January, 1989
January, 1989
EDITORIAL


Jonathan Erickson


Depending on who you talk to, the first IEEE international neural network
conference (which took place nearly two years ago) was either a surprising
success or a woeful failure. The naysayers point to a relatively small number
of people who showed up to hear a smattering of papers, share some theories,
and see a dozen or so exhibits. The yeasayers, however, insist that the
conference was successful simply because people showed up at all, especially
considering that as recently as a couple of years ago few people knew what
neural nets were, let alone how they could be used.
(As an aside, neural nets, as defined by Doug Palmer of Hecht-Nielsen
Neurocomputers, are highly parallel dynamical systems that process information
by means of response to continuous input. The structure of a neural net
consists of a large number of processing elements, each with multiple inputs,
but with a single output. In addition to the processing element, a neural net
is made up of an interconnection scheme [topology], a learning model, and
knowledge of the state of the system. Together, these elements are called the
network paradigm; currently, about 20 distinct paradigms have been
identified.)
There's little question, however, that the second international neural network
conference, which took place in July of 1988, was a success. Nearly two
thousand attendees--including members of the DDJ staff (see Michael Swaine's
"Programming Paradigms" column of October, 1988)--converged on San Diego to
find out what had happened in the world of neural nets over the space of a
year. What did they talk about? Among the dozens of papers presented were
those on topics such as "Neural Computation for Controlling the Configuration
of 2-Dimensional Truss Structure," "Neural Network Simulation at Warp Speed:
How We Got 17 Million Connections per Second," "Abilities and Limitations of a
Neural Network Model for Spoken Word Recognition," "Neural Network Models and
Their Application to Handwritten Digit Recognition," and "A Multilayer
Perception Network for the Diagnosis of Low Back Pain."
More significantly, the emphasis at the second conference, as well as more
recent technical conferences like the Fall ACM or the Pattern Recognition and
Advanced Missile System conferences, was on the practical, rather than
theoretical, side of neural networking. It's becoming apparent that neural
nets are rapidly moving out of research labs and into commercial development
environments because neural technology has the capability of providing
effective solutions to many kinds of complex computing tasks. A good example
of this is the kind of applications under development at companies like
Accurate Automation (a company located in Tennessee, a state that surprisingly
has become a hotbed of neural net development). AA is building commercial
neural net systems for tasks like monitoring the behavior of aircraft engines,
for real-time and process control, and even for monitoring of local area
networks and database activities using a form of neural nets called the
"nearest neighbor classifier" that looks for emergency patterns and reacts
accordingly. The basic questions being asked by programmers who are becoming
neural net aware concern which neural net paradigms should be applied in
specific situations, not whether or not neural nets are right at all.
With this backdrop, this month's examination of neural networks looks at the
technology from the practical side. Todd King's lead article will give you a
feel for what nets are about, and provide you with a new tool for solving old
programming problems. Casey Klimasauskas' article, and Steven Melinkoffs
sidebar, give you a chance to see how commercial neural net packages differ
from the roll-your-own approach taken by Todd. Together, our pair of neural
net articles also lets you examine the main types of problems--primarily
pattern recognition and noise filtering--neural networks are currently being
used to solve.
In short, neural nets have come a long way since DDJ last looked at them over
a year-and-a-half ago, and you can be assured that, over the next 18 months,
you'll see new uses and even more startling developments in this emerging
technology.















































January, 1989
LETTERS







Find that Function (Again)


Dear DDJ,
The article "Find that Function" caught my attention. I imagine every
programmer, at some time, wants a list of functions or other declarations,
particularly for a sizable program distributed over many source files.
I was surprised by the complexity of the code used to build the function list
in Hymowech's Listing One (pages 70-74), perhaps 400 lines of code and
comments. Surely there should a simpler way of doing this. The problem is how
to extract all instances of a particular grammar element, the identifier of a
function definition, from a source file presented as a character stream. This
is made difficult by the start of the definition (function type, name,
parameter declarations, and beginning of the function body) being commonly on
more than one line, having a varied arrangement of white space and
punctuation, comprising many different tokens, and possibly including comment
lines.
Lex, a generator of automatons for lexical analysis, is admirably suited for
this particular kind of pattern-matching problem. My example code gives Lex
sources for two filters and a wrapping shell script. The definitions
(FUNCSTRT, etc.) were written in the hope that they make self-explanatory the
few pattern-action rules given at the end of each source, following the %%
symbol. Mk_funclist.one and its dependents serve to build a table of functions
from a list of C source files.
The seven rules in the two Lex sources duplicate, I believe, the essential
actions of Listing One of "Find that Function." The design of Listing One
assumes that preprocessor directives within the function body do not alter the
curly brace balance. If this is not true, my second example shows that the
preprocessor itself can be used to sort things out. A few changes in Listing
One serve to change it into a generator of a list of all declarations, each on
one line, with parameter declarations and return value given on the line for
each function (a lint library).
There are tremendously powerful tools for pattern matching and extraction of
elements from text files. Public domain versions of Lex that run on non-Unix
systems have been available for some time. awk is sold for non-Unix systems
(should I say MS-DOS?). They run on simple hardware; the Lex sources of
Examples 1 and 2 were worked out on a PC AT. The problem is choosing the best
tool. For problems like the one addressed here, Lex is arguably the most
appropriate. Lex processes character streams simply, with new lines having no
special meaning, and so it differs from line-oriented tools, like sed and awk.
Lex also tests left and right context simply. Consider the last rule of
Listing One. It prints the NAME when the right context matches FUNCSTRT and
when the left-context flag NORM ensures that NAME is not inside a function
body. The same parsing likely could be accomplished also by use of awk or sed,
but the code for these tools would be more complicated. My intent is not to
push Lex as an all-purpose solution for text manipulation. The filters of
Listing Two, unlike those of Listing One, could just as easily have been
written for awk or sed.
Example 1

 # MK_FUNCLIST.ONE
 echo " /*****FUNCTION LIST*****/\n"
 for i; do
 echo "\n/* 'basename ${i} '*/"
 cat ${i}uncomment funcfilt 3
 done

 % {
 /*UNCOMMENT- based on usenet posting by: */
 /* Chris Thewalt; thewalt@ritz.cive.cmu.edu */
 %}
 STRING \"([^"\n]\\\")*\"
 COMMENT "/*"([^*\n]"*"+[^*/\n])*"*"*"*/"
 %%
 {COMMENT} ;
 {STRING} ECHO;
 .\n ECHO;

 %{
 /*-FUNCFILT3: print function names, indented 1 tab; */
 /*DECL & FUNCPTR may require change for ANSI compatibility */
 %}
 int curly;
 WLF ([ \t\n\f\r]*)
 NAME ([*]*[_a-zA-Z] [_a-zA-ZO-9]*)
 ARRAY (\[[0-9+-/*]*\])
 DECL ([;,]{WLF}{NAME}{ARRAY})
 FUNCPTR (\({DECL}*\)\({DECL}*\))
 DECLST ({DECL}{FUNCPTR})*
 FUNCSTRT ([ \t]*\({DECLST}\){DECLST}\{)
 STRING \"([^"\n]\\\")*\"
 SKIPALLQUOTED ({STRING}\'.\'\\.)
 %START CURLY NORM
 % {
 main() {
 /* if no shell wrapper, loop over files here */
 /* and run other filters using tmp files */
 BEGIN NORM;

 yylex();
 }
 %}
 %%
 <CURLY>\{ curly++;
 <CURLY>\} {if (--curly == 0) {BEGIN 0; BEGIN NORM;}}
 {SKIPALLQUOTED}.\n ;
 <NORM> {NAME}/{FUNCSTRT} (printf ("\t%s\n", yytext);
 curly=0;BEGIN 0;BEGIN CURLY;}

Example 2

 # MK_FUNCLIST.TWO
 echo" /*****FUNCTION LIST*****/\n"
 for i; do
 echo "\n/*'basename ${i}'*/"
 cat $ {i} funcfilt1/lib/cpp -P -Cl
 funcfilt2uncommentfuncfilt3
 done

 %{
 /*-FUNCFILT1: prepare for cpp execution of #ifdefs, etc.; */
 /*i.e., setup to restore #includes & remove code added by cpp */
 %}
 %%
 ^\#[ \t]*include.*$ {printf("/*%s*/\n", yytext);
 printf ("/*DINGDONGDELL*/\n");
 printf ("%s\n", yytext);
 printf ("/*DELLDONGDING*/\n:);}
 .\n ECHO;

 %{
 /*-FUNCFILT2: remove cpp-included code, restore #include's */
 %}
 %START DING
 %%
 ^"/*DINGDONGDELL*/"$ BEGIN DING;
 ^"/*DELLDONGDING*/"$ BEGIN O;
 <DING>^"/*#"[^*]*"*/"$ ;
 <DING>.\n ;
 ^"/*#"[^*]*"*/"$ {yytext [yyleng-2] = 0;
 printf ("%s", &yytext[2]);}
 ^[\t]*\n ;
 .\n ECHO;

With Lex, awk, and other such armament in hand, there would seem to be little
justification for coding the solution to a text manipulation problem from
scratch. Programmers should be encouraged to use the excellent tools available
and should have the knowledge to choose the one right for the job. I hesitate
to suggest that Dr. Dobb's should take its readers through discussions of the
internals or give tutorials on the use of "mature" tools like Lex, yacc, and
awk. You could, however, take care to give readers programming examples that,
rather than showing the long and wrong way to solve a problem, show them the
appropriate use of tools.
John Rupley
University of Arizona
Tucson, Arizona


Ada Aid


Dear DDJ,
I've read with interest the Ada articles in the September 1988 issue and hope
to see more such features in the future.
In "Object-Oriented Dimensional Units," John Grosberg presents a useful
example of how a dimensioned unit with a float-type value might be implemented
as a reusable component in Ada. However, I think one of his main points is
misleading.
In discussing the operations that the dimensional unit type Float_Unit.Class
inherits from the predefined type Float, Mr. Grosberg notes that some
operations are invalid. For example, 5 feet x 4 feet = 20 square feet, not 20
feet. He then states the "Ada provides no way to detect it (an invalid
operation) at compile time" and presents a way to detect the error at run
time.
One way to detect such an invalid operation at compile time is to implement
the type "Class" as a private type. Objects of a private type may be
manipulated only by the operations provided in the visible part of the spec.
If invalid operations are not explicitly provided, then they are not available
to a client. The integrity of the abstraction is thereby preserved, and
invalid operation attempts are discovered at compile time.

Use of private (and limited private) types is even more important when a class
is implemented as an array or record, which is often the case. Typically,
individual components of a composite type need to be protected from
manipulation by the clients. This is accomplished by use of private types
which provide a more accurate abstraction, a cleaner interface for the client,
more precise control of operations on the class, and minimization of side
effects of any changes to the internal representation of the type.
The code on page 12 shows the revisions to Mr. Grosberg's Float_Unit spec
required to make type "Class" a private type. "Class" is declared as type
private instead of type Float. The private part is inserted at the end of the
spec, and "Class" declaration is completed in the private part. The invalid
function declarations are then removed from the spec. The Units_Error
exception declaration is also removed.
In the Float_Unit body, only the removal of the invalid function bodies is
required. Package Float_Unit is shown in Listing One.
Glenn A. Edwards
St. Petersburg, Fla.
John responds: My statement that "Ada provides no way..." sounds more absolute
than I intended. I would have been more correct had I said "having chosen to
make float_unit.class public, Ada provides no way...." Any method of doing
something has advantages and disadvantages relative to the application. One
most choose the method that best fits the circumstances. Mr. Edwards' method
for making float_unit.class into a private type is the way I started the
design of the float_unit package. For my application, however, I eventually
chose to make the type public for two main reasons: First, I wanted to be able
to use floating point literals with variables of the various units types. For
example, I wanted to be able to say:
supply_voltage : volt.class := 5.2; or supply_voltage := 10.0;
Second, I wanted to be able to use the range declaration and checking features
that are available for floating point types in Ada. So, for example, I could
do something like this.
supply_voltage: volt.class range 0.0 .. 10.5;
I think it is safe to say neither of these capabilities is available if the
type is private. One could (if the type were private) obtain the effect of my
first example by adding a function to the package to convert floating point to
float_unit.class:
function to_class( f : float ) return class;
and use it as an inherited function like this:
supply_voltage : volt.class := volt.to_class(5.2);
But that seems less natural, and, besides, the range feature that I wanted
still wouldn't be available. In addition, there is little reason to hide the
fact that a dimensional unit is based on float, since that is the way we think
about them anyway. It is only necessary to "hide" or restrict the ways they
can be combined with each other. What I traded for the features I wanted was
to postpone the checking of some invalid operations to run-time. By the way,
if Mr. Edwards or any other readers are interested in an altogether different
method of handling units in Ada, I recommend they check out the article
"Dimensional Analysis in Ada" by Pat Rogers, published in the ACM Ada Letters,
vol. 8, no. 5, Sept./Oct. 1988.


Leftie Light Bulbs


Dear DDJ,
I enjoyed reading Steve Upstill's article on RenderMan Shading
Language("Photorealism in Computer Graphics," November 1988, but Figures 1 and
3 provided compelling examples of why software people have to be watched so
closely out in the real world. The threads on the light bulbs are left-handed.
And another thing: Threads are helixes, not spirals.
Fred Klingener
McLean, Virginia
Editor's note: Unfortunately, the slide was flopped during the printing
process. The error was not the programmer's.


Algorithmic Answer for Alan


Dear DDJ,
Software Manual for Elementary Functions (by William J. Cody Jr. and William
Waite, Prentice-Hall) should help Alan Clark, who needs a resource that lists
fundamental algorithms for high-level functions that do not exist in machine
code. (See "Letters," August 1988.) The book gives detailed implementation of
sqrt, alog, alog10, exp, power, sin, cos, tan, cot, asin, acos, atan, atan2,
sinh, cosh, tanh, and provides a test suite to boot.
Edmund Ramm
Kaltenkirchen, W. Germany





























January, 1989
USING NEURAL NETWORKS FOR PATTERN RECOGNITION


Recognizing and learning patterns is one thing neural nets do best




Todd King


Todd King is a programmer/analyst with the Institute of Geophysics and
Planetary Physics at UCLA. He is also associated with the NASA/JPL Planetary
Data Systems project. Todd can be reached at 1104 N. Orchard, Burbank, CA
91506.


By their very definition, neural networks are able to tolerate ambiguities in
much the same way as the human brain can. Consequently, the types of problems
that neural networks are best suited for unraveling are those in which
decisions need to be made based on close approximations to -- not exact
correspondence -- with the original input. One such class of problem is
pattern recognition, where the neural network can memorize a pattern, then
recognize it with 100 percent certainty when it sees the pattern again. And
when an entirely new pattern is presented to the network, the network can
determine which of the previously memorized patterns the new one matches the
most closely.
One issue faced by programmers who are interested in using neural nets to
solve pattern recognition problems is that there is more than one way to skin
a pattern recognition problem. In fact, research into the physiological and
cognitive processes of the mind has shown that there are several paradigms of
pattern recognition, each with its own usefulness, purpose, and inherent goal.
In this article I'll concentrate on the pattern associator and a called
variation of the classification paradigm.
Broadly speaking, the goal of a pattern associator is to handle pairs of
patterns -- one member of the pair being the input pattern and the other
member the output pattern. The network is configured (either explicitly or by
teaching) to associate the output pattern with the input pattern. Once this
has been done, every time the input pattern is presented to the network the
response will be the desired output pattern.
The classification paradigm is similar to the pattern associator in that it,
too, makes associations between input patterns and output patterns. Where it
differs is that the input patterns are placed into a fixed set of categories.
A benefit of this categorizing is that distorted or incomplete versions of the
input pattern can be presented to the network and the response will be the
correct output pattern.


Network Topologies


You can choose from a wide range of neural network topologies when attempting
to solve a particular problem. Choosing the right topology can mean a very
simple solution; choosing the wrong one can result in an overly complicated
solution or no solution at all.
After studying the problems that I discuss in this article - an executive
board simulator, logic gates, and an optical character recognition (OCR)
system -- I decided that all these examples could be implemented with the
following attributes: a three-layer network consisting of an input layer, a
hidden layer and an output layer. The number of neurons within each layer can
be set at run time. All neurons within each layer are fully connected with all
neurons in the next layer; that is, each neuron in one layer connects to every
neuron in the next layer (see Figure 1, page 16). The activation function is
the same for every neuron in every layer and is set to either a linear or
linear-threshold function. The propagation function is always a weighted sum,
and all connections activate in the direction of the output layer
(feed-forward network). Starting with this foundation, you can build a variety
of useful networks.


The Support Code


Let's first look at the support code that allows you to build neural networks.
A simple, general-purpose library, it consists of seven functions and five
pseudofunctions (defines) that allow you to build three-layer networks with
any number of neurons in each layer (up to a maximum). It also allows you to
set the activation, user input, and result output functions as well as the
weights between connections. Listing One, page 90, contains the source for the
support library. The pseudofunctions are in the include file in Listing Two,
page 90. The functions are simple with few or no arguments. Rather then detail
each function with words, I hope the names of the functions, the comments in
the listings, and the following examples will demonstrate how to use the
support code.


Building an Executive Board


A linear network is one of the simplest types of network. It operates by
passing the result from the propagation function directly to its output; that
is, the activation function performs no action on the input. To demonstrate
this type of network, let's consider an executive board that has five voting
members. Each member represents some number of stockholders who are voting by
proxy. There is one vote taker to whom all board members give their votes. The
vote taker then determines the net vote (how many for or against) by
considering how each board member votes and how many stockholders he or she
represents. The vote taker then reports the results to the board president,
who announces the outcome of the vote.
This problem can be represented as a neural network with the board members as
an input layer consisting of five neurons. The vote taker is a one-neuron
hidden layer and the board president is a one-neuron output layer. The weight
between the input layer and the vote taker is the number of stockholders a
board member represents. The vote taker is considered to be honest, so the
board president takes the results at face value (a weight of 1.0). Figure 2,
page 16, is a diagram of the network, and Listing Three, page 91, is the
source to implement the executive board neural network. When run, the program
prompts you for each member's vote and prints out whether the vote was for,
against, or a tie. The number of stockholders each member votes for is set to
the board member's number (for example, three for board member 3).


The Logic Gate


A slightly more complicated network is the linear-threshold-based network.
With this type of network, the output value is set to a specific value if the
net input to the neuron exceeds some threshold (specific value) and to
something different if the value is below the threshold. A good example of
such a network is one that emulates logic gates. With a logic gate, the gate
is either on or off, so you have what is called a binary linear threshold. For
a two-input OR and a two-input AND gate, you can use an input layer of two
neurons, a hidden layer of one neuron, and an output layer of one neuron. A
two-gate XOR requires a two-neuron hidden layer. The difference is because the
XOR logic gate needs to compare both inputs from two perspectives so it can
perform the exclusive operation.
Now let's look at how you can "teach" a network to recognize the two-bit input
patterns and produce the desired one-bit output pattern. Even though, with
more code and execution time, the network could teach itself, let's calculate
the weights for the connections between all neurons by deriving them
mathematically. The first thing to note is that with linear networks, when you
have a neuron with only one input connection, that neuron is not needed. The
input connection can bypass the one connection neuron and go directly to all
neurons that the one connection neuron connects to, without altering the
solution. This means that for the AND and OR logic gates, you really only need
two layers. Using three layers here means that the model is a more complicated
solution than is needed, but I'll accept the three-layer model so that all
examples can use the same support library. You can infer from this that the
weight between the hidden layer and output layer is to be 1.0 (face value).
Starting with the OR gate, you can detail the mappings between the input
pattern and the desired output pattern, as shown in Table 1, page 24.
Table 1. Mappings between input pattern and desired output pattern for an OR
gate

 Input Output
 --------------

 0 0 0
 1 0 1
 0 1 1
 1 1 1

From this you can derive the following simultaneous equations, which once
solved will tell you what weights the connections between the input and hidden
layer are to have. Let's use a threshold of 1.0 to differentiate between an on
state and an off state. The general equation is:

 w[1]i[1] + w[2]i[2] >= 0
where w[1] is the weight for input neuron 1 and w[2] is the weight for neuron
2, i[1] and i[2] are the input patterns, and 0 is the output pattern. The >=
is because you're using a threshold network. Because you have four patterns,
the equations after substitution of the values are:
 w[1] >= 1.0 W[2] >= 1.0 w[1] + w[2] >= 1.0
This not only gives you the answer immediately but also provides a check for
the answer. Let's choose 1.0 for both connections. You can derive the weights
for the AND gate. The values of w[1] = 0.5 and w[2] = 0.5 should be one of the
solutions.
The solution for the XOR gate is more complicated to derive and requires more
math, so I'll discuss the logic behind its solution. The weights between the
input and hidden layers must be adjusted so that when both inputs are on the
hidden-layer neurons are turned off. This is easy to do if, at any
hidden-layer neuron, the weights given to the input neurons are equal in value
but have opposite signs. Then, whenever both neurons are on, they cancel. This
makes the input-hidden layer pair a filter that ensures that both hidden-layer
neurons can never be on. The hidden-output-layer pair is then a simple OR
gate.
Figure 3, page 24, shows what all the logic gates look like as neural,
networks and Listing Four, page 91, shows how to build the logic gate
networks. When you run the code, it prompts you for the values for the input
pins for each type of gate, and the result is printed to the screen.


The OCR System


Now let's try something that is really suited for neural networks -- an
optical character recognition (OCR) system, which translates an optical
pattern composed of on and off bits to an ASCII code. The system will also
tell you just how certain it is that the input pattern is the character it
says it is. To do this I need to introduce one other concept -- namely, of
clustering. A cluster is a collection of neurons, all in the same layer, which
are in competition with one another to be the only one on. It's winner takes
all in a cluster, so the rule that is applied is that the neuron that is
closest to being on is forced on whereas all other neurons are forced off. The
closeness to being on is a measure of the certainty to which the input pattern
can be determined. In the cases in which two or more neurons have the same
value and are closest, the first neuron encountered is turned on. Doing this
doesn't affect the accuracy of the recognition because all neurons with equal
values have the same certainty.
For the OCR system, the input-hidden-layer pair is defined to be a binary
linear-threshold-based network in which neurons are either on or off. The
other layer pair, composed of the hidden layer and the output layer, is
defined to be a linear network. This approach allows you to associate any
value you like with the bit-image input pattern. Only the neurons in the
hidden layer are clustered, and all neurons in the hidden layer are placed in
the same cluster. A further restriction that is placed on the OCR network is
that the sum of all weights for all input connections to a neuron in the
hidden layer will be 1.0 or less. Collectively, the input-hidden-layer pair,
with all its features and restrictions, is a variety of what is commonly
called a perceptron (Rosenblatt, 1957). One aspect of a perceptron is that it
can classify its input patterns in as many groups as there are neurons in the
hidden layer. Thus, to uniquely classify the 26 characters in the alphabet the
hidden layer must have 26 neurons.
In order for the OCR network to recognize a pattern, it must be taught which
patterns produce which outputs. You can simplify the learning process by
understanding just what the network learns and then build in shortcuts that
will speed things up. For this example, let's limit each group so that it
recognizes only one character. By doing this the learning process is almost
instantaneous.
Each letter to learn is read and presented to the network. The next unaltered
hidden neuron is selected, and the weights from that neuron to all input
neurons are set. The weight for each input connection that is on is set to 1.0
divided by the total number of input connections that are on. The weight for
any input connection that is off is set to 0.0. The weight of the connections
from the hidden-layer neurons to the output neuron is set to the numeric
(ASCII, for example) value with which the input pattern is associated. The
function learn_ocr( ) in Listing Five, page 94, does all this. Once a network
has learned a set of characters, you can present it with any character pattern
and it will tell you which one of the known patterns the presented pattern
matches most closely.
Listing Five contains the code for the OCR system, and Figure 4, below, shows
the topology of the neural network.
When you run this program, you specify the characters it is to learn and test
on the command line as file names. Only one character is allowed in each file.
The patterns to learn are given first, then the word -test, and then the file
names of the patterns to test. The content of a file is the optical pattern
for a character as a 5x7-bit pattern, with the bits represented as 1s and 0s.
If the OCR system is to learn a pattern, the next value after the character
pattern is the numeric value that pattern is to be known by. A pattern that is
only to be tested does not need a numeric value because, when the OCR system
is testing a pattern, it reads only the pattern. The format of the file allows
the same file to be used for both learning and testing. Table 2, page 28,
contains values for the letters A, B, C, E, and O. A good invocation would be:
 ocr letter.a letter.b letter.c -test letter.a letter.e letter.o
Table 2: Values for letters A, B, C, E, and O. The values for each letter must
be in separate files when used with the OCR example. The bit image is shaded
to emphasize how it was derived.

 A: 00100 B: 11110 C: 00110
 01010 10001 01001
 10001 10001 10000
 11111 11110 10000
 10001 10001 10000
 10001 10001 01001
 10001 11110 00110
 65.0 66.0 67.0

 E: 11111 O: 00100
 10000 01010
 10000 10001
 11100 10001
 10000 10001
 10000 01010
 11111 00100
 69.0 79.0

The OCR system should respond with A is an A, E is a B, and O is C, which is
an intelligent deduction given that all the OCR network knows about are the
letters A, B, and C.


Benchmarks


On an XT-compatible rated at CI: 1.8 and DI: 1.1 with Norton's SI program, I
found that the OCR neural network, when compiled with Turbo C, could learn a
new pattern in 0.2 seconds. With four pattern groups it could determine a
pattern in 0.7 seconds. There was an overall start-up overhead of 0.2 seconds.
The program and library were compiled so that they were optimized for speed,
register use, and jumps, and floating-point emulation was in effect. Listing
Six, page 95, contains the project files to build the examples using Turbo C.
The source will also compile and run under Quick C.


Conclusion


I encourage you to experiment with neural networks. The support library
contained in this article is a toolbox you can use to do this. The most
difficult aspect of dealing with neural networks is the proper management of
the neurons. One way to do this is revealed in Listing Two, which is the
include file that is shared between the examples and the support library.
Even though neural networks have been a concept for almost as long as
computers have existed, there's still room for new ideas and innovations. This
article touches on only a small part of the topologies and functionality of
neural networks. If you'd like to learn more about neural networks, see the
bibliography for reading material.
I'm giving this neural network toolbox away freely. If you do use it to build
applications, please do not charge for the toolbox portion.
Please direct any questions about this article to the author at the address
given at the beginning of the article. If you need a response, include a
self-addressed, stamped envelope.


Bibliography



McClelland, James L. and Rummelhart, David E. Explorations in Parallel
Distributed Processing, Cambridge, Mass.: MIT Press, 1988.
NeuralWorks Professional Reference Manual. Sewickley, Penn.: NeuralWare Inc.,
1987.
Rummelhart, David E., et al. Parallel Distributed Process, vol. 1, Cambridge,
Mass.: The MIT Press, 1986.

_USING NEURAL NETWORKS FOR PATTERN RECOGNITION_

by Todd King


[LISTING ONE]

#include <stdio.h>
#define EXTERN extern
#include "neural.h"

/*-- MAKE_MIND ---------------------------------------
 Constructs a mental unit with the given number
 of input, hidden and output neurons.
------------------------------------------------------*/
make_mind(in, hid, out)
int in;
int hid;
int out;
{
 if ( in > MAX_NEURONS 
 hid > MAX_NEURONS 
 out > MAX_NEURONS ) return(0);
 if (in < 1 hid < 1 out < 1 ) return(0);
 Mind.n_input = in;
 Mind.n_hidden = hid;
 Mind.n_output = out;
 set_cluster_fun(NULL, NULL);
 set_all_weights(1.0);
 set_act_fun(pass);
 set_user_in_fun(prompted);
 set_result_fun(print_binary_state);
 strcpy(Prompt.string, "Input a value for neuron %d: ");
 Prompt.count = 1;
 return(1);
}

/*-- ACTIVATE_MIND -----------------------------------------
 Sets a mind in motion. Sequentially activating each neuron
-----------------------------------------------------------*/
activate_mind()
{
 int i;
 float net_input;

/* Activate input layer */
 Prompt.count = 1;
 for (i = 0; i < Mind.n_input; i++)
 {
 Mind.i_layer[i].value = Mind.user_in_fun();
 }

/* Activate hidden layer */
 for (i= 0; i < Mind.n_hidden; i++)
 {

 net_input = weighted_sum(i, HIDDEN);
 Mind.h_layer[i].value = Mind.act_fun(net_input);
 }

/* Activate feedback/certainty function (if one is set) */
 if ( Mind.certainty != NULL) Mind.cluster_fun(Mind.certainty);

/* Activate output layer */
 for (i=0; i < Mind.n_output; i++)
 {
 net_input = weighted_sum(i, OUTPUT);
 Mind.o_layer[i].value = Mind.act_fun(net_input);
 Mind.result_fun(Mind.o_layer[i].value);
 }
}

/*-- SET_ALL_WEIGHTS --------------------------------------
 Sets the weight of all connections between all neurons
 in all layers to the given value
----------------------------------------------------------*/
set_all_weights(value)
float value;
{
 int i, j;

/* Weights between input and hidden */

 for(i = 0; i < Mind.n_input; i++)
 {
 for(j = 0; j < Mind.n_hidden; j++)
 {
 Input_to_hidden[i][j].weight = value;
 }
 }

/* Weights between hidden and output */

 for(i=0; i< Mind.n_hidden; i++)
 {
 for(j = 0; j < Mind.n_output; j++)
 {
 Hidden_to_output[i][j].weight = value;
 }
 }
}

/*-- SET_WEIGHT -------------------------------------
 Sets the weight between two neurons to a given value.
------------------------------------------------------*/
set_weight(from, to, layer, value)
int from;
int to;
int layer;
float value;
{
 switch (layer)
 {
 case HIDDEN:
 if (from > Mind.n_input) return;

 if (to > Mind.n_hidden) return;
 Input_to_hidden[from][to].weight = value;
 break;
 case OUTPUT:
 if (from > Mind.n_hidden) return;
 if (to > Mind.n_output) return;
 Hidden_to_output[from][to].weight = value;
 break;
 default:
 break;
 }
 return;
}

/*-- WEIGHT_SUM --------------------------------------------
 Calculates the weighted sum for a given neuron in a given
 layer
----------------------------------------------------------*/
float weighted_sum(this_neuron, this_layer)
int this_neuron;
int this_layer;
{
 int i;
 float sum = 0.0;

 switch (this_layer)
 {
 case HIDDEN:
 for (i = 0; i < Mind.n_input; i++)
 {
 sum += (Mind.i_layer[i].value * Input_to_hidden[i][this_neuron].weight);
 }
 break;
 case OUTPUT:
 for (i = 0; i < Mind.n_hidden; i++)
 {
 sum += (Mind.h_layer[i].value * Hidden_to_output[i][this_neuron].weight);
 }
 break;
 default:
 break;
 }

 return (sum);
}

/*-- PASS ----------------------------------------------
 Returns the input value. A dummy activation function.
--------------------------------------------------------*/
float pass(value)
float value;
{
 return (value);
}

/*-- PROMPTED ---------------------------------------
 Prompts the user for an input value and returns the
 value. A user input function.
-----------------------------------------------------*/

float prompted()
{
 float value;

 printf(Prompt.string, Prompt.count++);
 scanf("%f", &value);
 return(value);
}

/*-- PRINT_BINARY_STATE -------------------------------
 Prints the output state of a neuron. If greater than
 0.0 the value printed is "on", otherwise "off".
------------------------------------------------------*/
float print_binary_state(value)
float value;
{
 printf("The output gate is: ");

 if (value > 0.0) printf("ON.");
 else printf("OFF.");

 printf("\n");
}



[LISTING TWO]

#ifndef _NEURAL_
#define _NERUAL_

#define MAX_NEURONS 35

#define HIDDEN 1
#define OUTPUT 2

/* Type definition for neurons and neural networks */

typedef struct {
 float value;
} NEURON;

typedef struct {
 int n_input;
 int n_hidden;
 int n_output;
 float *certainty;
 float (*cluster_fun)();
 float (*act_fun)();
 float (*user_in_fun)();
 float (*result_fun)();
 NEURON i_layer[MAX_NEURONS];
 NEURON h_layer[MAX_NEURONS];
 NEURON o_layer[MAX_NEURONS];
} MIND;

typedef struct {
 float weight;
} WEIGHTS;


typedef struct
{
 char string[80];
 int count;
} PROMPT;

/* Global Variables */

EXTERN MIND Mind;
EXTERN WEIGHTS Input_to_hidden[MAX_NEURONS][MAX_NEURONS];
EXTERN WEIGHTS Hidden_to_output[MAX_NEURONS][MAX_NEURONS];
EXTERN PROMPT Prompt;

/* Functions */

float weighted_sum();
float pass();
float prompted();
float print_binary_state();
float certainty_fun();
int activate_mind();

/* Pseudo-functions */

#define set_act_fun(f) Mind.act_fun = f
#define set_user_in_fun(f) Mind.user_in_fun = f
#define set_back_prop_fun(f) Mind.back_prop_fun = f
#define set_result_fun(f) Mind.result_fun = f
#define set_cluster_fun(f, x) Mind.cluster_fun = f; Mind.certainty = x

#endif







[LISTING THREE]

/* Linear network */

#define EXTERN
#include "neural.h"

#define MEMBERS 5

float print_vote_state();
#define plural(x) (x == 1 ? "" : "s")

main()
{
 int i;

 make_mind(5,1,1);
 set_result_fun(print_vote_state);
 strcpy(Prompt.string, "Ballot for member %d: ");
 for(i=0; i<MEMBERS; i++)

 {
 set_weight(i, 0, HIDDEN, (float)(i+1) );
 }
 printf("Ballot values: 1 = for, 0 = obstain, -1 = against\n\n");
 activate_mind();
}

float print_vote_state(value)
float value;
{
 int votes;
 printf("The vote is: ");

 votes = (int)value;
 if (votes > 0)
 printf("FOR, by %d vote%s", votes, plural(votes) );
 else if (votes < 0)
 printf("AGAINST, by %d vote%s", -votes, plural(-votes) );
 else
 printf("A TIE");

 printf(".\n");
}








[LISTING FOUR]

/* Simple linear threshold network.
 Demonstates logic gates */

#define EXTERN
#include "neural.h"

float linear_threshold();

main()
{
 int i;

/* OR gates work using the default weights (1.0) */

 strcpy(Prompt.string, "Logic state of gate %d: ");
 printf("Logic values: 1, on; 0, off\n\n");

 printf("OR logic gate.\n");
 printf("--------------\n\n");
 make_mind(2, 1, 1);
 activate_mind();

/* AND gates must have weights < 1.0 ( and > 0.0) */

 printf("\n");
 printf("AND logic gate.\n");

 printf("--------------\n\n");

 for(i = 0; i < 2; i++)
 {
 set_weight(i, 0, HIDDEN, 0.5);
 }
 activate_mind();

/* XOR gates are the most complicated */

 printf("\n");
 printf("XOR logic gate.\n");
 printf("--------------\n\n");
 make_mind(2, 2, 1);
 set_weight(0, 0, HIDDEN, 1.0);
 set_weight(1, 0, HIDDEN, -1.0);
 set_weight(0, 1, HIDDEN, -1.0);
 set_weight(1, 1, HIDDEN, 1.0);
 set_weight(0, 0, OUTPUT, 1.0);
 set_weight(1, 0, OUTPUT, 1.0);
 set_act_fun(linear_threshold);
 activate_mind();
}

/*-- LINEAR_THRESHOLD -------------------------------------
 If the input value is greater than zero then it returns
 1.0, otherwise it returns 0.0. A linear threshold
 activation function.
----------------------------------------------------------*/
float linear_threshold(value)
float value;
{
 if (value > 0.0) return(1.0);
 else return(0.0);
}









[LISTING FIVE]

/*
 Optical Character Recognition (OCR) neural network
 This is a hybrid between a linear threshold, fully
 interconnected network and a linear network. The
 transition being at the hidden layer. A Feedback neuron
 gaurantees a pattern match in the threshold layer.
*/
#include <stdio.h>
#define EXTERN
#include "neural.h"

float percep();
float print_ocr();

float certainty_cluster();

float Certainty;
FILE *Ocr_fptr;

main(argc, argv)
int argc;
char *argv[];
{
 int i;

 if(argc < 2)
 {
 printf("proper usage: ocr [<train_file> ...] [-test <test_file> ...]\n");
 exit(-1);
 }

 make_mind(35, 3, 1);
 set_user_in_fun(percep);
 set_result_fun(print_ocr);
 set_cluster_fun(certainty_cluster, &Certainty);
 set_all_weights(0.0);

/* Teach the network about the patterns */
 i = 1;
 while(strcmp(argv[i], "-test") != 0)
 {
 printf("Learning: %s\n", argv[i]);
 if( i > Mind.n_hidden)
 {
 printf("Too many pattern groups for the given topology, aborting.\n");
 exit(-1);
 }
 ocr_learn(argv[i], i - 1);
 i++;
 if(i >= argc)
 {
 printf("Nothing to test - exiting\n");
 exit(-1);
 }
 }

/* Classify each pattern based on what the network knows */
 i++; /* Skip over "-test" deliniator */
 while(i < argc)
 {
 printf("Testing %s\n", argv[i]);
 if ((Ocr_fptr = fopen(argv[i], "r")) == NULL)
 {
 perror(argv[i]);
 printf("Unable to open file, skipping pattern.\n");
 i++;
 continue;
 }
 activate_mind();
 fclose(Ocr_fptr);
 i++;
 }
}


/*-- PERCEP ------------------------------------------------
 Returns the value of the next pixel every time its called.
 The pixel state is determined from the contents of the
 pre-opened file pointed to by 'Ocr_fptr'.
----------------------------------------------------------*/
float percep()
{
 extern FILE *Ocr_fptr;
 int pixel_value;

 fscanf(Ocr_fptr, "%1d", &pixel_value);
 return( (float)pixel_value);
}

/*-- PRINT_OCR -------------------------------------
 Prints the character which the network determines
 it to be. Also prints the certainty of the match.
------------------------------------------------------*/
float print_ocr(value)
float value;
{
 extern float Certainty;

 printf("The character is '%c' (%d).\n", (int)value, (int)value);
 printf("with a certainty of %3.2f%.\n", Certainty);
}

/*-- OCR_LEARN -----------------------------
 Teach the network how to classify
 a pattern.
--------------------------------------------*/
ocr_learn(filename, group_id)
char filename[];
int group_id;
{
 int i;
 FILE *fptr;
 int pixel_cnt = 0;
 int pixel_value;
 float dist_weight;
 float output_value;

 if ((fptr = fopen(filename, "r")) == NULL)
 {
 perror(filename);
 printf("Skipping pattern.\n");
 return(0);
 }

/* Determine the number of "on" pixels, hence fractional weight */
 for(i=0; i < Mind.n_input; i++)
 {
 fscanf(fptr, "%1d", &pixel_value);
 if(pixel_value == 1) pixel_cnt++;
 }
 dist_weight = 1.0/pixel_cnt;
 rewind(fptr);


 /* Set fractional weight for each "on" connection */
 for(i=0; i < Mind.n_input; i++)
 {
 fscanf(fptr, "%1d", &pixel_value);
 if(pixel_value == 1) set_weight(i, group_id, HIDDEN, dist_weight);
 }

 /* Now set weight for output value for this character */
 fscanf(fptr, "%f", &output_value);
 set_weight(group_id, 0, OUTPUT, output_value);

 fclose(fptr);
 return(1);
}

/*-- CERTAINTY_CLUSTER ------------------------------------
 Performs a cluster function. It inhibits (sets to 0) all
 neurons in the cluster except the one which is closest to
 the value 1.0. This neuron is set to 1.0. The passed
 variable is assigned the certainty to which the closest
 neuron felt it matched the pattern
----------------------------------------------------------*/
float certainty_cluster(certainty)
float *certainty;
{
 int i;
 float highest = 0.0;
 int closest = -1;

 for(i=0; i<Mind.n_hidden; i++)
 {
 if(Mind.h_layer[i].value > highest)
 {
 closest = i;
 highest = Mind.h_layer[i].value;
 }
 }
 if(closest == -1) /* All are equally likely - choose the first */
 {
 closest = 0;
 }

 *certainty = Mind.h_layer[closest].value * 100.0;

/*
 Cause just enough feedback to the neuron which is closest
 to being "on" so that it is "on". That is set it "on"
 All others are given negative feedback to force them to
 zero. (set them to zero).
*/
 for( i = 0; i < Mind.n_hidden; i++)
 {
 if (i == closest) Mind.h_layer[i].value = 1.0;
 else Mind.h_layer[i].value = 0.0;
 }
}







[LISTING SIX]

Board:
 board.c (neural.h)
 neurlib.c (neural.h)

Logic:
 logic.c (neural.h)
 neurlib.c (neural.h)

OCR:
 ocr.c (neural.h)
 neurlib.c (neural.h)














































January, 1989
NEURAL NETS AND NOISE FILTERING


Back-propagation is a powerful adaptive method for filtering out noise or
identifying underlying signals




Casey Klimasauskas


Casimir C. "Casey" Klimasauskas is the president and founder of Neural-Ware, a
developer of neural network tools. He can be reached at 103 Buck-skin Ct.,
Sewickley, PA 15143.


Neural networks are an information-processing technology inspired by studies
of the brain and nervous system. Despite (or, perhaps, because of) their
origins, certain types of neural networks have a strong founding in
mathematics. One network type in particular, back-propagation, is a powerful
adaptive technique for approximating relationships between several continuous
valued inputs and one (or more) continuous valued output. This article
discusses applications of back-propagation to filtering out noise, or,
conversely, identifying fundamental underlying signals. An example, discussed
later, is in isolating EKG signals taken from a noisy environment.
This article addresses the problem of developing a limited number of "feature
detectors" that can account for the maximum amount of a signal (in a
least-mean-square sense). In doing this, the noise is assumed to be random and
of a higher frequency than the underlying signal. The approach taken here is
to find a way of encoding or compressing the input data and then reexpanding
it. The compression process eliminates portions of the input data, which
represent small or nonrecurring features. By selecting the number of
"encoders," you can vary the amount of detail retained in the transformation.
Figure 1, page 31, illustrates this process. Key to solving this problem is
the selection of a good set of feature detectors for the particular signal to
be to filtered.
Notice that in Figure 1 the 11 inputs have been reduced at the output of the
encoder stage to 3 outputs. If you were to represent each input as well as the
output of each encoder as a 32-bit floating-point number, it would take 352
bits (11 x 32) to represent the input to the system. It would take 96 bits (3
x 32) to represent the output of the system. From this perspective, you have
compressed the input data by a factor of 96/352. This form of data compression
is also known as dimensionality reduction. You have reduced the number of
dimensions from 11 to 3. For this system to work well, you must be sure that
the following two interrelated criteria are met: 1. There must be some
relationship between input variables; and 2. You must use an encoding system
that can represent the relationship.
It may not be apparent at first that these two criteria are related, but
consider the situation in which the input samples are from a
constant-amplitude sine wave. In this case, you could encode the data exactly
with a single "sine-wave" detector in which the output is related to the
position in the cycle. On the other hand, trying to use another type of
detector function may require several detectors to develop a "piece wise"
approximation of the function. If there is no relationship between the input
variables, dimensionality reduction often results in a series of fixed outputs
representing the "average" value of a combination of inputs. In the case of
noise filtering (where the inputs are a series of time-sequential samples),
there is usually a high degree of correlation (or relationship) between
adjacent time samples. Dimensionality reduction works very well in this
situation.
With a little thought, it becomes apparent that if the signal you are working
with has a noise component added to it, the contribution of the noise to the
signal will tend to be random or nonstationary with respect to other larger
features. As such, the noise will most likely be one of the features
eliminated in the encoding process. If the noise component is of a relatively
low frequency and accounts for a majority of the energy in the output signal
(for example, 60-cycle hum), this process will detect the noise. Subtracting
the output of the detector from the input signal removes the noise from the
signal.
To approach the problem of feature detection, first consider a simpler problem
of mapping from a single continuous input to a single continuous output.
Assume that the input and output are related by some continuous differentiable
function. Figure 2, this page, shows a plot of input values "X" and output
values "Y."
A first attempt to approximate the relationship uses a linear relationship.
This, too, is shown in Figure 2. The slope of this line can be determined
using statistical techniques (linear regression) or by using adaptive
signal-processing techniques (Bernard Widrow). Rather than using a straight
line, you could have used a quadratic polynomial. Figure 3, this page, shows
how a quadratic polynomial might fit the data shown. Both of these techniques
can be generalized to two, three, or more inputs. Notice that regardless of
the number of inputs, each feature detector has exactly one output.
For both of the cases just described, the decoded output is assumed to be
identical to the output of the encoders. Consider a more complex situation in
which you use a "sigmoidal" feature detector. Figure 4, page 34, shows an
example of a sigmoid function. You might use this function to create two
encoders as shown in Figures 4b and 4c. Now, if you use a decoder that takes
the difference of the output of the two encoders, you have the function shown
in Figure 4d. Using this method, you have been able to synthesize a "bump." By
varying the ai0 term, you shift the transition point. Varying the ai1
parameter changes the slope at the transition. In this way, you could have
created a "bump" of any size, any location, and independently vary the slope
of each of its sides. By adding more "bumps" together, you can create
arbitrarily complex functional relationships between one or more inputs and a
single output.
Though bumps can be used to create arbitrarily complex functions, this may not
always be the best method. At times, it may be better to use sine or other
types of functions in the encoders. Using "sine" encoders is equivalent to
Fourier decomposition of the inputs. Unlike the standard Fourier transform,
this technique accounts for both the phase and amplitude of the sampled input.
At this point, you might observe that what has been discussed sounds like
adaptive signal-processing, but with two stages instead of one. And you are
right to an extent. An adaptive unit is identical to the "linear" encoder
shown in Figure 2. If you were to take two linear encoders and try combining
their outputs together in a linear combination, you can easily show that this
is identical to a single linear element. The novel characteristics of the
neural network approach is that the output of the weighted sum (linear
combination of the inputs) is transformed using a non-linear function such as
a sigmoid or sine function. These nonlinearities make the creation of
multilevel systems possible and are responsible for several of the resulting
characteristics.
Why use this approach? The two basic reasons are as follows. First, it is
capable of retaining a level of detail greater than other techniques. Rather
than simply ignoring small features, it can allow them to pass if they are of
a stationary recurring nature. Second, the amount of detail retained by the
filtering process can be varied by varying the number of elements in the
hidden layer. From these perspectives, it is a technique providing additional
flexibility and control when the engineer analyzes noisy data.
More Details.
The approach described thus far is to develop a limited number of feature
detectors that encode a series of samples of an input signal. The output of
these encoders is used to reconstruct all or a portion of the input sample.
The use of a limited number of feature detectors ensures that some of the
information (preferably the noise) will be removed in the reconstruction.


Putting the Theory Into Practice


Until now, the problem of how to compute the various coefficients used by a
feature detector or encoder has been ignored. The process is a form of
regression. The word "regression" comes from a study done in the early 1900s.
In this study, the researchers attempted to find relationships between the
height of parents and that of their children. They developed a statistical
technique for fitting a straight line through the data (linear regression).
The results of the study showed that tall parents had shorter children and
that short parents had relatively taller children -- their heights were
regressively correlated. The term regression analysis applies to any technique
for fitting a curve to data.
The technique used to set the coefficients in a network is described as a set
of differential equations that modify the coefficients in such a way as to
reduce the mean-square-error of the output for the training set. These are
derived by defining the error for a single pass through a training set
(several presentations of input and expected, or desired, output) as the sum
of the squares of the difference of the actual and expected outputs. This is
called the square error.
The derivative of each weight (or coefficient) in the network is computed as a
function of the error. For most cases, these differential equations do not
have a closed-form solution. As such, the solution (or a solution) is derived
by using numerical methods. The numerical techniques compute the direction to
change the coefficients (gradient) and then change them slightly in that
direction. The mathematics behind this are explained in detail in Chapter 8 of
Parallel Distributed Processing by Rumelhart and McClelland (MIT Press 1986).
Listing One, page 96, presents a back-propagation network program that can
solve the "exclusive OR" problem described in the Rummelhart-McClelland book.
This program could also be extended to solve the noise filtering problem. If
you want to extend the program in Listing One to solve the "noise filtering"
problem, you must do three things:
1. Increase the number of processing elements in the input and output layers
to 19 each and the number of processing elements in the hidden layer to 5, 7,
10, or 19.
2. Delete all of the existing connections and fully connect the input layer to
the hidden layer and the hidden layer to the output layer. Likewise fully
connect the bias to both the hidden and output layers.
3. Write a module to load a series of noise data into memory and present them
to the network.
For the purposes of studying the noise filtering problem, I used a commercial
neural network development environment (NeuralWorks Professional II by
NeuralWare) for all of the examples that follow because neural network
development systems provide the infrastructure needed to solve the problem.
These systems provide the network building tools that make it easy to change
the network design, interfaces to assist in setting up the problem, tools for
investigating what is happening within the network itself, and a host of other
"bookkeeping" functions. A good development environment also enables you to
focus on the problem. Though most neural network types appear relatively
straightforward from the standpoint of mathematics, many programmers find it
difficult to debug this kind of numerical-oriented code. If a handcoded
network fails to converge, the problem may lie either in the "code" or the
network parameters. A neural network development environment solves the first
of these problems and makes the other problems easier to test.
Figures 5a, page 42, and 5 , 5c, and 5d, page 44, show screen dumps of four
tests of training a network to filter out noise. The examples use 5, 7, 10,
and 19 elements in the hidden layer. As you would expect from the previous
discussion, the more processing elements in the hidden layer, the more detail
is preserved from the encoding process to decoding process.
The data displayed in the lower window of each of the examples is the raw
input data. This is actually EKG data taken in a noisy environment (the noise
is caused primarily by fluorescent lights). The filtered output data is
displayed in the upper window. The networks used here each had 19 inputs and
19 outputs. The "filtered output" in the upper window is the output of the
center processing element of the output layer. Experimentally, the outputs of
the other processing elements in the output layer do a similar job of tracking
the input.
So far, we have looked at the problem from the perspective of doing
dimensionality reduction on 19 inputs to produce 19 outputs. The way we
trained the network was to randomly select a block of 19 sequential samples
from the raw data, and to use this as both the input and desired output of the
network. This was typically repeated 300,000 to 600,000 times for each of the
networks shown (overnight on a NEC Powermate III). Each time a block of inputs
was applied to the network, the weights were slightly adjusted to make the
actual network output closer to the desired output. Only the middle output
element is used as the "filtered" output.
You might wonder what would happen if you were to have only one output element
that was trained to reproduce the value of the middle input. Figure 6 , this
page, shows the results of such a network with five elements in the hidden
layer.
Closely examining the weights in the network, it becomes clear that the output
is affected almost exclusively by the center input. All other inputs are
ignored. Neural network techniques are sometimes smarter than you might
expect! The use of additional processing elements in the output layer forces
the feature detectors to come up with a more "general" way of encoding the
inputs.
During the recall process, the raw data is shifted through a 19-element long
shift register and the output computed with that set of inputs. The data is
shifted one position and the process repeated.
The training process is reasonably repeatable with the data shown. We have
tried it on a variety of other signals (manually constructed) with some
interesting results. Depending on the type of signal used, the processing
elements in the hidden layer should use a "sine" function rather than a
sigmoid. As mentioned previously, this causes the network to do a form of
Fourier decomposition and reconstruction.


Conclusion


As a general technique, neural network-based techniques for noise filtering
offer an interesting and potentially powerful approach. One of the
particularly interesting characteristics of neural nets is the capability to
develop an adaptive filtering technique that can be tuned to preserve varying
degrees of detail. Significant experimentation remains to develop these
techniques to the point where they are well characterized and understood.


Bibliography



D. Rumelhart and J. McClelland. Parallel Distributed Processing, Vol. 1.
(Boston Mass.: MIT Press; 1986).
B. Widrow and S.D. Stearns. Adaptive Signal Processing (Englewood Cliffs, NJ:
Prentice-Hall, 1985).


Neural Networks for Signal Processing: A Case Study


Steve Melnikof is an experimental particle physicist, who spends his working
hours in a multitasking fashion. When he isn't designing experiments or
consulting for GE, he can be found at home programming a Macintosh II to
investigate neural networkarchitectures.
Artificial neural networks (ANNs) show great promise in providing solutions to
signal processing or signal-noise reduction problems. In fact, these solutions
might be seen as more natural or intuitive when compared with traditional
techniques. As this note will indicate, however, there are a number of issues
and considerations that have to be faced when applying theory to real-world
situations. A few more words on how a neural network does signal processing
might help to clarify this point.
Recall that back-propagation networks configure themselves in such a way as to
establish a relationship between input and output variables that minimizes the
error between the network generated output and the exact or target output (in
the sense of least squares fitting). After minimizing the error or training
the network, we can say that it has learned to input/output relation. What's
amazing is that a network can and will produce an output close to and
consistent with the target value, even though one or more of the inputs have
been corrupted. If you still aren't impressed, remember that neural nets are
nonlinear devices! Again, inputs can be noisy, but networks usually generalize
well enough to produce an acceptable output. For example, we can train a
network on the input sequence of (1.0 2.0 3.0 4.0 5.0) with a target of (6.0)
and then input (1.0 3.0 3.2 4.1 4.9) to obtain (5.8).
Our working strategy is as follows: We pick an input sequence of data points
and then proceed to train the network using (as a target) the next point in
the signal sequence. We might, for instance, have a two input data and the
sixth as a target; for example, every third datum is a target. This way, we
encode the time structure of our signal directly into the network. Next, the
network is trained on a set of input sequences that together form the shape
(or representation) of a typical signal. Following this, we have a network to
process the noisy data.
To illustrate the process in an engineering application, Figure 1a shows the
training or target signal for a medical diagnostic (hemoglobin red blood cell)
device and the network output after different numbers of training cycles. For
our data a few hundred training epochs resulted in better than a one percent
difference between target and output. Figure 1b shows the end result of
applying this network on a noisy set of data. For this study I used either
two, five, or ten inputs along with ten hidden units; the presented results
are for the case of five inputs.
Now to important considerations when putting neural network theory into
practice. First, if we use a small input window (for instance, the number of
inputs is one or two), we will have a fine sampling of our signal. As such, we
will need a large number of hidden units to correctly encode the character of
the signal. This in turn involves more data processing, which could prohibit
real-time response without the use of parallel processors. Furthermore, with
only two inputs, if one or both are noisy, the success of the network in
distinguishing between signal and noise is diminished. Therefore, we might
want to increase the size of the input window in order to maximize the
network's noise handling capabilities. Remember, however, that we're also
trying to maintain as fine a sampling as possible to maximize signal detail.
The question then becomes: How many inputs and how many hidden units?
In general, the task is to find a balance between a number of variables
(besides the number of units), which at the present are more or less
determined empirically. In addition, we might decide to introduce feedback
(networks of this type are sometimes called Jordan networks) from either input
or output to further enhance the network's memory of the signal's time
structure. As mentioned in the main article, many of these options can be
explored more quickly using one of the commercially available network
simulation packages. For my work, I used, and highly recommend, The Cognitron
by Cognitive Software.
Figure 1c shows a final plot of our data representing the arithmetic
difference between the two curves of Figure 1b. You can see this difference is
relatively small, except for a number of spikes. Figure 1c is marked with two
horizontal lines to indicate these large variance points. In fact, these
spikes emerge just where we have a random noise pulse interposed as part of
the data!
To conclude, the network distinguished the signal from the noise without using
multiband filters, Fourier-Laplace transforms, and the like. You might even go
further to conclude that artificial neural networks seem to have an almost
human capability to process difficult, intrinsically nonlinear signal shapes.

_NEURAL NETS AND NOISE FILTERING_
by Casey Klimasauskas



[LISTING ONE]

/* (backprop.c) Back-propagation XOR example */

#include <stdio.h>

/************************************************************************
 * Back-propagation Exclusive OR Program *
 ************************************************************************
 Written by Casimir C. Klimasauskas. No copyright or other
 proprietary rights reserved.
 This program was compiled and tested using DataLight "C" version 3.14
 as follows: DLC -ms backprop.c
 NOTE: the structure "_conn" uses an index for the PE source. This was
 done (rather than a pointer to the processing element itself) to get
 around certain problems with handling circular references in structure
 definitions.
 */

#define MAXCONN 5 /* maximum number of conn */

typedef struct _conn { /* connection to a PE */
 int PESource; /* index of PE source */
 float ConnWt; /* connection Wt */
 float LastDelta; /* last weight change */
} CONN;

typedef struct _pe { /* processing element */
 float Output; /* PE output */
 float Error; /* Accumulated error */
 CONN Conns[MAXCONN+1]; /* connections */
} PE;

/************************************************************************
 * Network Topology *
 ************************************************************************

 The following diagram shows how the processing elements have been
 connected to solve the exclusive "OR" problem. This is taken from
 Volume 1 of "Parallel Distributed Processing" by Rummelhart and
 McClelland.

 +--------------------- PE5
 / \
 / \
 +-----------------/----PE4 \
 / \ 
 / \ 
 / \
 PE1 - bias PE2 PE3

*/
static PE pe1 = { 1.0, 0 }; /* bias */
static PE pe2 = { 0 }; /* inputs */
static PE pe3 = { 0 };
static PE pe4 = { 0, 0, 1,0,0, 2,0,0, 3,0,0 }; /* hidden */
static PE pe5 = { 0, 0, 1,0,0, 2,0,0, 3,0,0, 4,0,0 }; /* output */

/* --- Processing Elements (for reference by number) ---- */
static PE *PEList[] = { (PE *)0, &pe1, &pe2, &pe3, &pe4, &pe5 };

/* --- Layer definitions --- */
static PE *LayIn[] = { &pe2, &pe3, (PE *)0 }; /* input layer list */
static PE *LayMid[] = { &pe4, (PE *)0 }; /* hidden layer list */
static PE *LayOut[] = { &pe5, (PE *)0 }; /* output layer list */

/* --- Network List --- */
static PE **LayList[] = { &LayIn[0], &LayMid[0], &LayOut[0], (PE **)0 };

/************************************************************************
 * Sigmoid() - Compute the sigmoid of a value *
 ************************************************************************
 */
double sigmoid( x )
 double x;
 {
 double r; /* result */
 extern double exp();

 /* check special limiting cases to prevent overflow */
 if ( x < -10. ) r = 0.0;
 else if ( x > 10. ) r = 1.0;
 else r = 1.0 / (1.0 + exp( -x ));

 return( r );
 }

/************************************************************************
 * RRand() - Compute a random number in a range *
 ************************************************************************
 */
double RRand( low, high )
 double low, high; /* low / high limits */
 {
 double r; /* return result */
 extern int rand(); /* random number generator */


 r = (rand() / 32767.) * (high - low) + low;

 return( r );
 }

/************************************************************************
 * RandWts() - randomize all of the weights in a network *
 ************************************************************************
 */
void RandWts( low, high, LLp )
 double low, high; /* low / high limits for random */
 PE ***LLp; /* layer list pointer */
 {
 PE **PePP; /* PE Pointer */
 PE *PeP; /* PE itself */
 CONN *WtP; /* connection Pointer */

 for( ; (PePP = *LLp) != (PE **)0; LLp++ ) /* layer loop */
 for( ; (PeP = *PePP) != (PE *)0; PePP++ ) /* PE loop */
 for( WtP = &PeP->Conns[0]; WtP->PESource != 0; WtP++ )
 {
 WtP->ConnWt = RRand( low, high );
 WtP->LastDelta = 0.0;
 }
 return;
 }

/************************************************************************
 * Recall() - Recall information from the network *
 ************************************************************************
 */
void Recall( ov, iv, LLp, rcf )
 double *ov; /* output vector */
 double *iv; /* input vector */
 PE ***LLp; /* layer list pointer */
 int rcf; /* "recall" mode flag (0=learn) */
 {
 PE **PePP; /* PE Pointer */
 PE **LastPP; /* last non-zero PE list pointer */
 PE *PeP; /* PE itself */
 CONN *WtP; /* connection Pointer */
 double sum; /* weighted sum */

 /* copy the input vector to the inputs of the network */
 for( PePP = *LLp++; (PeP = *PePP) != (PE *)0; PePP++ )
 PeP->Output = *iv++;

 /* compute the weighted sum and transform it */
 for( ; (PePP = *LLp) != (PE **)0; LLp++ ) /* layer loop */
 {
 LastPP = PePP;
 for( ; (PeP = *PePP) != (PE *)0; PePP++ ) /* PE's in a layer */
 {
 /* weighted sum of the inputs */
 sum = 0;
 for( WtP = &PeP->Conns[0]; WtP->PESource != 0; WtP++ )
 sum += WtP->ConnWt * PEList[ WtP->PESource ]->Output;


 /* transform it using a sigmoidal transfer function */
 PeP->Output = sigmoid( sum );

 /* if "learn" mode, set the error to zero */
 if ( rcf == 0 ) PeP->Error = 0.0; /* (for learning) */
 }
 }

/* copy the results to the output array */
if ( rcf != 0 ) /* only if not learning */
 {
 for( ; (PeP = *LastPP) != (PE *)0; LastPP++ )
 *ov++ = PeP->Output;
 }
 return;
 }

/************************************************************************
 * Learn() - "learn" an association *
 ************************************************************************
 */
double Learn( ov, iv, LLp, alpha, eta )
 double *ov; /* output vector */
 double *iv; /* input vector */
 PE ***LLp; /* layer list pointer */
 double alpha; /* learning rate */
 double eta; /* momentum */
 {
 double MAErr; /* Maximum Absolute error */
 double rv; /* work value */
 double SigErr; /* back-propagated error */
 PE ***ALp; /* alternate layer pointer */
 PE **PePP; /* PE Pointer */
 PE **LastPP; /* last non-zero PE list pointer */
 PE *PeP; /* PE itself */
 CONN *WtP; /* connection Pointer */
 extern double fabs(); /* absolute value */

 Recall( ov, iv, LLp, 0 ); /* perform a recall */

 /* find the output layer */
 for( ALp = LLp; ALp[1] != (PE **)0; ALp++ )
 ;

 /* compute the square error in the output */
 for( MAErr = 0.0, PePP = *ALp; (PeP = *PePP) != (PE *)0; PePP++ )
 {
 rv = *ov++ - PeP->Output; /* output Error */
 PeP->Error = rv;
 if ( fabs(rv) > MAErr ) MAErr = fabs(rv);
 }

 /* back-propagate the error & update the weights */
 for( ; ALp > LLp; ALp-- ) /* layer loop */
 {
 PePP = *ALp;
 for( ; (PeP = *PePP) != (PE *)0; PePP++ ) /* PE's in a layer */
 {
 /* compute the error prior to the sigmoid function */

 SigErr = PeP->Output * (1.0 - PeP->Output) * PeP->Error;

 /* back-propagate the errors & adjust weights */
 for( WtP = &PeP->Conns[0]; WtP->PESource != 0; WtP++ )
 {
 PEList[ WtP->PESource ]->Error +=
 WtP->ConnWt * SigErr;
 rv = alpha * PEList[ WtP->PESource ]->Output * SigErr +
 eta * WtP->LastDelta;
 WtP->ConnWt += rv;
 WtP->LastDelta = rv;
 }
 }
 }
 return( MAErr );
 }

/************************************************************************
 * Main() - main driver routine to train the network *
 ************************************************************************
 */
static double iv1[] = { 0.0, 0.0 }; static double ov1[] = { 0.0 };
static double iv2[] = { 1.0, 0.0 }; static double ov2[] = { 1.0 };
static double iv3[] = { 0.0, 1.0 }; static double ov3[] = { 1.0 };
static double iv4[] = { 1.0, 1.0 }; static double ov4[] = { 0.0 };

static double *ivp[] = { &iv1[0], &iv2[0], &iv3[0], &iv4[0] };
static double *ovp[] = { &ov1[0], &ov2[0], &ov3[0], &ov4[0] };

main()
 {
 int wx; /* work index */
 int x; /* index into samples array */
 double r; /* work value */
 double MAErr; /* maximum Absolute error */
 double wo[sizeof(ivp)/sizeof(*ivp)];

 /* randomize the weights in the network */
 RandWts( -1.0, 1.0, &LayList[0] );

 MAErr = 0.0;
 for( wx = 0; ; wx++ )
 {
 x = wx % (sizeof(ivp)/sizeof(*ivp));
 if ( x == 0 && wx != 0 )
 {
 if ( (wx % 100) == 0 )
 printf( "Presentation %4d, Maximum Absolute Error = %.5f\n",
 wx, MAErr );
 if ( MAErr < .1 ) break;
 MAErr = 0.0;
 }

 r = Learn( ovp[x], ivp[x], &LayList[0], 0.9, 0.5 );
 if ( r > MAErr ) MAErr = r;
 }

 /* test the network */
 for( wx = 0; wx < (sizeof(ivp)/sizeof(*ivp)); wx++ )

 {
 Recall( wo, ivp[wx], &LayList[0], 1 ); /* perform a recall */
 printf( "Input: %.2f %.2f -> %.2f\n", ivp[wx][0], ivp[wx][1], wo[0] );
 }

 return;
 }























































January, 1989
UNIX STREAMS


If modularity and portability are part of your Unix problems, Streams may be
part of the solution




Michael W. Garwood and Andrew E. Schweig


Michael Garwood and Andrew Schweig are developers for Lachman Associates Inc.
and can be reached at 1901 N. Naper Blvd., Naperville, IL 60540-1301. Portions
of this article previously appeared in The European Unix User's Guide.


Not long ago, in an effort to overcome some of the shortcomings associated
with traditional Unix mechanisms for dealing with character-based I/O systems,
a new framework called Streams was introduced and implemented for Unix System
V, Release 3. This Streams framework incorporates several design strengths
that are important to the development of efficient, portable network protocols
for the Unix system.
Streams is a framework that provides for a full-duplex data connection between
a user process and a device or pseudodevice. This connection, termed a stream,
is composed of a module stack with which communication among neighboring
modules on the stack is achieved by passing messages through a well-defined,
intermodule interface. (Throughout this article, the terms module and driver
are used interchangeably; however, device driver refers to the software used
to directly control a particular hardware device or pseudodevice.) Messages
are passed "downstream" on the write half of the stream toward the device
driver and "upstream" on the read half of the stream toward the stream head or
user process. A module passes messages to a neighboring module by calling the
neighbor module's put procedure. Once a message is received by a module, it
may either be passed to a neighboring stream module or be enqueued for
subsequent processing by the module's service procedure.
A simple put and service strategy is shown in Example 1, page 53. The put
procedure (modput):
1. Processes all IOCTL messages immediately
2. Enqueues all messages of type M_DATA
3. Passes all other messages downstream
Example 1: A Streams module put and service procedure

 /*
 * module put procedure
 */
 modput (q, mp)
 queue_t *q;
 mblk_t *mp;
 {
 switch (mp->b_datap->db_type) {

 case M_IOCTL:
 do_ioctl (mp); /* process IOCTLs
 immediately */
 break;

 case M_DATA:
 putq(q, mp); /* enqueue for service
 processing */
 break;

 default:
 putnext (q, mp); /* pass downstream */
 break;
 }
 }
 /*
 * module service procedure
 */
 modsrv (q)
 queue_t *q;
 {
 mblk_t *mp;

 while (mp = getq( )) /* dequeue messages */
 if (canput (q->q_next)) {
 transform(mp); /* local transformation of

 data */
 putnext(q, mp);
 } else {
 putbq(q, mp);
 return;
 }
 }

The service procedure (modsrv) processes as much locally enqueued data as
possible until the downstream queues fill up. At that point the dequeued
message is reenqueued via putbq and the service procedure returns. The
flow-control mechanisms of Streams will reinvoke the service procedure when
the downstream blockage has subsided. Note that putq, put next, getq, canput,
and putbq are standard Streams utilities.
Information about a particular module is included in a pair of structures
called queues. One queue describes the write half of the module, and the other
describes the read half of the module. Information kept by the queue
structures contains, among other things, pointers to module entry points,
state flags, a data queue, and low and high watermarks used for flow control
on the stream.


Streams Benefits


The Streams design provides several advantages over the traditional Unix
method of establishing communication between a user process and character-type
driver. Most notably, it enforces (or at least strongly encourages) modularity
of design. It is certainly possible to write a module that does not conform to
the standard Streams interface among modules, but it would be a lot less
trouble to stay within the Streams' framework. For example, we developed a raw
IP driver to provide a TLI interface to the network-level IP driver without
making any modifications whatsoever to the network driver. In this way the
network driver can multiplex to other drivers (for example, TCP and UDP)
without the overhead imposed by a complex user interface.
The structure imposed by this framework delivers another bonus: portability.
Not only is Streams itself reasonably portable among different machine
environments, but modules that cooperate with the communication principles
laid down by the Streams framework also tend to be fairly portable.
Streams permits logical data (messages) to be split across several buffers.
The capability to efficiently construct large messages from several smaller
components proved to be of great benefit in the design of the data reassembly
portions of IP and TCP drivers we developed. Data can also be efficiently
duplicated (shared) without being copied in the Streams environment. This
feature allows a reliable transmission protocol (such as TCP) to send data and
keep a "copy" for possible retransmission without actually copying the data.
Another nicety in the Streams package is the clone driver feature. This
feature makes it possible to open a driver in "clone" mode and have the driver
itself pick an available minor device number. Gone are the days of a user
process sequencing through a long line of major-minor pairs in order to locate
an available device. The burden is more efficiently handled by the kernel
driver itself. For example, we wrote a socket interface that requires each
socket endpoint be attached to a specific protocol device major-minor pair.
Because the socket creation interface does not require an argument that would
specify a minor device number for a particular protocol driver, it would be
cumbersome to locate an available minor device number without the Streams
clone feature.
The use of this clone feature is demonstrated by the Streams driver open
routine in Example 2, below. If the driver open flag sflag is CLONEOPEN,
denoting a clone open, the driver must allocate and return an unused minor or
return OPENFAIL. A "normal" open should check to see if the minor is already
in use and, if so, should make certain that the queue pointer is the same as
the previous open. This avoids a small window in the current Streams
implementation.
Example 2: A Streams clone device driver open procedure

 struct queue_t *drvqueue [MAXDEV];

 drvopen (q, dev, flag, sflag)
 queue_t *q;
 dev_t dev;
 {
 unsigned int mindev;
 char error = 0;

 if (sflag == CLONEOPEN) {
 for (mindev = 0 ; mindev < MAXDEV; mindev++) /* allocate
 dev */
 if (drvqueue [mindev] == NULL)
 break;
 if (mindev >= MAXDEV)
 return (OPENFAIL);
 } else {
 mindev = minor(dev);
 if (mindev >= MAXDEV)
 error = ENXIO;
 else
 if (drvqueue [mindev] &&
 drvqueue [mindev] != q) /* one at a
 time */
 error = EBUSY; /* maybe
 EAGAIN */
 }
 if (error)
 u.u_error = error;
 else
 drvqueue[mindev] = q;
 return (mindev); /* return minor for clone
 driver */
 }

The resource allocation scheme used by the Streams package is simple and fast.
Because the resource pools (buffers, queues, message headers, and so on) are
statically allocated, freeing or allocating a resource is as simple as linking
or unlinking a pointer from a free list. This simplicity, however, does have
its drawbacks, as we will discuss later.



Streams Caveats


Despite the aforementioned advantages of Streams, there are still a few areas
in which the System V, Release 3, Streams implementation requires attention.
The relative severity of any of these items depends, for the most part, on the
particular system being implemented. The amount of memory allocated to a
particular Streams resource pool (that is, streams, queues, buffers, and so
forth) is determined by various configuration parameters. No dynamic
capability is currently built into Streams resource management. Once you've
hit a resource limit, you've hit it.
One twist with respect to resource management concerns data buffer allocation.
The buffer allocation scheme utilized by Streams divides data buffers into
nine, different, statically configured size classes (4, 16, 32, 64, 128, 256,
512, 1,024, and 4,096 bytes). Although the Streams allocation algorithm tries
a couple of different classes in an attempt to satisfy a buffer request, this
request may fail even when the buffer pool on the whole N is relatively
underutilized (for example, a request for a buffer in class N may fail when
there is a large number of buffers available in classes other than N or N+1).
These allocation failures may occur quite frequently on a heavily loaded or
improperly configured system; therefore, the driver must at all times be
prepared for such a failure, allowing for graceful recovery of the device or
system. It may be the case that the processing module is in a state whereby it
can wait synchronously for an available buffer; most of the time, however,
this is not the case as a module has no process context after open and before
close. (Normal character-based drivers have this restriction for
interrupt-side processing only.) This Streams design choice prevents service
routines from running as independent processes (which would allow them to
sleep whenever necessary), makes performance measurements difficult, and makes
necessary asynchronous buffer allocation recovery measures.
Streams provides a mechanism for asynchronous recovery from buffer allocation
failures. The driver interface to this mechanism, the function bufcall,
schedules an event to notify the driver when a buffer of a particular size
becomes available. At least three problems can occur here. First of all, this
notification is not a guarantee that the awaited buffer will be obtainable by
the driver; thus, the driver must be prepared for another allocation failure.
Second, bufcall may fail because the event structures that it uses are also
statically allocated and can be depleted at times of heavy load. Finally, the
buffer may become available and the event may be triggered too late, forcing
the driver to take (sometimes complicated) actions to avoid potential
problems.
We have attempted to get around some of the buffer allocation shortcomings by
building our own allocation routine around the Streams buffer allocator,
allocb. This new allocator tries a little harder than allocb by waiting for
buffers requested at low priority and/or attempting to cut and paste together
buffers from different-size classes to fulfill the size requested. In the
cases in which bufcall is required after an allocation failure, we check the
validity of the function argument supplied by the buffer event. In the event
that bufcall should fail, we use timeout. Example 3, this page, is a routine
that tries very hard to pass a buffer to its neighbor. Note that the buffer
may or may not have actually been sent upon return to the original caller.
Example 3: Usage of bufcall

 sendbuf(q)
 queue_t *q;
 {
 mblk_t *bp;

 if (!(bp = alloc(BUFSIZE, BPRI_MED))) {
 if (!bufcall (BUFSIZE, BPRI_MED, sendbuf, q))
 timeout (sendbuf, q, HZ); /* if all else
 fails */
 return(0);
 }
 putnext (q, bp);
 return (1);
 }

Each open driver has an associated pair of queues (one for read, one for
write) that contains, among other things, low and high watermarks that are
used to control the flow from neighboring modules. Although these may be
useful quantities to dynamically configure, Streams does not provide an
administrative interface that would support this easily. Although Streams does
define a special message type (M_SETOPTS) to allow a Streams module to set
streamhead read queue parameters, no method is provided for setting parameters
of other queues. We got around this problem to a certain extent by providing
each one of our drivers with an IOCTL that sets the low and high watermarks
for the various queues associated with the driver.
The driver queues also contain information about the module entry points and
data service routines. As such, these queues are important in scheduling the
module to run. These queues, however, are deallocated unconditionally when the
module is closed. Drivers such as TCP that still have processing to do after
the close returns to the user process can no longer be scheduled using the
ordinary queue-scheduling mechanism. We overcame this problem by using our own
built-in, queue-independent scheduler after the TCP queues were deallocated.
An alternative may have been to allocate a new pair of queues before the
original pair were deallocated, although this allocation may fail because of
static allocation problems discussed previously.
The Streams message architecture that allows data to be shared among multiple
messages also gives rise to a minor inconvenience. Given a reference into the
data portion of a buffer, it is difficult to find a reference back to the
buffer header itself. Our fragment reassembly code, for instance, uses fields
within the IP header itself to sequence incoming packet fragments properly;
however, when the time comes for the assembled packet to be passed upstream,
it is necessary to find the corresponding buffer header. We have circumvented
this problem by temporarily overlaying the header pointer onto the IP header
itself.


Streams Impact on High-Level Services


For the most part, Streams should be relatively transparent to a typical user
program or high-level application; however, there are a couple of new features
that are visible at the user level. First of all, there are new system call
(getmsg and putmsg) that can be used in addition to read and write to read and
write a stream endpoint. These new system calls enable a user process to
receive and send packetized control and normal stream data. A poll system
call, similar to the Berkeley select call, permits a user process to monitor
several stream endpoints for various events, including data arrival and
departure. Streams also provides several useful IOCTLs, which, among other
things assist in stream-administration, access control, and interprocess
communication.- A summary of these IOCTLs is given in Table 1, page 56.
Table 1: Streams' IOCTLs

 I_STR Send IOCTL message on stream
 I_LINK Link two streams
 I_UNLINK Unlink streams linked via I_LINK
 I_PUSH Push module onto stream below stream
 head
 I_POP Pop top module from stream
 I_LOOK Return name of first module on stream
 I_FIND Look for particular module on stream
 I_SETSIG Arrange for asynchronous stream event
 notification
 I_GETSIG Return registry of asynchronous events
 for which process is to be signaled
 I_PEEK Nonconsumptive read of stream
 I_NREAD Return size of first stream message
 I_FLUSH Send flush message on stream
 I_SRDOPT Set read characteristics of the stream
 head read queue
 I_GRDOPT Get read characteristics of the stream
 head read queue
 I_SENDFD Pass file descriptor through stream
 I_RECVFD Retrieve and set up previously sent
 (I_SENDFD) file descriptor
 I_FDINSERT Pass characteristic of one stream to

 another stream (for stream pipe creation)

The mechanics for sending an IOCTL are slightly different for a Streams device
than for a "normal," character-based device driver. An IOCTL on a non-Streams
device is fairly simple:
 ioctl(fd, IOCFOO, &arg);
For a Streams device, issuance of the IOCTL requires more preparation:
 struct strioctl strioc;

 strioc.ic_cmd = IOCFOO; strioc.ic_timout = -1; /* seconds, -1 ==
forever */ strioc.ic_len = sizeof(arg); strioc.io_dp = (char *) &arg; ioct(fd,
I_STR,
&strioc);


Conclusion


Despite any limitations that may exist in its current implementation under
Unix System V, Release 3, Streams has clearly been demonstrated as a practical
framework for networking. The general premise around which Streams is based is
a sound one; the problems that arise in using the framework are more related
to correctable defects in implementation than to any flaws in the main
structure itself. As a relatively new technology, Streams will certainly be
improved through new advances by AT&T and other Streams implementors.


Acknowledgments


The authors wish to thank Steve Alexander, Mike Eisler, and Michele Merens for
their assistance.


Bibliography


Ritchie, D.M. "A Stream Input-Output System." AT&T Bell Laboratories Technical
Journal 63(8) (October 1984).
Unix System V Release 3 Streams Programmer's Guide. Englewood Cliffs:
Prentice-Hall, 1986.
Presotto, D.L., and Ritchie, D.M. "Interprocess Communication in the Eighth
Edition Unix System." USENIX Conference Proceedings (Summer 1985).
































January, 1989
COMPARING MODULA-2 AND C++


These two languages have a lot in common -- but how they're different is what
really matters




Scott Robert Ladd


Scott Ladd is a full-time freelance computer journalist. You can reach him at
P.O. Box 61425, Denver, CO 80206.


Two of the hottest languages these days are Modula-2 and C++. They appeared in
the early 1980s, and both are gaining in popularity. Each was designed and
written by a single person: Niklaus Wirth created Modula-2, and C++ was
developed by Bjarne Stroustrup. An additional similarity is that both
languages are extended versions of earlier languages. This article contrasts
Modula-2 and C++; it assumes you have a cursory familiarity with both Pascal
and C (the root languages of Modula-2 and C++, respectively).


Modula-2 Background


Pascal was designed as a teaching language, to show students how to structure
programs properly. It lacks intrinsic support for character strings, file I/O,
and separately compiled modules. Pascal has many features that are useful in
software development, and the language gradually gained popularity outside the
academic community. Unfortunately, the vendors that implemented Pascal did not
develop a standard for extending the language to cover its weak points. Thus,
every Pascal compiler is unique, and porting programs between implementations
is difficult.
Wirth intended Modula-2 to be the successor to Pascal. Modula-2's name implies
one of its basic concepts: modular program design. Through the use of modular
facilities, Modula-2 supports data abstraction and encapsulation. All I/O is
done through procedures in modules as an aid to portability. As long as the
interface to the I/O procedures does not change, the implementation of those
procedures can be modified for different environments without having to make
alterations in the programs using them.
Among high-level languages, only Modula-2 and Ada support multiprocessing. In
addition, Modula-2 supports bit manipulation, generic types, and system-level
access. Modula-2 (unlike Pascal) is well suited to the writing of system
software such as operating systems and device drivers.


C++ Background


C is a powerful system-level language; in its original definition, however, it
lacked many features necessary for large, complex projects. Stroustrup
designed C++ to amend problems with the C language, in much the same way as
Wirth designed Modula-2 to correct the deficiencies of Pascal. C++ provides
capabilities for strong type checking, modular programming, and data
abstraction while maintaining full compatibility with existing C programs.
Stroustrup borrowed the object-oriented paradigm from Smalltalk; classes and
methods are the most significant additions he added to C in creating C++. In
an object-oriented language, objects (data items) belong to classes (types),
which have associated methods (functions). "Messages" are sent to objects via
their class methods; the messages tell the objects how to act. For example, an
object of class int would be given the message "add 1" by the ++ (increment)
method. Although this concept can take some getting used to, it can be more
appropriate than traditional program design methods.
C++ is often implemented as a translator. The C++ translator works much like
the standard C preprocessor, converting C++ programs into C programs. A C
compiler is then run on the output of the translator, producing executable
code. By its nature, this process is cumbersome and slow. Recently,
manufacturers have released true C++ compilers.
In addition to adding object-oriented capabilities to C, C++ offers other
enhancements, including in-line functions, function prototyping (since added
to the ANSI standard), and overloading. Because it retains all C's low-level
facilities, C++ can be used for a wide variety of applications.


General Language Features


Modula-2 and C++ have the following general features:
Source code format -- Source code is case sensitive in C++ and Modula-2. All
C++'s keywords must be in lowercase; Modula-2 requires uppercase.
Data types -- Both languages are rich in data types and allow the creation of
new types. They offer long and short, signed and unsigned, integers and reals.
Strings in both languages are handled as arrays of characters terminated by a
zero (NUL) byte. Both languages have pointer types. C++ provides the
capability to smoothly integrate new types through its class structure, while
Modula-2 provides generic types and better control over data item visibility.
Modula-2 provides SET types (including BITSET, which allows access to the
individual bits of an item), whereas C++ has C's bit structures.
Functions and procedures -- Modula-2 has procedures, whereas C++ has
functions. The purpose is the same in both languages: to provide callable
routines with parameters and local variables. Modula-2 allows the nesting of
procedures within procedures, using the same scoping rules as it does for
variables. Modula-2 and C++ parameters can be passed either by reference or by
value.
Control structures -- Here, too, there are no important differences. Although
the syntax may vary, the capability is the same. Both languages have loops
with tests at both the top and the bottom of the loop. The for, which iterates
through a succession of values, is available in both languages. There is (of
course) the ubiquitous if ..else ..endif conditional construct. Useful
multiple-condition branches (switch in C++ and CASE in Modula-2) are
available.
Library routines -- C++ uses the standard C library, which is extensive and
robust. Wirth defined several standard modules in his definition of Modula-2,
but these provided only minimal capabilities. Most Modula-2 vendors have
created expanded libraries for their implementations.


Unique Features of Modula-2


Modula-2 is not a superset of Pascal; rather, it is an evolution of the
earlier language. For example, one of Modula-2's most welcome enhancements
over Pascal is its simplified block structure. A Modula-2 program is not
filled with BEGIN...END pairs; each control structure has an implicit block
terminated by an END statement. Where Pascal has functions and procedures (the
former returns a value whereas the later does not), Modula-2 simply has
procedures. A Modula-2 procedure returns a value if it is specified as doing
so.
One fascinating feature of Modula-2 is its support for coroutines, which are
individual processes within a program that run concurrently. This allows
multiple tasks (within a program) to be performed simultaneously. A word
processor, for example, could use coroutines to allow a document to be printed
while another is being edited.
The strong type checking in Pascal prevents spurious errors but also causes
difficulties. For instance, it is impossible to create a general function in
standard Pascal that can accept arrays (such as character strings) of
different sizes. Modula-2 introduces the concept of the open array parameter
-- for example, a parameter to a procedure can be declared as an array without
bounds (array of char). Any length array of the specified type (in this case,
a character array) can be passed to that function. The procedure can then find
out the actual length of the array with the built-in HIGH procedure.
Modula-2 also provides the generic type. A WORD type is equal in size to the
default word size of the hardware it is running on. In the case of MS-DOS, a
WORD type is 16 bits long. Any type of the same size (usually the INTEGER and
CARDINAL types) is assignment compatible with WORD. An extension of this is
that any value can be passed to an open array of type WORD (that is, a
procedure parameter of type ARRAY OF WORD). The size of the array is the
number of WORD types required to hold the value being passed.
The concept of modules allows Modula-2 programmers to control access to
individually compiled portions of a program. The following object-oriented
example illustrates how modules can be used.


Unique Features of C++



C++ refines and expands C while including all C's strengths. Several features
of the emerging ANSI C standard are actually borrowed from C++. Examples of
this are function prototypes, const, and void. Function prototypes were added
so that C++ could do type checking or arguments; under the original K&R C,
values of incorrect types could be passed to functions, often causing obscure
errors. Const provides named constants. Adding void allowed the creation of
generic pointers and made it possible to declare that a function does not
return a value.
Overloading functions lets the programmers create several functions with the
same name but differentiated by their parameters. Instead of C's current crop
of absolute value functions (abs, dabs, fabs, and labs), a C++ implementation
could have the following:
 overload abs;
 int abs(int i);
 long abs(long l);
 float abs(float f);
 double abs(double d);
At compile time, the C++ compiler determines which version of abs to use based
on the parameters it is being passed. This facility can make programs easier
to understand.
It is possible to overload operators in C++. You can, for instance, create a
new class that can use the same operators as existing classes. The section on
object-oriented programming later in this article contains an example of
operator overloading.
The object-oriented features of C++ are prominent. Object-oriented programming
requires a change in thinking for many programmers; instead of thinking in
terms of the nuts and bolts of programming, programmers are required to
visualize a program as a series of processes applied to data items. This may
seem to be a subtle distinction, but it must be mastered to truly appreciate
and use an object-oriented language. The object-oriented features of C++ are
illustrated in the section on the subject in this article.
C++ provides dozens of other extensions to C, including in-line functions,
anonymous (unnamed) unions, and default function parameter values. There are
dangers to the many features provided by C++. Whereas C has always been famous
for providing the rope by which programmers hang their programs, C++ adds the
noose. Care must be taken to avoid "going wild," especially with the overload
capabilities.


An Object-Oriented Example


An object-oriented language is extensible, which means the programmer can
create new data objects that are integrated into a program. In order to show
how each language is used to create a new class of objects, I have implemented
complex numbers in each language. Complex numbers are a superset of the real
numbers, having both a real and an imaginary part. Complex numbers are used in
a number of scientific calculations; this is one reason why Fortran directly
supports complex numbers.
Both implementations are identical in function. First, you need to define the
data elements of a complex number. In this case, the complex number has two
floating-point components for its real and imaginary parts. Then, you
determine what operations can be performed on complex numbers. In the example,
the allowed operations are assignments, addition, subtraction, multiplication,
division, and output. I used Zortech C++, Version 1.06, and JPI TopSpeed
Modula-2, Version 1.11, to develop the examples.
In C++, a class is generally developed using two files. A header file contains
the class description, and a source file holds the actual implementation. The
example in Listings One - Three, page 102, follows this form.
A class in C++ looks like a structure; in fact, a structure is a special form
of class. Those items listed in the private section cannot be accessed outside
the class definition. Public data and functions are available for use outside
the class scope. The example has only two private items, which are the two
floating- point components of a complex value. (See Listings Four - Six, page
104.)
A "constructor" in C++ is called whenever an object of the class is created to
initialize the object. The counterpart of a constructor is a "destructor,"
which can be used to deallocate any resources used by an object (once that
object is no longer needed). The example does not require or implement a
destructor.
Function overloading is used to provide several different constructors. Each
constructor allows an object of class complex to be declared with different
initialization values. The type and number of the initializers determines
which constructor is used. The following code fragment shows how C++
determines which constructor to use:
 complex a; /* no values; uses 1st constructor */

 complex b(2.0,5.0); /* two real values; uses 3rd constructor */

 complex c(b); /* complex value; uses 2nd constructor */
Note that the first constructor is defined in the class definition; there is
no external function for it. This constructor is compiled into in-line code,
avoiding the overhead of a function call when declaring complex values without
initializers.
Modula-2 does not support a specific class structure, but the same effect can
be created through the use of modules. Modula-2 uses two files to create
modules called the definition and implementation modules. The definition
module defines the interface to the data elements and procedures stored in the
implementation. Unless it is defined in the definition module, an item is
private to the implementation.
In order to prevent programmers from manipulating the component values of a
complex item directly, an "opaque type" is used. The definition module lists
the type COMPLEX (allowing programs to create items of that type) but does not
expose the type's internal structure. Because Modula-2's opaque types must be
pointers, the implementation module defines the COMPLEX type as a pointer to a
structure containing two real values. The situation requires the functions
Create and Destroy in order to allocate and deallocate space for the complex
numbers. These are similar to C++ constructors and destructors, but they must
be called explicitly by the program.
Modula-2 does not support overloading, so there is no way to assign functions
to infix operators. C++ allows functions to be assigned to operators, so in
C++ it is possible to say:
 a = b + (c * d);
Modula-2 requires a less readable construct:
 Multiply(temp1,c,d); Add(a,b,temp1);
The process is identical; both languages use functions to simulate operators.
The primary difference is that C++'s notation is more natural, making it
easier for programmers to discern what is happening. Although the results are
the same, most programmers would prefer the clarity of the C++ version.
Both implementations provide a function for displaying the value of complex
number. The Modula-2 version uses a procedure that calls upon standard library
procedures. In the C++ version, a function is declared that accesses the
stream output functions of C++. The stream class is provided with C++. The <<
(left shift) operator has been overloaded in C++ to say "send the object on
the right to the object on the left." Streams can be easier to use for simple
output than library function calls. Complex numbers are displayed using
built-in stream output functions for characters, strings, and floating-point
numbers.
C++ offers other object-oriented features not shown in the example. New
classes can be "derived" from other, pre existing classes, and characteristics
can then be "inherited" from the original class.


Implementations


C++ and Modula-2 are young languages. Modula-2 has been more widely
implemented; on the other hand, it has had slightly longer to get into the
mainstream. C++ is just beginning to come into its own outside the Unix/AT&T
environment. Currently, half a dozen Modula-2 compilers are available for the
PC, whereas three C++ preprocessors and one C++ compiler are available. In the
Macintosh world, Apple has announced that it will extend its C compiler to be
a C++, and there are at least two Macintosh Modula-2 compilers.
The situation is similar when it comes to third-party add-on libraries. I know
of only one company that provides object libraries for C++. There are a few
companies that market products for Modula-2. Recently, several vendors of C
libraries have begun to market Modula2 versions of their products. As the
popularity of these products grows, so will the third-party support.


Conclusion


Modula-2 and C++ are powerful languages; both have bright futures. C++'s
classes, methods, and overloading make it a powerful tool, but it has few
restraints to keep programmers from being too "creative." Modula-2 is a
strongly organized language with restraints to keep programmers within
guidelines. It is not as extensible as C++, but it does implement its own
unique features, such as SETs, coroutines, and open arrays. Which of these
languages is more appropriate for a specific project will depend on the type
of application and the experience of the programmers involved. Both languages
are powerful and interesting to work with.


Bibliography


1. Stroustrup, Bjarne. The C++ Programming Language. Englewood Cliffs, NJ.:
Addison-Wesley, 1986.
2 Wirth, Niklaus. Programming in Modula-2. 3rd ed. New York,: Springer-Verlag,
1985.



_C++ VERSUS MODULA-2_

by Scott Ladd


[LISTING ONE]

=======================
[LISTING 1: COMPLEX.HPP]
=======================

// Header: Complex
// Version: 1.00
// Date: 10-Sep-1988
// Language: C++
// Purpose: Provides the class "complex" for C++ programs.
// Copyright 1988 by Scott Robert Ladd. All Rights Reserved.

#include "stream.hpp"

class complex
 {
 private:
 double real; // real part
 double imag; // imaginary part

 public:
 // constructors
 complex (void)
 {
 real = 0.0;
 imag = 0.0;
 }

 complex (complex &c);
 complex (double &r, double &i);

 // value extraction methods
 double get_real (void);
 double get_imag (void);

 // assignment method
 void assign (double &r, double &i);

 // calculation methods
 complex operator = (complex &c);
 complex operator + (complex &c);
 complex operator - (complex &c);
 complex operator * (complex &c);
 complex operator / (complex &c);

 // output method
 friend ostream& operator << (ostream &s, complex &c);
 };







[LISTING TWO]

=======================
[LISTING 2: COMPLEX.HPP]
=======================

// Module: Complex
// Version: 1.00
// Date: 10-Sep-1988
// Language: C++
// Purpose: Provides the class "complex" for C++ programs.
// Copyright 1988 by Scott Robert Ladd. All Rights Reserved.

#include "complex.hpp"
#include "stream.hpp"

// constructor: copy initializer
complex::complex (complex &c)
 {
 real = c.real;
 imag = c.imag;
 }

// constructor: real and imaginary parts specified
complex::complex (double &r, double &i)
 {
 real = r;
 imag = i;
 }

// retrieve real portion
double complex::get_real (void)
 {
 return real;
 }

// retrieve imaginary portion
double complex::get_imag (void)
 {
 return imag;
 }

// set the value of a complex object to a pair of real values
void complex::assign (double &r, double &i)
 {
 real = r;
 imag = i;
 }

// set the value of a complex number
complex complex::operator = (complex &c)
 {
 real = c.real;
 imag = c.imag;

 return *this;
 }


// add two complex numbers
complex complex::operator + (complex &c)
 {
 complex res;

 res.real = real + c.real;
 res.imag = imag + c.imag;

 return res;
 }

// subtract two complex numbers
complex complex::operator - (complex &c)
 {
 complex res;

 res.real = real - c.real;
 res.imag = imag - c.imag;

 return res;
 }

// multiply two complex numbers
complex complex::operator * (complex &c)
 {
 complex res;

 res.real = (real * c.real) - (imag * c.imag);
 res.imag = (imag * c.real) + (real * c.imag);

 return res;
 }

// divide two complex numbers
complex complex::operator / (complex &c)
 {
 complex res;
 double r, den;

 if (fabs(c.real) >= fabs(c.imag))
 {
 r = c.imag / c.real;
 den = c.real + r * c.imag;
 res.real = (real + r * imag) / den;
 res.imag = (imag - r * real) / den;
 }
 else
 {
 r = c.real / c.imag;
 den = c.imag + r * c.real;
 res.real = (real * r + imag) / den;
 res.imag = (imag * r - real) / den;
 }

 return res;
 }

// stream output of complex number

ostream& operator << (ostream &s, complex &c)
 {
 char buf[80];

 sprintf(buf,"%1g%+1gi",c.real,c.imag);
 return (s << buf);
 }







[LISTING THREE]

=======================
[LISTING 3: COMPTEST.CPP]
=======================

// Program: CompTest
// Version: 1.00
// Date: 10-Sep-1988
// Language: C++
// Purpose: Provides the class "complex" for C++ programs.
// Copyright 1988 by Scott Robert Ladd. All Rights Reserved.

#include "complex.hpp"

void main()
 {
 complex a(1.0,2.0),
 b(-1.5,-5.5),
 c;

 cout << "a = " << a << " b = " << b << " c = " << c << "\n";
 c = a + b;
 cout << c << "\n";
 c = c - b;
 cout << c << "\n";
 c = b;
 cout << c << "\n";
 c = b * a;
 cout << c << "\n";
 c = c / a;
 cout << c << "\n";
 c = b / b;
 cout << c << "\n";
 }





[LISTING FOUR]

=======================
[LISTING 4: COMPLEX.DEF]
=======================


DEFINITION MODULE Complex;
(*
 Version: 1.00
 Date: 11-Sep-1988
 Language: Modula-2
 Purpose: Provides the type "complex" for Modula-2.
 Copyright 1988 by Scott Robert Ladd. All Rights Reserved.
*)

TYPE
 COMPLEX;

PROCEDURE Create(VAR C : COMPLEX);

PROCEDURE Destroy(VAR C : COMPLEX);

PROCEDURE GetReal(C : COMPLEX) : LONGREAL;

PROCEDURE GetImag(C : COMPLEX) : LONGREAL;

PROCEDURE Assign(VAR C : COMPLEX; R, I : LONGREAL);

PROCEDURE Equate(VAR C1 : COMPLEX; C2 : COMPLEX);

PROCEDURE Add(VAR C : COMPLEX; C1, C2 : COMPLEX);

PROCEDURE Sub(VAR C : COMPLEX; C1, C2 : COMPLEX);

PROCEDURE Mult(VAR C : COMPLEX; C1, C2 : COMPLEX);

PROCEDURE Div(VAR C : COMPLEX; C1, C2 : COMPLEX);

PROCEDURE WriteComplex(C : COMPLEX);

END Complex.






[LISTING FIVE]

=======================
[LISTING 5: COMPLEX.MOD]
=======================

IMPLEMENTATION MODULE Complex;
(*
 Version: 1.00
 Date: 11-Sep-1988
 Language: Modula-2
 Purpose: Provides the type "complex" for Modula-2.
 Copyright 1988 by Scott Robert Ladd. All Rights Reserved.
*)

FROM Str IMPORT FixRealToStr;
FROM IO IMPORT WrStr, WrChar;

FROM Storage IMPORT ALLOCATE, DEALLOCATE;

TYPE
 COMPLEX = POINTER TO RECORD
 Real : LONGREAL;
 Imag : LONGREAL;
 END;

PROCEDURE Create(VAR C : COMPLEX);
 BEGIN
 NEW(C);
 C^.Real := 0.0;
 C^.Imag := 0.0;
END Create;

PROCEDURE Destroy(VAR C : COMPLEX);
 BEGIN
 DISPOSE(C);
END Destroy;

PROCEDURE GetReal(C : COMPLEX) : LONGREAL;
 BEGIN
 RETURN C^.Real;
END GetReal;

PROCEDURE GetImag(C : COMPLEX) : LONGREAL;
 BEGIN
 RETURN C^.Imag;
END GetImag;

PROCEDURE Assign(VAR C : COMPLEX; R, I : LONGREAL);
 BEGIN
 C^.Real := R;
 C^.Imag := I;
END Assign;

PROCEDURE Equate(VAR C1 : COMPLEX; C2 : COMPLEX);
 BEGIN
 C1^.Real := C2^.Real;
 C1^.Imag := C2^.Imag;
END Equate;

PROCEDURE Add(VAR C : COMPLEX; C1, C2 : COMPLEX);
 BEGIN
 C^.Real := C1^.Real + C2^.Real;
 C^.Imag := C1^.Imag + C2^.Imag;
END Add;

PROCEDURE Sub(VAR C : COMPLEX; C1, C2 : COMPLEX);
 BEGIN
 C^.Real := C1^.Real - C2^.Real;
 C^.Imag := C1^.Imag - C2^.Imag;
END Sub;

PROCEDURE Mult(VAR C : COMPLEX; C1, C2 : COMPLEX);
 BEGIN
 C^.Real := (C1^.Real * C2^.Real) - (C1^.Imag * C2^.Imag);
 C^.Imag := (C1^.Real * C2^.Imag) - (C1^.Imag * C2^.Real);
END Mult;


PROCEDURE Div(VAR C : COMPLEX; C1, C2 : COMPLEX);
 VAR
 r1, r2 : LONGREAL;
 BEGIN
 IF ABS(C2^.Real) >= ABS(C2^.Imag) THEN
 r1 := C2^.Imag / C2^.Real;
 r2 := C2^.Real + r1 * C2^.Imag;
 C^.Real := (C1^.Real + r1 * C1^.Imag) / r2;
 C^.Imag := (C1^.Imag - r1 * C1^.Real) / r2;
 ELSE
 r1 := C2^.Real / C2^.Imag;
 r2 := C2^.Imag + r1 * C2^.Real;
 C^.Real := (C1^.Real * r1 + C1^.Imag) / r2;
 C^.Imag := (C1^.Imag * r1 - C1^.Real) / r2;
 END; (* IF *)
END Div;

PROCEDURE WriteComplex(C : COMPLEX);
 VAR
 S : ARRAY [0..10] OF CHAR;
 OK : BOOLEAN;
 BEGIN
 FixRealToStr(C^.Real, 3, S, OK);
 WrStr(S);
 FixRealToStr(C^.Imag, 3, S, OK);
 IF C^.Imag >= 0.0 THEN
 WrChar('+');
 END; (* IF *)
 WrStr(S);
 WrChar('i');
END WriteComplex;

END Complex.






[LISTING SIX]

=======================
[LISTING 6: COMPTEST.MOD]
=======================

MODULE CompTest;
(*
 Version: 1.00
 Date: 11-Sep-1988
 Language: Modula-2
 Purpose: Tests the Complex module
 Copyright 1988 by Scott Robert Ladd. All Rights Reserved.
*)

FROM Complex IMPORT COMPLEX,
 Create, Destroy, Assign, Equate,
 Add, Sub, Mult, Div, WriteComplex;


FROM IO IMPORT WrStr, WrLn;

VAR
 a, b, c : COMPLEX;
 r1, i1 : LONGREAL;
 r2, i2 : LONGREAL;

BEGIN
 r1 := 1.0;
 i1 := 2.0;
 r2 := -1.5;
 i2 := -5.5;

 Create(a);
 Create(b);
 Create(c);

 Assign(a,r1,i1);
 Assign(b,r2,i2);
 Assign(c,r1,i2);

 WrStr("a = ");
 WriteComplex(a);
 WrStr(" b = ");
 WriteComplex(b);
 WrStr(" c = ");
 WriteComplex(c);
 WrLn;

 Add(c,a,b);
 WriteComplex(c);
 WrLn;

 Sub(c,c,b);
 WriteComplex(c);
 WrLn;

 Mult(c,b,a);
 WriteComplex(c);
 WrLn;

 Div(c,c,a);
 WriteComplex(c);
 WrLn;

 Div(c,b,b);
 WriteComplex(c);
 WrLn;

 Destroy(a);
 Destroy(b);
 Destroy(c);

END CompTest.




































































January, 1989
EXAMINING ROOM


MS-DOS Assemblers Compared




Michael Schmit


Mike Schmit is the president of Quantum Software. He can be reached at 19855
Stevens Creek Blvd., Ste. 154, Cupertino, CA 95014.


I'll admit it. I am biased about languages; I write almost all of my code in
assembly language. I do this for several reasons, but mostly I do it because
the resultant code is both compact and fast. In this article, however, I avoid
a debate over the merits of assembly language versus those of the various
high-level languages. Rather, I explore the choices you have after you've
already made the decision to program in assembly.
Programmers who are more familiar with high-level languages may be surprised
to find out that modern assembly languages include an equally rich set of
features. These often include powerful macros, data structures, defined
procedures, dynamic stack variables, several memory models, floating point
support, and a host of conditional assembly directives.
But to most programmers it isn't a matter of whether it can be done, but how
easy it is to accomplish the end result. For years I have contended that
assembly language need not be as tedious and difficult to learn as a typical
bit-banger might lead you to believe. Today's crop of assemblers have lent
much support to this contention. New products from Borland International and
SLR Systems, and new versions from Microsoft, have gone far toward making
assembly language programming more approachable, and the time spent doing it
more productive.
This article compares three MS-DOS assemblers: Microsoft's Macro Assembler 5.1
(MASM), Borland's Turbo Assembler 1.0 (TASM), and SLR System's OPTASM 1.5.
MASM is a well-establish standard that others must be measured against, the
baseline against which others should be compared. In recognition of this fact,
it's important to note that both OPTASM and TASM claim almost 100 percent
compatibility with Microsoft's MASM.
I will assume some familiarity with MASM, and will not dwell too long on
assembly language itself. Rather I'll concentrate on the differences between
the assemblers, the performance of the assembler, and the enhancements offered
in OPTASM and TASM.
Even though all three assemblers can translate the same source files, their
internal workings are radically different. MASM is a conventional two-pass
assembler, and TASM performs only one pass and then fixes forward references.
OPTASM, on the other hand, is an n-pass assembler, performing as many passes
as required to eliminate phase errors and extraneous NOPs.
Let's look at each separately.


MASM


Because of what might be best described as the least-common-denominator
effect, most magazine articles have standardized on MASM 4.0 for example code.
With no serious competition, Microsoft has, until recently, been slow to add
features and increased power to MASM. But now that's changed. With the 5.x
series Microsoft has added a number of nice features. For example, it added a
simplified system for declaring segments that works for standalone assembly
language programs and for interfacing to high-level languages. Other
enhancements include better performance, automatic allocation for stack
variables, local and global labels, continuation lines, OS/2 support and an
HLL-like interface for Basic, C, Fortran, and Pascal.
A number of problems arise naturally because of the way that different
languages pass arguments on the stack. MASM handles all of this automatically,
but there is no easy way to allow for an assembly language routine to do this,
unless the HLL model is duplicated exactly. It should be noted, however, that
argument passing convention problems are not unique to MASM, because the
calling conventions are defined by the HLL.
The reason for this is that some instructions have a variable number of bytes,
such as the JMP instructions. A JMP can have a displacement of 1, 2, or 4
bytes corresponding to destination labels that are SHORT, NEAR, or FAR. It's
even more complicated for a large segment on the 80386, because a NEAR label
can have a 4-byte offset and a FAR label can have a 6-byte offset.
This design was apparently modeled after the original 8086 assembler (Intel's
ASM-86), which assumes the most likely size for the operand (2 bytes). If only
1 byte is needed, then a NOP is filled into the other byte. If more than 2
bytes are needed, then an error is generated. ASM-86 keeps track of every
assumption, however, and generates an error message corresponding to the exact
instruction that is in error. The MASM designers took a shortcut and the
assumptions are checked only indirectly by ensuring that labels have the same
offset on each pass. If they don't, MASM generates the message "Phase error
between passes."
This message has ended the assembly language programming careers of many
programmers. The instruction with the bad assumption can be anywhere between
the instruction where the message was generated and the previous label. This
could be one instruction or a thousand. (You can examine a pass-one listing to
help find the problem, but if you're experienced enough to know about pass-one
listings, then you probably don't need help finding the problem.)
The manuals that come with MASM include, among others, a programmer's guide, a
guide to Codeview, and a guide to utilities. But the one that I use most is
the 150-page spiral-bound reference guide. This guide includes a list of every
directive, instruction, and math coprocessor instruction; it also includes
syntax, timings, brief explanations, flags affected, and more. The
programmer's guide is quite an improvement over the version 4.0 manual in that
there are examples on most every directive and instruction. There are also a
numerical error message listing and an excellent index.
MASM 5.1 comes with a number of utilities, including the Microsoft Linker,
Librarian, Cross-Reference utility, and Make utility. New with 5.1 is the
Microsoft Editor, a programmer's editor that allows compiling (or assembling)
from the editor. One major feature of MASM is a BIND utility, which allows the
creation of a program that runs under DOS or OS/2.


TASM


TASM was originally designed for internal use only, providing Borland with a
competitive advantage. Many of the familiar Borland products made heavy use of
TASM, including Turbo Pascal and Quattro. Perhaps because of its roots, the
Turbo Assembler is a departure from Borland's standard language product
strategy; it is not an integrated environment like the familiar Turbo Pascal
or Turbo C. Rather it is a standalone, command line program like MASM or
OPTASM.
TASM is a single-pass assembler with forward reference resolution (which
accounts for much of its speed advantage over MASM) and a number of
enhancements that exceed the capabilities of MASM. In fact, TASM (and OPTASM)
both claim to be more compatible with MASM than MASM is, based on the fact
that they support previous versions of MASM. This is important if you are
supporting older code or frequently use code from magazines, bulletin boards,
or the like.
TASM handles forward references in roughly the same way as MASM, but with
better error reporting for references that cannot be resolved. The manual,
however, admits that "The truth of the matter is that all sorts of forward
references can cause problems for Turbo Assembler, so you should avoid forward
references--that is, references to labels farther on in the code--whenever
possible." (According to Borland, however, forward references in TASM only
cause probems when the programmer makes explicit use of the two-pass nature of
MASM.ed.)
One nuisance that a programmer must deal with when writing code for the Intel
processors is that conditional jumps have only a 1-byte displacement (+127 or
-128 bytes). (Note: the 80386 has a 2-byte displacement, but most code is
still written for DOS [8086] or OS/2 [80286].)
This means that MASM programmers get a lot of "relative jump out of range"
error messages. If you can't adjust your design, then you must opt for the
inelegant 5-byte sequence (see Figure 1, page 71), which consists of two
jumps. Every veteran MASM programmer has done this hundreds of times. TASM has
a new JUMPS directive that, in most cases, automatically handles this
situation.
Figure 1: Expanded conditional jump example

 problem:
 ...
 cmp ax, 1
 je near_label
 ... > 128 bytes of code
 near_label:
 ...

 correction:

 ...
 cmp ax, 1

 jne $ + 3
 jmp near_label
 ... > 128 bytes of code
 near_label:
 ...

One of the most notable features of TASM is that it has two modes: MASM and
Ideal. The MASM mode can also be MASM51, which then allows some of the new
features in MASM 5.1. Based on the names you can guess which mode the Borland
programmers consider to be the better one.
Ideal mode, as defined by Borland, makes the expression parser accept only a
more rigid, type-checked syntax. Advantages and disadvantages of this mode are
listed in Table 1, page 78. One key feature is the fact that the same source
file may switch back and forth between MASM and Ideal modes. Borland claims a
30 percent speedup when using the Ideal mode. I believe that this is due to
the fact that the parser can isolate assembly directives and addressing modes
more quickly in the Ideal mode. Also I think that the assembler was written
for Ideal mode and MASM compatibility was added later; thus MASM compatibility
was not part of the primary design goal. All tests were run using the default
MASM mode.
Table 1: Highlights of Turbo Assembler's MASM and Ideal Modes

 Description Examples Comments

---------------------------------------------------------------------------------------------

 Directives that begin .model small ;MASM
 with a period have .code
 been renamed
 model small ;TASM Ideal
 codeseg


 Directives such as PROC, func1 PROC near ;MASM - Causes all directives to
 ENDP, SEGMENT, and be the first token except
 ENDS are reversed PROC func1 near ;TASM Ideal data declarations, EQUs,
 and =s


 Square brackets [] mov ax, var1 ;MASM - In someinstances this
 required for memory mov cl, array1[bx] removes ambiguity, but
 references prevents writing code
 mov ax, [var1] ;TASM Ideal that looks like an HLL
 mov cl, [bx+array1] array index



 Structure fields are mov ax, [bx].field_1 ;MASM - Allows the re-use of
 not global structure field names

 ;TASM Ideal - Allows UNIONs (two
 mov cx, [(struc_a PTR BX)].field_1] STRUC's for the same
 data items)
 - Messy notation when
 you want to overlay a
 structure onto data not
 explicitly declared, (i.e.
 data allocated from DOS)

 EQUs are always text A = 1 - Fixes an inconsistency
 based (MASM does not B = 2 in MASM
 handle EQUs and =s C EQU A + B
 in a consistent B = 3
 manner) var DW C ;MASM = 3
 ;TASM Ideal = 4


 SIZE returns actual msg1 DB 'message 1', 0 - Makes the SIZE operator
 size of first item len DB SIZE msg1 ;MASM = 1 much more useful
 in a list ;TASM Ideal = 9


 Floating point num1 DT 1E7 ;in MASM could - Prevents confusion if
 constants must ;be a radix 16 value you use radix 16 in
 include a decimal programs using floating
 point to prevent num2 DT ;required for point
 ambiguous values num2 DT 1.0e7 ;TASM Ideal



One of the key features of Ideal mode is a consistent syntax for the use of
directives, such as the declaration of a procedure. The Ideal syntax is:
 PROC Func1 near
 ...
 ENDP Func1 ; optional
 repeating of Func1
This allows a faster assembler, and the syntax makes sense because the first
token on the line always defines directive type, unlike the MASM syntax. But
you could make an equally sensible argument for the MASM syntax. The PROC
directive defines a program label, just as a label followed by a colon does,
or a label followed by a DW defines a data word. It would make no sense to
reverse the order of these. Then again, the MASM syntax for the ENDP doesn't
fit into any of these rules.
Perhaps the biggest difference between MASM and Ideal is that Ideal uses
square brackets ([]) in expressions as shown in Figure 2, page 76. In Ideal
mode you use square brackets to reference memory. This is easy to remember and
makes a lot of sense. (Ideal mode doesn't require square brackets, but it will
warn you if you don't use them. However, this warning can be disabled with the
NOWARN directive.ed.) For example, in MASM you would use:
 item_1 DW 10
 item_2 equ 20
 array1 DB 256 dup(0)

 mov ax, item_1
 mov dx, item_2
 mov cx, 3[bx]
 mov al, array1[bx]
Without the data declaration and equate in clear view, it is difficult to tell
the difference between the first two MOVs. There are times when you may not
care, but precise control is what assembly language is all about.
Mathematically the third MOV looks like three times something, when in fact it
is not. The fourth MOV looks just like what it is, using BX to index into an
array.
Figure 2: Turbo Assembler's Ideal mode example

 title MASM mode example program ; comment in title
 .model small

 stdin = 0
 stdout = 1
 buf_len = 128

 dosint MACRO function ; macro to invoke a DOS interrupt 21h function
 mov ah, function
 int 21h
 ENDM

 .code

 main PROC

 mov ax, @data ; load data segment
 mov ds, ax
 mov es, ax
 mov dx, OFFSET inbuf ; dest for input
 mov bx, stdin ; source of input
 mov cx, buf_len ; len for input
 dosint 3fh ; read file
 cmp ax, 2
 jle fin
 mov bx, ax
 mov inbuf[bx-2], 0 ; null at end (remove CR LF)
 mov cx, ax ; len for example
 sub cx, 2 ; removes CR LF
 call lower_line
 mov dx, OFFSET outbuf ; source of output
 mov bx, stdout ; dest for output
 dosint 40h ; write file

 fin:
 dosint 4ch ; exit to DOS

 main ENDP

 lower_line PROC near

 push cx
 mov si, OFFSET inbuf
 xor di, di ; for example no STOS for DI
 cld
 loop1:
 lodsb ; read a char
 or al, 20h ; convert to lower case
 mov outbuf[di], al ; store in outbuf
 inc di
 loop loop1 ; loop til end of string
 pop cx
 ret

 lower_line ENDP

 .data

 inbuf DB buf_len DUP (?)
 outbuf DB buf_len DUP (?)

 stk SEGMENT STACK ; reserve space for stack
 db 100 dup (0) ; using old segment method
 stk ENDS

 END main ; specify starting address

 ---------------------------------------------

 IDEAL
 %title "TASM IDEAL mode example program" ; comment not in title
 ; all directives affecting listing
 ; file begin with a percent sign (%)

 model small ; no periods in Ideal directives

 stdin = 0
 stdout = 1
 buf_len = 128

 MACRO dosint function ; label and MACRO reversed
 mov ah, function
 int 21h
 ENDM

 codeseg ; segmentation directive renamed

 PROC main ; label and PROC reversed

 mov ax, @data
 mov ds, ax
 mov es, ax
 mov dx, OFFSET inbuf

 mov bx, stdin
 mov cx, buf_len
 dosint 3fh
 cmp ax, 2
 jle fin
 mov bx, ax
 mov [bx+inbuf-2], 0 ; memory reference must be in []
 mov cx, ax
 sub cx, 2
 call lower_line
 mov dx, OFFSET outbuf
 mov bx, stdout
 dosint 40h
 fin:
 dosint 4ch

 ENDP main ; label and ENDP reversed

 PROC lower_line near

 push cx
 mov si, OFFSET inbuf
 xor di, di
 cld
 @@loop1:
 lodsb
 or al, 20h
 mov [di+outbuf], al ; memory reference in []
 inc di
 loop @@loop1 ; local labels available
 pop cx
 ret

 ENDP ; matching PROC name optional

 dataseg ; segmentation directive renamed

 inbuf DB buf_len DUP (?)
 outbuf DB buf_len DUP (?)

 SEGMENT stk STACK ; label and SEGMENT reversed
 db 100 dup (0)
 ENDS ; matching SEG name optional

 END main

By contrast, in Ideal mode you would use:
 item_1 DW 10 item_2 equ 20 array1 DB 256 dup(0)

 mov ax, [item_1] mov dx, item_2 mov cx, [bx+3] mov al, [bx+array1]
Keep in mind that MASM allows many of the Ideal mode expressions, but TASM (in
Ideal mode) requires them, possibly preventing an accidental construct that is
legal but does not get the intended insults. MASM veterans may have a hard
time accepting the Ideal mode. Beginners will probably learn assembly language
quicker.
There are about 25 differences between the Ideal and MASM modes (see Table 2,
page 80). The manual gives a description and an example of each difference
covering about 30 pages in all.
Table 2: TASM operating modes and quirks

 Features comparison

 MASM 5.1 OPTASM TASM
-------------------------------------------------------------------------


 MASM 3.x compatible x
 MASM 4.x compatible x x
 MASM 5.x compatible x 1 x

 80286 code x x x
 80287 code x x x
 80386 code x x
 80387 code x x
 8087 emulation x x

 Expand conditional jumps > 128 bytes x x
 away by adding an extra jump
 Expand LOOP's > 128 bytes away by x
 adding two additional jumps
 Remove extra NOPs x
 Eliminates phase errors x

 Externals can include size info x
 Global symbols (combines Public/Extrn) 2 x
 UNION directive (nested STRUC's) x
 Additional modes 3 4
 Length of data item 5 6
 Generation of Group offsets{7} x x
 Local labels 8 9 10
 Multiple PUSH and POP pseudo-op x
 Short Extrn directives (i.e. EXTB) x

 Wildcard filenames on cmd line x
 Built-in make x

 predefined symbols x x
 time
 date x x
 filename x
 version x

Notes:
1) OPTASM supports up to MASM 5.0 and some of MASM 5.1 features.
2) OPTASM has a Soft Extrn directive that allows an internal definition to
override the Extrn.
3) OPTASM has about 40 features that can be individually enabled or disabled
to override the normal MASM features.
4) TASM defaults to MASM 4.0 and has directives for MASM 5.0, MASM 5.1,
Quirks, and an Ideal mode (see text).
5) OPTASM has an option to allow the LENGTH operator to return the total
length of all items defined for a given label.
6) In Ideal mode the SIZE operator returns the total size for the first data
item in a data list.
7) When using the SEG operator and the OFFSET operator, MASM can generate
incorrect addresses when using simplified segmentation.
8) MASM allows local labels to be defined as "@@:" which can then be
referenced as @F (forward to next @@:) or @B (backwards to previous @@:). In
additional, when using an HLL model all labels are local, unless defined with
two colons.
9) OPTASM allows local labels by prefixing any label with a # sign or
suffixing it with a dollar ($) sign.
10) TASM allows local labels by prefixing with two @ signs (@@).

There is an additional submode called Quirks (see Table 3, page 83), which,
according to the manual, "allows you to assemble a source file that makes use
of one of the true MASM bugs." Although this statement is a shot at Microsoft,
more important is the fact that it refers to enabling several well-documented
features that Borland was apparently reluctant to include. The Manual lists
three main quirks:
1. Local labels are defined with @@ and referred to with @F and @B. 2. There
is a redefinition of variables inside PROCs. 3. C language PROCs are all
PUBLIC with leading underscores.
I agree that the first Quirk is a quirk. In my opinion, MASM's implementation
of local labels is brain damaged (both TASM and OPTASM have different
implementations). I believe that the last two quirks are actually a benefit
for anyone writing HLL utilities in assembly language. The second quirk allows
you to use the same names for arguments being passed from an HLL. The third
quirk is that all PROCs are declared public. When MASM51 is turned on and the
C language is specified as part of the .MODEL statement, leading underscores
are appended. This serves a quirk of the C language itself that requires that
functions have a hidden leading underscore in the public name passed to the
linker. (See Figure 3, page 84.).
The Turbo Assembler documentation includes a 580-page user's guide and a
300-page reference guide. The user's guide includes a 200-page tutorial that
is one of the best beginner's guides to the Intel architecture that I have
read. There are also chapters on the specifics of interfacing with the other
Borland languages: Turbo C, Turbo Basic, Turbo Pascal, and Turbo Prolog. Much
of this information is common with other languages.
The reference guide is primarily a description of all the directives and
operators. Each directive and operator is marked as to which modes (MASM or
Ideal) it is available in and any differences in syntax. But it is not
possible to tell what MASM mode syntax is a super set of Microsoft's MASM. In
other words, it is possible to write code in Turbo Assembler's MASM mode that
will not assemble with MASM, and the manual does not readily note the
differences. This is not a major concern if you are planning on using TASM
exclusively, but would be a problem if you are sharing code with other
programmers not using TASM.
Turbo Assembler comes with a number of additional programs. One program is the
Debugger. Turbo Debugger is a source-level debugger that supports debugging
from a remote system and has multiple windows, menus, on-line help, and
virtual 8086 debugging on an 80386 system. It is a significant product in its
own right; any further discussion is beyond the scope of this article.
Turbo Assembler also provides a utility that allows source debugging of files
linked for Codeview debugging. Additional utilities include Turbo Linker,
Turbo Librarian, TCREF (a cross-reference utility), and MAKE. The Turbo Linker
is not a complete replacement for the Microsoft Linker. The manual states,
"TLINK is lean and mean; ... it lacks some of the bells and whistles of other
linkers."


OPTASM



OPTASM, by SLR Systems, is a high-performance optimizing assembler that is
nearly 100 percent compatible with MASM. OPTASM supports incompatibilities
between MASM, Versions 3, 4, and 5. OPTASM is the clear winner in terms of
performance and code size, but, as is always the case, has some limitations.
These limitations are either minimal or severe, depending upon your needs. For
example, OPTASM does not support 80386 instructions, nor does it support some
of the MASM features gained in the jump from version 5.0 to 5.1. Another
downfall is that no linker is provided. Although MASM doesn't provide a linker
either, the high performance of OPTASM leads you to expect that it will.
(Note: OPTASM, Version 1.6, which will be released this month includes a
linker and debugger. No changes have been made to the assembler.) But then
after assembling at lightning speed, you must wait for the Microsoft Linker.
This is like using a 20-MHz 80386 and then shifting to a 4.77-MHz PC.
Table 3: Features conparison

 TASM Operating Modes

a) Normal MASM Emulates MASM 4.0, 5.0 without minor quirks.
b) QUIRKS Emulates MASM 4.0, 5.0 with minor quirks in those
 versions.
c) MASM51 Emulates those features of MASM 5.1 that conflict
 withMASM 4.0 and 5.0 operation but do not conflict
 with the operation of Borland's extensions that
 perform the similar functions.
d) MASM51 and Quirks Emulates MASM 5.1 fully.

Quirks as explained in the TASM HELPME! .DOC disk file

Mode Operations
Quirks Allows FAR jumps to be generated as NEAR or SHORT if
 CS assumes agree.
 Allows all instruction sizes to be determined in a
 binary operation solely by a register, if present.
 Destroys OFFSET, segment override, etc., information
 on '=' or numeric 'EQU' assignments.
 Forces EQU assignments to expressions with "PTR"
 or ":" in them to be text.
MASM51 Instr, Catstr, Substr, Sizestr, and "\" line
 continuation are all enabled.
 EQU's to keywords are made TEXT instead of ALIASes.
 Leading whitespace is not discarded on %textmacro in
 macro arguments.
MASM51 and Quirks Everything listed under Quirks above.
 Everything listed under MASM51 above.
 @@, @F, and @B local labels are enabled.
 Procedure names are PUBLICed automatically in
 extended MODELs.
 Near labels in PROCs are redefinable in other PROCs.
 "::" operator is enabled to define symbols that can
 be reached outside of current proc.
MASM51 and Ideal Ideal mode syntax and the MASM51 text macro
 directives are supported, i.e., Instr, Catstr,
 Substr, and Sizestr.

Figure 3: New MASM to C calling interface

 Old method (MASM 5.0 and before):

 Public _sample_func
 _sample_func proc near
 push bp
 mov bp, sp
 push bx
 push cx
 push dx

 mov bx, [bp+41] ; get
 ptr to var1
 ...


 pop dx
 pop cx
 pop bx
 pop bp
 ret
 _sample_func endp

 New method:

 .model small, C
 .code

 sample_func proc near USES BX
 CX DX, var1:PTR

 mov bx, var1 ; get
 ptr to var1
 ...

 ret
 sample_func endp

SLR Systems was the only company willing to discuss future products with me.
It is working on 80386 support, a linker, and full compatible MASM 5.1.
As mentioned before, OPTASM is an n-pass assembler. This unique design allows
OPTASM to perform as many passes as required to prevent all phase errors, and
it never inserts extra NOPs into your code. Although SLR calls OPTASM an
optimizing assembler, it really does not do optimizations in the sense, for
example, that a C compiler does optimizations. The truth is that MASM and TASM
perform unoptimizations by inserting NOPs where the programmer did not ask for
them; all OPTASM does in the way of optimizations is not to make this mistake.
OPTASM also contains a number of extensions to the basic MASM language. But
it's important to note that every feature can be either enabled or disabled.
The Make capability in OPTASM is not really a Make utility in the traditional
sense, because the MAKE routines are built into the assembler. There are both
advantages and disadvantages to this approach. You can still use your old Make
in the same way, or you can use your old Make along with the OPTASM Make.
In its simplest form the OPTASM Make will just process a file that has a list
of filenames. Actually you are required to enter the command tail, just as it
would appear in a DOS batch file, minus the OPTASM program name. The advantage
here is that OPTASM does not need to be reloaded by DOS between each file.
OPTASM is also smart enough to keep include files in memory, if possible. Just
as with a regular Make, you can add dependencies, so that files are assembled
only if they are out of date.
Version 1.0 of OPTASM came with no other utilities, but version 1.5 is now
shipped with a unique utility: OPTHELP. This is a memory-resident utility that
provides help on the instruction set and OPTASM. The help is fairly detailed,
including bit encodings for the various instructions. For example, the MOV
instruction has 15 pages of help.
OPTASM does not come with a librarian, but OPTLIB is available separately.
OPTLIB is ten times faster than the Microsoft Librarian.
OPTASM comes with one 320-page spiral-bound manual. The manual does not
include a tutorial, but is a complete description of the language, with all
features unique to OPTASM appearing in boxes marked as OPTASM features. When
reading about any feature, there is no doubt as to whether the exact syntax is
also available in MASM.


Local Labels


Each of the three assemblers implements local labels in a different manner.
(See Figure 4, below.) They are all incompatible with each other. In other
words, if you write a program that uses any of the methods for local labels,
then the other assemblers will not allow it. The only exception is that TASM
will accept the MASM methods if you use both the MASM and Quirks directives.
Figure 4: Local labels examples

 ; MASM example

 @@:
 inc ax
 cmp ax, bx
 je @f
 loop @b

 @@:
 ret

 ; TASM example

 @@loop:
 inc ax
 cmp ax, bx
 je @@exit
 loop @@loop
 @@exit:
 ret


 ; OPTASM example

 #1:
 inc ax
 cmp ax, bx
 je #2
 loop #1

 #2:
 ret

Local labels are a new feature in MASM 5.1. The idea is simple: you define a
local label with two at signs (@@) followed by a colon. To reference the
nearest preceding local label, use @B (back); to reference the next local
label, use @F (forward). When writing assembly language routines for use in
HLL programs, a much more structured feature is enabled. The manual describes
it as a local variable scope when you use the extended form of the .MODEL
directive. Labels ending with a single colon (and procedure arguments passed
on the stack and local stack variables) are considered local to the procedure
where they are defined. To make a label available from another procedure, it
must be defined with two colons.
This is an excellent feature of MASM, it promotes well-structured code and
highlights any labels that are used external to a procedure. The bad news is
that you must be using the extended form of the .MODEL directive, which is not
always appropriate. With TASM, on the other hand, local symbols are available
whether you are using the extended form of the .MODEL statement or not.
As stated earlier, TASM can emulate the MASM local labels. But in addition you
can define a local label by preceding a label with two at (@@) signs. The
scope of these local labels is in between any two regular labels. This feature
is automatically available in the Ideal mode and can be enabled in the MASM
mode with the Locals directive. The two at signs can be replaced with any
other characters that can start a label by using the Locals directive.
OPTASM 1.5 does not support the MASM local labels. However, OPTASM has a
unique feature called Procedure Local Labels. There are two formats:
 # n label
 n label $
The first format begins with a # sign and is followed by a series of digits
(n) or a standard label (label). The second format begins with a series of
digits (n), is optionally followed by a standard label (label), and is
terminated by a dollar sign ($). The assembler ignores the # and $ characters
when evaluating a label, and thus #10 and 10$ are considered the same. All
these labels are automatically local only to the procedure in which they are
defined. Therefore the same labels can be reused in other procedures.


The Results


I conducted a number of tests on all three assemblers on several machines,
including a 4.77-MHz XT, a 10-MHz AT, and a 16-MHz 80386. The results were
proportional for each class of machine, and so were based only on the 10-MHz
AT clone with a Seagate ST-251 drive (40 Mbyte, 38-ms access time).
The first test was based on a program that consisted of 15 source files,
ranging in size from 2K to 100K, totaling 535K. There were also four include
files totaling 16K. Based on how many times the include files were read, the
assembler had to read more than 600K of source to build the object files. All
times were in seconds and did not include linking or generating listing files.
The second set of tests consisted of assembling six 100K source files. The
third and fourth tests used the unique features of TASM and OPTASM to obtain
more speed. These tests were the same as the first two, except that I used
TASM's wildcard feature and OPTASM's internal make. The effect was that both
TASM and OPTASM obtained a speed advantage, because reloading the assembler
between files was not required. See Figure 3 for the test results.
Figure 5
Both Borland's Turbo Assembler and SLR System's OPTASM provided clear
advantages over Microsoft's Macro Assembler. The TASM/Debugger package seems
to be the best value, but both were much faster, with OPTASM clearly the
fastest. Plug and play, OPTASM also produced the most compact code, maybe 1
percent smaller than the other. While this is significant, view these results
in context. For example, with a bit of additional work, compaction could have
been improved in TASM through the use of overrides. Likewise, using a
different linker, like Phoenix's PLINK, is likely to shrink code size.
Figure 5: Test data

 Test description MASM TASM OPTASM

 test 1 (15 files, 600K total) 85 50 35
 test 2 (6 files, 100K each) 80 40 26
 test 3 (test 1 with wildcards/make) 85 34 24
 test 4 (test 2 with wildcards/make) 80 32 22

 Notes:
 1) Times are in seconds
 2) All tests on 10 MHz AT clone,
 zero wait states, Seagate
 ST-251 hard disk, no cache

There were four test cases run, as described in the article. Each test
consisted of just three data points, one for each of the assemblers.

Both TASM and OPTASM also provided a number of nice extras, such as fixing
conditional jumps that were out of range by putting in two jumps. OPTASM
clearly outperformed TASM based on the fact that it never inserted extra NOPs
and it handled all combinations of conditional jumps and loops. TASM, on the
other hand (if automatic jump sizing is used with forward references),
automatically assumed that the jump is going to be far and reserved 5 bytes
for the jump, padding the code with extra NOPs.
It is also worth nothing that both MASM and TASM support 80386 code, as will
OPTASM, sometime in the future, and for the present, at least, MASM is the
only assembler that runs in OS/2 protected mode.
Now that there is finally some competition in the macro assembler market, it's
about time users began sending their wish lists to the manufacturers. Keep in
mind that a job is only as easy as the tools make it.













January, 1989
USING EXTENDED MEMORY ON THE PC AT


Here's a program that lets you copy memory from real mode to extended memory -
and back again




Paul Thomson


Paul Thomson is a software engineer at Black Dot. He can be reached at 439 N
Lake Shore Dr., Palatine, IL 60067.


The PC AT BIOS contains a service -- service 87h of interrupt 15h --
forcopying memory in protected mode, thereby allowing you to copy memory
anywhere in the 16- Mbyte address range of the 80286. This means that you can
copy memory from real-mode memory to extended memory and back again. Before
calling this service routine, however, you must set up a data structure for
protected-mode operation. This structure is called a global descriptor table.
It contains six descriptors, each with 8 bytes. Two of these are segment
descriptors that describe the source and target buffers. The rest of the
descriptors are simply dummy descriptors that will be initialized by the BIOS
routine. The 8-byte segment descriptors are shown in Table 1, below.
Table 1: Segment descriptor formats

 Descriptor Format
 ------------------------

 16 bits Null: Reserved for future use
 8 bits Access rights byte: Contains attribute flags for the
 segment (writable?, data segment?, etc.)
 24 bits Physical address: Any real mode address must be
 converted to a 24 bit physical address by the
 following formula: (segreg * 16) + offset.
 16 bits Segment length in bytes: Words to transfer * 2.

Before calling the service routine, you must set up the registers as shown in
Table 2.
Table 2: Setup for registers

 AH = 87h
 ES:SI = Real mode address of
 global descriptor table

 CX = Count of words to copy

There is also another interrupt 15h service -- service 88h -- which is useful
for extended memory. Service 88h returns the number of Kbytes of extended
memory (which starts at 0x100000) in AX. Since service 88h will not return an
error when accessing non-existent memory, service 88h can be used to determine
if the move will involve valid memory locations.
Listing One, page 108, shows a Microsoft C (compact or large model) version of
the move routine (movphy), the top of memory routine (getext), and a test
routine (main) while Listing Two, page 119, is an assembly language version.
movphy will copy memory in 64K-byte chunks given the source and destination
addresses in 24-bit format. getext, which must be called once (before movphy
is called for the first time) sets up a static variable with the maximum
physical address of extended memory. The test copies a string between extended
and real-mode memory to show that the routine is working correctly. An error
can be discovered by checking the routine return code (ax in C, ah in the
assembler routine). Before you use the program, check that your extended
memory is not occupied by vdisk; the program can wipe out your virtual disk
files and/or render vdisk unusable.

_USING EXTENDED MEMORY ON THE PC AT_

by Paul Thomson


[LISTING ONE]

/***
 *** ROUTINE FOR COPYING MEMORY USING PHYSICAL ADDRESSES ON THE PCAT.
 *** WRITTEN BY PAUL THOMSON.
 *** COMPILE USING MICROSOFT C V4.0, LARGE OR COMPACT MODEL.
 *** DO NOT USE THIS ROUTINE WITH A VIRTUAL DISK IN EXTENDED MEMORY.
 ***/

#include <stdio.h>
#include <dos.h>


#define PHYS(s) (((long)FP_SEG(s) << 4) + FP_OFF(s)) /* CALC PHYS ADDR */
#define BUFSZ 16 /* BUF SIZE IN BYTES */
#define PHY_ADDR 0x100000L /* BEG OF EXT MEMORY */

char *buf="ORIGINAL MESSAGE";

/* TEST MOVPHY BY COPYING A MESSAGE TO EXTENDED MEMORY AND BACK */
main()
{
 extsize();
 puts(buf);
 movphy(PHY_ADDR, PHYS(buf), BUFSZ/2); /* MOVE TO EXTENDED MEMORY */
 sprintf(buf, "XXXXXXXXXXXXXXXX"); /* OVERWRITE BUFFER */
 puts(buf);
 movphy(PHYS(buf), PHY_ADDR, BUFSZ/2); /* MOVE BACK FROM EXTENDED MEMORY */
 puts(buf);
}

static long maxext; /* HOLDS MAX ADDRESS OF EXTENDED MEMORY */
extsize()
{
 union REGS r;

 r.h.ah = 0x88;
 int86(0x15, &r, &r); /* RETURNS SIZE OF EXTENDED MEM IN KBYTES */
 maxext = (r.x.ax + 1024L)*1024; /* FIND TOP OF EXTENDED MEMORY */
}

/* MOVE MEMORY USING PHYSICAL ADDRESSES */
movphy(target, source, wcount)
unsigned long target; /* PHYSICAL 24 BIT TARGET ADDRESS */
unsigned long source; /* PHYSICAL 24 BIT SOURCE ADDRESS */
int wcount; /* 16 BIT COUNT OF WORDS TO MOVE 0 - 32767 */
{
 int bcount;
 char gdt[48]; /* GLOBAL DESCRIPTOR TABLE (6 DESCRIPTORS*8) */
 char *g = gdt; /* POINTER TO gdt FOR MACROS FP_SEG & FP_OFF */
 union REGS r; /* HOLDS REGISTER VALUES FOR int86x CALL */
 struct SREGS s; /* HOLDS SEG REGISTER VALUES FOR int86x CALL */

 if(wcount <= 0) /* CHECK FOR WORD COUNT TOO BIG OR 0 */
 return(wcount);
 bcount = wcount*2; /* SIZE IN BYTES TO MOVE */

 if(target+bcount >= maxext source+bcount >= maxext)
 return(4);

 /* DESCRIPTORS 0 AND 1 ARE DUMMIES (NULL) */
 gdt[0]=gdt[1]=gdt[2]=gdt[3]=gdt[4]=gdt[5]=gdt[6]=gdt[7]=0;
 gdt[8]=gdt[9]=gdt[10]=gdt[11]=gdt[12]=gdt[13]=gdt[14]=gdt[15]=0;

 /* DESCRIPTOR 2: SOURCE */
 gdt[16] = bcount; /* BYTE COUNT */
 gdt[17] = bcount>>8;
 gdt[18] = source; /* PHYSICAL ADDRESS TO COPY FROM */
 gdt[19] = source>>8;
 gdt[20] = source>>16;
 gdt[21] = 0x93; /* ACCESS RIGHTS BYTE */
 gdt[22] = gdt[23] = 0; /* UNUSED ENTRIES */


 /* DESCRIPTOR 3: TARGET */
 gdt[24] = bcount; /* BYTE COUNT */
 gdt[25] = bcount>>8;
 gdt[26] = target; /* PHYSICAL ADDRESS TO COPY TO */
 gdt[27] = target>>8;
 gdt[28] = target>>16;
 gdt[29] = 0x93; /* ACCESS RIGHTS BYTE */
 gdt[30] = gdt[31] = 0; /* UNUSED ENTRIES */

 /* DESCRIPTORS 4 AND 5 ARE DUMMIES (NULL) */
 gdt[32]=gdt[33]=gdt[34]=gdt[35]=gdt[36]=gdt[37]=gdt[38]=gdt[39]=0;
 gdt[40]=gdt[41]=gdt[42]=gdt[43]=gdt[44]=gdt[45]=gdt[46]=gdt[47]=0;

 r.h.ah = 0x87; /* AH = SERVICE 0x87 */
 r.x.cx = wcount; /* CX = COUNT OF WORDS TO TRANSFER */
 s.es = FP_SEG(g); /* ES:SI = SEGMENT:OFFSET OF GDT */
 r.x.si = FP_OFF(g);

 int86x(0x15, &r, &r, &s); /* PERFORM BIOS INTERRUPT 0x15 */

 return(r.h.ah);
 /* RETURN STATUS:
 0: SUCCESSFUL MOVE
 1: RAM PARITY ERROR
 2: EXCEPTION ERROR
 3: ADDRESS LINE 20 FAILED
 4: MEMORY OUT OF RANGE
 <0: WORD COUNT > 32767 */
}






[LISTING TWO]

; ROUTINE FOR COPYING MEMORY USING PHYSICAL ADDRESSES ON THE PCAT.
; WRITTEN BY PAUL THOMSON.
; DO NOT USE THIS ROUTINE WITH A VIRTUAL DISK IN EXTENDED MEMORY.


.286p ; ALLOW 286 INSTRUCTIONS
code segment
assume cs:code,ds:code,es:code,ss:code

; TEST MOVPHY BY COPYING A MESSAGE TO EXTENDED MEMORY AND BACK
test proc near
 mov ax,cs ; ALLOW ACCESS OF DATA IN CODE SEG
 mov ds,ax

 call extsize ; FIND TOP OF EXTENDED MEMORY

 mov dx,offset mess1 ; PRINT MESSAGE
 mov ah,9
 int 21h

 mov dx,cs ; CALCULATE PHYS ADDR FROM REAL ADDR OF MESSAGE BUF

 shr dx,12 ; SI = BITS 0-15, DL = BITS 16-23
 mov ax,cs
 shl ax,4
 mov si,offset mess1
 add si,ax
 adc dl,0
 push si ; SAVE PHYS ADDRESS FOR LATER
 push dx

 mov dh,10h ; TOP OF EXTENDED MEMORY (100000h)
 mov di,0 ; DI = BITS 0-15, DH = 16-23

 mov cx,8 ; SIZE OF MESSAGE BUF IN WORDS
 call movphy ; MOVE MESSAGE TO EXTENDED MEMORY

 sub bx,bx ; OVERWRITE MESSAGE BUFFER
top:
 mov al,mess2[bx]
 mov mess1[bx],al
 inc bx
 cmp bx,16
 jl top
 mov dx,offset mess1

 mov ah,9 ; PRINT OVERWRITTEN MESSAGE BUFFER
 int 21h

 pop dx ; GET PHYS BUFFER ADDRESS FROM BEFORE
 pop di ; DI = BITS 0-15, DH = BITS 16-23
 mov dh,dl

 mov dl,10h ; TOP OF EXTENDED MEMORY (100000h)
 mov si,0 ; DI = BITS 0-15, DH = 16-23

 mov cx,8 ; SIZE OF MESSAGE BUF IN WORDS
 call movphy ; MOVE MESSAGE BACK FROM EXTENDED MEMORY

 mov dx,offset mess1 ; PRINT RESTORED MESSAGE
 mov ah,9
 int 21h

 mov ah,4ch ; EXIT
 int 21h
test endp
mess1 db 'ORIGINAL MESSAGE',0dh,0ah,'$'
mess2 db 'XXXXXXXXXXXXXXXX'

; extsize - GET PHYSICAL ADDRESS OF TOP OF EXTENDED MEMORY
; ADDRESS RETURNED IN max_hi,max_lo
extsize proc near
 mov ah,88h
 int 15h
 mov cx,1024
 add ax,cx
 mul cx
 mov max_hi,dl
 mov max_lo,ax
 ret
extsize endp

max_hi db ?
max_lo dw ?

; movphy - MOVE MEMORY USING PHYSICAL ADDRESSES
; CALLED WITH:
; dh:di = physical 24 bit target address.
; dl:si = physical 24 bit source address.
; cx = word count
; STATUS RETURNED IN ah:
; 0: successful move
; 1: RAM parity error
; 2: exception error
; 3: address line 20 failed
; 4: memory out of range
; 255: word count > 32767

public movphy
movphy proc near
 push ds
 mov ax,cs ; ALLOW ACCESS OF gdt IN CODE SEGMENT
 mov ds,ax
 mov es,ax ; es = SEGMENT OF gdt FOR int 15h

 mov ax,cx ; CALCULATE MAXIMUM TARGET ADDRESS
 shl ax,1
 mov bl,dh
 add ax,di
 adc bl,0
 cmp bl,max_hi ; CHECK IF TARGET ADDRESS OUT OF RANGE
 jl $target_ok
 jg $bad_range
 cmp ax,max_lo
 jge $bad_range
$target_ok:
 mov ax,cx ; CALCULATE MAXIMUM SOURCE ADDRESS
 shl ax,1
 mov bl,dl
 add ax,si
 adc bl,0
 cmp bl,max_hi ; CHECK IF SOURCE ADDRESS OUT OF RANGE
 jl $source_ok
 jg $bad_range
 cmp ax,max_lo
 jl $source_ok
$bad_range:
 mov ah,4 ; IF ACCESSING NON-EXISTENT MEMORY, RETURN ERROR 4
 jmp $xend
$source_ok:

 cmp cx,0 ; CHECK FOR WORD COUNT TOO BIG OR 0
 jg $wcount_ok
 mov ax,cx ; RETURN 255 IF WORD COUNT > 32767
 jmp $xend
$wcount_ok:

; DESCRIPTORS 0,1,4,5 ARE DUMMIES (NULL)
 sub ax,ax

 mov gdt,ax ; DESCRIPTOR 0

 mov gdt+2,ax
 mov gdt+4,ax
 mov gdt+6,ax

 mov gdt+8,ax ; DESCRIPTOR 1
 mov gdt+10,ax
 mov gdt+12,ax
 mov gdt+14,ax

 mov gdt+32,ax ; DESCRIPTOR 4
 mov gdt+34,ax
 mov gdt+36,ax
 mov gdt+38,ax

 mov gdt+40,ax ; DESCRIPTOR 5
 mov gdt+42,ax
 mov gdt+44,ax
 mov gdt+46,ax

 mov gdt+22,ax ; UNUSED ENTRIES IN DESCRIPTOR 2
 mov gdt+30,ax ; UNUSED ENTRIES IN DESCRIPTOR 3

 mov ax,cx ; CHANGE WORD COUNT TO BYTE COUNT
 shl ax,1
 mov gdt+16,ax ; BYTE COUNT DESCRIPTOR 2
 mov gdt+24,ax ; BYTE COUNT DESCRIPTOR 3

 mov gdt+18,si ; PHYSICAL ADDRESS TO COPY FROM
 mov al,dl
 mov ah,93h ; ACCESS RIGHTS BYTE
 mov gdt+20,ax

 mov gdt+26,di ; PHYSICAL ADDRESS TO COPY TO
 mov al,dh
 mov ah,93h ; ACCESS RIGHTS BYTE
 mov gdt+28,ax

; MAKE DOS CALL
 mov ah,87h ; SELECT SERVICE 87h
 mov si,offset gdt ; ES:SI = SEGMENT:OFFSET OF GLOBAL DESCRIPTOR TABLE
 int 15h ; PERFORM MEMORY MOVE
$xend:
 pop ds
 ret

 gdt dw 24 dup(?) ; GLOBAL DESCRIPTOR TABLE

movphy endp
code ends
end












January, 1989
C PROGRAMMING


The Surrogate Library




Al Stevens


Several readers have asked why I selected Turbo C for the "C Programming"
column project. Many of you use other compilers, and some of you feel left
out. One such reader goes so far as to say that the next several issues of DDJ
will be useless to him, and he will likely allow his subscription to expire.
That got my attention.
All this distress is because the programs being discussed are written with the
specific screen management functions of Turbo C. This particular reader felt
that the column should restrict itself to generic ANSI C whenever possible to
reach the widest possible audience. That argument has merit, and I do not want
to lose readers, particularly those who care enough to tell me when things are
not right. I must, therefore, deal with this concern.
First, let's consider why generic C is not practical for this project. The
program uses our own home-grown video windows on the IBM PC and compatibles.
Such a program must, if performance is to be considered, directly address
video memory. The program will also use the PC's serial port. These
requirements bind the program to the PC hardware architecture. Of course, most
of the many C compilers for the PC could be used to develop such a program,
and each such program would be somewhat different from the others. The methods
that you use for low-level hardware access are different enough with each
compiler that different programs will result. Perhaps a need exists for a
standard library for MS-DOS PC programs, but no such standard is in
acceptance. Some compilers attempt to be compatible with others, but these
attempts fall apart when the compiler developers are working on similar
extensions at the same time. Witness the different approaches to graphics
libraries in Turbo C and Microsoft C.
The news, however, is not all bad. A relatively small amount of the code in
our project is specific to and dependent upon Turbo C. All such code is in the
area of screen management, and is represented by a set of Turbo C functions
and a few internal Turbo C constructs. Most of the rest of the code is generic
at least at the PC level. Microsoft C and Turbo C do a lot of things the same
way. Remember, however, that the underlying theory that applies to our video
windows is based on the PC's video architecture; the functions that we are
developing are for the PC with MS-DOS. They are not CP/M, Unix, or VAX/VMS
programs, for example. This fact could give rise to a new uproar, and I
reluctantly wait for that heat and will try bravely to bear up under it.
Why did I chose Turbo C? The answer is simple: Turbo C is the compiler I use
for nearly everything else in MS-DOS. That is not so much an endorsement as a
mere fact. I use Turbo C because it has excellent support for the kind of
programs I write: TSRs and programs with video windows. Microsoft C has some
support for those areas, but not as much as Turbo C.
I might have selected Microsoft C for the project, in which case I would be
addressing this same discourse to the users of all the other compilers. I like
Microsoft C. Recently my work has extended into the OS/2 arena, and that
venture has me using Microsoft C once again. All that great Turbo C support
does not apply to OS/2. First, Turbo C does not come in an OS/2 flavor.
Second, one does not write the same kind of TSRs for OS/2 that one writes for
MS-DOS. Third, one does not do windows in OS/2; that will be the domain of
screen groups and the Presentation Manager, like it or not.


The TC to MSC Surrogate Library


What good does all this rationalization do for those of you dear readers who
do not use Turbo C and who want to use the code in the "C Programming" column?
None, unfortunately, and that is something I intend to partially rectify in
this column. This month I will provide a library of Microsoft C functions and
macros that turn the "C Programming" project into a Microsoft C-compatible
program. The library will mirror those. Turbo C-specific functions that I have
used so that you can compile the project's functions with Microsoft C. In
subsequent months I will add to this library (if need be).
How about the other compilers? At last count there were countless other MS-DOS
C compilers. Lattice, Aztec, Watcom, Zortech, MIX, High C, Whitesmith's, Mark
Williams, De Smet, QC88, Small C, Eco-C, are all names that come to mind. At
one time or another, I have used each of them. I do not, however, intend to
provide a compiler-independent program here. I will describe the Turbo C
functions that must be emulated for Microsoft C, and I will provide the source
code for that purpose. Users of those other fine compilers may use my example
to build their own libraries. I am doing it for Microsoft C because that will
satisfy the largest base of readers, and that wheel is squeaking rather
prominently just now, thus getting the grease. To take this same side trip for
all those compilers would use my allotted space in DDJ for about the next
year, and we wouldn't get anything else done.
This library development is not a big effort as you will see, but I do
encourage anyone who builds one for another compiler to send the code in;
we'll post it on CompuServe in the DDJ Forum. But let me caution you. In the
past I wrote and marketed a library of C functions, and I distributed versions
that worked with most of those compilers. It was a frustrating experience. The
effort required to keep up with the latest versions of the compilers was
significant. The targets would never hold still. Eventually I surrendered and
distributed only source code. The source code was a melange of compile-time
conditionals that managed the difference between compilers and, in doing so,
obscured the original meaning of the code. Such a mess is marginally
acceptable in a commercial product, but it should never be seen in a published
work where the purpose is as much to inform as to perform.
I built these surrogate functions and tested them with Microsoft C, Versions
5.0 and 5.1. I tried to get them running with QuickC as well, but the window.c
file causes an internal QuickC compiler error. I recompiled just that module
with Microsoft C, linked it with the rest of the program as compiled with
QuickC, and everything worked. A more recent version of QuickC might not have
this problem, but I have only the first version.
The library consists of three source files and a make file. The source files
are microsft.h (see Listing One, page 129), microsft.c (Listing Two, page
129), and vpeek.asm (Listing Three, page 131). The make file is twrp.mak
(Listing Four on page 132). The make file uses these new functions to build
last month's TWRP tiny word processor with Microsoft C. The microsft.h file is
intended to be appended to or included in the window.h file from my September
column. The microsoft.c file provides the surrogate functions that emulate
those features of the project that depend on things unique to Turbo C.
These emulations are not comprehensive -- some of them will not work as Turbo
C substitutes for everything you might do. Rather, they provide sufficient
support for the ways we used those functions in the "C Programming" project so
far. For this reason I call them surrogates rather than clones. Naturally, as
I add to the project, I will verify that all the new code works with both
compilers, and I will update this library if necessary. If you want a
comprehensive cross-compiler library, you might try buying the Turbo C runtime
library source code and compiling the pertinent functions with Microsoft C.
Such an ambitious endeavor is beyond the scope of this column. I need say no
more than that it would be a significant effort, fraught with peril.
Following is a discussion of each component of the library.
microsft.h -- The microsft.h file is meant to be appended to or included in
window.h from September. You must make one change to window.h to accommodate
differences between the two compilers' treatments of prototypes. In window.h
on about line 9 you will see this prototype:
 int select_window(int,int,int,int (*func)(int,int));
You must change it to the following:
 int select_window(int,int,int,nt (*)(int,int));
Microsoft C does not allow you to mix named and unnamed parameters in a
prototype while Turbo C does not seem to mind.
The microsft.h file begins with a test of the COMPILER compile-time
conditional flag. This flag is defined by the compile statement in the make
file, and it tells the compiler to use the code inserted from microsft.h. My
objective was to minimize changes to code already published.
The #define statements for movmem and setmem substitute the names of
appropriate MSC functions.
The setmem function sets all the bytes in a buffer to a specified character
value. The first parameter is a character pointer that points to the buffer.
The second parameter is the buffer's size in bytes. The third parameter is the
character value to be filled into the buffer. The corresponding Microsoft C
function is memset. Its parameters are in a different sequence than those of
Turbo C's setmem.
The movmem function takes three parameters: a source character pointer, a
destination character pointer, and a byte count. The function moves a block of
memory and accounts for overlapping source and destination blocks, moving from
the correct end of the buffer to prevent byte replication. The corresponding
Microsoft C function is memmove. Its parameters are in a different sequence
than those of movmem. Note that Microsoft C has a movmem function that
resembles Turbo C's movmem, but it does not correct for overlapping buffers
and cannot be used here.
The #define statements for cprintf, cputs, getch, and putch substitute the
names of functions in microsft.c for these names. Both Turbo C and MSC have
functions with these names, but their behaviors are different enough that we
must make substitutions.
Next come the prototypes for the functions of TC that MSC does not have. After
that are #define statements for the screen colors.
microsft.c -- This file has the functions that emulate the Turbo C functions
-- the Turbo surrogate. It starts with some #includes and prototypes. The
vpeek and vpoke prototypes are for the functions in vpeek.asm, which read and
write video memory compensating for video snow. If your system does not use
the color graphics adaptor (CGA), you do not need the assembly language
functions.
The compile-time global symbol, ADAPTOR, specifies the video adaptor your
program uses, and the VSEG symbol is automatically equated to the segment
address of display refresh memory, which is 0xb000 for the monochrome display
adaptor and 0xb800 for the others. The SNOW symbol is set to 0 if the adaptor
does not generate video snow when video memory is accessed. Otherwise it is
set to 1. If you use an adaptor other than the CGA, you can remove the
references to vpeek.asm and vpeek.obj from Listing Four, twrp.mak.
With Turbo C the tests for the video segment and snow are made at run time by
the compiled code and are not visible to you. For this subset emulation,
however, you must specify the video adaptor when you compile the program. This
is consistent with the philosophy of this program where configuration items
are controlled by compiled #define control statements.
The video structure is a duplicate of an external structure that is internal
to Turbo C and that is referenced in window.c. We declare it here because
Microsoft C has no such structure. By maintaining the values that our software
uses, we can make the program react just as it does when the Turbo C run-time
library (RTL) is running things.
The window function is used to establish a rectangle of memory as the current
video window. Its integer parameters are the left, top, right, and bottom
screen coordinates where the upper left screen coordinate is row 1, column 1,
and the lower right coordinate is row 25, column 80.
The _vptr function returns a far pointer to video memory based on the x and y
coordinates passed to it. This is an emulation of an external Turbo C function
that is normally only called from within the Turbo C run-time library but that
we used in window.c. You'll recall that I chastised myself for using it. Now I
pay.
The _vram function writes a linear block of program memory to video memory.
Its parameters are a far pointer to the start of the video memory location, a
far pointer to the program memory buffer, and the number of 2-byte integers to
write. Video memory consists of 2 bytes -- a video attribute and an ASCII
character -- for each screen character position. This function is also an
emulation of one that is normally only called from the TC RTL. We used it in
window.c to effect a smooth write to the screen without the annoyance of a
cursor flash across the write such as you see when you use cprintf and cputs.
The _getvram function is the reverse of _vram. It reads rather than writes
video memory. This function is not an emulation, but one that we need for the
gettext function described below. In Turbo C, the _vram function manages video
memory moves in both directions by using the segment addresses to determine if
the source or destination is the video RAM. Rather than go to that trouble, I
coded a simple _vram and then added _getvram as a reciprocal of _vram.
The gettext and puttext functions are higher-level read and write video memory
functions. They deal with linear buffers of program memory and rectangular
windows of video memory. They are used to save and restore the video space
that our windows occupy. The coordinates (left, top, right, bottom) are
relative to the full screen and begin with 1,1 at the top left. The movetext
function moves a video window. We use it for window scrolling, so this
emulated function works with vertical movements only. The Turbo C movetext
function is a lot smarter, being able to move a window from and to any screen
positions. The arguments to movetext are the four corner coordinates of the
original window (relative to the full screen) and the upper left coordinates
of where it is to be moved. If you are using the CGA, you can eliminate
movetext because we used BIOS to scroll CGA screens for performance reasons.
To avoid changing the scroll function in window.c, put a null function named
movetext in microsft.c in place of the one given here.
The gotoxy function positions the cursor at a location in the current window
(the one most recently defined by the window function) as specified by the x
and y parameters. These parameters are relative to the window, with 1,1 being
the upper left corner of the window.
The wherex and wherey functions return the current cursor x and y positions
relative to the current window.
The textcolor and textbackground functions set the colors that will be used
the next time a text display is written. Their parameters are integer values
as defined in microsft.h.
The wprintf function is substituted for cprintf. Both compilers have cprintf
functions that are similar, but they are both related to their own internal
text display processes. The cprintf macro in microsft.h replaces calls to
cprintf with ones to wprintf. This new function makes a simple translation
using the vsprintf function of Microsoft C. The resulting string is then
copied to video memory based on the current cursor position. Note that this
surrogate function does not attempt to deal with control characters such as \n
and \r. This omission is because we never use such controls in our calls to
cprintf. I hope I do not regret this shortcut later.
We have substituted wgetch and wputch for getch and putch because the existing
Microsoft C functions do not work in the context in which we are using them.
vpeek.asm -- The vpeek function has two unsigned parameters. The first is the
segment address of video memory (always the value #defined as VSEG) and the
second is the video offset address. The function returns the 2-byte value in
the video memory address. Before accessing video RAM, vpeek waits for a video
retrace cycle to avoid the snow-causing memory access conflict between the CPU
and the video controller.
The vpoke function inserts a 2-byte value into a video memory location with
the same snow-eliminating routine as vpeek. It has the same two video memory
address parameters as vpeek. Its third parameter is the two-byte word to be
inserted into video memory.



Header Files


Turbo C and Microsoft C define their function prototypes in header files in
mostly the same way. The same header files define the same ANSI standard
functions. There are two minor differences, however. Turbo C defines all the
memory allocation functions in alloc.h and Microsoft C uses malloc.h. Turbo C
uses mem.h for prototypes of functions similar to those that Microsoft C puts
in memory.h. To get around this problem, you can put surrogate alloc.h and
mem.h files in with your Microsoft C header files. The alloc.h file should
simply #include malloc.h. The mem.h will #include memory.h. I have not
included listings of these one-liners.


A C Crotchet: The ANSI Gotcha


In its goal to specify a standard C language that is all things to all
computers, the ANSI X3J11 committee has faced some thorny problems. Their
solutions do not always provide the best answer for everyone. Once upon a
time, if you coded this statement:
 char cp [] = "\00123";
you got a character array of four characters initialized with three characters
and a null terminator. The first character was the binary value 001 as
specified by the backslash octal sequence. The next two characters were the
ASCII values '2' and '3'.
The draft ANSI specification says that since some computers have character
sizes of greater than eight bits, the backslash sequence must allow for more
than three digits. Therefore, the statement will now declare a two-character
array with the octal value 123 (hex 53) followed by the null terminator. The
compiler's scan of digits to form an integral value continues as long as the
compiler sees octal digits.
Existing code can get broken by compilers that comply, particularly if the
array is compiled without warning messages. ANSI makes no provision for the
protection of existing code with the changes brought about by this new rule.
This discussion is, therefore, aimed at those who might be changing to an
ANSI-compliant compiler. Eventually that will include all of us. This will be
part of the future ANSI legacy, but let's see how it is effecting some of us
right now.
Borland decided to comply with this new rule of the standard in Turbo C 2.0.
This decision, I am told, was based on Borland's commitment to full compliance
with the ANSI standard. That position is hard to argue with even when the
consequences seem dire. Allow me to try.
When programmers complained that this new rule was breaking existing code,
they were offered a workaround: Use the ANSI string concatenation feature,
which works like this:
 char *cp = "\001" "123";
This, of course, is a workaround for developers of new code and does not
address the problem faced by those who compile large systems of existing code.
ANSI addressed this rule to solve portability issues. Some machines have 6-bit
characters, some have 8, some have 12. Traditionally, when a programmer ported
C code to a new machine, this was one of the portability considerations. Now,
however, the new ANSI rule -- or more specifically, the Borland compliance
with it -- gives us an unexpected portability issue and an unwanted surprise.
Programs that compiled correctly one way for years -- including with Turbo C
1.5 -- are now not portable to Turbo C 2.0, and the compiler issues no warning
and provides no way to turn off the new rule.
The only error message associated with the original format occurs when
coincidental octal digits following the octal constant happen to form an
integral value greater than 255. So if the initializer contains "\00377", the
compiler says your code is correct but compiles something other than what ten
years of tradition have conditioned us to expect. If, however, the string is
"\00400" you get a compile error. In the latter circumstance you can do
something about it. In the former, you have no clue that something is amiss
until the program stops working.
In my opinion, a compiler should issue a warning when it deviates from
tradition in the name of ANSI. The warning could be turned off by those who do
not need or want to see it.
Perhaps compilers for computers with 8-bit characters should ignore the new
rule. I doubt that many users would complain. That's right, your DDJ C
columnist advocates nonviolent civil disobedience sometimes. As Paul Newman
said in the movie Hud, "I've always believed in being lenient with the law.
Sometimes I lean one way, sometimes I lean the other."
Most PC programs, however, are written to execute from inside a user interface
that is tightly bound to the architecture of the PC. Our "C Programming"
column project is an example of such a program. The vast majority of programs
written in Turbo C will never be ported anywhere other than to the next
improved edition of Turbo C. I would guess that if Borland were to offer an
ANSI-only version of the compiler, there would be few takers.
Borland's official position on this issue is that the compiler exhibits
correct ANSI behavior, and is, therefore, correct. On the other hand, Borland
is a company that listens to its users. If enough of you say that you need
something changed, they will change it. I say we need a warning message.
Since version 1.0, I have recommended Turbo C without hesitation. Until this
problem is addressed, however, I suggest caution if Turbo C 2.0 is to be used
in projects that involve large helpings of existing code. Fortunately, Turbo C
1.5, Microsoft C 5.1, and other compilers have not adopted this rule, so you
have reasonable alternatives to Turbo C 2.0 if this new rule will be a
problem.


Other Turbo C 2.0 Offerings


Turbo C 2.0 has a bounty of new features. Most notable is the long-awaited
integrated debugger in the environment. Borland now markets a standalone Turbo
Debugger as well, and Turbo C programs can be debugged with it, too. If you
get the Turbo C Professional package, you get Turbo C 2.0, the Turbo Debugger,
and the new Turbo Assembler. Also included are the linker, librarian, make
utility, grep, and a host of other programmer's utility programs. All that is
missing is a multi-window, programmable programmer's editor in the class of
Brief or the Microsoft Editor.
One new feature of Turbo C 2.0 is a mixed blessing. Each object file is
encoded with the paths and names of the #include files that went into its
compilation. The Project Make facility has an option called "Auto
dependencies." When this option is on, the make process checks the dates of
the files that were included against the date of the object file. This is
useful in projects where the project make files might not be current. With
this feature you can get pretty sloppy about keeping the project make file up
to date.
Why a mixed blessing? The embedded file names can increase the size of the
object file significantly. One developer found that his commercial library now
required additional diskettes. There is an undocumented TLIB switch (/0) for
eliminating these strings. My sources say that Borland will post a utility to
strip the path names from object files and that the next release of Turbo C
will include a switch to suppress them. Now that I have them, though, I cannot
do without them. I would like to see them used by the command line MAKE
utility program.
Bugs in the version 1.5 cprintf and cputs functions were fixed in version 2.0.
What's that, you didn't know those functions were broken? Then you, like I,
did not read the documentation, which has always said that cprintf and cputs
do not expand the newline into a carriage return, line feed. That's what the
documentation said, but in TC 1.5, the new-line was expanded. In 2.0 you need
to code \r\n to get the same effect you got with \n before. This keeps the
compiler in synch with its documentation and also with Microsoft C. TC will
need to stay close to MSC with many such functions if they intend to get into
the OS/2 game. Many programmers used cprintf the way it worked rather than how
it was described and are not pleased with the change.


Coming up...


Next month we get on with the project. We'll discuss the weighty subject of
asynchronous communication and add serial port and modem functions to our
library.

_C PROGRAMMING_

by Al Stevens


[LISTING ONE]


/* -------------- microsft.h ---------------- */
/* #include this file at the end of window.h */

#if COMPILER == MSOFT

/* this line replaces the select_window prototype in window.h */
int select_window(int, int, int, int (*)(int,int));

#define setmem(bf,sz,c) memset(bf,c,sz)
#define movmem(fr,to,ln) memmove(to,fr,ln)

#define cprintf wprintf
#define cputs(s) wprintf(s)
#define putch(c) wputch(c)
#define getch() wgetch()

void window(int lf,int tp,int rt,int bt);
void puttext(int lf,int tp,int rt,int bt,char *sv);
void gettext(int lf,int tp,int rt,int bt,char *sv);
void movetext(int lf, int tp, int rt, int bt, int lf1, int tp1);
void gotoxy(int x,int y);
void textcolor(int cl);
void textbackground(int cl);
int wherex(void);
int wherey(void);
void wprintf(char *, ...);
void wputch(int c);
int wgetch(void);

#define BLACK 0
#define BLUE 1
#define GREEN 2
#define CYAN 3
#define RED 4
#define MAGENTA 5
#define BROWN 6
#define LIGHTGRAY 7
#define DARKGRAY 8
#define LIGHTBLUE 9
#define LIGHTGREEN 10
#define LIGHTCYAN 11
#define LIGHTRED 12
#define LIGHTMAGENTA 13
#define YELLOW 14
#define WHITE 15

#endif






[LISTING TWO]

/* ----------- microsft.c ------------ */

/*
 * Surrogate Turbo C functions
 * for Microsoft C users.
 */

#include <dos.h>
#include <string.h>
#include <stdio.h>
#include <stdarg.h>
#include <ctype.h>
#include <conio.h>
#include <bios.h>


/* -------- One of these is your Display Adapter -------- */
#define MDA 1 /* Monochrome Display Adapter */
#define CGA 2 /* Color Graphics Adapter */
#define EGA 3 /* Enhanced Graphics Adapter */
#define VGA 4 /* Video Graphics Array */

#define ADAPTER EGA /* Specifies the Display Adapter */

#if ADAPTER==MDA
#define VSEG 0xb000 /* VSEG is the video memory segment */
#else
#define VSEG 0xb800
#endif

#if ADAPTER==CGA
#define SNOW 1
/* --- assembly language vpeek.asm: manages CGA flicker --- */
void vpoke(unsigned adr, unsigned off, int ch);
int vpeek(unsigned adr, unsigned off);
#else
#define SNOW 0
/* ---- macros for vpeek and vpoke for non-CGA systems ---- */
#define MKFP(s,o) (((long)s<<16)o)
#define vpoke(a,b,c) (*((int far*)MKFP(a,b))=c)
#define vpeek(a,b) (*((int far*)MKFP(a,b)))
#endif

static union REGS rg;

/* --- a structure defined within Turbo C and used by us --- */
struct {
 char filler1[4];
 char attribute; /* saves the current video attribute */
 char filler2[5];
 char snow; /* says if the adapter snows */
} _video;

static int wlf,wtp,wrt,wbt; /* current window corners */
static int wx,wy; /* current window cursor */

/* ------- define a video window ---------- */
void window(int lf,int tp,int rt,int bt)
{
 wlf = lf;
 wtp = tp;
 wrt = rt;
 wbt = bt;
 _video.snow = (char ) SNOW;
}

/* ------ makes a video offset from x,y coordinates ----- */
#define vaddr(x,y) (((y)-1)*160+((x)-1)*2)

/* -- makes far pointer to video RAM from x,y coordinates -- */
void far * pascal __vptr(int x, int y)
{
 void far *vp;

 FP_SEG(vp) = VSEG;

 FP_OFF(vp) = vaddr(x,y);
 return vp;
}

/* ---- writes a block of memory to video ram ----- */
void pascal __vram(int far *vp, int far *bf, int len)
{
 while(len--) {
 vpoke(VSEG, FP_OFF(vp), *bf++);
 vp++;
 }
}

/* ---- gets a block of memory from video ram ----- */
void pascal __getvram(int far *vp, int far *bf, int len)
{
 while(len--) {
 *bf++ = vpeek(VSEG, FP_OFF(vp));
 vp++;
 }
}

/* ----- writes a memory block to a video window ----- */
void puttext(int lf,int tp,int rt,int bt,char *sv)
{
 while (tp < bt+1) {
 __vram(__vptr(lf, tp), (int far *) sv, rt+1-lf);
 tp++;
 sv += (rt+1-lf)*2;
 }
}

/* ----- reads a memory block from a video window ------ */
void gettext(int lf,int tp,int rt,int bt,char *sv)
{
 while (tp < bt+1) {
 __getvram(__vptr(lf, tp), (int far *) sv, rt+1-lf);
 tp++;
 sv += (rt+1-lf)*2;
 }
}

/* ------ moves a video window (used for scrolling) ------ */
void movetext(int lf, int tp, int rt, int bt, int lf1, int tp1)
{
 int nolines = bt - tp + 1;
 int incr = tp - tp1;
 int len, i;
 unsigned src, dst;

 if (tp > tp1) {
 src = tp;
 dst = tp1;
 }
 else {
 src = bt;
 dst = tp1+nolines-1;
 }
 while (nolines--) {

 len = rt - lf + 1;
 for (i = 0; i < len; i++)
 vpoke(VSEG, vaddr(lf1+i, dst),
 vpeek(VSEG,vaddr(lf+i, src)));
 src += incr;
 dst += incr;
 }
}

/* ----- position the window cursor ------ */
void gotoxy(int x,int y)
{
 wx = x;
 wy = y;
 rg.h.ah = 15;
 int86(16, &rg, &rg);
 rg.x.ax = 0x0200;
 rg.h.dh = wtp + y - 2;
 rg.h.dl = wlf + x - 2;
 int86(16, &rg, &rg);
}

/* ----- return the window cursor x coordinate ----- */
int wherex(void)
{
 return wx;
}

/* ----- return the window cursor y coordinate ----- */
int wherey(void)
{
 return wy;
}

/* ----- sets the window foreground (text) color ------- */
void textcolor(int cl)
{
 _video.attribute = (_video.attribute & 0xf0) (cl&0xf);
}

/* ----- sets the window background color ------- */
void textbackground(int cl)
{
 _video.attribute = (_video.attribute & 0x8f) ((cl&7)<<4);
}

void writeline(int, int, char *);

/* ------ our substitution for MSC cprintf -------- */
void wprintf(char *ln, ...)
{
 char dlin [81], *dl = dlin, ch;
 int cl[81], *cp = cl;
 va_list ap;

 va_start(ap, ln);
 vsprintf(dlin, ln, ap);
 va_end(ap);


 while (*dl) {
 ch = (*dl++ & 255);
 if (!isprint(ch))
 ch = ' ';
 *cp++ = ch (_video.attribute << 8);
 }
 __vram(__vptr(wx+wlf-1,wy+wtp-1),
 (int far *) cl, strlen(dlin));
 wx += strlen(dlin);
}

/* ------ our substitution for MSC putch -------- */
void wputch(c)
{
 if (!isprint(c))
 putch(c);
 wprintf("%c", c);
}

/* ------ our substitution for MSC getch -------- */
int wgetch(void)
{
 static unsigned ch = 0xffff;

 if ((ch & 0xff) == 0) {
 ch++;
 return (ch >> 8) & 0x7f;
 }
 ch = _bios_keybrd(_KEYBRD_READ);
 return ch & 0x7f;
}





[LISTING THREE]

;--------------------------- vpeek.asm ----------------------------
 dosseg
 .model compact
 .code
 public _vpoke
; -------- insert a word into video memory
; vpoke(vseg, adr, ch);
; unsigned vseg; /* the video segment address */
; unsigned adr; /* the video offset address */
; unsigned ch; /* display byte & attribute byte */
; ------------------------------------------
_vpoke proc
 push bp
 mov bp,sp
 push di
 push es
 mov cx,4[bp] ; video board base address
 mov es,cx
 mov di,6[bp] ; offset address from caller
 mov dx,986 ; video status port address
loop1: in al,dx ; wait for retrace to quit

 test al,1
 jnz loop1
loop2: in al,dx ; wait for retrace to start
 test al,1
 jz loop2
 mov ax,8[bp] ; word to insert
 stosw ; insert it
 pop es
 pop di
 pop bp
 ret
_vpoke endp

 public _vpeek
; -------- retrieve a word from video memory
; vpeek(vseg, adr);
; unsigned vseg; /* the video segment address */
; unsigned adr; /* the video offset address */
; ------------------------------------------
_vpeek proc
 push bp
 mov bp,sp
 push si
 push ds
 mov si,6[bp] ; offset address
 mov cx,4[bp] ; video board base address
 mov ds,cx
 mov dx,986 ; video status port address
loop3: in al,dx ; wait for retrace to stop
 test al,1
 jnz loop3
loop4: in al,dx ; wait for retrace to start
 test al,1
 jz loop4
 lodsw ; get the word
 pop ds
 pop si
 pop bp
 ret
_vpeek endp
 end





[LISTING FOUR]

#
# TWRP.MAK -- make file for TWRP.EXE with Microsoft C/MASM
#

.c.obj:
 cl /DCOMPILER=MSOFT -c -W3 -Gs -AC $*.c

twrp.obj : twrp.c editor.h help.h window.h

editshel.obj : editshel.c editor.h menu.h entry.h help.h \
 window.h microsft.h


editor.obj : editor.c editor.h window.h microsft.h

entry.obj : entry.c entry.h window.h microsft.h

menu.obj : menu.c menu.h window.h microsft.h

help.obj : help.c help.h window.h microsft.h

window.obj : window.c window.h microsft.h

microsft.obj : microsft.c

vpeek.obj : vpeek.asm
 masm /MX vpeek;

twrp.exe : twrp.obj editshel.obj editor.obj entry.obj menu.obj help.obj \
 window.obj microsft.obj vpeek.obj
 cl twrp editshel editor entry menu help window microsft vpeek











































January, 1989
STRUCTURED PROGRAMMING


Useful Utilities in Marvelous Modula




Kent Porter


Last September I did something I've done only twice before in dozens of
reviews: gave a rave to JPI's TopSpeed Modula-2. Now it's time to put it to
practical use.
With the spate of utilities that have appeared in this column, I've given some
thought to taking the lead from Peter Norton and calling them Porter's
Utilities. If I really followed Peter's example, though, I'd have to
abbreviate the name as PU: hardly an appealing prospect, so I guess they'll go
without a name.
At any rate, this month we'll present three utility programs that are
genuinely useful as well as illustrating some aspects of systems programming
in Modula.
Before I stray too far from the September review, though, there are a couple
of things to add. First is a correction to Table 6 on page 76. I reported that
Logitech's FP run time with a coprocessor was 45.28 seconds. Several readers
let me know that I had missed an obscure passage in the Logitech docs
regarding linkage with the proper floating point library, and that if I hadn't
overlooked it, the results would have been on the order of 12 or 13 seconds. I
haven't rerun the benchmark, but one reader got results similar to mine with
TopSpeed and 12.5 or thereabouts with Logitech. From that we can conclude that
Logitech's floating point performance with a coprocessor isn't spectacular,
but it's a whole bunch better than I reported and comparable with the others.
Sorry, guys.
The Modula review ranked twice as high as the second runner up in the reader
interest survey, and I got more feedback from that than from anything (maybe
everything) else I've ever written. There's an amazing amount of interest in
Modula-2 Out There. It is an up-and-coming language with some strong
supporting compilers, and it is a viable alternative to C, and I was gratified
to find that a lot of folks agree with me.
Don't get me wrong, I like C. I have two advanced C programming books coming
out in the next couple of months. While there's an element of shameless
commercialism in mentioning that, there's also an implicit endorsement of C as
a strong, vibrant language with a future. But frankly, C makes you work harder
than you have to and it's difficult to read when the code gets cold. I live in
a polyglot world of programming languages and believe me, life is easier on
the Pascal/Modula shores than it is on the C.
All of which is a roundabout way of getting to the subject of this month's
column.


Utility Number 1: How Much Free Memory Do You Have?


Software products keep getting bigger, while at the same time available memory
keeps shrinking as we find more and more TSRs that we can't live without. I
never once ran out of memory on my 64K CP/M machine (remember those days?),
but it happens often enough on my 640K AT to be a problem. Even modern
compilers with their interactive environments and dynamic linkers and
integrated debuggers and such often require upwards of 400K, which pushes
against the limits of a well-TSRed machine.
So the first utility in this group reports the amount of free memory. It
includes the space it occupies in the result; that is, the program measures
free space from the start of the area it occupies, and not just the space
above itself.
You can determine the beginning of the space a program claims by getting the
segment of its program segment prefix (PSP). This is a 256-byte data structure
that DOS builds when it loads a program for execution; the program itself
starts immediately above the PSP. When the program terminates, the memory
starting with the PSP becomes free again.
The PSP always begins on a paragraph boundary, a paragraph being a 16-byte
region expressed by the segment portion of the address. Therefore its offset
is always zero, and we know where the PSP is simply by finding out its
segment.
DOS provides two ways to get the PSP segment. One is the undocumented function
51h, and the other is function 62h, both under Int 21h. Function 62h is an
official part of DOS 3.0 and above; function 51h was the only way to get the
PSP in DOS 2.n. The latter has a bug that conflicts with Int 28h, which is
where DOS hangs out when it's looking for something to do. Because the MEMORY
utility (Listing One, page 134) might have to run under DOS 2.n, it uses
function 51h. No danger, since DOS isn't idle while this program is running.
The bug is only a problem with TSR's.
The PSPO procedure in MEMORY.MOD shows how to use function 51h to get the PSP
segment, which is returned in register BX. The usage for function 62h is
identical, but of course 62h doesn't work under DOS, Version 2. The PSPO
function returns the product of register BX times 16. Why the multiplication?
Because BX holds the PSP paragraph segment, so we multiply by 16 to get the
byte address of the PSP. This tells us, in effect, how much memory is used
below the utility.
The other critical value in this utility is the total size of main memory. It
happens that this value is stored in the ROM BIOS data area, a treasure trove
of machine-specific information based at paragraph 40h. The main memory size,
expressed in K, is a CARDINAL (unsigned integer) at offset 13h. Line 10 in
Listing One shows how to declare a variable at an absolute location using a
Modula-2 address constructor. The analogous declaration in Turbo Pascal would
be
 MemSize : WORD ABSOLUTE $0040:$0013;
The first statement in the main body of MEMORY casts MemSize to a LONGCARD,
which is a 32-bit unsigned integer, and multiplies by 1024 to convert K to
bytes. The rest is simple: Subtract the bytes below the PSP and report the
difference, which is the free memory in the machine.
There are a couple of things to note about the output instructions, First,
Modula-2 doesn't have a catch-all, high overhead statement like Pascal's
WriteLn, into which you can pour any number of parameters of varying types.
Instead, it has one output instruction for each data type. This makes for more
keystrokes to write a program.
That leads to the second point, which is unique to TopSpeed Modula-2. Niklaus
Wirth, the inventor of Modula-2, suggests some module and procedure names.
Most Modula implementors have followed Wirth's lead. JPI has been criticized
for deviating from these recommendations, but their approach cuts down on the
number of keystrokes by abreviating the Wirth names. For example, Wirth speaks
of a console input/output module called InOut, and a routine called
WriteString. Under his way of doing things, then, the statement at line 28
would read
 InOut.WriteString ('Available...');
which is nine more keystrokes than in TopSpeed. Multiply this by all the
console I/O statements in a typical program and you find a significant
difference in the sheer amount of work to write the source.
Speaking of module names, notice the IMPORT list at line 7 in Listing One. A
Modula-2 module has to explicitly import any libraries it uses. Specific
procedure calls are then bound to their supporting library using dot notation.
This clearly identifies which library the procedure comes from, and allows
similarly-named procedures to co-exist. For example, both IO and FIO (the file
I/O library) furnish a WrStr procedure. You specify which you mean through the
dot notation. There's also a way of importing specific elements from a library
to avoid use of this verbose dot notation; lines 9 and 52 - 53 in Listing
Three, page 134, illustrate how.
MEMORY is a handy utility to keep in your toolbox. Run it any time you need to
find out how much free memory you have. Also, you can run it before and after
loading a TSR to learn how much memory the TSR actually grabs.


Utility Number 2: What Are The Subdirectories?


The DOS DIR command and the SUB utility I published in this column a few
months back both list subdirectories, but not exclusively; they're sprinkled
randomly among other files, which makes them hard to spot. Often in the past I
used to start up something like XTREE simply to find out what subdirectories
were hanging off a specific directory. When that eventually got on my nerves
enough, I wrote SD.MOD, shown in Listing Two, page 134.
Put SD.EXE somewhere along your PATH. Then, wherever you are, you can type SD
and the program will list only the subdirectories in the current directory.
Unlike a DOS shell or the DOS TREE utility, SD doesn't list the children of
those subdirectories. It concedes that capability for the benefits of being
fast and requiring no command-line arguments.
SD relies heavily on the DOS findfirst/findnext functions (4Eh and 4Fh under
Int 21h). These functions are Siamese twins. Findfirst locates the first
instance of a filename pattern with a given file attribute and builds a data
structure. Findnext uses this data structure to step through the remainder of
the current directory, on each call pulling out the next file entry that
satisfies the parameters.
In TopSpeed Modula-2, these DOS functions are sugar coated with Modula syntax
as ReadFirstEntry and ReadNextEntry, and are located in the FIO library. They
return TRUE if the call is successful and FALSE otherwise. The involved data
structure is of type DirEntry, also from FIO. Thus lines 15 - 16 in Listing
Two make the first call, and then a loop begins at line 17. This loop will not
execute if ReadFirstEntry failed to find a match. In fact, for reasons we'll
discuss shortly, the loop will almost always execute at least twice despite
its structure. Within the loop's scope is the call to ReadNextEntry, which
also returns a Boolean to the Found variable controlling the loop. Thus the
loop reiterates until ReadNextEntry runs out of directory entries satisfying
the criteria in the DirEntry structure.
Note the parameters for ReadFirstEntry. They specify, in order, the pattern to
look for, the file attribute(s), and the name of a DirEntry object. The
attributes are a set of type FileAttr defined in FIO. The names describe the
possible file attributes: readonly, hidden, system, volume, directory, and
archive. In the case of SD, we want to look only for entries that have the
directory attribute, so that's the only set member we pass to ReadFirstEntry
as an argument.
The underlying DOS functions have a "feature," or in other words a fairly
well-known bug that the vendor doesn't intend to fix. The bug is that, even
when you specify the directory attribute alone, the functions return not only
directory entries, but also all normal (read/write) file entries. For that
reason, it's necessary to check the result in the attr field of the DirEntry
structure to make sure it has a directory attribute. That's the purpose of
line 18. It screens the results, allowing only legitimate directories to be
processed further.
Every DOS subdirectory (that is, every directory except the root) has the two
entries '.' and '..' referring to the current directory and its parent,
respectively. That explains why the loop beginning at line 17 will execute at
least twice; the only exception is when SD is looking at the root of an empty
disk. Because the existence of these entries is a given, there's no reason to
list them. Consequently the IF at line 19 filters them out.
SD is one of those little utilities that you quickly come to take for granted
because it's so useful and such a natural extension of DOS. Here it does
double duty, because it also lays the ground-work for the much more complex
directory search utility discussed next.


Utility Number 3: Where is ... ?


The final program in this set is WHERE.MOD (Listing Three). This utility
searches the entire directory structure of a disk, reporting the location of
every entry that satisfies a pattern.

The pattern can be any combination of literal and wildcard characters. For
example, if you want to know the location of every file with the extension
.OLD, type
 WHERE, OLD
or
 WHERE *.OLD
Similarly, to find all files whose names have AU as the first two characters,
type
 WHERE AU*
or
 WHERE AU*.*
Note that, unlike Modula-2 itself, the pattern is not case sensitive. You can
list specific filenames if you remember what the file is called but not where
it is. The command is also useful for chasing down unnecessary instances of a
file, as in
 WHERE COMMAND.COM
In response to the command, WHERE lists each instance of a matching file,
showing the drive, directory path, and filename.
WHERE can also search other drives. If you're running off drive C and you want
to find all the dBase files on drive D, type
 WHERE D:.DBF
or
 WHERE D:*.DBF
The utility interprets the command line argument to see if it contains a
drive, and performs the necessary drive change when it does. It also completes
wildcards, which is why some of the example commands have two forms.
And if you don't furnish a command-line argument, WHERE asks you for the
pattern with the query Filename?
The main body of WHERE begins by finding out the current directory and drive.
This information is needed in order to restore the user's environment when the
program quits. The next step is to get the command-line argument or, if not
present, ask for it.
The code beginning at line 119 interprets the search pattern. When the second
character is a colon, the program knows that the first character is a drive
designator, so it converts the letter into its numeric DOS equivalent (where 0
= A, 1 = B, etc.). Having done that, it selects the new drive and shifts the
pattern left to form a driveless pattern. (Note that the program doesn't
recognize a subdirectory as a starting point; all searches proceed from the
root, so a pattern containing a directory path won't produce meaningful
results.)
The output list will contain the drive prefix d:\ where d is a drive name.
This string is contructed starting at line 134.
Lines 140 - 146 patch in wildcards as needed. For example, if the user typed
 WHERE.BAK
this routine translates the pattern into *.BAK and, if
 WHERE SALES
into the pattern SALES.*. There is a distinction, by the way, between patterns
such as SALES and SALES*. If you type the first, the program reports only
files with SALES as the base name, and skips those such as SALESREP.XYZ. The
pattern SALES* finds the latter, because it gets translated into SALES*.*.
With all the preliminaries out of the way, the program can now undertake the
search. This always begins at the root and progresses through the entire
directory structure, checking every single file entry on the disk. Line 149
initiates the search.
The SearchDir procedure is recursive. That is, it calls itself, passing the
name of a child directory to search. A level of recursion thus exists for each
level in the generations of subdirectories. When a child directory has been
completely searched, the procedure returns to the parent level. When all
directories have been visited, the root level returns to the main program's
cleanup routines starting at line 151. Here's how it works in more detail.
SearchDir begins by changing to the directory it was passed as a parameter,
and reconstructing the full pathname including drive for reporting purposes.
It then searches the current directory for any entry satisfying the pattern.
The names of matching entries are listed on the display. When the matching
entry is a subdirectory, this first phase merely reports the name suffixed
with <DIR>; it doesn't actually branch to that directory yet.
That's what the second phase starting at line 66 does. When all entries in the
directory have been checked, line 67 starts over again, scanning specifically
for subdirectories much as SD does. However, here the loop issues a recursive
call to SearchDir, passing the name of the subdirectory. This call suspends
the parent invocation of SearchDir and restarts the procedure with different
data needed to search the child directory. This process continues through the
children of the children. Eventually all lines of inquiry along the path are
exhausted, and the child level returns to the parent level at line 71. The
child invocations changed the current directory, so line 71 restores the
parent directory and continues the loop seeking other child paths to search.
When it runs out of directories, it returns to its parent level of recursion,
and so on until eventually control returns to the main program at line 151,
thus ending the program run.
Note how the screen output is controlled. Line 49 displays the full name of
the path currently being searched. When a match occurs, lines 55 - 60 add the
filename and advance to the next display row, where line 61 again shows the
current path. Often a path contains no matches, and even when it does,
eventually it runs out of them. In either case, the next invocation of
SearchDir calls ClrSol from line 48. This routine erases the name of the old
exhausted path and overlays it with the next.
ClrSol (for clear to start of line) does this by using yet another field in
the ROM BIOS data area. When it gets control, the cursor is winking at the end
of the current path, which might be of any length. Offset 50 contains the
current cursor column in video page 0, the default video buffer for WHERE. The
procedure learns where the cursor is by checking this absolute variable. It
then backs up the cursor, replacing each position with a space, until the
cursor is in column 0, the left margin of the display area. Line 49 can then
write a fresh string indicating the new search path without worrying about
excess baggage left over from a longer line previously occupying the same row.
Similarly, the code beginning at line 151 erases the last fruitless path to
report the number of matches found.
TopSpeed Modula-2 includes a comprehensive high-level module called Window,
providing for advanced screen control. The TopSpeed interactive environment
itself was written using the Window module. However, Window takes more control
of the display than I deemed desirable for this program. For example, the
first call to any Window routine clears the display and sets the default text
color to bright white. I preferred that WHERE have the appearance of standard
DOS utilities, which is why I devised the ClrSol routine.
So there you have three useful utilities and a hands-on demonstration of
Modula-2 in action. Like C, Modula is a language inherently capable of
embracing the full spectrum of programming requirements, from low-level
systems utilities to advanced applications.


Handing Over the Reins


We all mourn the demise of Turbo Technix, Borland's excellent programming
magazine. Yet, even bad things have positive results. One of them is that we
were able to recruit Mike Floyd to come to work for DDJ as technical editor.
Another is that it freed Technix' editor-in-chief, Jeff Duntemann, to take
over this column.
Jeff needs little introduction to structured programming devotees. Through his
work at PC Tech Journal and Turbo Technix, as well as several authoritative
books, Jeff has had a substantial impact on the art of structured programming.
Indeed, one might fairly describe him as Mr. Pascal. He's also an accomplished
Modula programmer, and a superlative writer. Consequently, I leave the column
in capable hands.
Doing this column for the past year has been a lot of fun and earned me many
new friends. Had Technix not shut down and made Jeff available, I would gladly
have continued with it. But for a long time I've had a fantasy about doing a
graphics programming column. Cosmic forces created the opportunity, and we
decided to go for it.
You may not have noticed, but DDJ has been putting on weight. The number of
articles is up, and by changing the typeface we're able to get more content on
each page. These factors have created the space to run another column and
still keep the article count at its historic high.
So starting next month, you'll find Jeff Duntemann here and I'll be launching
an expedition into the magical world of graphics programming. See you there.

_STRUCTURED PROGRAMMING_

by Kent Porter


[LISTING ONE]


 1 MODULE Memory;
 2
 3 (* Reports amount of memory available, excluding this program *)
 4 (* JPI TopSpeed Modula-2 *)
 5 (* K. Porter, DDJ, January '89 *)
 6
 7 IMPORT SYSTEM, IO, Lib;

 8
 9 VAR MainMem, MemUsable : LONGCARD;
 10 MemSize [0040H : 0013H] : CARDINAL;
 11 (* ---------------------------------------------------------------- *)
 12
 13 PROCEDURE PSP() : LONGCARD; (* Return byte address of PSP *)
 14
 15 VAR Reg : SYSTEM.Registers;
 16
 17 BEGIN
 18 Reg.AH := 51H; (* Undocumented: same as 62H but works in DOS 2.n *)
 19 Lib.Intr (Reg, 21H);
 20 RETURN (LONGCARD (Reg.BX) * 16);
 21 END PSP;
 22
 23 (* ---------------------------------------------------------------- *)
 24
 25 BEGIN
 26 MainMem := LONGCARD (MemSize) * 1024; (* Total memory in bytes *)
 27 MemUsable := MainMem - PSP();
 28 IO.WrStr ('Available memory is ');
 29 IO.WrLngCard (MemUsable, 1);
 30 IO.WrStr (' bytes');
 31 IO.WrLn;
 32 END Memory.






[LISTING TWO]

 1 MODULE sd;
 2
 3 (* Lists all subdirectories in the current directory *)
 4 (* JPI TopSpeed Modula-2 *)
 5 (* K. Porter, DDJ, January '89 *)
 6
 7 IMPORT FIO, IO;
 8
 9 VAR F : FIO.DirEntry;
 10 Found : BOOLEAN;
 11 Count : CARDINAL;
 12
 13 BEGIN
 14 Count := 0;
 15 Found := FIO.ReadFirstEntry ("*.*",
 16 FIO.FileAttr {FIO.directory}, F);
 17 WHILE Found DO
 18 IF FIO.directory IN F.attr THEN
 19 IF F.Name[0] # '.' THEN
 20 IO.WrStr (F.Name); IO.WrLn;
 21 INC (Count);
 22 END;
 23 END;
 24 Found := FIO.ReadNextEntry (F);
 25 END;
 26 IO.WrCard (Count, 1);

 27 IO.WrStr (' directories found');
 28 IO.WrLn;
 29 END sd.





[LISTING THREE]

 1 MODULE Where;
 2
 3 (* Searches directory structure from the root, listing all occur- *)
 4 (* rences of a filename matching the search argument *)
 5 (* JPI TopSpeed Modula-2 *)
 6 (* K. Porter, DDJ, January '89 *)
 7
 8 IMPORT FIO, IO, SYSTEM, Lib, Str;
 9 FROM FIO IMPORT FileAttr, readonly, hidden, system, directory,
 10 archive;
 11
 12 TYPE string = ARRAY [0..79] OF CHAR;
 13
 14 CONST DefaultDrive = 0;
 15 Backspace = CHR(8);
 16
 17 VAR arg, curdir, xfer : string;
 18 DriveName : ARRAY [0..3] OF CHAR;
 19 count, p, s, d : CARDINAL;
 20 curdrive, newdrive : SHORTCARD;
 21 cx [40H:50H] : SHORTCARD; (* ROM BIOS cursor col *)
 22
 23 (* ---------------------------------------------------------------- *)
 24
 25 PROCEDURE ClrSol; (* Clear from cursor to start of line *)
 26
 27 BEGIN
 28 WHILE cx # 0 DO
 29 IO.WrChar (Backspace);
 30 IO.WrChar (' ');
 31 IO.WrChar (Backspace);
 32 END;
 33 END ClrSol;
 34
 35 (* ---------------------------------------------------------------- *)
 36
 37 PROCEDURE SearchDir (Path : ARRAY OF CHAR);
 38 (* Recursive directory search routine *)
 39
 40 VAR F : FIO.DirEntry;
 41 WholePath : string;
 42 Found : BOOLEAN;
 43
 44 BEGIN
 45 FIO.ChDir (Path); (* Set directory *)
 46 FIO.GetDir (DefaultDrive, WholePath); (* Get full pathname *)
 47 Str.Concat (WholePath, DriveName, WholePath); (* Add drive *)
 48 ClrSol; (* Clear to start of line *)
 49 IO.WrStr (WholePath); (* List directory *)

 50
 51 (* Search for filename matches in this directory *)
 52 Found := FIO.ReadFirstEntry (arg, FileAttr {readonly, hidden,
 53 system, directory, archive}, F);
 54 WHILE Found DO
 55 IF Str.Length (WholePath) > 3 THEN IO.WrChar ('\') END;
 56 IO.WrStr (F.Name);
 57 IF directory IN F.attr THEN
 58 IO.WrStr (" <DIR>");
 59 END;
 60 IO.WrLn; (* New line *)
 61 IO.WrStr (WholePath);
 62 INC (count); (* Count occurrence *)
 63 Found := FIO.ReadNextEntry (F); (* Get next match *)
 64 END;
 65
 66 (* Now recursively search any subs under this directory *)
 67 Found := FIO.ReadFirstEntry ("*.*", FileAttr {directory}, F);
 68 WHILE Found DO
 69 IF (directory IN F.attr) AND (F.Name[0] # '.') THEN
 70 SearchDir (F.Name); (* Recursive call *)
 71 FIO.ChDir (WholePath); (* Restore dir to this level *)
 72 END;
 73 Found := FIO.ReadNextEntry (F); (* Do next sub *)
 74 END;
 75 END SearchDir;
 76
 77 (* ---------------------------------------------------------------- *)
 78
 79 PROCEDURE GetDrive() : SHORTCARD; (* Get currently active drive *)
 80
 81 VAR Reg : SYSTEM.Registers;
 82
 83 BEGIN
 84 Reg.AH := 19H;
 85 Lib.Intr (Reg, 21H);
 86 RETURN (Reg.AL);
 87 END GetDrive;
 88
 89 (* ---------------------------------------------------------------- *)
 90
 91 PROCEDURE SetDrive (Drive : SHORTCARD); (* Set default drive *)
 92
 93 VAR Reg : SYSTEM.Registers;
 94
 95 BEGIN
 96 Reg.AH := 0EH;
 97 Reg.DL := Drive;
 98 Lib.Intr (Reg, 21H);
 99 END SetDrive;
 100
 101 (* ---------------------------------------------------------------- *)
 102
 103 BEGIN (* Main body of WHERE *)
 104
 105 (* Initialize *)
 106 FIO.GetDir (0, curdir); (* Remember where we are *)
 107 curdrive := GetDrive(); (* and drive *)
 108 count := 0;

 109 FOR p := 0 TO 3 DO DriveName[p] := CHR(0) END;
 110
 111 (* Get the name to search for *)
 112 IF Lib.ParamCount() > 0 THEN
 113 Lib.ParamStr (arg, 1);
 114 ELSE
 115 IO.WrStr ("Filename? ");
 116 IO.RdStr (arg);
 117 END;
 118
 119 (* Select another drive, strip out designator if necessary *)
 120 IF arg[1] = ':' THEN
 121 DriveName[0] := CAP (arg[0]);
 122 newdrive := SHORTCARD (ORD (CAP (arg[0])) - ORD ('A'));
 123 SetDrive (newdrive); (* Set new drive *)
 124 IF arg[2] = '\' THEN s := 3 ELSE s := 2 END;
 125 d := 0;
 126 FOR p := s TO Str.Length (arg) DO (* Strip out drive designator *)
 127 xfer[d] := arg[p];
 128 INC (d);
 129 xfer[d] := CHR (0);
 130 END;
 131 Str.Copy (arg, xfer); (* Copy back to arg *)
 132 END;
 133
 134 (* Build name of target drive *)
 135 IF DriveName[0] = CHR(0) THEN
 136 DriveName[0] := CHR (curdrive + 65)
 137 END;
 138 Str.Concat (DriveName, DriveName, ":\");
 139
 140 (* Add wildcard prefix/suffix as necessary *)
 141 IF arg[0] = '.' THEN
 142 Str.Concat (arg, "*", arg); (* Stick in wildcard prefix *)
 143 END;
 144 IF Str.Pos (arg, ".") = MAX (CARDINAL) THEN
 145 Str.Concat (arg, arg, ".*"); (* Append wildcard suffix *)
 146 END;
 147
 148 (* Begin search at root *)
 149 SearchDir ("\");
 150
 151 (* Report matches found *)
 152 ClrSol;
 153 IO.WrCard (count, 1);
 154 IO.WrStr (" matches found");
 155 IO.WrLn;
 156
 157 (* Restore user's original environment *)
 158 SetDrive (curdrive);
 159 FIO.ChDir (curdir);
 160 END Where.










January, 1989
PROGRAMMING PARADIGMS


Paradigms Past and Future




Michael Swaine


This issue marks the beginning of the 14th year of continuous publication for
Dr. Dobb's Journal. That makes this the 13th anniversary issue, and in honor
of the occasion I offer this Janus view of the paradigms of programming, past
and future. As you know, this is just the sort of schmaltzy occasion on which,
writers feel free to indulge in the most shameless journalistic excesses:
homespun anecdotes, romanticizations of the past, prognostications on the
future, and filling space with other people's words.
"The more often a problem has been solved on a desk machine, the more certain
it is that the methods developed need reexamining before translation into a
computer programme." --R.K. Livesley, An Introduction to Automatic Digital
Computers, 1957.
R.K. Livesley, in his little book about programming the Manchester computer in
the 1950s, cautions the reader that some of our favorite desk calculator
algorithms may be ridiculously inappropriate when implemented on a computer.
His book was one of the first to discuss programming techniques. I somehow
overlooked its sound advice, and similar advice in thousands of programming
books over the succeeding decades, and had to learn the lesson the hard way.


The Runaway Algorithm: A True Story


In the mid-70s, I was a graduate student picking up some extra money writing
statistical analysis programs in Fortran for faculty members and other
graduate students. All the conventional studies lent themselves to analysis
with the standard statistical packages such as SPSS, so the jobs I was picking
up were all a little unusual.
One day a graduate student brought me a magtape full of data and a book with
an equation circled and asked me to apply the latter to the former. I set to
work implementing the algorithm that the equation implied, tested it on very
small data sets against hand calculation from the book equation, scrutinized
boundary conditions, pronounced the program ready, and turned in the tape to
be loaded.
Insufficient memory. I banged my head against "insufficient memory" messages
far too long -- probably because after seven years on campus I knew too many
tricks for "finding" more core in the university's computer system -- before I
finally did some calculations and convinced myself that the data set I was
working with was not too large for my core allotment. I could load the entire
input data set into memory, as well as all the output matrices. Between input
and output, something was eating memory like a Core Wars champion. At that
point I saw what should have been obvious from the outset: that the book
equation made use of huge sparse matrices, ballooning the data out before
collapsing it back to manageable size at output. It was simply an
inappropriate algorithm for computer implementation.
So, having learned my lesson, I reworked the equation and everything was fine
-- at least until the student's dissertation committee discovered that her
analysis was based on my unpublished (therefore suspect) version of the
equation. There was a lesson in that, too, but I'm not sure I ever really
learned it.
But that's how I learned the lesson about the need to recast the algorithm
closer to the machine's desire. When I acquired my first personal computer a
couple of years later (a TRS-80 Model with Basic in ROM and 4K of user RAM),
the first thing I wrote for it was a set of sparse-matrix collapsing routines.
Just in case I had to do some heavy statistical analysis.


Beyond Space and Time


"The Ural II was exactly like a personal computer because it was just you and
the machine.... With 4K of memory and the slow speed, it was very similar to
the Altair, which was introduced in 1974. The excitement I experienced with
the Ural II in 1964 was the same kind of excitement that Bill Gates
experienced with the Altair in 1974." --Charles Simonyi, in Programmers at
Work by Susan Lammers.
My TRS-80 experience was well into the personal computer era. You could do a
lot with 4K of memory if you had Basic in ROM. It was harder earlier. Charles
Simonyi gave himself headaches programming in octal absolute on that Ural II.
And who, besides a hacker like Bill Gates, would have attempted to write a
Basic interpreter for the Altair back in 1975? A bunch of hackers, that's who.
Toward the close of 1975, when the typical personal computer programmer was
someone who had a friend across town with an Altair, the San Francisco Bay
Area-based People's Computer Company had a runaway project on its hands. It
wasn't an unusual situation; the group was filled with creative, energetic,
intelligent people who were fired up by the possibilities of computers for
empowering individuals, and had lots of projects to pursue. The group was,
after all, an offshoot of the same group that produced the Whole Earth
Catalog, subtitled Access to Tools.
The runaway project was a portable Basic interpreter that would run in the
remarkably small memory space of the Altair, or on one of the other
micro-computers then being designed. It was the famous Tiny BASIC, brainchild
and bandwagon of Dennis Allison and Bob Albrecht, and raison detre of this
magazine in its early days.
Those early days began in January 1976, and the magazine was originally titled
Dr. Dobb's Journal of Tiny BASIC Calisthenics and Orthodontia (Running Light
without Overbyte). The title said it all (a title that long had better say it
all): memory is tight, and code must be equally tight. Dr. Dobb's readers and
writers (then, as now, overlapping sets) were spending a lot of time recasting
their algorithms closer to the machine's desire. And what the machine desired
was tight code, memory-frugal algorithms. It was a time for running light
without overbyte.
It's always that time. Dr. Dobb's subhead could still be Running Light without
Overbyte. It's a paradoxical fact of programming life that memory space
remains precious, efficiency remains valuable, no matter how much RAM you
have. It's one of the two fundamental resources: The truism among programmers
is that the only commodities are time and space. Cycles and core. Ticks and
bits. And in some times and in some places, the truism is the only relevant
truth.


Environmental Action


Elsewhen and -where, though, it becomes appropriate to pay more attention to
the human commodity: The programmer. Rather than focusing exclusively on the
program and efficiencies with regard to its operation, to look at overall
effectiveness of the programming effort. To allocate programming effort
economically. It's not a new idea, of course.
I'm always pitching the importance of knowing many paradigms. That's the
subtext (at least) of this column. I'm not alone in promoting this. One reason
often put forth is that the separate streams of existing paradigms are
currently converging. It's not entirely clear whether the streams are
converging, though some, say, C. The truth is probably not so simple. In fact,
it may, to muddy the metaphor, be more coagulation than convergence.
In any case, things are getting clumpier. Nobody wants to sell you a compiler
anymore. What you need, you are told, is an environment. Well, you probably
do; who am I to scoff. But "environments" are more complicated than compilers,
and one of the ways in which they show signs of getting further complicated is
in terms of support for different paradigms, and that's my beat.
The future in C programming environments, for example, seems to include a C++
front end. C programmers can use C++ to work in an object-oriented paradigm
without quite leaving C, and while there may be arguments about whether C++ is
fully object-oriented, it certainly brings in non-procedural programming
considerations: Moving from C to C++ requires a paradigm shift.
Two products that have come across my desk recently, neither of which is a
programming language, but both of which include programming languages, seem
particularly catholic in their embracing of alternative paradigms. These are
Mathematica from Wolfram Research in Champaign, Ill. and HyperBase from Cogent
Software in Framingham, Mass.
Mathematica is an environment for doing mathematics. It is designed to run on
many machines, notably the Sun, Macintosh, and NeXT machines. It handles the
basically arithmetic calculations that spreadsheets know how to do, but it
also solves algebraic and calculus problems. You can integrate a function
symbolically or numerically. It supports arbitrary precision math. And it
embodies, according to its creator, Stephen Wolfram, several different
programming paradigms.
As a procedural language, Mathematica supplies the syntactic elements if,
while, and for to direct flow of control, as well as the dreaded goto. But the
language is more effectively used as a functional language like Lisp or APL.
The thought processes that go on when building a functional program are
different from those behind procedural development. Functional programming
makes little use of control structures or named variables. In fact,
Mathematica encourages eliminating names even for functions. Functional
programs can look extremely concise:
exprod[n_Integer] := Expand[ Product[x + i, {i, n}] ].
Possibly the most powerful paradigm supported by Mathematica, though, is
rule-based programming. This paradigm allows the programmer to build a
database of rules like these mathematical rules for differentiation:
 diff(x^n,x) = nx^(n-1) diff(log(x),x) = 1/x.
This technique brings with it a distinctly nonprocedural approach to
programming. It's easy, using this approach, to develop a program by a process
of successive approximations to the ultimate program. You simply add rules to
deal with additional cases.
An even more disorienting paradigm, from the procedural point of view, is the
approach of constraint propagation. In this approach (supported by
Mathematica) you specify constraints that variable must satisfy, but don't
tell the program how to satisfy them. Constraints like
 va = = vo r1 / (r1 + r2)
 va = = vi
 g = = vo / vi
make no assumptions about the direction of the relationships they define. The
solve function does its best to pull a solution out of the constraints. This
approach has similarities with using a spreadsheet, but with fewer,
constraints on the form of the constraints. Spreadsheets in effect implement
linear constraints, but Mathematica lets you define non-linear dependencies
among variables.
Both the rule-based and constraint propagation approaches sound like logic
programming a la Prolog, and in fact Mathematica supports something like the
logic programming paradigm. It differs, though, in being more directed, more
explicit, in the order in which it tries its rules.

Another consequence of Mathematica's aggressive ordering of its rules shows up
in its implementation of recursion. A recursive solution to a problem always
consists of at least one recursive (self-invoking) condition and at least one
non-recursive, terminal condition. Mathematica lets you drop these conditions
in as independent rules, like
 factorial[n_Integer] := n factorial[n-1]
 factorial[1] = 1,
without worrying about the order in which the rules should be applied.
Mathematica understands enough about recursion to recognize, usually, the
terminal condition and to apply it first.
HyperBase is a system for creating, what its author calls, intelligent
documents. It runs on MS-DOS machines. Intelligent documents are text and
graphics documents with attached code. The code is associated with elements of
the text or areas of the graphic images, and when the code effects jumps to
other nodes in the document, the system can be described as an implementation
of HyperText. But the code can be pretty much anything, including commands
that modify the text and graphics, in which case the system can be described
as an adaptive document. The underlying language is a full Prolog system.
The result is a system that feels on the surface something like a word
processor, and something like HyperCard or Owl International's Guide, while
digging deeper into the product takes you squarely into logic programming.
That's got to be a paradigmatic jolt.
























































January, 1989
OF INTEREST







Software


BrainMaker, a neural network simulation program, has been introduced by
California Scientific Software. BrainMaker is a system for designing,
building, training, testing, and running neural networks.
Using BrainMaker, users can build networks that do optical character
recognition, speech synthesis, and control systems applications. Programmers
can also use BrainMaker to investigate research topics such as speech
recognition and artificial vision.
BrainMaker runs at up to 500,000 neural connections per second and has I/O
facilities for visual or symbolic data manipulation. Menu-driven with color
and mouse support, BrainMaker comes with sample neural networks including
optical character recognition, speech synthesis, image recognition, image
enhancement, and Tic-Tac-Toe.
This product, which sells for $99.95, requires an IBM PC or compatible with
256K memory, a monochrome or color display, and PC-DOS or MS-DOS 3.0 or later.
Reader Service No. 20.
California Scientific Software 160 E Montecito Ave., #E Sierra Madre, CA 91024
818-355-1094
Cognitive Software has released two products: Cognitron, Version 1.1, a neural
network development tool; and Cognitron Prime 1.0 for use with INMOS T800
transputers. These products are development environments for creating networks
of processing units and connections. According to Cognitive Software, neural
networks developed on the Cognitron can solve problems such as forms of
diagnosis, problems in economics, project planning, best choice, financial
analysis, and forecasting.
Features of these products include simulated parallel processing, graphical
3-D modeling window with networking tools, editing window for programming the
processing units and weights between units, supporting windows and dialogues,
Lisp subset for use in programming units and weights, graphical user
interface, and simulation engine.
The Cognitron 1.1 is designed to run on the Macintosh Plus, Macintosh SE, and
Macintosh II. Cognitron 1.1 sells for $600, and Cognitron Prime 1.0 is priced
at $1,800. Educational discounts and group pricing are available. Reader
Service No. 21.
Cognitive Software Inc. 703 E 30th St., Ste. 7 Indianapolis, IN 46205
317-924-9988
Software Artistry has released PC Expert, a system development environment for
Microsoft C, Turbo C, JPI Modula-2, and Logitech Modula-2 compilers. The name
of the product has been changed from Turbo Expert, the original version that
supported only Turbo Pascal 4.0. PC Expert consists of a library module
containing an inference engine that can be embedded in applications developed
with one of the supported compilers.
According to Software Artistry, the inference engine takes up less than 64K of
code space yet offers features such as backward chaining, demons, agendas,
non-monotonic reasoning, tracing, data and time arithmetic, and communication
between the inference engine and the other parts of the program in which it is
embedded. Facilities are included to interrupt a consultation when a value is
needed, return control to the host program to obtain that value, and then
resume the consultation.
Software Artistry also offers a graphics package that can be used to create a
graphical front-end for expert systems or other projects. The graphics package
includes a memory-resident program that can be used to capture images created
with commercial paint programs and then display them during a consultation.
The package also includes routines to animate images, display graphical gauges
and dials, and add on mouse support.
PC Expert and the add on graphics package are $99.95 each. Reader Service No.
22.
Software Artistry 3500 DePauw Blvd., Ste. 2021 Indianapolis, IN 46268
317-876-3042


Utilities


C-DOC, a collection of tools that analyze and document C programs, has been
introduced by Software Blacksmiths. C-DOC modifies programs to include both
caller/called hierarchy documentation and global/local/parameter identifier
documentation in each procedure header.
The product also displays caller/ called hierarchy tree diagrams showing
interaction of procedures, produces table of contents files versus procedures,
documents both procedure and system usage of local/global identifiers,
produces both summary and detailed documentation, modifies programs (adjusts
indentation) to reflect their logic/control structure (and optionally adjusts
comment alignment), and lists programs with optional graphic action diagrams
of logic structure.
C-DOC requires an IBM PC or compatible; PC/MS-DOS, Version 2.x or 3.x, 256K
memory minimum (640K recommended); and independent of the C compiler used.
C-DOC, which includes four programs, sells for $99. Reader Service No. 23.
Software Blacksmiths Inc. 6064 St. Ives Way Mississauga, Ont. Canada L5N 4Ml
416-858-4466
Edward K. Ream has begun shipping Sherlock, a productivity tool that works to
trace, debug, and profile C programs. Sherlock consists of C language macros
and support routines called by these macros. The macros are inserted with a
separate tool (included with Sherlock), and the macros contain tracing and
debugging code that lies dormant until enabled during the execution of the
program.
Sherlock can be used during coding, testing, and maintenance phases of
programming and will work with Microsoft, Turbo C, or any full C compiler.
Programs containing Sherlock macros may also be debugged using traditional
debuggers. Sherlock can also be used with C++ compilers that produce C
language output.
Sherlock, which sells for $195, counts how many times each macro is
encountered and measures the time spent in functions. It comes with three
utility programs: The SPP program inserts macros into a file, the SDEL program
removes all Sherlock macros from a file, and the SDIF compares a file
containing Sherlock macros to a file without them. Reader Service No. 24.
Edward K. Ream 1617 Monroe St. Madison, WI 53711 800-922-4763
SLR Systems has added a linker and debugger to the OPT group of development
tools. OPTLINK is a program linker that works with modules in Intel OMF format
(.OBJ files), including those generated by the standard SLR IBM as well as
Microsoft assemblers and compilers. OPTDEBUG is a source level debugger that
provides the programmer with eight display types. These two products are
priced at $125 each.
Another new product is OPTLIB/2, an object module librarian that can create
and maintain master indexed files or libraries in the OS/2 environment.
OPTLINK/R works in the protected mode of OS/2 and generates real mode .EXE
files.
SLR Systems has also announced an update to its macro assembler, OPTASM. The
new version, 1.6, adds two new utilities: OLINK and ODEBUG. OPTASM, Version
1.6, sells for $125. Reader Service No. 25.
SLR Systems 1622 N Main St. Butler, PA 16001 412-282-0864


Neural Nets


The BRAIN simulator, announced by Abbot, Foster & Hauserman for IBM PCs,
simulates a neural circuit array on MS-DOS. By emulating a small section of a
human brain, the program gives users an understanding of how neuron circuitry
works.
Included with the program are several examples of the types of circuitry that
can be created with the neural model, including simple switching, memory, and
pattern-recognition circuitry. An accompanying booklet explains the operation
of biological neurons and describes how they are emulated by the program.
The BRAIN Simulator implements an array of 1,200 neurons using a digital model
of the human brain neuron developed by the Neural Network Laboratory. A
network of neurons is represented on the CRT; when a neuron fires, its CRT
location flashes.
Users can modify the sample networks or define new ones. Using a mouse to
indicate a particular neuron, users can display the existing synaptic
connections and add or delete new ones. Neurons can be forced to fire the
established initial conditions or emulate the input of a visual image. This
software is available for $99. Reader Service No. 26.
Abbot, Foster & Hauserman Co. 44 Montgomery St., 5th Floor San Francisco, CA
94101 415-955-2711 800-562-0025


Miscellaneous



Turbo Plus 5.5 for Turbo Pascal 5.0 is now available from Nostradamus. Turbo
Plus is a professional development package for Turbo Pascal programmers. One
of its new features is display mapping, which allows the user to create entry
screens with multiple fields using direct memory-mapping techniques.
The display-mapping procedures include features for I/O control; edit masks,
field highlighting with color bars, and insert/overwrite modes; delete key;
arrow keys; and home and end keys. I/0 mapping supports all Turbo Pascal data
types including extended floating-point numbers. Numeric I/0 fields can be
handled as decimal, hexadecimal, or scientific notation format. Version 5.5
also has display map user intervention routines that gain control on every key
pressed for user control of exit keys, cursor positioning, keyboard
redirection, and keyboard status.
Screen Genie, the screen painter included with Turbo Plus, has been upgraded
to include support for screen, window, and menu libraries. Library support is
also available to programmers as part of the programming tools.
Turbo Genie --the included source code generator for compiling screens,
windows, and menus into the .EXE file--has been upgraded to support libraries.
In addition, painted screens, windows, and menus can now be used in bit mapped
graphics CGA, EGA, and VGA modes, as well as text modes.
Over 70 new routines have been added to the Turbo Plus package, including new
graphics routines for saving and loading .PCX and .PCC files. Turbo Plus 5.5
sells for $149.95; current users of Turbo Plus 5.0 can upgrade for $50. Reader
Service No. 27.
Nostradamus Inc. 3191 S Valley St., Ste. 252 Salt Lake City, UT 84109
801-487-9662
Logical Systems has released the Transputer Toolset, which provides a C and
assembly language development environment for the INMOS transputer family. It
works with a single transputer or a transputer network.
The toolset provides optimization facilities and conformance to the ANSI C
standard. In addition to supporting features specific to the various models of
transputers, the product is provided in source code format.
The toolset facilitates C code generation for programmers developing parallel
and embedded system applications. The transputer, a 32-bit parallel processing
chip, has been implemented in products such as image systems, computer
graphics, industrial control, robotics, workstation and PC accelerator boards,
and supercomputers.
The toolset includes a C compiler, assembler, linker, librarian, and both a
single processor and network loader. The compiler supports in-line assembly
language and optionally generates in-line code for the C functions that map
into the transputer instruction set. The compiler can generate code for the
64-bit/32-bit ANSI floating point model or 32-bit only floating-point (or a
mixture).
The toolset is portable across MS- DOS, Apple Mac II (under both MPW and LSC),
and SYS5/BSD4.3 Unix systems, including Apollo, Sun, and DEC. The C library
contains both transputer runtime routines and host interface facilities. Host
I/O is provided by three standard servers, which handle file and system I/O.
The $995 price includes six months of support and updates. Reader Service No.
29.
Logical Systems P.O. Box 1702 Corvallis, OR 97339 503-753-9051
Cobalt Blue has announced the release of a Xenix version of FOR_C, a Fortran
to C translator running under MS-DOS. FOR_C converts standard Fortran-77 into
ANSI C, and MILSPEC extensions are supported.
FOR_C, Version 2.1, incorporates improvements to I/O translations; for
example, list directed WRITEs are translated to fprintfs and unformatted I/0
is translated to fprtunf and fscanunf, unformatted equivalents to their C
counterparts. Do loops are reduced, and function arguments are checked for
consistent usage and can be passed either by value or by address.
With C runtime source, FOR_C is priced at $750; with binary runtime (for Turbo
C or Microsoft C), the price is $550. The new Xenix version, with runtime
source for both ANSI C and Unix System V, is priced at $975. Both packages
include six months technical support and free upgrades. Reader Service No. 30.
Cobalt Blue 2940 Union Ave., Ste. C San Jose, CA 95124 408-723-0474
The release of REM86, a remote target debugger for the Future86 language, has
been announced by Development Associates. The REM86 extension completes the
Future86 development environment, and it works in both hosted or targeted
environments for 8Ox8x CPU and related CPU families.
REM86 consists of a target monitor (written in Future86) that is less than 2K
bytes. The remaining part of REM86 is the host control program that is
functionally similar (but extended) to FDT86, the Future86 host debugger.
This product supports a patching assembler, file loading, disassembler,
immediate execution assembler, high-level definition creation/execution,
single stepping, breakpoints, and symbolic debugging and control.
REM86 is supplied with sample monitor driver source code for both the NEC V25
DDK and the Vesta Technologies Micro 88 boards. Also included is sample
application source code. Selling for $479, REM86 requires Future86. Reader
Service No. 31.
Development Associates 1520 S Lyon St. Santa Ana, CA 92705 714-835-9512
Apex Software has released the Apex Database Library (ADL), a library of
functions enabling C programmers to create, access, and update dBase files.
ADL features include the ability to access the database at either the field or
record level, automatic updating of index files, and a tunable caching
algorithm.
With ADL, programmers can retrieve one field from the database record or
access the record reformatted into a standard C structure. ADL includes a
dBase expression parser for index file support, which allows automatic access
to database records based on the key expression of the current index file
(with a maximum of nine index files open per database). If database records
are updates, the parser evaluates and updates the key expression for all open
index files.
ADL's extension parser is provided in three different versions. Support is
provided for Microsoft C compilers, Version 4.0 and later, and Turbo C
compilers, Version 1.5 and later. System requirements are PC, AT, PS/2, or
compatible. The system also must have a minimum of 128K RAM and MS-DOS or
PC-DOS, Version 2.0 or later. A hard disk is recommended.
ADL is priced at $395 without source code or $795 with source code. Prices
include support and updates for 90 days. Reader Service No. 32.
Apex Software Corp. 4516 Henry St., Ste. 308 Pittsburgh, PA 15213-3785
412-681-4343
SOFTGRAF, Version 5.0, has been released by Sunrise-Littleton. SOFTGRAF is a
menu-driven TSR software driver that "fools" a monochrome PC, allowing it to
run CGA color graphics programs and games, without use of a color card or
monitor. It does this by setting the system's flags for a CGA environment,
intercepting graphics signals sent to the video buffer, and translating video
formats so that the monochrome pixel coordinates that of a CGA color system.
Colors appear as tones of amber or green.
SOFTGRAF operates from its own startup menu. Other software features include
running programs in large 40 character text mode, an AFC feature, and the
ability for the user to adjust the screen refresh rate interval from 16 to
1,024 times per second.
The upgraded version commits about 12K of RAM, and it senses to determine if
color graphics commands are being sent to the video buffer. If a
color-oriented program is detected, SOFTGRAF switches to CGA resolution and
then reverts back to the higher Hercules monochrome resolution as soon as the
application requiring color is terminated.
SOFTGRAF supports most commercial and public domain software and operates in
Hercules compatible or EGA environments. Systems such as a Tandy 1000 and
leading edge models are also supported.
Not copy protected, SOFTGRAF sells for $34.95. Reader Service No. 33.
Sunrise-Littleton Technology Corp. 15200 Shady Grove Rd. Rockville, MD 20850
301-963-0341





























January, 1989
SWAINE'S FLAMES


The Return of Insane Greatness







I went to the NeXT machine introduction. Read/write optical drive. Megapixel
display. Mathematica. The compete works of William Shakespeare. I want that
machine. It's so insanely great.
It's so insanely great that I don't care that there seems to be no way for a
software developer to make a buck on it. It's so insanely great that I don't
care that it's only going to be sold to higher education. It's so insanely
great that I don't care that there seems to be no way for a writer to make a
... Well, maybe it's not so great.


The Foot Metaphor


Steve Jobs insists that the NeXT machine will be sold, read his lips, Only to
Higher Education. Right. No new taxes.
Hard on the heels of NeXT's proof-of-concept media event comes a packet of
press releases from Microsoft announcing its support for higher education. Is
there a connection? Who knows, but speaking of being hard on the well heeled,
Bill Gates has been galling the Jobs kibe lately by bad-mouthing the NeXT
machine. Howcum?
My friend Thom Hogan thinks Bill's just annoyed that he can't get any
legitimate press out of the NeXT announcement. Thom has been immersed in Mac
technology since March, when he launched the Macintosh II Report(which I help
him with). He doesn't think there's anything in the NeXT machine that Apple
shouldn't be able to put into the Tower in a year or so. But he also admits,
after doing some research into the matter, that he doesn't see how NeXT can
make a decent profit selling the machine for $6,500, no matter how good a deal
it made on memory. Something's funny there.
Jobs' insistence that his only market is higher education ("Fortune 500
companies in disguise") makes one curious about the language of his
arrangement with Apple. I am very impressed with the machine. But I wonder
about the company. What else does it have to sell, and to whom, and how?
What's the rest of the NeXT strategy? Does Steve Jobs have another shoe to
drop? And if so, on whom?


Roget's Paradigm


Proponents of the object-oriented programming paradigm defend it as lucid and
realistic. It's lucid, they claim, in that it packages information neatly and
allows for suppression of detail. Objects encapsulate information clearly.
It's realistic, they claim, in that it represents the real world better than
other paradigms do. Developing OOP objects is really modeling real-world
objects and their interactions. Lucid and realistic.
Yes, but ... The sticky point comes when you consider multiple inheritance. No
one since Roget has believed that real-world objects can be gathered into a
single, all-encompassing hierarchy. The real world is saddled with real
multiple inheritance. The keyboard I'm pounding on as I write this belongs to
the class of tools and to the class of possessions of mine, and neither class
encompasses the other. And doesn't multiple inheritance blow away the vaunted
lucidity of object-oriented programming? Sure seems like it.
It begins to look like object-oriented programming can be lucid or realistic,
but not both. No sex, please, we're software engineers.
Nothing is lost that can be reinvented. Forty years ago, engineers knew how to
decompose a large problem into small tasks to be handled by separate
processors. The processors weren't transputer chips, to be sure. Technology
advances.
So does language. Back then, a computer was a human being. Rows of WACs, or in
Britain WRENS, each punching keys on a desk calculator, each computing her
tiny task of some large computing problem.
It's strange to reflect that electronic digital computers were invented to
replace women. Stranger still to realize that, for some (male) programmers,
the replacement seems to have succeeded beyond expectations. If this paragraph
has nothing to do with you, please take no offense. But if you recognize
yourself in this picture, take a break. Log off. Shut down. Talk to people.


Personal to Two Valley Boys


Stan Kelly-Bootle: OK, so I'm overusing the P word. But that's my job, Stan;
it's in my contract. It's clause 3b: "The party of the first part shall use
the word [P word deleted] at every opportunity."
Jeff Duntemann: Rumor is you're going to be writing for the premier
programmer's magazine. I think that's wonderful. Don't let them put any funny
stuff in your contract.




















February, 1989
February, 1989
EDITORIAL


Jonathan Erickson


For reasons that glimpse back to the past, while still looking ahead to the
future, this issue of Dr. Dobb's Journal is especially significant for us. As
you've probably already noticed, with this issue we've launched a new logo,
one that emphasizes the name Dr. Dobb's Journal and puts the descriptive
tagline "Software Tools for the Professional Programmer" back in what I
consider its proper (less dominant) perspective. Our reason for doing this is
quite straightforward: When most people, including those of us who put the
magazine together every month, think of this publication, it's DDJ, Dr.
Dobb's, or simply the Journal that comes to mind.
There's a lot of recognition, historical and otherwise, that goes with the
name Dr. Dobb's Journal and, to borrow from a recent article in American
Heritage magazine that acknowledged the 100th anniversary of the Wall Street
Journal, "I am pleased to report that the Journal's editors ... awakened to
the value of preserving an institutional memory." That's the way we feel about
DDJ--the magazine and its name.
Part of the history of DDJ revolves around some of the genuinely significant
articles we've published over the years. (Articles like "Tiny Basic" and
"Fattening Your Mac," for instance.) This month, we're continuing that
tradition with what I think will be one of the more important articles we'll
publish this year. Our lead feature, cowritten by DDJ s senior technical
editor Kent Porter and Intel engineer Robin Kar, describes a specification for
measuring real-time performance that we call the "Rhealstone." Over the past
few months, Kent's been working with key people across the real-time
industry--silicon vendors, software developers, and system designers -- to
generate the Rhealstone. As part of this process, Kent chaired a panel that
included Robin Kar, Ray Duncan (from Laboratory Microsystems Inc.), and Dan
Erickson (of Digital Research Inc.) at the Real-time Programming conference
where the Rhealstone was initially introduced. As Kent mentions in the
article, we're now presenting it to you and asking for your ideas, questions,
and comments. We'll then publish the final proposal, including code examples,
in this year's June issue.
Kent is also central to another of this month's developments. For the past
couple of years, Kent has been writing our "Structured Programming" column,
and he's recently become known in some circles as a Modula-2 evangelist.
Beginning this month, however, Kent is starting a new column, which is devoted
to high-performance graphics programming.
Our new "Structured Programming" columnist is Jeff Duntemann, no doubt
familiar to many of you as the former editor of Turbo Technix, a magazine that
until recently was published by Borland. Turbo Technix was a first-rate
magazine and Jeff deserves most of the credit for its excellence. Sad to say,
Borland discontinued publication of Turbo Technix a few months ago, which made
it possible for Jeff to join the DDJ family as a contributing editor and
columnist. While lamenting the demise of Turbo Technix, I'm equally pleased to
welcome Mike Floyd, a new technical editor for DDJ. Mike, too, comes from
Turbo Technix and he's already proven to be a valuable addition to our staff.
The long and short of all this is that DDJ is continuing its commitment to
publish the kind of information that, for the past 12 years, has made it the
magazine for serious programmers, and it's our intent for DDJ to remain so for
many years to come.















































February, 1989
LETTERS







Pessimistic Portrayal in Programming Paradigms


Dear DDJ,
I was intrigued by your [Mike Swaine's] worries about the U.S. software
industry as expressed in "Programming Paradigms" (August 1988). I believe you
are unduly pessimistic.
About 10 years ago, I learned that the British arm of Burroughs, as it was
then, delegated a substantial part of its programming to India. There is the
benefit that English is a common language, not only between Indians and
ourselves, but also within India where there are many other languages besides
the majority Hindi. There too is a tradition of scientific and technical
studies.
The point is this: I do not think the U.S. computer industry's use of Indian
labor was a very great secret -- at the time I was not working for an
especially favored customer -- and certainly would not be now. Yet, there is
no evidence of other firms following the Burroughs example; I don't even know
if Unisys continues the practice. There are great shortages of expertise
across a wide range of ADP applications. Salaries are sky high, and even
middle-aged programmers like me can still make a living. Closer to home is the
Republic of Ireland, dedicated to giving technical skills to one of the
youngest populations in Europe. Some recruitment into the U.K. has occurred,
but the great off-shore software industry just has not appeared.
I think that employers will always want people carrying out critical work to
be on-site. It is significant that Burroughs' Indian programmers were doing
largely program conversion -- work that had to be carried out to a high
standard of accuracy and to a tight deadline, but routine and not requiring a
high degree of interaction.
Just as people in business do not like paying for anything that they cannot
drop on their foot, they do not like employing people who are not there to
kiss the same. Cheap joke, but you know what I mean. They feel uncomfortable
without physical evidence of payroll.
An interesting manifestation of a similar phenomenon occurred here recently.
You must know that we had a postal strike. I, and probably many other
technically aware people, expected a large increase in traffic on Telecom Gold
and smaller communications networks. Not at all. The real boom was in
facsimile machines. Even now, none of the large computer recruitment agencies
I use has a communications link, in spite of the fact that the capital and
running costs are lower than fax. They all have fax. The physical appearance
of a document, even a poor copy, is more important than the content.
Many of us are trying to persuade employers in areas where recruitment is
difficult to employ us as "telecommuters." We work from home, don't have to
travel long distances (in U.K. terms), and need not be distracted by office
politics. In return, the employers pay substantially less than their local
going rate and do not have to provide accommodation, furniture, or equipment
apart from a terminal and a communications link. However, the concept has not
spread beyond a core of specialized areas.
You may ask why I am sending you a letter, rather than using electronic mail.
I would reply that I have had to move my computers to another part of the
house, it is too much trouble to fiddle around with cables and modems at the
moment, that it is more trouble to look up your BBS identity than your
business address...but I don't convince even myself. Atavism rules.
Frank Little
Clydach, Swansea


Blind to His Love's Faults?


Dear DDJ,
After some thought, I finally decided to write about a key issue left unspoken
in most discussion of "the new Basics." Bruce Tonkin's review of the Microsoft
and Borland offerings of Quick and Turbo Basic ("Inserting Elements into a
Basic Integer Array", November 1988) is a good case in point. After reading
pages of benchmarks and comments about the quality of editors and so on, I
still wondered "how large is the data segment?" In a program written last year
in QuickBasic 3.0, I ran up against the 64K data limit -- a limit of the
medium memory model. I wondered if this had been changed in QB4, the new Basic
compiler or in Turbo Basic. A re-reading of the article provided no further
clues.
A call to Microsoft straightened me out. QB4 and BASCOM, Version 6, continue
to use the medium memory model under DOS, even in the OS/2 world!
Of course, there are workarounds for the new Basics. Microsoft offers large
numeric arrays, which are a help if your data is numeric and if you don't mind
the execution speed penalty over using the heap in other languages. (Kent
Porter suggests this works out to about a 500 percent penalty.) Although
addressing limitations of the 8088/80x86 requires workarounds in Pascal and C,
they exact a more serious penalty on Basic. For many applications, this
limitation essentially prevents a new Basic from being considered the
platform. It is important for this information to be given and put in context
in a review aimed at software developers.
I always read Bruce Tonkin's articles and enjoy his often eloquent defense of
Basic's honor in an apparently hostile professional world. I am beginning to
wonder, though, if he is blind to his love's faults.
Jay van Santen
Topeka, Kansas
Bruce replies:
I think I'd better confess that I'm confused by Mr. van Santen's letter. I did
know that Microsoft used the medium memory model for both Basic 6.0 and
QuickBasic 4.0. Microsoft could have used the huge memory model, but that
would have made for slower data access and a larger amount of generated code
in most programs.
If Jay meant "Microsoft should have allowed a choice of memory model," I
agree, but it would have meant a more complex compiler and perhaps a less-easy
and less-general interface to other languages.
Large arrays are not limited to numeric data; both QB4 and Basic 6 permit
arrays of static strings and user-defined types of any size, limited by
memory. If he'd like variable-length string arrays larger than a single
segment, I agree. I've been asking for that for years.
I don't think he meant code size is limited. The medium-memory model allows
code up to a megabyte.
There is a bug with arrays of static strings and user-defined types for QB 4
and Basic 6.0. If the element lengths are not a power of two, the maximum
memory available to the array is about 128K.
Though data on the heap can be accessed faster, I've found Basic's string
functions faster, more flexible, and easier to use than those of Pascal or C.
Often, that more than makes up for any theoretical losses. For numeric arrays,
please note: The alternate math package in Basic 6.0 is the same one used by
the Microsoft C compiler.
Don't overlook the fact that large arrays are easy in QB or Basic 6.0, and not
always so in C or Pascal. If you have to do a little kludging, it seems to me
that you'd lose at least some performance and (more important) code clarity.
I'd never concede that QB or Turbo Basic is a poor development platform
compared to Pascal. There are a few jobs where C is better than Basic, and
jobs where assembler is better than either. Where Pascal might be better, I've
always found that C is better still. If time permits, assembler is always the
best choice in those same cases.
If I wrote different software, maybe I'd use C. I've had to write few device
drivers or systems software. Instead, I've been writing database management
tools, a word processor, and business programs. For those, Basic is and has
been a perfect choice. The medium memory model has been no constraint to me or
anyone I know.
I find it impossible to believe any of the applications mentioned could have
been more quickly written or debugged in C or Pascal. The small parts where
speed proved most critical were identified and then coded in assembler,
anyway, so I doubt the C version would show any better performance.


Mohr's Flames


Dear DDJ,
I have been a subscriber for years, and I like the magazine. I am a little put
off, though, by the endless PC stuff. I was forced to use one at work and
found it and its software abysmal. Aztec C had a number of errors in it, and
even though I was supposed to get updates of bug fixes, none ever came.
Flame One: Why is the programmer supposed to pick the large or small,
code/data model? Is this not a job suitable for the compiler and computer to
figure out? The entire Intel line beyond the 8080 seems to conglomerate the
problems instead of solve them. Maybe I am just stupid, but Motorola's 68000
series is much better designed.
Flame Two: I responded to an advertisement for the C club in Kansas, which had
lots of public domain (so they say) software. Well, that is a fallacy; it is
public domain as long as you own the BDS C compiler and library. Somehow I do
not think that is what public domain means. Am I naive or stupid in this
matter?
Flame Three: I have used a simple but powerful editor on a DEC 11/20, which is
a 64K machine called TECO, Version 28. It would (and could) do more than any
other editor I've seen for small (64K) or even ten times bigger (640K) IBM
PCs. EMACs, however, is an exception. Why is there no TECO for CPM machines?
We seem to have turned to a swamp of fancy junk editors rather than
maintaining a simple powerful editor like TECO.
End of Flames: I close with the wish that IBM and its clones drown in the
swamp of their own making, quietly and without fuss. Intel has pioneered the
lowest common denominator of computing hardware, and IBM, by farming it out,
has indeed pioneered the same in software for its machines -- a software
environment where wild cards sometimes work, sometimes not. Nothing is
consistent there. I have spent a lot of time using Unix and C, and the
comparison to the Intel/IBM systems show the latter to be sorely lacking in
both hardware and software. I would hope that Dr. Dobb's would lead the way
instead of just playing in the swamp. I eagerly await my NEXT computer, as it
seems like the first real step up from CPM.
Douglas Mohr

Boulder, Colo.





























































February, 1989
 RHEALSTONE: A REAL-TIME BENCHMARKING PROPOSAL


An independently verifiable metric for complex multitaskers




Rabindra P. Kar and Kent Porter


Rabindra P. Kar is a senior engineer with the Intel Systems Group in
Hillsboro, Oregon, Kent Porter is senior technical editor for DDJ. Kent can be
reached through CompuServe at 76704.51 or through MCI: KPORTER


Despite the growing importance of real-time systems, the industry lacks any
meaningful, objective way to measure real-time performance. We use Whetstones
and Dhrystones to benchmark the code generated by compilers and or the
throughput of hardware platforms, but to date there has been no equivalent
objective measure for real-time systems. In an effort to plug the gap, this
article proposes a standard methodology for objectively measuring real-time
performance and summarizing the components of performance in a figure of merit
called Rhealstones.
The Rhealstone metric chiefly helps developers select real-time computer
systems appropriate for their applications. A "real-time computer system" is
any marriage of hardware and systems software -- operating system, kernel,
executive, or these elements of systems software in combination -- that forms
a platform for a real-time application. A "real-time application" is a
limited-purpose computerized system that responds to defined circumstances as
they arise in time to influence subsequent events. An example is an aircraft
autopilot, which detects and corrects deviations from the plane's intended
flight profile.
A complete real-time solution consists of the computer system plus the
application software plus external devices. Rhealstones won't measure how good
the complete solution is, and thus they may not be an appropriate measure for
end users of real-time systems. Instead, Rhealstones are an engineering
measurement targeted specifically toward true multitasking solutions.
A multitasking system is one in which more than two tasks having different
priority levels run concurrently. By this definition, then, Rhealstones do not
apply to synchronous polling solutions, in which all tasks have equal priority
and are granted equal time and access to resources if active. Rhealstones
apply chiefly to complex systems running five to thirty concurrent processes.
The derivation of the name "Rhealstone" is obvious and -- to those concerned
with performance measurement -- so is the application of the result. The
similarities to Whetstone and Dhrystone end there.
The Whetstone and Dhrystone benchmarks are synthetic programs that try to
achieve a statistically balanced set of operations reflecting an "average"
workload: Whetstone for floating-point applications and Dhrystone for systems
programs such as compilers and operating system utilities. The benchmark
programs loop some number of times, and this number divided by the elapsed
time in seconds yields the figure of merit: the bigger, the better. The result
serves as a predictor of a given platform's performance given the "average"
job. Though useful, the Whetstone and Dhrystone metrics reveal nothing about
the relative performances of the various operations that contribute to the
single-figure result --and they have no applicability to real-time systems.
The Rhealstone measure, on the other hand, proceeds from the observation that
every real-time application is unique. One system may be highly
interrupt-driven, another rely heavily on message-passing among tasks, and
still another deal with contention for resources, and so on. Real-time systems
are almost always multitasking, with one task or another capturing the
attention of the processor based upon conditions within the specific
application's limited universe. The processor relies on some sort of real-time
systems software to help it do its job. Moreover, real-time systems react to
circumstances and by definition are endless programs that stop only when the
plug is pulled. All this makes the notion of statistical balance irrelevant.
The Rhealstone figure is consequently a sum obtained from six categories of
activity most crucial to the performance of real-time systems, irrespective of
the actual application. The categories, or components, of the Rhealstone sum
are:
Task switching time
Preemption time
Interrupt latency time
Semaphore shuffling time
Deadlock breaking time
Datagram throughput time
Using coefficients we'll discuss later, the system engineer can assign each
Rhealstone component a weight that reflects its relative importance in the
target application. But first, let's describe the components.


What Makes up a Rhealstone Number?


The Rhealstone metric consists of quantitative measurements of the six
components that most influence the performance of real-time systems:
Task switching time is the average time the system takes to switch between two
independent and active (that is, not suspended or sleeping) tasks of equal
priority, as Figure 1 illustrates. Task switching is synchronous and
nonpreemptive, as when, for example, the real-time control software implements
a time-slice algorithm for multiplexing equal-priority tasks.
Task switching time is a fundamental efficiency measure of any multitasking
system. Measurement seeks to assess the compactness of task control data
structures and the efficiency with which the executive manipulates the data
structures in saving and restoring contexts. Task switching time is also
influenced by the host CPU's architecture, instruction set, and features
(provided the executive uses them).
Additionally, task switching time is a measure of the executive's list
management capabilities because an executive typically organizes its data
structures into ordered lists and shuffles the nodes according to
circumstances.
Preemption time is the average time it takes a higher-priority task to wrest
control of the system from a running task of lower priority. Preemption
usually occurs when the higher-priority task moves from an idle to a ready
state in response to some external event. For example, when an attached device
generates an interrupt, the interrupt service routine attempts to wake up the
task to service the request. Preemption time is the average time the executive
takes to recognize an external event and switch control of the system from a
running task of lower priority to an idle task of higher priority.
Though conceptually similar to task switching (Figure 1), preemption usually
takes longer. This is because the executive must first recognize the wake-up
action and assess the relative priorities of the running and requested tasks,
and only then switch tasks if appropriate.
Virtually all multiuser/multitasking executives assign task priorities, and
many let the application designer change priorities dynamically. For this
reason preemption, along with interrupt latency, is the most significant
real-time performance parameter.
Interrupt latency time, illustrated in Figure 2, is the time between the CPU's
receipt of an interrupt request and the execution of the first instruction in
the interrupt service routine. Interrupt latency time reflects only the delay
introduced by the executive and the processor and does not include delays
occurring on the bus or interfaces to external devices.
Semaphore shuffling time is the delay between a task's release of a semaphore
(usually by calling the executive's "relinquish semaphore" primitive) and the
activation of another task blocked on the "wait semaphore" primitive. No other
tasks should be scheduled in between, although at least three tasks with
different priorities should be active.
The focus on semaphore shuffling time is to measure the overhead associated
with mutual exclusion. In most real-time systems, multiple tasks compete for
the same resources. Semaphore-based mutual exclusion is a convenient way to
ensure that a nonshareable resource serves only one master at a time.
Figure 3 illustrates semaphore shuffling. Here Task 1 runs for a time and then
takes control of a resource by requesting its semaphore. Eventually Task 1 is
suspended and Task 2 starts. After a time, Task 2 requests the semaphore
presently owned by Task 1. Unable to continue, Task 2 stops, and Task 1 again
awakens. At the end of its period, Task 1 relinquishes the semaphore. The
executive recognizes that the semaphore is now free/available for suspended
Task 2. Semaphore shuffling time is the period between Task 1 releasing the
semaphore and the resumption of Task 2.
Advanced real-time executives recognize relative priorities in a "wait
semaphore" situation; that is, when multiple tasks are waiting for a
semaphore, the executive schedules them so that the highest-priority task goes
to the head of the queue. The Rhealstone measure doesn't give an executive
extra credit for prioritized semaphore queues, but such a capability may be
important to the software designer.
Deadlock breaking occurs when a higher-priority task preempts a lower-priority
task that holds a resource needed by the higher-priority task. The deadlock
breaking metric measures the average time it takes the executive to resolve
the conflict.
Deadlocks are a common multitasking problem, yet not all executives handle
deadlocks effectively, if at all. A common executive solution is to
temporarily raise the priority of the running task above that of the
interrupting task until the lower-priority task releases the needed resource.
At that point, the temporary priority is lowered and the new task can run.
Figure 4 illustrates deadlock breaking along a time line. Here, low-priority
Task 1 takes ownership of a resource and is then preempted by Task 2, which
has medium priority. This task runs until the highest-priority task preempts
it. Task 3 presently requests the critical resource, which is still held by
the suspended Task 1. The first phase of deadlock breaking time then occurs as
the executive decides what to do about the contention. Eventually the
executive raises the priority of Task 1 so that it can run. When Task 1
releases the critical resource, the executive suspends it and enters the
second phase of deadlock breaking, which entails reinstating Task 3 and giving
it control of the resource.
Deadlock breaking is thus the sum of times required to resolve an ownership
dispute between a low-priority task holding a resource and a higher-priority
task that needs it.
Datagram throughput time is the number of kilobytes per second one task can
send to another via calls to the primitives of the real-time executive
--without using a pre-defined common message buffer in the system's address
space or passing a pointer. The sending task must receive an acknowledgement,
as Figure 5 depicts. Executives typically provide pipes, message queues,
and/or stream files for this purpose.
The goal of measurement in the datagram throughput category is to average
intertask communications speed. Datagram throughput is of primary importance
in applications where one task collects data from the outside world and sends
it to another task for processing. An acknowledgement of receipt is essential
to assure the sending task that the data is safely in the hands of the
receiver before the sender overwrites its buffers with new data.


Computing the Rhealstone Number



Measurement of these six aspects of real-time performance yields a set of time
values in the microsecond to millisecond range. In order to combine the six
results into a single meaningful figure, express all times in seconds and
invert them arithmetically. For example, if activity N takes 200 microseconds,
the time in seconds is 0.0002 and the Rhealstone component is 1/0.0002 = 5000.
The frequency of N is 5000 per second. (Note: Datagram throughput time is
already expressed in kilobytes per second, so no further conversion is
necessary.)
More Details.
There are two related reasons for expressing Rhealstone components in terms of
frequency per second. Performance becomes directly proportional to value --the
bigger the number, the better the performance. This leads to the second reason
--the Rhealstone metric is then consistent with other industry benchmarks such
as Whetstones and Dhrystones.
An objective Rhealstone number can now be calculated as
 r1 + r2 + r3 + r4 + r5 + r6 = objective Rhealstones/second
where r1 is the task switching time component, r2 is preemption time, and so
on.
The objective Rhealstone number sum is useful for expressing the overall
performance of a real-time platform: for example, the "claim" number used by
the vendor of real-time executive A running on microprocessor X. The objective
Rhealstone sum is based on the assumption that all Rhealstone components are
equally influential in determining system performance. This may be true in
general, but it is probably not true of any specific real-time application.


Weighting the Rhealstone Components


Evaluators can tailor the Rhealstone figures to their applications by using
weighting coefficients. Herein lies one of the strongest features of the
Rheastone measurement.
Say a designer estimates that interrupts will occur five times as often as
task switches; semaphores and intertask messaging will not be used at all. In
this case, then, the weighting coefficients for semaphores and datagrams are 0
and the weighting coefficient for interrupts is numerically five times as
great as the weight for task switching, say 10 and 2, respectively. The other
two components (preemption and deadlock breaking), having unknown weights,
receive "default" coefficients of 1.
Such information leads us to an application-specific Rhealstone equation of
 n1*r1 + n2*r2 + n3*r3 + n4*r4 + n5*r5 + n6*r6 = application
Rhealstones/second
where the n factors are weight coefficients. Note that a coefficient must be
either zero (when the component is irrelevant to the specific application's
performance) or a positive value that gives the Rhealstone component its
relative importance in the application's performance.
An application designer typically has several alternatives among
microprocessors and real-time systems software. Using a matrix in which the
coefficients are uniformly applied to each alternative, the designer can
easily select the best platform for the application: The platform with the
largest application-specific Rhealstone value "wins."
The beauty of the Rhealstone metric lies in its ability to express real-time
performance objectively, while allowing designers to tailor and measurements
to specific applications. The Rhealstone number allows vendors to state --in
terms universally understood --the relative performances of products and
application designers to extrapolate these performances to a given real-time
system.


Data Acquisition


Real-time systems are, in general, "black boxes" operating within closed,
well-defined universes of possibilities that change dynamically. A real-time
system runs forever, dealing with circumstances as they arise. Unlike a
compiler or some other utility program, a real-time system has no defined
beginning and ending. This makes it difficult to measure real-time
performance.
A number of tools let us look into the black box and find out where it spends
its time. One is a hardware probe or in-circuit emulator, which feeds software
that reports how many times each machine instruction executes. Another is a
statistical profiler, a software monitor awakened by an event such as a clock
tick, which collects and reports information about what the observed system is
doing each time it looks. There are other tools as well, such as software
simulators driven by scripts.
The Rhealstone methodology specifies which aspects of performance must be
measured and how they are to be treated in order to arrive at a figure of
merit; the methodology does not describe how data for each component should be
obtained. The only stipulation is that someone else, using the same
configuration and measurement techniques, must be able to arrive independently
at the same performance components. The verifiability of both the individual
component figures and the Rhealstone sum depends, of course, on compliance
with a uniform method for reporting Rhealstones.


Reporting Rhealstone Results


A Rhealstone report must include the following:
The hardware platform for which results are shown, including the processor
type and speed in MHz.
Relevant details of the platform configuration. Examples are the major/minor
version of any software, whether cache memory is employed for instruction
prefetch by pipelined processors and memory wait states.
The method by which the Rhealstone components were obtained.
The individual Rhealstone components.
The last is the most stringent requirement of a Rhealstone report. Inclusion
of the individual Rhealstone components allows others to apply performance
figures to their circumstances. In the absence of by-category results, an
objective Rhealstone number has little meaning.
The overall intent is reproducibility; anyone should be able to take the same
configuration, apply the same measurement techniques, and come up with the
same results with a reasonable margin for error, regardless of the actual
application. Independent verification is the key to reliable Rhealstone
measurements.


Giving Credit Where Due


The Rhealstone concept was originally devised by Rabindra P. Kar of the Intel
Systems Group, Hillsboro, Ore., in response to the need to measure
enhancements to iRMX and Intel's other real-time executives. Recognizing that
this idea could lead to an industry standard for measuring real-time
performance, Frank Vaughan, the division's PR manager, approached DDJ's
editor-in-chief Jon Erickson. Jon enthusiastically endorsed the concept, and
senior technical editor Kent Porter is coordinating it.
From its inception, the objective of this project has been to remove any
vendor preference from the Rhealstone metric. This paper has already been
circulated to a number of vendors, users, and academics for comment. The
Rhealstone measurement was also the subject of a panel discussion at the
Real-Time Programming Conference held in Anaheim, Calif., last November. Now
DDJ invites interested readers to submit written suggestions and criticisms,
(see sidebar) so that we can forge this proposal into an industrywide,
vendor-independent measure of real-time performance.


Request for Suggestions


The Rhealstone proposal published here is a draft. DDJ invites interested
readers to submit suggestions for improvements. To the fullest extent
possible, we will incorporate reader feedback into the Rhealstone standard.
Current plans call for the final version to be published (along with a model
set of benchmark tasks written in C) in the June 1989 issue of DDJ.
Please send suggestions in writing to Kent Porter at DDJ, 501 Galveston Dr.,
Redwood City, CA 94063. Alternatively, address them to Kent at CompuServe
76704,51 or MCI KPORTER. Include your name, title, company, and a brief
summary of your real-time experience.
We cannot accept suggestions by telephone, so please DO NOT CALL.
Deadline for suggestions is March 1, 1989. A list of all contributors to this
standard will be published with the final specification.

--K.P.

































































February, 1989
REAL-TIME MODELING WITH MS-DOS


Who says MS-DOS isn't a real-time operating system?




David Bowling


David Bowling is a project engineer at Honeywell Defense Avionics Systems
Division and is responsible for the software design and development of
non-linear real-time simulators. His current projects include simulators for
the B-52, C-135, C-130, and C-17 aircraft. He can be reached at 6213 Antigua
NE, Apt. F, Albuquerque, NM 87111.


The phrase real-time has been thrown around a lot in the past few years,
mostly as a buzzword to impress people. Yet the question still asked is, what
is a real-time system? Basically, real-time systems are those systems that are
able to respond to specific events within a specified time. Given this
definition, even Unix-based computers are real-time systems if we specify our
response time in seconds -- which, in some cases, is very acceptable. Because
the temperature in a 100,000-sq ft building can't change much in a few
minutes, a heating and cooling system in a large building will work fine if it
can respond to a change in the building's temperature in less than a minute.
At the other end of the spectrum, the stuff I work with -- flight simulators
and flight controllers -- requires response times down in the milliseconds.
For example, an autopilot on a small fighter aircraft must respond to any
disturbances in less than about 20 milliseconds to keep the airplane from
going unstable (crashing).
As stated earlier, real-time systems respond to events. An event can be almost
anything. The most common type of real-time event in a computer system is a
hardware interrupt. One common hardware interrupt is generated when a key is
pressed. Generally, the desired real-time response to this hardware interrupt
is the appearance of a character on the computer screen. On a heavily loaded
minicomputer, this simple task can sometimes take seconds. The delay from the
time the event occurs to the time the computer responds is called interrupt
latency. Interrupt latency is a scale by which real-time computers are
measured. Using this scale, MS-DOS excels as a real-time operating system.
Because of the standard hardware and the wealth of information on how to
program it, functions can be directly tied to hardware interrupts. The
interrupt latency with MS-DOS ends up being the time required to jump to an
interrupt service routine -- which is not very long.
When modeling something in real time, the event we respond to is a
clock-generated interrupt. The clock generates its hardware interrupt at set
intervals known as the frame time. In real-time modeling, we respond to the
clock interrupt by computing or modeling what a system will do over the frame.
If, at every clock pulse, we model what our system did since the last clock
pulse (over one frame), we end up predicting what the system will do at the
precise time the actual system would do it. A familiar example of real-time
modeling is the game Flight Simulator. Flight Simulator models, or predicts,
what a Cessna 182 will do at the precise time an actual Cessna 182 would do
it.


Designing a Real-Time Multitasking Environment


The first thing we need to implement a real-time model is a multitasking
environment. Because MS-DOS isn't multitasking, we're going to implement our
own simple, real-time multitasking system. Our system will have two tasks --
one foreground and one background. The foreground task will do the real-time
modeling; the background task will be used to input data and display the
output of the model.
The best way to illustrate how the two tasks interact is with time lines (see
Figure 1). Every time we get a clock interrupt, the foreground task is
activated. The foreground task then runs to completion (see Figure 1a). The
obvious limitation of the foreground task is that it must take less time to
complete than the duration of one frame. The background task will be an
endless loop that continuously checks for input and displays output (see
Figure 1b). But, the only time allotted for the background task is the time
between the end of the foreground task and the next clock interrupt. Because
the background task has no knowledge of the real-time task the trick is
getting both of the tasks to work together. The foreground task interrupts the
background task, does its thing, and when finished resumes the background task
where it was interrupted (see Figure 1c).
You might have noticed that this scheme sounds a lot like a simple interrupt
service routine. Well, that's all it is. All we really did was give all the
functions different titles. The program becomes the background task and the
interrupt service routine becomes the real-time task. The only real difference
in implementation is that the computer spends more time in the interrupt
service routine (our real-time task) than in the program (our background
task).


Creating a Program Structure


By creating an interrupt-driven foreground task that interrupts a
continuous-loop background task, we can implement a real-time system with
MS-DOS. But the problem of how to create and start both tasks from our main
program still remains.
Our main program can be viewed as consisting of an initialization section, a
background task loop, and a termination section. In the initialization
section, we set up our foreground and background tasks and initiate the
foreground interrupt. In the termination section, we disable the foreground
interrupt and restore normal operation. Although all this can be done in one
long main program, I find a modular approach to programming facilitates
understanding as well as modification.
Using a modular approach, I developed a generic main program for any simple
real-time model (see Listing One, page 78). 1 used C because of its powerful
instruction set, which lends itself to manipulating hardware interrupts and
timer chip programming.
The initialization section of the generic program consist of calling three
setup modules: two set_up_user...modules to perform any special foreground or
background initialization for our particular real-time application and one to
set up the real-time interrupt. Similarly the termination section of the
generic program consists of an error handler and three set-down modules: one
to turn off the real-time interrupt and two set_down_user ... modules to
perform any special foreground or background termination for our real_time
application. As an example, in the simple spring-mass system discussed later
in this article, I use the set_up_user_background_task( ) and
set_down_user_background_task( ) functions to get the EGA card in and out of
graphics mode.


Spawning the Real-Time Task


But what's inside the functions set_up_real_time_task( ) and
set_down_real_time_task( ) ? As in setting up any other interrupt service
routine, we redirect the interrupt vector to our real-time function
(real_time_task( )). This is where Borland's Turbo C really comes in handy.
Turbo C has two functions -- getvect( ) and setvect( ) that make manipulating
the interrupt vectors a snap (see Listing Two, page 78).
Setting up the real_time task involves four simple steps:
1. Save the clock interrupt service routine address (hardware interrupt vector
0x08). The Turbo C function getvect( ) is used for this.
2. Set an unused interrupt vector to point to the original clock interrupt (I
used interrupt 0x50). We will want to continue calling the clock interrupt so
we don't alter our computer's system time. The Turbo C function setvect( ) is
used for this.
3. Overwrite the address of the original clock interrupt with the address of
our real-time function. The Turbo C function setvect( ) is also used for this.
4. Reprogram the system clock. I'll talk more about this in the next section,
"Setting the Frame Time."
Setting down the real-time task is even simpler:
1. Overwrite the address of our real-time task with the original clock
interrupt. Remember we saved it when setting up the real-time task.
2. Reset the system clock to its boot-up configuration.
Actually, I lied earlier. We can't tie our real-time function directly to the
interrupt; we have to do a little housekeeping before we call the actual
real-time task, user_defined_real_time_task( ) (see Listing Three, page 78).
This is the function we will write to do our modeling. When we called set up
real time task( ), we overwrote the vector to the clock interrupt that keeps
the system time. If we do nothing, time will stand still while the program is
running. The first time I realized this, the computer told me it was 10:30
when actually the time was 1 o'clock in the morning.
So, to avoid losing track of time, we need to call the original clock
interrupt routine in conjunction with our real-time task. It is also a good
idea to enable interrupts during the real-time function to make sure things
such as the keyboard are still serviced. But most important, we need to have
some way to check and control if we reenter the real-time function. If the
real-time task takes longer to run than the frame time, we end up reentering
the function. In simpler, more confusing, terms if we haven't finished the
real-time function when the next clock interrupt occurs, we call the real-time
function again, before we finish it the first time. This has a tendency to
bring the computer to a halt very quickly (see time line of Figure 2).
To prevent frame overruns from locking up the computer and to report the
occurrence of the overrun we use two flags--the flag running and the flag
over_run. By setting the flag running to TRUE in the interrupt routine, before
the user-defined function is called, and then resetting it to FALSE before
returning to the background task, it provides us with an indication of task
completion within the allotted frame time. If we enter our foreground
interrupt service routine and running is TRUE this means we are in the middle
of the user-defined function from the last interrupt and we've just reentered
the function. When this happens we set the flag over_run to TRUE and
immediately return to our previous foreground routine, still executing (see
Figure 2b). Once over_run is TRUE, we never again call the user-defined
function. Upon the completion of each background loop we examine over_run in
the main program. If set, we jump out of the background loop, enter the
termination section, and report the overrun error to the user.


Setting the Frame Time


It turns out that the frame time of the system clock is never what we want. I
usually find it's too large. This can be easily corrected by reprogramming the
timer that controls the system clock. But we also need to take into account
that once we change the clock timer, the computer might no longer keep
accurate time. I choose to limit the reprogramming of the timer to integer
divisions of the original frame time. If we use only integer divisors, we can
maintain the time of day simply by calling the original clock interrupt every
"divisor" frames. Remember, because we saved the address of the original clock
interrupt in old_clock_func and then set interrupt 0x50 to point to it, we can
call the original clock interrupt by generating a software interrupt 0x50.

PC clones use variations of the Intel 8253 timer chip to generate the system
clock interrupts. When MS-DOS boots, it sets up the 8253 to generate
interrupts every 54.9 msecs. This happens to be the longest interval that a PC
can generate, the shortest being 838 nsecs. The longest interrupt is generated
by setting the counter to count 65,536 clock ticks (this is done by setting
the 16-bit decrement counter to 0). Because the timer counter is an integer,
not only do we need to use a divisor that's an integer but also it needs to
divide evenly into 65,536 (no remainder). Any power of 2 will work. We set the
divisor in the function set_up_user_real_time_task( ) by assigning it to the
external variable user_define_divisor. For example, if we want interrupts
about every 14 msecs, we program the clock with 65,536/4=16,384
(user_define_divisor = 4). This actually gives a frame time of 13.73 msecs.
Just remember to use 13.73 msecs in all your real-time equations. The actual
code that sets the timer is shown in Listing Four, page 78. Once reprogrammed,
we only need to call the original clock interrupt every four clock interrupts
(see Listing Three).


Communicating Between Tasks


Now that we have the two tasks running, we need to get them talking to each
other. The easy way to do this is with memory common to each function
(external variables). The common term for this memory is DATAPOOL. The best
way to demonstrate this is with a simple example (see Listing Five, page 78).
The code has a background task that continually prints out the value of i, and
the real-time task continually changes i at the rate of the frame time. The
variable i is common to both tasks and allows them to communicate with each
other.


Limitations


There aren't too many limitations to either the background or the foreground
task, but if ignored all heck win break out:
The real-time task can't take more time to run than the frame time. Actually
it should take less than about 75 percent of the frame time. If no time is
left, the background task will never have time to run.
MS-DOS is not reentrant. If DOS calls are made in one task, they can't be made
in the other. If the background task is interrupted by the real-time task
while in a DOS function, then the real-time task makes a DOS call, and the
computer will probably crash. This doesn't really create a problem because
most DOS functions can't be completed in a frame. If you need to talk to a
device in real-time, you should write your own device driver.
Turbo C's void interrupt function saves all the 8Ox86's registers. What it
fails to do is save the registers and status of the 8Ox87. This means that
either the background or the real-time task; but not both, can use
floating-point math (if you don't have a coprocessor, ignore this). In many
cases this also doesn't pose a problem. In a lot of cases the background task
doesn't do any math, it just outputs data. If you have an assembler, inserting
the three lines of the in-line assembler of Listing Six, page 78, in
real_time_task( ) will rectify the problem.


Debugging Real-Time Tasks


Debugging the real-time part of a system can pose a problem. If something
happens, the interrupts run wild and the computer usually crashes.
Debuggers not designed for real-time systems usually don't work. In effect,
debuggers stop the program at each line in the code. This is fine except a
debugger will not stop the clock interrupts. This means that if the debugger
is in the real-time function, at the next clock interrupt, the real-time
function will reenter itself. This is sure to cause a system crash.
The old-fashion-way to debug a program is by placing print statements at
strategic locations in the code. This works fine if the code doesn't have to
be completed in a specified time. The problem with a real-time task, again, is
that the print statement will usually take longer to complete than the framed
time, causing the task to reenter itself.
The way to get around this is not to debug in real time. Set up both the
background and real-time functions as background tasks, calling one after the
other (see Listing Seven, page 78). Now you can use any method of debugging.


A Simple Spring-Mass System


Now what we've all been waiting for, a real-time model. I've included a
real-time program that models a simple spring-mass system. You're probably
wondering why you need a real-time system to model a spring-mass system. You
really don't, but you can impress you're friends when you show them the
program and tell them time model. Actually, a spring-mass system is an
excellent application to demonstrate the concepts of real-time modeling. It
turns out that the frequency at which the system should oscillate (how often
it bounces back and forth) can be computed easily.
The system shown in Figure 3 has a period of:
 ______________
 taud=2pi\/m/k(1 - zeta{2})
The term taud is how often the mass should bounce up and down in seconds. The
spring-mass program uses the following values for m, k, and zeta: m =
1.0182969 slugs, k = 10.0 lbs/inch, and zeta - 0.1. These values give a period
of 2.0 seconds.
This is the period of a real spring-mass system. If we went out and bought a
lead weight that had a mass of m, hung it under a spring with a constant k and
damping ratio zeta, set the system in motion, and used a stopwatch to time how
long it took to complete one cycle, the stopwatch would read 2.0 seconds.
 ________________________
 taud = 2pi\/1.0134145/10.0 (1 - 0.01{2})

 taud = 2.0 seconds
If we just write a normal program to model this (non-real-time), the rate at
which the program will oscillate (the time as measured by the stopwatch) is
anyone's guess. What we're going to do in our real-time model is model what
the mass would do at the exact time the lead weight would do it. Now, if
everything works, the program will display a graphical model of a mass having
a period of two seconds. If we hang the lead weight next to the computer
screen and start both the weight and computer at the same time, the lead
weight and computer model will move in unison. You'll have to run the program
to prove it to yourself.
And what's amazing is, it doesn't matter what computer you run the program on,
from the slowest XT to the fastest 386 the graphics display will always have
the same period--that is, as long as the computer has a coprocessor (you'll
overrun the frame time without one). Ah, the magic of real time. The only
thing that will change on different computers is how often the screen is
updated. The faster your computer, the faster the display will be updated.


Implementing the Spring-Mass System


Using a little college-level dynamics, the differential equation of motion can
be written for the system as follows:
 mx + cx + kx = 0
This equation gives us the acceleration of the mass; but what we want is the
position of the mass. All we have to do to get the position is solve this
equation (a second-order-homogeneous, linear differential equation)--that is:
 x = e{-zetaomega[n]t} (zetax{0}/ \/1 - zeta{2} sin \/1 - zeta{2}omega[n]t +
 __________
x[0]cos\/1 - zeta{2}omega[n]t)
where:
x[0] is the initial velocity of the system. This is set to 0 in our model.
omega[n] is the circular frequency of the system; with omega[n] = square root
of (k/m)
t is the current time. We increment this by the frame tune every frame -- that
is, t = t + frame time.
x[0] is the initial position of the system. This is set to 30.0 in our model.
zeta is the damping ratio of the spring-mass system, with zeta = C[c]/C

Once we substitute these values into our position equation, we get:
 x = e{-zetaomega[n]t} (x[0] + zetaomega[n]x[0]
 __________ __________
 omega[n]\/1 - zeta{2} sin \/1 - zeta{2}omega[n]{t}
 _____________
 + x[0] cos\/1 - zeta{2}omega[n]t)
This equation is the essence of our real-time model. The only thing left is
the frame time. For our spring-mass system, I picked a frame time of 54.9/4 =
13.73 msec, or 0.01373 seconds.


Conclusion


The real-time modeling program is broken into three files. The first file
contains all the real-time kernel functions (see realtime.c in Listing Eight,
page 78). The next two files contain all the user functions. massfor.c (
Listing Nine, page 80) contains all the real-time, or foreground,
functions--set_up_user_real_time_task( ), set_down_user_real_time_task( ), and
user_defined_real_time_task( ). massbck.c (Listing Ten, page 80) contains all
background functions -- set_up_user_background),set_down_user_background( ),
and user_defined_background_task( ).
To implement your own real-time model, just replace massfor.c and massbck.c
with your own . . ._user_. . . functions. And who says MS-DOS isn't a
real-time operating system?


Bibliography


Foster, Caxton C. Real-Time Programing -- Neglected Topics. Reading, Mass.:
Addison-Wesley, 1982.
Peterson, James L. and Silberschatz, Abraham. Operating System Concepts.
Reading, Mass.: Addison-Wesley 1985.
Savitzky, Stephen. Real-Time Microprocessor System. New York: Van Nostrand
Reinhold, 1985.
Thomson, William T. Theory of Vibration with Applications. Englewood Cliffs,
N.J.: Prentice-Hall, 1981.

_Real-Time Modleing with MS-DOS_
by David Bowling


[LISTING ONE]


 #include <dos.h>

 #define TRUE -1
 #define FALSE 0

 void user_defined_background_task();
 void set_up_user_background_task();
 void set_down_user_background_task();
 void set_up_user_real_time_task();
 void set_down_user_real_time_task();
 void set_up_real_time_task();
 void set_down_real_time_task();

 int not_done = TRUE;
 int over_run = FALSE;

 main(){
 set_up_user_real_time_task(); /*initialization section*/
 set_up_real_time_task();
 set_up_user_background_task();

 while( not_done && ! over_run){ /*background task loop*/
 user_defined_background_task();
 }

 set_down_real_time_task(); /*termination section*/
 set_down_user_background_task();
 set_down_user_real_time_task();


 if( over_run )
 printf("Error exit, frame over run\n");
 }







[LISTING TWO]

 void interrupt real_time_task();
 void interrupt (*old_clock_func)();
 void set_timer();
 int user_define_divisor = 1; /*initialize in case user forgets*/

 void set_up_real_time_task()
 {
 void interrupt (*getvect())();

 old_clock_func = getvect( 0x08 );/*save original clock vector*/
 setvect( 0x50, old_clock_func );/*store in unused location*/
 setvect( 0x08, real_time_task ); /*overwrite with real-time*/
 set_timer( user_define_divisor ); /*set system timer*/
 }

 void set_down_real_time_task(){
 setvect( 0x08, old_clock_func ); /*restore clock vector*/
 set_timer( 1 ); /*reset system timer*/
 }






[LISTING THREE]


 int running = FALSE;
 int inter_count = 0;

 void interrupt real_time_task(){
 enable();

 if( !running && !over_run ){
 running = TRUE;
 user_defined_real_time_task(); /*real-time function*/
 }else{
 over_run = TRUE;
 };

 if( inter_count == user_define_divisor ){
 geninterrupt( 0x50 ); /*call system clock*/
 inter_count = 0;
 }else{
 outport( 0x20, 0x20 ); /*8259 end of interrupt routine*/

 inter_count += 1;
 };

 running = FALSE;
 }









[LISTING FOUR]

 void set_timer( divisor )
 int divisor;
 {
 int cnt;
 int lo, hi;

 cnt = 65536 / divisor;

 lo = cnt % 256;
 hi = cnt / 256;

 outportb( 0x43, 0x36 );
 outportb( 0x40, lo ); /*write tic counter*/
 outportb( 0x40, hi );
 }






[LISTING FIVE]

 int i = 0; /* DATAPOOL */

 user_defined_background_task(){
 printf("i = %d\n", i );
 }

 user_defined_real_time_function(){
 i += 1;
 }








[LISTING SIX]

 union{

 char coprocessor_state[94];
 int control_word;
 }float_save;

 /* save coprocessor state */
 asm fsave float_save.coprocessor_state
 asm fldcw float_save.control_word
 .
 .
 .
 /* restore coprocessor state */
 asm frstor float_save.coprocessor_state







[LISTING SEVEN]


 while( not_done && ! over_run ){ /* non real-time debugging*/
 user_defined_background_task();
 user_defined_real_time_task();
 }







[LISTING EIGHT]

 #include <dos.h>

 #define TRUE -1
 #define FALSE 0

 void user_defined_background_task();
 void set_up_user_background_task();
 void set_down_user_background_task();
 void set_up_user_real_time_task();
 void set_down_user_real_time_task();
 void set_up_real_time_task();
 void set_down_real_time_task();

 int not_done = TRUE;
 int over_run = FALSE;

 main(){
 set_up_user_real_time_task(); /*initialization section*/
 set_up_real_time_task();
 set_up_user_background_task();

 while( not_done && ! over_run){ /*background task loop*/
 user_defined_background_task();
 }


 set_down_real_time_task(); /*termination section*/
 set_down_user_background_task();
 set_down_user_real_time_task();

 if( over_run )
 printf("Error exit, frame over run\n");
 }

/******************************************************/

 void interrupt real_time_task();
 void interrupt (*old_clock_func)();
 void set_timer();
 int user_define_divisor = 1; /* initialize in case user forgets */

 void set_up_real_time_task()
 {
 void interrupt (*getvect())();

 old_clock_func = getvect( 0x08 ); /*save original clock vector*/
 setvect( 0x50, old_clock_func ); /*store in unused location*/
 setvect( 0x08, real_time_task ); /*overwrite with real-time*/
 set_timer( user_define_divisor ); /*set system timer*/
 }

 void set_down_real_time_task(){
 setvect( 0x08, old_clock_func ); /*restore clock vector*/
 set_timer( 1 ); /*reset system timer*/
 }

/******************************************************/

 union{
 char coprocessor_state[94];
 int control_word;
 } float_save;

 int running = FALSE;
 int inter_count = 0;

 void interrupt real_time_task(){
 /* save coprocessor state */
 asm fsave float_save.coprocessor_state
 asm fldcw float_save.control_word

 enable();

 if( !running && !over_run ){
 running = TRUE;
 user_defined_real_time_task(); /*real-time function*/
 }else{
 over_run = TRUE;
 };

 if( inter_count == user_define_divisor ){
 geninterrupt( 0x50 ); /*call system clock*/
 inter_count = 0;
 }else{

 outport( 0x20, 0x20 ); /*8259 end of interrupt routine*/
 inter_count += 1;
 };

 running = FALSE;

 /* restore coprocessor state */
 asm frstor float_save.coprocessor_state
 }

/******************************************************/

 void set_timer( divisor )
 int divisor;
 {
 int cnt;
 int lo, hi;

 cnt = 65536 / divisor;

 lo = cnt % 256;
 hi = cnt / 256;

 outportb( 0x43, 0x36 );
 outportb( 0x40, lo ); /*write tic counter*/
 outportb( 0x40, hi );
 }

/******************************************************/







[LISTING NINE]

double x; /* DATAPOOL */
extern int user_define_divisor;

#define m 1.0134145 /* define spring-mass constants */
#define k 10.0
#define zeta 0.01
#define x_o 30.0

#define frame_time 0.013725

double t = 0.0; /* real-time */
double c1; /* real-time constants */
double c2;
double c3;
double c4;

void set_up_user_real_time_task(){
 double omega;
 double temp;
 double sqrt();


 user_define_divisor = 4; /* set user divisor counter */

 omega = sqrt( k / m );
 temp = sqrt( 1.0 - zeta * zeta );

 c1 = - zeta * omega; /* compute real-time constants */
 c2 = zeta * x_o / temp;
 c3 = temp * omega;
 c4 = x_o;
}

void set_down_user_real_time_task(){ /* no set down necessary */
}

void user_defined_real_time_task(){
 double cos();
 double sin();
 double exp();
 /* spring-mass model */
 x = exp( c1 * t ) * ( c2 * sin( c3 * t ) + c4 * cos( c3 * t ) );

 t += frame_time;
}







[LISTING TEN]

#include "graphics.h"

#define FALSE 0
#define TRUE -1

extern int not_done;
extern double x; /* DATAPOOL */

int x_off = 320;
int y_off = 100;

stationary[11][4] = { { 0, 0, 0, -5 }, /* base */
 { 0, 0, 7, 0 },
 { -40, -5, 40, -5 },
 { -35, -5, -30, -12 },
 { -25, -5, -20, -12 },
 { -15, -5, -10, -12 },
 { -5, -5, 0, -12 },
 { 5, -5, 10, -12 },
 { 15, -5, 20, -12 },
 { 25, -5, 30, -12 },
 { 35, -5, 40, -12 } };

void set_up_user_background_task(){
 int i, j;
 int g_driver = EGA;
 int g_mode = EGAHI;

 char d_path[] = {""};
 int g_error;

 if( registerbgidriver( EGAVGA_driver ) < 0 ){ /* EGA driver */
 printf("ERROR: can't register ega/vga driver\n");
 exit();
 };

 initgraph( &g_driver, &g_mode, d_path );
 g_error = graphresult();
 if( g_error < 0 ){
 printf("ERROR: %s\n", grapherrormsg(g_error) );
 exit( 0 );
 };

 setcolor( YELLOW );

 for( i = 0; i < 2; ++i ){ /* setup spring */
 setactivepage( i );
 for( j = 0; j < 11; ++j ){
 line( stationary[j][0] + x_off, stationary[j][1] + y_off,
 stationary[j][2] + x_off, stationary[j][3] + y_off);
 };
 };

}

void set_down_user_background_task()
{
 closegraph();
}

double stretch[12][4] = { { 7.0, 0.0, -7.0, 5.0 }, /* spring */
 { -7.0, 5.0, 7.0, 10.0 },
 { 7.0, 10.0, -7.0, 15.0 },
 { -7.0, 15.0, 7.0, 20.0 },
 { 7.0, 20.0, -7.0, 25.0 },
 { -7.0, 25.0, 7.0, 30.0 },
 { 7.0, 30.0, -7.0, 35.0 },
 { -7.0, 35.0, 7.0, 40.0 },
 { 7.0, 40.0, -7.0, 45.0 },
 { -7.0, 45.0, 7.0, 50.0 },
 { 7.0, 50.0, -7.0, 55.0 },
 { -7.0, 55.0, 7.0, 60.0 } };
int move[ 6][4] = { { -30, 5, 30, 5 }, /* mass */
 { -30, 40, 30, 40 },
 { -30, 5, -30, 40 },
 { 30, 5, 30, 40 },
 { 0, 0, 0, 5 },
 { 0, 0, 7, 0 } };

void user_defined_background_task(){
 double ratio;
 int x_spring;
 int i, j;
 static int start = 1;
 static int buff[2][100][4];
 static int cnt[2];
 static int b = 0;

 static int p = 0;

 if( start ){
 set_page( p );
 p = ( p )? 0: 1;
 setactivepage( p );
 };

 if( kbhit() ){
 not_done = FALSE;
 };

 x_spring = x + 30.0;
 ratio = 1.0 + ( (double)x / 60.0 );

 cnt[b] = 0;

 setcolor( RED ); /* draw mass */
 for( i = 0, j = cnt[b]; i < 6; ++i, ++j ){
 buff[b][j][0] = move[i][0] + x_off;
 buff[b][j][1] = move[i][1] + y_off + x_spring + 30;
 buff[b][j][2] = move[i][2] + x_off;
 buff[b][j][3] = move[i][3] + y_off + x_spring + 30;
 line( buff[b][j][0], buff[b][j][1], buff[b][j][2], buff[b][j][3] );
 };
 cnt[b] += 6;

 setcolor( GREEN ); /* draw spring */
 for( i = 0, j = cnt[b]; i < 12; ++i, ++j ){
 buff[b][j][0] = stretch[i][0] + x_off;
 buff[b][j][1] = (int)( stretch[i][1] * ratio ) + y_off;
 buff[b][j][2] = stretch[i][2] + x_off;
 buff[b][j][3] = (int)( stretch[i][3] * ratio ) + y_off;
 line( buff[b][j][0], buff[b][j][1], buff[b][j][2], buff[b][j][3] );
 };
 cnt[b] += 12;

 b = ( b )? 0: 1;

 set_page( p );
 p = ( p )? 0: 1;
 setactivepage( p ); /* switch page */

 if( ! start ){
 setcolor( BLACK ); /* undraw picture */
 for( i = 0; i < cnt[b]; ++i )
 line( buff[b][i][0], buff[b][i][1], buff[b][i][2], buff[b][i][3] );
 }else{
 start = 0;
 };
}

set_page(n) /* set visual page */
 int n;
{
 int far *farptr;
 int addr;

 setvisualpage( n );


 farptr = (int far *)0x00400063; /* status register address */
 addr = *(farptr) + 6;

 while( ( inport( addr ) & 0x08 ) == 0x08 ); /* while in vert retrace */
 while( ( inport( addr ) & 0x08 ) != 0x08 ); /* while not in vert retrace */

}






















































February, 1989
A BENCHMARK APOLOGIA


Credible benchmarks are oftentimes the exception, rather than the rule




G. Michael Vose and Dave Weil


G. Michael Vose is the editor of "OS Report: News and Views on OS/2," and he
can be reached at Box 3160, Peterborough, NH 03458. Dave Weil is the Microsoft
C project manager, a co-author of the Microsoft C Compiler, and the principal
representative from Microsoft on the ANSI C Standard Committee. He can be
contacted at 16011 NE 36th way, Box 97010 Redmond, WA 98073.


Battered and berated, benchmarks might remind you of Wiley E. Coyote: They
keep getting blown up but they never seem to go away. Consistently outsmarted
by those cunning optimizing compiler-writing road-runners, benchmarks put on a
new face almost yearly and dive back into the performance-testing fray.
Why do we bother with benchmarks? What have they done for us lately? If they
are so easy to trick, what is to be gained by testing software with
benchmarks? Will compiler technology always be one step ahead of benchmarks,
which supposedly help us evaluate new software? Which comes first: the test
suite or the product to test?
On the plus side, benchmarks can help us sort out real compiler performance
from compiler vendor hype. They can help uncover the flaws in a compiler or
show where a compiler is strong. Benchmarks even occasionally expose bugs and
language implementation errors.
But benchmarks can also mislead us into thinking a compiler doesn't perform
well, when the real problem is the benchmark itself. Conversely, a compiler
may show outstanding performance on a benchmark only to prove mediocre in
real-world situations. Knowing where the problem lies--in the compiler or in
the benchmark--can be tricky.
Ignoring the vast problems encountered in using benchmarks to measure computer
hardware performance, let's look at the practice of benchmarking
language-compiler software to see if benchmarks can tell us anything useful.
This discussion can also highlight the pitfalls we might encounter in
designing and using compiler benchmarks and evaluating their results.


What To Test?


Benchmarks often try to do too much. They often attempt to test several
different qualities of a compiler and then represent the results of those
tests as one monolithic total. This is sort of like using a college graduate's
overall grade point average to figure out how good he or she will be at math.
This flaw in benchmarks raises a fundamental question about what they are
trying to measure.
The primary performance issues for compiler testers encompass several
fundamental compiler operations: code quality, including its compactness and
its speed of execution; compiling speed; compiler features, such as support
for different memory models and compiler settings, or switches; and standards
conformance.
The secondary compiler issues subject to testing by benchmarks include the
environment of the compiler and the operation of the compiler on different
hardware platforms. So the first step in designing benchmarks is to figure out
what to measure. After you decide what you want to measure, then you can set
out to design a test algorithm to generate some data.
Testing individual statements or expressions is extremely difficult. Designing
a test to determine the speed of pointer dereferencing or integer math
operations exposes the benchmark writer to a variety of pitfalls. These
pitfalls include the use of loops to extend the execution of the test code to
some meaningful, measurable time period. Using loops to iterate some piece of
test code fifty thousand to one hundred thousand times to obtain a benchmark
time of one to two seconds raises the question of whether the benchmark is
measuring the test code or the compiler's loop overhead. In benchmarks that
run for only one to two seconds, a time differential of only one-third of a
second may create times too small to measure accurately.
Loops also frequently fall victim to compiler optimizations. A good compiler
can pull loop invariant code out of a loop and thereby distort the result of a
benchmark. In some extreme cases, as we'll explain in one of the examples that
follow, an optimizing compiler might do away with a loop altogether. Compilers
can also unroll a loop and break down the loop to a serial sequential
operation, while leaving others inside; this also will distort a benchmark's
results unless you are trying to measure a compiler's ability to optimize loop
code. But such a test may not tell you anything about the quality of the code
generated for the body of the loop.
One way to lessen the problems of using loops in benchmarks is to place a
large body of code within the loop. Large code blocks do not necessarily
prevent loop optimizations from taking place, but they do ensure that the loop
overhead is not significant compared with the time spent doing the operations
inside the loop--supposedly what you are trying to measure.
Trying to test individual statement execution probably isn't useful anyway.
Even if you could determine that a compiler dereferenced pointers at blazing
speeds, an application probably wouldn't spend much of its time dereferencing
pointers. Testing individual expressions might not paint an accurate portrait
of a compiler's performance for a total application. If you wanted to know how
well a compiler handles floating-point operations, the best test you could
devise would be a floating-point intensive application, such as an FFT
algorithm. Using real working code can often yield more meaningful performance
data about compilers than an artificially contrived test can.
Knowing exactly what you are testing becomes crucial in a variety of
situations. One common area of confusion is that gray area where a compiler
and an operating system overlap. For example, does a disk I/O benchmark test a
compiler's ability to handle files or its calls to the operating system? Does
that matter as long as you always benchmark compilers under the same operating
system? Do disk I/0 routines ever go directly to the hardware and bypass the
operating system?
A recently published operating system benchmark suffered from a
misunderstanding of the gray area where compilers and operating systems
intersect. The benchmark sought to test the operating system's virtual memory
manager. It created a large array and accessed elements in the array. But the
compiler used to compile the benchmark mapped the array in such a manner as to
cause a segment swap with each array-element access. The resulting test of the
virtual memory manager turned out to be a test of the compiler's memory
manager instead. The operating system got blamed for poor performance, but the
compiler was the culprit.
A common pitfall in designing compiler tests is a failure to isolate the
routines that do the work that you actually want to measure. To measure the
speed of matrix multiplication, for example, a test might set up two arrays
and then multiply them. Measuring the time needed to set up the arrays, as
well as the time needed to make the matrix multiplications, does not tell you
how well the compiler generates code to do matrix math. The timing routine
needs to be placed around the multiplication operation, not around all the
declaration and set-up code.
Today you must possess a knowledge of optimization techniques in order to test
compilers. If you want to measure code compactness, particularly for a modern
optimizing compiler, you must create test algorithms that cannot be reduced in
size by a given optimization technique unless you specifically want to measure
the implementation of that optimization. Not knowing how a given piece of test
code might be optimized leaves you wide open to generating misleading data.
Another complication is that you also can't always assume that a compiler
optimizes code just because the vendor claims it can; one well-known Unix C
compiler purports to do common subexpression elimination but actually performs
this optimization in only the simplest of cases.


Evaluating Results


Possibly the hardest part of using benchmarks is interpreting their results.
The proper design and coding of benchmarks minimizes this problem. But even
well-done tests can be overinterpreted or occasionally misinterpreted.
Many benchmarks leave themselves open to improper interpretation because they
fail to produce verifiable results. If a compiler's code executes an algorithm
25 percent faster than its rivals but calculates a wrong answer, how do you
interpret its performance? If you don't know it calculated an incorrect
answer, you may arrive at an unwarranted conclusion. Benchmarks should always
calculate a verifiable solution to a problem.
Interpreting benchmarks also suffers from an intrinsic fault of benchmarks:
their size. Benchmark code, because it needs to be portable and easily read,
frequently is brief. But can tests using a small piece of code be extrapolated
to similar performance characteristics when code size grows to hundreds or
thousands of lines? Small-test code usually gets compiled with a small- or
medium-memory model; will the results of these restricted-memory operations
apply to large- and huge-memory model programs?
Compilers may store data differently for specific memory models, particularly
for large data models. One compiler may store data in the near data segment
until this segment overflows and then store subsequent data in far data
segments; another may choose to store all data larger than a certain threshold
size--or all uninitialized data--in far data segments. For example, Microsoft
C always stores uninitialized array data in a far data segment. The compiler
does not know how big the array will be, because arrays can have different
sizes in different modules (as long as they are not initialized), and the
linker resolves the space allocation problem by allocating space for the
largest size. This is called common model; the name is taken from the Fortran
concept of Common data. Initializing the array will ensure that it goes into
the near data segment if it is less than 32K in size. This memory model
specific treatment of data can affect the performance of a compiler on a
benchmark, such as the Sieve. These factors can be controlled in most
compilers via initialization, by keywords such as near or far, or by control
of the data threshold size.
Now that we've entered the 32-bit era, another frequent benchmark problem is
whether ints are 16 or 32 bits. One compiler vendor recently related that
several of his customers called to complain that his new compiler was 25
percent slower than a popular competing compiler on a certain integer-math
benchmark. The benchmark really only used 16-bit integers, but people testing
a 32-bit 80386 compiler declared all the integers as 32 bits. The compiler had
to add extra code to cope with storing 16-bit numbers in 32-bit data
areas--code that slowed it down by 25 percent. Recoding in 16 bits showed this
compiler to be superior to the competing compiler. This incident shows how the
assumptions made when running and interpreting benchmarks can create false
impressions.
Another common problem with interpreting benchmark results is inequalities in
testing procedures. For example, a tester once walked into a room full of
80386 machines and started plugging in a disk containing an executable file of
what he thought was a good benchmark. As he went from machine to machine
recording test results, he uncovered some wide disparities. Later snooping
revealed that the machines had different-sized caches and different expanded
memory managers; both the caches and memory managers had been installed at
boot time. Therefore the benchmark results were skewed by factors not
immediately apparent. The tester's mistake, of course, was in not carefully
controlling all the factors that could affect the results.
Another example of inequitable test procedures shows up in some tests of file
I/O. One recent benchmark attempted to measure the efficiency of low-level
file I/O by writing a large file to disk and then performing random seeks in
the file, reading and writing pieces of the file. When this operation used
media such as a floppy disk, running the same program consecutively with
different compilers while deleting the test file between runs resulted in each
compiler performing the test more slowly than the previous one. Was this
because the compilers were really less efficient? The problem here stemmed
from the fact that the files were created on different areas of the disk
(farther out each time) because of the operating system's scheme for
allocating disk sectors. Therefore, the disk head-seek time became significant
(because of the way the file was written) and affected each subsequent run.
When the test was rerun with a reformatted disk to ensure that the file was
always created in the same place on the disk, each compiler ran the benchmark
in almost the identical amount of time. Were they all equally good at
low-level file I/O? Profiling this benchmark revealed that it spent more than
95 percent of its time in the operating system and less than five percent of
its time in the actual filehandling code, so the benchmark was really
measuring the operating system. The only compiler that had a significantly
different time (faster) on this benchmark did some internal buffering to cut
down on the number of system calls made (this, however, subverted the use of
low-level I/O) to force information directly to the disk.
A compiler's options also complicate the benchmark problem. When testing
compilers you must know what compiler switches to set to generate the
equivalent code from each compiler to make the tests as fair as possible.
Optimization levels are a prime example. If you decide to run all benchmarks
using the default switch settings for all the compilers you test, some
compilers will default to no optimization, others will default to full
optimization, and still others will fall somewhere in between. This lack of
care in testing can mislead for the following reasons.
You provide no information on what these defaults were for each compiler.
The testing compares apples to oranges.
Most users do not exhibit the same lack of care in running their compiler,
except perhaps when first getting accustomed to it.
Stack checking is another example of an area where compiler defaults can
differ. Some compilers turn stack checking on by default, and others turn it
off. On a benchmark such as the Fibonacci, stack checks could have a huge
effect on execution time even though virtually every compiler provides some
user control over stack checks. Benchmarkers must be sensitive to the need for
compilers to be compared on equal terms.


The Good and the Bad



There are many examples of both good and bad benchmarks. The Dhrystone,
Whetstone, and Sieve benchmarks have been around for quite some time, and,
although not perfect, are at least well understood.
The Dhrystone benchmark, Version 1.1, is susceptible to some string-handling
optimizations. It also produces no output that you can check for accuracy. But
the Dhrystone still is one of the better benchmarks, because it does test a
good mix of statements and operations. One major exception is floating-point
operations: Dhrystone does no floating-point tests even though some testers
make the mistake of judging floating-point performance by running the
Dhrystone. A new version of the Dhrystone currently being written will address
these problems.
The Whetstone benchmark does test floating-point operations and has also been
around for a long time. Whetstone performance can get a boost from compilers
that insert in-line code for certain routines, and this benchmark also
contains some loop invariant code that can be taken out of a loop. The
Whetstone benchmark also exhibits a frustrating lack of verifiability, because
it produces no output that you can check but performs many complex operations
that open the door to a lot of potentially wrong answers.
The Sieve benchmark tests a compiler's ability to perform integer operations,
memory references, and simple control structures, such as loops. The Sieve
calculates a testable result, prime numbers, and deserves high marks as a test
whose output you can verify. But, like other benchmarks, the Sieve tests just
a few operations in a small memory environment and therefore cannot tell you
much about the many operations that it doesn't test. Plus, as mentioned
earlier, the Sieve's results can be misleading if you don't account for an
individual compiler's method of handling uninitialized arrays.
Other well-known benchmarks leave a lot to be desired. The so-called Loop and
Float benchmarks have major flaws, as do some disk I/O benchmarks. One version
of the Fibonacci benchmark frequently gets misinterpreted.
The Float benchmark provides a classic example of how an optimizing compiler
can destroy a poorly written test algorithm. The Float algorithm (see -
Example 1 ) performs a series of multiplication and division operations inside
a loop. An optimizing compiler can reduce this code to a simple assignment
operation, c = b. The compiler first propagates constants and discovers that
they have a sequence of assignments of the form
 for(...) {
 c = CONST_VAL_1;
 c = CONST_VAL_2;
 c = CONST_VAL_1;
 c = CONST_VAL_2;
 ....
 }
The compiler will then throw away the extra stores to c, leaving only C =
CONST_VAL_2. Next the loop optimizer hoists this code out of the loop, because
it is invariant, leaving simply c = b. This program could be modified in one
of several simple ways to prevent this optimization. Initializing the
variables a and b with a function call, as in init(&a,&b);, will prevent
constant propagation. Alternatively, modifying the body of the loop so that a
or b or both also get modified would prevent this optimization, unless the
compiler was also smart enough to do the following induction analysis:
 for(....) {
 c = a * b;
 a *= a;
 b-=a;
 ...
 }
The Whetstone benchmark provides good examples of this kind of unoptimizable
loop.
Example 1

 /* The floating point arithmetic benchmark
 * does repeated multiplications &
 divisions in a loop that is
 * large enough to make the looping time
 insignificant. This
 * program is from the August 1983 issue
 of Byte magazine.
 */
 #include <stdio.h>

 #define CONST1 3.141597E0
 #define CONST2 1.789032E4
 #define COUNT 50000

 main ()
 {
 double a, b, c;
 unsigned int i;

 time_0();

 a = CONST1;
 b = CONST2;

 for ( i = 0; i < COUNT; ++i)
 {
 c = a * b;
 c = c / a;
 c = a * b;
 c = c / a;
 c = a * b;
 c = c / a;
 c = a * b;
 c = c / a;
 c = a * b;
 c = c / a;

 c = a * b;
 c = c / a;
 c = a * b;
 c = c / a;
 }
 fprintf (stderr, "%d\n", time_n());
 }

The Fibonacci test provides another example of how benchmarks can mislead if
the results are not interpreted correctly. This benchmark (see Example 2)
computes a Fibonacci number sequence using recursion. But using iteration
might be more efficient. This benchmark really tests function-call overhead.
The issue here boils down to the question: What do the results mean?
Example 2

 /* The Fibonacci benchmark tests
 recursive procedure calls
 */

 #include <stdio.h>

 #define NTIMES 34
 #define NUMBER 24

 main()
 {
 int i;
 unsigned value, fib();

 time_0();

 printf ("%d iterations: ", NTIMES);

 for( i = 1; i <= NTIMES; i++)
 value = fib(NUMBER);

 printf ("\n");

 fprintf (stderr, "%d\n", time_n());
 }
 unsigned fib(x)
 int x;
 {
 if( x > 2)
 return (fib(x-1) + fib(x - 2)):
 else
 return (1);
 }

This benchmark is small enough and straight forward enough that most compilers
generate similar code for it. But test results show that compilers that often
do well on most other benchmarks sometimes lose badly on this one. The reason
has to do with register variables. If a compiler does not implement register
variables, the code for the path that computes fib(x-l) + fib (x-2); will look
like this (in pseudo-assembler):
 push(x-l)
 call fib
 push ax ;save result in memory
 push(x-2) call fib

 pop bx ;retrieve saved result
 add ax,bx ;add current result and saved result
A compiler that implements register variables produces this code:
 push(x-1)
 call fib
 mov si,ax ;save result in register
 push(x-2)
 call fib

 add ax,si ;add current and save results
This latter sequence will always execute faster. Because these compilers must
save and restore the value of the register si every time the function is
called--and because this path is taken only one-half the times that fib is
called--the overhead of the save/restore on each call is greater than the
overhead to save/restore the intermediate result in memory. The compiler that
uses register variables, contrary to what your expectations may be, runs
slower than one that does not.
In most cases, however, register variables help produce more compact code and
enhance an algorithm's execution speed. This Fibonacci algorithm really is an
exception rather than the rule (other ways to alleviate this problem include
saving/restoring si only in the path where it is used). But benchmarkers
seldom engage in this kind of analysis.
The Loop benchmark was originally created to factor out loop overhead in
benchmarks that used loops to artificially increase a benchmark's running time
rather than increase the number of operations at the source level. The problem
with this technique is that many compilers that perform loop optimizations
will remove these loops, not as a benchmark ploy, but because other loop
optimizations may result in empty loop bodies. For example, loop unrolling on
loops that iterate a small number of times, as in
 float a[4];,
 ...
 for (i = 0; i < 4; i++) {
 a[i] = (float)i;

 }
might well be expanded to four individual assignments and the loop removed
entirely.
A better solution to the problem of dealing with loop overhead is to simply
make sure that the body of the loop is large and complex enough that the loop
overhead is insignificant.
So how can you protect yourself from misleading benchmark claims? Always
insist on benchmarks that provide verifiable output so that you can at least
be certain the code a compiler generates will correctly execute instructions.
Read benchmark code whenever you can to judge the vulnerability of the test
algorithms to optimization. And be sure to find out if all the conditions were
the same when a given test was performed on competing systems.


The Acid Test


There will probably never be a perfect compiler benchmark. Compiler technology
marches forward at a steady rate, and performance tests almost always follow
the creation of new compiler techniques.
To protect yourself as you evaluate compilers, you need to remember a few
simple but important ideas:
Understand what a test really measures.
Understand optimizations and how they affect code.
Understand that compilers make assumptions about data storage, optimizations,
and so on.
Understand the importance of an equal environment for running tests.
Take care in interpreting benchmark results.
The true measure of any compiler is how well it suits your programming style
and application needs. Benchmarks might be more useful if they measured
compiler performance on a variety of common tasks, ranging from heavily
compute-bound tasks (for example, the FFT program for floating-point
operations, [scalar] matrix inversion for array manipulation, table lookups or
tree manipulation for pointers) to I/O bound operations (for example, an
object decoder, such as the one used at Microsoft). Similar kinds of
applications could test other library related items, such as string
manipulation, memory allocation, and the like. Benchmarks that are too simple
tell you little, and most compilers end up looking much the same.
Developments of the future will no doubt create a need to test multitasking
compiler operations, real-time applications, and multiprocessor systems. These
complex systems will only add to the problems of creating realistic and
credible benchmarks. But knowing the pitfalls can help us differentiate the
signal from the noise.

 _A Benchmark Apologia_ by G. Michael Vose and Dave Weil

[EXAMPLE 1]

/* The floating point arithmetic benchmark
 * does repeated multiplications & divisions in a loop that is
 * large enough to make the looping time insignificant. This
 * program is from the August 1983 issue of Byte magazine.
*/
#include <stdio.h>

#define CONST1 3.141597E0 #define CONST2 1.7839032E4 #define COUNT 50000

main ()
 {
 double a, b, c;
 unsigned int i;

 time_0();

 a = CONST1;
 b = CONST2;

 for( i = 0; i < COUNT; ++i)
 {
 c = a * b;
 c = c / a;
 c = a * b;
 c = c / a;
 c = a * b;
 c = c / a;

 c = a * b;
 c = c / a;
 c = a * b;
 c = c / a;
 c = a * b;
 c = c / a;
 c = a * b;
 c = c / a;
 }
 fprintf (stderr, "%d\n", time_n());
 }







[EXAMPLE 2]

/* The Fibonacci benchmark tests
 recursive procedure calls
*/

#include <stdio.h>
#define NTIMES 34
#define NUMBER 24

main()
 {
 int i;
 unsigned value, fib();

 time_0();

 printf ("%d iterations: ", NTIMES);

 for( i = 1; i <= NTIMES; i++ )
 value = fib(NUMBER);

 printf ("\n");

 fprintf (stderr, "%d\n", time_n());
 }
unsigned fib(x)
int x;
 {
 if( x > 2 )
 return (fib(x-1) + fib(x - 2));
 else
 return (1);
 }






































































February, 1989
A C++ MULTITASKING KERNEL


In a multitasking environment, C++ lets you define tasks as object




Tom Green


Tom Green is an engineer at Central Data Corp. in Champaign, Ill. He can be
reached at 217-359-8010.


About two years ago, the company I work for started using a multitasking
kernel on our Multibus II SCSI and communications controllers. This was my
first exposure to a multitasking kernel, and I was curious about the
algorithms required to run multiple threads of code on a single processor. I
decided to try to write a small preemptive kernel (a kernel that switches
tasks by using a timer) for the MS-DOS environment. For my first attempt I
used Turbo C, and I wrote a small working kernel in a couple of weeks. The
kernel stole the MS-DOS timer-tick interrupt and switched between tasks (C
functions that repeat forever) with each timer tick.
When the Zortech C++ compiler came out for MS-DOS machines, I immediately
bought it and started to rewrite my kernel. C++ objects, I thought, were
perfect for a multitasking kernel. Tasks would be objects that the
task_control object would communicate with and switch between. C++ would also
allow task objects to make all of their data private, and task objects could
be created with a constructor that required parameters. The task_control
object would also keep all data private and supply several routines so that
users could add tasks, start and stop multitasking, and so on.
I used Modula-2 for the model when designing task communication. The signal
object is modeled on the Modula-2 data type that you see in many Modula-2
packages. This object is used by a task to send or receive a message, and it
supplies a fast and lean interface. I added a block routine (which allows a
task to voluntarily give up control) and a task-control routine that allows a
task to stop task switching if it is calling a routine that is not reentrant
(MS-DOS, BIOS, or C library routines).
The kernel presented here may seem like a very minimal kernel, and that's
because it was meant to be. If you understand how the kernel works, however,
you will find that it is easy to expand and customize. There are many
different design philosophies for kernels, but I think that this kernel
supplies what you need for most programming situations.


Objects and Task-Switching Overview


This kernel uses three C++ classes: task_control, task, and signal. If you are
new to C++, a quick explanation might be in order.
A "class" is similar to a C structure; in fact, in C++ a structure is a class
in which all members are public. The data in a class can be public or private.
A class can also have "methods" (another name for a function) associated with
it. An object is a variable of a specific class. When an object is
instantiated, space is allocated for all of the data (public and private) and
a set of methods (functions) then exist to interface with that object. These
methods are used just like data in a structure, with the . (period [structure
field selection]) operator or the -> (arrow [structure field selection with
indirection]) operator. This does not mean that there are separate copies of a
method's code for each object. The Zortech compiler (and also, I assume, C++
preprocessors) passes to the method a hidden pointer to the object. This
pointer is used by the method to access the object's data.
When I talk about objects, I will also refer to "constructors" and
"destructors." A constructor is called when an object is declared; it is used
to initialize the object. A destructor is called when an object goes out of
scope; it is most often used to free the memory allocated by the constructor.
All of the data in each of this kernel's objects is private to prevent someone
from accidently destroying the workings of the kernel. A few public routines
(or methods) are provided for initialization and for communicating with the
task_control object.
A signal object is a queue of task objects. The data for a signal object is
two task-object pointers, which are used to create a queue (a head and tail
pointer). This is the simplest object in the kernel. It has only two methods:
get_task_q, which gets the next pointer to a task object from the queue, and
put_task_q, which appends to a task object pointer to the queue. These methods
are private and can only be called by the task_control object (which is a
friend of a signal object). A constructor for signal objects initializes the
queue head and tail pointers.
A task object contains several pieces of data that are used by the
task_control object. Except for a constructor and a destructor, there are no
methods for interfacing with a task object. Signal objects and the
task_control object are "friends" of task objects. This means that they may
directly manipulate the private data of a task object. I did this to avoid the
overhead that I might incur by using methods to interface with task object
private data. C++ supports in-line functions, but, unfortunately, the Zortech
compiler does not generate in-line code in all cases.
When a task object is declared, the constructor is called and the private data
is initialized. A workspace or stack is allocated for the task object. A task
"image" is set up in the allocated memory. The "image" is an area of memory
that has an image of all of the registers (8086) of the task when it was last
stopped. In this case, we want to set up a register image to be used the first
time the task is switched to. The task object carries a pointer to the task
image which is loaded into the stack pointer when you want to activate the
task.
See the structure called task_image in Listing One, (task.hpp) page 84, to get
an idea of how this task image looks. It shows how the stack would look after
an interrupt handler had been entered and all of the registers had been saved,
which is how the kernel saves the image of a running task before switching to
another task.
When the task image is set up, a routine called getCS is called to get the
code segment of the task. This is necessary because of a compiler bug that
returns the contents of the DS register when you try to get the segment of a
near function pointer. Because of this, the kernel will only work with the
small model of the compiler.
The task object also carries a task-state flag and a pointer to a task object.
This next_task pointer is used by the signal's methods to append and remove
task objects from a signal queue. The destructor for a task object frees the
memory allocated for a task object's stack or workspace.
The task_control object takes care of switching between tasks and provides an
interface to the outside world. Its methods allow a task to wait for or send a
signal, turn on and off task switching, and so on. Before multitasking is
started, task objects must be added to the task_control object with a call to
add_new_task. The task_control object has a signal object called ready_q to
which all tasks that are ready to run are added. This object is not really
used to signal tasks, but it is used because a signal object is a queue. After
all of the task objects have been added, a call to start_tasks saves the old
timer-tick interrupt handler, installs a new interrupt handler, and starts up
the first task when the first timer interrupt occurs. The task_control object
continues to switch between tasks until stop_tasks is called. When this
happens, the original timer-tick interrupt handler is reinstalled, and
execution is resumed at the point after the start_tasks routine was called.
When the kernel is running, tasks are switched 18.2 times a second. When a
timer-tick interrupt occurs, the task that is running is appended to ready_q
and the next task to run is removed from the head of ready_q. The new task
runs until the next timer interrupt and the process is repeated.
Tasks can also voluntarily give up control and call the kernel to run the next
task on the ready_q. A task may block, send a signal, or wait for a signal.
Using send and wait allows tasks to communicate.
A task can use task_control::block(), which takes no parameters and is called
by a task to voluntarily give up control. The calling task is appended to
ready_q and the next task to run is removed from the head of ready-q. This is
used by a task to allow other tasks to run when it is done.
A task calls task_control::wait() with the address of a signal object to wait
for a signal from another task. The calling task is appended to the signal
object's queue, and the next task to run is removed from the head of ready-q.
Tasks call task_control::send() with the address of a signal object to send a
signal to a waiting task. If there are any task objects in the signal object's
queue, the task from the head of the queue is removed and appended to ready-q.
The calling task object is then appended to ready-q. The next task to run is
removed from the head of ready_q.
The task_control object has two methods to enable and disable task switching
by the timer-tick interrupt. This allows a task to prevent a switching if it
is calling a routine (MS-DOS, BIOS, or C library) that is not reentrant. Call
task_control::lock() to disable task switching and task_control::unlock() to
reenable task switching. These routines do not affect a task that is
voluntarily giving up control and calling the kernel (block, send, or wait).
One warning about the task_control object: there can only be one. If you look
in Listing Two on page 84, you will find a global variable called gl_tptr,
which is a pointer to a task_control object. In the constructor for a
task_control object, this pointer is initialized to the address of the
task_control object (the "this" pointer). The pointer gl_tptr is used by the
timer-tick interrupt handler and the save_image routine in Listing Three
(timer.asm), page 88. This is the hidden pointer (passed to a C++ routine)
that is needed for calling task_control::task_switch(). This code is a little
ugly, but it allows the task-switching code to be written in C++, and it
allows task_switch to access the private data of task and signal objects (by
being a friend).


Details of Task Switching


Task switching would not be possible if you could not save a task image of all
of the 8086 registers. This is accomplished by the timer_handler and the
save_image routines in Listing Three. These routines both do the same thing:
they save the task image on the task's stack by pushing the necessary
registers and then call the C++ task_switch routine. Let's look at these
routines more closely.
The new timer-tick interrupt handler is timer_handler. It saves all necessary
registers and restores the DS register so the C++ code can find its data. The
original timer-tick interrupt handler is called to take care of the hardware
needs for the timer interrupt. Next, several parameters are pushed to prepare
for a call to the C++ task_switch routine. I will discuss these parameters
later. After the task_switch routine is called, a far pointer to the new task
object's image is returned. The sp and ss registers are loaded with this
pointer, and the new task object's registers are popped from the stack. An
iret instruction is executed, and we are running our new task.
The save_image assembly routine is called by the C++ block, send, and wait
methods. This routine is used to save the image of a task just as
timer_handler does. This routine is a little tricky because it is passed two
parameters (a flag and a signal pointer) on the stack. These parameters are
popped off the stack to pass to the C++ task_switch routine, then pushed back
on, and then the stack is rearranged so the flags and a far return address are
on the stack (as if an interrupt occurred). The rest of the registers are
pushed as in timer_handler. The rest of the routine is just like the
timer_handler routines; the C++ task_switch method is called, and on return
the SS and SP registers are set up to run the next task.
The task_switch method is called by the timer_handler and save_image routines.
These routines pass a far pointer to the image of the currently running task.
The pointer to the task object is put on ready-q or on the queue of a signal.
The flag passed by the routines is used to tell task_switch where it was
called from. task_switch needs to know if a timer interrupt or a send, wait,
or block call occurred. Once task_switch decides what task should run, the far
pointer to this task's image is returned to the save_image or timer_handler.
The task runs after registers are restored and an iret is executed.
I hope that this has shown how easy task switching really is. The important
part of task switching is saving all of the registers of the running task and
then saving a far pointer to that area of memory. Each task has its own stack
(memory that has been allocated with malloc). The far pointer points into this
allocated stack to the area where the registers were pushed. Restoring a task
to its last running state just requires getting the far pointer, loading it
into the ss and sp registers, restoring the registers, and doing an iret. The
new task will then be running.


A Multitasking Demo Program


To compile the multitasking program shown in Listing Four, page 90, which
demonstrates the concepts discussed so far, type ztc taskdemo task timer. The
Zortech ztc program looks at each of the source files and runs the C++
compiler or assembler as necessary. The linker then creates the executable
file.

The taskdemo program has five tasks that demonstrate all of the ways that
tasks can be switched. Not much happens in the program. The five tasks run,
and counters are incremented. The task0 task takes care of printing the
counters for the other four tasks. Let's look at each task.
The task0 task makes a lock( ) call to make sure it is not switched until it
has finished writing to the screen. The counter for each of the other four
tasks is printed. Then stop_tasks( ) is called if a key has been pressed. If
no key has been pressed, the unlock( ) routine is called to reenable task
switching. The block( ) routine is then called to allow other tasks to run.
The task1 and task2 tasks increment their counters and will run until a timer
interrupt occurs. These tasks will receive a big chunk of time to run, so the
counters increment very quickly.
The task3 task increments its counter and then calls wait(), which allows the
next task to run. Next, task4 increments its counter and then calls send(),
which causes task3 to be placed on ready_q. The call to send() allows the next
task on the ready_q queue to run. The net effect of this is that the counters
of task3 and task4 increment very slowly, because each task allows another
task to run after incrementing its counter.


Multitasking Kernel Philosophy


It should be easy to change or add features to this simple kernel. For
example, in the Modula-2 implementations of send() that I have seen, if there
is no task on the signal queue, then a flag is set so the next task to wait on
the signal queue will get this message. In my version of the kernel, this
message would be lost. If you do not think a message should be lost, then just
add a flag to the signal class.
Modula-2-style task communication was the perfect approach for the "simple and
lean" kernel approach. It allows the task_control object to get by with a very
small data structure, because signal objects carry around all of the data they
need. The interface to these signal objects consists of a couple of simple and
fast methods to get and put pointers on a queue. The task_control object uses
the same simple methods to get the next task to run off of ready-q. If you
would like to see a more complete explanation of Modula-2-style task
communication, I recommend Chapter 16 of the book Modula-2: A Software
Development Approach.
I do have a couple of suggestions on how to improve this kernel. Assigning a
priority to a task can be a handy feature. This would simply entail adding a
ready_q to the task_control object for each priority level, or perhaps using
an array of ready_q queues. Do not use too many different priority levels;
five would probably be enough. The task_switch routine would then search the
ready_q queues, highest priority first, to find the next task to run. When a
task was appended to a ready_q, you could simply check the task's priority and
append it to the correct ready_q.
Adding a way for a task to sleep for a number of clock ticks might be useful
as well. This would be a little more complicated, and there are several
approaches you could take. I hope you can see, however, that adding these
features or customizing the kernel will be very simple.


Suggested Reading


Ford, Gary A., and Weiner, Richard S. Modula-2: A Software Development
Approach, John Wiley & Sons. New York,: 1985.
Weiner, Richard S., and Pinson, Lewis J. An Introduction to Object-Oriented
Programming and C++, Addison-Wesley. Reading, Mass.: 1988.

_A C++ Multitasking Kernel_
by Tom Green


[LISTING ONE]


/********************************************/
/* TASK.HPP */
/* Tom Green */
/********************************************/
/* this file contains classes needed to use multi-tasking kernel */
/* include this file in your source code and then link with */
/* task.cpp and timer.asm */

/* this is used when a task is initialized */
/* this is a pointer to a function */
typedef void (*func_ptr)(void);
/* this is used for interrupt handler to call old interrupt handler */
/* this is a far pointer to a function */
typedef void (far *far_func_ptr)(void);

/* this is how the registers will look on the stack for a task */
/* after they have been saved for a task switch. the sp and ss */
/* registers will point to this when a task is started from the */
/* interrupt handler or save_image */

typedef struct task_image{
 unsigned int bp;
 unsigned int di;
 unsigned int si;
 unsigned int ds;
 unsigned int es;
 unsigned int dx;
 unsigned int cx;
 unsigned int bx;
 unsigned int ax;
 unsigned int ip;

 unsigned int cs;
 unsigned int flags;
}task_image;

/* a task object. contains information needed by task_control object */
/* to do task switching and a pointer to the task's workspace (stack) */

class task{
 private:
 friend class task_control; // task_control object needs access
 friend class signal; // signal needs access to next_task
 task_image far *stack_ptr; // task stack ("image") pointer
 unsigned char task_state; // task state flag
 unsigned char *workspace; // address of allocated task stack
 task *next_task; // pointer to next task in queue
 public:
 task(func_ptr func,unsigned int workspace_size); // constructor
 ~task(); // destructor
};


/* this is a queue for tasks */
/* it is called signal so user can define a signal for task communication */

class signal{
 private:
 friend class task_control; // task_control needs access
 task *head;
 task *tail;
 task *get_task_q(void); // get next task off of queue
 void put_task_q(task *tptr); // append task to queue
 public:
 signal(void){head=tail=0;}; // constructor
};




/* task_control object */
/* routines and methods to interface with and control tasks */
/* this object will initialize and restore interrupt vectors, */
/* keep track of timer ticks, and switch execution between the */
/* task objects */

class task_control{
 private:
 signal ready_q; // queue of tasks ready to run
 task *current_task; // current active task
 task_image far *old_stack_ptr; // return to this stack when done
 unsigned int task_running; // task switching enabled flag
 unsigned long timer_ticks; // 18.2 ticks/second
 unsigned int task_lock; // lock out task switching
 task_image far *task_switch(task_image far *stk_ptr,
 unsigned int flag,
 signal *sig);
 public:
 task_control(void); // constructor
 void add_new_task(task *new_task); // add new task object to ready q
 void start_tasks(void); // start switching tasks on ready_q

 void stop_tasks(void){task_running=0;};
 unsigned long get_timer_ticks(void){return(timer_ticks);};
 void lock(void){task_lock=1;}; // current task can not be switched
 void unlock(void){task_lock=0;}; // allow task switching
 void send(signal *sig); // put task from sig q on ready q
 void wait(signal *sig); // put task on sig q
 void block(void); // task allows next to run
};






[LISTING TWO]

/********************************************/
/* TASK.CPP */
/* by Tom Green */
/********************************************/

/* this file implements the methods used by task_control and task */
/* objects */

#include <stdio.h>
#include <stdlib.h>
#include <dos.h>
#include <int.h>
#include "task.hpp"

/* task states */
#define TASK_INACTIVE 0
#define TASK_ACTIVE 1
#define TASK_READY 2
#define TASK_WAITING 3
#define TASK_ERROR 0xff

/* flags for interface routines */
#define TASK_TIMER_INTR 0
#define TASK_SEND 1
#define TASK_WAIT 2
#define TASK_BLOCK 3

/* system timer interrupt or "timer tick" */
#define TIMER_INT 8

/* routines we need from timer.asm */
unsigned int getCS(void);
extern void timer_handler(void);
extern void save_image(unsigned int flag,signal *sig);

/* global for timer_handler to call old interrupt routine */
far_func_ptr old_timer_handler;

/* this is really ugly. */
/* when constructor for task_control object is called we save the */
/* this pointer for task switch routine to call our task_control object */
/* task_switch. this means we can only have 1 task_control object. sorry */
task_control *gl_tptr;


/* constructor for a new task. workspace will be the stack space for */
/* the task. when the timer interrupt happens the tasks "image" */
/* is saved on the stack for use later and the task_image *stack_ptr */
/* will point to this image */

task::task(func_ptr func,unsigned int workspace_size)
{
 task_image *ptr;

 /* get stack or "workspace" for task */
 if((workspace=(unsigned char *)malloc(workspace_size))==NULL){
 task_state=TASK_ERROR; // do not let this one run
 return;
 }

 /* now we must set up the starting "image" of the task registers */
 /* ptr will point to the register image to begin task */
 ptr=(task_image *)(workspace+workspace_size-sizeof(task_image));

 /* now save the pointer to the register image */
 stack_ptr=MK_FP(getDS(),(unsigned int)ptr);

 ptr->ip=(unsigned int)func; // offset of pointer to task code
 ptr->cs=getCS(); // segment of pointer to task, compiler bug
 ptr->ds=getDS();
 ptr->flags=0x200; // flags, interrupts on
 task_state=TASK_INACTIVE; // task is inactive
 next_task=0;

/* destructor for a task object */

task::~task(void)
{
 free(workspace);
}

/* get the next task off of a task queue */

task *signal::get_task_q(void)
{
 task *temp;

 temp=head;
 if(head)
 head=head->next_task;
 return(temp);
}

/* append a task to the end of a task queue */

void signal::put_task_q(task *tptr)
{
 if(head)
 tail->next_task=tptr;
 else
 head=tptr;
 tail=tptr;
 tptr->next_task=0;

}

/* constructor for task_control */
/* inits private stuff for task control */

task_control::task_control(void)
{
 gl_tptr=this;
 task_running=0;
 current_task=0;
 timer_ticks=0L;
 task_lock=0;
}

/* adds a task to the task ready_q */
/* call this routine after creating a task object */

void task_control::add_new_task(task *new_task)
{
 if(new_task->task_state!=TASK_ERROR){
 new_task->task_state=TASK_READY;
 ready_q.put_task_q(new_task);
 }
}

/* call to start up tasks after you have created some */
/* and added them to the ready_q */

void task_control::start_tasks(void)
{
 unsigned int offset,segment;

 task_running=1;
 /* get address of old timer interrupt handler */
 int_getvector(TIMER_INT,&offset,&segment);
 old_timer_handler=(far_func_ptr)(MK_FP(segment,offset));
 /* set up our new timer interrupt handler */
 int_setvector(TIMER_INT,(unsigned int)timer_handler,getCS());
 /* tasks will now start running */
 while(task_running)
 ; // do nothing, trick to wait for tasks to start up
 /* falls through to here when multi-tasking is turned off */
}

/* gets the next task off of sig queue and puts it */
/* on the ready_q. this suspends operation of the calling */
/* task which is also put on the ready queue */

void task_control::send(signal *sig)
{
 save_image(TASK_SEND,sig);
}

/* puts the calling task on the sig queue to wait for a signal */

void task_control::wait(signal *sig)
{
 save_image(TASK_WAIT,sig);
}


/* this causes the current task to be placed on the ready queue */
/* and a switch to the next task on the ready_q */

void task_control::block(void)
{
 save_image(TASK_BLOCK,(signal *)0);
}

/* this routine is called to do a task switch. it is */
/* passed a task_image far * to the current stack or task "image". */
/* also pass a flag (described above) and a signal pointer if needed. */
/* a task_image * to the "image" of the next task is returned */

task_image far *task_control::task_switch(task_image far *stk_ptr,
 signal *sig)
{
 task_image far *temp;
 task *tptr;

 if(flag==TASK_TIMER_INTR) // increment clock if it is a timer interrupt
 timer_ticks++;

 /* this saves a pointer to stack when we first start multi-tasking */
 /* allows us to return to where start_tasks was called */
 if(!current_task){ // no current task so save state for restoring
 old_stack_ptr=stk_ptr; // save stack pointer
 current_task=ready_q.get_task_q(); // set up a current task
 }

 /* we have an active task, so do task switch if we can */
 if(current_task->task_state==TASK_ACTIVE){
 current_task->stack_ptr=stk_ptr; // save stack pointer
 current_task->task_state=TASK_READY; // task is ready to go
 /* do not allow task switching if tasks are locked and */
 /* it is timer interrupt */
 if(!task_lock flag!=TASK_TIMER_INTR){
 /* check and see what caused task_switch to be called */
 switch(flag){
 case TASK_WAIT:
 current_task->task_state=TASK_WAITING;
 sig->put_task_q(current_task);
 break;
 case TASK_SEND:
 if((tptr=sig->get_task_q())!=0)
 ready_q.put_task_q(tptr);
 // fall through
 case TASK_BLOCK:
 case TASK_TIMER_INTR:
 current_task->task_state=TASK_READY;
 /* put old task on ready queue */
 ready_q.put_task_q(current_task);
 break;
 }
 /* get next task to go */
 current_task=ready_q.get_task_q();
 }
 }


 /* if we are still multi-tasking, get task ready to run */
 if(task_running){
 current_task->task_state=TASK_ACTIVE;
 temp=current_task->stack_ptr; // get stack pointer of task
 }
 /* multi-tasking has stopped, get ready to return where we started */
 else{ // someone called stop_tasks
 int_setvector(TIMER_INT,FP_OFF(old_timer_handler),
 FP_SEG(old_timer_handler));
 temp=old_stack_ptr; // get back original stack
 }
 /* return far pointer to stack_image to do task switch */
 return(temp);
}





[LISTING THREE]

;*****************************************************************************
; TIMER.ASM
; by Tom Green
; Timer interrupt handler
; Timer interrupt handler calls original handler first and then calls the
; task_control object task switcher. a pointer to the stack "image"
; of the new task is returned by the task switcher.
; getCS
; returns current code segment
; save_image
; saves "image" of task as if interrupt had happened and then calls the
; task_control object task switcher. a pointer to the stack "image"
; of the new task is returned by the task switcher.
;*****************************************************************************

 .MODEL SMALL
 .8086

 .DATA

extrn _old_timer_handler:dword
extrn _gl_tptr:word
extrn __task_control_task_switch:near

 .CODE

;*****************************************************************************
; unsigned int getCS(void) - returns current code segment.
; this is needed because of compiler bug. when a function is cast
; to a far function, and you try to get the segment, DS is returned
; instead of CS.
;*****************************************************************************
_getCS proc near
public _getCS
 mov ax,cs
 ret
_getCS endp


;*****************************************************************************
; timer_handler - this replaces the MS-DOS timer tick interrupt (8H).
; this routine saves everything on stack, calls original timer interrupt
; handler, and then calls task_control object task switcher.
;*****************************************************************************
_timer_handler proc near
public _timer_handler
 push ax ;save everything
 push bx
 push cx
 push dx
 push es
 push ds
 push si
 push di
 push bp
 mov bp,dgroup
 mov ds,bp ;get our data segment back
 pushf
 call dword ptr dgroup:_old_timer_handler ;call original handler
 sti
 xor dx,dx
 mov ax,ss
 mov bx,sp
 push dx ;push 0 for last 2 parameters
 push dx ;meaning timer interrupt
 push ax
 push bx
 mov dx,_gl_tptr ;push hidden pointer for C++ object
 push dx
;stack is now set up for call to task_control object task_switch
 cli ;turn off interrupts for task switch
 call __task_control_task_switch
;no need to clean up the stack because it will change
 sti
 mov ss,dx ;new ss returned in dx
 mov sp,ax ;new sp returned in ax
 pop bp ;restore registers
 pop di
 pop si
 pop ds
 pop es
 pop dx
 pop cx
 pop bx
 pop ax
 iret
_timer_handler endp

;*****************************************************************************
; void save_image(unsigned int flag,signal *sig) - send, wait, block
; etc. all call through here to save the "image" of the task. this
; code simulates what will happen with an interrupt by saving the task
; image on the stack. the flag passed is passed on to the task_control
; object task switcher so it knows if it was called by the timer
; interrupt handler, send, wait, block, etc. the second parameter
; is a signal pointer which is used by send and wait and is passed
; through to the task switcher.
;*****************************************************************************

_save_image proc near
public _save_image
;since this is a C call we can destroy some registers (ax bx cx dx),
;so now we will set up the stack as if an interrupt call had happened.
;leave parameters on stack, because calling routine will adjust on
;return. bx and cx will have the parameters that were passed.
 pop ax ;get return address offset on stack
 pop bx ;get first parameter off stack
 pop cx ;get second parameter off stack
 push cx ;put them back on stack
 push bx
 pushf ;save flags for iret
 mov dx,cs ;get code segment
 push dx ;save code segment for return address
 push ax ;push saved return address offset
 push ax ;save everything
 push bx
 push cx
 push dx
 push es
 push ds
 push si
 push di
 push bp
 sti
 mov ax,sp ;stack pointer parameter
 push cx ;second parameter passed
 push bx ;first parameter passed
 mov bx,ss
 push bx ;far pointer to stack, parameter passed
 push ax
 mov ax,_gl_tptr ;push hidden pointer for C++ object
 push ax
;stack is now set up for call to task_control object task_switch
 cli ;turn off interrupts for task switch
 call __task_control_task_switch
;no need to clean up the stack because it will change
 sti
 mov ss,dx ;new ss returned in dx
 mov sp,ax ;new sp returned in ax
 pop bp ;restore registers
 pop di
 pop si
 pop ds
 pop es
 pop dx
 pop cx
 pop bx
 pop ax
 iret
_save_image endp

 end







[LISTING FOUR]

/********************************************/
/* TASKDEMO.HPP */
/* by Tom Green */
/********************************************/

/* this file is a demonstration of how to use the C++ multi-tasking */
/* kernel. 5 tasks are run and the various means of task switching */
/* and communication are shown */

/* you must have the Zortech C++ compiler version 1.5 and linker and */
/* Microsoft MASM 5.xx to compile this code. */
/* type "ztc taskdemo task timer" and the ztc.com will take */
/* care of compiling, assembling, and linking */

#include <stdio.h>
#include <disp.h>
#include "task.hpp"

void task0(void);
void task1(void);
void task2(void);
void task3(void);
void task4(void);

/* our task_control object (just 1 please) */
task_control tasker;

void main(void)
{
 /* task objects */
 task t0((func_ptr)task0,1024);
 task t1((func_ptr)task1,1024);
 task t2((func_ptr)task2,1024);
 task t3((func_ptr)task3,1024);
 task t4((func_ptr)task4,1024);

 /* add task objects to our task_control object ready q */
 tasker.add_new_task(&t0);
 tasker.add_new_task(&t1);
 tasker.add_new_task(&t2);
 tasker.add_new_task(&t3);
 tasker.add_new_task(&t4);

 /* use zortech display package */
 disp_open();
 disp_move(0,0);
 disp_eeop();

 /* start tasks up and wait for them to finish */
 tasker.start_tasks();

 disp_move(0,0);
 disp_eeop();
 disp_close();
}

static unsigned long counter[]={0L,0L,0L,0L,0L};

static signal sig;

/* task 0 prints the values of the counters for the other 4 tasks. */
/* lock is used to prevent task switching while the screen is being */
/* updated. when the task is finished, block is called to transfer */
/* control to the next task on the ready q */

void task0(void)
{
 while(1){
 /* disable task switching */
 tasker.lock();
 disp_move(5,10);
 disp_printf("Task 1 %lx",counter[1]);
 disp_move(5,50);
 disp_printf("Task 2 %lx",counter[2]);
 disp_move(15,10);
 disp_printf("Task 3 %lx",counter[3]);
 disp_move(15,50);
 disp_printf("Task 4 %lx",counter[4]);
 /* if key pressed then stop the kernel and return */
 if(kbhit())
 tasker.stop_tasks();
 /* re-enable task switching */
 tasker.unlock();
 /* let next task run */
 tasker.block();
 }
}

/* tasks 1 and 2 just update counters. these tasks will run until */
/* a timer interrupt occurs, so they get a very large chunk of time */
/* to run, so the counters increase rapidly */

void task1(void)
{
 while(1){
 counter[1]++;
 }
}

void task2(void)
{
 while(1){
 counter[2]++;
 }
}

/* task 3 waits for a signal from task 4 each time the counter is */
/* incremented. when a task waits, it is put on a signal q and the */
/* next task on the ready q is run. this means task 3 and 4 counters */
/* will increment very slowly. in task 4 when a signal is sent, the */
/* task signal q is checked for a task to put on the ready q. the task */
/* sending the signal is then placed on the ready q */

void task3(void)
{
 while(1){
 counter[3]++;

 /* wait for a signal from task 4 */
 tasker.wait(&sig);
 }
}

void task4(void)
{
 while(1){
 counter[4]++;
 /* send signal to task 3 */
 tasker.send(&sig);
 }
}

















































February, 1989
A TIMED EVENT NETWORK SCHEDULER IN FORTH


Forth is particularly well suited to the development of real-time kernels




Gregory R.S. Ilg and R.J. Brown


Gregory Ilg and R.J. Brown are consultants involved in the development of
microprocessor- and minicomputer-based real-time systems. Mr. Brown can be
reached at Elijah Laboratories Inc., P.O. Box 833, Warsaw, KY 41095. Mr. Ilg
can be reached at Computer Strategies, Inc., P.O. Box 180, Evon Lake, OH
44012.


The Timed Event Network Scheduler (TENS) is a scheduler we have developed that
is well suited for real-time process control and industrial automation
applications. One of its main features is that it has its own high-level
language, which is an extension of Forth, in which dependencies among
different events are specified. When any action or time duration in the system
is altered, other modifications are not necessary in order to maintain and
preserve these dependencies. At compilation time, the critical path through
the network and its path length (measured in units of time) are displayed.
Because of this approach, you can easily see which processes need to be
streamlined in order to increase system throughput. The TENS paradigm was
inspired by the critical path method (CPM) scheduling techniques used to
manage large projects.
We implemented TENS using Laboratory Microsystems' UR/FORTH, Version 1.03,
under PC-DOS 3.3 running on several XT- and AT-compatible computers. So far,
TENS has been used to control the time sequencing of the fluid and pneumatic
valves in a piece of complex medical equipment. Our intent was for a
mechanical engineer to be able to tune the system timing without the aid of a
software engineer. The results were quite favorable. The mechanical engineer
quickly grasped the concepts of network sequencing and was able to develop
some of the network topology without software engineering assistance and was
easily able to tune the timing of the network to meet stringent design
requirements.
This article describes the TENS system and provides examples of its use. The
programs shown in Listings One and Two, page 92, do, however, make use of
several Forth packages that, because of their length, are not reproduced here.
The packages are included in the companion DDJ source disk and in the DDJ
Forum on CompuServe. The TENS system can be used for any purpose as long as
credit is given to Elijah Laboratories.


Description of TENS


Each node in a TENS network has associated with it an action to be performed
when the node is first activated, a delay time, and an action to be performed
after the delay time. The node is then exited. Both actions behave like normal
Forth colon definitions and may use any number of normal Forth words.
Each node knows its immediate successors and predecessors in the network and
cannot be activated until all its predecessors have exited. When a node exits,
it informs all its successors that it has done so. Each node keeps track of
which of its predecessors have already exited by means of a flag for each
predecessor. When notified, this node examines all these flags to determine if
it is now eligible to run. When the conditions for node activation have been
met, all the flags are cleared before the node is activated.
Each TENS network is distinguished by a single entry and a single exit, called
the head and tail, respectively. Between the network head and tail there may
be zero or more nodes, subject to the constraint that there are no cycles. (A
cycle would exist in the situation that an ultimate successor of a node could
be that node's predecessor.) A network is known by its head, and typically the
head has a name that appears as a Forth word in the dictionary. To run such a
network, you execute its name as you would any other Forth word. These
networks can in turn be used to construct larger networks.


A TENS Example


Let's suppose TENS were used in the development of the Egg-Master vending
machine illustrated in Leo Brodie's book entitled Thinking FORTH. (Figure 1 is
an adoption of the concept.) One of the networks it might use could be
represented as shown in Figure 2 and Listing One. This network is responsible
for handling the omelette selection available on the Egg-Master. Similar
networks would handle the other selections. The word EGG_MASTER in Listing One
handles the selection of which network to run and would be hooked to the
power-up vector of the machine. The network in Figure 2 , whose head is
Start-Cook-Omelette, runs the nodes Preheat-Griddle and Mix-Omelette in
parallel. The node Pour-Mixture waits for both the nodes Mix-Omelette and
Preheat-Griddle to complete before it runs. This is obviously necessary
because the ingredients must be mixed and the griddle must be hot before the
mixture is poured onto the griddle.
The node Mix-Omelette' in the Cook-Omelette network is itself a network that
is a subnetwork of Cook-Omelette, but it can also be a subnetwork of other
networks (such as a Cook-Scrambled network, should the Egg-Master Supreme ever
make it out of the R&D lab). This is the network analogy to the
linear-threaded concept of factoring. Because factoring is fundamental to
writing good Forth code, we felt it was important to carry this practice into
the parallel data-flow paradigm. TENS permits factoring of networks, thereby
increasing their reusability and test-ability.


Run-Time Data Structures


Each network is implemented by a network-head data structure that contains
pointers to the initial and terminal nodes of the network. The network head
also provides storage for the critical path length.
Each node is implemented as a compound data structure. It is composed of a
fixed area containing the CFAs of the entry and exit action words, the delay
time, its path length from the start node, a pointer to a linked list of
successors, and a variable-length array of pointers to its predecessors. This
array is terminated with a NULL pointer. Each node in the linked list of
successors is implemented as a pair of cells. The first cell points to the
next node in the linked list, and the second cell points to the successor node
in the TENS network (see Figure 3).


Run-Time Processing


Each TENS network is created by a CREATE DOES> word whose DOES> clause
references the word TENS (see Listing Two). This word sets a trapdoor with
CATCH; entry-action throws back to it when the entire network has finished
executing. After this trapdoor is established, the timer is started and a
pointer to the first node is passed to entry-action, which throws back to the
trapdoor if the pointer is NULL. If it is not NULL, then the entry-action CFA
is performed. Next, this node is put on the timer queue for the required delay
time.
The timer queue is managed in accord with "An Efficient Algorithm for Large
Priority Queues," by R. J. Brown (DDJ, June 1987). The priority is the
dispatch time, which causes each node to be removed from the queue when its
dispatch time occurs. As the flow of control proceeds through the network,
there will be many nodes on the timer queue.
When a node is dispatched from the timer queue, its pointer is passed to
exit-action, which causes that node's Exit-Action to be performed. At this
point, the node is exiting, and its successors are notified. If there are no
successors, then the entire network has finished running and the trapdoor is
thrown back to. As each node receives notification, it checks to see if all
its other predecessors have also exited. If they have, then that node is
activated. This activation and exiting process is applied iteratively as the
TENS scheduler traverses the network. When subnetworks are present, recursion
occurs because more than one network is active at a time.
Although TENS implements a data-flow paradigm for parallelism, it is not a
multitasking system. All of the parallelism results from having active nodes
on the timer queue waiting for the delay interval between their Entry-Action
and their Exit-Action. For this reason, these actions should have negligible
processing time relative to the delay time.


Compile-Time Processing


During compilation the run-time data structures must be built (see Listing
Two). The network head is instantiated by the defining word NETWORK, which
leaves a pointer to this head on the stack for use by END-NETWORK.
Each node in the network can know only its predecessors at compile time
because Forth does not generally allow forward references. After all nodes of
the network have been compiled, the word END-NETWORK lexically delimits the
entire network and causes additional compile-time processing to occur.
The first (and the only essential) post-processing operation is the generation
of successor links. This is performed by a depth-first traversal of the
network, starting at the tail and penetrating toward the head. Each pair of
cells on a successor list is allocated as it is needed.
Once the successor links have been established, the length from the head to
each node is computed by a similar depth-first search and then is saved with
that node. Therefore, the critical path length will be found in the tail node.

The names of the nodes on the critical path are displayed starting with the
tail. By choosing the predecessor with the largest path length, we are able to
stay on the critical path. As each node on the critical path is encountered,
its name (as stored in the dictionary header) is printed on the screen.
As a result of this compile-time post-processing, not only is the network
linked in both directions but also the length of the critical path is computed
and the names of the nodes it contains are displayed. The length of the
critical path is a useful statistic for the designers because only those nodes
that are actually on the critical path can have an effect on the critical path
length. Thus, designers need direct their tuning efforts only to these nodes
-- by changing the delay time of any node, they can affect the time required
to run the entire network. When the duration of a node is changed, it is not
necessary to change the times of any other nodes to keep the process in step.
This is the beauty of the network-scheduling data-flow paradigm.
The display of the critical path and its length, together with the cell in
each node to hold that node's path length, can be removed after development to
produce a cross-compiled system without the extra overhead if so desired.


Conclusion


By having nontrivial processing done at compile time, a large amount of work
is factored into the compilation of the network instead of into its execution.
The Forth language is particularly well suited to the implementation of this
approach because the full facilities of the language are available not only to
execute the program but also to build that same program. Most other
higher-level languages have no such facility. The C language has a limited
compile-time processing facility. Most good assemblers have a macro
capability, but these are separate languages distinct from the language that
specifies the run-time behavior of the program. Lisp is the only other
well-known language that has as good a compile-time processing facility;
however, Lisp is generally too slow to be effective in real-time applications
on microprocessors. Forth is deliberately designed to provide real-time
performance on micros.


Bibliography


Brodie, Leo. Thinking FORTH. Engelwood Cliffs, N.J.: Prentice Hall, 1984.
Brown, Robert Jay. "An Efficient Algorithm for Large Priority Queues." DDJ 128
(June 1987).

_A Timed Event Network Scheduler in Forth_
by Gregory Ilg and R.J. Brown

[NOTE: FORTH SCREENS ACCOMPANY THESE LISTINGS]


[LISTING ONE]

\ Hypothetical Network To Handle Omelette Selection On
\ Brodie's Egg-Master


 CONSULT TENS \ Timed Event Network Scheduler

184 EQU ticks/10secs \ system clock time constant

: MS ( miliseconds -- ticks ) \ time units conversion
 184 10000 */ ;

: sec ( seconds -- ticks )
 184 10 */ ;

: min ( minutes -- ticks )
 60 * sec ;


( Define stubs for un-implemented words... )

 STUB[ Break-Egg seasoning-valve open
 close mixer on
 off griddle pour-valve
 wait-for-coins Cook-Fried Cook-Poached
 Cook-Hard-Boiled Cook-Benedict ]


( Define buttons as an enumeration set... )

 1 ENUM[ Fried-button
 Poached-button
 Hard-Boiled-button
 Benedict-button
 Omelette-button ]


: read-a-button Omelette-button ; \ always choose "omelette"

: None ; IMMEDIATE ( noise word to improve readability )



NETWORK Mix-Omelette \ subnetwork also used
 \ in fancier omelettes

 NODE Start-Mix-Omelette \ dummy for single entry
 Predecessors NIL
 Entry-Action None
 Delay 0 sec
 Exit-Action None
 END-NODE


\ Wrap up subnetwork Break-Egg. Break-Egg not detailed here

 NODE Crack-Egg
 Predecessors Start-Mix-Omelette
 Entry-Action Break-Egg
 Delay 0 sec
 Exit-Action None
 END-NODE


\ concurrent with Crack-Egg

 NODE Add-Seasoning
 Predecessors Start-Mix-Omelette
 Entry-Action seasoning-valve open
 Delay 100 MS
 Exit-Action seasoning-valve close
 END-NODE


 NODE Blend
 Predecessors Crack-Egg
 Add-Seasoning
 Entry-Action mixer on
 Delay 3 sec
 Exit-Action mixer off
 END-NODE


END-NETWORK ( Mix-Omelette )




NETWORK Cook-Omelette \ net head to process...
 \ ...Omelette-button

 NODE Start-Cook-Omelette \ dummy for single entry
 Predecessors NIL
 Entry-Action None
 Delay 0 sec

 Exit-Action None
 END-NODE


 NODE Preheat-Griddle
 Predecessors Start-Cook-Omelette
 Entry-Action griddle on
 Delay 30 sec
 Exit-Action None
 END-NODE


\ subnetwork...

 NODE Mix-Omelette'
 Predecessors Start-Cook-Omelette
 Entry-Action Mix-Omelette
 Delay 0 sec
 Exit-Action None
 END-NODE


 NODE Pour-Mixture
 Predecessors Preheat-Griddle
 Mix-Omelette'
 Entry-Action pour-valve open
 Delay 2 min
 Exit-Action pour-valve close
 griddle off
 END-NODE


END-NETWORK ( Cook-Omelette )



\ This word is hooked to the power-up vector on the
\ Egg-Master vending machine.


: EGG_MASTER ( -- )
 BEGIN wait-for-coins read-a-button CASE
 Fried-button OF Cook-Fried ENDOF
 Poached-button OF Cook-Poached ENDOF
 Hard-Boiled-button OF Cook-Hard-Boiled ENDOF
 Benedict-button OF Cook-Benedict ENDOF
 Omelette-button OF Cook-Omelette ENDOF
 ENDCASE AGAIN ;





[LISTING TWO]

\ TIMED EVENT NETWORK SCHEDULER
\ Copyright (c) 1988 Elijah Laboratories Inc.



( This package makes use of a number of programs that are
not detailed here. The word CONSULT is used to load these
packages. The packages are included in the companion DDJ
source disk. )


 CONSULT STRUC \ structure definitions
 CONSULT PRIQUE \ priority queue manager
 CONSULT MACROS \ for the evaluator
 CONSULT BALLS \ for backtracking control
 CONSULT XSHEETS \ transient storage words


 CREATE TQ NIL , \ the timer queue

 : nothing ; *MITT* SETQ nothing \ no default THROW handler
 6 INFLATE BALL tens.ball \ trap door for exit
 BALL te.ball \ ditto


( A timed event network is a linked data structure composed of
the following timed event nodes. )

sizeof pq struc TEnode \ a timed event node
1w TEnode word-1 \ execute before delay
1w TEnode delay \ # ticks to delay
1w TEnode path-length \ path length from start
1w TEnode word-2 \ execute after delay
1w TEnode ^Succ's \ ptr to successor list
0 TEnode Pred's \ start of predecessor list


( These words start and stop the timer cell, and fetch its
value, and add its value to the top-of-stack, thereby
converting a time interval into an absolute time value. )

 VARIABLE now \ the timer cell

: start-now ( -- ) -2 now ! now TICKER DROP ; \ start the timer

: stop-now ( -- ) now -TICKER ; \ stop the timer

: now@ ( -- #tics ) now @ NEGATE ; \ fetch the timer

: now+ ( delay -- time ) now@ + ; \ add timer value


( This word defines the processing that occurs when a node is
activated. This consists of performing the entry action
routine and enqueueing for the required delay time. )

: entry-action ( node -- ) \ start up a node
 >R \ save node pointer
 R@ NIL = IF T tens.ball THROW THEN \ if none, we're done!
 R@ BODY> >NAME CR .NAME \ display its name
 R@ word-1 PERFORM \ do pre-delay stuff
 R@ delay @ now+ R@ key ! \ figure dispatch time
 TQ R> pq-enque ; \ put on timer queue



( This word waits until the head of the timer queue is past
its dispatch time, and then dequeues the head element from the
timer queue for subsequent processing. )

: wait-till ( -- node ) \ take next node off queue
 TQ @ 0= ABORT" Empty timer queue "
 BEGIN TQ @ key @ now@ < UNTIL \ wait until its time
 TQ pq-deque \ then remove it
." DQ " ;


( The following structures are used in the word "notify". The
first is passed as a parameter block, and the second is SPREAD
as a SHEET. )

0 struc n.parm \ passed parameters...
1w n.parm n.pred \ ptr to predecessor
1w n.parm n.succ \ ptr to its successor

0 struc n.temp \ temporaries...
1w n.temp n.trig \ trigger indicator


( This word searches the predecessor list of a node for a
match with the cfa of a completing predecessor. When the match
is found, the low order bit of the predecessor address is set
as a flag to indicate that that predecessor has completed. If
all such flags are set, "n.trig" is set to trigger the node. )

: set-done-flag ( _n.temp -- _n.temp ) \ set pred's done fg
 AT #[ parm n.parm n.succ ]+ SH@ Pred's \ pt to succ's preds
 BEGIN DUP @ ?DUP WHILE \ for all preds
 AT #[ parm n.parm n.pred ]+ SH@ = \ we got a match?
 IF DUP 1 ! THEN \ yes, set done flag
 DUP @ AT n.trig SH@ AND \ figure new trigger
 AT n.trig SH! w+ REPEAT DROP ; \ save it & continue


( This word clears all the done flags that were set to trigger
the activation of a node. This word is performed just prior to
activating the node, thus preparing it for re-triggering at a
later time. )

: clear-done-flags \ clear succ's done flags
 AT #[ parm n.parm n.succ ]+ \ point to successor
 SH@ Pred's \ point to his predecessors
 BEGIN DUP @ ?DUP WHILE \ for all predecessors
 [ 1 NOT ]# AND OVER ! \ clear its done flag
 w+ REPEAT ; \ loop to end of list


( The "notify" word tells a successor to a node that one of
its predecessors has completed. If all of its predecessors
have completed, then that successor node is started. )

: notify ( pred succ -- ) \ notify a successor
 sizeof n.temp SPREAD \ make room for temps
 1 AT n.trig SH! \ cock the trigger

 set-done-flag \ set pred's done flag
 AT n.trig SH@ IF \ has succ been triggered?
 clear-done-flags \ yes, clear his done flags
 CRUSH NIP entry-action \ & start him up!
 ELSE CRUSH 2DROP THEN ; \ no, just return...


( This word is executed after a node has been removed from the
timer queue after waiting its required delay time. It causes
the exit action routine for that node to be performed, and then
notifies all the successors of that node that it has completed
execution. )

: exit-action ( node -- ) \ complete a node
 DUP word-2 PERFORM \ do after delay stuff
 DUP ^Succ's DUP @ NIL = \ point to successor chain
 IF T tens.ball THROW THEN \ if none, we're done!
 BEGIN @ ?DUP WHILE \ for all his successors
 2DUP w+ @ notify \ notify them he's done
 REPEAT DROP ; \ then clean up & exit


( This is the "Timed Event Network Scheduler", the entry point
for the DOES> word of a timed event network. Running such a
network is done by executing the name of the network, which
calls TENS with the address of the network list head. )

: TENS ( net -- ) \ run a timed event net
 tens.ball CATCH IF \ exit via trap door?
 DROP EXIT THEN \ yes, stop timer, exit
 start-now @ entry-action BEGIN \ no, start first node
 wait-till exit-action AGAIN ; \ wait, then finish it


( This word links a node into the successor lists of all of
its predecessors. After this has been done recursively for
all the successors of that node, ad infinitum, then the entire
event network will be linked both forwards and backwards. )

F: link-to-pred \ declare forward reference

: link-to-pred's ( node -- ) \ fix up forward lnks in network
 DUP Pred's >R \ point to it predecessor list
 BEGIN R@ @ ?DUP WHILE \ for all its predecessors...
 OVER SWAP link-to-pred \ link this node to its pred
 R> w+ >R REPEAT \ point to next pred ptr
 DROP R> DROP ; \ tidy up stacks

R: link-to-pred ( node pred -- ) \ ping pongs with word above
 DUP >R \ remember predecessor ptr
 ^Succ's \ point to successor ptr
 BEGIN DUP @ ?DUP WHILE \ for all successors...
 NIP DUP w+ @ 2 PICK = \ duplicate?
 IF R> 3DROP EXIT THEN \ yes, skip this node
 REPEAT \ until end of list
 HERE SWAP ! \ append new CONS cell
 NIL , \ CDR is NIL
 , \ CAR is successor node
 R> link-to-pred's ; \ fix his predecessors too!



( This word kicks off the action to reverse link the event
network by passing the address of the first node to the
recursive network traversal algorithm. It takes the address
of the network list head as its parameter. )

: fix-Succ's ( head -- ) \ generate successor lists
 w+ @ \ point to last node
 link-to-pred's ; \ link it to its predecessors


( This word recursively traverses the network accumulating the
maximum path length to each node. This length from the first
to last nodes is the critical path length, or network delay. )

: set-path-len ( ^node old-path-length -- )
 SWAP >R \ save this node's pointer
 R@ delay @ + \ add this delay to old len
 R@ path-length @ \ get current len
 MAX DUP \ new len is max of the two
 R@ path-length ! \ save new len
 R> Pred's >R \ point to pred ptrs list
 BEGIN R> DUP w+ >R @ ?DUP WHILE \ for all predecessors...
 OVER RECURSE REPEAT \ set their path length too
 DROP R> DROP ; \ clean up and exit


( This word calls the recursive network traversal routine to
compute the length of the critical path through the network.
The result is stored in the network's list head. )

: set-path-lengths ( nethead -- )
 DUP w+ @ \ point to last node
 0 \ initial path length is zero
 set-path-len \ recursively set path lengths
 DUP @ path-length @ \ get computed critical path length
 SWAP 2 w*+ ! ; \ save it in network head


( This word displays the name of a network and its critical
path length in milliseconds, It is used as a tuning aid. )

: .pathlen ( nethead -- ) \ display critical path length
 CR \ start on a new line
 ." Critical path of " \ indentify it
 DUP BODY> >NAME .NAME \ show network name
 ." is " \ more verbiage
 2 w*+ @ \ fetch length in ticks
 10 184 */ \ convert to seconds
 . \ display the number
 ." seconds long. " \ more verbiage
 CR ; \ and a new line


( This word displays the critical path on the screen at
compilation time to facilitate tuning. )

: .critpath ( nethead -- )

 ." Critical path is: " w+ @ \ point to last node
 BEGIN ?DUP WHILE \ for each node on critpath
 CR 4 SPACES \ indent to look pretty
 DUP BODY> >NAME .NAME \ display its name
 Pred's >R 0 0 BEGIN \ for each predecessor
 R> DUP w+ >R @ \ get a predecessor pointer
 ?DUP WHILE \ as long as they exist
 DUP path-length @ \ get its path length
 2 PICK OVER MAX \ take maximum of old & new
 OVER = IF 2SWAP THEN \ update max & ptr
 2DROP REPEAT R> 2DROP \ discard excess baggage
 REPEAT CR ; \ loop till done


( The Top-of-Stack is used by END-NETWORK to generate the
successor lists for all of the nodes in the network. For this
reason, the last node instantiated *MUST* be the unique
terminal node for the network. Likewise, the first node
instantiated *MUST* be the unique initial node for the
network. )

: END-NETWORK ( head first last -- )
 ROT >R SWAP \ save head, put first on top
 R@ ! \ save first in head
 R@ w+ ! \ save last in head
 R@ fix-Succ's \ generate successor lists
 R@ set-path-lengths \ compute critical path length
 R@ .pathlen \ display critical path length
 R> .critpath ; \ display critical path


( The following word is used to begin the definition of a
timed event network. The network is terminated by the word
END-NETWORK. )

: NETWORK ( -- ) \ begin a timed event network
 CREATE HERE \ give it a name and remember where
 NIL , NIL , \ initialize first & last pointers
 NIL NIL \ initialize 2 pointers on stack
 0 , \ initialize critical path length
 DOES> TENS ; \ runs the scheduler when called


( This is the format for declaring a node in a TENS network.

 NODE <node-name>
 Predecessor pred1 ... predn
 Entry-Action word1 ... wordn
 Delay n \ tics
 Exit-Action word1 ... wordn
 END-NODE
)


 NIL EQU ^NODE \ pointer to current node

: NODE ( -- ) \ define network step node
 CREATE \ make a dictionary header
 HERE EQU ^NODE \ remember where it is

 sizeof TEnode ALLOT \ make room for it
 0 ^NODE path-length ! \ initialize path length
 NIL ^NODE ^Succ's ! \ terminate successor chain
 DROP ?IF ^NODE \ set 1st
 ELSE ^NODE DUP THEN ; \ & last pointers


: Predecessors \ start predecessor list
 DUP \ insert cushion
 te.ball CATCH NIL = \ to end predecessor list
 IF BEGIN DEPTH >R EVAL \ mark stack & eval token
 DEPTH R> \ did eval return a value
 - 1 = IF , \ yes, store it as a pred
 ELSE \ no, more than one value?
 T ABORT" Illegal Predecessor " \ yes, abort with a msg!
 THEN \ no value, treat as comment
 AGAIN THEN \ build predecessor list
 DROP \ remove cushion
 CP @ ^NODE word-1 ! \ make entry action header
 HERE PFA, nest JMP, \ stuff cfa and pfa into it
 ] ; \ compile entry action


: Entry-Action NIL , T te.ball THROW ; \ end predecessor list


: Delay \ specify delay time
 COMPILE EXIT \ last word in entry action
 [COMPILE] [ \ set interpret state
 ; IMMEDIATE \ must run while compiling


: Exit-Action \ specify exit action
 ^NODE delay ! \ save delay time
 CP @ ^NODE word-2 ! \ make exit action header
 HERE PFA, nest JMP, \ stuff cfa & pfa into it
 ] ; \ force compile state


: END-NODE \ terminate node definition
 COMPILE EXIT \ last word in exit action
 [COMPILE] [ \ set interpret mode
 ; IMMEDIATE \ must run while compiling



















February, 1989
BENCHMARKING C STATEMENTS


Getting the most out of your C programs




David L. Fox


David has been developing programmer's tools for several years. He is the
chief scientist at Minimum Instruction Set Computer Inc. and can be reached at
P.O. Box 1528, Golden, CO 80402.


Programmers will expend considerable (sometimes excessive) effort to reduce
the execution times of their programs. Widely held maxims such as "pointers
are faster than array indexing" or "multiply is faster than divide" don't
always apply to a particular situation; any program might benefit from a
little tool-aided exploration and the optimization of any code consuming a
disproportionate share of CPU cycles. Most programmers go first to a profiler,
which measures the fractions of the total execution time consumed by various
parts of a program. Once the profiler has identified the time critical
portions of code, the programmer can fine tune those parts with sbench.
Sbench is a program generator. It reads a list of C statements and creates a
test program to measure the execution time of each statement individually.
This allows a quick and easy comparison of the performances of various
alternative methods of accomplishing a given task. Sbench also makes it much
easier to explore the performance of a compiler in a variety of situations.
Sbench is of primary interest to programmers of real-time systems and others
concerned with optimizing small regions of code. Limitations on the
performance of a program imposed by the execution of a few lines of bottleneck
code are not unusual. Furthermore, it is not always obvious where and when
such bottlenecks occur.


Sbench


I was inspired to write sbench by Ralph E. Griswold's "Benchmarking Icon
Expressions," (19 October 1987, The Icon Project, University of Arizona),
which describes a similar tool for the Icon programming language. Icon
presents a programmer with even more operators and alternatives than C, but
the question of which statement would be faster arises often enough in C to
warrant a tool to help answer it.
In order to be really useful a C statement benchmarking program must satisfy a
number of requirements. First, it should be easy to use; that is, the program
should require a minimum from the programmer. In sbench, often the only
information required is a list of the statements to be compared. Sbench
generates a test program from such a list and compiles, links, and executes
it.
C programs, however, contain more than simple executable statements. In order
to compare various ways of reading data from a file, for example, buffers and
file descriptors must be declared and the file must be opened before any input
statements can be executed. Some mechanism must be provided to allow for
declaring variables (both global and local), including other files, and
executing the initialization code required for the statements to be tested.
The most important requirement a C statement benchmarking program must
satisfy, of course, is that it provided some way to measure the execution time
of the statements under test. The draft ANSI C standard contains the library
function clock(), which seems ideal for this purpose. clock() returns an
implementation's best approximation of the processor time used by a program.
Unfortunately, clock() is not universally available, even on compilers that
claim to support ANSI C. In most MS-DOS implementations, the resolution of the
clock function is 18.2 ticks per second, inadequate for timing individual
statements that may take only a few microseconds to execute.
sbench solves both problems by providing a customized version of clock() with
much higher timing resolution. Sbench's timing function can be implemented on
the IBM PC and compatibles by reprogramming the 8253 timer chip. Byron
Sheppard's "High-Performance Software Analysis on the IBM PC" (Byte, January
1987) details this programming of the 8253 chip. Note that this solution
requires a high degree of hardware compatibility.
For other systems with low-resolution clock() functions, the statement under
test can be enclosed in a loop and repeated many times. This increases the
total time required to a value that can be measured with a low-resolution
timer. Putting the statement under test inside a loop is not an ideal solution
because it can lead to multiple side effects and increase the confusion an
optimizing compiler can create. Sbench provides for both the higher-resolution
version of clock() and the repeating loop solutions to the timing problem.
The final requirement of a C benchmarking tool is that its generated test
programs should run on a wide variety of systems and compilers. The emerging
ANSI C standard will make portability much easier to achieve in the future,
but unfortunately, compiler support of the draft standard is uneven. Sbench's
solution is to try to generate programs that avoid system specific features
and provide for various implementations of C by use of #if conditional
compilation directives. ANSI C is used in the version of sbench presented
here, but "old style" C (K&R) alternatives are provided too. I have compiled
the generated test programs with four different MS-DOS C compilers. I do not
have access to other hardware, so portability to non-MS-DOS systems is
untested.
The code for sbench itself does not maintain the level of compiler
independence the programs it generates do. The sbench code uses several
features introduced in the ANSI draft standard and not supported by most K&R C
compilers. In particular, function prototypes and concatenation of adjacent
string constants are used throughout the code. I have not added the
conditional compilation switches needed to allow compilation with either K & R
or ANSI style of compiler because the switches would have increased the size
and decreased the readability of the code.
Benchmark programs rarely do anything more than print times, and consequently,
they can be invalidated by sufficiently smart optimizing compilers. Sbench is
susceptible to this problem. One solution is not to use the optimizer for the
test code. This has the severe drawback, however, that the times will not
correspond to the times in the real, optimized code. The best solution is to
include enough realistic code in your test that the optimizer will make the
same improvements it would in the actual application.
There are also some tricks you can use to prevent the optimizer from
completely removing your test code. Few microcomputer C compilers optimize
across function calls, so passing the results of any calculations to an
external function will usually convince an optimizer that the results are
being used. Also helpful in preventing the optimizer from removing the loops
is to use the loop index variable in any calculation. Finally, note that some
optimizers make fewer changes in code using global variables than in code with
local variables.


Sbench Code


The code for sbench, the statement benchmark generator, is in Listing One,
page 98. The operation of the program is quite simple -- sbench collects
statements from the input into three linked lists, one for global
declarations, one for local declarations, and one for the statements to be
timed. Global lines are copied into the beginning of the generated test
program, external to any function. Local lines are copied into the main
function of the test program before any executable statements. Finally, the
statements to be timed are copied into the test program. The three elements
are combined with fragments of code to make a complete program. The additional
code fragments required to complete the program are given by the pgm* string
variables defined in lines 70 to 95.
Global lines must occur first in the input and are separated from local lines
by a line beginning with %%. Local lines must precede any test statements and
are indicated by a single initial % character. Any statement may be split over
several physical lines by the usual C convention of ending lines to be
continued with a \ character.
Once the test program is created, sbench attempts to compile and execute it
with the system commands in lines 215 to 225. The programmer can use a
command-line option (-g) to prevent compilation of the test program. Unless
the programmer gives it another name with the -f option, the test program is
called statbm.c. Sbench will not delete the test program so it may be
(re)compiled if the programmer wants to compare various compilers and/or
options.
Listing Two on page 101, contains the code for the routine getline( ), which
reads a line of arbitrary length and combines continued physical lines into
one logical line. Listing Three, page 101, contains the command-line option
parsing routine getopt( ). getopt( ) is a Unix function, available in only
some micro-computer C implementations. Because getopt() is not specified in
the ANSI draft standard, I have included it here.
clock( ), the microsecond resolution clock function, is given in Listing Four,
page 102. clock() is very hardware and compiler dependent. The code in Listing
Four should be portable to IBM PC compatible systems with C compilers that
have the ability to control interrupts and access hardware ports. Listing Four
can be used as a guide for implementation on other systems. Its function has
been described in detail in Byron Sheppard's Byte article.


Results


Listing Five(a), page 103, is an example of an input file for sbench. No
global variables are used, so the global section is empty except for a
comment. Note that in addition to variable declarations, the global section
could contain #include commands, definitions of functions called by the test
code, or any other valid C code. Several local variables are declared in lines
3 to 6. Executable code to perform required initialization could also be
placed in the local section.
The first two test statements (Listing Five's lines 7 and 8) add two float and
double variables, respectively. The results in Listing Five(b) were obtained
using floating point software with the Zortech C compiler. They show clearly
why float variables should not be used when speed is important. When the
optimizer is used in compiling the test code, the results in Listing Five(c)
are obtained. The execution time appears to have been reduced to almost zero
-- the optimizer has recognized that the results of the additions are not used
anywhere and has completely removed all the code generated by the additions.
Inspection of the object code confirms this. The apparent results of 1 to 2
microseconds in Five(b) are due to just round off error and uncertainty in the
timing function.
The remainder of the test input compares using subscripts and pointers for
initializing and copying an array. Each statement tested is a for loop split
into two or more lines to illustrate the use of the continuation character (
\). Using pointers for the unoptimized code is slightly faster for
initializing and copying. The optimizer removes the advantage in the
initialization loop but increases the advantage in the copy loop. The
conclusion to be drawn from this is that micro performance will vary
substantially depending on the exact details of the code. When tuning code for
maximum performance, you have to test the specific code for your application.
Sbench makes it easy to try a large number of variations.


Further


Like all tools, sbench doesn't do everything, and there are a few ways in
which it could be improved or extended. One simple improvement would be to
provide a mechanism for passing options from sbench's command line through to
the compiler and/or to the test program itself when compiling the test
program. Another would be to extend the program to generate test programs in
other languages by providing alternate definitions of the code fragments in
the pgm*strings --a fairly easy improvement to make. Finally, note that the
idea of using programs to generate or manipulate other programs is extremely
powerful. Code similar to sbench's could be used to insert commands in a test
program for profiling by routine, block, or line.



A Caution


While the fine tuning encouraged by this tool has its place, don't get carried
away by the notion of speed. Faster is not always better; readability,
maintainability, and portability are at least as important as speed for most
programs. As always, the programmer must weigh the importance of conflicting
requirements.

_Benchmarking C Statements_
by David Fox


[LISTING ONE]
 1 /* sbench.c -- C statement benchmark generator. */
 2 /* Released to the public domain by David L. Fox, 1988. */
 3
 4 /* This program will read a list of C statements and generate a test
 5 program to time the execution of each of the statements in the list.
 6 The list should have the form:
 7 zero or more global declarations
 8 %%
 9 %zero or more local declarations or statements
 10 one or more test statements
 11 The global and local declarations and statements are not timed.
 12 Every local statement must have a % in column 1. Every test statement
 13 should be on one logical line. Physical lines ending with \ are
 14 continued, as in C, so long logical lines may be split.
 15
 16 The program is run with a command of the form:
 17 sbench [ -cx ] [ -f name ] [ -n xx ] [ -o ] [ -g ] [ files ... ]
 18 If no input files are given sbench reads the standard input.
 19 Available options are:
 20 -c x Select compiler: x=a Aztec, x=d Datalight,
 21 x=e Ecosoft, x=t Turbo, x=z Zortech
 22 -f name Write generated program to file name.
 23 -n xx Execute each statement xx times.
 24 -o Optimize when compiling benchmark.
 25 -g Generate test program only, do not try to compile.
 26 */
 27 #include <stdio.h>
 28 #include <stdlib.h>
 29 #include <string.h>
 30 #include <ctype.h>
 31
 32 #define OUTFNAME "statbm.c" /* Default name of generated program. */
 33
 34 struct listel { /* One element in a linked list of lines. */
 35 struct listel *next; /* Link to next element. */
 36 char *line; /* Pointer to the text of the line. */
 37 };
 38 struct llist {
 39 struct listel *head; /* Head of the list. */
 40 struct listel *tail; /* Tail of the list. */
 41 };
 42
 43 /* The following enum type is used to identify the compilers used. */
 44 enum cctag { aztec = 'a', datalight = 'd', ecosoft = 'e',
 45 turboc = 't', zortech = 'z'};
 46 struct cmdline { /* Contains parts of command lines for compilers. */
 47 enum cctag tag; /* Identifies compiler. */
 48 char *cmd; /* Beginning of command line. */

 49 char *optimize; /* Command line option to invoke optimizer. */
 50 char *lib; /* Libraries added to end of command line. */
 51 /* The libraries named should contain the high resolution */
 52 /* clock() function if it is used. The exact contents of the */
 53 /* lib strings will depend on how your hard disk is organized. */
 54 } compilers[] = { /* Compilers[0] is the default. */
 55 { aztec, "c -lx -lm -dMu_CLOCK", "+f", "" },
 56 { datalight, "dlc -dMu_CLOCK", "-o", "\\lib\\dlc\\xs.lib" },
 57 { ecosoft, "ecc -dMu_CLOCK -lecox", "", "" },
 58 { turboc, "tcc -dMu_CLOCK", "-G", "\\lib\\tc\\xs.lib" },
 59 { zortech, "ztc -dMu_CLOCK", "-o", "\\lib\\ztc\\xs.lib" }
 60 };
 61
 62 struct cmdoptns { /* Information extracted from command line. */
 63 unsigned int ntimes; /* Repeat count for SUT. */
 64 char *outfname; /* Name of output file. */
 65 int doopt; /* Flag, non-zero optimizes test program. */
 66 struct cmdline *compiler; /* Pointer to command line details. */
 67 } options;
 68
 69 /* These strings form parts of the generated program. */
 70 char *pgmincl = "#include\t<stdio.h>\n#include\t<time.h>\n\n";
 71 char *pgmhdr = "#ifdef\tMu_CLOCK\n" /* Using high-res clock()? */
 72 "#undef\tCLK_TCK\n#define\tCLK_TCK\t1000000L\n#endif\n"
 73 "#ifdef\t__STDC__\n" /* __STDC__ indicates an ANSI */
 74 "int main(int argc, char **argv) {\n" /* conforming implementation */
 75 "\tvoid sb_dummy(unsigned int);\n" /* with prototypes. */
 76 "#else\n" /* otherwise use K&R style function definitions. */
 77 "int main(argc, argv)\nint argc;\nchar **argv; {\n\tint sb_dummy();\n"
 78 "#endif\n\tclock_t sb_t0, sb_time, sb_empty;\n"
 79 "\tunsigned int sb_index, sb_nexpr, sb_niter;\n";
 80 char *pgmend =
 81 "\treturn 0;\n}\n#ifdef\t__STDC__\nvoid sb_dummy(unsigned int i) {\n"
 82 "#else\nint sb_dummy(i)\nunsigned int i; {\n#endif\n\treturn;\n}\n";
 83 char *pgminit = "clock();\n" /* Start clock, avoid first call overhead. */
 84 "sb_nexpr = 0;\nsb_niter = %u;\n";
 85 char *pgmstart = "sb_t0 = clock();\n";
 86 char *pgmloop = "for (sb_index=0; sb_index < sb_niter; ++sb_index) {\n"
 87 "sb_dummy(sb_index);\n"; /* Keep optimizers from gutting loop. */
 88 char *pgmnoloop = "{\n"; /* Put expression to be timed in a block. */
 89 char *pgmempty = "}\nsb_empty = clock() - sb_t0;\n"; /* Time empty stmt */
 90 char *pgmstop = "}\nsb_time = clock() - sb_t0;\n" /* Print results. */
 91 "printf(\"Statement %u required %6.1f microseconds to execute, "
 92 "%u iteration%s timed.\\n\",\n++sb_nexpr,"
 93 "(((double)(sb_time-sb_empty))*(1000000./CLK_TCK))/sb_niter, sb_niter,"
 94 "sb_niter > 1 ? \"s\" : \"\");\n";
 95
 96 /* Function prototypes. */
 97 void docmdline(int argc, char **argv, struct cmdoptns *optptr);
 98 char *getline(FILE *);
 99 int getopt(int argc, char *argv[], char *);
100 void lladd(struct llist *last, char *str);
101 void outerr(void);
102 void writelines(struct llist *list, FILE *outfile);
103 void usage(void);
104
105 /* External variables communicate with getopt(). */
106 extern int optind; /* Index of next command line argument in argv. */
107 extern char *optarg; /* Argument for a command line option (-o arg). */

108 int
109 main(int argc, char **argv) {
110 char cmdbuf[256], *str;
111 FILE *infile, *outfile;
112 struct llist *llp,
113 globall, /* Global declaration lines. */
114 locall, /* Local declaration lines. */
115 testl; /* Lines to be tested. */
116 struct listel *lep;
117
118 /* Set up default values */
119 infile = stdin;
120 options.outfname = OUTFNAME;
121 options.ntimes = 1;
122 options.doopt = 0; /* Don't optimize test code. */
123 options.compiler = &compilers[0];
124 docmdline(argc, argv, &options); /* Process command line options. */
125
126 globall.head = globall.tail = locall.head = locall.tail =
127 testl.head = testl.tail = NULL;
128
129 /* Once through the loop for each input file. */
130 do {
131 if (optind < argc) {
132 /* Try to open a file from the command line. */
133 if ((infile = fopen(argv[optind], "r")) == NULL) {
134 perror(argv[optind]);
135 fprintf(stderr, "Failed to open input file\n");
136 continue; /* Try the next file. */
137 }
138 }
139 else {
140 /* There are are no files on the command line, */
141 /* use the standard input. */
142 infile = stdin;
143 if (isatty(fileno(stdin)))
144 fprintf(stderr, "Enter list of statements:\n");
145 }
146
147 /* Now read the file and append its contents to
148 a linked list of lines. */
149 llp = &globall; /* First section contains global statements. */
150 while ((str = getline(infile)) != NULL) {
151 if (strncmp(str, "%%", 2) == 0)
152 llp = &testl; /* Found end of global section. */
153 else if (*str == '%')
154 lladd(&locall, str+1); /* This is a local declaration. */
155 else lladd(llp, str);
156 }
157 if (ferror(infile)) { /* Getline returns NULL on error or EOF. */
158 perror(argv[optind]);
159 fprintf(stderr, "Read error\n");
160 exit(1);
161 }
162
163 if (fclose(infile)) {
164 perror(argv[optind]);
165 fprintf(stderr, "Error closing input file!\n");
166 exit(1);

167 }
168 } while (++optind < argc);
169
170 /* Done with input, begin output phase. */
171 if ((outfile = fopen(options.outfname, "w")) == NULL) {
172 perror(options.outfname);
173 fprintf(stderr, "Can't create output file\n");
174 exit(1);
175 }
176
177 /* Select and write the pieces of the generated program. */
178 if (fprintf(outfile, "%s", pgmincl) < 0) outerr();
179 writelines(&globall, outfile); /* Write global lines. */
180
181 if (fprintf(outfile, "%s", pgmhdr) < 0) outerr(); /* Main program. */
182 writelines(&locall, outfile); /* Write local lines. */
183
184 /* Write code to initialize variables. */
185 if (fprintf(outfile, pgminit, options.ntimes) < 0) outerr();
186
187 /* Write code to time execution of an empty expression. */
188 if (fprintf(outfile, "%s", pgmstart) < 0) outerr();
189
190 if (options.ntimes > 1) { /* Need looping code? */
191 if (fprintf(outfile, "%s", pgmloop) < 0) outerr();
192 }
193 else {
194 if (fprintf(outfile, "%s", pgmnoloop) < 0) outerr();
195 }
196 if (fprintf(outfile, "%s", pgmempty) < 0) outerr();
197
198 /* Write code to time execution of statements. */
199 for (lep = testl.head; lep; lep = lep->next) {
200 if (fprintf(outfile, "%s", pgmstart) < 0) outerr();
201 if (options.ntimes > 1) { /* Need looping code? */
202 if (fprintf(outfile, "%s", pgmloop) < 0) outerr();
203 }
204 else {
205 if (fprintf(outfile, "%s", pgmnoloop) < 0) outerr();
206 }
207 /* Here is where the expression is written into the program. */
208 if (fprintf(outfile, "%s", lep->line) < 0) outerr();
209
210 if (fprintf(outfile, "%s", pgmstop) < 0) outerr(); /* Output */
211 }
212 if (fprintf(outfile, "%s", pgmend) < 0) outerr(); /* Wrap up. */
213 if (fclose(outfile)) outerr();
214
215 if (options.compiler != NULL) {
216 /* Compile and execute test program. */
217 sprintf(cmdbuf,"%s %s %s %s", options.compiler->cmd,
218 options.doopt ? options.compiler->optimize : "",
219 options.outfname, options.compiler->lib);
220 if (system(cmdbuf)) { /* MS-DOS doesn't return error codes. */
221 fprintf(stderr, "Compilation error\n");
222 exit(1);
223 }
224 *strchr(options.outfname, '.') = '\0'; /* Chop off ".exe" */
225 system(options.outfname);

226 }
227 return 0;
228 }
229
230 /* lladd -- Add a new line to the tail of a linked list. */
231 void
232 lladd(struct llist *list, char *str) {
233 struct listel *lep;
234
235 /* Create a new list structure. */
236 if ((lep = (struct listel *)malloc(sizeof(struct listel))) == NULL) {
237 fprintf(stderr, "Not enough memory\n");
238 exit(1);
239 }
240 lep->line = str; /* Add the line to it */
241 lep->next = NULL; /* and mark it as the tail. */
242 if (list->tail != NULL)
243 list->tail->next = lep; /* Link it to the old tail. */
244 list->tail = lep; /* Make it the new tail. */
245 if (list->head == NULL)
246 list->head = lep; /* First element is head. */
247 return;
248 }
249
250 /* docmdline -- Extract options from command line. */
251 void
252 docmdline(int argc, char **argv, struct cmdoptns *optptr) {
253 int c, i;
254 enum cctag tag;
255
256 while ((c = getopt(argc, argv, "c:f:n:og")) != EOF) {
257 switch(c) {
258 case 'c': /* Select compiler. */
259 tag = tolower(*optarg);
260 for (i = 0; i < sizeof(compilers)/sizeof(compilers[0]); ++i) {
261 if (tag == compilers[i].tag) { /* Is this the one? */
262 optptr->compiler = &compilers[i];
263 break;
264 }
265 }
266 if (i >= sizeof(compilers)/sizeof(compilers[0])) {
267 /* Didn't find compiler. */
268 fprintf(stderr, "Unknown compiler: %s\n", optarg);
269 usage();
270 }
271 break;
272 case 'f': /* Name of generated program from command line. */
273 if (NULL == (optptr->outfname = malloc(strlen(optarg)+1))) {
274 fprintf(stderr, "Not enough memory\n");
275 exit(1);
276 }
277 strcpy(optptr->outfname, optarg);
278 break;
279 case 'g': /* Don't compile. */
280 optptr->compiler = NULL;
281 break;
282 case 'n': /* Execute statement n times. */
283 optptr->ntimes = strtol(optarg, NULL, 0);
284 break;

285 case 'o': /* Use optimizer when compiling benchmark. */
286 optptr->doopt = 1;
287 break;
288 default:
289 usage();
290 }
291 }
292 }
293
294 void
295 usage(void) { /* Print help message and exit. */
296 fprintf(stderr, "Usage: sbench [ options ] [ files ... ]\n"
297 " Available options are:\n"
298 " -c x Select compiler: x=a Aztec, x=d Datalight,\n"
299 " x=e Ecosoft, x=t Turbo, x=z Zortech\n"
300 " -f name Write generated program to file name.\n"
301 " -n xx Execute each statement xx times.\n"
302 " -o Optimize when compiling benchmark.\n"
303 " -g Generate test program only, do not try to compile.\n");
304 exit(1);
305 }
306
307 /* writelines -- Output all lines in a list. */
308 void
309 writelines(struct llist *list, FILE *outfile) {
310 struct listel *lep;
311
312 for (lep = list->head; lep != NULL; lep = lep->next) {
313 if (fprintf(outfile, "%s\n", lep->line) < 0) outerr();
314 }
315 return;
316 }
317
318 /* outerr -- Display a message and abort following an output error. */
319 void
320 outerr(void) {
321 perror(options.outfname);
322 fprintf(stderr, "Error writing output file");
323 exit(1);
324 }






[LISTING TWO]

 1 /* getline.c -- Read a line from a stream. */
 2 #include <stdio.h>
 3 #include <stdlib.h>
 4 #include <string.h>
 5
 6 #define MEMINCR 256 /* Size of memory block used for (mre)alloc. */
 7 /* getline -- Read one line from file infile into a malloc'ed string.
 8 Lines ending with \ are combined with the following line.
 9 Return NULL on error or EOF, otherwise a pointer to the string. */
 10 char *
 11 getline(FILE *infile)

 12 { char *p, *q;
 13 int c;
 14 unsigned avail;
 15
 16 p = q = malloc((unsigned)MEMINCR);
 17 avail = MEMINCR - 1;
 18 for (;;) {
 19 if ((c = getc(infile)) == EOF) {
 20 *p = '\0';
 21 if (p > q) return q;
 22 else return NULL;
 23 }
 24 if ((*p++ = c) == '\n') {
 25 if (p <= q+1 p[-2] != '\\') {
 26 *p = '\0';
 27 return q;
 28 } /* else Continued line. */
 29 }
 30 if (--avail == 0) {
 31 /* Need more memory. */
 32 *p = '\0';
 33 if ((q = realloc(q, (size_t)(p-q + MEMINCR + 1))) == NULL)
 34 return NULL;
 35 avail = MEMINCR;
 36 p = strchr(q, '\0'); /* Find end of string, in case it moved. */
 37 }
 38 }
 39 }







[LISTING THREE]

 1 /* getopt.c -- UNIX-like command line option parser. */
 2 #include <stdio.h>
 3 #include <string.h>
 4 int optind; /* Index of next argument in argv[]. */
 5 char *optarg; /* Pointer to option argument. */
 6
 7 /* getopt -- Returns option letters one at a time,
 8 EOF when no more options remain, and ? for unknown option.
 9 Optstr contains legal option letters. A colon following a
 10 letter indicates that option requires an argument. */
 11 int
 12 getopt(int argc, char *argv[], char *optstr)
 13 { static char *cp;
 14 char *p;
 15
 16 if(optind == 0)
 17 cp = argv[++optind] + 1; /* First call */
 18 if(*argv[optind] != '-' optind >= argc)
 19 return EOF; /* No more options. */
 20 if(*cp == '-')
 21 { ++optind;
 22 return EOF; /* -- indicates end of options */

 23 }
 24 if((p = strchr(optstr, *cp++)) == NULL)
 25 { fputs("unknown option: ", stderr);
 26 putc(*(cp-1), stderr);
 27 putc('\n', stderr);
 28 if(*cp == '\0')
 29 cp = argv[++optind] + 1;
 30 return '?';
 31 }
 32 if(*(p+1) == ':')
 33 { if(*cp == '\0') /* Get argument. */
 34 optarg = argv[++optind];
 35 else
 36 optarg = cp;
 37 cp = argv[++optind] + 1;
 38 }
 39 else if(*cp == '\0')
 40 cp = argv[++optind] + 1; /* Set up for next argv. */
 41 return *p;
 42 }







[LISTING FOUR]

 1 /* clock.c -- Microsecond resolution clock routine. */
 2 /* Implements in C the timer chip tweaking described by */
 3 /* Byron Sheppard, _Byte_, Jan 1987, p 157-164. */
 4 /* Replaces standard clock() from the library. */
 5 /* The definition of CLK_TCK in time.h may have to */
 6 /* be changed to 1000000L. */
 7 /* Does not correctly handle intervals spanning */
 8 /* midnight or intervals greater than about 6 hrs. */
 9 #include <time.h>
 10
 11 /* Interrupt handling and i/o ports are compiler dependent. */
 12 /* The following set of preprocessor directives selects the */
 13 /* correct include files and macros for various compilers. */
 14 #ifdef __ZTC__
 15 #include <dos.h>
 16 #include <int.h>
 17 #define inportb inp
 18 #define outportb outp
 19 #else
 20 #ifdef __TURBOC__
 21 #include <dos.h>
 22 #define int_off disable
 23 #define int_on enable
 24 #else
 25 #error Unknown compiler
 26 #endif
 27 #endif
 28
 29 /* Constants */
 30 #define CONTVAL 0x34 /* == 00110100 Control byte for 8253 timer. */

 31 /* Sets timer 0 to 2-byte read/write, mode 2, binary. */
 32 #define T0DATA 0x40 /* Timer 0 data port address. */
 33 #define TMODE 0x43 /* Timer mode port address. */
 34 #define BIOS_DS 0x40 /* BIOS data segment. */
 35 #define B_TIKP 0x6c /* Address of BIOS (18.2/s) tick count. */
 36 #define SCALE 10000 /* Scale factor for timer ticks. */
 37 /* The following values assume 18.2 BIOS ticks per second resulting from
 38 the 8253 being clocked at 1.19 MHz. */
 39 #define us_BTIK 54925 /* Micro sec per BIOS clock tick. */
 40 #define f_BTIK 4595 /* Fractional part of micro sec per BIOS tick. */
 41 #define us_TTIK 8381 /* Micro sec per timer tick * SCALE. (4/4.77 MHz) */
 42
 43 clock_t
 44 clock(void) {
 45 unsigned char msb, lsb;
 46 unsigned int tim_ticks;
 47 static int init = 0;
 48 unsigned long count, us_tmp;
 49 static unsigned long init_count;
 50
 51 if (0 == init) {
 52 init = 1; /* This is the first call, have to set up timer. */
 53 int_off();
 54 outportb(TMODE, CONTVAL); /* Write new control byte to timer. */
 55 outportb(T0DATA, 0); /* Initial count = 0 = 65636. */
 56 outportb(T0DATA, 0);
 57 init_count = *(unsigned long int far *)MK_FP(BIOS_DS, B_TIKP);
 58 int_on();
 59 return 0; /* First call returns zero. */
 60 }
 61 int_off(); /* Don't want an interrupt while getting time. */
 62 outportb(TMODE, 0); /* Latch count. */
 63 lsb = inportb(T0DATA); /* Read count. */
 64 msb = inportb(T0DATA);
 65 /* Get BIOS tick count (read BIOS ram directly for speed and
 66 to avoid turning on interrupts). */
 67 count = *(unsigned long far *)MK_FP(BIOS_DS, B_TIKP) - init_count;
 68 int_on(); /* Interrupts back on. */
 69 tim_ticks = (unsigned)-1 - ((msb << 8) lsb);
 70 us_tmp = count*us_BTIK;
 71 return us_tmp + ((long)tim_ticks*us_TTIK + us_tmp%SCALE)/SCALE;
 72 }







[LISTING FIVE]

 1 /* Test data for statement benchmark generator. */
 2 %%
 3 %float f, g = 10., h = 20.;
 4 %double c, d = 10., e = 20.;
 5 %int i = 10, j, *p, *q;
 6 %int array[50], another[50];
 7 f = g+h; /* Float addition. */
 8 c = d+e; /* Double addition */

 9 /* Initialize an array using subscripting. */\
 10 for (i=0; i < 50; ++i)\
 11 array[i] = 0;
 12 /* Initialize an array using pointer. */\
 13 for (p = array; p < array + 50; ++p)\
 14 *p = 0;
 15 /* Copy using array subscripts. */\
 16 for (i=0; i < 50; ++i)\
 17 array[i] = another[i];
 18 /* Copy using pointers. */\
 19 for (p = array, q=another; p < array + 50; ++p, ++q)\
 20 *p++ = *q++;
 21 (a)
 22 Test input
 23
 24 Statement 1 required 1215.0 microseconds to execute, 1 iteration timed.
 25 Statement 2 required 560.0 microseconds to execute, 1 iteration timed.
 26 Statement 3 required 1671.0 microseconds to execute, 1 iteration timed.
 27 Statement 4 required 1606.0 microseconds to execute, 1 iteration timed.
 28 Statement 5 required 1820.0 microseconds to execute, 1 iteration timed.
 29 Statement 6 required 1794.0 microseconds to execute, 1 iteration timed.
 30 (b)
 31 Results
 32
 33 Statement 1 required 2.0 microseconds to execute, 1 iteration timed.
 34 Statement 2 required 1.0 microseconds to execute, 1 iteration timed.
 35 Statement 3 required 813.0 microseconds to execute, 1 iteration timed.
 36 Statement 4 required 813.0 microseconds to execute, 1 iteration timed.
 37 Statement 5 required 1112.0 microseconds to execute, 1 iteration timed.
 38 Statement 6 required 777.0 microseconds to execute, 1 iteration timed.
 39 (c)
 40 Results with optimizer on






























February, 1989
DEBUGGING TSR PROGRAMS


Turbo Debugger can take the pain out of buggy TSRs -- if you know the trick




Costas Menico


Costas Menico is a senior software developer and part owner of The Software
Bottling Company, where he has worked with assembly language and TSR programs
for more than five years. He can be reached at 6600 Long Island Expressway,
Maspeth, NY 11378.


One of the most unnerving aspects of writing terminate-and-stay-resident (TSR)
programs is debugging them. Compounding this troublesome fact is that many
commercial debuggers, including Debug, don't have the ability to debug TSR
programs. One solution, of course, is to buy a sophisticated (and expensive)
debugger, but this isn't always the most attractive or economical solution
unless you are developing TSR programs on a full-time basis.
Because I found Borland's Turbo Debugger to be a good, low-cost debugger, I
decided to investigate ways of making it handle TSR programs in an easy and
respectable manner. The techniques I developed, and that I describe here, have
been used to debug commercially available TSR programs (Software Bottling's
Flash-Up and Speed Screen, for instance).
Furthermore, there's no reason why the same techniques can't be applied to
debuggers other than Turbo Debugger. In this article, I'll briefly describe
how TSR programs work, then talk about how you can debug them when things go
wrong. As an example, I'll use a typical (but bare boned) TSR program called
KEYSWAP (assembled in Turbo Assembler) and look into why you can't normally
debug it with Turbo Debugger. This sample TSR program makes the keys F8 and
F10 act as if you had pressed the up arrow and down arrow, respectively. In
many ways, KEYSWAP is the foundation of a keyboard macro program.


The TSR Problem


A TSR program really has two parts: The first part actually performs the
functions you designed it to perform, and the second part consists of the code
that loads the program, chains to any interrupt vectors, and terminates with a
special DOS int 21h call to stay resident. Example 1 illustrates the flow of a
typical TSR program.
Example 1: Typical structure of a TSR program

 ; This portion remains resident
 tsr data
 tsr interrupt handlers
 tsr other programs

 ; This portion is freed when the DOS Terminate
 ; and stay resident function is executed.
 data area
 load entry point:
 create stack
 call other check routines
 (e.g., DOS version, parameters, available memory, etc.)
 chain tsr interrupt handlers
 determine size of tsr to keep
 call DOS TSR function
 end

If you run a TSR program such as KEYSWAP with a debugger in the normal manner
--that is, you do a file/load and then run it --you will simply get a message
saying that the program and terminated and some error code. When you quit the
debugger, your program will not be resident because both the debugger and your
program will terminate. If you've chained to any interrupts, your system will
probably crash because the chained interrupt vectors will still be there but
your program and the debugger will be gone. This means any chained interrupt
vectors will be pointing into what could be random instructions and data.
The concept behind the solution to this sort of problem is actually quite
simple. Instead of terminating KEYSWAP using the normal
terminate-but-stay-resident DOS interrupt function, I take advantage of the
DOS load-and-execute function to load and execute COMMAND.COM. The effect of
this is twofold: first KEYSWAP seems to be memory-resident, and second, Turbo
Debugger is there at the touch of a hot key if you need to debug the program.


Using and Debugging a Typical TSR Program


To use the sample TSR program as an executable program, assemble KEYSWAP.ASM
(Listing One, page 104) into KEYSWAP.OBJ by typing TASM KEYSWAP (assuming
you're using Turbo Assembler). Do not include the file TSRDEBUG.ASM in Listing
Two, page 105. Next create the executable file KEYSWAP.COM from KEYSWAP.OBJ by
typing TLINK KEYSWAP /T. You can then load the TSR program into memory by
typing KEYSWAP at the DOS prompt. (As this article is about debugging TSR
programs and not about how to write them, I have not provided KEYSWAP with the
ability to check if it has been already loaded in memory or the ability to be
unloaded from memory, except by rebooting.)
To debug the program, you assemble and link KEYSWAP.ASM as before, except now
you include the file TSRDEBUG.ASM (Listing Two). To assemble KEYSWAP.ASM and
include the special debugging code in file TSRDEBUG.ASM, type TASM KEYSWAP
/DDEBUG. A file called KEYSWAP.OBJ is then created. Next, link KEYSWAP.OBJ to
create KEYSWAP .COM and KEYSWAP.MAP by typing TLINK KEYSWAP /M /T. Finally,
type TMAP KEYSWAP to create the file KEYSWAP.TDS, which contains symbolic
information for Turbo Debugger, from KEYSWAP.MAP.
You should now have files called KEYSWAP.COM and KEYSWAP.TDS ready to use for
debugging. All other files are not required by the debugger.
To run, start Turbo Debugger and load KEYSWAP.COM using the file/load menu
selection, then press F9 to execute. KEYSWAP will chain int16 handler on
interrupt 16h. It will then call the procedure tsr_simulate, which frees all
memory starting from the label load_tsr. It then proceeds to chain
int9handler. After saving the stack registers into a memory location, KEYSWAP
loads and executes COMMAND.COM, which must be in the root directory of drive
C:. At this point you will get the DOS prompt. Your program and Turbo Debugger
are now both memory-resident, and you can debug and set breakpoints by
activating the debugger.
To activate the debugger, press Ctrl-Enter. The program will be interrupted
because int9handler checks whether Ctrl-Enter was pressed, and if it was, the
handler executes the breakpoint int 3, which is the breakpoint interrupt to
activate any debugger. (If you have a hardware switch, known as an NMI switch,
you may use it instead of Ctrl-Enter.)
The CPU window will show the point at which execution was interrupted. Because
the int 3 instruction is in KEYSWAP, you can press PgUp or PgDn and view both
code and symbols. You can also go to a procedure by pressing Alt-G (Goto) and
entering a label that is public in your program.
You can now set breakpoints in the KEYSWAP code at the points where you want
to break and start tracing. As an example, set a breakpoint at the label
f10pressed, which is in int16handler and was declared public. The instruction
at this label is executed when you press the F10 key while in your
application. Press Alt-F2 and enter the label name. This sets a breakpoint.
Press F9 (Run) to exit from the debugger and return to the DOS prompt.
(Make sure that you do not alter any registers or flags or start executing at
another instruction. Do not try to do any disk access either, such as loading
a different program. You may have interrupted DOS in the middle, and because
DOS is not reentrant, another DOS call is all DOS needs to crash your system.)

To run your application and use KEYSWAP, press the F10 key when you want to
move the cursor down, and your debugger will activate because the breakpoint
has been reached. Trace to check what your program is doing, and debug as you
would with any other program. Pressing F9 (Run) will return you to your
application.
To exit from the debugger and terminate KEYSWAP, type the Exit command at the
DOS prompt. This command returns you to KEYSWAP at the instruction immediately
following the load-and-execute function in the tsr_simulate procedure. The
procedure will then unchain the interrupt handlers and terminate using the
normal DOS terminate function. (For a list of the DOS functions used in this
program, see Example 2.) You can now quit the debugger or restart KEYSWAP.
Example 2: DOS functions used in KEYSWAP
Note: The following is a list of the DOS INt 21h functions used in KEYSWAP.
Remember that if any DOS function returns with the CARRY flag set, the
function had a problem. Usually reason for the error is returned in the AX
register. For more information, see the DOS Technical Reference manual or one
of the dozens of books on the subject.

 Set the vector of interrupt:
 Input: AH = 25h, AL = interrupt #
 DS:DX = segment & offset value of new interrupt handler

 Terminate but stay resident:
 Input: AH = 31h, AL = errorlevel
 DX = # of paragraphs to keep

 Get the vector of interrupt:
 Input:
 AH = 35h, AL = interrupt #
 Output:
 ES:BX = segment & offset value of the current interrupt handler

 Shrink allocated memory:
 Input: AH = 4Ah, ES = Segment to shrink
 BX = paragraphs to keep

 Load & Execute:
 Input: AH = 4Bh, AL = 0 (If AL = 3 then it only loads)
 ES:BX = segment and offset of params block
 (for param block format see the program)
 DS:DX = segment & offset of command (ASCIIZ string)
 Output: None, but all registers including SS and SP are
 destroyed.

 Terminate program:

 Input: AH = 4Ch, AL = error level.


Flow of KEYSWAP Without the DEBUG Code


When KEYSWAP starts up, it first sets up its own stack. Always use your own
stack as if it were your own parachute. The next step is to chain the
int16handler keyboard handler so that you can check for the F8 and F10 keys.
The last step is to determine the size of the program you wish to keep in
memory. You normally do not want to keep the loading portion of KEYSWAP, so
you figure out the size by taking the offset of the loader's starting address,
which is the label load_tsr. You now call DOS to terminate but stay resident.
On entry, int16handler checks to see if the keyboard function called for is a
"getkey". If not, it jumps directly to the old interrupt vector. If the
function is a get key, int16handler calls the old interrupt to get the key.
Upon return, it inspects the ax register for an F8 or F10 key. If either of
those keys was pressed, it substitutes the corresponding up or down arrow keys
into ax and returns to the original caller.


KEYSWAP with the Debug Code


To debug KEYSWAP you must compile it with the /ddebug option, as noted
earlier. You will notice that I use the conditional-assembly directives if ...
def ... else ... endif to include the debugging code. This code is in the file
TSRDEBUG.ASM. The debugging code adds the following procedures: int9handler,
tsr_simulate, and unchain_vector.
The procedure int9handler checks to see if the Ctrl-Enter key was pressed and
then executes an int 3 to activate Turbo Debugger. The procedure tsr_simulate
frees memory, starting at the label load_tsr. It then executes COMMAND.COM, at
which point the DOS prompt appears. When you terminate your debugging and wish
to go back, type Exit at the DOS prompt, which returns you to the debugger to
recover the stack registers and unchains the vectors int9handler and
int16handler. It will then execute the normal DOS terminate function. The
procedure unchain_vector unchains the int16handler vector when you have
finished and puts back the original one. If you do not do this, and exit from
the debugger, you will eventually crash the computer. The debug code also adds
working storage for these three procedures.


Caveats, Conditions, and Other Warnings


While debugging your memory-resident program with Turbo Debugger, the program
memory size is the size of Turbo Debugger plus your program size. Do a CHKDSK
after your program runs to see how much memory is left.
If you interrupted your program by pressing Ctrl-Enter or with a breakpoint,
you should not try to use any of Turbo Debugger's menu functions that access
the disk. These include such choices as file/load and options/save. If you do
attempt to use filing choices, you may crash DOS. The reason is that DOS is
not reentrant, and so it cannot be called again when busy. This includes calls
to DOS from the debugger. Microsoft provides an undocumented function to
detect if Dos is busy but I will not discuss it here.
If Ctrl-Enter does not respond, the chances are that your program crashed in
some loop and interrupts have been turned off, which means that the keyboard
interrupts are ignored by the CPU. If you have an NMI switch, try using it to
see if your program is stuck in a loop. Beware, though, because the only way
out may be by rebooting.
Never leave int 3 (breakpoint) instructions in your finished program. Although
they are supposed to be harmless if no debugger is loaded, I have known some
machines that will not run programs containing int 3s. (I had to write a TSR
program to trap the int 3s!)
The reason why I use Ctrl-Enter and int9handler to activate the debugger is
because Turbo Debugger's hot key (Ctrl-Break) will not activate when inside
applications. You could, however, remove int9handler if you have an NMI
hardware switch. I also recommend changing the Turbo Debugger hot key from
Ctrl-Break to something else.
The debugging concept presented in this article will work in a similar manner
for .EXE programs that run TSR programs, although you do have to take the
different segments into account. You can also extend this concept to languages
such as C and Pascal when you create TSR programs using them. These languages
have a built-in load-and-execute function; the trick is to know how much
memory to free before you call it.



Conclusions


No matter how well we design, no matter how well we code, and no matter how
much we know, our programs will not necessarily run bug free once compiled.
That goes twice for TSR programs. The trick is to minimize the time it takes
to debug using tools such as Turbo Debugger. I hope this article will give you
an extra tool in the battle against Tyranno-Saurus Rex (TSR) bugs.

_Debugging TSR Programs_
by Costas Menico


[LISTING ONE]

;
; Set TABS to 8 for viewing with editor.
;
;-------------------------------------------------------;
; KEYSWAP program ;
; By Costas Menico. ;
; Software Bottling Company ;
; 6600 Long Island Expressway ;
; Maspeth, N.Y. 11378 ;
; 718-458-3700 ;
; This program will swap the F8 & F10 key for the ;
; Up and Down arrow keys. ;
; Demonstrator program for debugging memory resident ;
; program using Turbo Debugger. ;
;-------------------------------------------------------;

 .model small

; Set key values.
; For the SCAN and ASCII codes of any other keys, see the BIOS manual
f8key equ 4200h ; define the F8 key
f10key equ 4400h ; define the F10 key
upkey equ 4800h ; define the up arrow key
downkey equ 5000h ; define the down arrow key

DOSCALL equ <int 21h> ; DOS interrupt call
LOADSEG MACRO segr1, segr2 ; Load segment register
 push segr2 ; macro.
 pop segr1
 ENDM


; Publics area
public start, oldint16, int16handler, exit, test_for_key
public load_tsr, tsr_terminate, chain_vector, f10pressed

; Start Code
 .code
 assume cs:_text, ds:_text, es:_text, ss:nothing

;-------------------;
; Our program start ;
;-------------------;
 org 100h ; COM file starting program address.
start:
 jmp load_tsr ; Load our program resident.


; Data area. Since this is COM file all data
; and code are in the Code segment.
 evendata
oldint16 dd 0 ; Area for old interrupt 16h vector.
stackbot dw 128 dup('?'); Our stack bottom.
stacktop equ $-2 ; Our stack top.

;-----------------------------------------------;
; Entry to keyboard handler. ;
; We get here when any application or DOS ;
; executes an INT 16h. ;
; Input: AH=0 - Get a key ;
; Output: The key in AX ;
; Input: AH=1 - Check for key ;
; Ouput: The available key in AX ;
; Input: AH=2 - Get shift flags ;
; Output: The shift flags in AL ;
;-----------------------------------------------;
int16handler proc far
 pushf ; Save flags status.
 or ah, ah ; Are we getting a key?
 je test_for_key ; Yes - goto to test for F10.

 popf ; No - pop flags and
 jmp cs:oldint16 ; jump to old interrupt handler.

; Get a pressed key and test if it was
; and F10
test_for_key:
 popf ; Pop the original flags.
 pushf ; Push them again
 call cs:oldint16 ; and simulate INT 16h.

 pushf ; Save flags.
 cmp ax, f10key ; is this an F10 key?
 jne is_it_f8 ; no - check for F8.
f10pressed:
 mov ax, downkey ; yes - set swap key in ax.
 jmp short exit
is_it_f8:
 cmp ax, f8key ; is this an F8 key?
 jne exit ; no - exit
 mov ax, upkey ; yes - set swap key in ax.
exit:
 popf ; Pop the saved flags.

 iret ; Return to caller with key in ax.
int16handler endp

IFDEF DEBUG
 ; Assemble this section to debug the TSR with Turbo Debugger.
 ; You must assemble with the /DDEBUG option.
 include tsrdebug.asm
ENDIF

;===============================================;
; Load KEYSWAP, and terminate but stay resident.;
; Once KEYSWAP is loaded all code from here on ;

; is discarded. ;
;===============================================;

load_tsr:

 ; Set SP to our internal stack area
 cli ; Do not interrupt while
 mov sp, offset stacktop ; stack pointer is being adjusted.
 sti ; Now interrupt.

 call chain_vector ; Install our keyboard interrupt.

IFDEF DEBUG
 ; Assemble this section to debug the TSR with Turbo Debugger.
 ; You must assemble with the /DDEBUG option.
 call tsr_simulate
ELSE
 ; Assemble this section to run without the debugging.
 call tsr_terminate
ENDIF

;-----------------------------;
; Save old interrupt 16 vector;
; and point it to our handler ;
;-----------------------------;
chain_vector proc uses ax bx dx es
 mov ax, 3516h ; Get the int 16h keyboard vector.
 DOSCALL
 mov word ptr oldint16, bx; Save old vector in our data area.
 mov word ptr oldint16[2], es

 mov ax, 2516h ; Set the new vector of our
 mov dx, offset int16handler; interrupt 16h keyboard handler.
 DOSCALL
 ret
chain_vector endp

;-----------------------------;
; Terminate and stay resident ;
;-----------------------------;
tsr_terminate proc
 mov dx, offset load_tsr ; Get offset address of program
 mov cl, 4 ; code to discard.
 shr dx, cl ; Convert to pargraphs.
 inc dx ; Round off to the next paragraph.
 mov ax, 3100h ; Terminate & stay resident function.
 DOSCALL
tsr_terminate endp

@CurSeg ends

 end start





[LISTING TWO]


;---------------------------------------------------------------;
; Include file TSRDEBUG.ASM ;
; ;
; Assemble this code if you will debug with Turbo Debugger ;
; You must assemble with the /DDEBUG option. ;
;---------------------------------------------------------------;

public param, command_com
public tsr_simulate, unchain_vector, int9handler

BREAKPOINT equ <int 3> ; Debugger break point
ctrlflag equ 4 ; Ctrl key BIOS indicator flag.
enterscan equ 28 ; Scan code for Enter key.
kbdflags equ 417h ; BIOS keyboard flags location

; Load & Execute parameter block structure
execparam struc
envstr_addr dw 0 ; Environment pointer
cmdline_ofs dw 0 ; Offset to command line.
cmdline_seg dw 0 ; Segment to command line.
fcb1_ofs dw 0 ; Offset of fcb1.
fcb1_seg dw 0 ; Segment of fcb1
fcb2_ptr dw 0 ; Offset of fcb2.
fcb2_seg dw 0 ; Segment of fcb2.
execparam ends


oldint9 dd 0 ; Area for old int 9 vector.
paramvals equ <0,offset cmd_line,?,offset fcb1,?,offset fcb2,?>
;
; Shows below how the data above is initialized.
;
comment ^
 0, ; Use current environment
 offset cmd_line,? ; Point to command line.
 offset fcb1,? ; Offset & Seg of fcb1.
 offset fcb2,? ; Offset & Seg of fcb2.
^
param execparam <paramvals>
command_com db 'c:\command.com',0
cmd_line db 0, 0dh ; Command line is null.
savess dw 0 ; Save SS here.
savesp dw 0 ; Save SP here.
fcb1 db 16 dup(0) ; Blank area for FCB1
fcb2 db 27 dup(0) ; Blank area for FCB2

;-----------------------------------------------;
; Int 9 handler. Ctrl-Enter key check. ;
; Each time a key is pressed it is checked ;
; if the Ctrl & Enter key were pressed ;
; simultaneously. If they were then we ;
; interrupt Turbo Debugger ;
;-----------------------------------------------;
int9handler proc far
 pushf ; Save the flags
 push ax ; Save ax
 push ds ; Save dx
 xor ax, ax ; Point DS to segment 0
 mov ds, ax

 ; Is the Ctrl key pressed?
 test byte ptr ds:[kbdflags], ctrlflag
 je exitint9 ; No - call old int 9 & exit.

 in al, 60h ; Yes - get the scan code of key.
 cmp al, enterscan ; Is it the Enter key?
 je breakkey ; Yes - go to break Turbo Debug
exitint9: ; No - call old int 9 & exit

 pop ds ; Pop ds & ax before jump.
 pop ax
 popf ; Pop flags
 jmp cs:oldint9 ; Jump to the old int 9

breakkey:
 pop ds ; Pop ds & ax before jump.
 pop ax
 popf ; Pop flags

 in al, 61h ; Reset the keyboard controller
 mov ah, al ; for next key.
 or al, 80h
 out 61h, al
 xchg ah, al
 out 61h, al
 mov al, 20h ; Enable interrupts
 out 20h, al

 BREAKPOINT ; Execute our break point.
 ; Turbo Debugger STOPS HERE!.
 ; To find out what the program
 ; was doing press F7 (Trace)
 ; twice.
 iret
int9handler endp

;---------------------------------------;
; Simulate Terminate and stay resident ;
; Frees unecessary memory and loads & ;
; executes COMMAND.COM ;
;---------------------------------------;
tsr_simulate proc
 ; Free unused memory.

 LOADSEG ES, CS ; Point ES to our segment.
 mov bx, offset load_tsr ; Get offset address of program
 mov cl, 4 ; code to discard.
 shr bx, cl
 inc bx ; round off to the next paragraph
 mov ah, 4ah ; Shrink allocated block function
 DOSCALL ; call DOS

 ; Save stack registers
 mov savess, ss ; Save stack segment.
 mov savesp, sp ; Save stack pointer.

 ; Chain unto int 9 for debugger break key.
 mov ax, 3509h ; Get the int 9 keyboard vector.
 DOSCALL

 mov word ptr oldint9, bx; Save old vector in our data area.
 mov word ptr oldint9[2], es
 mov ax, 2509h ; Set the new vector of our
 mov dx, offset int9handler; interrupt 9 keyboard handler.
 DOSCALL

 ; Load and execute COMMAND.COM

 LOADSEG ES, CS ; Point ES to our segment
 mov dx, offset command_com; Point to COMMAND.COM file path.
 mov bx, offset param ; Get address of param block.
 mov [bx].cmdline_seg, ds; Get segment of the command line.
 mov [bx].fcb1_seg, ds ; Get segment of fcb1
 mov [bx].fcb2_seg, ds ; Get segment of fcb2
 mov ax, 4b00h
 DOSCALL ; Load and execute COMMAND.COM.


 ; Upon return form load and execute, all registers are
 ; destroyed and the necessary ones, must be recovered.

 cli ; Turn interrupts off.
 mov ss, cs:savess ; Recover stack segment.
 mov sp, cs:savesp ; Recover stack pointer.
 sti

 LOADSEG DS, CS ; Recover data segment

 ; Unchain all our handlers, and terminate program.

 mov ax, 2509h ; Unchain our int 9 vector and put
 lds dx, oldint9 ; original in.
 DOSCALL

 call unchain_vector ; Unchain our keyboard handler

 mov ax, 4c00h ; Terminate program.
 DOSCALL
tsr_simulate endp

;----------------------------------------;
; Unchain vectors before exiting debugger ;
;----------------------------------------;
unchain_vector proc uses ds ax
 mov ax, 2516h ; Unchain our vector and put
 lds dx, oldint16 ; original in.
 DOSCALL
 ret
unchain_vector endp




[Example 1: Typical structure of a TSR program]


; This portion remains resident
 tsr data
 tsr interrupt handlers

 tsr other programs

; This portion is freed when the DOS Terminate
; and Stay resident function is executed.
 data area
load entry point:
 create stack
 call other check routines
 (e.g. DOS version, parameters, available memory, etc.)
 chain tsr interrupt handlers
 determine size of tsr to keep
 call DOS TSR function
end




[Example 2: DOS functions used in KEYSWAP]

Note: The following is a list of the DOS INT 21h functions used
in KEYSWAP. Remember that if any DOS function returns with the
CARRY flag set, the function had a problem. Usually the error
reason is returned in the AX register. For more information, see
the DOS Technical Reference manual, or one of the dozens of books
on the subject.

Set the vector of interrupt:
Input: AH = 25h, AL = interrupt #
 DS:DX = segment & offset value of new interrupt handler.

Terminate but stay resident:
Input: AH = 31h, AL = errorlevel
 DX = # of paragraphs to keep.

Get the vector of interrupt:
Input:
 AH = 35h, AL = interrupt #
Output:
 ES:BX = segment & offset value of the current interrupt handler.

Shrink allocated memory:
Input: AH = 4Ah, ES = Segment to shrink
 BX = paragraphs to keep.

Load & Execute:
Input: AH = 4Bh, AL = 0 (If AL = 3 then it only loads)
 ES:BX = segment and offset of params block
 (for param block format see the program)
 DS:DX = segement & offset of command (ASCIIZ string).
Output: None, but all registers including SS and SP are
 destroyed.

Terminate program:
Input: AH = 4Ch, AL = errorlevel




































































February, 1989
APL PLUS SYSTEM II


An inside look at the evolution of APL*PLUS/PC from STSC




Chris Burke


Chris Burke is an actuary for Manufacturers Life in Toronto, Canada.


STSC's recent introduction of its APL*PLUS System II (SII) for 80386-based PC
compatibles may be one of the most significant events in the APL world for
some time. SII succeeds STSC's highly successful APL*PLUS/PC (APL/PC) as its
flagship microcomputer product. Enhancements include nested arrays, improved
handling of workspaces, and more.
APL/PC had some limitations, mainly a consequence of its dependence on the
8088 processor. I had hoped STSC's new interpreter would take full advantage
of the 80386 and include fixes to the more noticeable shortcomings of APL/PC
while maintaining essentially the same interpreter and close compatibility
between the two systems.
What STSC has produced is different. You do get the expected 80386
enhancements, plus nested arrays. SII, however, carries over some of the
problems of APL/PC and adds a few new ones. SII is twice as expensive as
APL/PC, with no special price for an upgrade. It requires an 80387 coprocessor
to take full advantage of its extra speed; and there are many changes from
APL/PC, making SII incompatible with its predecessor.
Before getting into a detailed discussion of SII, let's take a brief look at
the overall system. Once the basics have been covered, this review looks at
the new product from the point of view of the APL/PC user, focusing on the
enhancements that SII provides.


Functionality


Like APL/PC, SII is an implementation of standard APL in the commonly accepted
sense: A program written in standard VSAPL, not using shared variables or
system-specific code, can be ported to SII and run with little or no
modification.
SII includes most of the extensions to the APL language found in APL/PC. These
extensions include:
APL files
/FMT, /FI, /VI
/SI, /IDLOC, /SIZE
/VR, /DEF (vector function representations)
Event handling (via /ELX, /ALX, /SA)
Primitives match, without, find
In addition, SII includes several PC-specific features found in APL/PC, such
as:
/CALL (to support machine-language subroutines)
/PEEK, /POKE
/EDIT Full-screen editor
Full screen management via /WIN, /WGET, /WPUT
/GPRINT, and so forth, for graphics management
Access to native files * Calls to DOS commands via /CMD
Direct access to DOS directory functions (for example, /CHDIR)
New features in SII include nested arrays, and system functions, and variables
such as /ED, /FHIST, /PR, /SAVE, and /PSAVE. Missing are detached I/O
facilities (such as /IN and /OUT), and several /PEEK and /POKE locations have
been disabled. Also removed are the little-used system variables /KEYW,
/HNAMES, and /HT0PIC.
I should mention that SII's workspaces and files are not compatible with
APL/PC. SII has no means of reading an APL/PC workspace directly, but the
system does include a workspace of defined functions that allow full access to
APL/PC files. Thus, files can be transferred back and forth using these
functions, and hence workspaces, via the standard functions WSTOFILE on the
source system and FILETOWS on the target system.


Enhancements


SII offers several enhancements. Most notably, SII uses Phar Lap's DOS
extender to run 80386 protected mode under standard DOS, Version 3.3 or later.
Use of the 80386 processor has allowed STSC to make several fixes to APL/PC's
shortcomings.
New are 4-byte integer and bit Boolean data types. APL/PC stored both in 2
bytes. In fact, APL/PC Booleans were one of that system's weak points. Not
only did they take up far too much space but also the interpreter had no
special code for Boolean calculations. In executing +/A=B, for example, the
interpreter now knows A=B is Boolean and therefore is able to calculate the
plus reduce efficiently. APL/PC did not do so.
Another enhancement is that there is no limit on variables nor a performance
penalty for large arrays. Earlier versions of APL/PC limited object size to
64K. Although this limitation was relaxed in later versions, the manipulation
of objects greater than 64K was rather sluggish.
There is, however, one caution. Because SII uses protected mode, \CALL machine
language written for APL/PC will not work.


Workspaces


Workspaces are now limited only by the amount of available memory. (The
theoretical limit is 2 gigabytes.) Part of the APL system (the DOS extender
and APL session manager) is located in the first 640K of system memory, and
the APL interpreter is located in memory above 1 Mbyte. Any other free memory
above 1 Mbyte is available for the APL workspace. STSC recommends a minimum of
2-Mbytes RAM, leaving a workspace of about 750K. This compares to about 400K
of workspace with APL/PC.
Several workspaces are supplied, including complex numbers, eigenvalue
calculations, upload and download to mainframe APL systems, printer drivers,
and some utilities. A major feature here is extensive support of GSS*CGI
graphics, which consists of a workspace plus several files accommodating a
wide range of graphics devices.



Nested Arrays


SII now includes a fairly basic nested array capability made up of the
following:
enclose and disclose
split and mix
type
depth
each operator
Enclose, split, and mix can be modified by the axis operator. SII also
supports strand notation and scattered assignment, and it allows user-defined
functions as arguments to operators. User-defined operators, however, are not
supported.
Nested arrays presented one pleasant and unexpected surprise when I reviewed
the system. My programs typically depend on the ability to package APL objects
(such as representing a list of APL objects in a single object). In using SII,
I was initially concerned that most of my systems would not work without a
good package capability. Of course, programmers can implement packages with
ordinary APL functions, but I had expected this to be much too slow, as it had
been with APL/PC.
Nested arrays came to the rescue. Using them I was able to write, in a couple
of hours, a complete package emulator that is efficient enough for general
use. The package runs an order of magnitude slower than the APL/PC assembler
functions, but I find that acceptable.
Indeed, only a couple of problems remain to be solved before dispensing
altogether with assembler or system package functions. Because emulators store
functions in their vector representations, locked functions can't be handled,
and the intermediate compilations produced when a function is first executed
are thrown away.


Session Manager


Several enhancements have been made to the user interface. A session manager,
implemented independently from the APL interpreter, now handles the APL
session and one or more edit sessions (the ring editor). New functions that
you'll find include search and replace (in any session), copy or block move
within or among sessions or APL objects, undo several erasures, restore
current line, and search for matching parenthesis or bracket. Especially
useful are token search and replace and the ability to set stop and trace when
editing functions. Additionally, pop-down menus provide an alternative means
of selecting various options.
Various keystrokes have been added and others changed to support these
capabilities. In fact, the underlying keyboard driver itself has been
extensively redesigned. Some changes would be needed to APL/PC functions
referencing keyboard numbers, but overall, the use of the keyboard is much
better and more logical than with APL/PC.
There are a couple of minor problems, though. A drawback to the new ring
editor is that only one editing session at a time can be handled under program
control. Also, the session manager appears slightly slower than the APL/PC
when displaying output.


Character Set


Like APL/PC, SII offers both the traditional APL keyboard and a text keyboard
called the Unified keyboard. I am not happy with either. The traditional APL
keyboard is a nuisance to use because it remaps several standard keys, such as
the apostrophe, colon, and semicolon, to new positions. Thus, it is difficult
to become fluent in both the APL and standard keyboard layouts. The whole
question of the character set and keyboard layout was hotly debated a few
years ago in the APL industry for this reason.
Two different solutions were proposed for the PC, both using Alt or Shift-Alt
key combinations to represent special APL characters. The main difference lay
in the choice of the primary alphabet. STSC's Unified keyboard treats the
uppercase alphabet as primary and lowercase as emphasized, whereas Sharp's
Union keyboard uses the reverse. In retrospect, it is quite clear that the
Union keyboard is correct; it follows the normal practice of written languages
and computer languages that allow both uppercase and lowercase alphabets. In
practice, I find the reversal of the usage of uppercase and lowercase
alphabets in the Unified keyboard disconcerting, and I still use the APL
keyboard (as does nearly everyone else). At some stage, STSC should reverse
its policy and its alphabets!


Documentation and Support


The documentation is up to STSC's usual excellent standard. It includes a
comprehensive reference manual, a users' guide, a manual of GSS graphics, and
a pocket quick reference.
I have used STSC's customer-support help line several times and am happy to
report its staff members are invariably courteous and efficient. STSC also
goes out of its way to ensure that a variety of hardware is supported by its
system.


Minor Improvements


Several minor improvements are noteworthy. First, note that screens other than
25 rows by 80 columns are now fully supported. Also, notice symbol table sizes
are now increased automatically so that using )SYMBOLS when first creating a
workspace is unnecessary. Maximum symbol size is now 32767.
Improvements to the default formatting of numbers are as follows. First,
formatting of small numbers switches to exponential notation only when the
exponent is - 7 or smaller. In contrast, APL/PC had exponential notation
starting at - 3, which often produced unreadable numeric displays. Second,
numeric matrices are now formatted column by column so that each column is
formatted to its own width instead of to the width of the widest column.
Curiously enough, formatting of numeric matrices in APL/PC was so slow that it
was more efficient to format using a defined function that formatted each
column separately and catenated the results. Even in SII, using the defined
function is only marginally slower; clearly an inefficient algorithm is being
used here.
Printer support is extensive, but perhaps I might make one request. I use a
LaserJet II printer, for which STSC provides an APL font. It's a 12-point
font, however, which is too large for function listings. Condensed mode is an
improvement but still is still unsatisfactory. I'd like to see STSC supply APL
fonts in a variety of sizes.


Problems


SII does have a few problems. To start with, there are some changes that bring
little or no benefits, yet they invalidate a lot of existing software, making
life more difficult for programmers.
For instance, automatic formatting applied when defining functions has been
disabled, leaving the original source unchanged. This formatting was extremely
useful: It made code much easier to read; it meant programmers need not waste
time manually prettying the code; and utilities such as line relabelers,
comparison functions, and search and replace functions could take advantage of
the standard format.
STSC supply a utility [triangle]VR, which (supposedly) converts any vector
representation to either its canonical vector representation in APL/PC or a
similar representation in SII. In neither case, however, does this work
correctly!
Next, APL files can now be accessed by giving their DOS path rather than the
APL library number used previously. (Incidentally, the library number is still
recommended by STSC for compatibility.) Thus, rather than referring to a file
as 11 TABLES, I can now use a few more keystrokes and enter C:
\APL\UTILS\TABLES. This seemingly innocent extension confers no real benefit
to users, but it has the serious drawback that code accessing APL files may no
longer work properly and algorithms that access files have to be much more
complex than before.
A related issue is that there is no longer a default APL library, which raises
the question of how to access a file or workspace given only its name (not its
library number or full DOS path). This feature was especially useful in
systems intended for distribution when the library number or DOS path was not
known in advance.


Timing



This was a real surprise for me. On my machine, there is no overall
significant difference in performance between SII and APL/PC. In fact, some
common functions consistently perform slower in SII. An example is the
standard vector-to-matrix utility, which typically runs about 20 percent
slower (even after compilation). These results differ from STSC's advertised
speedups of between 50 to 500 percent. There are, however, several factors to
take into account.
The use of 80386 machine language should, in theory, enable quite significant
improvements over the same programs written in 8088 code. I suspect this has
happened but has been offset by the more complex interpreter required by
nested arrays.
TSC strongly recommends using a coprocessor (which I do not have) because SII
makes much better use of coprocessing than does APL/PC. In particular, with a
coprocessor on-board, STSC claims a six time speedup for integer arithmetic
and four times for floating-point arithmetic and mixed primitives. The
company's claim for overall speedup is more modest: 30 to 50 percent in total
processing time.
SII also has a form of intermediate compilation for its functions. The first
time a function is executed, it runs slowly while the function itself is
internally redefined. It runs faster the next time around. The
vector-to-matrix function, for example, doubled in size from 532 bytes to
1,024 bytes, while timing improved about 12 percent. This is nice, but it
requires functions to be workspace-resident. Those who pefer to store their
systems on file (essential for large applications) will find there is no way
of storing the compiled code, so functions must be recompiled each time they
are run.
I think it's unrealistic to use standalone benchmarks to compare the
performance of SII and APL/PC. The larger workspace of SII allows more data to
be operated on simultaneously and can allow more efficient nonlooping
algorithms to be written. The use of 4-byte integers enables more use of
integer arithmetic rather than floating point, and bit Boolean manipulation is
much faster than with APL/PC's 2-byte Booleans.
The overall performance of systems written in SII should therefore be better
than those written with APL/PC -- even for users without a coprocessor. At the
same time, there appears to be some room for improvement in efficiency. APL/PC
users hoping to move up to a blindingly fast implementation may be
disappointed.


Conclusion


Overall, SII is a more attractive product than APL/PC. In particular, it fits
in with STSC's long-term strategy for its PAL product range. In the past, STSC
has been criticized for supporting too great a variety of APL interpreters,
which were not closely compatible with each other and which apparently
stretched STSC's development resources too thin. The plan now is to support
one primary product, the Portable APL*PLUS System, with implementations on the
PC (SII), VAX, and various Unix machines; 370 (VM\XA and MVS\XA)
implementations are to follow. APL/PC will continue to be supported but will
not become part of this product range. Sooner or later, therefore, serious
users of APL/PC will want to bite the bullet and make the change to SII.

















































February, 1989
PROGRAMMING PARADIGMS


Tough Choice




Michael Swaine


In picking Stepstone's Objective C as a key element in the NeXT development
platform, Jobs and company made what had to be a tough choice. It's the kind
of choice that will mean more as time goes on: It's what one writer calls a
wye.
By wyes, novelist John Barth means those choice points in people's lives,
roads taken or not, that lead to someplace very different from the alternate
routes. The letter Y is used as a metaphor for significant choice points.
That's what this column is about: the wyes of object-oriented programming.
Some of these represent choices among products, some are just bullets on
feature lists, and none is as widely discussed as it should be, at least if
you believe Chuck Duff of The Whitewater Group.
I talked with Duff and Digitalk's Jim Anderson, developers, respectively, of
Actor and Smalltalk/V, and both exponents of a "pure" approach to
object-oriented programming. What follows is based on their experiences and
perspectives out on their branches of the various OOP wyes. It is not an
objective, balanced view of the choices a programmer faces, but rather a look
at the choices two programmers made, and how they came to make them.


Pure vs Hybrid OOP


NeXT sidestepped the pure object-oriented approach exemplified by Smalltalk
and Actor, and opted for a hybrid approach to object-oriented programming.
There are compelling reasons for taking the branch NeXT took, as there are
compelling reasons for following the purer path. (This use of the word "pure"
is intended to be nonjudgmental and relative. Imagine quotes around the word
wherever it appears.) But the paths do seem to be divergent, and the choice of
whether to build on a pure object-oriented model or to employ object-oriented
extensions to a familiar language model will take the chooser to radically
different places. "Where Smalltalk and Actor came from is one world, and this
is really an entirely different world," Duff says.
Although NeXT chose Objective C, it's another C-based object-oriented system
--C++ --that is getting the bulk of the attention just now. And although
Bjarne Stroustrup developed C++ from Simula, the same root from which all
object-oriented languages grew, it is a fundamentally different approach from
the Smalltalk model (as is Objective C). Stroustrup is explicit about his
intentions in developing C++, and they include efficiency and compatibility
with C. C++ is an extension of C, and as such permits programming style that
is more procedural than object-oriented; it's this mixing of metaphors, more
than any particular missing features, that seems to disturb the purists.
From the pure object-oriented programming branch, this approach of extending C
in an object-oriented direction looks at best quaint and at worst
wrong-headed. Anderson expresses what seems like genuine puzzlement at the
NeXT decision, and Duff clearly sees C++ as a sell-out of the paradigm,
popular only because of AT&T and its visibility in the public domain. "From my
point of view," Duff says, "it really compromises a lot of the benefits of
object-oriented programming."
Jim Anderson agrees. He sees one of the main benefits of object-oriented
programming is that the solution is close to the problem. "You don't lose
sight of your problem in the solution; in fact, you develop an increased
understanding in the system you create." That benefit, he thinks, requires an
unadulterated paradigm. Of Digitalk's customer base, he says, "We know that
the nonprofessional programmer, the scientist, the doctor is a common
customer. Programming isn't his job; he needs results quickly, and
(appreciates) the solution being close to the problem, which I think Smalltalk
gives you. But we're also getting a lot of programmers, because they're
getting bloody noses with C or even with C++."
Duff and Anderson, of course, have axes to grind: They are selling products
that lie on the other arm of the wye. But unlike most programmers, Duff has
been down both branches. When he talks about C++, he's seeing it in the light
of Neon. As he tells the story:
"My initial focus was on Forth. (When) I was at Kriya Systems, I was the
author of Typing Tutor, and I had to write Typing Tutor for the Macintosh.
Forth, as an extensible language, was not really working, in my view, because
the extensibility was just too powerful. I was intrigued by the Smalltalk
books, and that seemed to represent a situation in which you could channel the
extensibility into a more pragmatic framework. If you extended the system (and
I extended) the system, we could probably understand each other's extensions,
because there was this consistent technology for doing it."
His researches into object-oriented programming and his background in Forth
led to Neon: "Neon was an object-oriented extension to Forth, but it was a
hybrid language in that you still had Forth around and you could still blow
yourself out of the water any time you wanted to and completely corrupt the
object metaphor. My focus in Neon was on making Smalltalk practical, because
at that time there was no Smalltalk that was really useful on a personal
computer; in fact I think Neon was the first object-oriented language that was
commercially available for a personal computer."
Neon, however, failed to take the world by storm, and Duff himself had some
second thoughts about the approach. After the Neon experience, he was
convinced that a pure object-oriented model was better, if it could be
achieved without compromising efficiency. In the hybrid approach, he saw too
many ways for the base language to collide with the object model. It was
appealing to envision a language in which even the basic syntactical
constructs would be constructed according to the messaging paradigm. That
appeal, and some frustration with Smalltalk as it existed then, led Duff to
develop Actor.
Today, there are several pure object-oriented languages available on personal
computers but surprisingly little direct comparison of these products with one
another or with the hybrids. Although an advertisement for Interactive
Software Engineering's pure object-oriented programming language Eiffel does a
feature-list comparison with C++, you won't find many comparative reviews.
(That ad also adds insult to injury by quoting Stroustrup repeatedly on
object-oriented features that C++ lacks.) Duff thinks that the reason we don't
see much direct comparison of the hybrid vs the pure implementations is that
the two approaches are seen as incommensurable: "Right now, it's just sort of
parallel evolutionary paths."
"C++ has an immense amount of support, but a lot of it is (from) C
programmers, to (whom) it's just a better C. They don't have anything to
compare it to." Some C vendors pitch C++ as precisely a better C, scarcely
mentioning its object-oriented nature in their ads.
Duff wants to see the hard questions asked in print. "Eventually what's going
to have to happen is that we're going to have to say, 'What is this technology
about? What are the benefits that we're looking for? And what language
features have to be there to provide those benefits?' We have to look behind
the scenes, and that's not happening right now. We're going to have to take a
hard look at what we want the technology to do for us, what compromises we're
willing to make in the language to accomplish that."


Inheritance vs Flat Programming


The key to the object-oriented paradigm is inheritance. It's what delivers the
benefits, and it's the essence of the choice to OOP or not to OOP, Duff
believes. Anderson sees reusability as its greatest benefit: It buys the
programmer reusability by factoring out the general, the abstractions, and
letting general solutions be specialized.
It was precisely the quest for reusability that initially turned Anderson on
to object-oriented programming. If Duff found his path only after following a
Forth thread, Anderson's enlightenment came from watching the world's biggest
software company fail and realizing that he knew where it had gone wrong.
Anderson was working for Computer Science Corp. when it got a contract with
Basic 4 Corp. Basic 4 was selling a business minicomputer with a very good
Business Basic and getting a lot of similar jobs: to do an accounting system
for a bakery, to do an accounting system for a trucking company, and so on.
Basic 4 had tried contracting the development out to individual consultants
but wasn't happy with the result, when they hit upon the idea of contracting
with CSC to do a general accounting system that could be tweaked for the
particular applications.
CSC found itself unable to reuse the software in going from job to job. "It
was just too much of a mess to snip out the pieces, the common stuff that
could be used for all." As it was becoming clear that the project was not
working out, Jim Anderson, working in another branch of CSC, was reading the
1981 Byte issue on Smalltalk and making connections.
"That was one of the things that I saw in Smalltalk: the reusability because
of inheritance. Basic doesn't have much structure. What you need is to look at
the similarities and the differences. The similarities are the abstractions,
and the general things that apply to all, and the differences are the
specializations in subclasses. Basic just doesn't give any help in that area."
CSC's problem in creating a generalized accounting system ultimately led
Anderson and George Bosworth to start Digitalk and to develop a Smalltalk
implementation for the PC and now for the Mac. "I haven't been able to go back
and solve that problem, but I can certainly envision having multiple
applications in Smalltalk: abstract accounts receivable, then Trucker's
accounts receivable, and so on, all in there at once."
Duff points to the granularity of reusable code that inheritance provides.
"You can isolate a smaller section of code that is more likely to be reusable
as a unit than if it were five or ten times its size. As size increases, the
likelihood of reusability goes down, because the more functionality there is,
the more likely it is that it'll be inconsistent with future uses."
In fact, Duff finds that the object-oriented style seems naturally to lead
programmers to break functionality down into small pieces. That's a value that
rings true for Duff, an old Forth coder. One of Chuck Moore's rules of Forth
programming is that no Forth word should be longer than two or three lines.
"In a sense I've found the same thing to be true in Actor: that short methods
and short classes and a larger number of smaller classes is usually a better
situation because it gives you that granularity where you can pick and choose
what you want to reuse."
But inheritance supplies a level of quality or reliability to the reusable
code that, Duff argues, would not be there otherwise. "If I develop a library
and I give it to you and you need it to work a little bit differently, you
start to edit my code. There is a portion that we share, but it's no longer
cleanly delineated from what I initially wrote. This means that we now have
two separate maintenance problems. I can no longer simply give you a bugfix
for you to blindly apply to your code, because you might have changed it. That
problem is made much less severe by inheritance, because we can agree that you
don't edit my code; you override pieces of it. Then I can give you updates and
it's not going to compromise your system."
Inheritance is also what makes object-oriented programming hard for many
programmers to adapt to. Novices seem to have less trouble, suggesting that
it's an unlearning, rather than a learning problem. At Carlton University in
Ottowa, they now use Smalltalk in the Introduction to Programming course in
the computer science department. They've progressed: A few years ago it was
being used only in a graduate AI course, and last year it was introduced in a
second-year data structures class. But there is, as Anderson puts it,
"definitely a weirdness factor."
"One of my problems in learning Smalltalk," he confesses, "was that I wanted
to translate it into something familiar. I was constantly fighting the ideas.
Nonprogrammers don't have to make this kind of translation all the time, so I
think they accept it more quickly."
And the weirdness factor hits immediately, as you are approaching the problem,
conceptualizing, analyzing, not thinking about code yet. Anderson again: "I
can remember programming in Pascal, approaching a problem, and the first
thought would be: 'OK, what kind of record structures do I need to define for
this problem?' It's starting to go into Smalltalk and approach a new problem
and the question is, 'OK, what kind of (preexisting) collections do I use for
this problem?' You don't start by defining something new but by reusing
something that exists."


Multiple Inheritance vs Single Inheritance


Multiple inheritance means the possibility of belonging to two different
classes at once, neither of which is a subclass of the other. It's a deviation
from a strict tree-structured arrangement of classes, such as the Linnean
classification of organisms. In the real world, objects belong to different
classes. Some object-oriented systems have multiple inheritance, most don't.
Neither The Whitewater Group's Actor nor Duff's Smalltalk supports multiple
inheritance, but both companies are looking into the technique. When a
customer asks about multiple inheritance, the companies ask what the customer
has in mind to do with it.
Duff says that the customers usually haven't thought about it very much. "It's
a bullet on a features list," he says. "The people who have thought about it
know it's fraught with pitfalls."
What are these pitfalls of multiple inheritance? Well, it's naive to think
that you can combine an A and a B to come up with a meaningful AB. A and B
typically will have similar but not quite identical properties that collide
with each other. Consider the case of two multiply inherited classes with
instance variables having the same name. Each may want to preserve its own
copy of that instance variable, or they may want to share it. There are
equally valid cases of each kind. When faced with these collisions, you have
to decide what to do about them; specifically, you have to establish a
collisions-resolving philosophy. Do you just employ an arbitrary rule? That
really isn't good enough. Do you ask the programmer on a case-by-case basis to
make the resolution? Do you develop a collision-resolving expert system? It's
not a simple matter.
"It's things like this that have kept us from approaching multiple inheritance
until just recently," Duff says. Anderson can think immediately of one
instance in the design of his Smalltalk system when multiple inheritance would
have come in handy. "It's nice to have a lot of things (that are not
descendants of the stream class) that act like streams." He managed to get the
effect of multiple inheritance without the structure, but "if we had had
multiple inheritance we would have used it there." But would the costs in
complexity that multiple inheritance brings with it have justified the
convenience that would have been gained in this case? Anderson doubts it.

Duff says you have to weigh these costs in designing a language. "You have to
say, 'Is the productivity benefit there? Are they spending more time in the
multiple inheritance technology than they would have just getting around it
with single inheritance?'" The Whitewater Group regularly teaches programmers
how to simulate multiple inheritance in a single inheritance system. In a
single inheritance environment, you can achieve multiple inheritance by adding
the classes that you want to inherit from as components or instance variables
and then using those components of the object as sort of experts to fill in
that domain of behavior. Then you just pass through the protocol of those
objects to the outside by introducing methods in the class. It works.
The approach suggests a broad design principle for object-oriented
programming, and it applies particularly strongly to multiple inheritance.
Duff says that his people have found that it's possible to overuse
inheritance. "In general, it's better to err in the direction of encapsulating
too much, in other words including something as an instance variable instead
of inheriting from it, because it's very likely that you might inherit
inappropriate behavior."
Inherit inappropriate behavior? Here's what he has in mind: "You want to
create something that holds a collection of other things, let's say a picture
class that holds a collection of graphical objects. One way to do that is to
make it a descendant of one of the collection classes, like Array. A picture
is an array, only it's a special kind of array. The problem is, you may want a
picture that can have a dictionary instead of an array as its collection.
Also, there are many things that are appropriate for arrays that are not
appropriate for pictures. You end up having to circumvent a lot of
inappropriate inherited behavior."
Multiple inheritance exacerbates the problem because with multiple inheritance
you have not just one but many classes from which you need to filter out
inappropriate behavior, in addition to resolving the redundancies. But Actor
is likely to support multiple inheritance before long.
"The direction that I'm heading," Duff explains, "is decoupling inheritance of
protocol --or methods --from inheritance of representation --or instance
variables. One way to do that is to be able to group a piece of code into a
thing that's like a class, except that it doesn't specify any representation.
It's just a set of methods that, since they don't refer to instance variables,
can be generically inherited. You can then say, in addition to the class that
I inherit from, I also inherit these protocols that allow me to behave like an
object of this virtual type --virtual (in that) you're not specifying anything
about its representation. It simplifies a lot of these resolution problems.
You don't have to worry about instance variables; you can just deal at the
level of what functionality I inherit."
And it's really not accurate for me to say that Digitalk doesn't support
multiple inheritance; it's there in the company's Smalltalk products. It's
just not visible. "In ST/VB-286 and the Mac product," Anderson says, "we have
actually implemented the mechanism for multiple inheritance, but we don't have
a (supporting) user interface. We don't know how to present it to the user."
The user interface is an area of active research at Digitalk, and Anderson
feels that the company was really founded to make object-oriented programming
accessible. Accessibility means a lucid user interface, he believes. If and
when they figure out how to present the technique so that it doesn't
complicate the programmer's life too much, they will no doubt implement
multiple inheritance for real. If and when they do, the solution is likely to
involve context, a concept that Digitalk programmers are currently exploring
avidly.
From the point of view of putting valuable functionality in the programmer's
hands, it's good that Digitalk and The Whitewater Group are moving toward
multiple inheritance, because there are real benefits in having multiple
inheritance: The real world has multiple inheritance, and if you want to
simulate aspects of the real world, should you try to get along without this
natural technique? On the other hand, the programmer has to decide if it is
worth the hassle. It's really not a trivial choice for the language designer
or for the programmer.
As Jim Anderson says, "It's definitely a controversial topic."






















































February, 1989
C PROGRAMMING


TINYCOMM: The Tiny Communications Program




Al Stevens


This month's column is about asynchronous serial communication and how it fits
into the "C Programming" column project. We will develop the first of a series
of tools designed to connect one computer to another with modems and the
telephone system. To illustrate these tools, we will build TINYCOMM, a tiny
communications program with the ability to originate or answer calls with
another computer, converse interactively with the keyboard and screen, capture
messages to a disk log file, and upload message files.
Just as December's TWRP, the Tiny Word Processor, is no competition for the
high-end word processors, TINYCOMM is no threat to Smartcom, Procomm, or
Qmodem. Our "C Programming" column project is slowly gathering a collection of
small but useful tools that we will eventually integrate into a program
targeted to a specific purpose. In the meantime we can learn from these small
examples, seeing how the C language is applied in building such applications.
You do not need an in-depth understanding of serial input/output and modems to
use this month's code, but it could help you. I recommend Mastering Serial
Communications by Peter W. Gofton, 1986, Sybex Inc. The book belabors the
complexity of serial I/O, and that is a genuine concern. You will learn here,
however, that the basic steps for making a modem connection to a remote
computer involve a minimum of C code.


Communications


When humans converse, we use channels of communication appropriate to the
conversations and our proximity to one another. The most common channel is the
spoken word issued within earshot of the participants. Less common are fortune
cookies, convicts clanking on water pipes, and bottled messages floating in
the ocean, but whatever the medium and the message, some are better suited
than others.
People can choose convenient media as time and opportunity permits, but when
computers converse, the medium must be exactly matched to the message. A
participant in an electronic conversation must use a medium known, agreeable,
and available to the other participants.
Therefore, if you want to send a message to your mother and she has a
computer, modem, and communications software, your computer can call her
computer. If hers can answer the phone and receive the message, she will
eventually get to read it. That may or may not require the participation of
one or both of you at the time of the transmission. She'd probably prefer a
call.


The Communicating Hardware


Programs that communicate with remote computers must manage two layers of
hardware: the modem and the serial port. The local modem translates the local
program's data into tones that are sent across telephone lines to the remote
modem. The local modem then receives the remote modem's tones and translates
them into data. The computer sends and receives data values to and from the
modem through the serial port. Management of this communication requires
functions for the serial port and the modem. Which way you select to do this
depends on the hardware itself. We will use the IBM PC serial ports and
Hayes-compatible modems.


The Asynchronous Serial Port


A serial port is a bit stream device that is used in applications where eight
data lines are not available and/or the distances between devices preclude the
use of the lower signal strengths of the computer.
The asynchronous serial port adds a start bit and one or two stop bits to each
byte. There may be a parity bit that may use odd or even parity. A byte may
consist of from five to eight data bits. The transmission speed is expressed
in bits per second and is called the "baud rate." Senders and receivers of
asynchronous serial messages must agree on the number of data bits; the number
of stop bits; whether odd, even, or no parity exists; and the baud rate of the
transmission.
The IBM PC can have one or two serial ports. A program can communicate with
one of these serial ports by reading and writing the port registers directly,
by calling the serial I/O ROM BIOS functions, or by using DOS. Each approach
has advantages and disadvantages. The book mentioned earlier explains these
trade-offs; space prevents me from offering you the comprehensive treatment
the subject requires. For our purposes, you may assume that I have weighed the
benefits and made the best selection for the examples at hand. We will use
direct port addressing and an interrupt service routine for the receiver side
of the serial transmissions.
These functions employ an XON, XOFF protocol where the receiving computer
effectively turns the sender's transmissions on and off. When its input buffer
gets nearly full, the receiver sends the XOFF character to the sender. The
sender suspends transmitting until the receiver sends the XON character. The
receiver watches its buffer as characters are removed and sends the XOFF when
the level is below the safety margin.
File transfer protocols such as XModem and Kermit use their own packet
management techniques to handle such delays. XModem can be a problem in
networks because the XModem protocol specification includes fixed timeout
periods that can be exceeded by latent delays introduced by the network. The
local communications software must know when one of these protocols is in use,
because they are often used to transmit binary files -- files that might have
the XON or XOFF characters as valid data bytes. During these transmissions,
the XON/XOFF protocol is disabled.
Listing One, page 138, is serial.h. This header file declares the prototypes
and macros for the serial port functions. It also defines status register
signals, the timer interrupt vector, and some control parameters for our
program. The comstat macro reads the status of the serial port. The
input_char_ready macro returns a true value if there is a byte waiting in the
serial input buffer and a false value if not.
Listing Two, page 138, is serial.c. These functions manage serial port
initialization, input, and output. The initcomport function initializes the
communications port by using the structure named initcom to contain the
initialization parameters that are written to the 8250 universal asynchronous
receiver/transmitter (UART). The integers named COMPORT, PARITY, STOPBITS,
WORDLEN, and BAUD are initialized with the default values assumed by the
functions.
The initcomport function sets up the serial receiver interrupt process. In
this process, all serial input characters are read by the interrupt service
routine named newcomint, which collects the characters as they are read into a
circular buffer. The initialization steps and the serial input and output
operations use definitions derived from the port number for COM1 or COM2.
Those definitions are in serial.h. (You can modify the code to work with other
machines that use the 8250 UART and the 8259 Programmable Interrupt Controller
by changing the base port address defined in serial.h as BASEPORT and the
interrupt request line named IRQ, also in serial.h.) First the current
contents of the serial interrupt vector are read and saved away. Then the
vector is initialized with the address of newcomint. The 8250 UART has two
registers that must be initialized. The program asserts the data terminal
ready (DTR), request to send (RTS), and user defined output 2 (OUT2) signals
in the modem control register, and writes the interrupt enable register with
the data available signal set to generate interrupts. The 8259 programmable
interrupt controller is written to allow the proper IRQ line to cause an
interrupt. Then the interrupt controller is reset and all UART input registers
read to clear any stray interrupts that might be hanging around.
The restore_serialint function in serial.c should be called at the end of a
program that has used serial input/output. This function returns the serial
interrupt vector to its original value.
The clear_serial_queue function resets the serial receiver's circular buffer
to an empty condition. This function is used by the modem manager to get rid
of any unneeded text condition responses from the modem.
The interrupt service routine newcomint is for serial input. It resets the
interrupt controller, reads the serial input byte, and stuffs it into the
circular buffer unless the character is an XON or XOFF with XON/XOFF protocols
enabled. In this case the function sets a flag so the transmitter knows to
suspend or resume transmissions. If XON/XOFF is enabled and the buffer is at
or beyond its threshold, newcominttransmits the XOFF character to tell the
sender to hold up for a while.
The readcomm function waits for a byte to be available in the receiver's
circular buffer and then extracts it for the calling function. A program that
polls for serial input should call this function only after getting a true
return from the input_char_ready macro because readcomm waits for a character
to be received. When readcomm sees that the buffer is at a safe level after
newcomint has transmitted the XOFF, readcomm transmits the XON character to
tell the sender to resume transmissions.
Programs that read ASCII text from a remote computer with seven data bits and
odd or even parity should logically AND the return from readcomm with 0x7f to
strip the parity bit. If you are reading a binary stream, such as an archived
file, you should accept the full eight-bit value.
The writecomm function sends the byte in its parameter to the remote computer
through the serial port. Note that readcomm and writecomm return a false value
if the port times out as a result of the elapse of the programmed TIMEOUT
value with no character being received or written. TIMEOUT is defined in
serial.c and is expressed in seconds.
The timer functions in serial.c provide another example of the use of the
interrupt function type. These timer functions process timeouts or suspend
processing for a specified number of seconds. Any program that will use the
functions in serial.c must first call intercept_timer to attach to the timer
interrupt vector. Before returning to DOS, the program must call restore_timer
to reset the vector. If you use the modem functions in modem.c you do not need
to make these calls, because the modem functions do it for you.
serial.h has two macros named set_timer and timed_out that operate the timer.
The first macro sets a timer value in seconds. The second macro returns a true
value if the most recent setting has elapsed. set_timer puts a value into the
ticker variable. The value in ticker is approximately 18.2 times the number of
seconds to count; the timer interrupt occurs 18.2 times per second. The
newtimer interrupt chains the interrupt; then, if ticker is greater than zero,
newtimer decrements it. When the timed_out macro finds that ticker is not
greater than zero, it returns a true value.
sleep uses the timer to suspend the program for a specified number of seconds.
Turbo C already has just such a function in its library, but Microsoft C does
not.
Note that the Turbo C functions getvect and setvect are used to read and write
the serial and timer interrupt vectors. If you are using Microsoft C, these
are changed to _dos_getvect and _dos_setvect, the MSC equivalents. If you are
porting this software to a compiler that does not support the interrupt
function type, you must use assembly language for the entry and exit to each
of the interrupt functions. The book I mentioned earlier has an example of
this technique.
The serial port detects the break condition, framing error, parity error, and
overrun error, any of which might happen. Why do we not check and correct for
these conditions? We do not for two reasons: First, the exchange of
human-readable ASCII data assumes that the user can tell when the characters
being read are incorrect. You can visually compensate for these so-called
"line hits" or terminate the transmission if the errors exceed a tolerable
threshold. Second, the other exchanges--uploading and downloading files with
XModem, Kermit, et al.--have their own error-correcting protocols involving
checksums, timeouts, and retries. We will explore these techniques in the
months to come.


The Modem



Modems are semi-intelligent devices that connect remote computers with
telephone lines and that can be programmed to operate in different ways. This
programming implies a language for the modem--a way to set the modem's modes
and read the modem's status. The accepted standard for this language is the
Hayes command set.
Because a modem supports terminal connections, it recognizes and responds with
language that people can understand. Well, almost. The Hayes command set is
hardly what you would call a natural language, but a person can learn it.
Oddly, the modem speaks a more cogent language than it understands. You tell
it ATEOM1V1S0 = 0 and it answers OK. You say ATDT 17033710188 and it says
BUSY, CONNECT, NO ANSWER, or NO CARRIER. You can learn its language if you
want, but any communications program worth its salt will know the language and
hide it from you.
Listing Three, page 139, is modem.h. It has configuration parameters and
prototypes for the modem functions. The parameters' strings initialize and
reset the modem, dial a number, and answer an incoming call. In some
communications programs, these parameters are maintained in a configuration
file built by a setup program. We'll just hardcode them this way. The curly
characters in the RESETMODEM, INITMODEM, and HANG UP parameters are not part
of the Hayes command set. Rather, they tell the modout function to wait one
second while sending a command to the modem.
Vendors who sell so-called Hayes-compatible modems do not always get it right.
Also, many modems have configuration micro switches that set default values.
The string value of the INITMODEM parameter is one that works with the Toshiba
T-1000's internal modem and the US Robotics Courier 2400. This is a black art.
If you have modem problems with this program, you might need to mess with the
parameters, the micro switches, or both.
Listing Four, page 139, modem.c contains the functions that control the modem.
The initmodem function initializes the serial port and the modem. It calls
intercept_timer to let the program hook the timer interrupt vector. Programs
should call the next function, release_modem, before they terminate. It
restores the timer and serial interrupt vectors and resets the modem. If you
fail to use this procedure, your computer will go off into the reeds and the
bulrushes when you exit to DOS. The placecall function dials the number in the
PHONENO parameter. The answer-call function prepares the modem to
automatically answer the phone. The disconnect function disconnects the modem
from the phone line. The modout function is used by the others to send a
command to the modem. The function tests for the curly character in the
command string and modout calls the sleep function in serial.c to tell the
program to wait one second for each curly character.
A program that supports direct connection of the serial ports of two computers
with no modems will set the direct_connection variable to a true value, which
suppresses the modem commands. This mode is one way to test a communications
program when two phone lines are not available. You must connect the serial
ports with a "null modem" cable, which crosses the send and receive lines --
usually pins 2 and 3 -- in the cable connectors.


TINYCOMM


The code in serial.h, serial.c, modem.h, and modem.c represents the primitive
functions required to communicate by modem. To use these functions, you need a
higher-level communications program. Listing Five, page 139, is tinycomm.c,
the bare beginnings of such a program. Its purpose is to demonstrate the
application of the communications functions, but it has a good bit of
functionality packed into such a tiny package.
When you run TINYCOMM, you can specify the serial port -- 1 or 2 -- and the
telephone number on the command line as shown here:
 C>tinycomm 2 555-1212
After this command, you see this menu.
 ------ TINYCOMM Menu ------
P-lace Call
A-nswer Call
H-ang Up
L-og Input [OFF]
S-end Message File
T-elephone Number (???-????)
E-xit to DOS

Enter Selection >
The TINYCOMM menu selections allow you to place a call, tell TINYCOMM to
prepare to answer a call, hang up, turn the disk logging off traffic on and
off, send an ASCII file to the other end, reprogram the telephone number, and
exit to DOS.
Once a connection is made, TINYCOMM sends the characters you type and displays
the characters it receives. These displays include the modem's messages, such
as CONNECT and NO CARRIER. TINYCOMM expects you to read these messages and
react appropriately. There is no automatic recognition of any text input to
TINYCOMM. You get in and out of the menu by pressing the Esc key.
TINYCOMM's message, upload, and log features use a straight ASCII protocol
with XON/XOFF enabled. This is acceptable for message traffic but would not
work at all for the transfer of binary formats such as archived or executable
files. File transfer protocols will come in a future installment.


Ctrl Break


When a program takes over an interrupt vector, the program must not terminate
abnormally. If it does, the interrupt vector will still point to the memory
formerly occupied by the program. The next time the interrupt happens, the
system will be awry. TINYCOMM hooks the timer and serial interrupt vectors, so
we must not allow it to be terminated other than through the normal exit point
of the program where these vectors are restored.
When you press Ctrl C or Ctrl Break, DOS normally displays the ^C token and
aborts the program. This would be one of those unwanted terminations I just
mentioned. You can take over the Ctrl Break and Ctrl C interrupt vectors (0x1b
and 0x23) and prevent the termination, but the stupid ^C token still gets
displayed, messing up your well-planned screen display and moving your cursor.
One way to defeat this dubious feature of DOS is to avoid using DOS for
keyboard input and screen output functions. TINYCOMM uses Turbo C's getch and
putch or its own keyboard and screen functions for Microsoft C, and these
measures effectively avoid DOS.
TINYCOMM polls the keyboard before calling getch and polls the serial buffer
before calling getcomm. This technique allows either device to get a character
in edgewise, which is necessary in a full duplex communications operation. The
two usual ways to poll the keyboard are the kbhit function and the bioskey
(TC) or _bios_key (MSC) function. Unfortunately, these functions involve the
Ctrl Break and Ctrl C logic, so we cannot use them. Instead we must use BIOS
to see if a key has been pressed. Both compilers have the int86 function, and
we can use this function to call BIOS interrupt 0xl6, which manages the
keyboard. The keyhit function in tinycomm.c uses int86 and the problem is
solved -- well, not quite. BIOS returns its results in the zero bit of the
CPU's flags register. Turbo C's int86 includes the flags register in the REGS
union written by int86; the Microsoft C version does not. As a consequence,
Microsoft C provides no way that I can find to poll the keyboard without the
offensive ^C showing up and perhaps aborting the program. To overcome this
obstacle, we use the assembly language function found in Listing Six,
keyhit.asm (page 140).
The keyhit function is the only Ctrl Break defensive measure we need with
Turbo C. Microsoft C is not as easy. The MSC getch, putch, and gets functions
let the break operation get into the act. Therefore, the mscgetch, mscputch,
and mscgets functions are added to MSC-compiled versions of tinycomm.c. The
gotoxy and clrscr functions are clones of similar Turbo C functions.
Listing Seven, page 140, is tinycomm.prj, the Turbo C environment project make
file. Set your compiler defines option (Alt-O/C/D) to these parameters:
 MSOFT=1;TURBOC=2;COMPILER=TURBOC
Listing Eight, page 140, is tinycomm.mak, the make file for Microsoft C. It
uses the small memory model.
Next month we'll overhaul the TINYCOMM program to use windows, help, menus,
and the other tools in our collection. We will add features to download files,
program the serial port's parameters from a menu, use the direct connection
features of the modem manager, access a phone directory, and maintain the
program's setup in a configuration file. We will insert the hooks to add file
transfer protocols (but not the protocols yet). As I develop this program, I
am testing it by using it for all my online activities, so if I disappear some
night in the middle of a heated online exchange, you'll know why.
_C Programming Column_
by Al Stevens


[LISTING ONE]

/* ---------- serial.h ---------------
 * Serial Port Definitions
 */
extern int ticker, COMPORT;
extern char *nextin, *nextout;
/* ----------- serial prototypes ----------- */
void initcomport(void);
int readcomm(void);
int writecomm(int);
void clear_serial_queue(void);
/* -------- timer prototypes --------- */

void sleep(unsigned);
int set_timer(unsigned);
void intercept_timer(void);
void restore_timer(void);
void restore_serialint(void);
/* ----------------- macros ------------------- */
#define comstat() (inp(LINESTATUS))
#define input_char_ready() (nextin!=nextout)
#define timed_out() (ticker==0)
#define set_timer(secs) ticker=secs*182/10+1
#define XON 17
#define XOFF 19
/* ---------------- serial port addresses ----------------- */
/* - 8250 UART base port address: COM1 = 3f8, COM2 = 2f8 - */
#define BASEPORT (0x3f8-((COMPORT-1)<<8))
#define TXDATA BASEPORT /* transmit data */
#define RXDATA BASEPORT /* receive data */
#define DIVLSB BASEPORT /* baud rate divisor lsb */
#define DIVMSB (BASEPORT+1) /* baud rate divisor msb */
#define INTENABLE (BASEPORT+1) /* interrupt enable */
#define INTIDENT (BASEPORT+2) /* interrupt ident'n */
#define LINECTL (BASEPORT+3) /* line control */
#define MODEMCTL (BASEPORT+4) /* modem control */
#define LINESTATUS (BASEPORT+5) /* line status */
#define MODEMSTATUS (BASEPORT+6) /* modem status */
/* --------------- serial interrupt stuff ------------------ */
#define IRQ (4-(COMPORT-1)) /* 0-7 = IRQ0-IRQ7 */
#define COMINT (12-(COMPORT-1)) /* interrupt vector 12/11*/
#define COMIRQ (~(1 << IRQ))
#define PIC01 0x21 /*8259 Programmable Interrupt Controller*/
#define PIC00 0x20 /* " " " " */
#define EOI 0x20 /* End of Interrupt command */
#define TIMER 0x1c /* PC timer interrupt vector */
/* --------------- line status register values ------------- */
#define XMIT_DATA_READY 0x20
/* ------------ modem control register values -------------- */
#define DTR 1
#define RTS 2
#define OUT2 8
/* ----------- interrupt enable register signals ------------ */
#define DATAREADY 1
/* ------------- serial input interrupt buffer -------------- */
#define BUFSIZE 1024
#define SAFETYLEVEL (BUFSIZE/4)
#define THRESHOLD (SAFETYLEVEL*3)
#ifndef TRUE
#define TRUE 1
#define FALSE 0
#endif



[LISTING TWO]]

/* ---------- serial.c ---------------
 * Serial Port Communications Functions
 */
#include <stdio.h>
#include <conio.h>

#include <dos.h>
#include "serial.h"

#if COMPILER == MSOFT
#define getvect _dos_getvect
#define setvect _dos_setvect
#endif

char recvbuff[BUFSIZE];
char *nextin = recvbuff;
char *nextout = recvbuff;
int buffer_count;
int COMPORT = 1; /* COM1 or COM2 */
int PARITY = 0; /* 0 = none, 1 = odd, 2 = even */
int STOPBITS = 1; /* 1 or 2 */
int WORDLEN = 8; /* 7 or 8 */
int BAUD = 1200; /* 110,150,300,600,1200,2400 */
int TIMEOUT = 10; /* number of seconds to time out */
int xonxoff_enabled = TRUE;
static int waiting_for_XON;
static int waiting_to_send_XON;
int ticker;

/* ----- the com port initialization parameter byte ------ */
static union {
 struct {
 unsigned wordlen : 2;
 unsigned stopbits : 1;
 unsigned parity : 3;
 unsigned brk : 1;
 unsigned divlatch : 1;
 } initbits;
 char initchar;
} initcom;
static void (interrupt far *oldtimer)(void);
static void interrupt far newtimer(void);
static void (interrupt far *oldcomint)(void);
static void interrupt far newcomint(void);

/* -------- initialize the com port ----------- */
void initcomport(void)
{
 initcom.initbits.parity = PARITY == 2 ? 3 : PARITY;
 initcom.initbits.stopbits = STOPBITS-1;
 initcom.initbits.wordlen = WORDLEN-5;
 initcom.initbits.brk = 0;
 initcom.initbits.divlatch = 1;
 outp(LINECTL, initcom.initchar);
 outp(DIVLSB, (char) ((115200L/BAUD) & 255));
 outp(DIVMSB, (char) ((115200L/BAUD) >> 8));
 initcom.initbits.divlatch = 0;
 outp(LINECTL, initcom.initchar);
/* ------ hook serial interrupt vector --------- */
 if (oldcomint == NULL)
 oldcomint = getvect(COMINT);
 setvect(COMINT, newcomint);
 outp(MODEMCTL, (inp(MODEMCTL) DTR RTS OUT2));
 outp(PIC01, (inp(PIC01) & COMIRQ));
 outp(INTENABLE, DATAREADY);

 outp(PIC00, EOI);
/* ----- flush any old interrupts ------ */
 inp(RXDATA);
 inp(INTIDENT);
 inp(LINESTATUS);
 inp(MODEMSTATUS);
}

/* ------ restore the serial interrupt vector ---------- */
void restore_serialint(void)
{
 if (oldcomint)
 setvect(COMINT, oldcomint);
}

/* ------- clear the serial input buffer --------- */
void clear_serial_queue(void)
{
 nextin = nextout = recvbuff;
 buffer_count = 0;
}

/* ---- serial input interrupt service routine ------- */
static void interrupt far newcomint(void)
{
 int c;
 outp(PIC00,EOI);
 if (nextin == recvbuff+BUFSIZE)
 nextin = recvbuff; /* circular buffer */
 c = inp(RXDATA); /* read the input */
 if (xonxoff_enabled)
 if (c == XOFF) /* test XON */
 waiting_for_XON = 1;
 else if (c == XON) /* test XOFF */
 waiting_for_XON = 0;
 if (!xonxoff_enabled (c != XON && c != XOFF)) {
 *nextin++ = (char) c; /* put char in buff*/
 buffer_count++;
 }
 if (xonxoff_enabled && !waiting_to_send_XON &&
 buffer_count > THRESHOLD) {
 while ((inp(LINESTATUS) & XMIT_DATA_READY) == 0)
 ;
 outp(TXDATA, XOFF); /* send XOFF */
 waiting_to_send_XON = 1;
 }
}

/* ---- read a character from the input buffer ----- */
int readcomm(void)
{
 set_timer(TIMEOUT);
 while (!input_char_ready())
 if (timed_out())
 return FALSE;
 if (nextout == recvbuff+BUFSIZE)
 nextout = recvbuff;
 --buffer_count;
 if (waiting_to_send_XON && buffer_count < SAFETYLEVEL) {

 waiting_to_send_XON = 0;
 writecomm(XON);
 }
 return *nextout++;
}

/* ---- write a character to the comm port ----- */
int writecomm(int c)
{
 while (waiting_for_XON)
 ;
 set_timer(TIMEOUT);
 while ((inp(LINESTATUS) & XMIT_DATA_READY) == 0)
 if (timed_out())
 return FALSE;
 outp(TXDATA, c);
 return TRUE;
}

/* ---- intercept the timer interrupt vector ----- */
void intercept_timer(void)
{
 if (oldtimer == NULL) {
 oldtimer = getvect(TIMER);
 setvect(TIMER, newtimer);
 }
}

/* ---------- sleep for n seconds ------------ */
void sleep(unsigned secs)
{
 set_timer(secs);
 while (!timed_out())
 ;
}

/* ---- restore timer interrupt vector ------- */
void restore_timer()
{
 if (oldtimer)
 setvect(TIMER, oldtimer);
}

/* ------ ISR to count timer ticks ------- */
static void interrupt far newtimer()
{
 (*oldtimer)();
 if (ticker)
 --ticker;
}




[LISTING THREE]

/* -------- modem.h ------------
 * Modem Definitions
 */

/* -------- Hayes modem control strings --------- */
#define RESETMODEM "ATZ\r~"
#define INITMODEM "ATE0M1S7=60S11=55V1X3S0=0\r~"
#define HANGUP "~+++~ATH0\r~ATS0=0\r~"
#define ANSWER "ATS0=1\r~"
/* --------- prototypes ---------- */
void initmodem(void);
void placecall(void);
void answercall(void);
void disconnect(void);
void release_modem(void);



[LISTING FOUR]

/* ------------ modem.c --------- */

#include <dos.h>
#include <conio.h>
#include "serial.h"
#include "modem.h"

char DIAL[] = "ATDT";
char PHONENO[21];

int direct_connection; /* true if connected without a modem */

/* ----------- write a command to the modem ------------ */
static void modout(char *s)
{
 while(*s) {
 if (*s == '~')
 sleep(1);
 else if (!writecomm(*s))
 break;
 s++;
 }
}

/* ----------- initialize the modem ---------- */
void initmodem(void)
{
 intercept_timer();
 initcomport();
 if (!direct_connection) {
 modout(RESETMODEM);
 modout(INITMODEM);
 }
}

/* -------- release the modem --------- */
void release_modem(void)
{
 if (!direct_connection)
 modout(RESETMODEM);
 restore_timer();
 restore_serialint();
}


/* ----------- place a call -------------- */
void placecall(void)
{
 if (!direct_connection) {
 modout(DIAL);
 modout(PHONENO);
 modout("\r");
 clear_serial_queue();
 }
}

/* ------------- answer a call ------------ */
void answercall(void)
{
 if (!direct_connection) {
 modout(ANSWER);
 clear_serial_queue();
 }
}

/* ------------ disconnect the call ----------------- */
void disconnect(void)
{
 if (!direct_connection) {
 modout(HANGUP);
 clear_serial_queue();
 }
}


[LISTING FIVE]

/* ------ tinycomm.c ---------- */

#include <stdio.h>
#include <string.h>
#include <ctype.h>
#include <conio.h>
#include <stdlib.h>
#include <dos.h>
#include "serial.h"
#include "modem.h"

#if COMPILER==MSOFT
#define getch mscgetch
#define putch mscputch
#define gets mscgets
static void mscgets(char *);
static int mscgetch(void);
static void mscputch(int);
static void gotoxy(int,int);
static void clrscr(void);
#endif
int keyhit(void);

#define TMSG "\r\n\r\nTINYCOMM: << %s >>\r\n"
#define LOGFILE "tinycomm.log"
#define ESC 27

extern char PHONENO[];
extern int COMPORT;
static FILE *logfp;
static FILE *uploadfp;
static int running=1;
static int connected,answering;
static int forcekey, forcecom, forcemenu;
static union REGS rg;
/* ----- prototypes ------ */
static void tinymenu(void);
static void log(int);
static void upload(void);

void main(int argc, char *argv[])
{
 int c;
 if (argc > 1)
 COMPORT = atoi(argv[1]);
 if (argc > 2)
 strcpy(PHONENO, argv[2]);
 initmodem();
 while (running) {
 if (!connected forcemenu) {
 forcemenu = 0;
 tinymenu(); /* display and process the menu */
 }
 /* ------ poll for a keystroke --------- */
 if (keyhit() forcekey) {
 c = forcekey ? forcekey : getch();
 forcekey = (answering && c == '\r') ? '\n' : 0;
 if (c == ESC)
 tinymenu();
 else if (connected) {
 if (answering)
 log(c); /* answerer echos his own key */
 writecomm(c); /* transmit the keystroke */
 }
 }
 /* ------- poll for serial input ---------- */
 if (input_char_ready() forcecom) {
 c = forcecom ? forcecom : readcomm();
 forcecom = (answering && c == '\r') ? '\n' : 0;
 log(c); /* display the serial input */
 if (answering)
 writecomm(c); /* answerer echos serial input */
 }
 }
 release_modem();
}

/* ------- display and process the TINYCOMM menu --------- */
static void tinymenu(void)
{
 int c;
 clrscr();
 gotoxy(20,5), cprintf("------ TINYCOMM Menu ------");
 gotoxy(20,7), cprintf("P-lace Call");
 gotoxy(20,8), cprintf("A-nswer Call");
 gotoxy(20,9), cprintf("H-ang Up");

 gotoxy(20,10), cprintf("L-og Input %s",
 logfp == NULL ? "[OFF]" : "[ON]");
 gotoxy(20,11), cprintf("S-end Message File");
 gotoxy(20,12), cprintf("T-elephone Number (%s)",
 PHONENO[0] ? PHONENO : "???-????");
 gotoxy(20,13), cprintf("E-xit to DOS");
 gotoxy(20,14), cprintf(connected ?
 "Esc to return to session" : "");
 gotoxy(20,16), cprintf("Enter Selection > ");
 c = getch();
 putch(toupper(c));
 switch (toupper(c)) {
 case 'P': /* Place a call */
 if (!connected) {
 cprintf(TMSG, "Dialing");
 initmodem(); /* initialize the modem */
 placecall(); /* dial the phone number */
 connected = 1;
 cprintf(TMSG, "Esc for the menu");
 }
 break;
 case 'A': /* Answer a call */
 if (!connected) {
 cprintf(TMSG, "Waiting");
 initmodem(); /* initialize the modem */
 answercall(); /* wait for an incoming call */
 answering = connected = 1;
 cprintf(TMSG, "Esc for the menu");
 }
 break;
 case ESC: /* Return to the session */
 if (connected)
 cprintf(TMSG, "Esc for the menu");
 break;
 case 'L': /* Log input on/off*/
 if (logfp == NULL)
 logfp = fopen(LOGFILE, "a");
 else {
 fclose(logfp);
 logfp = NULL;
 }
 forcemenu++;
 break;
 case 'E': /* Exit to DOS */
 cprintf(TMSG, "Exiting");
 running = 0;
 case 'H': /* Hang up */
 if (connected) {
 cprintf(TMSG, "Hanging up");
 disconnect();
 connected = answering = 0;
 }
 break;
 case 'S': /* Send a message file */
 upload();
 break;
 case 'T': /* Change the phone number */
 cprintf(TMSG, "Enter Telephone Number: ");
 gets(PHONENO);

 forcemenu++;
 break;
 default:
 putch(7);
 break;
 }
}

/* --------- upload an ASCII file ---------- */
static void upload(void)
{
 char filename[65];
 int c = 0;
 if (uploadfp == NULL && connected) {
 cprintf(TMSG, "Enter file drive:path\\name > ");
 gets(filename);
 if ((uploadfp = fopen(filename, "r")) == NULL)
 cprintf(TMSG, "Cannot open file");
 else {
 cprintf(TMSG, "Press Esc to stop sending file");
 while ((c = fgetc(uploadfp)) != EOF) {
 if (c == '\n') {
 writecomm('\r');
 log(answering ? '\r' : readcomm());
 }
 writecomm(c);
 log(answering ? c : readcomm());
 if (keyhit())
 if (getch() == ESC) {
 cprintf(TMSG, "Abandoning file");
 break;
 }
 }
 fclose(uploadfp);
 uploadfp = NULL;
 }
 }
}

/* ----- echo modem or keyboard input and write to log ----- */
static void log(int c)
{
 putch(c);
 if (logfp)
 fputc(c, logfp);
}

/* --------------------------------------------------------
 Clone functions to keep Ctrl-Break from crashing the
 system by aborting before interrupt vectors get restored
 -------------------------------------------------------- */
#if COMPILER==TURBOC
/* --------- use bios to test for a keystroke -------- */
int keyhit()
{
 rg.h.ah = 1;
 int86(0x16, &rg, &rg);
 return ((rg.x.flags & 0x40) == 0);
}

#else
/* ------- substitute for getch for MSC --------- */
static int mscgetch(void)
{
 rg.h.ah = 0;
 int86(0x16, &rg, &rg);
 return rg.h.al;
}

/* ------- substitute for putch for MSC --------- */
static void mscputch(int c)
{
 rg.x.ax = 0x0e00 (c & 255);
 rg.x.bx = 0;
 int86(0x10, &rg, &rg);
}

/* -------- gotoxy clone ------------ */
static void gotoxy(int x, int y)
{
 rg.h.ah = 2;
 rg.x.bx = 0;
 rg.h.dh = (char) y-1;
 rg.h.dl = (char) x-1;
 int86(0x10, &rg, &rg);
}

/* -------- clrscr clone ------------- */
static void clrscr(void)
{
 rg.x.ax = 0x0600;
 rg.h.bh = 7;
 rg.x.cx = 0;
 rg.x.dx = (24 << 8) + 79;
 int86(0x10, &rg, &rg);
}

/* ----------- gets clone ------------- */
static void mscgets(char *s)
{
 int c;
 while (1) {
 if ((c = mscgetch()) == '\r')
 break;
 mscputch(c);
 *s++ = (char) c;
 }
 *s = '\0';
}
#endif



[LISTING SIX]

; ------------- keyhit.asm ---------------
;
; Use this in MSC C programs in place of kbhit
; This function avoids Ctrl-Break aborts

;
_text segment para public 'code'
assume cs:_text
public _keyhit
_keyhit proc near
 mov ah,1
 int 16h
 mov ax,1
 jnz keyret
 mov ax,0
keyret: ret
_keyhit endp
_text ends
 end



[LISTING SEVEN]

tinycomm (serial.h, modem.h)
modem (serial.h, modem.h)
serial (serial.h)


[LISTING EIGHT]


# TINYCOMM.MAK: make file for TINYCOMM.EXE with Microsoft C/MASM
#

.c.obj:
 cl /DMSOFT=1 /DCOMPILER=MSOFT -c -W3 -Gs $*.c

tinycomm.obj : tinycomm.c serial.h modem.h window.h

modem.obj : modem.c serial.h modem.h

serial.obj : serial.c serial.h

keyhit.obj : keyhit.asm
 masm /MX keyhit;

tinycomm.exe : tinycomm.obj modem.obj serial.obj keyhit.obj
 link tinycomm+modem+serial+keyhit,tinycomm,,\lib\slibce


















February, 1989
GRAPHICS PROGRAMMING


Pixelmania




Kent Porter


Graphics programming is enthralling because it gives immediate feedback,
translating your ideas into vivid visual images. It's also challenging --in
many ways it's one of the most demanding forms of programming --and overcoming
its complexities enhances the satisfaction of it. End users have come to
expect more and better graphics, which makes the discipline increasingly
important. For all these reasons and more, we embark this month on a voyage
into the enchanted world of graphics programming.
In this column I'm going to discuss the whole scope of computer graphics,
starting with the writing of pixels --the basic unit of visual information
--and building from there. In time I'll get to such lofty subjects as shading
and light sourcing and photorealism.
High-level graphics rest upon a hierarchy of lower-level functions. As a
result, one of the chief projects of this column will be to develop a graphics
application program interface (API), a library of routines supporting ever
more sophisticated techniques for producing dazzling visual images. The API
will unfold month by month, building not only a useful toolset, but also an
understanding of the concepts and algorithms.
We're going to write the API in generic ANSI C, along with some assembly
language routines for performance. A number of C compilers come with built-in
graphics libraries, but because there's no consistency from one to the next,
the API will be completely independent of any vendor's library or C dialect.
The hardware model is the EGA/VGA on IBM PCs and compatibles. Note that the
CGA isn't included. The reason is simple: You can't do serious graphics
development on a board that provides only four colors at low resolution. Some
might say you can't on the EGA, either, which furnishes 16 out of 64 colors.
While there's merit to that point, the EGA is widely available, and it
provides a foundation for the VGA. Initially we'll work with the EGA, but
eventually we'll move on to the VGA's much richer set of 256 colors at a time.
This puts you at a disadvantage if you're working with a different hardware
environment, but it doesn't exclude you. Virtually all of the low-level
machine-specific stuff will appear between now and June: routines that operate
on pixels or otherwise interface directly with the video controller. Other
hardware graphics systems differ in details, but all must read and write
pixels. So, if you're developing on a different machine or video board, write
low-level routines that parallel those presented here. Most of the
higher-level functions in C can then be used "as is" to keep you in step with
the column. I wish I could publish equivalent low-level routines for other
systems, but if I did, I'd never get anything else done. And my purpose is,
after all, to explore the meatier issues and techniques of graphics
programming.
The C code written here compiles with Turbo C 2.0 and Microsoft C 5.1. For the
assembly language routines, I'm using Microsoft MASM 5.1 syntax, which also
assembles under Borland's TASM.
Now let's get started.


How to Write a Pixel


The elementary particle of graphics information is the pixel, a contraction
for "picture element." A pixel has two characteristics: position and color.
Its position is expressed by a coordinate pair consisting of the X (or
horizontal) location and the Y (vertical) position. The top left corner of the
display is at {0, 0}. X coordinates increase to the right, and Y coordinates
increase downward. Thus, on a display of 640x350, which is the best EGA
resolution, the lower right corner of the display is at {639,349}, and the
center is at {320, 175}. Note that the X coordinate is always expressed first
in the pair.
Most PC-based graphics systems --the EGA, VGA, and even more exotic chips such
as the TI 340X0 --use an indexed color scheme. Here, a pixel's color is not
absolute but instead is taken from a set of color registers called the
palette. You tell a pixel to assume a certain value. That value refers to a
palette register that contains the actual color, which is a zoned bitfield
containing a combination of red, green, and blue hues that comprise the color
blend appearing at the pixel position. Thus, the pixel value is an index to a
color register. As it refreshes each pixel, the video adapter uses the pixel
value to look up the palette register where the real color is found.
In its default condition, the EGA/VGA has certain predictable values in the
palette registers. For example, palette register 1 contains medium blue. Thus,
PC programmers get used to thinking that 1 means blue. By changing the content
of palette register 1, however, you can instantly change all pixels with value
1 to a different color.
I'll talk more about color control in April. Meanwhile, use the default
palette with the understanding that a pixel value refers to a palette
register, which can contain any of 64 colors. The EGA has 16 palette
registers, and so does the VGA when running in EGA emulation mode; other VGA
modes have 256 palette registers with a much broader selection of colors.
The simplest way to write a pixel is to call the PC's ROM BIOS video services
via interrupt 10h, function 0Ch. In C notation, you might write such a call as
follows:
 void draw_point (int x, int y, int pxval)
 {
 union REGS r;
 r.h.ah = 0x0C;
 r.h.al = pxval;
 r.x.cx = x;
 r.x.dx = y;
 int86 (0x10, &r, &r);
 }
To place a blue pixel in the center of the screen, then, you'd write
 draw_point (320, 175, 1);
Unfortunately, simplest is not always best. The ROM BIOS video services are
shockingly inefficient. On my 10-MHz AT clone, the best throughput is a lazy
2,000 pixels per second using this C function. By going around the ROM BIOS to
interface directly with the 6845 video controller using assembly language, you
can achieve a 10.5X increase in throughput --to around 25,000 pixels per
second --without sacrificing reliability. And in the special case discussed
next month, a 250X improvement (yes, an amazing half a million pixels per
second) is possible. To accomplish this miracle, it's necessary to understand
something about the 6845 video controller chip (VCC) and the way pixel values
are represented.


Programming 6845 Write Mode 2


Video memory in the EGA/VGA consists of color planes. There are as many color
planes as there are possible bits in a pixel value. In EGA 16-color mode 10h,
a pixel can have any value from 0h to 0Fh, or four bits, and thus there are
four color planes.
The exact details of how this is implemented are not important, because the
VCC manages it for us. What is important to realize is that pixels are grouped
by byte across the width of the screen. With a horizontal granularity of 640
pixels, the display is conceptually 80 bytes wide. But under each byte are
layered a number of other bytes that you can't see: in EGA mode 10h, three
others for a total of four.
A pixel's value is determined by finding its bit position within the topmost
byte, then peering downward through the corresponding bit positions in the
other three layers. The bits comprising the pixel value are thus "stacked,"
with each bit on a different plane. A plane is a region of display memory:
28,000 bytes for a 640x350 display. The pixel at the top left corner derives
its value from the high-order bits of the bytes at offsets 0, 28,000, 56,000,
and 84,000.
At first glance, such a scheme makes you wonder how much the designers had to
drink before they dreamed it up. Closer examination, however, reveals the
method of the madness: You can add another color plane simply by tacking an
additional 28,000 bytes onto the end of the display memory, without
rearranging the mapping of bytes to pixels.
The 6845 VCC takes care of all the bit-twiddling required to read and write
pixels. By writing a series of values to the VCC via ports in the range 3C4h
through 3CEh, you program the 6845 to perform certain pixel-oriented
operations. Once you've programmed the VCC, read from or write to the pixel's
plane 0 video buffer address, and the VCC transparently intervenes to perform
the plane indexing.
In order to update individual pixels, set up the VCC for Write Mode 2. It's
also necessary to load a bit mask specifying the pixel(s) within the byte
(which represents eight pixels) to be updated. Then read the pixel's byte
location in plane 0 of the video buffer. This loads the four plane bytes into
the 6,845 latch registers. By writing 0 to the video buffer address, you clear
the mask-specified pixel(s); then you write the new palette register value to
the same address. The VCC updates the affected bits in the latch registers and
copies the latches back to their proper memory locations. The pixel change
takes effect instantly.
Listing One, page 142, is the low-level assembly language routine DRAWPT. ASM,
which performs this 6845 VCC interface function. It's written to be called
from C with the prototype
 void far draw_point (int x, int y);
As mentioned earlier, this function achieves over ten times the performance of
the ROM BIOS equivalent.
Note that public and external symbols must be prefixed in assembly language
with an underscore. This complies with C linkage conventions. Note also that
DRAWPT.ASM relies on an external variable called color1 to furnish the new
pixel value. I discuss where that variable comes from next.



Starting the GRAFIX API


The development of a graphics API begins with the bare-bones library of four
functions described in GRAFIX.H (Listing Two, page 146). These functions are:
Put the video system into a graphics mode.
Restore the system to its original text mode.
Write a pixel.
Select the current pixel value.
Of these functions, one is a separate program (DRAWPT.ASM from Listing One),
while the other three are found in GRAFIX.C (Listing Three, page 146). After
assembling DRAWPT.ASM and compiling GRAFIX.C, create a library with the DOS
utility command
 LIB GRAFIX + GRAFIX+DRAWPT;
This pulls GRAFIX.OBJ and DRAWPT. OBJ into the single object file GRAFIX.LIB,
with which you can link your application programs.
Let's briefly discuss the elements in GRAFIX.C. The init_video( ) function
places the graphics subsystem into the mode specified via the argument. The
two modes currently supported are EGA and VGA16, defined in GRAFIX.H. This
function does a number of things, as its length suggests.
Lines 46 - 76 inventory the video subsystem. Jeff Duntemann's first column
arrived as I was writing this routine. By a happy coincidence it contained a
method for detecting the video equipment that's simpler than mine, so I
adapted Jeff's code to suit. For more on how it works, see the "Structured
Programming" column in this issue. Note that the VGA emulates EGA modes, so
both the ega and vga variables are set to TRUE when a VGA is detected.
Assuming that an EGA or VGA is present, lines 78 - 98 perform a number of
set-up operations:
Select the default pixel color.
Save the current text mode, video page, and cursor position.
Copy the text screen to a save area on the heap so that it can be restored
intact later when you return from graphics mode.
Lines 97 - 109 then switch the system to graphics, providing that the desired
mode is supported. Success is indicated by setting the result variable to
TRUE. Lines 110 - 117 check the result and either deallocate the text screen
save area when unsuccessful or save the graphics mode and register
pc_textmode( ) as an exit function when the system has been put into graphics.
The function returns the result variable so that the caller can find out if
the call succeeded.
init_video( ) is by far the most complex function in the library, and it will
probably remain so. But we're not done with it yet. In later installments I'll
expand it to accommodate new video modes, as well as to initialize the color
palette and a data structure describing the display characteristics. It's
complete enough for now, though.
The function pc_textmode( ) restores the graphics system to its original
state, with the screen as it was prior to invoking graphics and the cursor
correctly placed. This function runs only if the system is in graphics; the
oldmode variable is always 0 (FALSE) when in text mode, and something else
when in graphics. There's a reason: init_video( ) registers, pc_textmode( ) as
an exit function, which is automatically called on program termination. There
is no C function to undo the atexit( ) call, so pc_textmode( ) will be called
when the program ends no matter what. If you were already in text mode and
this safeguard didn't exist, the call to pc_textmode( ) would crash the system
or corrupt the display.
The final function is set_color1. It simply stores the argument --the
currently-selected palette register for pixels --into the color1 variable.
This variable is used by the pixel-writing routine and will also be used by
the fast line-drawing routines to be presented next month.
The GRAFIX library is an absolute minimal API. Though it lacks functionality,
it can be made to do some useful work. The STRIPES.C program (Listing Four,
page 146) furnishes a small program that you can use to test your copy of the
library. The program draws a solid rectangle in the center of the display with
multicolored stripes running northeast to southwest. It freezes the display
until you press a key and then restores the original text screen. After
compiling the program, link it with GRAFIX.LIB and the runtime library for the
compiler and memory model.


Must Reading for PC Graphics Programmers


If you do much graphics programming, a necessary addition to your reference
bookshelf is Richard Wilton's Programmer's Guide to PC & PS/2 Video Systems,
published by Microsoft Press in 1987. This book could almost be subtitled
"More Than Anyone Would Ever Want to Know About..." Its 531 pages bulge with
excruciatingly detailed information about the CGA, MCGA, EGA, VGA, and
Hercules, including numerous listings in assembly language and a fair number
in C as well. Wilton presents alternative algorithms for such challenging
problems as drawing circles and doing flood fills, with lucid discussions of
which is best for a given circumstance. My copy is getting dogeared from heavy
use, and the only criticism I have so far is that the listings are in faded
green, which is hard to read. This book is just super.
So there we have it, the modest beginnings of an epic voyage into the
infinitely fascinating innards of graphics programming. I invite you to rejoin
me here each month as we explore the crystal caverns of this magical craft.
_Graphics Programming Column_
by Kent Porter


[LISTING ONE]

; DRAWPT.ASM: Writes pixel directly to 6845 Video Controller
; Microsoft MASM 5.1
; C prototype is
; void far draw_point (int x, int y);
; To be included in GRAFIX.LIB
; K. Porter, DDJ Graphics Programming Column, February '89

.MODEL LARGE
.CODE
 PUBLIC _draw_point
 EXTRN _color1 : BYTE ; From GRAFIX.LIB
x EQU [bp+6] ; Arguments passed from C
y EQU [bp+8]

_draw_point PROC FAR
 push bp ; Entry processing
 mov bp, sp

; Point ES to video memory segment
 mov ax, 0A000h
 mov es, ax


; Row offset = y * 80;
 mov bx, y ; Get y argument
 mov ax, 80
 mul bx ; Result in AX
 mov bx, ax ; Row offset in BX

; Column offset = x SHR 3
 mov ax, x ; Get x
 mov cl, 3 ; Shift operand
 shr ax, cl ; Column offset

; Complete address of pixel byte
 add bx, ax ; ES:BX = address

; Build bit mask for pixel
 mov cx, x ; Get x again
 and cx, 7 ; Isolate low-order bits
 xor cl, 7 ; Number of bits to shift
 mov ah, 1 ; Start bit mask
 shl ah, cl ; Shift for pixel
 mov cl, ah ; Save it

; Use write mode 2 (single-pixel update)
 mov dx, 03CEh ; 6845 command register
 mov al, 5 ; Specify mode register
 mov ah, 2 ; Write mode 2
 out dx, ax ; Send

; Set 6845 bit mask register
 mov al, 8 ; Specify bit mask register
 mov ah, cl ; al = mask
 out dx, ax ; Send bit mask

; Draw the pixel
 mov al, es:[bx] ; Load 6845 latch registers
 xor al, al ; Clear
 mov byte ptr es:[bx], al ; Zero the pixel for replace
 mov al, _color1 ; Get the pixel value
 mov es:[bx], al ; Write the pixel

; Restore video controller to default state
 mov dx, 03CEh
 mov ax, 0005h ; write mode 0, read mode 0
 out dx, ax
 mov ax, 0FF08h ; default bit mask
 out dx, ax
 mov ax, 0003h ; default function select
 out dx, ax
 xor ax, ax ; zero Set/Reset
 out dx, ax
 mov ax, 0001h ; zero Enable Set/Reset
 out dx, ax
 mov dx, 03C4h ; 6845 address reg
 mov ax, 0F02h ; Data reg, enable all planes
 out dx, ax

; Exit
 mov sp, bp
 pop bp

 retf
_draw_point ENDP
 END


[LISTING TWO]

 /* Library source file GRAFIX.C */
 /* EGA/VGA graphics subsystem */
 /* Following library routines are external: */
 /* DRAWPT.ASM Feb '89 */
 /* K. Porter, DDJ Graphics Programming Column */
 /* ------------------------------------------ */

 #include <dos.h>
 #include <stdlib.h>
 #include <string.h>
 #include <stdio.h>
 #include "grafix.h"

 #if !defined TRUE
 #define FALSE 0
 #define TRUE !FALSE
 #endif

 /* Variables global to this library */
 int color1, /* foreground color */
 oldmode = 0, /* pre-graphics mode */
 grafixmode = 0, /* default graphics mode */
 ega = FALSE, /* equipment Booleans */
 vga = FALSE,
 colormonitor = FALSE,
 curpos, /* text cursor position */
 textpage; /* active text page */
 unsigned vidmem; /* video buffer segment */
 char far *vidsave; /* video buffer save area */

 /* -------------------------------------------------------- */

 int far init_video (int mode)

 /* Initializes video adapter and defaults for mode */
 /* Sets up pc_textmode() to be called on pgm termination */
 /* Returns TRUE or FALSE indicating success */
 /* This function will be expanded in a later version */
 {
 union REGS r;
 int result = FALSE;

 /* Determine attached adapter and monitor */
 r.h.ah = 0x1A; /* VGA inquiry function */
 r.h.al = 0;
 int86 (0x10, &r, &r); /* ROM BIOS call */
 if (r.h.al == 0x1A)
 switch (r.h.bl) {
 case 4 : ega = TRUE; /* EGA color */
 colormonitor = TRUE;
 break;
 case 5 : ega = TRUE; /* EGA mono */

 break;
 case 7 : ega = TRUE; /* VGA mono */
 vga = TRUE;
 break;
 case 8 : ega = TRUE; /* VGA color */
 vga = TRUE;
 colormonitor = TRUE;
 }
 else { /* No VGA, so check for EGA */
 r.h.ah = 0x12;
 r.x.bx = 0x10;
 int86 (0x10, &r, &r);
 if (r.x.bx != 0x10) { /* if EGA present... */
 ega = TRUE; /* set flag */
 r.h.ah = 0x12;
 r.h.bl = 0x10; /* find out which monitor */
 int86 (0x10, &r, &r);
 if (r.h.bh == 0)
 colormonitor = TRUE; /* EGA color */
 }
 }

 /* Proceed only if EGA or VGA present */
 if (ega vga) {
 set_color1 (15); /* default pixel color */

 r.h.ah = 0x0F; /* get current screen mode */
 int86 (0x10, &r, &r);
 oldmode = r.h.al; /* store it */
 textpage = r.h.bh; /* also active text page */

 if (colormonitor) /* point to video memory */
 vidmem = 0xB800;
 else
 vidmem = 0xB000;
 vidsave = malloc (4096); /* allocate save area */
 movedata /* save text screen contents */
 (vidmem, 0, FP_SEG (vidsave), FP_OFF (vidsave), 4096);

 r.h.ah = 3; /* get text cursor position */
 r.h.bh = textpage;
 int86 (0x10, &r, &r);
 curpos = r.x.dx; /* and save it */

 if ((mode == EGA) && ega) {
 r.h.ah = 0;
 r.h.al = mode; /* set EGA mode */
 int86 (0x10, &r, &r);
 result = TRUE;
 } else
 if ((mode == vga) && vga) {
 r.h.ah = 0;
 r.h.al = mode;
 int86 (0x10, &r, &r);
 result = TRUE;
 }
 }
 if (!result) { /* unable to switch to graphics */
 oldmode = 0; /* so cancel text screen save */

 free (vidsave);
 vidsave = 0;
 } else { /* successful, so... */
 grafixmode = mode; /* save mode */
 atexit (pc_textmode); /* register exit function */
 }
 return result;
 } /* ------------------------------------------------------ */

 void far pc_textmode (void)
 /* SPECIFIC TO MS-DOS */
 /* Restore text mode */
 /* Automatically called on pgm termination */
 {
 union REGS r;

 if (oldmode) { /* if not in text mode now... */
 r.h.ah = 0;
 r.h.al = oldmode; /* restore text mode */
 int86 (0x10, &r, &r);
 movedata /* restore text screen */
 (FP_SEG (vidsave), FP_OFF (vidsave), vidmem, 0, 4096);
 free (vidsave); /* free allocated memory */
 vidsave = 0; /* zero pointer */
 oldmode = 0; /* reset */
 r.h.ah = 2; /* restore old cursor position */
 r.h.bh = textpage;
 r.x.dx = curpos;
 int86 (0x10, &r, &r);
 }
 } /* ------------------------------------------------------ */

 void far set_color1 (int palette_reg)
 /* Select pixel color from palette register */
 {
 color1 = palette_reg;
 } /* ------------------------------------------------------ */



[LISTING THREE]

/* Library source file GRAFIX.C */
/* EGA/VGA graphics subsystem */
/* Following library routines are external: */
/* DRAWPT.ASM Feb '89 */
/* K. Porter, DDJ Graphics Programming Column */
/* ------------------------------------------ */

#include <dos.h>
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
#include "grafix.h"

#if !defined TRUE
#define FALSE 0
#define TRUE !FALSE
#endif


/* Variables global to this library */
int color1, /* foreground color */
 oldmode = 0, /* pre-graphics mode */
 grafixmode = 0, /* default graphics mode */
 ega = FALSE, /* equipment Booleans */
 vga = FALSE,
 colormonitor = FALSE,
 curpos, /* text cursor position */
 textpage; /* active text page */
unsigned vidmem; /* video buffer segment */
char far *vidsave; /* video buffer save area */

/* -------------------------------------------------------- */

int far init_video (int mode)

/* Initializes video adapter and defaults for mode */
/* Sets up pc_textmode() to be called on pgm termination */
/* Returns TRUE or FALSE indicating success */
/* This function will be expanded in a later version */
{
union REGS r;
int result = FALSE;

 /* Determine attached adapter and monitor */
 r.h.ah = 0x1A; /* VGA inquiry function */
 r.h.al = 0;
 int86 (0x10, &r, &r); /* ROM BIOS call */
 if (r.h.al == 0x1A)
 switch (r.h.bl) {
 case 4 : ega = TRUE; /* EGA color */
 colormonitor = TRUE;
 break;
 case 5 : ega = TRUE; /* EGA mono */
 break;
 case 7 : ega = TRUE; /* VGA mono */
 vga = TRUE;
 break;
 case 8 : ega = TRUE; /* VGA color */
 vga = TRUE;
 colormonitor = TRUE;
 }
 else { /* No VGA, so check for EGA */
 r.h.ah = 0x12;
 r.x.bx = 0x10;
 int86 (0x10, &r, &r);
 if (r.x.bx != 0x10) { /* if EGA present... */
 ega = TRUE; /* set flag */
 r.h.ah = 0x12;
 r.h.bl = 0x10; /* find out which monitor */
 int86 (0x10, &r, &r);
 if (r.h.bh == 0)
 colormonitor = TRUE; /* EGA color */
 }
 }

 /* Proceed only if EGA or VGA present */
 if (ega vga) {

 set_color1 (15); /* default pixel color */

 r.h.ah = 0x0F; /* get current screen mode */
 int86 (0x10, &r, &r);
 oldmode = r.h.al; /* store it */
 textpage = r.h.bh; /* also active text page */

 if (colormonitor) /* point to video memory */
 vidmem = 0xB800;
 else
 vidmem = 0xB000;
 vidsave = malloc (4096); /* allocate save area */
 movedata /* save text screen contents */
 (vidmem, 0, FP_SEG (vidsave), FP_OFF (vidsave), 4096);

 r.h.ah = 3; /* get text cursor position */
 r.h.bh = textpage;
 int86 (0x10, &r, &r);
 curpos = r.x.dx; /* and save it */

 if ((mode == EGA) && ega) {
 r.h.ah = 0;
 r.h.al = mode; /* set EGA mode */
 int86 (0x10, &r, &r);
 result = TRUE;
 } else
 if ((mode == vga) && vga) {
 r.h.ah = 0;
 r.h.al = mode;
 int86 (0x10, &r, &r);
 result = TRUE;
 }
 }
 if (!result) { /* unable to switch to graphics */
 oldmode = 0; /* so cancel text screen save */
 free (vidsave);
 vidsave = 0;
 } else { /* successful, so... */
 grafixmode = mode; /* save mode */
 atexit (pc_textmode); /* register exit function */
 }
 return result;
} /* ------------------------------------------------------ */

void far pc_textmode (void)
/* SPECIFIC TO MS-DOS */
/* Restore text mode */
/* Automatically called on pgm termination */
{
union REGS r;

 if (oldmode) { /* if not in text mode now... */
 r.h.ah = 0;
 r.h.al = oldmode; /* restore text mode */
 int86 (0x10, &r, &r);
 movedata /* restore text screen */
 (FP_SEG (vidsave), FP_OFF (vidsave), vidmem, 0, 4096);
 free (vidsave); /* free allocated memory */
 vidsave = 0; /* zero pointer */

 oldmode = 0; /* reset */
 r.h.ah = 2; /* restore old cursor position */
 r.h.bh = textpage;
 r.x.dx = curpos;
 int86 (0x10, &r, &r);
 }
} /* ------------------------------------------------------ */

void far set_color1 (int palette_reg)
/* Select pixel color from palette register */
{
 color1 = palette_reg;
} /* ------------------------------------------------------ */



[LISTING FOUR]

/* STRIPES.C: Demos pixel-writing with GRAFIX library */

#include <conio.h>
#include <stdio.h>
#include <stdlib.h>
#include "grafix.h"

void main ()
{
int x, y, color;

 if (!init_video (EGA)) {
 puts ("No EGA/VGA present in system: Program ended");
 exit (1);
 } else {
 for (y = 95; y < 255; y++) {
 color = y % 16; /* first color this row */
 for (x = 160; x < 480; x++) {
 set_color1 (color); /* set color this pixel */
 draw_point (x, y);
 if (++color == 16) color = 0; /* wrap around palette */
 }
 }
 getch(); /* hold for keypress */
 }
}


















February, 1989
RUN LENGTH ENCODING


RLE is an efficient -- yet simple -- way of reducing storage requirements




Robert Zigon


Robert Zigon is a senior software engineer and can be reached at 4505
Candletree Circle, Apt. 7, Indianapolis, IN 46254.


Run Length Encoding (RLE) is one of several techniques that can be used to
reduce the storage requirements of text files, databases, and digital images.
The algorithm is very simple to implement, it produces output files that, on
the average, require 80 percent of the input file space, and executes quickly.
Because the compression factor is of prime interest, it should be pointed out
that the worst case performance of the algorithm causes the output file to
double its size. My use of the algorithm, however, for compressing and saving
the digitized output of a frame-grabber board has never resulted in this
degeneracy.
The algorithm works as follows: An input buffer is scanned for sequences of
identical characters. When a character transition occurs the repetition count
of the previous character (along with the character itself) is sent to an
output buffer.
This continues until the end of the input buffer is reached. My implementation
currently uses 1 byte for the repetition count. This allows for sequences of
identical characters to be reduced to exactly 2 bytes. The output for the
input buffer looks like this:
 Input Buffer: AAAA BBBBBB CC D EEEEE Output Buffer: 4A 6B 2C 1D
5E
In this example, the 18 characters in the input buffer were reduced to 10
characters in the output buffer (the spaces in the input and output buffers
are there to make it easier to read). Notice that no space savings was gained
by compressing the CC and that the space requirements of the D doubled.
Listing One contains a routine called PACK that can be used to compress an
input buffer. The code is written in the assembly language of the Intel 80X86
family. The routine is designed so that multiple calls will result in having
the output concatenated to the contents of the previous call.
After the information is packed and saved to disk, the day will come when you
will need to unpack it. Unpacking is significantly easier. A repetition count
is read from the packed buffer, and the corresponding character is sent to the
output buffer that many times. The string load (LODSB) mnemonic permits the
efficient reading of the pairs, while the string store (STOSB) and REP prefix
perform the duplication and restoration of the input file. Listing One also
contains the necessary UNPACK routine to accomplish this.
Run Length Encoding is a fast and efficient technique for reducing the space
requirements of an input file. Though more complex algorithms exist with
better compression factors (and worse compression times), its elegance lies in
its simplicity.

_Run Length Encoding_
by Robert Zigon


[LISTING ONE]

;--------------------------------------------------------------------
; RLE.asm Run Length Encoding Routines
; Author : Bob Zigon
;--------------------------------------------------------------------
; -------------------------------------------------------------------
; PACK
; This routine will pack a source buffer into a destination buffer
; by reducing sequences of identical characters down to a 1 byte
; repetition count and a 1 byte character.
; INPUT : DS:SI -- Points to source buffer to pack
; ES:DI -- Points to destination
; DX -- Number of characters in source buffer to pack
; OUTPUT: DI -- Is updated to allow for multiple calls.
; -------------------------------------------------------------------
pack proc near
 push ax
 push bx
 push cx
 lodsb
 mov bl,al
 xor cx,cx ; Counts number of characters packed
 cld ; All moves are forward

p10: lodsb ; Get chara into AL
 inc cx ; Inc chara count
 sub dx,1 ;
 je p30 ; Exit when DX = 0

 cmp cx,255 ; 255 characters in this block yet?
 jne p20 ; If not then jump

 mov [di],cl ; output the length
 inc di
 xor cx,cx
 mov [di],bl ; output the character
 inc di

p20: cmp al,bl ; Has there been a character transition?
 je p10 ; NOPE ... so go back for more

 mov [di],cl ; output the length
 inc di
 xor cx,cx
 mov [di],bl ; output the character
 inc di

 mov bl,al ; Move Latest chara into BL
 jmp p10 ; Loop back for more
;
; We will get here when DX finally goes to zero.
;
p30: mov al,cl ; Output the length
 stosb
 mov al,bl ; Output the character
 stosb

 pop cx
 pop bx
 pop ax
 ret
pack endp

; -------------------------------------------------------------------
; UNPACK
; This routine will unpack a sequence of characters that were
; previously PACKed.
; NOTE : Source Region must be terminated with a NULL.
; INPUT : DS:SI -- Source buffer to unpack
; ES:DI -- Destination buffer to unpack into
; OUTPUT:
; -------------------------------------------------------------------
unpack proc near
 push ax
 push cx
 xor cx,cx ; Make sure CH is zero
 cld ; All moves are forward

unp10: lodsb ; Load into AL a character from DS:[SI]
 cmp al,0 ; Length of zero is end of region
 je unp40 ;
 mov cl,al ; Length goes into CL
 lodsb ; AL has chara to repeat

 rep stosb ; Store AL to [DI] and repeat while CX <> 0
 jmp unp10 ; Loop back forever

unp40: pop cx

 pop ax
 ret
unpack endp



























































February, 1989
STRUCTURED PROGRAMMING


The Return of the Shower Curtain Salesman




Jeff Duntemann KI6RA


It's a crazy business, these magazines. The very first computer magazine I
bought was a Dr. Dobb's Journal, back in my COSMAC days of late '76. Still,
it's got to be one of the very few serious mags I haven't yet written for. So
let me drive a stake into that 12-year record by introducing myself as your
new "Structured Programming" columnist.
Not a few of you will recognize me as that grinning gremlin from the editorial
page of Turbo Technix, Borland's incandescent programmers' magazine that
vividly demonstrated what happens when the tail of a circular queue of dollars
catches up with and seriously overtakes the head. After a heartbreak of that
magnitude, I had thought about laying aside the magic keyboard forever.
However, it was either write or sell shower curtains to discount stores in the
Missouri/Iowa/Minnesota territory --and if you're reading this, well, my
decision should be obvious.
I believe that in starting a project like this it's only fair to be up front
about my biases. With all due respect to the Dr. Dobb's majority, I don't
think all that much of C. Therefore, a significant part of the mission of this
column will be to demonstrate that there's nothing you can do in C that you
can't do in Modula-2 or Pascal, and with less need for Band-Aids and Mad-Dog
Colombian Buzz-Beans. (Tell you about those some day ...)
I strongly favor object-oriented programming, and while OOP is wasted on
mongrels like C++, it's true that there are few OOP options for DOS outside
the C sphere. This is why I'll be talking about Smalltalk and Actor from time
to time. Whether or not there will ever be a NeXT machine for the south side
of $10,000, OOP is hot. Read up on it. Try it if you get the chance.
(Digitalk's $99 Smalltalk/V is an excellent learning vehicle.)
Finally, I've been a Borland employee in the past and still have ties to the
company, so I will politely decline any invitations to review Pascal or
Modula-2 compilers here or elsewhere. While I will be mentioning programming
tools of various sorts as I go, the emphasis will be on awareness rather than
evaluation. No matter what they tell you, a lot of powerful stuff still comes
out of garages, and I'll be pointing out the best of it as I find it.
I recognize that Pascal and Modula aren't the only non-C languages kicking
around. Ada? I don't know. If I see a good one that doesn't cost six percent
of the gross national product, I'll try it out and let you know. Cobol or
Fortran? No chance. And while I don't promise to talk about Basic I don't
promise not to, either.
Your input is solicited. What languages would you like me to discuss? (Is
anybody really using Smalltalk on the PC?) What do you want to know how to do?
Do you care about OS/2? Windows? TSRs? High level? (How to Design a Coherent
Program?) Low level? (How to Load an EGA Font?) Assembler interface? Send me a
laundry list. Also, tips are welcome; I only have two eyes and so many hours
in the day. Send me your opinions and your best mini hacks. What's hot? What's
not? Be my eyes and ears. Bearer of the best tip of each month gets my secret
recipe for Structured Chicken Soup plus a little bag of Mad-Dog Colombian
Buzz-Beans.


Looking For Mr. Goodboard


The problem with keeping video on the back plane (as the PC does and the Mac
does not) is that the Vendor Gorilla (guess who) keeps changing boards on us.
There are now two different text video devices for the PC (monochrome and
color) and four different graphics devices (CGA, EGA, MCGA, and VGA), not
counting the unlamented PCjr. To avoid programming to the least common
denominator (and there isn't any truly common denominator), you have to be
able to figure out which board is on the bus before you go out and try to do
anything adventurous like set a mode or, lord knows, turn the cursor off.
It's easier than it sounds. You don't have to do anything as underhanded as
examining bits in undocumented I/O locations. It's all a question of working
with video-specific BIOS calls.
BIOS support for video exists in three layers, corresponding to the three
generations of IBM video boards. The first generation consists of the original
color graphics adapter and monochrome display adapter. Think of this as the
innermost video BIOS layer, the one in planar ROM. (There is no ROM on those
two first generation video boards.)
The introduction of the EGA carried with it ROM BIOS enhancements on the video
board. These enhancements were attached to the main planar BIOS through a
feature called ROM scan. During power-on self-test (POST), the PC's BIOS scans
memory looking for ROM BIOS extensions identified by a special signature. In
essence, the INT 10H (video) support on the mother-board was replaced by a
software interrupt service routine on the EGA board, simply by repointing the
INT 10H vector in the vector jump table to the routine in the EGA board's ROM.
VGA and MCGA BIOS support was provided in the same way (on-board BIOS
connected to the planar BIOS through ROM scan), but it is a superset of the
EGA's. Wrapped around EGA, BIOS support is a third layer of additional BIOS
video services that don't exist on any board prior to the VGA and MCGA.
The algorithm for detecting adapters comes down to this: First, identify the
generation of video boards by looking for each layer. In other words, check
for VGA/MCGA-specific features. If you can't find that outermost layer, you
know you don't have a VGA or MCGA. Next, look for the second layer, EGA
support. If you can't find that, you know you don't have an EGA. If there's no
EGA, you've defaulted to the first generation video boards.
Once you have the generation, you can use BIOS services to distinguish between
a VGA and an MCGA or between a CGA and an MDA. Because the EGA is the only
board in its generation, there's nothing further to be done once you've
identified the generation as the EGA's.
There is one additional complication. Beyond the first generation of boards,
all IBM-style video boards may be connected to either a color or monochrome
monitor. The type of monitor connected to the board affects which modes may
legally be set, and drastically changes the mechanics of working with certain
modes. (And a minor complication to this complication ... the creaking,
ancient composite video monitors may be either color or monochrome but are
considered color monitors and may be used only with the CGA.)
The differences in semantics between an EGA/VGA/MCGA connected to a color
monitor and an EGA/VGA/MCGA connected to a monochrome monitor are so severe
that for coding purposes I recommend thinking of the two situations as two
separate types of display boards. I consider this supportable since monitors
are changed rarely and (because it's madness to consider swapping boards with
the power on) never during the course of a DOS session.
One way to represent the different types of display devices is through an
enumerated type:
TYPE AdapterType = (None,MDA,CGA,EGAMono,EGAColor, VGAMono,
VGAColor,MCGAMono,MCGAColor);
Note that the EGA, VGA, and MCGA each have two different representations,
depending on which monitor is connected to the installed board. The None
constant exists to capture that plaguey hundredth chance that the host machine
is running its console through a serial link with no video board of any kind
on the backplane.
A video board identifier function should return a value of type AdapterType.
I've implemented that function as QueryAdapterType. The Turbo Pascal 5.0
version is given in Example 1, while the TopSpeed Modula-2 version is given in
Example 2.
Example 1: QUERYDSP.SRC

 FUNCTION QueryAdapterType : AdapterType;

 VAR
 Regs : Registers;
 Code : Byte;

 BEGIN
 Regs.AH := $1A; { Attempt to call VGA Identify Adapter Function }
 Regs.AL := $00; { Must clear AL to 0 ... }
 Intr ($10, Regs);
 IF Regs.AL = $1A THEN { ...so that if $1A comes back in AL... }
 BEGIN { ...we know a PS/2 video BIOS is out there. }
 CASE Regs.BL OF { Code comes back in BL }
 $00 : QueryAdapterType := None;
 $01 : QueryAdapterType := MDA;
 $02 : QueryAdapterType := CGA;
 $04 : QueryAdapterType := EGAColor;
 $05 : QueryAdapterType := EGAMono;
 $07 : QueryAdapterType := VGAMono;

 $08 : QueryAdapterType := VGAColor;
 $0A, $0C : QueryAdapterType := MCGAColor;
 $0B : QueryAdapterType := MCGAMono;
 ELSE QueryAdapterType := CGA
 END { CASE }
 END
 ELSE
 { If it's not PS/2 we have to check for the presence of an EGA BIOS: }
 BEGIN
 Regs.AH := $12; { Select Alternate Function service }
 Regs.BX := $10; { BL=$10 means return EGA information }
 Intr ($10, Regs); { Call BIOS VIDEO }
 IF Regs.BX <> $10 THEN { BX unchanged means EGA is NOT there...}
 BEGIN
 Regs.AH := $12; { Once we know Alt Function exists...}
 Regs.BL := $10; { ...we call it again to see if it's...}
 Intr($10,Regs); { ...EGA color or EGA monochrome.}
 IF (Regs.BH = 0) THEN QueryAdpterType := EGAColor
 ELSE QueryAdapterType := EGAMono
 END
 ELSE { Now we know we have an CGA or MDA; let's see which: }
 BEGIN
 Intr ($11, Regs): {Equipment determination service }
 Code := (Regs.AL AND $30) SHR 4;
 CASE Code of
 1 : QueryAdapterType := CGA;
 2 : QueryAdapterType := CGA;
 3 : QueryAdapterType := MDA
 ELSE QueryAdapterType := None
 END { Case }
 END
 END;
 END;

Example 2 QUERYDSP.JPI

 PROCEDURE QueryAdapterType() : AdapterType;

 VAR
 Regs : Registers;
 Code : SHORTCARD;

 BEGIN
 Regs.AH : = 1AH; (* Attempt to call VGA Identify Adapter Function *)
 Regs.AL : = 0; (* Must clear AL to 0 ... *)
 Intr (Regs,10H);
 IF Regs.AL = 1AH THEN (* ...so that if $1A comes back in AL... *)
 (* ...we know a PS/2 video BIOS is out there. *)
 CASE Regs.BL OF (* Code comes back in BL *)
 0 : RETURN None 
 1 : RETURN MDA; 
 2 : RETURN CGA; 
 4 : RETURN EGAColor; 
 5 : RETURN EGAMono; 
 7 : RETURN VGAMono; 
 8 : RETURN VGAColor; 
 0AN, 0CH : RETURN MCGAColor; 
 0BH : RETURN MCGAMono; 
 ELSE RETURN CGA

 END (* CASE *)
 ELSE
 (* If it's not PS/2 we have to check for the presence of an EGA BIOS: *)
 Regs.AH : = 12H; (* Select Alternate Function service *)
 Regs.BX : = 10H; (* BL=$10 means return EGA information *)
 Intr (Regs, 10H); (* Call BIOS VIDEO *)
 IF Regs.BX <> 10H THEN (* BX unchanged means EGA is NOT there...*)
 Regs.AH : = 12H; (* Once we know Alt Function exists... *)
 Regs.BL : = 10H; (* ...we call it again to see if it's... *)
 Intr (Regs, 10H); (* ...EGA color or EGA monochrome. *)
 IF (Regs.BH = 0) THEN RETURN EGAColor
 ELSE RETURN EGAMono
 END
 ELSE (* Now we know we have an CGA or MDA; let's see which: *)
 Intr(Regs,11H); (* Equipment determination service *)
 Code : = SHORTCARD (BITSET(Regs.AL) * BITSET(4..5)) >> 4;
 CASE Code OF
 1 : RETURN CGA 
 2 : RETURN CGA 
 3 : RETURN MDA
 ELSE RETURN None
 END (* Case *)
 END
 END
 END QueryAdapterTYPE;

Identifying the presence of a PS/2-generation video BIOS hinges on a service
IBM first incorporated into the VGA BIOS: Service 1AH, Identify Adapter.
Service 1AH is quite courteous in that it always copies register AH into
AL.IBM has always used the convention that if you attempt to call a video BIOS
service that does not exist, all registers are left unchanged. Because video
services are selected by placing their number in register AH before calling
INT 10H, if you call service 1AH and find 1AH in register AL, service 1AH (and
hence a PS/2 VGA or MCGA BIOS) exists. A good safety measure is to zero
register AL before calling service 1AH so that there is no chance that a
garbage bit pattern of 1AH will not be passed to INT10H and returned unchanged
in AL.
Now, if service 1AH exists at all, it does all the rest of the work. It
returns a code in BL that specifies which adapter is on the bus. A simple CASE
statement translates the code into a value of type AdapterType and we're done.
If service 1AH is not found, there's no VGA or MCGA on the bus, and we have to
go looking for the EGA layer. The plan is similar: The EGA added a video BIOS
service called alternate function, 12H. Service 12H does not promise to copy
AH into AL, but (as with all video BIOS calls) if service 12H does not exist,
no registers will be changed. Service 12H contains several subfunctions, one
of which is return EGA information, subfunction 10H. Subfunction numbers are
selected by placing them in BL. If video service 12H exists, it will place EGA
information in the various registers. If service 12H does not exist, all
registers will be returned unchanged.
By a (highly) fortunate coincidence, the subfunction number 10H is not valid
EGA information. Therefore, if after a call to service 12H with subfunction
number 10H in BL we still see 10H in BL, then service 12H does not exist, and
the device is one of the first-generation CGA or MDA boards.
If BL changes, there's an EGA on the bus, and we can use some of that returned
EGA information to determine whether the attached monitor is color or
monochrome. BH contains the goods: If BH is found to be 0, a color monitor is
attached; otherwise a monochrome monitor is there, and QueryAdapterType can
return the appropriate value of EGAColor or EGAMono.
This sounds more complicated than it is. The code itself is quite
straightforward; only the reasons for the BIOS's behavior are complex.
If after searching for the outer two layers we turn up nothing, then by
default the attached board is either the CGA or MDA. (Or, as mentioned before,
no adapter is installed at all.) The way to tell the difference is to call
BIOS interrupt 11H, Equipment Determination. A word value encoded with a
summary of installed equipment will be returned in register AX.Bits 4 and 5
specify the attached video adapter. Shifting the adapter bits 4 bits to the
right translates them into a simple numeric code that may be translated to an
AdapterType value with a CASE statement.


The Machine Told Me; Now who Told The Machine?


That's about all there is to detecting standard IBM display adapters. In most
cases it devolves to asking the machine what its got. Now, I can never quite
rid myself of the fear that the Anthropomorphic Fallacy (that is, this habit
of "asking the machine" as though the machine were the little guy behind the
counter at the liquor store) will throw me a curve. It's worth wondering how
the machine knows what it has --and how likely the machine is to be right.
In the case of the EGA, VGA, and MCGA the machine knows because during POST,
ROM scan attached the video board's BIOS chip to the planar BIOS. Because what
you are doing when you are calling INT 10H is calling a routine in a ROM chip
on the video board, you would like to assume that the board knows what it is.
If only IBM made video boards, that would be a good bet. However, I've seen
enough totally blatant BIOS bugs in some of the scruffier Yuk Foo EGA clones
to downgrade that certainty to about 99 percent. Not enough to worry about,
but keep it in the corner of your mind if a client ever reports truly bizarre
behavior on the part of your software.
For the CGA and MDA, what the BIOS does is simply report the state of the
display adapter DIP switches (PC and XT) or CMOS RAM bits (AT). Since human
beings set these switches or bits, you'd think the chances for error here are
greater. Not so --ironically, if a machine tells you through Query-AdapterType
that it contains a CGA or MDA, you can believe it, because POST double checks
the switches and bits. The MDA and CGA are fairly simple creatures whose
differences far outweigh their similarities, and are easy to tell apart by
examining refresh buffers and I/O locations.
There is still the possibility that a weirdly designed CGA/MDA combination
board might fool POST into making the wrong determination, but I've used a few
such boards and POST has pegged them correctly every time.
Keep QueryAdapterType handy. Several routines and utilities in upcoming
columns will require it.


Dare to Hack the Hardware


I guess I was marked forever when I decided to wirewrap my first computer (a
COSMAC Elf, lordy) out of loose parts. Those people who view with terror the
idea of hooking loose wires onto serial or parallel ports remind me of James
Thurber's granny, who worried endlessly about electricity leaking out of empty
light bulb sockets and slithering around the house, crackling at her.
Then again, if you decided you wanted your machine to control the motion of a
telescope, what would you do? Regardless of your answer, before doing
anything, lay hands on Bruce Eckel's new book Computer Interfacing With Pascal
and C. With wit and some superb technical figures, Bruce captures the essence
of making electrons out of bits and vice versa. Among the described devices
are temperature sensors, stepper motor drivers, TRIAC switches, digital/analog
converters (DACs), op-amps, optoisolators, and numerous others. Most of the
I/O is through standard PC serial and parallel ports. Bruce does, however,
relate how to create your own PC bus prototyping board.
Bruce assumes that you all know how to program, so the emphasis is on
practical interface electronics. The driver code is in Turbo C and Turbo
Pascal and is disarmingly simple. Most of the interface devices can be cobbled
together out of Radio Shack parts without a heavy investment in money or
skill. Having done some of this stuff myself, I can tell you that it's easier
than it may seem. With Bruce's book in front of you, it might even be fun.
Highly recommended.


BOO!


It's Halloween, and I just handed our last Reese's Peanut Butter Cup to a
six-foot tall cardboard milkshake with a cowlick. Nothing much more will get
done tonight, I suspect. We'll pick up the subject of PC display adapters next
time. For now, it's time to turn out the lights and go engage in some
devil-try of my own. Perhaps I'll be truly wicked and scrawl GOTO 100 on my
neighbor's garage door ... naw, too nasty.
Anybody know where there's a good outhouse?



Products Mentioned


Computer Interfacing With Pascal and C Bruce Eckel eisys 1009 N. 36th Str.
Seattle, WA 98103 Book $29.95 (postpaid) Listing disk $19.95


_Structured Programming Column_
by Jeff Duntemann

[EXAMPLE 1]

FUNCTION QueryAdapterType : AdapterType;

VAR
 Regs : Registers;
 Code : Byte;

BEGIN
 Regs.AH := $1A; { Attempt to call VGA Identify Adapter Function }
 Regs.AL := $00; { Must clear AL to 0 ... }
 Intr($10,Regs);
 IF Regs.AL = $1A THEN { ...so that if $1A comes back in AL... }
 BEGIN { ...we know a PS/2 video BIOS is out there. }
 CASE Regs.BL OF { Code comes back in BL }
 $00 : QueryAdapterType := None;
 $01 : QueryAdapterType := MDA;
 $02 : QueryAdapterType := CGA;
 $04 : QueryAdapterType := EGAColor;
 $05 : QueryAdapterType := EGAMono;
 $07 : QueryAdapterType := VGAMono;
 $08 : QueryAdapterType := VGAColor;
 $0A,$0C : QueryAdapterType := MCGAColor;
 $0B : QueryAdapterType := MCGAMono;
 ELSE QueryAdapterType := CGA
 END { CASE }
 END
 ELSE
 { If it's not PS/2 we have to check for the presence of an EGA BIOS: }
 BEGIN
 Regs.AH := $12; { Select Alternate Function service }
 Regs.BX := $10; { BL=$10 means return EGA information }
 Intr($10,Regs); { Call BIOS VIDEO }
 IF Regs.BX <> $10 THEN { BX unchanged means EGA is NOT there...}
 BEGIN
 Regs.AH := $12; { Once we know Alt Function exists... }
 Regs.BL := $10; { ...we call it again to see if it's... }
 Intr($10,Regs); { ...EGA color or EGA monochrome. }
 IF (Regs.BH = 0) THEN QueryAdapterType := EGAColor
 ELSE QueryAdapterType := EGAMono
 END
 ELSE { Now we know we have an CGA or MDA; let's see which: }
 BEGIN
 Intr($11,Regs); { Equipment determination service }
 Code := (Regs.AL AND $30) SHR 4;
 CASE Code of
 1 : QueryAdapterType := CGA;
 2 : QueryAdapterType := CGA;

 3 : QueryAdapterType := MDA
 ELSE QueryAdapterType := None
 END { Case }
 END
 END;
END;


[EXAMPLE 2]

PROCEDURE QueryAdapterType() : AdapterType;

VAR
 Regs : Registers;
 Code : SHORTCARD;


BEGIN
 Regs.AH := 1AH; (* Attempt to call VGA Identify Adapter Function *)
 Regs.AL := 0; (* Must clear AL to 0 ... *)
 Intr(Regs,10H);
 IF Regs.AL = 1AH THEN (* ...so that if $1A comes back in AL... *)
 (* ...we know a PS/2 video BIOS is out there. *)
 CASE Regs.BL OF (* Code comes back in BL *)
 0 : RETURN None 
 1 : RETURN MDA; 
 2 : RETURN CGA; 
 4 : RETURN EGAColor; 
 5 : RETURN EGAMono; 
 7 : RETURN VGAMono; 
 8 : RETURN VGAColor; 
 0AH,0CH : RETURN MCGAColor; 
 0BH : RETURN MCGAMono; 
 ELSE RETURN CGA
 END (* CASE *)
 ELSE
 (* If it's not PS/2 we have to check for the presence of an EGA BIOS: *)
 Regs.AH := 12H; (* Select Alternate Function service *)
 Regs.BX := 10H; (* BL=$10 means return EGA information *)
 Intr(Regs,10H); (* Call BIOS VIDEO *)
 IF Regs.BX <> 10H THEN (* BX unchanged means EGA is NOT there...*)
 Regs.AH := 12H; (* Once we know Alt Function exists... *)
 Regs.BL := 10H; (* ...we call it again to see if it's... *)
 Intr(Regs,10H); (* ...EGA color or EGA monochrome. *)
 IF (Regs.BH = 0) THEN RETURN EGAColor
 ELSE RETURN EGAMono
 END
 ELSE (* Now we know we have an CGA or MDA; let's see which: *)
 Intr(Regs,11H); (* Equipment determination service *)
 Code := SHORTCARD(BITSET(Regs.AL) * BITSET{4..5}) >> 4;
 CASE Code OF
 1 : RETURN CGA 
 2 : RETURN CGA 
 3 : RETURN MDA
 ELSE RETURN None
 END (* Case *)
 END
 END
END QueryAdapterType;































































February, 1989
THE FORTH COLUMN


Forth News




Martin Tracy


The German Forth community has moved a step closer to claiming the domain of
real-time programming. According to Klaus Schleisiek-Kern (DELTA-t), a German
real-time congress with a strong Forth component is now being planned.
Echtzeit '90 will take place in Nuremberg, the home of the first German Forth
chip, in the summer of 1990. Sponsored by E-T-A GmbH, the chip was developed
by Fraunhofer-Gesellschaft Erlangen, and will be produced by ELMOS GmbH. Klaus
will talk about it at the 1988 FORML convention.
In other Forth news, Friends of Forth are invited to brush up on their syntax
by participating in Jack Brown's ongoing Forth tutorial on the British
Columbia Forth Board (604-434-5886). It is also being networked to GENIE
(800-638-9636 sign-up) and the East Coast Forth Board (703-442-8695). The
lessons, two of them so far, are well written and have excellent examples and
homework problems. You can leave your solutions on the board for correction
and feedback. The course is based on Tom Zimmer's public-domain F-PC Forth.
The latest version, 2.15, is available on the same bulletin boards, although,
it will take quite a while to download it. Or you might call Offete
Enterprises (415-574-8250) to see if they already have it on a low-cost disk.


ANS X3J14 Meeting Number 5


The ANS Forth Technical Committee held its fifth meeting August 10 - 13 in
Portland, Ore. For the first time, a BASIS document was presented that differs
radically in format from the Forth-83 Standard. The content and wording of
this document, nicknamed "Brave New Basis" (BNB), received quite a bit of
attention at the meeting. Several new terms were defined, such as execution
token, the token that ' ("tick") returns for EXECUTE to use. You can get a
copy of the latest BASIS by sending $5 to Martin Tracy, FORTH Inc., 111 N.
Sepulveda Blvd., Manhattan Beach, CA 90266.
The meeting dealt with over 70 technical proposals, more than double of the
previous meeting. There was a feeling that the BASIS would become the draft
proposal ANS Forth Standard (DPANS) sometime in 1989. This optimism abated
somewhat when Charles Moore, the founder of the Forth language, left the
meeting abruptly after one of his proposals was defeated. I have posted the
draft minutes of the meeting and the current technical proposal. Log on GENIE
as files TCMINS5.ARC and TECHPROS.ARC, respectively.
Here are some of the technical highlights of that meeting:
The " (quote) string literal compiler was added. The sequence "ccc" inside a
colon definition will cause the address and length of the string ccc to be
pushed on the stack, with the length on top.
2@, 2!, 2DUP, 2DROP, 2SWAP, 2OVER, 2R>, and 2>R are now required words.
ASCII and [ASCII] have been added. ASCII used outside of a definition leaves
the ASCII value of the first character of the following word on the stack.
[ASCII] has an equivalent action for use within a definition.
A floating-point extension was added. It includes the functions F+, F-, F*,
F/, F0<, F0=, F<; the memory access words F@ and F!; the (separate) stack
manipulators FDUP, FDROP, FSWAP, FOVER, and FROT; and the defining words
FCONSTANT and FVARIABLE. At present, there is no proposal for floating-point
input or output, nor is there any way to make a floating-point number.
The text file operators READ-LINE and WRITE-CR have been added to the file
extension.
The next ANS X3J14 committee meeting is scheduled for the end of January 1989.
It will be held in Los Angeles, Calif., and is hosted by Ray Duncan of
Laboratory Microsystems. Observers are welcome. Call chair Elizabeth Rather at
FORTH Inc. (213-372-8493) if you are planning to attend.


1988 ASYST International Conference


The first ASYST International Conference took place October 9 - 10 at the
University of Rochester, N.Y. ASYST is one of the better known data
acquisition and analysis packages and features a flexible graphics interface,
disk data library support, and a rich library of mathematical functions, from
statistics to signal processing. And best of all, it's programmable! ASYST is
written in Forth, and much of the language is accessible at the user level.
To create a four-cycle, 256-point sine wave, for example, you would first make
an array of 256 successive integers: 256 REAL RAMP
The elements generated by RAMP range from 1 to 256 and are located somewhere
in a heap. A token representing this array is left on the stack. To adjust the
integers to range from 0 to 255, subtract one: 1 -
The subtraction operator knows to subtract the scalar one from each element in
the array. A token pointing to the resulting array is left on the stack. The
index array is then normalized to range from 0 to a little less than 2 PI:
 256 / 2 * PI *
Finally, take four times the sine of each element:
 RAD 4 * SIN
To duplicate the array and plot is a simple matter:
 DUP Y.AUTO.PLOT
Of course, this could all be compiled into a colon definition:
 :FOURSINE ( - array)
 256 REAL RAMP 1 - 256 /
 2 * PI * 4 * SIN
 DUP Y.AUTO.PLOT;
ASYST supports the GPIB bus and more than two dozen IBM PC data acquisition
and control drop-in boards. I was impressed by their (copyrighted) DAS driver
specification, which describes a logical data acquisition and control board
--timers, triggers, multi-channel A/D, DMA, the whole nine yards. This
specification is not included in the ASYST package but comes with the notice
that "ASYST authorizes any person or entity in possession of this
specification to reproduce and distribute copies of this specification free of
charge, provided that such copies are complete and entire and contain all
copyright notices indicating ASYST's ownership of the copyright as are
contained in the original." Call them at 716-272-0070.
The conference was smoothly executed by veteran Larry Forsely, who also puts
together the Rochester Forth Conference. Here are some of the papers you
missed:
"A General Purpose Stimulus Presentation and Data Acquisition System for the
Cognitive Psychology/Evoked Potential Laboratory," by Dr. George Fein, UC San
Francisco.
"Femtosecond Spectroscopy with ASYST," by Wayne Know, AT&T Bell Labs.
"Cables and Bits (the assessment of lung dysfunction)," by Dr. Daniel Rayburn,
Walter Reed Institute of Research.
There were more than 40 papers in all, an impressive technical program for a
first conference. I will let you know when the proceedings are available.


A Simple Mandelbrot



Hats off this month to Marc Hawley for his elegantly simple Mandlebrot
program. The plotting routines, in their entirety, can be found on screen 2
(see Listing One , page 149). They use DOS calls to plot pixels on an IBM
high-resolution (black and white) graphics screen, but you could alter them
for fancier pictures. The routine itself is on screen 3. Several different
views are developed by simply copying the screen and editing the parameters.
The familiar inkblot appears from one to ten minutes after execution. The
MANDLZEN.ARC file itself was downloaded from the GENIE FORTH bulletin board.
[<LISTING .+>\]/,"[$1]")

[LISTING ONE]


\ MANDLZEN 9-16-88 M.HAWLEY
\
 This file contains a screen for graphics words for an IBM-PC
BIOS compatible. The word GRAPH puts the computer in high res.
graphics mode. PIXEL-ON takes two numbers off the stack and uses
them as X,Y coordinates to plot one pixel. PIXEL-OFF does the
same but turns the pixel off. All other words are FORTH-83
written in L&P F83.
 The load screen loads screen 2 , the graphics words
and screen 3 which draws a small sketch of the Mandelbrot Set
in under 8 minutes.
 The other screens draw bigger versions and closeups. Full
blown versions take up to 4 hours to draw on my 8086 based PC.
LOAD THE SCREEN YOU WANT TO RUN. You can't load them all at once
because they all use the same variables and constants.

\ M.HAWLEY
2 3 THRU














\ M.HAWLEY
HEX
 CODE VIDEO AX POP DX POP CX POP
 BP PUSH SI PUSH 10 INT SI POP BP POP
 NEXT END-CODE
: TEXT 0 0 2 VIDEO ;
: GRAPH 0 0 6 VIDEO ;
CODE PLOT AL POP DX POP CX POP
0C # AH MOV
 BP PUSH SI PUSH 10 INT SI POP BP POP
 NEXT END-CODE
CODE PIXEL-ON 0C01 # AX MOV DX POP CX POP
 BP PUSH SI PUSH 10 INT SI POP BP POP NEXT END-CODE
CODE PIXEL-OFF 0C00 # AX MOV DX POP CX POP
 BP PUSH SI PUSH 10 INT SI POP BP POP NEXT END-CODE
DECIMAL ;
\ MANDLZEN One screen Mandlbrot sketch M.HAWLEY

 VARIABLE CX VARIABLE CY VARIABLE X
-8192 CONSTANT CYBASE CYBASE CY ! -12000 CX !

 160 CONSTANT CXSTEP 400 CONSTANT CYSTEP
 : MANDLZEN GRAPH 269 X ! 370 270 DO 1 X +!
 125 75 DO 0 0 X @ I PIXEL-ON
 30 0 DO 2DUP DUP 8192 */ SWAP DUP 8192 */
 2DUP + 0< IF X @ J PIXEL-OFF 2DROP LEAVE THEN
 SWAP - CX @ + -ROT 4096 */ CY @ + LOOP 2DROP
 CYSTEP CY +!
 LOOP CYBASE CY !
 CXSTEP CX +! LOOP ." Mandelbrot by Marc Hawley "
." POB 716, Mt. Vernon, IN 47620 " ;


\ MANDL1 Full screen Mandelbrot portrait M.HAWLEY

 VARIABLE CX VARIABLE CY VARIABLE X
-8192 CONSTANT CYBASE CYBASE CY ! -12000 CX !
 33 CONSTANT CXSTEP 82 CONSTANT CYSTEP
 : MANDL1 GRAPH 0 X ! 500 0 DO 1 X +!
 200 0 DO 0 0 X @ I PIXEL-ON
 50 0 DO 2DUP DUP 8192 */ SWAP DUP 8192 */
 2DUP + 0< IF X @ J PIXEL-OFF 2DROP LEAVE THEN
 SWAP - CX @ + -ROT 4096 */ CY @ + LOOP 2DROP
 CYSTEP CY +!
 LOOP CYBASE CY !
 CXSTEP CX +! LOOP ." Mandelbrot by Marc Hawley "
." POB 716, Mt. Vernon, IN 47620 " ;


\ MANDL1 full screen M.HAWLEY

-200 CONSTANT CYBASE CYBASE CY ! 2000 CX !
 1 CONSTANT CXSTEP 2 CONSTANT CYSTEP 0 X !
 : MANDL1 GRAPH 500 0 DO 1 X +!
 200 0 DO 0 0 X @ I PIXEL-ON
 80 0 DO 2DUP DUP 8192 */ SWAP DUP 8192 */
 2DUP + 0< IF X @ J PIXEL-OFF 2DROP LEAVE THEN
 SWAP - CX @ + -ROT 4096 */ CY @ + LOOP 2DROP
 CYSTEP CY +!
 LOOP CYBASE CY !
 CXSTEP CX +! LOOP ;




\ MANDL3 full screen M.HAWLEY

-8192 CONSTANT CYBASE CYBASE CY ! -1024 CX !
 2 CONSTANT CXSTEP 5 CONSTANT CYSTEP 0 X !
 : MANDL3 GRAPH 500 0 DO 1 X +!
 200 0 DO 0 0 X @ I PIXEL-ON
 80 0 DO 2DUP DUP 8192 */ SWAP DUP 8192 */
 2DUP + 0< IF X @ J PIXEL-OFF 2DROP LEAVE THEN
 SWAP - CX @ + -ROT 4096 */ CY @ + LOOP 2DROP
 CYSTEP CY +!
 LOOP CYBASE CY !
 CXSTEP CX +! LOOP ;





\ MANDL4 full screen M.HAWLEY

-8192 CONSTANT CYBASE CYBASE CY ! -1024 CX !
 2 CONSTANT CXSTEP 5 CONSTANT CYSTEP 0 X !
 : MANDL4 GRAPH 500 0 DO 1 X +!
 200 0 DO 0 0 X @ I PIXEL-ON
 30 0 DO 2DUP DUP 8192 */ SWAP DUP 8192 */
 2DUP + 0< IF X @ J PIXEL-OFF 2DROP LEAVE THEN
 SWAP - CX @ + -ROT 4096 */ CY @ + LOOP 2DROP
 CYSTEP CY +!
 LOOP CYBASE CY !
 CXSTEP CX +! LOOP ;




\ MANDL5 full screen M.HAWLEY

-8192 CONSTANT CYBASE CYBASE CY ! -824 CX !
 1 CONSTANT CXSTEP 2 CONSTANT CYSTEP 0 X !
 : MANDL5 GRAPH 500 0 DO 1 X +!
 200 0 DO 0 0 X @ I PIXEL-ON
 90 0 DO 2DUP DUP 8192 */ SWAP DUP 8192 */
 2DUP + 0< IF X @ J PIXEL-OFF 2DROP LEAVE THEN
 SWAP - CX @ + -ROT 4096 */ CY @ + LOOP 2DROP
 CYSTEP CY +!
 LOOP CYBASE CY !
 CXSTEP CX +! LOOP ;




\ MANDL6 full screen M.HAWLEY

-7250 CONSTANT CYBASE CYBASE CY ! -424 CX !
 1 CONSTANT CXSTEP 2 CONSTANT CYSTEP 0 X !
 : MANDL6 GRAPH 500 0 DO 1 X +!
 200 0 DO 0 0 X @ I PIXEL-ON
 20 0 DO 2DUP DUP 8192 */ SWAP DUP 8192 */
 2DUP + 0< IF X @ J PIXEL-OFF 2DROP LEAVE THEN
 SWAP - CX @ + -ROT 4096 */ CY @ + LOOP 2DROP
 CYSTEP CY +!
 LOOP CYBASE CY !
 CXSTEP CX +! LOOP ;




\ MANDELZEN DOCUMENTATION 9-16-88 M.HAWLEY
VARIABLES CX and CY are the X and Y coordinates of the
starting point for the graph. Everything is scaled up by a
factor of 8192, so -1 is expressed as -8192, .02 is expressed
as 164 and so on. The variable X is a kludge I had to use to
access the outermost index of three nested loops while in the
innermost. My version of L&P F83 has I and J for the first
two indexes but no I' ( drat ).
 To explore different parts of the Mandelbrot Set, change
the starting point by editing CX and CY.

CONSTANTS CYBASE is the base value for CY. After the program
scans through the range of values being tested along the Y-axis
CY is reset to CYBASE for the next vertical scan.
 CXSTEP and CYSTEP are the increments by which CX and CY
are changed each time. To explore a large part of the Set or
all of it, use large STEPs.
\ DOCs cont. M.HAWLEY
To zoom in and magnify a small part of the Set, use small
STEPs. For the best proportion, at least on my screen, CYSTEP
should be 2 or 2 1/2 times as big as CXSTEP.

PIXEL-ON and PIXEL-OFF are specific to IBM-PC BIOS ROM
compatibles. Given two numbers on the stack, they turn on
or off the pixel at that (X,Y) location on the screen.

You will notice that the closeup screens magnify the view
of screen 3 ( MANDLZEN ) up to 160 diameters just like a
160 power telescope looking at a celestial object. Yet, the
program is entirely in 16 bit scaled integer math ---
no floating point. I originally thought that THIS program
would HAVE to be in floating point.

\ DOCs M.HAWLEY
 I first wrote it in floating point. It ran 6 TIMES SLOWER
than present version. For my fellow intermediate programmers
take notice. I now finally begin to understand why FORTH
programmers scoff at floating point.

THE ALGORITHM Two numbers are kept on the stack representing
the real and imaginary components of a complex number. This
number is repeatedly put through the transformation:
 Z --> Z*Z + C
The complex number is sqared and added to another complex
number, C , which is the point being tested to determine whether
it is in the Set. C is represented as CX , the real part, and
CY the imaginary part. The sum is again squared and added to C.
This is repeated untill the test is satisfied ( Z stays small )
or failed ( Z gets too big ).
\ DOCs M.HAWLEY
 The outer loops simply scan through the x and y coordinates
of the screen or some part of the screen and update the
variables.
 The inner loop is the repetative test which finds Z*Z + C.
The odd thing about squaring an imaginary number is that the
result is always negative. ( A positive OR negative real number
squared is always positive, of course. ) So what we need on the
stack for the real part of Z*Z + C is ZR*ZR-ZI*ZI + CX
and for the imaginary part 2*ZR*ZI + CY.
 Why ? Well :
 Z*Z = (ZR + ZI)*(ZR + ZI)
 = ZR*ZR + 2*ZR*ZI + ZI*ZI but ZI*ZI is negative so...
 = ZR*ZR + 2*ZR*ZI - ZI*ZI
 The real part is ZR*ZR - ZI*ZI , imaginary 2*ZR*ZI
 Add the C : ZR*ZR-ZI*ZI+CX , 2*ZR*ZI+CY qed.
\ DOCs M.HAWLEY
The inner loop first puts 0 0 on the stack for starters.
 WORD : STACK
 --> 0 0
 X @ I --> 0 0 X I

 PIXEL-ON --> 0 0 plot the point being checked
 --> ZR ZI the values being represented
 2DUP --> ZR ZI ZR ZI
 DUP --> ZR ZI ZR ZI ZI
8192 */ --> ZR ZI ZR ZI*ZI scaled down by 8192
 SWAP --> ZR ZI ZI*ZI ZR
 DUP --> ZR ZI ZI*ZI ZR ZR
8192 */ --> ZR ZI ZI*ZI ZR*ZR scaled
 2DUP --> ZR ZI ZI*ZI ZR*ZR ZI*ZI ZR*ZR
 + --> ZR ZI ZI*ZI ZR*ZR ZI*ZI+ZR*ZR
 this is the square of the magnitude
\ DOCs M.HAWLEY
The magnitude of a complex number is its distance from the
origin, the 0,0 point. The X and Y coordinates are two sides
of a triangle and the hypotenuse is the magnitude. Using the
Pythagorean Theorem, Mag*Mag = ZR*ZR + ZI*ZI
If the magnitude of Z is over 2 the point will continue to
grow and is not in the Set. But the square of the magnitude
is easier to find, so check if it is over 4. Here is a trick.
If you scale up by a factor of 1000 you will be checking for
a magnitude of 4000. Fine. For more detail you might try a
scale of 2000 and test for 8000. Still fine. Try a scale of
8000. Trouble. We are then testing whether a number is greater
than 32000, but if it is over 32768 it will show up as a
NEGATIVE number and pass the test it should fail. SO AHA !
Use a scale of 8192 and test for 32768 which is similar to
testing for a negative number. Mag. should not be negative.
\ DOCs M.HAWLEY
 --> ZR ZI ZI*ZI ZR*ZR ZI*ZI+ZR*ZR
 --> ZR ZI ZI*ZI ZR*ZR magnitude.squared
 0< --> ZR ZI ZI*ZI ZR*ZR TF less than zero ?
 IF --> ZR ZI ZI*ZI ZR*ZR
X @ J PIXEL-OFF if test failed, erase pixel
 2DROP if failed, drop two numbers
LEAVE THEN ZR ZI failed, exit test
 --> ZR ZI ZI*ZI ZR*ZR if test passed
 SWAP --> ZR ZI ZR*ZR ZI*ZI
 - --> ZR ZI ZR*ZR-ZI*ZI real part of Z*Z
 CX @ --> ZR ZI ZR*ZR-ZI*ZI CX real part of C
 + --> ZR ZI ZRnew
 -ROT --> ZRnew ZR ZI
4096 */ --> ZRnew 2*ZR*ZI scaled ( 2/8192 = 1/4096)

\ DOCs M.HAWLEY
 --> ZRnew 2*ZR*ZI
 CY @ --> ZRnew 2*ZR*ZI CY imag. part of C
 + --> ZRnew 2*ZR*ZI+CY this is new ZI
 --> ZRnew ZInew
 LOOP --> test again ...
 2DROP --> clear stack when done
CTSTEP CY +! --> increment CY , the Y axis variable
 LOOP --> cycle through the Y axis
CYBASE CY ! --> reset CY for the next CX cycle
CXSTEP CX +! --> increment CX, the X axis variable
 LOOP --> cycle through the X axis
 ; --> That's all, folks !
I would like to hear your comments and improvements.
Marc Hawley POB 716, Mt. Vernon, IN 47620
 EXPLORE AND ENJOY THE MANDELBROT SET































































February, 1989
OF INTEREST







Tools


REGULUS-386 Builder, a software development kit for 80386-based PC AT and
Compaq compatible systems, has been released by Alcyon Corp. This product
contains the REGULUS-386 operating system as well as a C compiler, assembler,
linker, debugger, and support upgrades for 90 days.
A Unix-compatible, real-time operating system, REGULUS-386 supports system
calls and kernel features. Real-time operating features include prioritized
tasks, context switching, intertask communications, contiguous files, and
direct access to interrupts.
REGULUS-386 Builder sells for $1,500. Perpetual upgrades and continued support
are available at additional cost. Reader Service No. 20. Alcyon Corp. 6888
Nancy Ridge Dr. San Diego, CA 92121-2232 619-587-1155
Abraxas Software has announced an OS/2 CASE tool called PCYACC/2, Version 2.0,
a program generator capable of generating C source code for building
assemblers, compilers, browsers, page description language, language
translators, syntax directed editors, and query languages for OS/2.
PCYACC/2 is designed to generate ANSI C source code optimized for the
Microsoft and Lattice OS/2 compilers. The generated source code can then be
compiled to generate the final product. Runtime library and example sources
are provided to be used as application skeletons for new products.
Demo application sources include a desktop calculator, an infix to postfix
translator, and an implementation of the PIC(ture) language. Also included are
a code execution engine to display the graphics on PS/2 screens and a C++ to C
translator. Grammar for YACC, C++, ISO Pascal, ANSI C, SQL, Apple Hypertalk,
Smalltalk-80, Prolog, and dBase III+ and IV is included.
Other features include the ability to generate code for large grammars on
PS/2s, syntax trees generated at runtime by target products for debugging, and
error recovery support for target products.
PCYACC/2 sells for $395; multiple copy discounts and site licenses are
available. Reader Service No. 21. Abraxas Software Inc. 7033 SW Macadam Ave.
Portland, OR 97219 503-244-5253
Version 1.3 of Pro-C has been released by Vestronix Inc. Pro-C is a source
code applications generator for MS-DOS, QNX, Xenix, and Unix. Windows, one of
the product's features, gives developers the ability to create applications
with moving windows, multiple windows, dynamically sized windows, scrolling
regions, and subscreens.
Other features include dbase III interface, multiple records, help functions,
and color support. Pro-C, which does not require a runtime environment, sells
for $495. Reader Service No. 23. Vestronix Allen Square 180 King St. S, Ste.
230 Waterloo, Ontario, Canada N2J IP8 519-745-2700
Island Systems has announced the availability of Turbo Meta-Menu, a software
utilities package that uses the MetaWindow graphics driver (from MetaGraphics
Software Corp.) for Turbo Pascal 4 and 5 to produce a user interface to
graphics application programs.
Turbo Meta-Menu allows software developers to create horizontal and vertical
pull-down menus, pop-up messages, button menus, and more. All structures
autosave the underlying graphics image and provide a mouse-drag feature.
The Turbo Pascal 4 and 5 version of Turbo Meta-Menu sells for $149; the
library source is an additional $75. The base package comes with a reference
manual, demo program with source, six sample programs with source, and two
additional utility programs. The package includes menu and message utilities
as well as a cursor icon editor and an automatic menu-making program. Reader
Service No. 22. Island Systems 7 Mountain Rd. Burlington, MA 01803
617-273-0421
Trio Systems has begun shipping a new release of its C-Index Database Toolkit
developed for Turbo C. The C-Index library is now available in a version ready
to run with Borland's Turbo C. A professional development tool, the C-Index
Database Toolkit supports single user, multiuser, and network applications
with file-management facilities.
The C-index Database Toolkit features B+ tree indexing, variable-length
records, direct and sequential access, and multiple record formats per file.
The C-Index Database Toolkit for Turbo C uses application program interface
(API), which allows the library to be implemented using nine subroutine calls.
This product sells for $99. Reader Service No. 24. Trio Systems 2210 Wilshire
Blvd., Ste. 289 Santa Monica, CA 90403 213-394-0796
Solution Systems' C-Worthy Interface Library includes features such as screens
and windows, form interface library, menus, error handling, DOS interface
library, and system and context sensitive help.
With C-Worthy, creating a window consists of three function calls: Define the
window's characteristics, select it as the current window, and display it. Up
to 50 windows can be active at one time. The windows feature also includes
screens and color palettes.
The optional form interface library is designed to help users display data
input forms and to gather and validate user input. C-Worthy's menus include
pop-up, Lotus style, pull-up, and pull-down (MS Windows style). C-Worthy's
system error library is a collection of over 120 error handling routines that
report errors returned from C-Worthy functions.
The DOS interface library has routines to manage data files, dates, times,
disk drives, subdirectories, memory, and other DOS operations. The help
feature creates program help screens interactively using C-Worthy's full
screen text editor. Each help message can have multiple pages of text.
C-Worthy supports Microsoft C 4.0 and 5.0, Quick C, Turbo C 1.0 and 1.5, and
Lattice C 3.2. System requirements include hard-disk media with 256K RAM,
MS-DOS 2.0 or later, and IBM PC or compatible, TI Professional, NEC APC III,
or Vicor 9000. G-Worthy Interface Library sells alone for $195, with Form
Interface Library for $295, and with Forms and Library source for $495. Reader
Service No. 25. Solution Systems 541 Main St., Ste. 410 S Weymouth, MA 02190
617-337-6963 800-821-2492


Debuggers


Wendin Inc. has released VM-DEBUG, a debugging tool for IBM PCs, XTs, ATs, and
compatibles. VM-DEBUG, which stands for the virtual machine debugger, is an
interpreter whose language is 8088 machine code extended with the real-mode
instructions of an 80286.
According to Wendin, the VM-DEBUG interpreter can stop the execution of a
program at any point, examine or alter memory or register, examine the
program, and determine where the program has been. In addition, the product
can trace DOS, or the ROMs, and set breakpoints within ROM. VM-DEBUG sells for
$99. Reader Service No. 26. Wendin Inc. Box 3888 Spokane, WA 99220-3888
509-624-8088
Language Processors Inc. (LPI) has announced the availability of CodeWatch, an
interactive source-level debugger to be used with LPI's Basic, C, Cobol,
Fortran, Pascal, and PL/I programming languages operating in the Unix and
Xenix environments.
Code Watch uses the language of the source code (not machine language) and
allows commands to be entered in an abbreviated form. Features include action
lists, macros, stepping, tracing, and program execution controlled by the
developer so that execution may be suspended via breakpoints or resumed at a
developer-specified point.
Available on most Motorola 680X0 series systems and Intel's 80386-based
systems under Unix and Xenix, CodeWatch is priced from $495 to $2,495,
depending on the number of ports and the processor. Reader Service No. 27.
Language Processors Inc. 959 Concord St. Framingham, MA 01701-4613
508-626-0006


Prototyping


Rapid Prototyping System (RPS) is Genesis Data Systems' new presentation
design program incorporating several tools for project prototyping,
interactive tutorials, product demonstrations, program design, and front-end
software management.
RPS features a prototyping module that enables the user to manipulate and join
text, graphics, and music into presentations through the use of animation,
transition, branch, subroutine, and numerical/text variable commands.
RPS provides a screen design module that uses the IBM extended ASCII set and
255 color combinations for the creation of screens. Graphics or text screens
can also be captured from other programs with an RPS memory-resident module.
Both user-designed and captured screens can be used in the prototyping module.
RPS contains a music module that uses standard musical notation for the
creation of new tunes or the duplication of familiar ones. Tunes can be saved
to a file and accessed by the prototyping module. Tunes may be played in the
background, either as part of a presentation or directly from DOS.
RPS runs on IBM PCs, PS/2s, and compatibles equipped with a floppy or hard
drive and a color, monochrome, or TTL monitor. The $249.95 program requires
256K and DOS 2.0 or later, and it is not copy protected. Reader Service No.
28. Genesis Data Systems 8415 Washington Place NE, Ste. A Albuquerque, NM
87113 505-821-9425 800-777-1437


Ada


Intel Corp. has demonstrated its validated Ada-386 compilation package with
the Intel 32-bit, real-time kernel iRMK and has announced plans to introduce
an Ada-960 cross-compiler.

The Ada-386 cross compilation package runs under VAX/VMS and features a set of
language tools, which a designer uses to go from code development to
optimizing, downloading, and debugging.
The package enables designers to generate 32-bit 386 microprocessor code
supported by an 80387 numerics coprocessor. The "pragma interface" permits
calls to high-level languages, including Intel's ASM-386, PL/M-386, and C-386.
Optional download and debug paths are available via either a ROM-resident
debug monitor or by using Intel's ICE-386 for transparent, real-time
emulation.
Intel anticipates similar capabilities in the future for designers using the
new Ada-960 development tools. The validated Ada-960 cross-compiler will be
available in the first half of this year.
The Ada-386 cross compilation package is available now. Pricing is dependent
upon the class of VAX host: The price for MicroVAX version is $36,000. The
price for the Ada-386 cross compilation package includes Intel's 90-day
support offering plus 12 months of maintenance. Reader Service No. 29. Intel
Corp. 3065 Bowers Ave. Santa Clara, CA 95052-8065 503-696-2233 800-548-4725
Texas Instruments and Tartan Laboratories have announced an agreement to
jointly develop an Ada compiler for a digital signal processor (DSP) chip.
Target processor for the compiler is TI's SMJ320C30, a third-generation DSP
being developed for military applications requiring advanced signal processing
capabilities, such as radar, sonar, image processing, missile guidance and
tracking, and communications. The companies will work together to define the
capabilities of the compiler. Completion of the compiler is expected late this
year.
The compiler, to be hosted on a DEC-VAX/VMS system, will implement Ada as
defined in ANSI/MIL-STD-1815A-1983 and will target the SMJ320C30 instruction
set architecture.
Still in development, the 32-bit SMJ320C30 is a third-generation device in
TI's family of DSP chips for military applications. The SMJ320C30, built in
CMOS technology, executes single-cycle instructions in 60 nanoseconds and can
perform more than 33 million floating-point operations per second (MFLOPs).
The chip can perform complex algorithms in real time. Reader Service No. 30.
Texas Instruments Inc. Semiconductor Group (SC-871) P.O. Box 809066 Dallas, TX
75380-9077 800-232-3200, ext. 700


Object Oriented Programming


Digitalk Inc. began shipping Smalltalk/V Mac, the Macintosh version of its
object-oriented programming system for PCs. This product consists of a
development language and an interactive development environment. Small pieces
of Smalltalk code can be created, highlighted with the mouse, and then
immediately tested.
Applications developed in Smalltalk/V or Smalltalk/V 286 (for PCs and
compatibles) can be ported to Smalltalk/V Mac, and the application "senses"
the environment. The resulting application, without being modified, will
display standard Macintosh windows, as well as standard zoom, close, and grow
boxes.
Smalltalk/V Mac provides access to the Macintosh toolbox and MultiFinder
compatibility, including background processing. It also provides
multi-processing within Smalltalk/V Mac applications, such as background
sorting. The environment includes a pushbutton debugger. Smalltalk/V,
including a tutorial/user guide and example files, sells for $199.95. Reader
Service No. 31. Digitalk Inc. 9841 Airport Blvd. Los Angeles, CA 90045
213-645-1082


Miscellaneous


SofTools Inc. has released CASE:W, a computer-aided software engineering tool
that generates the windows portion of applications. It includes a programming
environment to generate pretested window code.
The package features a front-end prototyper that provides a way to describe
the application program's windows and controls. Using an inference engine,
CASE:W then evaluates the prototype specification, applies the stored
programming knowledge, and generates the window-based application.
CASE:W produces commented code and the operating controls for windows
applications; it also supports windows controls, such as menu bars, pop-up
menus, and dialogue boxes. CASE:W generates only C code, but programmers may
use application routines in languages that can receive a C language call (such
as assembler, Pascal, Fortran, Basic) using Microsoft's mixed language
conventions.
The product also interfaces with a variety of text editors and has a facility
that regenerates programmer-added codes into future versions of a program.
Additionally, the CASE:W programming environment encompasses the Microsoft
Software Development Kit components (icon editor, font editor, and dialogue
editor) and 12 other configurable tools, such as a debugger, Spy, Heapwalker,
and Shaker.
CASE:W can be used on a 286-based machine or a 386 with at least 2 Mbytes of
main memory. The package also requires the Microsoft Software Development Kit,
the C Compiler, make utility and linker, and a DOS-or Windows compatible text
editor. The company recommends the use of Microsoft's Code-View debugger.
CASE:W sells for $1,495. Reader Service No. 32. SofTools Inc. 1 Dunwoody Park,
Ste. 130 Atlanta, GA 30338 404-399-6236





































February, 1989
SWAINE'S FLAMES


Time, Inc.







This month's Rhealstone project is in the DDJ tradition. This magazine began
as a project to promulgate, and to allow programmers to refine a Basic
interpreter. The very subject matter of the Rhealstone, though, points to the
reason there isn't more of this community design work in these pages: A
magazine does not live in real time.
I got a call from Lee Felsenstein last week (that is, right around
Thanksgiving, real time). Lee has a new project in mind, a non-Apple hardware
platform for stackware: HackerCard, if you will. The project implies some
interesting programming challenges, not to mention the legal hurdles, that
would be fun to hash out in DDJs pages. But given the time factor, the
hashing-out will likely be done elsewhere, like on line.
We in magazine publishing call this communication barrier "lead time," and the
effect is exactly as though we were radioing from somewhere beyond Pluto (the
weeklies from Titan). I mentioned mail order electro-magnate Drew Kaplan in a
column I wrote in October, but when Tyler was chatting (real time) with Drew a
month later at Comdex, that column was still navigating the asteroid belt.
To give you an outside-time perspective, then: As I write this, it's 15
shopping days 'till Christmas, and the airwaves are filled with reminiscences
of the thousand days of the New Frontier. As you read this, it's somewhere
around Martin Luther King's birthday, approaching the thousandth hour of the
Bush Fringe. Either way, a good time to be outside time.


Time Travel Makes Me Tense


Sometimes we in magazine publishing will have to deal with this time problem
or we'll be relegated to rehashing the past. As I write this, both the 35th
anniversary issue of Playboy and the 20th anniversary issue of The Whole Earth
Review are remembering Jack Kerouak.
Skipping a beat, bimonthlies wobble in a different orbit altogether: I already
have the (recommended) Jan./Feb. '89 issue of Micro Cornucopia, which reminds
me that Dave Thompson, (although his magazine sometimes seems to lie outside
the plane of the ecliptic,) is a serious and skilled professional.
As is Nick Herbert. You've probably noticed that DDJ doesn't cover quantum
physics very thoroughly, and I doubt that that's going to change soon. Let me
then recommend a good book, and a very good writer. Nick Herbert's Quantum
Reality is more readable than most science books, and it deals with some
remarkably difficult concepts, including Bell's Theorem.
Bell's Theorem permits a kind of instantaneous connection-at-a-distance that
almost suggests that we could re-orient the arrow of time. Nick Herbert's
second book is out now, it's called Faster Than Light: Superluminal Loopholes
In Physics. Uh-oh, Toto. Herbert is exploring the possibility of superluminal
communication at his National Science Foundation in the Santa Cruz mountains.
Maybe faster-than-light communication can make magazine publishing timely.


Over the Entrepreneurial Edge


A year ago in this space I made fun of the merchandising of Peter Norton,
accusing him of aspirations to the title of Mr. Cultural Icon of 1988. Since
then, Peter's been Dewar's Profiled and Inc. Five Hundreded. I'd like to take
this opportunity to apologize to Peter for my snide remarks and to offer
readers of DDJ a 50 percent discount on the Mike Swaine coffee mug when
purchased with the Mike Swaine tee shirt.
That December Peter Norton Commemorative Issue of Inc. magazine, by the way,
was an intriguing study in self-criticism. (But not like John Dvorak's
still-asteroidal March 1989 MacUser column on MacUser columnists, which I read
in manuscript in November.)
To get on the Inc. 500 list, your company has to grow real fast. Getting on
the Inc. 500 list, suggest the Inc. editors in that selfsame Inc. 500 list
issue, may not be a healthy goal for a small company. Making the list, they
hint, may even be a sign of instability. Fascinating. Usually we in publishing
apologize for our errors after we make them, if at all. Bundling the apology
with the error is an innovative way to beat the lead time.
That same issue also featured the Cary Lu Lynching. Cary had claimed that the
personal computer industry is undergoing enormous and rapid change, that no
major software company is planning to introduce a new MS-DOS product after
1990, and that users will soon find themselves in a "microcomputer mess."
Inc. readers didn't care for Cary's negative thinking; and frankly, I suspect
that it's just this sort of naysaying that's kept Cary from getting a Dewar's
Profile all these years.


























March, 1989
March, 1989
EDITORIAL


Windows, Window Dressing, and Ethics




Jonathan Erickson


It hasn't been that long since the only windowing system widely used by PCs
was that on the Macintosh. Today there are more windowing systems than you can
shake a light pen at, and you can bet that there'll be a lot more before we're
through with them. To give you an idea of what I mean, consider that a few
months ago the Open Software Foundation (OSF, the group formed to counter AT&T
and Sun's attempt at Unix standardization) asked software developers to submit
windowing systems for the user interface of the OSF's upcoming Unix
implementation. When 39 windowing systems were submitted for consideration,
even the OSF folks were a little surprised.
With this backdrop, this month we're examining a few of those windowing
systems, focusing on Presentation Manager, Microsoft Windows, and X Windows.
And because of the growing importance of windowing systems, we'll continue
this examination over the coming months by looking at interfaces (the "Rooms"
interface from Xerox PARC and more on X Windows and MS Windows) and windowing
tools.
There's no question that a consistent user interface makes life easier for
users if application development rules are set and, more importantly, if
developers adhere to those rules. That's one thing I like about the Macintosh.
Because Apple laid down some pretty clear interface rules early on, most Mac
applications look more or less the same and, when you come face-to-face with a
new program, you can get started quickly, relative to most PC applications
anyway.
It remains to be seen how or if windowing systems will make life easier for
programmers. What you can expect to see, however, is the continued emergence
of new cross-platform development tools like the XVT Toolkit (reviewed by
Margaret Johnson in the issue) and Glockenspiel's CommonView (to be promoted
by Microsoft). These tools let you write common code, while the toolkit takes
care of system specific necessities at compile time--the idea being that your
application can run under a variety of windowing systems (PM, Windows,
Macintosh, X Windows, or whatever) without any extra coding on your part.
Incidentally, as Kee Hinckley points out in this issue, the decision reached
by the OSF wasn't for a single windowing system, but instead for an
amalgamation consisting of Microsoft's Presentation Manager, HP's 3-D
technology (as present in NewWave), and DEC's X Windows Toolkit. This is
particularly interesting since, with Presentation Manager, it will be possible
for users to eventually see a consistent graphical interface across dissimilar
hardware platforms. With Microsoft reportedly developing a non-OSF Unix
version of PM, the day when diverse systems appear alike comes a little
closer.
Every now and then a little ethical tune-up seems to be in order. Granted,
Mike Swaine has for years been a self-appointed watchdog of journalistic
ethics, pointing out foibles whenever they come to his attention and, a few
years ago, Phil Lemmons gave a discourse on computer journalism and ethics in
an editorial in Byte. But it seems time for another reminder.
What brought this to mind was a message that recently crossed my desk whereby
a prominent public relations firm said it was discontinuing its policy of
paying members of the computer press a $1,000 finder fee for putting the PR
firm in contact with companies that eventually become clients. Now there are
many fine folks in the PR business; they have a tough job, and the good ones
work hard. (I know because I talk to some of them just about every day). And,
for that matter, the press has an equally tough job. (At least that's what I
keep telling my boss.) Even though the two professions must endure a symbiotic
relationship, it is important to remember that a PR agency's allegiance is to
the client while a journalist's responsibility is to you, the reader. Working
together is one thing, but money changing hands goes beyond the bounds of
decency. Shame on the those who offered the money, but more shame on those who
took it. The only good thing about the whole muddle is that it has ended.
To be fair, I asked myself if my reaction was sour grapes because I didn't
find out about this opportunity until after it ended. Maybe so. Perhaps if I'd
received one of those checks just before Christmas, I'd be more
understanding...but no, it's wrong and no amount of justification can make it
right.
If you'll remember, last month we changed what the outside of the magazine
looked like when we introduced our new DDJlogo. This month we've extended that
redesign into the interior. Redesigning a magazine isn't an endeavor that is
undertaken simply for the sake of change. There are practical reasons too, the
most important being that the redesign has enabled us to get more text on the
same size page without shrinking the size of the type. This in turn means more
articles and more program listings for you.







































March, 1989
LETTERS







Forth Source for Floating Point


Dear DDJ,
First, I like DDJ. I think it's the best magazine around (together with Byte).
I fully agree with Mr. Price ("Letters," August 1988); I think standardization
is useful only in case the language has been designed by a committee. I doubt
if it's smart to standardize a language some 15 years after its introduction!
Although I program mostly in Basic or C (yes, that is possible on a little CBM
64, praise the Lord!), I'm playing around a little with Forth. But I've got a
problem, and I think your readers could help me.
I'm looking for Forth source (FIG-FORTH) for floating point mathematics and
for a decompiler for Forth words (or just hints on how to do it so I can
program them myself.) Also, could someone tell me how it's possible that
certain C functions, such as PRINTFO, can deal with a variable number of
parameters?
Mark van Atten Krabbestraat
11 3052 NR Rotterdam
The Netherlands


Will Simulator Support SubLOGIC Scenery Disks?


Dear DDJ,
We have been swamped with phone calls from owners of Microsoft Flight
Simulator, Version 3, concerned that the product will not support SubLOGIC
Scenery Disks.
The following information is crucial for loading and operating Scenery Disks
with Flight Simulator and applies to Europe Scenery Disk as well as Scenery
Disks 1 - 7, Japan and San Francisco.
Although Flight Simulator's manual does not include any reference to Scenery
Disk support outside a mention on page 75, a readme.doc file on the disk
itself does explain the procedure for converting a Scenery Disk to a scenery
file and loading the file to a hard disk.
You can load a Scenery Disk in either of two ways: as a copy-protected disk
from drive A or as a scenery file loaded into the same hard disk subdirectory
as Flight Simulator. To load the scenery disk from drive A, you first need to
load Flight Simulator from the hard drive or a floppy drive (preferably on
drive B, but you can operate both from drive A in a pinch).
When you're positioned for take-off, press <Shift-E> to bring up the scenery
library menu. Choose option 1 (floppy disk in drive A) or the number
associated with the scenery file you wish to use. Then invoke the NAV/COM menu
and choose selection A to set your position withi the scenery area.
This should have Flight Simulator soaring through the scenery in no time. Of
course, if further assistance is required, the SubLOGIC customer support
department can be reached toll free outside Illinois at 800-637-4983 or
217-359-8482 within the state.
Thanks for helping to get the word out.
Deb Israel
SubLOGIC Corp.
Champaign, Ill.


Sold on Stony Brook


Dear DDJ,
I certainly enjoyed Kent Porter's Modula-2 compiler review in the September
1988 issue ("Examining Room"). My company uses the Stony Brook Modula-2
compiler to develop accounting applications that run under Microsoft Windows.
We are always on the lookout for a new and better compiler, so we read your
review with a great deal of eagerness. However, none of the compilers except
Stony Brook comes close to meeting our needs. This was such a surprise to me
that I thought your readers might want to know why.
1. Stony Brook is the only Modula-2 compiler that can be used to develop
well-behaved Microsoft Windows applications.
2. Stony Brook was the first compiler (anywhere?) to be able to compile all
types of OS/2 code.
3. The memory models offered by Stony Brook are far superior to the choices
offered by the other compilers.
4. The interlanguage support offered by Stony Brook is superior to that of
other models. Others don't do an adequate job because of limited memory
models, calling conventions, and naming conventions.
5. Stony Brook's fast compile times become more important when compiling large
commercial grade programs with lots of modules.
It's true that the other tools are somewhat crude, but the editor is certainly
customizable--you get the source code!
Stony Brook will be coming out with (if they haven't already) an integrated
environment that will include a debugger. It should compete very well with
TopSpeed's environment and debugger. Thanks for giving this fledgling language
space in your magazine.
Seth K. Pratt
Accounting By Design
Berkeley, Calif.


Marvin's Turn



The following is Marvin Hymowech's response to the letter of James R. Van
Zandt of Nashua, New Hampshire ("Letters," December 1988) regarding Marvin's
August 1988 article "Find That Function."
Thanks for the corrections. Here are some other fixes communicated to me by
Lee Meador of Richardson, Texas:
1. Using the macros isspace(), isalnum(), etc., requires that you first check
that isascii() is true. Therefore:
(a) In get_fn_name() in bldfuncs.c, replace:
while( isspace(*name_prt)) /* skip trailing white space*/ name_ptr--;
with:
while( isascii(*name_ptr) && isspace(*name_ptr)) name_ptr-;
Also, replace:
while (isalnum(*name_ptr) II name _ptr=='') && name_ptr>=line) name_ptr-;
with:
while( isascii(*name_ptr)) && (isalnum(*name_ptr) II name_ptr =='') &&
name_ptr>=line) name_ptr-;
(b) Similarly, in get_names_one_file() in bldfuncs.c, replace:
/*skip white space */ while ((c = filter_data(fp_source)!= EOF && isspace(c));
with:
/* skip white space*/ while ((c = filter_data(fp_source) != EOF && isascii(c)
&&
isspace(c));
2. Once you are in a quoted string, you have to call fgetc( ) for the quoted
characters, rather than calling filter_cmt( ), to allow for possible comment
delimiters within quoted strings. Therefore, in filter_quotes( ) in
bldfuncs.c, replace:
 switch ( c2 = filter_cmt(fp_source))
with:
 switch ( c2 = fgetc(fp_source))
Similarly, replace:
 case '\\':/* beginning of an escape sequence */ filter_cmt(fp_source); /* so
eat next
char */
with:
 case'\\':/* beginning of an escape sequence */
fgetc(fp_source); /* so eat next
char */
3. If your C compiler does not allow nested comments, then in filter_cmt() in
bldfuncs.c, replace:
 cmt_level++;/* descend one comment level */
with:
 cmt_level = 1;/* disregard nested comments */


The Squeaky Wheel...


Dear DDJ,
I am writing to you regarding the lack of industry-standard software upgrade
policies. Here is an example that frustrated me recently: I purchased
Micrografx Draw, and, within a week, the company released a replacement
product called Micrografx Draw Plus. The upgrade cost will be $149.95. This is
an outrageous price; other companies, such as Microsoft, offer upgrade
software at a more reasonable cost.
I talked to Micrografx representatives about their policies and suggested that
they offer a lower upgrade plan for those who have recently purchased their
product. For their part, they insisted that this is a new product and not just
an upgrade and that their policy will not change. I questioned them on how
long Micrografx Draw will be sold, and they said that it is replaced by Draw
Plus. I do not believe software producers should be able to change the name of
a product just to escape their obligation to customers to make upgrades
available at a reasonable cost.
I see several other companies with equally poor upgrade policies. Let's get
the software producers to realize that this is an important issue that needs
to be changed.
Chris A. Friend
Friend Dialogues Inc.
Shelby, North Carolina
Micrografx representative Kenneth Mecca responds: We appreciate Dr. Dobb's
Journal and Mr. Friend for pointing out this problem; we had just begun to
notice an increase in negative customer feedback concerning this upgrade.
Based on these events, we reviewed our upgrade policy for Micrografx Draw Plus
and made a change to reflect Mr. Friend's suggestions, as well as those from
our other customers. Effective immediately, our new upgrade policy for
Micrografx Draw Plus calls for free upgrades to recent purchasers of a
previous version (30 days or for any customer that purchases an obsolete
version from a dealer) and a $99 upgrade price for current users (longer than
30 days).
In addition, customers that have already purchased the $149.95 upgrade can get
a cash refund for the difference in price or a free Micrografx ClipArt library
(worth $79) of their choice.
It has always been our intent to make upgrading as painless as possible, and
we certainly support any effort that will promote standardization of upgrade
practices.


Errata


"Real-Time Modeling with MS-DOS" by David Bowling, (February issue): In Figure
1, page 27, the first two time lines are reversed. The value given for "zeta",
page 32, column 3, is incorrect. The value of the proof is 0.1 --it should be
0.01. The equations for "x" on page 34 are reversed. The equation in column 2
should be interchanged with the equation in column 3.








March, 1989
A PRESENTATION MANAGER APPLICATION TEMPLATE


Here's a PM application template that can be used as the starting point for
your PM programs




Herbert Schildt


Herb Schildt is the author of more than two dozen computer books, with topics
ranging from C to Modula-2. This article is an adaption from his book OS/2
Programming: An Introduction. He can be reached at RR #1, Box 130, Mahmomet,
IL 61853.


Beginning with version 1.1, OS/2 has included the Presentation Manager (PM) as
the default user interface. The Presentation Manager provides the user with a
windowed, graphical interface in which much of the functionality of the system
is readily displayed on the screen, thus making the operation of the computer
more intuitive than that of the traditional command line interface. As you
will see, however, the ease of end-user operation has a price: the extra time
and effort it takes to create a PM-compatible program.
This article explains the general operation of a Presentation Manager
application program and develops a Presentation Manager application template
in C. Programs compatible with the Presentation Manager share a common
structure. The elements shared by these PM-compatible programs are examined
here. The PM application template shown at the end of this article can be used
as the starting point for your own programs.
I chose C for this article, because C is the de facto language for OS/2 and
Presentation Manager. There are two main reasons for this. First, C was the
only language available when OS/2 was released. Second, the OS/2 API services
strongly resemble the C standard library functions and appear to be optimized
for it. The code in this article was compiled with Microsoft C, Version 5.1,
using the Microsoft OS/2 Software Developer's Toolkit version of OS/2 and the
Presentation Manager. With minor changes, however, the code should be able to
be compiled using any OS/2-compatible C compiler.


What Is the Presentation Manager?


Technically, the Presentation Manager is a user-interface shell that runs on
top of the OS/2 operating system base. From a programming point of view,
however, the Presentation Manager virtually appears to be the operating
system. This is because the Presentation Manager provides several hundred
Application Program Interface (API) services, which, to a large extent,
replace those provided by the basic OS/2 system. From the programmers's point
of view, the Presentation Manager is one giant toolbox of interrelated
services that allows the creation of application programs that share a common
interface. For many PM-compatible programs, the original OS/2 API services are
irrelevant.
The goal of the Presentation Manager is to enable an end user who has basic
familiarity with the system to sit down and run virtually any application
without prior training. So in theory, if you can run one Presentation Manager
program, you can run them all. In actuality, of course, most useful programs
will still require some sort of end-user instruction, but at least this
instruction can be restricted to what the program does, not how the user must
interact with it.
Not every program that runs under Presentation Manager will necessarily
present the user with a PM-style interface. As the programmer, you can
override the basic Presentation Manager philosophy; but if you do, you had
better have a good reason, because otherwise the users of your programs will
be disturbed. If you are writing application programs for OS/2, they should
conform to the general PM application interface philosophy if they are to be
successful in the marketplace.
One more important point: Because the Presentation Manager must have complete
control of the screen, a PM-compatible program will not be able to use any of
C's standard console input or output functions. This means, for example, that
your PM-compatible programs should not call get( ) or printf( ). In fact, one
reason there are many PM API services is that a large amount of the C standard
library had to be rewritten when window support was introduced.
Let's look at a few of the more important features of the Presentation
Manager.


The DeskTop Model


With few exceptions, the point of a window-based user interface is to provide
the equivalent of a desktop on the screen. On a desktop you may find several
pieces of paper, one on top of another, often with fragments of different
pages visible beneath the top page. The equivalent of the desktop in the
Presentation Manager is the screen. The equivalents of pieces of paper are
windows on the screen. On a desk you can move pieces of paper about, maybe
switching which piece of paper is on top or how much of another is exposed to
view. Presentation Manager allows the same type of operations on its windows.
By selecting a window you can make it current, which means putting it on top
of all other windows. You can enlarge or shrink a window, or move it around on
the screen. The Presentation Manager lets you control the surface of the
screen the way you control the surface of your desk.


The Mouse


Unlike DOS and the original version of OS/2, the Presentation Manager allows
the use of the mouse for almost all control, selection, and drawing
operations. Of course, to say that it allows the use of the mouse is an
understatement. The fact is that the PM interface was designed for mouse
input: It allows the use of the keyboard! Although it is possible for an
application program to ignore the mouse, it does so only in violation of a
basic PM design philosophy principle.
In general, to activate a feature, move the mouse pointer to that feature and
double-click the left mouse button. A double-click is achieved by pressing the
button twice in rapid succession. Presentation Manager allows you to drag
objects around by moving the mouse pointer to the object, pressing and holding
the left button, and moving the mouse pointer and object to a new location.


Icons and Graphic Images


The Presentation Manager allows (but does not require) the use of icons and
bit-mapped graphic images as a means of conveying information to the user. The
theory behind the use of icons and graphic images is found in the old adage "A
picture is worth a thousand words."
An icon, in OS/2 terminology, is a small symbol that is used to represent some
function or program that can be activated by moving the mouse to the icon and
double-clicking on it. A graphic image is generally used to simply convey
information quickly to the user.


Menus and Dialog Boxes


In addition to standard windows, the Presentation Manager also provides
special-purpose windows. The most common of these are the menu and dialog
boxes. Briefly, a menu is a special window that contains only a menu from
which the user makes a selection. Instead of having to provide the menu
selection functions in your program, however, you simply create a standard
menu window using PM services.
A dialog box is essentially a special window that allows more complex
interaction with the application than that allowed by a menu. For example,
your application might use a dialog box to input a filename. With few
exceptions, nonmenu input is accomplished in the Presentation Manager via a
dialog box.


General Operation of a PM Application



There is one important point that you must fix in your mind: A PM application
program's flow is fundamentally different from a "normal" application. You
will need to abandon any preconceived notions of how information moves in and
out of your program as well as what constitutes a program's main loop. Before
we look at any concrete PM services or examples, let's look at the theory
behind all PM-compatible programs.


PM Application Theory


Programs that are compatible with the Presentation Manager share a common
skeleton. In its most straightforward implementation, when the program begins,
it performs the following functions, in the order shown.
Initializes the Presentation Manager relative to the program
Establishes a message queue
Registers a special function, referred to as the window function, which
receives input from the Presentation Manager;
Creates a window of the registered class; and
Executes a loop, which reads messages from the queue and dispatches these
messages to the window function, which takes appropriate action.
The window function (sometimes called wind-proc or windowproc) is a special
function that is called only by the Presentation Manager, not by your program.
It receives, in its parameters, a message from the message queue, established
in the second step. It then takes different actions based upon the value of
each message. (We will look at messages in a moment.)
In a very real way, a PM application program is functionally similar to an
interrupt-driven program. At any point in time, the program may receive a
message from the Presentation Manager to which it must respond. For example,
your program might be told to refresh its screen or that the user has made a
menu selection. The point is that a PM-compatible program is event driven. If
you have never written interrupt-driven programs, then programming for the
Presentation Manager will require that you adjust the way you think about your
programs and their execution.
When a PM application ends, it must perform these three steps:
1. Destroy the window.
2. Destroy the message queue.
3. Terminate the window environment relative to the application.


The Message Loop


Except for creating and destroying the windows required by your program,
generally the only other thing that the main() function does is receive and
dispatch messages. To accomplish this it uses a loop that looks something like
this:
while (program is still running) { get a message; send the message to the
proper window; }
Essentially, the Presentation Manager communicates with your program by
putting messages into its message queue. Your program then extracts a message
from the queue and dispatches it to the proper window by calling another PM
service. This process continues until the program is terminated. For the most
part, messages are the only way in which your program receives input.
(Remember, a PM program cannot, for example, call scanf() to read input from
the keyboard.) Although the form of a message varies somewhat, depending upon
what type of message it is, all messages are integers. Now that you know some
of the theory behind the Presentation Manager and its windows, let's look at
some specifics.


Obtaining an Anchor Block Using WinInitialize


One of the first things that you will want your PM application to do is to
obtain an anchor block handle by calling WinInitialize, whose prototype is
shown here.
void far * WinInitialize (unsigned short handle)
Here, handle must be NULL. Notice that the function returns a void far
pointer, which points to the region of memory used by the Presentation Manager
to hold various bits of information about the window environment relative to
the application program. This region of memory is called the anchor block, and
the pointer to it is called the anchor block handle. If the system cannot be
initialized, a NULL is returned. The anchor block handle is required as a
parameter by many PM services.
Unlike the core API services, which generally return 0 for success, many of
the PM services return 0 (NULL) on failure.
(The Microsoft OS/2 Software Developer's Kit has defined a great many type
names. For example, they have defined USHORT as an unsigned short integer. I
do not use the Microsoft-type names in this article, however, for two reasons.
First, when learning about a new environment, it is important to know exactly
what type of data you are dealing with. Although the type names defined by
Microsoft are convenient, they disguise the nature of the actual data. Second,
and more important, all the type names defined by Microsoft are copyrighted by
Microsoft. Hence, other OS/2C compilers may not be able to provide them.)


Creating a Message Queue


After initializing the window system, all PM applications must create a
message queue using WinCreateMsgQueue, which has this prototype:
 void far *WinCreateMsgQueue(void far *anchor_block, unsigned short size);
The anchor_block is the handle obtained using WinInitialize. The size of the
queue is determined by the value of size, or, if size is NULL, then the system
default is used. Generally, the system default queue size is acceptable.
Each element in the message queue is contained in a structure, called QMSG by
Microsoft, defined like this:
 struct {
 void far *hwnd; /* handle of the recipient window */
 unsigned short msg; /* the message */
 void far *mp1; /* additional message info */
 void far *mp2; /* additional message info */
 unsigned long time; /* time message was generated */

 POINTL ptl; /* position of mouse pointer */

 }QMSG;
The POINTL structure is defined like this:
 struct {

 long x;
 long y;
 } POINTL;
WinCreateMsgQueue returns a handle to the message queue or NULL if the request
fails.


Registering a Window Class


Before you can actually create a window, you must register its class using
WinRegisterClass, whose prototype is shown here:
 unsigned short WinRegisterClass (void far *anchor_block,
 char far *classname,
 (pascal far *window_func)(),
 unsigned long style,
 unsigned short storage_bytes)
Here, anchor_block is a pointer to the anchor block. The string pointed to by
classname is the name of the window class being registered. This may be any
name of your own choosing. The address of the window function must be passed
as the third parameter. The style of the window is specified by style.
Finally, the number of bytes of additional storage beyond that needed by the
window is specified by storage_bytes. Your program may use this extra storage
for its own purposes. For the example in this article, this field will be 0.
The sort of window being registered is described by the value of style. The
only style that we will be using in this article has the value 4L and is
defined as CS_SIZEREDRAW in the PMWIN.H header file provided by Microsoft.
Using this style causes the Presentation Manager to inform your program each
time the window is resized. The WinRegisterClass service returns nonzero if
successful and NULL on failure.


Creating a Window


Once you have initialized the window system relative to your application,
created a message queue, and registered the class, it is time to create a
window.
All Presentation Manager windows begin with a frame, which is essentially a
box. Onto this frame are added a number of optional, but desirable, additions.
In OS/2 these additional features are actually windows in their own right,
which are attached to the frame. It is easier, however, to think of them as
options to the frame. Let's look at these options now.
There are two options that are virtually essential to all windows. The first
is the border. The border is important because it allows the user to move or
resize the window using the mouse. The second is the system menu. The system
menu is a standard menu that, minimally, allows the user to perform the
following operations: restore the window to its original size, move the
window, resize the window, minimize or maximize the window, and close the
window. Although the border allows a more convenient method of moving or
resizing the window, these operations can also be activated from the system
menu. When a window is minimized, it is shown in its iconic form and is moved
to the icon region of the screen. Your program can specify what the iconic
form of a window will look like or simply let the system decide. When a window
is maximized, it takes over the entire screen. Closing a window removes it
from the screen, and if this is the program's top-level window, it terminates
the program.
Most of the time you will also want to add three other features to your
windows: maximize icons, minimize icons, and a title that identifies the
window. Although it is possible to maximize and minimize the window using the
system menu, it is quicker if the maximize and minimize icons are available,
because the user can activate them by clicking on them with the mouse. When
the screen holds several windows, a title is almost necessary in order to
remind the user which window is which.
Finally, if applicable to your program, you will want to add vertical and
horizontal scroll bars to the window. By clicking on the appropriate points on
these scroll bars, the user causes the contents of the window to scroll up,
down, left, or right. The region enclosed by the frame and used by your
application program is called the client area.
The organization of a standard PM window is shown in Figure . (Remember, not
all options will necessarily be found on all windows.) Each PM-compatible
program creates at least one main window. A main window is at the topmost
level. The main window is the one that the user associates with the program.
Closing the main window terminates the program.
There are two general categories of windows: parents and children. It is
possible to create a window inside of another window. In this case the newly
created window is a child of the main window and is enclosed by the parent. A
Child window can, in turn, create a child of its own, and so on up to the
limits imposed by the size of the screen.
Each window is associated with a class. There are several built-in classes,
such as menus, frames, scroll bars, and the like. Windows that you create,
however, will need to be given class names, and these classes must be
registered with the Presentation Manager. All windows define the lower left
corner as location 0,0. The maximum X and Y dimensions are dynamically defined
as the window changes size and shape. The maximum, however, is determined by
the resolution of the screen.


Using WinCreateStdWindow


The easiest way to create a window is to use WinCreateStdWindow Presentation
Manager Service. Its prototype is shown here:
 void far *WinCreateStdWindow(void far *parent_handle,
 unsigned long style,
 void far *frame_data;
 char far *classname,
 char far *title,
 unsigned long client_style,
 unsigned module,
 unsigned short resource,
 void far **client_handle);
The parent_handle must be the handle of the parent window. When a program
begins execution, its parent is the screen, which has 1 for its handle.
Microsoft defines this value by the macro HWND_DESKTOP. The value of style
determines how the window will first appear. Its most common values are shown
in Figure 2, along with the macro names given them by Microsoft.
Figure 2: The most common values for the WinCreateStdWindow-style parameter.

 Macro Name Value Meaning

---------------------------------------------------------------------------------------------

 WS_VISIBLE 0x80000000L make window visible
 WS_MINIMIZED 0x01000000L minimize window
 WS_MAXIMIZED 0x00800000L maximize window

The value pointed to by frame_data sets various flags that detemine how the
window will behave. This value can be any combination of the values shown in
Figure 3, along with the macro names given them by Microsoft.
Figure 3: The values for the frame_data parameter to point to.


 Macro Name Value Meaning

---------------------------------------------------------------------------

 FCF_TITLEBAR 0x00000001L include title bar
 FCF_SYSMENU 0x00000002L include system menu
 FCF_MENU 0x00000004L include user menu
 FCF_SIZEBORDER 0x00000008L include sizing
 FCF_MINBUTTON 0x00000010L include minimize icon
 FCF_MAXBUTTON 0x00000020L include maximize icon
 FCF_MINMAX 0x00000030L include both min and max icons
 FCF_VERTSCROLL 0x00000040L include vertical scroll bar
 FCF_HORZSCROLL 0x00000080L include horizontal scroll bar

The classname parameter points to the string that identifies the class. This
should be the same string that was used in the call to WinRegisterClass. The
string pointed to by title will be used as the title of the window for
identification purposes. For most purposes the client_style parameter should
be 0L, indicating that the client window should be of the same style as the
window class.
The resource and module parameters are used to identify a resource module. For
the examples in this chapter, no resource modules are needed, so these
parameters should be NULL and 0, respectively. A handle to the client window
is put in the handle pointed to by client_handle. The WinCreateStdWindow
service returns a handle to the frame if successful and NULL on failure.


WinGetMsg and WinDispatchMsg


To process messages, your program will require the use of WinGetMsg and
WinDispatchMsg. The WinGetMsg() prototype is shown here:
 unsigned short WinGetMsg(void far *anchor_block,
 QMSG far *message,
 void far *window,
 unsigned short first,
 unsigned short last)
The message retrieved from the queue is put in the queue structure pointed to
by message. If window is not NULL, then it causes WinGetMsg to retrieve
messages directed to only the specified window. Most of the time your
application will want to receive all messages. In this case window should be
NULL. All messages are integers. The first and last parameters determine the
range of messages that will be accepted by defining the end points of that
range. If you wish to receive all messages, then first and last should both be
zero.
The return value of WinGetMsg is important. It returns true unless a
termination message is received, in which case it returns false. This is the
way your program will know it is being terminated.
In most situations, once a message has been received it is simply dispatched
to the correct window without further processing by your program within the
message loop. The service that sends messages along their way is
WinDispatchMsg, whose prototype is shown here:
 void far *WinDispatchMsg(void far *anchor_block, QMSG far *message)
By calling this function the message will automatically be routed to the
proper window function. WinDispatchMsg returns the value returned by the
window function.


Program Termination


Before your program terminates, it must do three things: Close any active
windows, close the message queue, and deactivate the window system interface
created by the WinInitialize service. To accomplish these things the
Presentation Manager provides the services WinDestroyWindow,
WinDestroyMsgQueue, and WinTerminate. Their prototypes are shown here:
 unsigned long WinDestroyWindow(void far *handle_window);
 unsigned long WinDestroyMsgQueue(void far *handle_msgQ);
 unsigned long WinTerminate(void far *anchor-block);
Here, handle_window is the handle of the window to be closed. The handle_msgQ
is the handle to the message queue to be destroyed. And the window system is
disconnected by calling WinTerminate with the anchor block handle.


The Window Function


As mentioned earlier, all programs that are compatible with the Presentation
Manager must pass to the PM the address of the window function that will
receive messages. This function must be declared as shown here:
 void far * pascal far window_func(void far *handle,
 unsigned short message,
 void far *param1,
 void far *param2);
The window function receives the PM messages in its parameters. In essence,
the PM sends your program a message by calling the window function. The value
of handle is the handle of the window receiving the message. The message
itself is contained in the integer message. Finally, some messages require
further information, which is put into the param1 and param2 parameters.
The Presentation Manager can generate several different types of messages.
Some of the more common ones are shown in Figure 4, along with the macro names
assigned to them by Microsoft.
Figure 4: Some common messages.

 Macro name Value Meaning

---------------------------------------------------------------


 WM_BUTTON1DOWN 0x0071 button 1 down
 WM_BUTTON1UP 0x0072 botton 1 up
 WM_BUTTON1DBLCLK 0x0073 double click on button 1
 WM_BUTTON2DOWN 0x0074 button 2 down
 WM_BUTTON2UP 0x0075 button 2 up
 WM_BUTTON2DBLCLK 0x0076 double click on button 2
 WM_BUTTON3DOWN 0x0077 button 3 down
 WM_BUTTON3UP 0x0078 button 3 up
 WM_BUTTON3DBLCLK 0x0079 double click on button 3
 WM_CHAR 0x007A keystroke occured
 WM_CREATE 0x0001 window has been created
 WM_DESTROY 0x0002 window is being destroyed
 WM_ERASEBACKGROUND 0x004F OK to erase background request
 WM_HSCROLL 0x0032 horizontal scroll
 WM_MOVE 0x0006 window is being moved
 WM_MOUSEMOVE 0x0070 mouse has moved
 WM_PAINT 0x0023 window display needs to be refreshed
 WM_SHOW 0x0005 window is shown or removed from the screen
 WM_SIZE 0x0007 window is being resized
 WM_VSCROLL 0x0031 vertical scroll
 WM QUIT 0x002A window being terminated

The window function does not need to explicitly process all the messages that
it receives. In fact, it is common for an application to process only a few
types of messages. What happens, then, to the rest of the messages received by
the window function? They are passed back to the PM for default processing
using the WinDefWindowProc service. Its prototype is shown here:
 void far *WinDefWindowProc(void far *handle,
 unsigned short message,
 void far *param1,
 void far *param2)
As you probably guessed, the WinDefWindowProc simply passes back to the
Presentation Manager the parameters with which it was called.


Putting Together the Pieces


Now that you have seen services needed to initialize and run a simple windowed
application template, it is time to put together the pieces. The program shown
in Listing One (page 81) creates a window that includes a system menu, a
title, a sizing border, and scroll bars. You can move the window around the
screen, minimize or maximize it, and change its shape. For the moment, don't
concern yourself with the window function, window_func( ); it will be
explained shortly.
The program can be compiled using Microsoft C, Version 5.1, using the OS/2
Software Developer's Kit. You may need to make some small changes if you are
using a different compiler. Notice that the program defines INCL_WIN. This
definition is needed to cause the prototypes and definitions for the window
system to be loaded if you are using the Microsoft C compiler. Before you try
to compile this program, read the next two sections.


The Definition File


A definition file is used to specify various options that affect your program.
All definition files end with the .DEF extension. If you have been working
with OS/2 for a while, then definition files are probably no stranger to you.
(A detailed discussion of definition files is beyond the scope of this
article, but the interested reader will find coverage of this topic in my book
OS/2 Programming: An Introduction (Berkeley, Calif.: Osborne/McGraw-Hill,
1988). But unlike many non-PM-compatible programs for which a definition file
is optional, you must define a definition file in the link line for any
PM-compatible program you write. Aside from other reasons, the overriding
reason for this is that you will need to specify more stack space for the PM
application than it will receive by default. The Presentation Manager template
in this article is allocated 4096 bytes, but real world applications may need
more or less space. Also, it is a good idea to specify a heap size. Again, I
allocated 4096 bytes for this purpose, but your programs may need a different
amount of space. You must also include an EXPORTS statement in the file that
specifies the name of the window function. The definition file for the
template is shown in Listing Two (on page 81).


Compiling PM Programs


You will need to specify some different compiler options when compiling a
Presentation Manager program than you did for a standard program. You can use
this batch file if you are using the Microsoft C compiler:
 cl -c -G2sw %1.c
 link %l,,, os2, %1.def;
The -G2sw option tells the compiler to use a 32-bit address for all code and
data references, to turn off stack checking, to assume that the value of the
DS register is different from the one in SS, and to generate 80286
instructions. Because the Presentation Manager requires, minimally, an 80286
processor, there is no harm in generating 80286 instructions. (It is possible
that you will have to use a different set of options even if you are using
Microsoft C, because of future changes to the compiler or the PM environment.)


Understanding How the Template Works


The operation of the main( ) function is straightforward. It initializes the
link between the Presentation Manager and the program, registers a new window
class, creates a window, and executes its message loop. As messages are
received they are dispatched to the window_func( ) by calling WinDispatchMsg.
The message loop terminates when the WM_QUIT is received. This message is
generated by choosing the close option in the window's system menu.
In a PM application, the most important single function is the window
function. It receives the messages sent by the PM and takes appropriate
action. The template shows entries in the switch statement for only the most
common of the several messages that can be generated by the PM. (Keep in mind
that any message that your program does not wish to process must be passed
back to the PM via the WinDefWindowProc service.) Let's look at the meaning of
some of these messages.
When a window is created, the WM_CREATE message is sent to the window
function. This allows your program to initialize values or to perform other
startup operations. As you know, the Presentation Manager allows the user to
move and resize windows. It also allows the user to cover part of a window
with another. These operations imply that all or part of the window must be
redrawn at some point in time. The PM generates the WM_PAINT message whenever
the contents of the window must be refreshed. One common misconception about
the Presentation Manager is that it handles the reconstruction of a window
that has been overlaid. The truth is that your program must reconstruct its
own window whenever the WM_PAINT message is received.

The WM_ERASEBACKGROUND message tells your program that the window needs to be
erased, perhaps because the window is being moved. By returning TRUE, you are
allowing the PM to do this for you. Otherwise your program must do it.
Each time a key is pressed, the WM_CHAR message is generated. Each time the
user requests a vertical scroll, the WM-VSCROLL message is generated. Each
time a horizontal scroll is requested, the WM_HSCROLL message is generated.
The mouse messages are self-explanatory.
When you run this program, a window will pop up. This window can be resized,
moved, minimized, or maximized. Other than those basic window operations, the
program does nothing else.
Because this program is a template for future applications, it does not do
anything with the messages. But your applications will need to. Also, keep in
mind that when your program does not actually need to concern itself with a
message, such as a program that does not have scroll bars, its message can be
removed from the switch statement. In this case the default processing will
handle it.


_A PRESENTATION MANAGER APPLICATION TEMPLATE_
by Herbert Schildt


[LISTING ONE]


/* A Presentation Manager Application skeleton. */

#define INCL_PM

#include <os2.h>
#include <stddef.h> /* get definition of NULL */

void far * pascal far window_func(void far *, unsigned short,
 void far *, void far *);

char class[] = "MyClass";

main()
{
 void far *hand_ab;
 void far *hand_mq;
 void far *hand_frame, far *hand_client;
 QMSG q_mess;
 unsigned flags = FCF_SYSMENU 
 FCF_SIZEBORDER FCF_TITLEBAR 
 FCF_VERTSCROLL FCF_HORZSCROLL 
 FCF_MINMAX;

 hand_ab = WinInitialize(NULL);

 hand_mq = WinCreateMsgQueue(hand_ab, 0);

 if(!WinRegisterClass(hand_ab, /* anchor block */
 class, /* class name */
 window_func, /* address of window function */
 CS_SIZEREDRAW, /* window style */
 0)) /* no storage reserved */
 exit(1);

 hand_frame = WinCreateStdWindow(HWND_DESKTOP,
 WS_VISIBLE,
 (void far *) &flags,
 (char far *) class,
 (char far *) "My Window",
 0L, /* resource modules */
 NULL,
 0,
 &hand_client); /* client handle */

 /* message loop */
 while(WinGetMsg(hand_ab, &q_mess, NULL, 0, 0))

 WinDispatchMsg(hand_ab, &q_mess);

 WinDestroyWindow(hand_frame);

 WinDestroyMsgQueue(hand_mq);
 WinTerminate(hand_ab);
}

/* This is the window function. */
void far * pascal far window_func(void far *handle,
 unsigned short mess,
 void far *parm1,
 void far *parm2)
{

 switch(mess) {
 case WM_CREATE:
 /* Perform any necessary initializations here. */
 break;

 case WM_PAINT:
 /* Refresh the window each time the WM_PAINT message
 is received.
 */
 break;

 case WM_ERASEBACKGROUND:
 /* By returning TRUE, the PM automatically erases
 the old window each time the window is resized
 or moved. Without this, your program must
 manually handle erasing the window with it changes
 size or location.
 */
 return(TRUE);

 case WM_CHAR:
 /* Process keystrokes here. */
 break;

 case WM_HSCROLL:
 /* Process horizontal scroll request. */
 break;

 case WM_VSCROLL:
 /* Process vertical scroll request. */
 break;

 case WM_MOUSEMOVE:
 /* Process a mouse motion message. */
 break;

 case WM_BUTTON1DOWN:
 /* 1st mouse button is pressed. */
 break;

 case WM_BUTTON2DOWN:
 /* 2nd mouse button is pressed. */
 break;


 case WM_BUTTON3DOWN:
 /* 3rd mouse button is pressed. */
 break;

 /* If required by your application, you may also need to
 process these mouse messages:

 WM_BUTTON1UP
 WM_BUTTON1DBLCLK
 WM_BUTTON2UP
 WM_BUTTON2DBLCLK
 WM_BUTTON3UP
 WM_BUTTON3DBLCLK
 */
 }
 /* All messages not handled by the window_func,
 must be passed along to the PM for default
 processing.
 */
 return WinDefWindowProc(handle, mess, parm1, parm2);
}





[LISTING TWO]

NAME skeleton
HEAPSIZE 4096
STACKSIZE 4096
EXPORTS window_func






























March, 1989
DYNAMIC LINK LIBRARIES UNDER MICROSOFT WINDOWS


Sharing code through DLLs is one way to deal efficiently with excess baggage
in a multitasking environment




Margaret Johnson and Mark Solinski


Margaret K. Johnson is a software engineer at Beckman Instruments. She can be
reached at 2500 Harbor Blvd., M/S D-33-B, Fullerton, CA 92634. CompuServe ID
74706,2325. Mark Solinski is director of technical services for the Whitewater
Group. He can be reached at 906 University Place, Evanston, IL 60201.


With so many people considering the benefits of OS/2, dynamic link libraries
(also known as dynalinks or DLLs) have become a hot topic. Few people,
however, realize that DLLs have been available to programmers for more than
two years as part of the Microsoft Windows operating environment. As a matter
of fact, most of the Microsoft Windows environment is implemented as DLLs.
This article discusses the virtues of DLLs and also provides a clear example
of a DLL that can be integrated into any Windows' application. It is our hope
that developers currently creating run-time libraries for MS-DOS, with plans
for future products under OS/2, would consider DLLs Microsoft for windows as
one of the platforms for their products.


DLLs Versus Run-Time Libraries


In the MS-DOS world, code is compiled and linked with external libraries or
object modules to create an executable file. Frequently, routines in libraries
are generic, consisting of often-used functions--for example, the run-time
library included with the compiler. Each executable that calls a routine from
the library contains a copy of the code for that fu ction. Under typical DOS
single-tasking conditions, and assuming it doesn't matter how large the
executable file is, run-time libraries are just fine.
However, having each application carry around a copy of common code is
inefficient in a multitasking system such as Windows. Windows is an operating
environment that sits on top of DOS and creates a nonpreemptive, event-driven
multitasking environment for its applications. This common code "baggage" can
cause particularly severe problems because Windows, DOS, and your application
have to share the infamous 640K of memory (assuming your system has no
expanded memory and the himem.sys driver is not installed).
Windows relieves some of this congestion by allowing multiple instances of the
same application to share the same code space, but, unfortunately, if multiple
applications are loaded, code is not shared. For example, if each application
calls the same function to smooth data from a run-time library, then Windows
will allocate code space for the smooth function for each application. As
anyone who has run multiple applications under Windows knows, memory is to
Windows what water is to the desert--a precious resource that should not be
wasted.
If a function resides in a DLL, then only one copy will be loaded no matter
how many applications call it. This approach keeps the application's
executable small because the common code is not included in each application.
Also, the code is dynamically loaded at run time, so as long as the parameters
passed in a function call do not change, you can change the innards of a
function in a DLL at any time without having to recompile and relink the
application. Thus, judicious use of DLLs lets you customize and upgrade your
applications without producing whole new versions.
Loading the library at run time provides other intriguing opportunities. Some
packages-for example, Actor from Whitewater, Excel from Microsoft, and SQL
Windows from Gupta --can integrate DLL functions within their programs. This
ability allows developers to use their own warm and comfy functions in
extremely powerful existing applications. But wait, there's more! DLLs need
not contain any code or data at all. They can also be completely made of
system resources such as fonts, bit maps, icons, or cursors --a great way to
allow customization of applications without having to distribute source code,
a must for library developers.


Problems with Creating DLLs


As is usually the case in life, additional benefits are not realized without
some pain, and this is certainly true of DLLs. There are two big differences
between code in a DLL and code in a standard run-time library under MS-DOS.
The first one is on the tip of the tongue of anyone who has ventured into the
DLL domain - the infamous SS!=DS issue. The second difference concerns the use
of global variables.
Normally, when C code is compiled, it is assumed the data segment (DS) is the
same as the stack segment (SS). This assumption is valid when passing the
address of a parameter to a function for the small (one code segment and one
data segment) and medium (multiple code segments and one data segment) models,
because only the 16-bit offsets of the addresses are passed.
DLLs, however, have their own data segment and use the stack of the caller. In
this case, the stack segment does not equal the data segment (SS!=DS) and the
16-bit offsets are not valid addresses because all automatic variables -
parameters and local variables--are kept on the stack and all
others--globaland static variables --are contained in the data segment. This
means that an address passed to a function must contain both the segment and
offset as 32-bit far pointers, and as a result, some of the standard C
run-time library functions cannot be used. These functions are listed in an
appendix of the Microsoft Software Development Kit (SDK) for Windows. In
general, any of the buffered I/O functions fprintf, fscanf, sscanf, and soon),
exec functions (for example, execlp, execv), and spawn functions (for example,
spawnl, spawnv) are off limits.
The second problem occurs because of our disposition and familiarity with
writing single-tasking software. Because these programs need not be reentrant,
it is common to sprinkle code generously with global variables. This approach
can be a source of grief to the person who wishes to convert existing code to
work in a DLL. Although code is not reentrant in Windows, the data is
reentrant because DLLs have only one data segment, which is shared by all
applications. You cannot expect a variable that is set by one function of the
DLL, to contain that value when retrieved in another function.
An example of this might be a DLL that contains routines for plotting the
values of variables. Usually a sequence of calls consisting of initialization,
setting plot variables, graphing the plot, and cleanup would be used to
perform this function. Putting the plotting variables in global memory assumes
each application that makes this series of calls has its own data area to hold
these variables. If two applications (A and B) coexist and application A makes
calls to set the plotting variables, and then B does the same before A can
make a call to plot the variables, the variables used by the DLL will be set
to B's preferences, not A's.
Most of these problems are inherent in multitasking and shared-code systems
and are not particular "quirks" of Microsoft Windows. The solutions to these
problems, implemented through careful software design, will certainly be of
benefit to developers who wish to port their applications and libraries to
multitasking environments, including OS/ 2. The example that follows
illustrates how a simple, extendable help facility can be added to any
application, and can be customized without recompiling and relinking the main
application.


Help Library


The help library we have created follows the convention put forth in the
Microsoft Windows Application Style Guide. All applications that follow these
guidelines will have the same visual appearance and present a consistent user
interface (keyboard, mouse, screen, and other ports). Bells should be ringing
inside your head when you hear phrases such as "All (your) applications . .
.", "same visual. . .," and "consistent user. . . ." These are usually some of
the criteria for producing function libraries. Dynamic libraries are an even
better choice because sometimes standards change and these changes can be
incorporated into a new library and become immediately available to all the
applications that used the old library without your having to recompile and
relink.
This discussion illustrates another feature of DLLs -- the ability to isolate
customization code into different libraries. Thus, developers need only
publish function specifications for the routines in the library. Microsoft
uses this technique for its hardware drivers. Printer manufacturers, for
instance, can write DLL libraries that take advantage of the best features of
their printers and provide the same "functional" appearance to Microsoft
Windows. The helplib.c file (Listing One, page 82) defines the "public"
protocol for the help library. The two functions we create are Topics() and
Screen(). Topics() displays a list of available help topics; Screen() displays
the text relating to a specific topic. There is a one-to-one mapping between a
screen and a topic. This mapping is enforced by using the same token name in
the resource file, helplib.rc (Listing Two, page 82), for the topic strings
and the user-defined resources. For example, the token for the "caveats" topic
string, CAVEATS, is the same as the token for the name of the user-defined
resource named CAVEATS. The text for this screen is located in the ASCII file
CAVEATS.ASC (see Example 1). This file can be created with any editor. There
is one file per screen. Each application that shares the help library would
have a sequential list of tokens (and associated text) for the help topics.
Examples 2, 3, and 4 list other user-defined resources used by our library.
Example 1: The user-defined resource name CAVEATS.ASC caveats
As is usually the case in life, it takes a little more to get a little more.
This is certainly true with DLLs. There are two big differences between code
in a DLL and code in a standard run-time library under MS-DOS. The first one
is on the tip of the tongue of anyone who has ventured into the DLL domain.
This is the SS != DS issue. The second concerns the use of global variables.
Normally, when C code is compiled, it is assumed that the data segment (DS) is
the same as the stack segment (SS). This is valid when passing the address of
a parameter to a function for the small (i.e., one code segment and one data
segment) and medium (i.e., multiple code segments and one data segment) models
since only the 16-bit offsets of addresses are passed.
DLLs, however, have their own data segment and use the stack of the caller. In
this case, the stack segment does not equal the data segment (SS!=DS) and the
16-bit offsets are not valid addresses since all automatic variables (i.e.,
parameters and local variables) are kept on the stack and all others (i.e.,
global and static variables) are contained in the data segment. This means
that any addresses passed to a function must contain both the segment and
offset (i.e., 32-bit far pointers).
Because of this, some of the standard C run-time library functions cannot be
used. These are listed in an appendix of Microsoft's Software Development Kit
(SDK) for Windows. In general, any of the buffered I/O functions (e.g.,
fprintf, fscanf, sscanf), exec functions (e.g., execlp, execv), and spawn
functions (e.g., spawnl, spawnv) are off limits.
Because programs in DOS need not be reentrant, it is common to sprinkle code
generously with global variables. This can be a source of grief to the person
who wishes to convert existing code to work in a DLL. Although code is not
reentrant, the data is. A DLL has only one data segment, which is shared by
all applications. Don't expect a variable that is set by one function of the
DLL to contain the set value in another function. A common case is plotting
libraries. Usually the sequence of calls consists of setting plot variables,
graphing the plot using the plot variables set previously, then performing any
cleanup.

Putting the plotting variables in global memory assumes each application that
makes this series of calls has its own data area to hold these variables. If
these routines reside in a DLL, then the global variable that sets the x
origin within the DLL is the same variable used by any application that calls
this routine. For example, if two applications (A and B) coexist and
application A makes calls to set the plotting variables, and then B does the
same before A can call to plot, the variables used by the DLL will be set to
B's preferences.
Example 2: Another user-defined resource, this one called COOKBOOK.ASC

 cookbook

 Steps to Follow when Creating a DLL:


 1. Create the resource file.
 2. Create the library source files.
 3. Make sure you have an initialization function.
 4. Create the module definition file.
 5. Create a make file to:
 a. Compile library and initialization source files.
 b. Use rc compiler to compile any resources.
 c. Use the link4 linker to create the .EXE file.
 d. Attach the resources to the .EXE file.
 e. Use the implib tool to create an import library.

Example 3: A user-defined resource that provides reference information

 references

 Two books are definite musts:

 Programming Windows by Charles Petzold
 (Microsoft Press, 1988)

 852 pages jam-packed with Windows
 programming tips and useful code. This is an
 incredible source for Windows developers.

 Inside OS/2 by Gordon Letwin (Microsoft
 Press, 1988)

 289 pages from the chief architect for
 systems software at Microsoft. This book
 is great for getting a feel for what OS/2
 has to offer and the philosophy behind
 it. It also gives a good feeling for the
 differences between the API for Windows
 and OS/2 without getting bogged down in
 the code.

Example 4: A fourth user-defined resource, stored in the ASCII file
VSRUNTIME.ASC

 versus run-time libraries

 In the MS-DOS world, code is compiled and linked with external libraries
 or object modules to create an executable. Frequently, routines in
 libraries are generic, consisting of often-used functions. An example
 of this is the run-time library included with the compiler. Each
 executable that calls a routine from the library contains a copy of the
 code for that function. Under typical DOS single-tasking conditions,
 and assuming it doesn't matter how large the executable file is,
 run-time libraries are just fine.

 Having each application carry around a copy of common code is
 inefficient in a multitasking system such as Windows. Windows is an
 operating environment that sits on top of DOS and creates a
 nonpreemptive, event-driven multitasking environment for its
 applications. In fact, with Windows, this is even more important
 since Windows is constrained to sit on top of DOS, and the environment
 that DOS sits in only allows 640K of memory (assuming no expanded
 memory). In Windows, if multiple instances of the same application are
 loaded, they will share the same code space. If multiple applications

 are loaded, however, code is not shared. If each application call the
 same function to smooth data from a run-time library, then Windows will
 allocate code space for the smooth function for each application. As
 anyone who has run multiple applications under Windows knows, memory is
 to Windows what water is to the desert--a precious resource that should
 not be wasted.

 If a function resides in a DLL, then only one copy will be loaded no
 matter how many applications call it. This also keeps the application's
 executable small by keeping the common code out of each application. Not
 only that, but since the code is dynamically loaded at run time, as long
 as the parameter sequence of the function call does not change, the
 innards of a function in a DLL can be changed without forcing the
 application's executable to be recompiled and relinked.

 Loading the library at run time allows other exciting opportunities.
 Some packages allow functions in a DLL to be called within their program.
 These include Actor from Whitewater, Excel from Microsoft, and SQL
 Windows from Gupta. Thus, the developers can use his warm and comfy
 functions in extremely powerful existing applications. But wait, there's
 more! DLLs need not contain any code or data at all! They can also be
 completely made of resources such as fonts, bit maps, icons, or cursors.

Both functions take as input an LPSCREEN type variable (as defined in
helplib.h, Listing Three, page 82). This type is a far pointer to a SCREEN
type that is typedefed as a structure containing the variables wStart, wEnd,
and wScreen. Topics() uses the wStart and wEnd token settings to reference the
range of character strings to load. Both Topics() and Screen() use wScreen to
reference the user-defined ASCII file resources. All public functions (that
is, those having the ability to be invoked from outside the library) must be
declared as FAR PASCAL. Declaring a function FAR allows it to be accessed from
another application that will not have the same code segment. The PASCAL
declaration specifies the use of the Pascal calling sequence, which means that
parameters are pushed on the stack from left to right instead of from right to
left, as in C.
Figure 1 shows the dialog box that pops up when Topics() is called. The dialog
box displays a list box containing the strings mentioned previously and the
two push buttons Help and Cancel. The Screen() function is called if you wish
to get help on the highlighted topic. As shown in Figure 2, Screen() displays
a dialog box that contains the help text in a scrollable window along with the
four push buttons Topics, Next, Previous, and Cancel. Pushing Topics returns
you to the Topics dialog box. Pushing Next displays the text for the next help
topic listed in the Topics dialog box list; and pushing Previous displays the
previous one. You can also call Screen() directly to display a specific help
text.
The "private" protocol of the DLL is implemented in the file, helpdlg.c (
Listing Four, page 82). The ScreenDlgProc, Screen WndProc, and TopicDlgProc
functions process the messages that Microsoft Windows sends to the dialogs.
Typically, we are most concerned with the user-interface messages such as
scrolling the window and moving it around and the system messages such as
redisplaying a portion of the screen that is now visible because it was moved
in front of a window that was previously obscuring it. A full discussion of
the Microsoft Windows message system is beyond the scope of this article and
we recommend that those people interested in the subject get a copy of
Programming Windows by Charles Petzold (Microsoft Press, 1988) -- an
invaluable aid for all Windows programmers.
Another important part of the private protocol of the DLL is the maintenance
of the screen information. One "feature" of a DLL is reentrancy. This
complicates matters if you need to maintain global data. One way of keeping
track of global data is to allocate extra bytes to the window structure when
registering the window class. We have done this by setting the cbWndExtra
field to sizeof(HANDLE) bytes when registering the help screen window class in
libinitc.c (Listing Five, page 87). When Screen WndProc() receives a WM_CREATE
message, it allocates a chunk of local memory to hold the window's screen
information and puts the handle into the window structure at offset 0. This
allows easy retrieval when an event occurs that requires the help window's
screen data.
DLL libraries are different from applications in that they don't and can't
invoke the C run-time/Windows initialization. Any initialization that the
library needs must be done by the developer, usually with a short assembly
program that in turn calls a C function in the library to perform the
initialization. libintc.c and libinita.asm ( Listing Six, page 87) provide
this service.
The final piece of our help library puzzle is the module definition file,
helplib.def (Listing Seven, page 88). The primary purpose of the module
definition file is to define the characteristics of the data and code
segments, the local heap, and the exported functions. If this were a module
definition file for a Windows application, it would also include a parameter
for the stack size, which is not necessary in a DLL because the DLL uses the
calling application's stack. Also because all libraries have only one data
segment, the option SINGLE must be used on the DATA definition line.
The EXPORTS section defines the functions that are accessible to the outside
world. Of course, Screen() and Topics() are in this list, but so are the
functions ScreenDlgProc, ScreenWndProc, and TopicsDlgProc. This is necessary
because these DLL routines are called by Windows to process user input and
that is why these functions are declared FAR PASCAL. The LINK4 linker uses the
module definition file to add the necessary information to the .EXE file,
which allows run-time linking and execution of the library routine.
The make file, helplib (Listing Eight, page 88) uses the files discussed
previously to create the library files helplib.lib and helplib.exe. The
compiler switch that is necessary for a DLL is the - Alnw switch (or -Asnw for
a small model). This switch is needed to ensure that SS!=DS.


Testing the Library


The help library we provide gives an overview of this article and a
step-by-step description of the process of creating a library. We have also
included two examples of how the library can be used. The first, helpdemo.exe,
and its associated source code (in Listings Nine - Eighteen, pages 89 - 91)
shows how easy it is to call a DLL from a Windows application. The function
HelpDemoMsg() sets up the screen information and calls the Topics() function
in the DLL whenever the F1 key is pressed or the Help! menu item is selected.
The second example, helplib.act (Listing Nineteen, page 91), is written in
Actor, an incrementally compiled language for developing Windows applications.
This example illustrates how different DLLs can be specified at run time. As
long as the application knows what functions a library contains, it can load
the library, execute the function, and then free the library. Actor provides a
way to experiment with changing libraries on the fly and seeing how these
changes affect the way your application behaves.


Conclusion


Dynamic link libraries provide a great way to reuse code. With correct modular
design, applications can be customized without having to recompile and relink.
DLLs perform best when they are called to process data and then return. This
is because the DLL is shared by all the applications in the system that call
the library, and therefore global data cannot be maintained. One way to
overcome this limitation is to store information specific to the window in the
window's data structure.
We hope this article gives you a taste of the flexibility of DLLs. Developers
who have built DLLs for Microsoft Windows have had an early taste of OS/2
programming. Those people who develop third-party libraries for sale to the PC
programming community will find great acceptance from the Microsoft Windows
community and will also provide an easy platform for porting their libraries
to OS/2.

_DYNAMIC LINK LIBRARIES UNDER MICROSOFT WINDOWS_
by Margaret Johnson and Mark Solinski


[LISTING ONE]

/****************************************************************************
 * MODULE: HELPLIB.C
 * COMMENTS: contains the functions Screen and Topics (see HELPDLG.C)
 ****************************************************************************/
#include <windows.h> /* used by all modules written for Windows */
#include "helplib.h" /* library's include file */
#define HELPLIB

#include "prothelp.h" /* function prototypes */
/****************************************************************************
 * external variables
 ****************************************************************************/
extern HANDLE hInst; /* set by the initialization function in libinitc.c */
/****************************************************************************
 * global variables
 ****************************************************************************/
LPSCREEN lpsc;
BOOL TOPICS;
BOOL SCR;
 /****************************************************************************
 * Local variables
 ****************************************************************************/
 static FARPROC lpfnScreenDlgProc;
 static FARPROC lpfnTopicsDlgProc;
/************************************************************************
 * FUNCTION: Screen
 * PURPOSE: Display help text on a topic.
 ************************************************************************/
BOOL FAR PASCAL Screen( LPSCREEN sc )
{MSG msg;
 HWND hWnd;
 LockData(0);
 lpsc = sc;

 if (!(lpfnScreenDlgProc = MakeProcInstance(ScreenDlgProc,hInst)))
 return FALSE;

 if (!(hWnd = CreateDialog(hInst,"HELP_BOX",GetActiveWindow(),
 lpfnScreenDlgProc)))
 return FALSE;
 while (GetMessage (&msg,NULL,0,0))
 {if (!IsDialogMessage(hWnd,&msg) )
 {TranslateMessage(&msg);
 DispatchMessage(&msg);
 }
 }

 FreeProcInstance(lpfnScreenDlgProc);

 if (TOPICS)
 {Topics(lpsc);
 }

 UnlockData(0);
 return TRUE;
}
/************************************************************************
 * FUNCTION: Topics
 * PURPOSE: to present a listbox of currently available help topics.
 ************************************************************************/
VOID FAR PASCAL Topics( LPSCREEN sc )

{LockData(0);
 lpsc = sc;

 lpfnTopicsDlgProc = MakeProcInstance(TopicsDlgProc,hInst);
 DialogBox(hInst,"TOPICS_BOX",GetActiveWindow(),lpfnTopicsDlgProc);

 FreeProcInstance(lpfnTopicsDlgProc);

 if (SCR)
 {Screen(lpsc);
 }

 UnlockData(0);
 return;
}





[LISTING TWO]

/****************************************************************************
 * FILE: helplib.rc
 * PURPOSE:resource file for the helplib DLL
 ***************************************************************************/
#include <style.h>
#include "helplib.h"
#define TABGRP (WS_TABSTOP WS_GROUP)

VSRUNTIME TEXT vsruntime.asc
CAVEATS TEXT caveats.asc
COOKBOOK TEXT cookbook.asc
REF TEXT ref.asc
A TEXT a.asc
B TEXT b.asc
C TEXT c.asc

STRINGTABLE
 BEGIN
 IDS_MEMERROR "Out of Memory"
 VSRUNTIME, "versus run time libraries"
 CAVEATS, "caveats"
 COOKBOOK, "cookbook"
 REF, "references"
 A, "a"
 B, "b"
 C, "c"
 END

rcinclude HELP.dlg
rcinclude TOPICS.dlg







[LISTING THREE]

/***************************************************************************
 * FILE: helplib.h
 * PURPOSE: include file for the helplib DLL
 **************************************************************************/

typedef struct {
 WORD wScreen;
 WORD wStart;
 WORD wEnd;
 }SCREEN, FAR * LPSCREEN, NEAR *NPSCREEN;
#define ID_SCREEN_HELP 100
#define ID_LB_TOPICS 101
#define ID_NEXT_HELP 102
#define ID_PREVIOUS_HELP 103
#define ID_TOPICS_HELP 104
#define ID_SCROLL_HELP 105

#define VSRUNTIME 300
#define CAVEATS 301
#define COOKBOOK 302
#define REF 303

#define A 500
#define B 501
#define C 502

#define IDS_MEMERROR 1000







[LISTING FOUR]

/**************************************************************************
 * MODULE: HELPDLG.C
 **************************************************************************/

#include <windows.h> /* required for all Windows applications */
#include "helplib.h" /* library's include file */
#define HELPLIB
#include "prothelp.h" /* function prototypes */
#include "string.h" /* strlen */

/*************************************************************************
 * to allow multiple screens from different apps/instances
 *************************************************************************/
typedef struct screenStruct {
 WORD wScreen;
 WORD wStart;
 WORD wEnd;
 int nPage;
 int nTopics;
 int nNumLines;
 HWND hScroll;
 int nVscrollPos;
 }HELPSCREEN, *NPHELPSCREEN;
/****************************************************************************
 * local variables
****************************************************************************/
#define GWW_SCREENHANDLE 0
#define MAXBUFLEN 80

#define MAXLINES 250
#define LOCAL static
/*************************************
 * scroll bar positioning variables
*************************************/
LOCAL int nVscrollMax;
/*************************************
 * buffer to hold the help text
*************************************/
LOCAL char szText[MAXLINES][MAXBUFLEN];
/*******************************************
 * screen information
 *******************************************/
LOCAL NPHELPSCREEN sptr;
/****************************************************************************
 * local function prototypes
****************************************************************************/
LOCAL VOID NEAR getText ( VOID );
LOCAL VOID NEAR setScroll ( VOID );
LOCAL VOID NEAR setNewHelp ( HWND );
LOCAL HANDLE NEAR setToScreen ( HWND );
LOCAL BOOL NEAR differentScreen ( HWND );
LOCAL BOOL NEAR initScreen ( HWND );
LOCAL VOID NEAR freeScreen ( HANDLE );
/****************************************************************************
 * external variables
****************************************************************************/
extern HANDLE hInst; /* set by the initialization function in libinitc.c */
extern LPSCREEN lpsc; /* passed in by the calling function */
extern BOOL TOPICS;/* used by ScreenDlgProc when the user requests Topics */
extern BOOL SCR; /* used by TopicsDlgProc when the use requests Help */
/****************************************************************************
 * Function: ScreenDlgProc
 * Purpose: To respond to the Push Buttons: Topics, Next, Previous, and
 * Cancel on the Screen() dialog box.
 ****************************************************************************/
BOOL FAR PASCAL ScreenDlgProc(HWND hDlg, WORD wMessage, WORD wParam,
 LONG lParam)
 {int i;
 static BOOL bImoved=FALSE;
 HWND hWndScreen;
 HANDLE hScreen;
 switch(wMessage)
 {
 case WM_INITDIALOG:
 hScreen = setToScreen(hDlg);
 sptr->nTopics = sptr->wEnd - sptr->wStart + 1;
 sptr->hScroll = GetDlgItem(hDlg,ID_SCROLL_HELP);
 getText();
 LocalUnlock(hScreen);
 break;
 case WM_MOVE:
 bImoved=TRUE;
 break;
 case WM_VSCROLL:
 if (bImoved)
 {hScreen = setToScreen(hDlg);
 getText();
 setScroll();

 LocalUnlock(hScreen);
 bImoved=FALSE;
 }
 hWndScreen = GetWindow(hDlg,GW_CHILD);
 SendMessage(hWndScreen,WM_VSCROLL,wParam,lParam);
 break;
 case WM_COMMAND:
 hScreen = setToScreen(hDlg);
 hWndScreen = GetWindow(hDlg,GW_CHILD);
 switch(wParam)
 {
 case ID_PREVIOUS_HELP:
 sptr->wScreen = sptr->wStart + (sptr->wScreen-sptr->wStart+
 (sptr->nTopics-1)) % sptr->nTopics;
 setNewHelp(hWndScreen);
 LocalUnlock(hScreen);
 break;
 case ID_NEXT_HELP:
 sptr->wScreen = sptr->wStart + (sptr->wScreen-sptr->wStart+1)
 %sptr->nTopics;
 setNewHelp(hWndScreen);
 LocalUnlock(hScreen);
 break;
 case ID_TOPICS_HELP:
 TOPICS = TRUE;
 case IDCANCEL:
 freeScreen(hScreen);
 DestroyWindow(hDlg);
 break;
 default:
 return FALSE;
 }
 break;
 case WM_ACTIVATE: /* when the dialog is activated, check to
 * see if the correct screen mode is
 * is selected.
 */
 if (!wParam)
 break;
 case WM_PAINT: /* User could be switching 'tween n application's
 * (or n instances) Help Dialogs
 */
 hScreen = setToScreen(hDlg);
 if (differentScreen(hDlg))
 {getText();
 }
 if (WM_ACTIVATE == wMessage)
 {SetFocus(sptr->hScroll);
 setScroll();
 }
 LocalUnlock(hScreen);
 return FALSE;
 default:
 return FALSE;
 }
 return TRUE;
 }
/****************************************************************************
 * Function: ScreenWndProc

 * Purpose: To respond to messages received by the Screen() window.
 * Notes: The WM_VSCROLL, WM_PAINT message handling was derived from
 * Programming Windows by Charles Petzold, pp. 117-122.
 ****************************************************************************/
long FAR PASCAL ScreenWndProc(HWND hWnd, WORD wMessage, WORD wParam,
 LONG lParam)
{PAINTSTRUCT ps;
 LOCAL int xChar,yChar;
 LOCAL int yClient;
 LOCAL TEXTMETRIC tm;
 int i;
 int nVscrollInc;
 int nPaintBeg,nPaintEnd;
 HDC hDC;
 HANDLE hScreen;
 switch(wMessage)
 {
 case WM_CREATE:
 if (!initScreen(hWnd))
 break;
 hDC = GetDC(hWnd);
 GetTextMetrics(hDC,&tm);
 ReleaseDC(hWnd,hDC);
 xChar = tm.tmAveCharWidth;
 yChar = tm.tmHeight + tm.tmExternalLeading;
 TOPICS = FALSE;
 break;
 case WM_SIZE:
 hScreen = GetWindowWord(hWnd,GWW_SCREENHANDLE);
 sptr = (NPHELPSCREEN)LocalLock(hScreen);
 yClient = HIWORD(lParam);
 sptr->nPage = yClient / yChar ;
 LocalUnlock(hScreen);
 break;
 case WM_SETFOCUS:
 hScreen = GetWindowWord(hWnd, GWW_SCREENHANDLE);
 sptr = (NPHELPSCREEN)LocalLock(hScreen);
 SetFocus(sptr->hScroll);
 LocalUnlock(hScreen);
 break;
 /* the WM_VSCROLL message is sent by the ScreenDlgProc, which has
 * already taken care of sptr
 */
 case WM_VSCROLL:
 switch (wParam)
 {
 case SB_TOP:
 nVscrollInc = -sptr->nVscrollPos;
 break;
 case SB_BOTTOM:
 nVscrollInc = nVscrollMax - sptr->nVscrollPos;
 break;
 case SB_LINEUP:
 nVscrollInc = -1;
 break;
 case SB_LINEDOWN:
 nVscrollInc = 1;
 break;
 case SB_PAGEUP:

 nVscrollInc = min(-1, -sptr->nPage) ;
 break;
 case SB_PAGEDOWN:
 nVscrollInc = max(1, sptr->nPage) ;
 break;
 case SB_THUMBPOSITION:
 nVscrollInc = LOWORD(lParam) - sptr->nVscrollPos;
 break;
 default:
 nVscrollInc = 0;
 }
 if (nVscrollInc = max(-sptr->nVscrollPos,
 min(nVscrollInc, nVscrollMax - sptr->nVscrollPos)))
 {sptr->nVscrollPos += nVscrollInc;
 ScrollWindow(hWnd, 0, -yChar * nVscrollInc, NULL, NULL);
 UpdateWindow(hWnd);
 SetScrollPos(sptr->hScroll,SB_CTL,sptr->nVscrollPos,TRUE);
 }
 break;
 case WM_PAINT:
 hScreen = GetWindowWord(hWnd, GWW_SCREENHANDLE);
 sptr = (NPHELPSCREEN)LocalLock(hScreen);
 BeginPaint( hWnd, &ps );
 nPaintBeg = max(0,sptr->nVscrollPos + ps.rcPaint.top/yChar - 1);
 nPaintEnd = min(sptr->nNumLines,
 sptr->nVscrollPos + ps.rcPaint.bottom / yChar);
 for (i=nPaintBeg; i < nPaintEnd; ++i)
 {TextOut(ps.hdc,xChar,yChar * (1 - sptr->nVscrollPos+i),
 szText[i],strlen(szText[i]));
 }
 EndPaint(hWnd,&ps);
 LocalUnlock(hScreen);
 break;
 case WM_DESTROY:
 PostQuitMessage(0);
 break;
 default:
 return DefWindowProc( hWnd, wMessage, wParam, lParam );
 }
 return 0L;
}
/****************************************************************************
 * Function: TopicsDlgProc
 * Purpose: To respond to the list box and push button messages
 * of the Topics() dialog box.
 ****************************************************************************/
BOOL FAR PASCAL TopicsDlgProc(HWND hDlg, WORD wMessage, WORD wParam,
 LONG lParam)
 {int i;
 LOCAL char szBuff[MAXBUFLEN];

 switch(wMessage)
 {
 case WM_INITDIALOG:
 SetSysModalWindow(hDlg);
 for (i=lpsc->wStart;i<=lpsc->wEnd;++i)
 {LoadString(hInst,i,szBuff,MAXBUFLEN);
 SendDlgItemMessage(hDlg,ID_LB_TOPICS,LB_ADDSTRING,0,
 (LONG)(LPSTR)szBuff);

 }
 SendDlgItemMessage(hDlg,ID_LB_TOPICS,LB_SETCURSEL,0,0L);
 SCR = FALSE;
 break;
 case WM_COMMAND:
 switch(wParam)
 {
 case ID_LB_TOPICS:
 if ((HIWORD(lParam)) != LBN_DBLCLK)
 {break;
 }
 case IDOK:
 lpsc->wScreen = (WORD)SendDlgItemMessage(hDlg,ID_LB_TOPICS,
 LB_GETCURSEL,0,0L) + lpsc->wStart;
 SCR=TRUE;
 case IDCANCEL:
 EndDialog(hDlg,TRUE);
 break;
 default:
 return FALSE;
 }
 break;
 default:
 return FALSE;
 }
 return TRUE;
 }
/****************************************************************************
 * Function: getText
 * Purpose: To retrieve the help text for the current screen from the
 * Help library's resource file.
 ****************************************************************************/
LOCAL VOID NEAR getText( )
 {LPSTR lpText,lpTextBeg;
 HANDLE hRes;
 hRes = LoadResource(hInst,
 FindResource(hInst,MAKEINTRESOURCE(sptr->wScreen),"TEXT"));
 lpText = LockResource(hRes);
 sptr->nNumLines=0;
 lpTextBeg = lpText;
 while (*lpText != '\0' && *lpText != '\x1A' )
 {if (*lpText == '\r')
 {*lpText = '\0';
 lstrcpy(szText[sptr->nNumLines++],lpTextBeg);
 if (sptr->nNumLines >= MAXLINES)
 break;
 *lpText='\r';
 lpText = AnsiNext(lpText);
 if (*lpText = '\l')
 lpText = AnsiNext(lpText);
 lpTextBeg = lpText;
 }
 else
 lpText = AnsiNext(lpText);
 }
 *lpText = '\0';
 lstrcpy(szText[sptr->nNumLines++],lpTextBeg);
 GlobalUnlock(hRes);
 FreeResource(hRes);

 }
/****************************************************************************
 * Function: setScroll
 * Purpose: To set the scroll bar control's range and initial position.
 ****************************************************************************/
LOCAL VOID NEAR setScroll( )
 {nVscrollMax = max(0, sptr->nNumLines + 2 - sptr->nPage);
 SetScrollRange(sptr->hScroll, SB_CTL,0, nVscrollMax, FALSE);
 SetScrollPos (sptr->hScroll, SB_CTL,sptr->nVscrollPos,TRUE);
 }
/****************************************************************************
 * Function: setNewHelp
 * Purpose: Get the right screen help info
 ****************************************************************************/
LOCAL VOID NEAR setNewHelp(HWND hWnd)
 {sptr->nVscrollPos = 0;
 getText();
 setScroll();
 InvalidateRect(hWnd,NULL,TRUE);
 SetFocus(sptr->hScroll);
 }
/****************************************************************************
 * Function: setToScreen
 * Purpose: Lock the screen pointer to the correct screen info.
 ****************************************************************************/
LOCAL HANDLE NEAR setToScreen(HWND hWnd )
{HANDLE hScreen;
 HWND hWndScreen;
 hWndScreen = GetWindow(hWnd,GW_CHILD);
 hScreen = GetWindowWord(hWndScreen, GWW_SCREENHANDLE);
 sptr = (NPHELPSCREEN)LocalLock( hScreen );
 return hScreen;
}
/****************************************************************************
 * Function: differentScreen
 * Purpose: Find out if hDlg references the same window as the last call
 ****************************************************************************/
LOCAL BOOL NEAR differentScreen(HWND hDlg)
{LOCAL HWND hWnd = NULL;
 if (hWnd != hDlg)
 {hWnd = hDlg;
 return TRUE;
 }
 return FALSE;
}
/****************************************************************************
 * Function: initScreen
 * Purpose: Set up the screen on a WM_CREATE message.
 ****************************************************************************/
LOCAL BOOL NEAR initScreen(HWND hWnd)
 {HANDLE hScreen;
 char szMemErr[15];
 if (!(hScreen = LocalAlloc(LHND,sizeof(HELPSCREEN))))
 {LoadString(hInst,IDS_MEMERROR,szMemErr,15);
 MessageBox( GetParent(hWnd), szMemErr, NULL, MB_ICONEXCLAMATION);
 return FALSE;
 }
 sptr = (NPHELPSCREEN)LocalLock(hScreen);
 sptr->wScreen = lpsc->wScreen;

 sptr->wStart = lpsc->wStart;
 sptr->wEnd = lpsc->wEnd;
 LocalUnlock(hScreen);
 SetWindowWord( hWnd, GWW_SCREENHANDLE, hScreen );
 return TRUE;
 }
/****************************************************************************
 * Function: freeScreen
 * Purpose: All done with the screen, so release the memory.
 ****************************************************************************/
LOCAL VOID NEAR freeScreen(HANDLE hScreen)
{lpsc->wScreen = sptr->wScreen;
 lpsc->wEnd = sptr->wEnd;
 lpsc->wStart = sptr->wStart;
 LocalUnlock(hScreen);
 LocalFree(hScreen);
}






[LISTING FIVE]

/*
 * Utilities library C language initialization.
 */
#include <windows.h>
#include "helplib.h"
#include "prothelp.h"
/**********************************************************************
 * function prototypes
 **********************************************************************/
#define LOCAL static
int NEAR PASCAL LibInitC ( HANDLE );
LOCAL BOOL NEAR registerWindow ( VOID );
/**********************************************************************
 * Global vars
 **********************************************************************/
HANDLE hInst;
/**********************************************************************
 * C init called from the asm entry point init.
 * We require that the library have exactly one DS. See libinita.asm.
 * The DS can (should) be moveable.
 ***********************************************************************/
int NEAR PASCAL LibInitC(HANDLE hInstance)
{
 hInst = hInstance;
 return(registerWindow());
}
/************************************************************************/
LOCAL BOOL NEAR registerWindow ( )
{WNDCLASS WndClass; /* Window class structure */
 WndClass.style = CS_HREDRAW CS_VREDRAW;
 WndClass.lpfnWndProc = ScreenWndProc;
 WndClass.cbClsExtra = 0;
 WndClass.cbWndExtra = sizeof(HANDLE);
 WndClass.hInstance = hInst;

 WndClass.hIcon = NULL;
 WndClass.hCursor = LoadCursor(NULL, IDC_ARROW);
 WndClass.hbrBackground = COLOR_WINDOW+1;
 WndClass.lpszMenuName = NULL;
 WndClass.lpszClassName = "Screen";

 return RegisterClass(&WndClass);

}






[LISTING SIX]

; LIBINITA.ASM - - - Define entry point and perform initialization
; for libraries that have their own data segments.
 TITLE LIBINITA
?PLM = 1
?WIN = 1
memM = 1

.xlist
include cmacros.inc
.list

externFP <LocalInit>
externFP <UnlockSegment>
externNP <LibInitC>
;
; cx = size of the heap as defined in the .def file.
; di = "Instance handle". This is the C hInstance passed to WinMain.
; NOTE: For a WINDOWS library, hModule is interchangeable with
; hInstance.
; This is a handle to the global object containing the DS if there
; is one. If there is no DS for a library, it is the module handle,
; which is also a pointer to the module since that's a fixed global
; object.
; NOTE: The meaning and contents of hInstance are undocumented and
; should not be relied upon. That is, you may asuume that the value
; in di may be passed to any routine expecting hInstance. Making
; any other assumptions is VERY dangerous since the contents of
; hInstance may change in future versions.
; ds = data segment for our heap.
; es:si = pointer to the command line.

_INIT SEGMENT BYTE PUBLIC 'CODE'
assume CS: _INIT ; ???? assume vs assumes? ???????????????????
assumes DS,NOTHING ; ???? assume vs assumes? ???????????????????
assumes ES,NOTHING ; ???? assume vs assumes? ???????????????????

cProc LibInitA, <FAR, PUBLIC, NODATA>, <si,di>
cBegin
 xor ax,ax ; Return failure if there is no heap.
 jcxz ourexit
 cCall LocalInit,<ds,ax,cx> ; Set up our DS for doing LocalAllocs.


 or ax,ax
 jz ourexit

 cCall LibInitC,<di> ; Do any C initialization.
 ; di = hInstance.
 push ax ; Save the return value.
 mov ax,-1
 cCall UnlockSegment,<ax> ; NOTE that we leave DS unlocked.
 ; This implies that we must use
 ; Lock/UnlockData as we enter and
 ; any routines which access our
 ; data segment.
 pop bx

 or ax,ax ; Check if either one failed.
 jz ourexit
 or bx,bx
 jnz ourexit
 xor ax,ax

ourexit:
cEnd
_INIT ENDS
end LibInitA






[LISTING SEVEN]

LIBRARY helplib
DESCRIPTION 'Copyright 1988, mkj'
STUB 'WINSTUB.EXE'
CODE LOADONCALL MOVEABLE DISCARDABLE
DATA MOVEABLE SINGLE
HEAPSIZE 2048
CODE MOVEABLE
SEGMENTS _INIT PRELOAD MOVEABLE DISCARDABLE
 HELPLIB MOVEABLE LOADONCALL DISCARDABLE

EXPORTS
 Screen @1
 Topics @2
 ScreenDlgProc @3
 ScreenWndProc @4
 TopicsDlgProc @5







[LISTING EIGHT]

#
# FILE: helplib

# PURPOSE: make file for the helplib DLL
#
COMP = -c -Alnw -Gsw -Zp -Os -FPa -W2 -D LINT_ARGS
OBJS = helplib.obj helpdlg.obj libinita.obj libinitc.obj helplib.res
LOBJS = helplib+helpdlg+libinita+libinitc

.rc.res:
 rc -r $*.rc

.asm.obj:
 masm $*;

helplib.res: helplib.rc helplib.h vsruntim.asc caveats.asc \
 cookbook.asc ref.asc help.dlg topics.dlg

helplib.obj: helplib.c helplib.h prothelp.h
 cl $(COMP) -NT $* $*.c

helpdlg.obj: helpdlg.c helplib.h prothelp.h
 cl $(COMP) -NT HELPLIB $*.c

libinita.obj: libinita.asm
 masm $*;

libinitc.obj: libinitc.c
 cl $(COMP) -NT _INIT $*.c

helplib.exe: $(OBJS) helplib.def
 link4 $(LOBJS),helplib.exe/align:16,/map/li,mwinlibc mlibw mlibca /NOE
,helplib.def
 rc helplib.res helplib.exe
 implib helplib.lib helplib.def






[LISTING NINE]

 /**********************************************************************
 * FILE: helpdemo.c
 * PURPOSE: to demonstrate the use of the helplib DLL
 **********************************************************************/
#include <windows.h>
#include "helpdemo.h"

HANDLE hInstance; /* The Instance handle */
char szClass[10]; /* Window class name (see the .rc file) */
char szTitle[40]; /* Window title (see the .rc file) */
char szAbout[40]; /* About box string (see the .rc file */

static HWND hWnd;

long FAR PASCAL WndProc (HWND, unsigned, WORD, LONG);
BOOL NEAR Initialize( HANDLE, int );
 /***********************************************************************
 *Application main program.
 ***********************************************************************/
int PASCAL WinMain( hInst, hPrevInst, lpszCmdLine, nCmdShow )

 HANDLE hInst; /* Our instance handle */
 HANDLE hPrevInst; /* Previous instance of this application */
 LPSTR lpszCmdLine; /* Pointer to any command line params */
 int nCmdShow; /* Parameter to use for first ShowWindow */
 {
 MSG msg; /* Message structure */
 HANDLE hAccel; /* Accelerator handle */
 /* Save our instance handle in a global variable */
 hInstance = hInst;
 /* Initialize application, quit if any errors */
 if( ! Initialize( hPrevInst, nCmdShow ) )
 {return FALSE;
 }
 /* Main message processing loop. Get each message, then translate
 * keyboard messages and finally dispatch each message to its window
 * function.
 */
 hAccel = LoadAccelerators(hInstance,szClass);
 while( GetMessage( &msg, NULL, 0, 0 ) )
 {if (!TranslateAccelerator(hWnd, hAccel, &msg))
 {TranslateMessage( &msg );
 DispatchMessage( &msg );
 }
 }
 return msg.wParam;
 }
 /************************************************************************
 *Initialize the application.
 *************************************************************************/
 BOOL NEAR Initialize( hPrevInst, nCmdShow )
 HANDLE hPrevInst; /* Previous instance handle, 0 if first */
 int nCmdShow; /* Parameter from WinMain for ShowWindow */
 {
 WNDCLASS WndClass; /* Class structure for RegisterClass */
 HMENU hMenu; /* Handle to the (system) menu */
 if( ! hPrevInst ) {
 /* Initialization for first instance only */
 /* Load strings from resource file */
 LoadString( hInstance, IDS_CLASS, szClass, sizeof(szClass) );
 LoadString( hInstance, IDS_TITLE, szTitle, sizeof(szTitle) );
 /* Register our window class */
 WndClass.style = CS_HREDRAW CS_VREDRAW;
 WndClass.lpfnWndProc = WndProc;
 WndClass.cbClsExtra = 0;
 WndClass.cbWndExtra = 0;
 WndClass.hInstance = hInstance;
 WndClass.hIcon = LoadIcon( NULL, IDI_APPLICATION );
 WndClass.hCursor = LoadCursor( NULL, IDC_ARROW );
 WndClass.hbrBackground = COLOR_WINDOW + 1;
 WndClass.lpszMenuName = szClass;
 WndClass.lpszClassName = szClass;
 if( ! RegisterClass( &WndClass ) )
 return FALSE;
 }
 else
 {/* Initialization for subsequent instances only */
 /* Copy data from previous instance */
 GetInstanceData( hPrevInst, szClass, sizeof(szClass) );
 GetInstanceData( hPrevInst, szTitle, sizeof(szTitle) );

 }
 /* Initialization for every instance */

 /* Create the window */
 hWnd = CreateWindow(
 szClass, /* Class name */
 szTitle, /* Window title */
 WS_OVERLAPPEDWINDOW , /* window style */
 CW_USEDEFAULT, /* x */
 0, /* y */
 CW_USEDEFAULT, /* x width */
 0, /* y width */
 NULL, /* Parent hWnd (none for top-level) */
 NULL, /* Menu handle */
 hInstance, /* Owning instance handle */
 NULL /* Parameter to pass in WM_CREATE (none) */
 );

 /* Insert "About..." into system menu */
 LoadString( hInstance, IDS_ABOUT, szAbout, sizeof(szAbout) );
 hMenu = GetSystemMenu(hWnd, FALSE);
 ChangeMenu(hMenu, 0, NULL, 999, MF_APPEND MF_SEPARATOR);
 ChangeMenu(hMenu, 0, (LPSTR)szAbout, IDS_ABOUT, MF_APPEND MF_STRING);

 /* Make the window visible */
 ShowWindow( hWnd, nCmdShow );

 /* Got all the information, update our display */
 UpdateWindow( hWnd );

 return TRUE;
 }






[LISTING TEN]

 /***************************************************************************
 * FILE: wndproc.c
 * PURPOSE: The functions WndProc and About for the helplib DLL demo.
 ***************************************************************************/
#include "windows.h"
#include "helpdemo.h"
extern HANDLE hInstance;
 /******************* Message function prototypes *******************/
extern VOID HelpDemoMsg ( HWND, WORD, LONG );
 /******************* local function prototype **********************/
BOOL NEAR paint ( HWND );
long FAR PASCAL WndProc( HWND, unsigned, WORD, LONG );
BOOL FAR PASCAL AboutDlgProc( HWND, unsigned, WORD, LONG );
long FAR PASCAL WndProc( hWnd, message, wParam, lParam )
 HWND hWnd;
 unsigned message;
 WORD wParam;
 LONG lParam;
 {

 FARPROC lpprocAbout;
 switch (message)
 {
 case WM_SYSCOMMAND:
 switch (wParam)
 {
 case IDS_ABOUT:
 /* Bind callback function with module instance */
 lpprocAbout = MakeProcInstance( (FARPROC)AboutDlgProc,
 hInstance);
 DialogBox( hInstance, MAKEINTRESOURCE(ABOUTBOX), hWnd,
 lpprocAbout );
 FreeProcInstance( (FARPROC)AboutDlgProc );
 break;
 default:
 return DefWindowProc( hWnd, message, wParam, lParam );
 }
 break;
 case WM_COMMAND:
 switch(wParam)
 {
 case IDM_HELP:
 HelpDemoMsg (hWnd, wParam, lParam);
 break;
 }
 case WM_PAINT:
 paint( hWnd );
 break;
 case WM_DESTROY:
 case WM_CLOSE:
 PostQuitMessage( 0 );
 break;
 default:
 return DefWindowProc( hWnd, message, wParam, lParam );
 break;
 }
 return(0L);
 }
 /*************************************************************************
 * Paint procedure (for processing of WM_PAINT messages)
 **************************************************************************/
BOOL NEAR paint ( hWnd )
 HWND hWnd;
 {
 PAINTSTRUCT ps;
 BeginPaint(hWnd,&ps);
 EndPaint(hWnd,&ps);
 return TRUE;
 }
 /**************************************************************************
 * About Box
 **************************************************************************/
BOOL FAR PASCAL AboutDlgProc( hDlg, message, wParam, lParam )
 HWND hDlg;
 unsigned message;
 WORD wParam;
 LONG lParam;
 {
 if (message == WM_COMMAND)

 {EndDialog( hDlg, TRUE );
 return TRUE;
 }
 else if (message == WM_INITDIALOG)
 return TRUE;
 else return FALSE;
 }






[LISTING ELEVEN]

 /***************************************************************************
 * The message functions.
 ***************************************************************************/
#include "windows.h"
#include "helpdemo.h"
#include "helplib.h" /* library's help file */
#include "prothelp.h" /* prototypes for library's functions */

extern HANDLE hInstance;

 /******************* local function prototypes *******************/
VOID HelpDemoMsg ( HWND, WORD, LONG );

 /**************************************************************************/
VOID HelpDemoMsg( hWnd, wParam, lParam )
 HWND hWnd;
 WORD wParam;
 LONG lParam;
 {SCREEN sc;
 sc.wStart = VSRUNTIME;
 sc.wEnd = REF;
 Topics(&sc);
 }






[LISTING TWELVE]

/**************************************************************************
 * FILE: prothelp.h
 * PURPOSE:function prototypes for the helplib DLL
 *************************************************************************/
#ifndef HELPDLG
 #define HELPDLG extern
#endif
#ifndef HELPLIB
 #define HELPLIB extern
#endif

HELPLIB BOOL FAR PASCAL Screen ( LPSCREEN );
HELPLIB VOID FAR PASCAL Topics ( LPSCREEN );

HELPDLG BOOL FAR PASCAL ScreenDlgProc ( HWND, WORD, WORD, LONG );
HELPDLG BOOL FAR PASCAL TopicsDlgProc ( HWND, WORD, WORD, LONG );
HELPDLG LONG FAR PASCAL ScreenWndProc ( HWND, WORD, WORD, LONG );
extern LPSTR FAR PASCAL lstrcpy ( LPSTR, LPSTR );
extern int FAR PASCAL lstrlen ( LPSTR );






[LISTING THIRTEEN]

/*************************************************************************
 * FILE: helpdemo.h
 * PURPOSE: include file for the help DLL demonstration window
 *************************************************************************/

/************** strings **************************************************/
#define IDS_CLASS 0 /* String Table ID for the Class Name */
#define IDS_TITLE 1 /* String Table ID for the Title */
#define IDS_ABOUT 2 /* String Table ID for the About box */

/************** menus ****************************************************/
#define ABOUTBOX 3 /* About dialog resource ID */
#define IDM_HELP 1000 /* Menu resource ID */


[LISTING FOURTEEN]

NAME HELPDEMO
DESCRIPTION 'Help DLL Demonstration, Copyright 1988, mkj'
STUB 'WINSTUB.EXE'
CODE LOADONCALL MOVEABLE DISCARDABLE
DATA MOVEABLE MULTIPLE
HEAPSIZE 2048
STACKSIZE 4096
EXPORTS
 WndProc @1
 AboutDlgProc @2







[LISTING FIFTEEN]

 /**********************************************************************
 * FILE: helpdemo.rc
 * PURPOSE: used with the helplib DLL for demonstration
 **********************************************************************/

#include <style.h>
#include "helpdemo.h"

STRINGTABLE
BEGIN

 IDS_CLASS "HelpDemo"
 IDS_TITLE "Help Demonstration"
 IDS_ABOUT "About..."
END

ABOUTBOX DIALOG 22, 17, 154, 75
STYLE WS_POPUP WS_DLGFRAME
BEGIN
 CTEXT "DLL Example" -1, 0, 5, 154, 8
 CTEXT "Help Library Demonstration" -1, 0, 14, 154, 8
 CTEXT "Version 1.00" -1, 30, 34, 94, 8
 CTEXT "Copyright ) 1988, mkj" -1, 0, 47,154, 9
 DEFPUSHBUTTON "Ok" IDOK, 61, 59, 32, 14, WS_GROUP
END

HelpDemo MENU
BEGIN
 MENUITEM "\aF1=Help", IDM_HELP,HELP
END

HelpDemo ACCELERATORS
BEGIN
 VK_F1, IDM_HELP, VIRTKEY
END






[LISTING SIXTEEN]

TOPICS_BOX DIALOG LOADONCALL MOVEABLE DISCARDABLE 28, 19, 206, 79
STYLE WS_DLGFRAME WS_POPUP
BEGIN
 CONTROL "", ID_LB_TOPICS, "listbox", LBS_NOTIFY WS_BORDER 
WS_VSCROLL WS_CHILD TABGRP, 12, 18, 144, 49
 CONTROL "&Help", IDOK, "button", BS_DEFPUSHBUTTON WS_TABSTOP 
WS_CHILD TABGRP, 165, 17, 32, 14
 CONTROL "Cancel", IDCANCEL, "button", BS_PUSHBUTTON WS_TABSTOP 
WS_CHILD , 165, 43, 32, 14
 CONTROL "Help Topics on DLL's", 103, "static", SS_LEFT WS_CHILD,
12, 5, 100, 11
END







[LISTING SEVENTEEN]

HELP_BOX DIALOG LOADONCALL MOVEABLE DISCARDABLE 10, 9, 262, 131
STYLE WS_BORDER WS_CAPTION WS_POPUP WS_SYSMENU WS_VISIBLE
CAPTION "Help"
BEGIN
CONTROL "", ID_SCREEN_HELP, "Screen", WS_BORDER WS_CHILD TABGRP,
9, 9, 234, 90

CONTROL "", ID_SCROLL_HELP, "scrollbar", SBS_VERT WS_CHILD 
WS_CLIPSIBLINGS, 242, 9, 8, 90
CONTROL "&Next", ID_NEXT_HELP, "button", BS_DEFPUSHBUTTON 
WS_TABSTOP WS_CHILD, 76, 105, 44, 16
CONTROL "&Previous", ID_PREVIOUS_HELP, "button", BS_PUSHBUTTON 
WS_TABSTOP WS_CHILD, 139, 105, 44, 16
CONTROL "Cancel", IDCANCEL, "button", BS_PUSHBUTTON WS_TABSTOP 
WS_CHILD, 201, 105, 44, 16
CONTROL "&Topics", ID_TOPICS_HELP, "button", BS_PUSHBUTTON 
WS_TABSTOP WS_CHILD, 14, 105, 44, 16
END







[LISTING EIGHTEEN]

#
# FILE: helpdemo
# PURPOSE: make file for the helplib DLL demo
#
COMP = -c -AS -D LINT_ARGS -Os -Zp -Gsw -FPa
ASM =

.rc.res:
 rc -r $*.rc

.c.obj:
 cl $(COMP) $*.c

.asm.obj:
 masm $(ASM) $*;

helpdemo.res: helpdemo.rc helpdemo.h

helpdemo.obj: helpdemo.c helpdemo.h

wndproc.obj: wndproc.c helpdemo.h

msgs.obj: msgs.c helpdemo.h helplib.h prothelp.h

helpdemo.exe: helpdemo.obj wndproc.obj msgs.obj helpdemo.res
link4 helpdemo+wndproc+msgs,/align:16,/map/li,slibw slibca
helplib /NOE /co , helpdemo.def
 rc helpdemo.res







[LISTING NINETEEN]

/* **************************************
 FILE: helplib.act

 PURPOSE: example of specifying and loading
 the library at run-time instead of
 at link (link4). The code is written in
 Actor, an object-oriented language for
 developing MS-Windows applications.
 ************************************** */

/* **************************************
 INITIALIZATION
 ************************************** */
/* define constants */
#define VSRUNTIME 300
#define CAVEATS 301
#define COOKBOOK 302
#define REF 303

#define A 500
#define B 501
#define C 502
!!

/* create a new C struct */
SCREEN := new(CStruct);
!!

init(SCREEN, #(
 int wScreen 1
 int wStart 1
 int wEnd 1
));
!!
/* initialize help library */
HelpLib := new(Library); /* create new library */
HelpLib.name := "HELPLIB.EXE"; /* set the file name of the library */
add(HelpLib, #SCREEN, 0, #(1)); /* add the names of the exported
 functions in the library that you
 will call. The '0' is the return
 type (0=int,1=long), and the
 '#(1)' lists the arguments - one
 long variable */
!!
/* load library */
load(HelpLib);
!!
/* **************************************
 BODY
 ************************************** */
/* fill C Struct */
SCREEN[#wStart] := VSRUNTIME;
SCREEN[#wEnd] := REF;
SCREEN[#wScreen] := CAVEATS;
!!
/* call DLL to put up help dialog */
pcall(HelpLib.procs[#SCREEN], lP(SCREEN));
freeHandle(SCREEN);
!!
/* **************************************
 CLEAN-UP
 ************************************** */

/* free the library */
free(HelpLib);
!!



























































March, 1989
WRITING PORTABLE APPLICATIONS WITH X/GEM


How does X/GEM compare with other windowing systems?




Bill Fitler


Bill Fitler is the X/GEM project manager with Digital Research (DRI) in
Monterey, Calif. He can be reached at 408-649-3896.


What are the main issues in creating portable graphics applications? For
example, how do you implement foreign language versions of your software? And
how do you handle the fundamental differences among various hardware
platforms? To achieve portability, you need to follow certain general
guidelines. The rules depend partly on what your application does, and partly
on what you want to port to. This article explores these and other problems,
showing how it's done with X/GEM, and comparing the X/GEM solution with other
common windowing systems.
Let's begin by discussing what X/GEM is.
X/GEM is based on the older Graphical Environment Manager (GEM), introduced in
1984 by Digital Research (DRI). GEM enables graphics applications to run on a
broad range of computers, ranging from Ataris to low-end 8088 based PCs and
compatibles. X/GEM provides a GEM compatible application program interface
(API) that functions on higher-end workstations, thus allowing GEM programs to
be more easily ported to these more powerful computers. X/GEM currently works
with DRI's FlexOS (a multitasking, multiuser, real-time operating system), and
versions are in the works for Presentation Manager and the X Window system.
The X/GEM API provides two sets of run-time application services, shown in
Figure 1. The first, called the virtual device interface (VDI), is where
graphics portability is implemented. The second is the application environment
services (AES), which provides the user interface building blocks for an
application. Built on top of the VDI, the AES also provides functions, that
enhance portability to both single tasking and multitasking operating
environments. X/GEM doesn't provide file system functions. Instead, GEM
applications rely on the services provided by the underlying operating system.
This basic structure of the X/GEM API --a user-interface-function set resting
atop device independent graphics -- is similar to Windows and the Macintosh.
All three systems implement a user interface "policy:" that is, the basic look
and feel shared by all applications running in that windowing environment.
X Window takes a different approach. Its implementors went to great lengths to
provide a "policy-free" interface. For example, the Xlib interface does not
contain any window control mechanisms. Instead it abdicates this policy to an
X "window manager" responsible for providing the look and feel of the
interface. This design has merits, because all X applications can be made to
behave in a fashion deemed appropriate by the window manager. At MIT, before X
was developed, there were many different vendors supplying software and
hardware, leading to a Tower of Babel effect. Now for applications that use X,
a person needs to learn only one window manager in order to use an application
from any vendor. This also allows new ways of interacting with the user to be
developed, which is useful because the art of user-interface design is still
in its infancy.
The X Window philosophy, however, also has drawbacks. One is increased
application complexity due to the need to negotiate with a window manager
which, for example, may not allow a window to be arbitrarily located on the
display device. A second problem is that this design requires more computing
resources. That might not be an issue in a workstation environment, but it is
on a low-end personal computer.
Another X Window problem is the proliferation of window managers, each of
which sets forth its own user-interface policy. As of this writing no clear
winner has emerged, with the result that end users are confused about which
window manager to select. The Open Software Foundation (OSF) hopes to end the
dilemma by choosing one as the X Window user-interface standard. A number of
vendors have proposed their solutions to OSF, including DRI with X/GEM.
Whatever the outcome, end users could benefit by having a single style of user
interface across the entire range of supported applications and computers.
Another beneficiary is programmers, who will then be able to write portable
applications for a consistent environment with few worries about hardware and
operating system differences.
Now let's consider some specific issues that arise when you want to make your
graphics application portable.


Dealing with Graphics Devices


Windowing systems such as Presentation Manager and X Window provide a set of
functions that draw to the screen as independently of the actual physical
device as possible. These functions also return information about the device
so that the program can deal with specifics and idiosyncrasies consistently.
In X/GEM, the VDI handles graphics-device portability. The VDI is built around
a conceptual graphics model called a workstation (not to be confused with
those powerful high-end personal computers going by the same name). X/GEM VDI
workstations are specific graphical devices available to an application. Thus,
the display and the printer are both VDI workstations. When an application
opens a workstation, the appropriate graphical device driver is loaded. The
device driver translates high-level VDI commands into the low-level actions
necessary to produce graphical information on the target device or to receive
information from it.
This approach is similar to that taken by other windowing systems. In Windows
the approximate equivalent is called a display context, on the Macintosh it is
called a grafPort, and in Xlib it is called a graphics context.
X/GEM furnishes multiple classes of workstations. These include screens,
plotters, printers, cameras, scanners, and metafiles (files containing sets of
graphic commands). In general, the application can send a given set of
commands to the screen and then issue the same set of commands to the printer
or metafile device to create a more enduring copy of the output. Graphic
systems such as the Macintosh and X don't have the same kind of generality
built into their graphics models, choosing instead to support primarily the
screen display device.
One of the most important issues in writing portable graphics applications
concerns the concept of uniform rendering. Graphical devices span a wide range
of addressability and resolution. Many displays for PCs --the Macintosh being
a notable exception --use pixels that are not square. This means that a square
box with ten units of measurement in height and width will show up as a
rectangle. The ratio of height to width is called the aspect ratio of the
device.
Microsoft Windows handles the aspect ratio by providing some half-dozen
functions to scale coordinates for a display context. The application
programmer selects from coordinate systems that are metric, English,
arbitrary, or raster dependent. Each coordinate is scaled automatically at
output time.
In contrast, the X/GEM VDI uses primarily raster coordinates. This means the
application must do its own scaling. The GEM designers felt that the
application is best able to decide how to do the scaling in the most efficient
manner. For example, some CAD applications need to scale in floating point,
whereas many publishing applications can use integer or fixed point scaling.
The X/GEM VDI provides aspect ratio information so that the application can
handle coordinate translation in the most appropriate manner. The height and
width of a pixel (in microns) is returned by the VDI call that opens the
workstation. Listing One, page 92, illustrates how this is done with a simple
integer scaling function.


Speaking in Foreign Tongues


As new markets open, it becomes increasingly important to be able to translate
applications into other languages. Doing this easily requires a strategy other
than the traditional use of embedded text literals in the source code.
With X/GEM, foreign language portability can be achieved by placing all the
application's text messages in resource files. One of the system's most
important portability feature is the AES function for object and resource
handling. The GEM Programmer's Tool to build and edit these resources is
called the resource construction set or RCS.
RCS provides tools for constructing and displaying menus, windows, and dialog
boxes in files independent of the application code. Each object has a position
and size, along with type and data information. The objects are organized
hierarchically, thus allowing them to be defined, displayed, and manipulated
relative to each other. Examples include icons, text, boxed text, boxes,
editable text fields, and programmer-defined objects.
The collection of objects for a given application is grouped together in a
resource file. When a GEM application initializes itself at run time, it calls
the AES to load the resource file. The resource file contains all the text
that the user sees, separate from the executable code. This allows a
programmer to prepare a resource file for each supported foreign language
without recompiling the application.
One warning: If you intend to translate menu and dialog items, leave plenty of
extra visual space in the enclosing boxes for the translated messages. For
example, translating from English to German usually requires between 50 and
100 percent more characters per message.
The RCS also allows an application designer to get quick feedback about the
user interface, even before a large amount of code is written. Instead of
using the application's programming language, you use the interactive RCS
tools to specify exactly what gets placed where. Thus, for example, you can
use RCS to place a string object in a dialog box, change the text as
necessary, reposition and tweak it, and then label the object for access by
the application. The difference between constructing a moderately complex
dialog with the RCS versus a compiled language is incredible.
The RCS produces a resource file. It can also produce a file format suitable
for compiling directly into the program. You might do this for a variety of
reasons, such as protecting the resource information or reducing distribution
complexity by eliminating the resource file altogether. The penalty for doing
so is loss of portability to some extent.
The resource file shown in Listing Two, page 92, contains a single dialog box
with two button objects. Without going into detail about the construction of a
resource, there are a few things to note. First, it took less than three
minutes from start to finish to create the dialog. Figure 2 shows the RCS
display during the design. Next, the structure of the resource file is well
defined and therefore described by the first few defines, which give critical
array sizes. Finally, the RCS generates an include file with the contents:
 #define SAMPLDLG 0 /* TREE*/
 #define BTNCANCL 4 /* OBJECT in TREE #0 */
 #define BTNOK 3 /* OBJECT in TREE #0 */
The #defines give indices into the resource array so that the application can
refer to specific objects. These data objects change values according to user
input, thus allowing the application to determine exactly what the user
entered.


What about the OS?



The capabilities and characteristics of various operating systems (DOS vs
FlexOS vs Unix vs OS/2) also influence strategy in writing portable
applications. Two critical areas are file and event handling. Many GEM
applications call directly the native operating system's file-handling
primitives. This is done primarily to avoid the overhead of C run-time
libraries. Another reason for avoiding the run-time libraries is that they
usually build in bulky support to handle screen output, such as printf( ). In
general, applications running under a windowing manager such as GEM and
Windows cannot use the operating system's screen and keyboard I/O services at
all; all I/O must go through the environment manager.
An application targeted toward several operating systems needs a file system
interface that can be readily ported to any of them. One way to deal with this
problem is to use the standard routines (that is, fopen (), fclose (), fread
(),fwrite (), and so on) provided with the C run-time library, and pay the
price in overhead for portability. Another way is to emulate the file system
calls of one operating system on another. For example, DRI furnishes a DOS
emulation run-time library for the X/GEM product on FlexOS.
The AES provides a degree of operating system independence by providing a
file-selector mechanism enabling an application to solicit a file
specification from the end user. This lets the user navigate through the file
system and change directories as necessary to find or specify the desired
file. DRI plans to tailor the mechanism for each supported operating system in
order to hide the subtleties of file system navigation from the user.
The AES also provides an event driven input model with which applications can
work effectively in a broad range of environments. The application waits for
multiple events, where an event could be a keypress, a mouse button press, a
timer tick, or certain kinds of messages. These messages include notifications
that the user has requested a menu item, or changed the size or position of a
window. A message can also be information from other applications. The point
is that the application doesn't have to actively poll the environment looking
for something to happen. Instead it suspends activity until a specified event
occurs. This greatly enhances portability to multitasking environments; the
application becomes a "good neighbor" in the computing environment by not
wasting unnecessary CPU resources.
The X/GEM event structure in Listing Three, page 92, looks complicated, but
it's easy to understand. The program fills in the e_flags field with a bitmask
containing those events that it wants to wait for. When a bit for a specific
event is turned on, the structure element controlling that event must be
filled in. In the example shown in Listing Four, page 92, the program waits
for one of two things to happen: either a keypress or an elapsed time of ten
seconds. The application can also wait for a button press, a message, or for
the mouse to enter or leave one of two rectangular regions on the screen.
In Windows and in X, the application notifies the system that it will wait for
a number of different types of messages. This is roughly equivalent to the
evnt_event() mechanism of GEM, except that events are serialized because the
application can only receive a single message at a time. The Macintosh has a
much cruder mechanism, where it cycles through a loop, polling certain
functions to see if anything has happened. This works fine when a single
application is running on the machine, but it causes conflicts and excessive
overhead when multiple applications are polling at the same time.


Moving Among Different Processors


The wide differences among microprocessor architectures demand a careful
strategy where portability is in the picture. The main requirements are
programming in a portable language such as C, along with careful attention to,
and independence of, byte ordering, integer size, and correct and consistent
use of pointers. These and other concerns were discussed in Greg Black-ham's
article "Building Software for Portability" (DDJ, December 1988).
With respect to X/GEM programs, there is one additional set of portability
issues. In order to minimize the amount of code in GEM applications, the X/
GEM bindings are defined to use the mixed model programming applicable to the
segmented Intel architectures. To simplify, an application can use either long
pointers --access to the entire address space at the expense of code
compactness --or short pointers, which decrease executable program size but
can only address up to 64K from a fixed location. GEM applications are mostly
in small models (short pointers) with certain kinds of pointers being kept in
long format. Although this is an extra burden to the application programmer,
it can be readily handled by the strict parameter type checking available in
most up-to-date C compilers.
Writing portable applications takes a different design philosophy and tool set
than developing for a single system. There's nothing mysterious about the
process; it's just different, requiring observance of some common sense rules.
A well-crafted user-interface subsystem such as X/GEM, which was specifically
created for multiplatform, multi-device portability, can also greatly ease the
headaches of moving software from one environment to another that differs
radically.

_WRITING PORTABLE APPLICATIONS WITH X/GEM_
by Bill Fitler


[LISTING ONE]

 /* GEM uses 16 bit quantities for raster
 coordinates */
 typedef short int WORD;
 WORD pxl_width, pxl_height;
 WORD work_in[11], ws_handle, work_out[57];
 ...
 /* Open the workstation after filling in initial
 defaults into work_in[] */
 if( v_opnwk(work_in,&ws_handle,work_out) == FAILURE )
 { /* Handle fatal error */ }
 /* After successful open, ws_handle identifies the
 desired workstation and work_out contains 57
 units of information about the device,
 including the width and height of each pixel in
 microns */
 pxl_width = work_out[3];
 pxl_height = work_out[4];
 ...
 /* Returns a value in y units that is scaled to x
 units */
 WORD scale_y(WORD raw_y)
 {
 return raw_y * pxl_width / pxl_height;
 }
 ...
 /* Draw a square on the screen that is "x_units"
 wide and looks square */
 VOID draw_square(WORD x, WORD y, WORD x_units)
 {
 WORD xy[10];
 xy[0] = x; xy[1] = y;
 xy[2] = x+x_units; xy[3] = y;
 xy[4] = x+x_units; xy[5] = y+scale_y(x_units);
 xy[6] = x; xy[7] = y+scale_y(x_units);
 xy[8] = x; xy[9] = y;
 v_pline(ws_handle,5,xy);
 }







[LISTING TWO]

SAMPLE SOURCE OUTPUT FROM RESOURCE CONSTRUCTION SET


 #define T0OBJ 0
 #define FREEBB 1
 #define FREEIMG 1
 #define FREESTR 4

 BYTE *rs_strings[] = {
 "This is a Sample Dialog",
 "with an image and two buttons.",
 "OK",
 "Cancel"};

 WORD IMAG0[] = {
 0x7FF, 0xFFFF, 0xFF80, 0xC00,
 0x0, 0xC0, 0x183F, 0xF03F,
 0xF060, 0x187F, 0xF860, 0x1860,
 0x187F, 0xF860, 0x1860, 0x187F,
 0xF860, 0x1860, 0x187F, 0xF860,
 0x1860, 0x187F, 0xF860, 0x1860,
 0x187F, 0xF860, 0x1860, 0x187F,
 0xF860, 0x1860, 0x187F, 0xF860,
 0x1860, 0x187F, 0xF860, 0x1860,
 0x187F, 0xF860, 0x1860, 0x187F,
 0xF860, 0x1860, 0x183F, 0xF03F,
 0xF060, 0xC00, 0x0, 0xC0,
 0x7FF, 0xFFFF, 0xFF80, 0x0,
 0x0, 0x0, 0x3F30, 0xC787,
 0x8FE0, 0xC39, 0xCCCC, 0xCC00,
 0xC36, 0xCFCC, 0xF80, 0xC30,
 0xCCCD, 0xCC00, 0x3F30, 0xCCC7,
 0xCFE0, 0x0, 0x0, 0x0};

 LONG rs_frstr[] = {
 0};

 BITBLK rs_bitblk[] = {
 0L, 6, 24, 0, 0, 1};

 LONG rs_frimg[] = {
 0};

 ICONBLK rs_iconblk[] = {
 0};

 TEDINFO rs_tedinfo[] = {
 0};

 OBJECT rs_object[] = {
 -1, 1, 5, G_BOX, LASTOB, SHADOWED, 0x21100L, 0,0, 43,9,

 2, -1, -1, G_STRING, NONE, NORMAL, 0x0L, 10,4, 23,1,
 3, -1, -1, G_STRING, NONE, NORMAL, 0x1L, 7,5, 30,1,
 4, -1, -1, G_BUTTON, SELECTABLE, NORMAL, 0x2L, 10,7,
 8,1,
 5, -1, -1, G_BUTTON, SELECTABLE, NORMAL, 0x3L, 24,7,
 8,1,
 0, -1, -1, G_IMAGE, LASTOB, NORMAL, 0x0L, 19,1, 6,3};

 LONG rs_trindex[] = {
 0L};

 struct foobar {
 WORD dummy;
 WORD *image;
 } rs_imdope[] = {
 0, &IMAG0[0]};

 #define NUM_STRINGS 4
 #define NUM_FRSTR 0
 #define NUM_IMAGES 1
 #define NUM_BB 1
 #define NUM_FRIMG 0
 #define NUM_IB 0
 #define NUM_TI 0
 #define NUM_OBS 6
 #define NUM_TREE 1

 BYTE pname[] = "SAMPLE.RSC";






[LISTING THREE]

/* mevent.h - define structure for GEM evnt_event() */

 typedef struct mevent
 {
 UWORD e_flags; /* events to wait on */
 UWORD e_bclk; /* num button clicks */
 UWORD e_bmsk; /* which mouse buttons */
 UWORD e_bst; /* button up or down */
 UWORD e_m1flags; /* return on entry or exit */
 GRECT e_m1; /* rect 1 x,y,width,height */
 UWORD e_m2flags; /* return on entry or exit */
 GRECT e_m2; /* rect 2 x,y,width,height */
 LONG e_mepbuf; /* message buffer pointer */
 ULONG e_time; /* time to wait (ms) */
 WORD e_mx; /* return x */
 WORD e_my; /* return y */
 UWORD e_mb; /* return which buttons */
 UWORD e_ks; /* return kb state */
 UWORD e_kr; /* return kb code */
 UWORD e_br; /* return num button clicks */
 CHAR e_reserved[24]; /* for system use */
 } MEVENT;


 WORD evnt_event( MEVENT * );







[LISTING FOUR]

 MEVENT ev_str;
 ev_str.e_flags = E_KEYBD E_TIMER;
 ev_str.e_time = 10000L; /* in milliseconds */
 ret_flags = evnt_event( &ev_str );
 if( ret_flags & E_KEYBD )
 {
 /* The user pressed a key:
 Key code of key pressed returned in ev_str.e_kr;
 State of shift and control keys in ev_str.e_ks */
 }
 if( ret_flags & E_TIMER )
 {
 /* 10,000 milliseconds have elapsed since the call
 */
 }





































March, 1989
NETWORK WINDOWING USING THE X WINDOW SYSTEM


For networks, X is the biggest game in town




Jim Gettys


Jim Gettys was an original member of MIT's Project Athena development team and
was part of the research project that eventually led to X. Jim is now a
consulting engineer with the DEC Cambridge Research Lab. and can be reached at
1 Kendall Square, Bldg. 700, Cambridge, MA, 02139.


The X Window system is a device independent, multitasking windowing, and
graphics system designed to operate across heterogeneous networks. As such, X
supports high-performance graphics and window-management mechanisms to provide
a hierarchy of overlapping, resizable windows. The X system is based upon a
client/server architecture whereby the client (that is, the application
program) requests that the server (the program that controls the
user-interface at the workstation) be responsible for drawing text, windows,
and other objects. Through this client/server architecture, applications can
run on any machine in the network and can be accessed by any workstation or PC
running on an X server.
The X Window system was originally designed at the Massachusetts Institute of
Technology (MIT) to provide a generic, network windowing environment for
dissimilar bit-mapped workstations. The system makes it possible for MIT to
network its collection of incompatible computers, which have accumulated at
the institute over the years. X, which is now a de facto standard and is
publicly available, provides efficient workgroup computing, connecting the
dissimilar computing environments that exist in many situations.
The X Window system includes the Xlib graphics subroutine library, the X
network protocol (which handles transmission between client applications and
server processes running on remote workstations), an X toolkit (which
programmers use to build graphical-user interfaces), and several window
managers. The window-manager program is distinct from the base window-system
server and provides part of the user interface for manipulating existing
applications on the screen. The application itself, however, in combination
with an X toolkit, actually specifies the bulk of the window interface and
defines the application's "look and feel." X contains mechanisms for
implementing many interface styles and, unlike some interfaces (such as the
Macintosh), does not mandate a single style. Because X is supported by
virtually every major workstation vendor and more than 24 organizations, any
application residing on a multi-vendor network and adhering to the de facto X
standard for windowing on high-performance bit-mapped workstations can be
accessed by all X-based workstations. X defines an open-systems architecture
--one that is independent of devices, networks and operating systems --that
accommodates a range of workstations. X servers have been implemented on
workstations and PCs from Digital Equipment Corp., Apollo Computer, Sun
Microsystems, and Apple Computer, as well as MS-DOS and OS/2-based PCs.


Network Transparency


The X Window system, written in C, provides a network transparent
client/server architecture that spares you from coding communications services
between connected clients and servers. Clients establish connections with any
number of servers. These servers handle graphics and windowing functions
(called X server processes) and receive requests from clients in the form of
Xlib graphics library calls. Fast asynchronous communication between clients
and servers is performed by the X Network Protocol.
X applications and toolkits usually interface to the Xlib subroutine library
on the host. Xlib in turn converts the parameters passed to the procedural
interface into the X network protocol format, and translates messages from the
server into return values for the application. Xlib also provides a set of
utility routines needed by most applications.
In addition to Xlib, applications can interface to a variety of programming
libraries as needed. These include the X Toolkit (used to build graphical-user
interfaces), industry-standard libraries (such as GKS and PHIGS) that can be
layered on top of Xlib, and extension libraries that provide programming
interfaces to X server extensions such as 3-D graphics, Display PostScript,
and imaging.


Common X Applications


X applications allow you in effect, to reach out across a network via a
consistent and intuitive graphical-user interface to access remote
applications and high-performance resources anywhere on the network. This
network transparency lets you intermix operating systems in workgroups and
access applications regardless of the system those applications run on. For
example, you can display in separate windows on a PC applications running on a
VAX/VMS system, a Unix workstation from Digital, Sun Microsystems,
Hewlett-Packard, and IBM computers. Through X, you can access multiple
applications and display and control them within hierarchical-style windows on
this workstation. Applications can be running either on the workstation or on
remote hosts. The X Window system's network transparency makes remote
applications appear as if they were running on your workstation.
Although an X application can be accessed by any bit-mapped graphics
workstation on the network, the program actually executes on the whatever
available computer is best suited to process the program. For example, a PC
that lacks the performance and memory to run a weather prediction program can
access a Cray supercomputer running the program while you interact with the
application through a window on the PC. Again, X masks the differences in
operating systems.
Applications can be designed with mouse-driven front ends that allow you to
scan and navigate through multiple databases scattered throughout a network. A
stockbroker, for instance, can review and compare multiple documents presented
on his screen at the same time, each containing stock information from various
news wires. The broker can "cut and paste" information of the same data
formats from these windows into another window, and electronically mail the
contents of the window immediately to a client with whom he is talking on the
phone. The client, after reviewing the information from a desktop terminal,
can place an order to purchase stock while the broker fills in an order form
in another window and sends it to the client for confirmation. The stock
information can be coming from news wires running on one system that knows
nothing about the mail system, and nothing about the order form application
and its host.


Software Portability


In addition to providing an architecture for distributed applications, X also
simplifies the porting of software. This should be music to your ears if you
traditionally have had to customize an application to match each hardware and
software environment that you want your application to run on. Through X, one
common application interface reaches all platforms because X is independent of
machine architecture, operating system, and resolution and display
characteristics. Application programs written to support X can often run on
another vendor's system simply through recompilation.
The portability of X has been amply demonstrated by the X Testing Consortium,
a group of vendors jointly developing tests to help firms validate their X
implementations. To date, the test software runs on all the computer
architectures of contributing companies with virtually no changes. Developers
have had prerelease test software (containing more than 200 programs) up and
running in one day.
Because X is device independent, you do not need to rewrite, recompile, or
relink an application for each new hardware display. Applications can be
written to be independent of monochrome and color displays or displays with
varying resolution and other characteristics. Furthermore, every graphics
function defined by the system will work on virtually every supported display.
If graphics functions were not made to work on all displays, "inquire"
operations (like the GKS-graphics-standard inquire) could be used to determine
the set of implemented functions for a particular display at run-time.
(However, using GKS inquire operations would require run-time analysis for
every application, adding overhead and producing inconsistent user
interfaces.)


X Extensions


Vendors implementing their own versions of X can extend the system over time.
The most recent version of X, Version 11, provides hooks to extend X to
support functions such as the Display PostScript imaging model, PEX 3-D
graphics (see accompanying box), and standard compound-document interchanges.
X11 also adds a graphics state for improved performance and defines precise
semantics for output routines. The X11 tape, publicly available for a nominal
fee from MIT, includes a sample X server, the Xlib library of X routines, the
X Toolkit, several window managers, and other contributed software.
Unlike other windowing systems, X has a basic philosophy of providing only
low-level mechanisms and defers policy decisions to developers. X will create
a window, but the application must tell it whether the window should have a
particular border or color. X gives low-level "raw" functionality in order to
provide a simple, clean system on which to build. If policy decisions were
made at a low level, the system would not grow so easily or allow for future
advances in user-interface technology.


X User Interface Toolkits


As previously stated, X provides the mechanisms to move, resize, and
manipulate windows but does not dictate the actual appearance of the windows.
These are defined by various developers. One such toolkit is the public-domain
Xtk toolkit included with the MIT distribution tape. Another is Digital's
DECwindows X User Interface (XUI), which is a collection of user-interface
components called "widgets," which are analogous to objects. These widgets
include scroll bars, pop-up menus, window borders, and dialog boxes. Digital's
widget library is built on top of the lower-level "intrinsics" found in the X
Toolkit. Intrinsics are the basic set of rules governing widgets: how the
widgets are created and destroyed, how they receive input events, how they are
stored in a resource file to be initiated at runtime, and other
characteristics. You can build interfaces on their applications that maintain
a high degree of portability and consistency with the X standard using
Digital's XUI.
To build graphical-user interfaces, you design any number of custom widgets
from the X Toolkit, or use sample widgets from the X Toolkit as well as
widgets provided or sold separately by vendors. Because all widgets sit on top
of the same foundation (the intrinsics), an application and its widgets can be
ported from one X-based computer to another. Each computer provides a portable
foundation, yet each application is customized and differentiated by its own
"look and feel." Consistent graphical-user interfaces enable applications to
have similar looks and feels, function similarly, and operate intuitively
through mouse-driven graphical icons and pull-down menus. This simplifies the
learning process for users.
From the three-level stack of programming interfaces --widgets, intrinsics,
and Xlib --an application calls on any one interface, or any combination of
interfaces as required by the program. If you are developing a spreadsheet,
for instance, your program might call on intrinsics to customize a widget that
displays cells in a spreadsheet, and access Xlib routines to draw graphics on
the screen. Although it is possible for applications to directly access the
server via the X protocol, you should use the higher-level Xlib graphics
routines to manage communication.
The X protocol defines data structures used to transmit requests between
clients and servers. X transmission is asynchronous. This enables requests to
be sent without waiting for the completion of previous requests. Pipelining
techniques in both the server and Xlib speed the processing of requests. Any
requests depending on the completion of other requests are blocked, pending
execution of those other requests. Errors are also generated asynchronously,
and clients must be prepared to receive error messages at arbitrary times.

In general, the X protocol also describes connections between clients and
servers, windows (which allow interaction between you and the application),
events (which notify the application of mouse and keyboard actions and provide
a way to control communication between multiple applications), and graphics
routines (which allow an application to draw information on a display). These
are described later.


Connections


Because X is network and operating system independent, applications can run on
any machine in the network. The X protocol defines data structures used to
transmit requests between clients and servers. Applications do not generate
protocol requests themselves. Instead, applications call Xlib and other
layered libraries. X uses asynchronous stream-based interprocess communication
instead of the traditional procedure call or kernel-call interface. This
asynchronous communication improves network speed by enabling requests to be
sent without needing to wait for the completion of previous requests. Nearly
any form of reliable data transport may be used. Current implantations include
TCP/IP and DECnet.
Pipelining techniques in the server and Xlib help accelerate processing of
requests. Some X requests, however, have return values (state queries, for
example) that depend on the completion of previous requests. The X protocol
will block any further requests until the server has generated a reply and
sent it back to the client. Errors are generated asynchronously so clients
must be prepared to receive error replies at arbitrary times after the
offending requests.
A connection (that is, the communication path between the server and client
program) can exist between processes on the same machine or on different
machines. A client program usually has one connection to a server over which
requests and events are sent. When processes are on the same machine, the X
protocol is often sent using shared memory or other local transport facilities
of the system, rather than TCP or DECnet.
To interact with you, a client must first open a connection with an X Server
using a common transport mechanism. (DECwindows, for example, uses DECnet/OSI
or TCP/IP). The client passes version and authorization protocol information
in a packet to the server along with a code that indicates the byte order used
by the client. If the byte order differs from the server's machine
architecture, the server will use the byte order code to swap the bytes of
incoming requests. A swapping takes place, for example, between a server
running on a Macintosh and a client application running on a VAX processor.
If a request to open a connection is successful, a reply is sent back to the
client. This reply contains information about the server and the associated
display hardware including display resolution, physical dimensions,
color-handling abilities, and a vendor identification string.


Windows


Once a connection is made, you can interact with multiple applications that
are displayed within windows employing a window manager, which is simply
another client program, that helps you manipulate windows on the screen. From
the programmer's perspective, windows are hierarchical and can be created
inside other windows to any depth necessary for an application.
Each screen has a root window that displays a background color or pattern and
serves as the root of the window tree for that display. Windows can be
displayed fully on the screen, partially, or completely hidden. To display a
window, the client sends the server a Map-Window request. Graphics output to a
window is clipped to the boundaries of the window. A window, therefore,
becomes a virtual graphics terminal for an application, allowing multiple
applications to share a screen and not overwrite another application's output.
Each window has a height and width and Z position that indicates its position
within a stack of other windows. In addition, windows carry other attributes
that identify their location on the screen, their mapped state, and their
relationship to parent and sibling windows. The border pixel attribute is used
to draw a border around a window. Through a background pixel and pixmap
attribute, an application can specify either a single pixel value or a
complete pixmap (a rectangular array of pixels in main memory) as the window
background. Through this attribute, a server can redraw a window's background
color itself without sending a request to an application.
More Details.
An application that creates a window can specify bit and window "gravity" to
the server to indicate which pixels should be retained when a window is
resized, or how children windows should be positioned when the parent window
is resized. For example, a text application might specify "NorthWest-Gravity"
to indicate that the upper-left information should be preserved when the
window is reduced to size.
Many Xlib routines are used primarily by a window manager or toolkit rather
than by applications. Typical routines include changing the parent of a
window, grabbing the pointing device or keyboard, altering event dispatching
and processing, changing the keyboard encoding, determining the resident color
maps, and modifying the list of hosts that have access to the server.
All pixels in X have uninterpreted color values, although the application can
allocate and define a color map to gain control of the mapping between pixel
values and colors displayed on screen. X encourages sharing of color maps
between applications. Pixel values can be allocated as read only, and shared
in a color map (optionally by name), or as read/write and exclusive in a color
map. Applications that use low-level X routines are expected to query the
hardware capabilities at connection set-up time and adjust their usage
accordingly.
X graphics routines can be directed to a window or to an arbitrary pixmap.
Pixmaps and windows are referred to as "drawables" and all X drawing
operations are passed drawable as a parameter. Instead of passing all
parameters that describe a drawing operation to the server on each graphics
request, the server keeps state in a data structure called a graphics context
(GC). The GC is passed as an argument to each graphics call and includes
information about the foreground and background colors, line widths and
styles, polygon fill rule, stipple patterns, text fonts, and a client-supplied
clipping region. Applications can create more than one GC to alternate quickly
between states on sequential output calls.


Events


X applications are event driven with events being sent to an application from
a number of sources, including the X server and X toolkit, as well as other
applications. Events are generated by the X server when you type on the
keyboard or move the mouse or other pointing device. Some event types are
generated as side effects of client requests. Each event includes a time
stamp, a bitmask indicating the up/down state of all modifier keys and mouse
buttons just before the event, the window the mouse is in, and details about
the change the event describes.
An application is notified when the pointing device or cursor enters or leaves
a window. A single window is globally designated as the "input focus." This
window receives all keyboard input specified by the event until the input
focus is set to a different window. An event is generated when a window gains
or loses input focus. In X, applications are expected to regenerate, on
request, any information displayed in windows. When a window changes size or
becomes visible, the server may need to tell the application which parts of
the window to redraw. This triggers an event. Some X implementations may
invoke backing store and save orders to reduce repainting, but applications
must still be able to repaint a window on request.


Graphics Performance


Graphics operations in X are designed to be simple and fast. They are
relatively low level compared to PostScript, PHIGS, or GKS, but are still
well-suited to create high-performance, visually sophisticated applications.
Tasks requiring a higher-level, graphics-oriented interface can use layered
graphics libraries or intermix calls to the layered libraries with basic X
graphics functions.
The Xlib contains about 300 routines that either map directly to X Protocol
requests or provide utility functions to the client. Xlib routines allow a
client application to create, destroy, manipulate, and configure windows.
There are also routines for lines, polygons, arcs, text, block pixel
transfers, stipple and tile filling, and color-map manipulation. Routines such
as Polyline and PolyRectangle perform multiple operations based on a list of
points.
Operating on an array of objects is more efficient than making multiple
graphics calls due to the X requests overhead. The protocol to draw one or
more rectangles is PolyRectangle, which takes a drawable (a pixmap or window),
a graphics context, and a list of rectangles as parameters.
For example, X shows that graphics performance over an Ethernet network is
excellent, and usually functions at the speed of the display device (often
higher when the application is running remotely rather than locally!).
Although the semantics of server operations are tightly connected to the X
protocol, a fair degree of freedom exists in the actual design and
implementation of the server itself. The quality of the server implementation
is one way vendors can add value to their competing X offerings.
The MIT sample server (on the MIT distribution tape) consists of a section of
highly portable code, and a section of device-dependent code. The sample
server was designed to make device-independent code as large as possible, thus
simplifying implementation at the expense of performance. Reimplementing the
server to be entirely device dependent may provide the best performance, but
would require a major effort to support each new workstation product.


Conclusion


Over time, extensions to support 3-D graphics, imaging, and even live video
will be added to both the X architecture and to development tools with the
goal of providing added functionality, but not at the expense of
compatibility. With this in mind, software developers should consider the
following criteria when evaluating a specific implementation of X: quality and
robustness of code, performance between clients running X applications and
remote workstations running X servers, the vendor's X development environment,
and the level of difficulty required to integrate a software developer's own
extensions and software into the X environment.


PEX 3-D Graphics for X


While the PEX project is supported by a number of companies and organizations
-- including DEC, Tektronic, Hewlett-Packard, Apollo, Sun Microsystems, and
the Open Systems Foundation -- the actual implementation work will be done by
Sun under the direction of Robert Scheifler, director of the X Consortium. Sun
will develop a public implementation of PEX and provide the full network and
graphics code necessary to generate 3-D graphics on an X display.
Scheifler, who was the principal architect of X, says "PEX adds a significant
new functionality to X." He went on to tell DDJ that the PEX project is
especially significant, and personally gratifying, because "it is another
indication of how well the industry can pull together when the right
technology is recognized." Scheifler added that "interest in PHIGS throughout
the world has been heating up recently, especially because it is about to
become an official ISO standard. It is the right thing at the right time."
The preliminary release of the software will be in mid-1989, with public
release (including documentation) scheduled for late 1990. The PEX
implementation will become part of the MIT X Consortium software release and
will be available at distribution cost with no licensing restrictions.
On a related topic Scheifler indicated that similar projects may be announced
by the X Consortium sometime in the future, particularly projects targeted at
object-oriented programming. "The consensus is that object-oriented
programming is fundamental to user-interface building," he said. "We are
looking into application development environments and the next generation of
toolkits of which object-oriented languages are a key ingredient."

--eds.
































































March, 1989
EXTENDED DIRECTORY SEARCHES USING C++


Here's what you can do when DOS's DIR just won't do the job




John M. Dlugosz


John is a programmer with Conductor Software and can be reached at 9208 W.
Royal Lane, Irving, TX 75063 or on CompuServe at 74066, 3717.


Any decent program that deals with files must have some way to parse and
construct file names. Consider, for example, a simple non-interactive program
that accepts input and output file names on the command line. A simple program
would accept a pair of names and blindly use them. A more robust program would
check for validity, check for the extension and supply a default if needed,
and generate the output name from the input name if the second parameter is
missing.
In this article I'm presenting a program called MATCH.CPP that lets you use
C++ to extract drive, path, basename, and extension components from an MS-DOS
filename, and create a filename from component parts. This program makes use
of a C++ class called dir_scanner that is much more convenient than similar
methods traditionally used in C. MATCH.CPP, when used with the other files
included in this article (DIRSCAN.HPP, DIRSCAN.CPP, and SCANHELP.ASM) and with
your standard library, form a complete program. The latter half of main( )
shows the use of a dir_scanner, as well as a suggestion of how you can send a
dir_data to a text file.


Dirpath


The first class, called "dirpath," is defined in Listing One (DIRPATH.CPP),
page 99, as a structure. In C++ a structure (struct) is a class. The only
difference between the two is that class members are private unless stated
otherwise, while struct has all public members. Notice that the members of the
structure are character arrays to hold the various pieces of a file name. You
can read their contents or change them, just like any ordinary structure. The
magic is in the member functions: The << operator parses a string into the
structure, and >> concatenates all the pieces together into a full name.
In addition to the two operators, a pair of constructors is provided as well.
If a dirpath is initialized with a single string, it is parsed into the
structure (it simply calls operator<< inline). Alternatively, you can
initialize a dirpath with four strings and they are copied into the four
fields. Listing Two, page 99, illustrates the usage of the dirpath.
Now let's take a close look at the guts of dirpath. Both constructors are
inline functions, and are stated in the header. Listing Three (DIRPATH.CPP),
page 99, shows the implementation of << and >>. Defining the << operator is
just like defining any other function whose name is operator<<. The operator
is used between two items, but only one parameter is given. The left argument
is this, which is automatically present in any member function. The this
parameter is always a pointer to the class of which the function is a member.
You rarely need to refer to the name this, because the members may be referred
to directly--that is, you could say this->drive, but drive alone will do.
Dirpath is straightforward. It checks to see if a drive letter is present, and
places it into drive if found. Next, the string is scanned for the last slash
(subdirectory separator) and dot. The pieces found go their separate ways, via
the copy( ) function. Dirpath copies a string but will not overflow the
destination.
The operator >> reconstructs a name from the components. It is rather simple
but is loaded with failsafes. Even though the parse will have put a colon
after the drive name, a backslash after the path, and begun the extension with
a dot, you might have replaced some of the fields and not been as neat. The
operator >> will generate a valid filename.


Dirscan


Programs that deal with filenames will probably want to deal with generic
filenames. In DOS, the wildcard characters * (asterisk) and ? (question mark)
are used to abbreviate a name or to select a group of similar names. The use
of * is very weak, however; for example, the * must be the last character in
the base or extension. It would be much nicer if you could have as many *s as
you like and put them anywhere in the name.
This leads us to a class called dir_scanner. In C you would probably use
functions like dirfirst( ) and dirnext( ) or perhaps opendir( ) and readdir(
). These functions are difficult and clumsy to use and have subtle differences
among compilers. In C++ an iterator class makes things much easier. Take a
look at Listing Four (DIRSCAN.HPP), page 99.
The structure for DOS's directory entries is described by the class dir_data.
Notice that this is a class and not a struct. The reserved area is private.
Other than that, dir_data behaves like an ordinary struct in that the members
may be freely used, like in C. Because the class has no member functions and
no friends, the reserved area cannot be accessed anywhere in the program. In C
you could say that this area is hands-off; in C++ you can mean it.
When you want a directory listing, you declare a variable of type dir_scanner.
The constructor takes the wild file specification and the mode. If the second
parameter is missing, zero is taken as a default.
The dir_scanner class is an iterator. Iterators come in different forms,
reflecting the way they are intended to be used. The simplest case is a single
function that returns the next value each time it is called, and some sentinel
value or error code when no elements remain. (There might be an infinite
number of elements, in which case you do not have to worry about the end of
the series.) A two-step iterator, on the other hand, has two steps. One
function advances to the next element and reports on the success or failure of
the operation. Another function fetches the current element. A two-step
iterator is popular for two reasons: You might refer to the current element
several times before advancing, and it may be easier to structure the loop if
you know in advance that an element is present. ( Listing Eight [MATCH.CCP],
page 100, demonstrates the use of class dir_scanner.)
Both parts of the iterator are not ordinary functions. The fetch function is
operator( ). This is a popular choice for iterators or other classes where one
function is dominant. Overloading the function call operator may look strange
at first but is quite handy. This function may be defined with any number of
arguments (zero, in this case). If arguments are present, they are placed
between the ( ) just like in a function call. Defining the function, the name
operator( ) is given followed by an argument list. This function returns an
item of type dir_data by reference. Other languages (like Pascal) let you pass
by reference, but C++ lets you return by reference as well. Here, the function
is inline, and return by reference is used for efficiency.
The function to advance and test is not an ordinary function either. This is
an overloading of the int operator. Casting a dir_scanner to an int causes
this function to be called. This result is handy because you can use a
dir_scanner in a while loop. If d is a dir-scanner, while(d) {... causes d to
be implicitly casted to an int. The construct (int)d is familiar to C
programmers, and int(d) is equivalent and preferred in C++. Both would cause
operator int to be called.
If the use of the operators is confusing, you might want to imagine them as
functions bool next( ) and void fetch( ).


Implementation


The dir_scanner must somehow find out which files are on the disk. The class
dir_scanner is portable and easy to use. It is in the implementation of the
class that you can use all the tricks and environment-specific knowledge. The
implementation in Listing Five (DIRSCAN.CPP), page 99, is for MS-DOS.
Most C libraries include functions to read the directory, they are, however,
all different. Most also have a way to call DOS functions, but, again,
differences in implementations make calling DOS functions more trouble than
it's worth. The functions are quite simple, so it is easier to call them in
assembly language than it is to figure out how to do it in C++ for any given
implementation.
In DOS, the "find first" and "find next" call uses a data area that must be
preserved. Because it is allowed to have more then one dir_scanner going at
the same time, each instance must have its own data -- another reason not to
use the functions that came with the library. In the case of Zortech, for
example, the supplied functions use an internal static buffer. Listing Six
(SCANHELP.ASM), page 100, contains the two assembly language functions.
The constructor for dir_scanner calls dirscan_findfirst( ), and the advance
and test function calls dirscan_findnext( ). In Listing Five, this accounts
for about ten lines. There is much more here then necessary for simply
providing a shell for DOS's directory searching functions.


Wildcard Names


Under DOS's filename search (int 21 functions 4e and 4f), the file name may
contain wildcards. A ? matches any character, and an * finishes out the name.
The * must be the last character in the base or extension. I decided to
implement a much more robust system. First, asterisks should be allowed
anywhere in the name, and multiple *s should be allowed. So names like A*ED.*,
*188.*F*, and *I*.*I* are legal. (The last example will find any name that
contains an I.)
The key is to ask DOS for all files, and sort out the matches within the
dir_scanner. A function that matches enhanced wildcard specifications is the
cornerstone. The way to allow multiple *s is to use recursion. The algorithm
(Listing Seven, version a, page 100) is "the strings match if the first
characters match and the rest of the strings match." There is no need to worry
about breaking the name into base and extension because the pattern will
contain a dot, it must match the dot in the filename, and the filename will
always contain a dot.
The function matchname( ) looks at the first character of the pattern. If the
first character is an *, it tries to match the rest of the pattern with the
name. If it does not match, it skips a character in the name and tries again.
It continues this way until a match is found (the * can stand for any number
of characters) or until the end of the name is reached. This recursive
handling of * is fundamental to the enhanced pattern matching. If the first
character of the pattern is a ?, the first character of the name can be
anything except a nul.
The technique in Listing Seven(a) would be great in Prolog, but it has
something to be desired in C. The overhead of recursion is high. On the other
hand, the name is only 13 characters long, so maybe it is OK. But some obvious
improvements can be made with regards to consecutive ordinary characters. With
the exception of *, all the recursion is at the end of execution. Whenever the
recursive call is being returned, you can replace the call with a jump (rather
than pushing parameters, assign the new values to them, then jump back to the
beginning of the function).

Listing Seven(b) shows the result of applying tail-end recursion elimination
to Listing Seven(a). The meaning is the same, but it has much less recursion.
It is a rather ugly sight in C++, though. The last step (Listing Seven[c]) is
to get rid of all the GOTOs. Sometimes the literal translation of the
algorithm can be transformed in good code. This was much simpler than trying
to come up with a good function from scratch.
Now the dir_scanner could use matchname( ) to match filenames, but it does
more than this. The matchname( ) function is instead called from multi_match(
). The dir_scanner accepts search strings with multiple wild names. Separating
the names by + finds matches to the first or the second. Separating with a
semicolon will find all the names that match the first and do not match the
second. Furthermore, the multiple name specification can be preceded by drive
and path information. The MATCH.CPP program (Listing Eight, page 100) lets you
try out these searches.
The constructor dir_scanner.:scanner( ) separates the path from the search
string by breaking it at the last slash. The name is replaced by *.* and sent
to the DOS primitive. It returns all files in the directory and multi_match( )
decides which ones to keep. The search specification is then sent to
buildlists( ).
The multiple names are separated by buildlists( ) into two lists. The goodlist
includes all the regular names, and the badlist contains the names that were
preceded with semicolons. It scans the string to count the names, and
allocates storage for the two lists. The lists point into the single buffer,
and nulls placed into the buffer (overwriting the + or ;) break it into
individual strings. This works better than allocating a bunch of small strings
from the heap.
The function multi_match( ) decides if a name matches the search request. The
name is accepted if it matches any of the names in the goodlist and does not
match any of the names in the badlist. Each of the tests uses the matchname( )
function with its enhanced wildcards.
But matchname only works if the names are normalized. Specifically, both name
and pattern must contain a dot. If the filename does not contain a dot, it is
appended on the end. The search string is processed more so it accepts more
flexible input.
Now you have several more tools for your C++ programs. These should let you
write more powerful programs and write programs easier. After all, that was
the idea when you moved up to C++.
More Details.
Table 1: Sample uses of the Match program.

 Command Use
 -------------------------------------------------------------------------

 *.EXE[+]*.COM Examines all EXE and all COM files

 T*T.*; TEST.BAT Searches for all files that begin and end in T
 except for TEST.BAT

 *.*;COMMAND.COM Searches for all files except COMMAND.COM

 D:\FOO\*.EXE+*.COM;*K*.* Finds all EXE and COM files in D:\FOO\ that do
 not contain the letter K in their names


_EXTENDED DIRECTORY SEARCHES USING C++_
by John M. Dlugosz


[LISTING ONE]

// DIRPATH.HPP Copyright 1988 by John M. Dlugosz

struct dirpath {
 char drive[3];
 char path[65];
 char base[9];
 char ext[5];
 void operator<< (char const*name); //throw a name into the structure
 void operator>> (char *name); //extract name from structure
 dirpath(char const* name) {*this << name;}
 dirpath(char const* d, char const* p, char const* b, char const* e)
 {strcpy(drive,d);strcpy(path,p);strcpy(base,b);strcpy(ext,e);}
 dirpath() {}
 };





[LISTING TWO]

// example using dirpath
// by John M. Dlugosz

#include <stream.hpp>
#include <string.h>
#include <dirpath.hpp>

char infile[67], outfile[67];


void checkfiles (char const* name)
{ /* find input and output filenames */
dirpath d (name);
if (*d.ext == '\0') //supply default extension for input file
 strcpy(d.exe, ".C"); //notice dot in included
d >> infile;
strcpy (d.ext, ".OBJ");
d >> outfile;
}







[LISTING THREE]

/*****************************************************
File: DIRPATH.CPP
Copyright 1988 by John M. Dlugosz, all rights reserved
 parse and combine file names
*****************************************************/

// you must define NULL for your compiler. Most have a way of checking the
// model being compiled under.
#ifdef LPTR
#define NULL 0L
#else
#define NULL 0
#endif

#include <string.h>
#include "dirpath.hpp"

static void copy (char *dest, char const* source, char const* const limit, int
count)
{
while (count-- && source <= limit) *dest++ = *source++;
*dest= '\0';
}

/* /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ */

void dirpath::operator<< (char const* name)
{
//takes a pathname and splits it up into its components.
//any or all of the components may be missing.

//first, locate DRIVE letter
if (strlen(name) >= 2 && name[1]==':') {
 drive[0]= name[0];
 drive[1]= ':';
 drive[2]= '\0';
 name += 2; }
else drive[0]= '\0'; //no drive found

//locate last slash and dot
char *p;

char *last_slash= NULL, *dot= NULL;
for (p= name; *p; p++)
 if (*p == '\\' *p == '/') last_slash= p;
 else if (*p == '.') dot= p;
// p now points to the \0 at the end of the string
if (last_slash) { //path exists. copy up to and including the last slash
 copy (path, name, last_slash, 64);
 name= last_slash+1;
 }
else *path= '\0';
if (dot) { //break int base and ext
 copy (base, name, dot-1, 8);
 copy (ext, dot, p-1, 4);
 }
else { //rest of string is base name, no ext.
 copy (base, name, p-1, 8);
 *ext= '\0';
 }
}

/* /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ */

void dirpath::operator>> (char *name)
{
//concatenate the parts together into a full name.
//the name parameter better be long enough.
if (*drive) {
 *name++ = *drive;
 *name++ = ':'; }
if (*path) {
 strcpy (name, path);
 name += strlen (name);
 if (name[-1] != '/' && name[-1] != '\\')
 *name++ = '\\';
 }
strcpy (name, base);
if (*ext) {
 if (*ext != '.') strcat (name,".");
 strcat (name, ext);
 }
}






[LISTING FOUR]


// DIRSCAN.HPP Copyright 1988 by John M. Dlugosz

/* these values for attribute parameter in constructor
 are passed through to DOS's function 4e */
#define fa_READ_ONLY 0x01
#define fa_HIDDEN 0x02
#define fa_SYSTEM 0x04
#define fa_LABEL 0x08
#define fa_DIR 0x10

#define fa_ARCHIVE 0x20

class dir_data {
 char reserved[21];
public:
 unsigned char attribute;
 unsigned time;
 unsigned date;
 unsigned long size;
 char name[13];
 };

class dir_scanner {
 char **goodlist, **badlist;
 int goodcount, badcount;
 dir_data current_entry;
 bool done;
 bool firsttime;
 void near buildlists (char *filespec);
 bool near multi_match();
public:
 dir_scanner (char *filespec, unsigned attribute=0);
 ~dir_scanner();
 operator int(); //look up next entry, return 1. If no next, return 0.
 dir_data& operator() () {return current_entry;} //fetch current entry
 };






[LISTING FIVE]


/*****************************************************
File: DIRSCAN.CPP
Copyright 1988 by John M. Dlugosz, all rights reserved
 implementation of class dir_scanner, which does
 enhanced file matching in a directory list.
*****************************************************/

enum bool {FALSE,TRUE};
#define NULL 0L //define NULL properly for your compiler and model
#include <string.h>
#include "dirscan.hpp"
// these two function in assembly language (file SCANHELP.ASM)
extern int dirscan_findfirst (char const far *name, unsigned attribute,
dir_data const far *dta);
extern int dirscan_findnext (dir_data const far *dta);

/* /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ */

static void near dir_scanner::buildlists (char *pattern)
{
goodcount= badcount= 0;
for (char *p= pattern;;) { // pass 1 -- count them
 while (*p == ' ' *p == '\t') p++;
 if (*p == '\0') break;
 if (*p == ';') badcount++;

 else goodcount++;
 do p++; while (*p != ' ' && *p != '\t' && *p != ';' && *p != '+' && *p !=
'\0');
 }
typedef char* POINTER; //I need a type name so I can use NEW
goodlist= new POINTER [goodcount]; // allocate arrays
badlist= new POINTER [badcount];
int gcount=0, bcount= 0;
/* I remember the location of the end of the substring in lastbreak. After
 parsing the next string, I stick a '\0' here. I do this one string
 behind because it will overwrite the '+' or ';' */
char *lastbreak= NULL; //NULL means none found yet
for (;;) { // pass 2 -- chop it up
 while (*pattern == ' ' *pattern == '\t') pattern++;
 /* Embedded whitespace tolerated. You can even omit the '+' and seperate
 names with spaces. But if you do have a '+' or ';', it must
 immediately preceed the name */
 if (*pattern == '\0') break; //end of the string
 if (*pattern == ';') //add to BAD list
 badlist[bcount++]= ++pattern;
 else { //add to GOOD list
 if (*pattern == '+') pattern++;
 goodlist[gcount++]= pattern; }
 if (lastbreak) *lastbreak= '\0';
 while (! (*pattern==' ' *pattern=='\t' *pattern=='\0' 
 *pattern==';' *pattern=='+')) pattern++;
 lastbreak= pattern;
 }
}

/* /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ */

dir_scanner::dir_scanner (char *filespec, unsigned attribute)
{
char dir[67], *lastslash= NULL, *p;
int max= 63;

firsttime= TRUE;
// seperate the path information from the search specification
for (p= filespec; *p; p++)
 if (*p == ':' *p == '/' *p == '\\') lastslash= p;
 // notice that both \ and / are acceptable.
p= dir;
if (lastslash) //copy directory information
 // if it is too long, it is truncated. You might want to check and
 // return an error if this happens (max will be -1 after the loop)
 while (max-- && filespec <= lastslash) *p++= *filespec++;
 // if it is too long, it is truncated. You might want to check and
 // return an error if this happens (max will be -1 after the loop)
strcpy (p, "*.*");
buildlists (lastslash ? lastslash+1 : filespec);
/* the first name is read differently then the others. So read the first
 here, and set firsttime so the advance and test function will not
 advance. */
if (0 != dirscan_findfirst(dir, attribute, &current_entry)) done= TRUE;
else {
 done= FALSE;
 return; }
}


/* /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ */

dir_scanner::~dir_scanner()
{
delete goodlist;
delete badlist;
}

/* /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ */

static bool near matchname (char const* pattern, char const* name)
{
while (*pattern) {
 if ((*pattern == *name) (*pattern == '?' && *name)) {
 pattern++;
 name++;
 }
 else if (*pattern == '*') {
 for (;;) {
 if (matchname (pattern+1, name)) return TRUE;
 if (*name) name++;
 else return FALSE;
 }
 }
 else return FALSE;
 }
return *name == '\0';
}

/* /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ */

static char * near normalize (char const* pattern)
{
static char buf[16];
if (*pattern == '.') {
 // if no basename, insert '*'. i.e. ".EXE" => "*.EXE"
 buf[0]= '*';
 strcpy (buf+1, pattern);
 return buf; }
else if (!strchr(pattern,'.')) {
 // if no dot, extension of '*'. i.e. "FOO" => "FOO.*"
 strcpy (buf, pattern);
 strcat (buf, ".*");
 return buf; }
return pattern;
}

/* /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ */

static void near check_for_dot (char* name)
{ /* if name does not contain a dot, add one to the end */
while (*name)
 if (*name == '.') return;
 else name++;
*name++ = '.';
*name= '\0';
}

/* /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ */


static bool near dir_scanner::multi_match()
{
/* a match occurs if the name does not match anything on the badlist,
 and does match something on the goodlist */
check_for_dot (current_entry.name);
int count;
for (count= 0; count < badcount; count++)
 if (matchname (normalize(badlist[count]), current_entry.name)) return FALSE;
for (count= 0; count < goodcount; count++)
 if (matchname (normalize(goodlist[count]), current_entry.name)) return TRUE;
return FALSE;
}

/* /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ */

dir_scanner::operator int()
{ /* this is the advance and test function */
while (!done) {
 if (!firsttime) {
 if (0 != dirscan_findnext (&current_entry)) {
 done= TRUE;
 return FALSE; }
 }
 else firsttime= FALSE;
 if (multi_match()) return TRUE;
 }
return FALSE;
}

/* /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ */







[LISTING SIX]

;SCANHELP.ASM Copyright 1988 by John M. DLugosz, all rights reserved

;extern int dirscan_findfirst (
; char const far *name, unsigned attribute, dir_data const far *dta);
;extern int dirscan_findnext (dir_data const far *dta);

 .MODEL LARGE,C
; to adapt to any memory model, make sure the RET is the right kind, and
; that the parameters are accessed properly on the stack frame. In
; MASM 5.1, just change the preceeding line. LARGE and SMALL versions
; are enogth to service any model, because the data size does not matter
; thanks to the prototypes.


 .CODE

dirscan_findfirst proc uses DS, fname:DWORD, attribute:WORD, dta:DWORD
 lds DX,dta
 mov AH,1ah

 int 21h ;set disk transfer area
 lds DX,fname
 mov CX,attribute
 mov AH,4eh
 int 21h ;find it.
 jc error
 xor AX,AX ;return 0 for ok.
error: ;error code is already in AX
 ret
dirscan_findfirst endp


dirscan_findnext proc uses DS, dta:DWORD
 lds DX,dta
 mov AH,1ah
 int 21h ;set disk transfer area
 mov AH,4fh
 int 21h ;find it.
 jc error
 xor AX,AX
error: ;error code already in AX
 ret
dirscan_findnext endp

 END






[LISTING SEVEN]


// Version A

static bool near matchname (char const* pattern, char const* name)
{
if (*pattern == '?')
 return (*name != '\0' && matchname (pattern+1, name+1);
else if (*pattern == '*') {
 while (*++name)
 if (matchname (pattern+1, name)) return TRUE;
 return FALSE
 }
else if (*pattern == *name) {
 if (*pattern == '\0') return TRUE;
 else matchname (pattern+1, name+1);
 }
else return FALSE;
}



// Version B - apply tail-end recursion elimination

static bool near matchname (char const* pattern, char const* name)
{
restart:

if (*pattern == '?') {
 if (!*name) return FALSE;
 pattern++;
 name++;
 goto restart;
 }
else if (*pattern == '*') {
 while (*++name)
 if (matchname (pattern+1, name)) return TRUE;
 return FALSE
 }
else if (*pattern != *name) return FALSE
else if (*pattern == '\0') return TRUE;
else {
 pattern++;
 name++;
 goto restart;
 }
}


// Version C - get rid of goto's

static bool near matchname (char const* pattern, char const* name)
{
while (*pattern) {
 if (*pattern == *name) (*pattern == '?' && *name) {
 pattern++;
 name++;
 }
 else if (*pattern == '*') {
 for (;;) {
 if (matchname (pattern+1, name) return TRUE;
 if (*name) name++;
 else return FALSE;
 }
 }
 else return FALSE;
 }
return *name == '\0';
}







[LISTING EIGHT]

/*****************************************************
File: MATCH.CPP
Copyright 1988 by John M. Dlugosz, all rights reserved
 demonstration of dir_scanner
*****************************************************/

enum bool {FALSE, TRUE};
#include <stream.hpp>
#include "dirscan.hpp"

#include <ctype.h> //toupper() needed

/* The nl macro might already be in stream.hpp. If it is not,
 you might want to add it. Otherwise, define it here. */
#define nl << "\n"

/* /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ */

ostream& operator<< (ostream& o, dir_data& d)
{
/* let output streams handle directory entries. This just prints the
 name, but you might want to enhance it to prinmt the date, size, etc. */
o << d.name;
return o;
}

/* /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ */

void upcase (char* s)
{ /* convert string to all caps */
while (*s) {
 *s= toupper (*s);
 s++;
 }
}

/* /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ */


main (int argc, char* argv[])
{
char s[80];
dir_data f;
if (argc < 2) { //prompt user
 cout << "enter search request: ";
 cin >> s;
 cout nl;
 }
else strcpy (s, argv[1]); //use command line if present
// the dir_scanner is assumes parameter is ALL CAPS
upcase (s);
dir_scanner d (s); //notice second parameter is missing, so default is used.
int number= 0; //count how many found
while (d) { //advance and test function called here
 f= d(); //fetch function called
 number++; //count it
 cout << f nl; //display it
 }
cout << number << " files found." nl;
}












March, 1989
COPING WITH COMPLEX PROGRAMS


Quality control using software metrics




Karanjit S. Siyan


Karanjit S. Siyan is a consultant with Freedom Technologies and Integrated
Computer Systems. He be reached at Box A, Corwin Springs, MT 59021.


At one time or another, almost every software developer has had to face the
beast of software complexity. One of the most common methods of measuring
software complexity is counting the number of lines of code in the program.
This is an example of a "software metric." Generally speaking, a software
metric measures the characteristics of a program in quantitative terms.
Experience reveals that the number of lines of code is a simplistic and
unrealistic way of measuring the complexity of a program. A better software
metric is needed --one that can measure the "content" of a program. Another
metric is one that can provide an estimate of the difficulty in understanding
the program, writing a similar program, and estimating how many errors you can
expect to see in this program.
Freedom Technologies has been using several software metrics for managing
development efforts. Many of these measures are based on intuitive
understanding of how complex a program is. For example, a program that
exhibits excessive decision making can be somewhat difficult to follow. A
program that makes use of many global variables can be extremely difficult to
modify and maintain because of the tight degree of unwanted coupling and
"side-effects," and the dreaded "ripple-effect" where one innocuous change can
result in the program not working.


Size and Flow Metrics


Different approaches have been suggested to measure those characteristics that
make up complex programs. Broadly speaking, these approaches can be
categorized into two groups: size metrics and control-flow metrics.
Size metrics are based on the premise that the "larger" the program, the more
complex it is. The size of a program is not necessarily based on the number of
lines of code. A more meaningful measure of this size would be based on the
number of procedures, number of operators, number of variables, and so forth.
Control-flow metrics attempt to measure complexity based on the decisions --
statements such as if, for, while, case, etc. --used in a program, and the
levels of nesting. Some measures use graph theory by viewing the abstract flow
of program control as a directed graph.
The two most popular families of metrics that fall into this category are the
Software Science (size metric) and the Cyclomatic Complexity (control-flow
metric). At least one tool from SET Labs Inc. (Portland, Ore.), called
PC-METRIC, computes these metrics and runs on MS-DOS-based systems for
languages such as C, Modula-2, and Pascal.


Assessing Validity of a Metric


To determine if measurement of a certain characteristic of a program
contributes to the complexity of a program, the following two facts must be
established: 1. Does the characteristic computed by the metric actually
contribute to complexity? and 2. Does the computation of a metric reflect how
much complexity is being contributed?
The answers to these questions are based on a statistical analysis of the
programs. Some aspect of the programming process --development time, number of
errors during development, and so forth --is compared with the computed
metric. As an example, if, for a large sample of programs, the time to develop
a program increases with a larger value of the computed metric, then the
computed metric indicates more complex programs. The degree to which a
computed metric is related to the programming process is called correlation. A
well-developed theory exists in statistics for measuring the correlation
between variables. The better the correlation between the software metric
(product metric) and an attribute of the programming process (process metric),
the greater the effectiveness of the software metric.
Another approach is to partition the set of sample programs into groups based
on some characteristic (such as the number of decisions). The average number
of errors can be measured in these groups. If the number of errors is found to
increase for groups that have a larger number of decisions, the software
metric of "number of decisions" does contribute to program complexity.


Software Science


The Software Science metrics were developed in the early 1970s by Maurice
Halstead of Purdue University. Halstead observed that all programs are made up
of operators and operands. By coming up with a measure based on operators and
operands, you can get a better idea of how complex the program really is. The
following four parameters can be counted for any program:
n1 = number of unique operators
N1 = total number of operators
n2 = number of unique operands
N2 = total number of operands
Halstead's entire theoretical framework is based on these four parameters. For
example, the "length" of a program (N) is defined as the total number of
operators and operands used. Also, the vocabulary (n) of a program is the
total number of unique operators and operands.
 N (Length) = N1 + N2 an (Vocabulary) = n1 + n2


Software Science Counting Rules


What is an "operator," and what is an "operand" in a program? How do you
compute the values of n1, N1, n2, N2 for a program? The best way to illustrate
this is through an example. Figure 1 is a program fragment and Figure 2 shows
a tally of the operators and operands. Notice that paired items such as {}, (
) are counted as operators. In some situations, there may be differences in
opinion as to what is an operator and what is an operand. The statement goto
Label can be legitimately viewed as consisting of the operator goto and the
operand Label. You could also treat the whole statement as an operator.
Several researchers have shown that minor differences in counting rules have
little impact on the effectiveness of Software Science as long as the rules
are applied consistently.


The Length Equation and Purity Ratio


Several interesting relations are derived based on the count of operators and
operands. The first one is the predicted "length" of a program. Remember, that
the actual "length" (N) is defined as N1 + N2 or simply the count of all the
operators and operands in a program. Halstead theorized that a well-written
program should have a predicted length, N^ (pronounced N-hat) as shown here:

Figure 1: Program fragment 1

 int atoi(NumString)
 char NumString[I]
 {
 int Number;
 int i;
 i = 1;
 Number = O;
 while(NumString[I] !='\0')
 {
 Number = Number + NumString[I] - '0';
 i ++;
 }
 return Number;
 }/* end
 atoi */

Figure 2: Software science counting rules applied to the program in Figure 1

 Operator Count Operand Count
 ___________________ ______________________________

 = 3 i 4
 ; 5 Number 4
 () 1 NumString 2
 [] 2 '/O' 1
 {} 2 'O' 1
 != 1
 + 1 N2 = 12
 ) 1 n2 = 5
 ++ 1 n1 = 9
 n = 14
 N1 = 17 N = 29

 N^ = [n1 X log[2](n1)] + [n2 x log[2](n2)]
This equation is called the length equation. If the actual length, N, is
different from the predicted length, the program could have any of the
following six classes of impurities: canceling of operators, ambiguous
operands, synonymous operands, common subexpressions, unnecessary
replacements, or unfactored expressions.
Based on N and N^, the purity ratio is defined as N^/N. A purity ratio of 1
would imply few impurities. C programs usually have a purity ratio greater
than 1. This means that the actual length, N, is less than the predicted
length, N^. This could be because logic can be expressed very succinctly in C.
What is interesting about the length equation is that the first term [n1 x log
[2] (n1)] can be treated as a constant. The rationale for this is that the
number of unique operators in a programming language is finite, and most
programmers have a small subset of operators they regularly use. The second
term [n2 x log[2] (n2)] is the one that has a greater impact on the predicted
length. If you can compute an accurate estimate of how many operands will be
used in a program, you can predict how large a program is going to be.


Program Volume and Estimated Errors


Halstead also theorized that if a program has "n" (n = n1 + n2), unique
operators and operands, the number of bits used to represent or encode these
as a binary number would be log[2](n). Since there are "N" (N = N1 + N2) such
usages of these operators and operands, the total number of bits used to
represent the program would be N x log[2](n). This expression is termed the
volume (V) of a program.
 V (Volume) = N x log[2](n)
Consider two programs, each with 200 uses of operators and operands (N = 200).
The first program uses 16 unique operators and operands, while the second uses
64 unique operators and operands. The volume for the first program is 200 x
log[2] (16) = 800, while the volume for the second program is 200 X log[2]
(64) = 1200. The volume is a better estimate of program size than a simple
count of operators and operands (N), since the value is larger with a larger
number of unique operators and operands.
Another way of looking at the program volume is that N total operators and
operands will require log[2](n) mental lookups using the best search
algorithm. The total number of mental lookups is N x log[2](n), same as the
program volume. On the average, people tend to make mistakes every E0 mental
comparisons. Work done by psychologists shows that the value of E0 is about
3,000 to 3,200. The number of estimated errors, B^ is
 B^ = V/E0 = [N x log[2](n)]/E0
A programmer with superior ability will have a larger value of E0, and someone
with an inferior ability will have a lower value of E0. The programmer with
superior ability will make fewer errors. If the actual number of errors is
recorded for a number of programs, a project manager can determine the E0
values for the team members.


Program Effort and Estimated Time


You can expect that the effort required to develop a program will be greater
for a program with larger program volume. A parameter called "effort" (or E)
is defined as
 E = V/L
where V is the volume of a program and L is an estimate of the level of
abstraction in a program. The level of abstraction is defined as
 L = V*/V

where V is the volume measure (as previously defined) and the new V* is the
volume a program would have if it could be represented as a procedure call
within the language. An example will clarify this.
Consider a program to print a report from a data file. A procedure call to
this program could be written as
 PrintReport(DataFile)
The volume (V*) of this statement would be 3 x log[2](3) = 4.74. The volume
(V*) is called the potential volume. An estimate for L (L^) is often used
instead of the equation L = V*/V, which is based on the number of operators
and operands. This estimate is given by the following empirical formula:
 L^ = 2/n1 x n2/N2
A good thing about this estimate for the level of abstraction is that it is
easily computed by a simple analysis without knowing too much about the
internal structure of a program.
What do all the given equations mean? Halstead suggested that the effort (E)
represents the number of "mental discriminations" a programmer would have to
perform in order to write the program. A psychologist, Stroud, found that
humans are capable of making five to twenty mental discriminations per second.
This number is called the Stroud Number, S. You now have the following
equation:
Estimated time to develop a program,
 T^ = E/S
A common value of "S" for programmers is 18.


Cyclomatic Complexity


Tom McCabe suggested that the cyclomatic number of a program's control flow
graph would be an accurate estimate of how complex a program's flow of control
is. The control flow graph is similar to a flowchart. Each node in this graph
consists of a basic block. A basic block is a segment of code that has a
single entry point and a single exit point, and there is no transfer of
control within the basic block. Figure 3 is a fragment of code and Figure 4 is
its control graph. The cyclomatic number of a flow graph is defined as
 Cyclomatic Number of graph "G," V(G) = e - n + 2
where n is the number of nodes in the graph, and e is the number of edges or
lines connecting each node. In Figure 4, n is 4, and e is 4, giving a
cyclomatic number of 2.
A simpler way of calculating the cyclomatic number is to add 1 to the sum of
all the decision-making statements (if, while, for, and so on). In the example
in Figure 3, there is only one if statement, and adding 1 gives 2.
Extended Cyclomatic Complexity is defined as including not just
decision-making statements, but also decision-making predicates. Examples of
decision-making predicates are the logical expressions in an if statement that
are connected by logical and, or. A shortcut method of calculating the
Extended Cyclomatic Complexity is
Figure 3: Program fragment 2

 Init( ): /* Node A */
 if (ReportOption)
 {
 GenReport(ReportOption); /* Node B */
 }
 else
 {
 ReportError(ReportOption); /* Node C */
 }
 Finish( ); /* Node D */

Sum of the number of decisions, ANDs, ORs + 1.
The following if statement has an Extended Cyclomatic Complexity of 5 because
there is 1 if statement, 2 AND (&&), 1 OR ( ).
 if (_dbStatus == 0 && IsValid( ) _dbStatus > 10 && IsExpr()) { return
0; }


Uses of Software Metrics


The useful thing about all these metrics is that they actually help develop
better code. Here are some examples.
After writing a piece of code, a software developer can compute the software
metrics (using a tool, of course,) to provide feedback about potentially
troublesome aspects of a program. The software metrics can be compared against
a norm that is user-definable. Remedial action, if necessary, can then be
taken. This could be a different approach or algorithm that will produce less
complex code, breaking the program down to simpler modules. There are
applications where even the best solution may be very complex. In this case,
software metrics can identify the modules that need better and extra
documentation.
A small percentage of modules in a software system has an inordinate amount of
errors. Software metrics can identify the more error-prone modules, and
greater resources can be allocated to testing them. During the maintenance
phase, software metrics can be used as one of the tools to measure the amount
of effort and time required to make changes to existing modules.
Software metrics should be viewed as one of the tools (among many) to help
manage the process of developing software. However, this tool is not a
replacement for common sense.
In some programs, the metrics may not accurately capture complexity and flag
the programs as being overly complex or less complex than they actually are.
Common sense will aid in determining if the metric results are reasonable.
Applying the metrics to one or two individual programs may be disappointing.
Software metrics work best when applied to a large group of programs.


Validity of Software Science and Cyclomatic Complexity


While much of the literature in software metrics reports studies supporting
the software metrics described in this article, several researchers have
expressed doubts about its validity. Shen, Conte, and Dunsmore voice most of
the objections raised by researchers. Much of the criticism is directed at
certain assumptions made by Halstead and the methodology used by those studies
that support Software Science. These objections drive home the point that
software metrics are effective in a statistical sense when applied
consistently to a large number of programs.


References


Much of the information for this article was taken from Reference 1.
References 2 and 3 are classics by the founders of the metrics examined in
this article. Reference 4 is a critique on Software Science and gives examples
of situations where it does not work.

1. SET Labs Inc., PO Box 03627, Portland, OR 97203, "PC-METRIC."
2. Halstead, M., "Elements of Software Science," Elsevier North Holland, New
York 1977.
3. McCabe, T., "A Complexity Measure," IEEE Transactions on Software
Engineering (Dec. 1976): 308 - 320.
4. V. Shen, S. Conte, and H. Dunsmore, "Software Science Revisited: A Critical
Analysis of the Theory and Its Empirical Support," IEEE Transactions on
Software Engineering (March 1983): 155 - 165.


























































March, 1989
THE PORTABILITY DREAM


Now you can develop an application that will run under Windows and the Mac




Margaret Johnson


Margaret K. Johnson is a software engineer at Beckman Instruments. She can be
reached at 2500 Harbor Blvd., M/S D-33-B, Fullerton, CA 92634. Compu-Serve ID:
74706,2325.


Some might say that the creation of portable code between windowing
environments, such as Microsoft Windows and the Macintosh User Interface, is
an impossible dream. After all, the architectures within which these
environments operate are widely disparate. The Extensible Virtual Toolkit
(XVT), however, is a package from the Advanced Programming Institute that
promises to virtualize the code between these diverse environments to the
point where the process of porting code between them should be just a matter
of source transfer, creation of resources (such as menus, dialog boxes, and
strings), and recompilation. XVT is a collection of C functions that attempts
to abstract the major features of a windowing environment's application
programming interface (API). XVT is a virtual toolkit, because it acts as the
interface to the native toolkit. A user of this package stays away from direct
calls to either the Windows Software Development Toolkit (SDK) or the
Macintosh User Interface Toolbox.
XVT is currently available only for the Mac and Windows. Advanced Programming
Institute is working on a version of XVT for the OS/2 Presentation Manager,
and also promises a future version for X-Windows. Compilers supported by XVT
include MS-C (Version 5.x) for Windows, and both Lightspeed C (Version 3.0)
and MPW C (Version 2.0.2) on the Mac.
Those who stand to gain the most from XVT are developers who are familiar with
C but are new to graphical user interface (GUI) programming, developers who
don't want to invest the time and energy necessary in order to feel
comfortable developing programs for either (or both) Windows or the Mac, and
developers who don't require features that are unique to a particular API. The
package contains enough built-in code to also benefit the developer who
creates applications specific to the Mac or to Windows. XVT's built-in
routines include functions for file handling, printing, managing the
clipboard, font handling, dialog management, and a facility for easily
including a help system within an application.


Portability Between Environments


To reach the dream of 100 percent portability, the process of virtualization
cannot compromise too many features of an environment. The application
determines which features can be compromised. Although the Mac and Windows are
conceptually similar, there are enough differences in their implementations to
make abstractions to some of their features difficult, if not impossible. Both
APIs have terrific features that are specific to their respective
environments. For example, Windows allows child windows, the sending and
posting of intratask messages, dynamic link libraries, the dynamic data
exchange for intertask messages, mapping modes (MM_LOENGLISH, MM_ANISOTROPIC,
and so forth), window property lists, subclassing, and atom tables. Because
Windows provides a nonpreemptive form of multitasking, its API contains
functions that allow an application to yield to other Windows applications
during processing.
On the other hand, the Mac supports more choices for menu styles, such as
hierarchical, tear-off, and popup menus. It also allows nonrectangular and
shadowed windows. In addition, the Mac's API contains functions that support
internationalization and file handling dialogs; Windows does not include these
functions. The differences between Windows and the Mac can be interpreted as
constraints to the XVT API. With the exception of the standard
file-open-and-save dialogs, the special features listed above for Windows and
the Mac are not part of the XVT package. In addition, the version that I
reviewed (1.2) does not include support for bitbltting (bit block transfers),
the user of color, internationalization, RS-232 communications, sound, and
mapping modes.


Handing Over Control


When I began reviewing XVT, I noticed that I took the side of what the product
lacks, rather than what it has to offer. This perspective most likely results
from both my long hours spent becoming familiar with the Windows SDK, and my
current interest in Mac development. It's the perspective of a developer who
is asked to move from a low-level language, such as assembler, to a
higher-level language, such as C: Some control must be taken away. This is a
hard thing to come to terms with. Once I gave XVT a chance, however, I was
surprised at how many of the more important features are available.
These features include support for dialog boxes, windows (in the sense of
Microsoft's Multiple Document Interface, as described in the MS-Window's SDK
Application Style Guide), text, file handling, printing, drawing primitives,
handling the clipboard, and 15 of what XVT's developers believe are the most
popular events that can occur to a window. The dialog boxes can contain push
buttons, radio buttons, check boxes, scroll bars, static text, edit boxes, and
listbox controls. The drawing primitives allow the drawing of rectangles (with
or without rounded corners), ovals, arcs, pie pieces, lines (with or without
arrows), and polygons.
XVT's 15 Window-related events fall far short of the approximately 100
possible event messages that are available under Windows. Although the Mac
also has 15 events, they're not identical (in meaning) to XVT's events. XVT's
events are probably sufficient, however, for the 90 percent case. They include
messages for mouse down, mouse up, mouse movement, mouse double click,
keyboard input, window update, window activation/deactivation, window
destruction, vertical scroll bar activity, horizontal scroll bar activity,
menu commands, close window, resize window, selection from the font or style
menu, and a request to quit the application.


Taking the Plunge


The XVT product that I received included a loose-leaf binder jampacked (almost
to the point of explosion!) with pages including the "XVT Programmer's
Manual," a question and answer sheet covering the most often asked questions,
information on enhancements and changes since version 1.1, two 5 1/4-inch 360K
floppy disks for the PC, two 3 1/2-inch 800K floppy disks for the Mac, and a
registration form. The programmer's manual is divided into sections for
installation and usage, a user's guide, a reference to each function
(documented with a description, example, and list of associated functions),
technical notes (giving insight on how to implement techniques not covered
elsewhere), and a quick reference.
I felt the documentation did a good job of explaining both the objective of
SVT and the functions that are implemented to reach that objective. The only
major complaint is a lack of an index for each section. This is promised for a
later release.
To use XVT on the PC, it's necessary to first install the Windows operating
environment (2.x), the SDK (2.x), and Microsoft's C compiler (5.x). On the Mac
side, I used Lightspeed 3.0. Note that all of this software assumes the
presence of a hard disk.
Installation of XVT on the PC side is easy. First create the five
subdirectories \XVT\BIN, \XVT\EXAMPLES, \XVT\INCLUDE, \XVT\LIB, and
\XVT\SOURCE on the hard disk. Then copy the contents of the subdirectories on
the XVT floppy to the corresponding subdirectories on the hard disk. The
second floppy contains a program called XVTDraw that highlights the usage of
XVT in a Windows draw package. It is for demonstration purposes only.
Once XVT is installed, life is easier if the lines that set the INCLUDE and
LIB paths in AUTOEXEC.BAT are modified to allow the XVT paths. For example:
SET INCLUDE=\MSC\INCLUDE;\XVT\INCLUDE SET LIB=\MSC\LIB;\XVT\LIB
Note that all this discussion about paths assumes \XVT is the parent
directory. If this isn't the case, a minor annoyance is encountered. The file
XVTRSRC.HRC (located in the XVT INCLUDE subdirectory) and the .rc files
included with the examples, hardcode pathnames that assume the parent
directory to be \XVT. This minor inconvenience should be handled by the
installation.
The Mac version of XVT contains the BIN, EXAMPLES, INCLUDE, and LIB folders. I
loaded the INCLUDE and LIB folders into the Think C folder, and created a new
folder to contain the BIN and EXAMPLES folders.
XVT includes nine examples that show off the package's abilities. The best way
to sally up to XVT is to first read the technical overview and then follow the
examples. These examples cover the use of the clipboard, scrolling, the use of
the font style menu, tracking with the mouse, directory manipulation, and
dialog boxes. All examples are well thought out and provide a great starting
point for applications development.


Taking Issue


I wish XVT handled some characteristics on the PC side differently. In
particular, the XVT functions are not in a dynamic link library (DLL), and
thereby zapping more of my valuable memory resource and data area than
necessary (for more insight into DLL, see "Dynamic Link Libraries Under
Microsoft Windows," elsewhere in this issue). Putting the XVT functions into a
DLL would offer considerable advantages. The code would be shared not only
between instances of an application, but also between applications. Also, any
static or global data defined and used exclusively by the library would not
eat into an application's precious local data. The Advanced Programming
Institute plans to implement the OS/2 library as a DLL, but there are no plans
to do the same for the Windows library.
Another problem is that although the XVT functions were created using the
medium model, the code segments were not separated. This approach creates a
huge code segment (around 60K) that degrades the performance of Windows and,
consequently, the performance of the application. If the application is of any
size, the performance of Window's memory manager is really improved by
breaking the code up into multiple segments. Luckily, this step can be handled
by creating a map listing and adding the code segment names to the module
definition file. An example is provided in the Installation and Users Guide.
Note that windows in XVT are created with a NULL brush for the background.
This means that an application must paint the client rectangle each time it
receives an update event. The time this unnecessary call takes to paint the
client rectangle each time an update event occurs is quite noticeable.
In Windows, once the application's background brush is set to a non-null
brush, the application need not worry about updating the background. When XVT
is used on a Mac, application calls EraseRect( ) to update the background. In
both of these cases, XVT should take care of updating the background.
The size of the application's main window is predefined as the full screen.
This makes the main window look maximized, even though the size boxes are the
same as those used for a normal window. If the programmer is not allowed to
enter the screen coordinates for the main window, then the size of the main
window should default to the default overlapped window size that Windows
provides.
One final suggestion that I would make for future versions of XVT is that the
system font be used as the default.



Hello World!


The simplest application created with XVT must contain the following:
a menu
an initial window data structure
an application initialization function named appl_init( )
an application cleanup function named appl_cleanup( )
a main event function named main_event( ) that is called by XVT when one of
the 15 defined events occurs
Listings One through Three, illustrate the difference between XVT code
(Listing One, page 102), Mac (Listing Two, page 102), and Windows code
(Listing Three, page 104) for the classic "Hello World!" example. Figure 1
shows the output from either of these examples. Although the application is
very simple, I think it gives the flavor of the way that XVT abstracts the
features of the PC and Mac environments. I used Lap-Link (Mac) to transfer
files between the two machines. XVThello.c compiled without a hitch in both
environments. Once I set up the resource and library files, the programs I
tested ran as advertised.


Final Note


To summarize, whether XVT will fill a particular need is clearly up to the
application. Some important features, such as the dynamic data exchange in
Windows and the more advanced intertask communications in OS/2 Presentation
Manager, would be difficult to reproduce in the Mac environment, and are thus
not included into XVT. Every day, there is talk about packages soon to be
introduced that promise improved portability between GUI implementations. For
instance, Microsoft and Glockenspiel have announced a product called
CommonView. According to information that I can gather at this time,
CommonView is implemented as a DLL and provides C+ + classes for most Windows
objects. It claims a high percentage of portability between Windows and OS/2
PM. The developers also hope to have ports for X-Windows, News, and the Mac at
some unknown future date. But that's the future, and XVT is available now. I
am impressed with both the amount of effort invested in this package and with
its capabilities. The folks at the Advanced Programming Institute have shown
the impossible dream to be a reality for a significant number of features
found in a windowing environment.

_EXAMINING ROOM - THE PORTABILITIY DREAM_
by Margaret Johnson


[LISTING ONE]

/**********************************************************************
 XVT "Hello World"
 **********************************************************************/
#include "xvt.h" /* standard XVT header */
#include "xvtmenu.h" /* standard XVT menu tags */


/*
 Required application setup structure.
*/
APPL_SETUP appl_setup = {
 0, /* menu bar resource ID (use default) */
 0, /* about box resource ID (use default) */
 "Hello World!", /* application's name */
 W_DOC, /* type of initial window */
 TRUE, /* size box on initial window? */
 FALSE, /* vert. scroll bar on initial window? */
 FALSE, /* horz. scroll bar on initial window? */
 TRUE, /* close box on initial window? */
 FALSE, /* want std. font menu? (includes sizes) */
 FALSE /* want std. style menu? */
};
/*********************************************************************
 * Main application entry point.
 *********************************************************************/
void main_event(win, ep)
WINDOW win;
EVENT *ep;

{ RCT rct;

 switch(ep->type)
 {
 case E_UPDATE:

 get_client_rect(win,&rct);
 set_pen(&white_pen);
 draw_rect(&rct);
 draw_text(10,100, "Hello World!", -1);
 break;
 case E_COMMAND:
 if (ep->v.cmd.tag == M_FILE_QUIT)
 terminate();
 break;
 case E_CLOSE:
 terminate();
 break;
 case E_QUIT:
 if (ep->v.query)
 quit_OK();
 else
 terminate();
 break;
 }
}

/***********************************************************************
 * Application cleanup. Nothing to do.
 ***********************************************************************/
void appl_cleanup()
{
}
/**********************************************************************
 * Application initialization.
 **********************************************************************/
BOOLEAN appl_init()
{
 return(TRUE);
}





[LISTING TWO]

/***********************************************************************
 * Mac "Hello World"
 ***********************************************************************/
#include <QuickDraw.h>
#include <WindowMgr.h>
#include <ControlMgr.h>
#include <EventMgr.h>
#include <DeskMgr.h>
#include <MenuMgr.h>

GrafPtr w_port;
Rect drag_rect, grow_bounds;
WindowRecord w_record; /* storage for a window's information */
WindowPtr hello_window; /* a pointer to that storage */

#define mk_long(x) (*((long *)&(x)))

main()

{
 init_process(); /* do all the initialization */
 make_window();
 event_loop();
}

/**********************************************************************/
 init_process()
 {
 init_mgrs();
 set_parameters();
 }

/**********************************************************************/
init_mgrs()
{
 InitGraf(&thePort);
 InitFonts();
 FlushEvents(everyEvent,0);
 InitWindows();
 InitCursor();
}
/**********************************************************************/
set_parameters()
{
 drag_rect = thePort->portRect;
 SetRect(&grow_bounds, 64, 64, thePort->portRect.right,
 thePort->portRect.bottom);
}
/**********************************************************************/
make_window()
{
 hello_window = GetNewWindow(128,&w_record,-1L);
}
/**********************************************************************/
event_loop()
{
 EventRecord event;

 while (1)
 {SystemTask();
 GetNextEvent(everyEvent, &event);
 switch(event.what)
 {
 case mouseDown:
 do_mouse_down(&event);
 break;

 case updateEvt:
 do_update(&event);
 break;

 case activateEvt:
 do_activate(&event);
 break;

 default:
 break;
 }

 }
}
/**********************************************************************/
do_mouse_down(eventp)
 EventRecord *eventp;
 {
 WindowPtr mouse_window;

 switch(FindWindow(mk_long(eventp->where),&mouse_window))
 {
 case inContent:
 if (mouse_window != FrontWindow())
 SelectWindow(mouse_window);
 break;

 case inDrag:
 DragWindow(mouse_window, mk_long(eventp->where),&drag_rect);
 break;

 case inGrow:
 grow_window(mouse_window, mk_long(eventp->where), &drag_rect);
 break;

 case inGoAway:
 if (TrackGoAway(mouse_window,mk_long(eventp->where)))
 finish();
 break;

 default:
 break;
 }
 }
/**********************************************************************/
do_update(event)
 EventRecord *event;
{
 GrafPtr save_graf;
 WindowPtr update_window;

 if (FindWindow(mk_long(event->where),&update_window) != inSysWindow)
 {if (update_window == hello_window)
 {GetPort(&save_graf);
 SetPort(update_window);
 BeginUpdate(update_window);
 ClipRect(&update_window->portRect);
 EraseRect(&update_window->portRect);
 DrawGrowIcon(update_window);
 draw_content(update_window);
 EndUpdate(update_window);
 SetPort(save_graf);
 }
 }
}
/**********************************************************************/
do_activate(event)
 EventRecord *event;
{
 WindowPtr event_window = (WindowPtr)event->message;
 if (event_window == hello_window)

 {DrawGrowIcon(event_window);
 if (event->modifiers & 1)
 SetPort(event_window);
 }
}
/**********************************************************************/
grow_window(window,mouse_point)
 WindowPtr window;
 Point mouse_point;
{
 long new_bounds;

 inval_bars(window);
 new_bounds = GrowWindow(window, mk_long(mouse_point),&grow_bounds);
 if (0 == new_bounds)
 return;
 SizeWindow(window,LoWord(new_bounds),HiWord(new_bounds),TRUE);
 inval_bars(window);
}
/**********************************************************************/
inval_bars(window)
 WindowPtr window;
{
 Rect temp_rect, port_rect;

 port_rect = window->portRect;

SetRect(&temp_rect,port_rect.left,port_rect.bottom-16,port_rect.right,port_rect.bottom);
 InvalRect(&temp_rect);

SetRect(&temp_rect,port_rect.right-16,port_rect.top,port_rect.right,port_rect.bottom);
 InvalRect(&temp_rect);
}
/**********************************************************************/
draw_content(window)
 WindowPtr window;
{
 MoveTo(100,100);
 DrawString("\pHello World!");
}
/**********************************************************************/
finish()
{
 exit(0);
}





[LISTING THREE]

#include <windows.h>
#include "hello.h"

BOOL NEAR Initialize( HANDLE hInst, HANDLE hPrevInst, int nCmdShow );
long FAR PASCAL WndProc ( HWND hWnd, WORD wMessage, WORD wParam, LONG lParam);
static char szClass[40];
static char szTitle[40];

 int PASCAL WinMain( hInst, hPrevInst, lpszCmdLine, nCmdShow )

 HANDLE hInst; /* Our instance handle */
 HANDLE hPrevInst; /* Previous instance of this application */
 LPSTR lpszCmdLine; /* Pointer to any command line params */
 int nCmdShow; /* Parameter to use for first ShowWindow */
 {
 MSG msg; /* Message structure */

 if( ! Initialize( hInst, hPrevInst, nCmdShow ) )
 return FALSE;

 while( GetMessage( &msg, NULL, 0, 0 ) ) {
 TranslateMessage( &msg );
 DispatchMessage( &msg );
 }

 return msg.wParam;
 }
/************************************************************************
 Initialize the application.
 Returns TRUE if initialization succeeded, FALSE if failed.
 ************************************************************************/
 BOOL NEAR Initialize( hInst, hPrevInst, nCmdShow )
 HANDLE hInst; /* Our Instance handle */
 HANDLE hPrevInst; /* Previous instance handle, 0 if first */
 int nCmdShow; /* Parameter from WinMain for ShowWindow */
 {
 WNDCLASS WndClass; /* Class structure for RegisterClass */
 HWND hWnd; /* The window handle */
 HMENU hMenu; /* Handle to the (system) menu */

 if( ! hPrevInst )
 {
 LoadString( hInst, IDS_CLASS, szClass, sizeof(szClass) );
 LoadString( hInst, IDS_TITLE, szTitle, sizeof(szTitle) );
 WndClass.style = CS_HREDRAW CS_VREDRAW;
 WndClass.lpfnWndProc = WndProc;
 WndClass.cbClsExtra = 0;
 WndClass.cbWndExtra = 0;
 WndClass.hInstance = hInst;
 WndClass.hIcon = LoadIcon( NULL, IDI_APPLICATION );
 WndClass.hCursor = LoadCursor( NULL, IDC_ARROW );
 WndClass.hbrBackground = GetStockObject(WHITE_BRUSH);
 WndClass.lpszMenuName = NULL;
 WndClass.lpszClassName = szClass;

 if( ! RegisterClass( &WndClass ) )
 return FALSE;
 }
 else
 {
 GetInstanceData(hPrevInst, szClass, sizeof(szClass));
 GetInstanceData(hPrevInst, szTitle, sizeof(szTitle));
 }
 hWnd = CreateWindow(
 szClass, /* Class name */
 szTitle, /* Window title */
 WS_OVERLAPPEDWINDOW, /* window style */
 CW_USEDEFAULT, /* x */
 0, /* y */

 CW_USEDEFAULT, /* x width */
 0, /* y width */
 NULL, /* Parent hWnd (none for top-level) */
 NULL, /* Menu handle */
 hInst, /* Owning instance handle */
 NULL /* Parameter to pass in WM_CREATE (none) */
 );
 ShowWindow( hWnd, nCmdShow );
 UpdateWindow( hWnd );

 return TRUE;
 }
/***********************************************************************
 Process the messages
***********************************************************************/
 long FAR PASCAL WndProc(hWnd, wMessage, wParam, lParam)
 HWND hWnd;
 WORD wMessage, wParam;
 LONG lParam;

 {PAINTSTRUCT ps;

 switch (wMessage)
 {case WM_PAINT:
 BeginPaint(hWnd,&ps);
 TextOut(ps.hdc,10,100,"Hello World!",12);
 EndPaint(hWnd,&ps);
 break;
 default:
 return DefWindowProc( hWnd, wMessage, wParam, lParam );
 break;
 }
 return 0L;
 }




























March, 1989
THE OSF WINDOWING SYSTEM


The OSF user interface is built around several graphical user-interface
technologies




Kee Hinckley


Kee is a software engineer currently on loan to the Open Software Foundation
from Apollo Computer and he can be reached at 11 Cambridge Center, Cambridge,
MA 02142.


Two months after its formation in 1988, the Open Software Foundation (OSF)
released a Request For Technology (RFT) to organizations to submit their X
Window System-based user-interface technologies for inclusion in the OSF's
future implementation of Unix. After reviewing the 39 systems that were
submitted, the members of the OSF staff responsible for the "user environment
component" or UEC (that is, the graphical user interface) culled the list to
23 qualified candidates. The UEC staff then divided these into four areas of
interest; Window Managers, User Interface Toolkits, User Interface Management
Systems, and Graphical Desktops.
Early this year, the OSF announced a hybrid UEC windowing system selected
called the "OSF/Motif user interface system" built around, not one, but
several graphical user interface technologies: Digital Equipment Corp.'s (DEC)
toolkit to provide the interface toolkit, Hewlett-Packard to provide the
window appearance, and Microsoft's Presentation Manager for the user
interface's behavior.


The Selection Scope


OSF intentionally worded the initial RFT so as to encourage submissions in a
variety of areas that otherwise might not have been recognized at the outset
of the process. This revealed a wide variety of submissions exhibiting diverse
strengths. Many of these submissions were of potential interest for inclusion
in the core offering, but were not yet appropriate for a variety of reasons.
Some are in technology areas where the market has not yet settled on a single
model or set of techniques. Others, depended on technologies not yet widely
available to some OSF members, or required base technologies that as yet are
not standardized. Rather than ignore these products completely, OSF created a
model to facilitate channeling them in directions compatible with the OSF base
offerings, and potentially into the base offering set itself.
At the center of this model is the UEC core offering made up of a style guide,
which provides a description of the behavior and a reference appearance; user
interface toolkit, which provides a standard graphical user interface layer;
presentation description language, which provides quick turn-around and easy
modification of user interfaces; and window manager, which gives the user a
standard environment for manipulating application windows.
The second ring of the technology model is the OSF catalog program. The
catalog program encompasses technologies that use or are integrally compatible
with the core offerings, but which OSF is not ready to include in the core.
Submissions currently under consideration for inclusion in the catalog program
include: open dialogue, a presentation and dialog management system submitted
by apollo computer; base/open user interface management system, a dialog
management system submitted by the swedish telecom group; and generic window
manager, a UIMS for building window managers, submitted by Groupe Bull.
The final ring of the technology model is the OSF research program. This
includes areas of interest in which OSF is encouraging investigation, but
which are not necessarily compatible with the core offerings and may not be as
portable. UEC offerings currently under consideration for this program include
the Andrew Window System, submitted by Carnegie-Mellon University.
The core offering is currently available to OSF members in snapshot form (that
is, prerelease versions in source code form). The first version of the UEC,
available in the summer of 1989, will be portable to both System V and BSD
Unix systems. The initial catalog and research technology offerings will be
available in a similar time frame.
The OSF/Motif toolkit is based on the X Intrinsics, a toolkit framework
provided with MIT's X Window System (Version 11, Release 3). The Intrinsics
use an object-oriented model to create a class hierarchy of graphical objects
known as "widgets." Widgets have an associated set of "resources," some of
which are specific to a particular class of widget, others inherited from
superclasses. Resource values are specifiable both by the application program
(either directly or through the presentation description language) and by the
user (through a standard set of files that are read by the Intrinsics). They
determine the actual appearance and behavior of any particular instance of a
widget and give the user and application designer a great deal of latitude for
customization. For example, a PushButton widget has resources to set the top
and bottom "shadow" colors that give it a 3-D appearance. Other resources set
the color of the button, the color of the text, and a list of callback
procedures for each of a wide variety of events (arm, disarm, activate...).
Most applications need not set more than a few resources; the OSF/Motif
Toolkit provides reasonable defaults for the rest, on both color and black and
white systems.
One of the strengths of the OSF/Motif toolkit is the extensibility provided by
its object-oriented model. It is easy for the application designer to add new
widgets, even without access to the OSF/Motif source code. Thus, vendors and
third parties can create entirely new graphical objects that are specific to
their applications, or they can sub-class from an existing class to add
functionality to a particular widget (for example, a Text widget that does a
specific type of command completion). This extensibility allows the toolkit to
grow and advance in the future as new user interface techniques are developed.


The Style Guide


The user interface portion of a program can be broken into two parts--its
"appearance" and its "behavior" (that is, its "look and feel" respectively).
It was the opinion of the OSF membership, consultants, and staff that the
behavior was by far the more important of the two. As a result, the OSF/Motif
style guide sets forth the manner in which the application should interact
with the user, but not how the graphical objects themselves should appear. It
does, however, address some layout issues, because the location of certain
common functions (such as the Help button) clearly affects the user's
behavior. The visuals furnished with the toolkit provide a distinctive and
intuitive 3-D appearance, however, vendors and software developers are free to
modify this appearance as they wish.
The behavior of the OSF offering is compliant with the Presentation Manager,
as specified by the joint HP/Microsoft submission. This provides the user with
a well-designed and well-tested interface. In addition, it provides an
interface that is familiar to anyone having used Macintoshes, PCs, or other
systems descended from the Xerox user interface research. This creates a high
level of "user-transferability" between personal computers and workstations, a
goal clearly set forth by the OSF membership.


The Window Manager


Unlike many window systems, the X Window System is "policy-free" with regards
to application windows. Instead a separate application is provided, which
allows the user to move, resize, iconize, or otherwise manipulate windows.
(Unix-based window systems traditionally have a concept of an "icon" different
than that in graphical desktops. Rather than representing a file or folder,
these icons represent running programs that the user temporarily doesn't want,
taking up large amounts of screen space.) Window managers provide the window
"dressing" (special areas around the window border, which are used to perform
these functions) and they can implement policies concerning where certain
windows can go, whether windows can overlap, and where to put the icons.
The OSF/Motif Window Manager provides the user with standard Presentation
Manager behavior. It is highly customizable, however, and allows the user to
redefine the contents of the window manager menus, control the amount and
behavior of the window dressing, and alter many other aspects of
window-related interactions.


The Presentation Description Language


The PDL is a language for describing widget resources. Normally the
application programmer must make a series of calls to build up a widget
description, and then create the widget. With the OSF/Motif PDL, the
programmer (or interface designer) creates a text file that contains a
description of each of the widgets and their resources. This description is
then compiled into a resource file. The application code simply makes a call
to load this file and the widgets are automatically created and initialized.
This separation of application and interface allows the interface designer to
make many changes to the overall appearance and layout of an application
without having to modify, recompile, or relink the application itself.


Native Language Support


One of the primary goals of the RFT process was to support non-English
languages. While this area still requires a great deal of work (particular in
the area of simultaneously supporting 8-bit and 16-bit character sets), the
OSF offering does begin to address the problems. This is done by using a Unix
"environment variable" (the X/Open specified LANG variable) to determine the
current default language. (Environment variables are set from the standard
Unix command interpreters --usually when the user logs in --and can be read
via a system call by any interested program.) The toolkit uses this
information to load the user and application resource files appropriate for
that language. Because the text of an interface is stored in the resource
file, it allows the developer to easily construct interfaces that can be
localized to particular language environments. In addition, this facility can
be used by programs that wish to support more than one language. By specifying
the language name explicitly they can override the default and load all, or
part of their resources, from a specific language file; even if that file did
not exist when the application was originally built.


Summary



The OSF UEC system represents nearly six months of open input and close
scrutiny of a wide variety of user interface technologies. Its availability
across a wide range of Unix platforms, and its high-level user interface
toolkit should increase the incentive of software vendors to port to the Unix
market. In addition, the UEC work at OSF does not end with the current
offering. Future activities being considered include: work on providing a
printing/imaging toolkit; system information APIs for graphical desktops,
shells or cyberspace decks; higher-level user interface APIs that don't
require access to the underlying system; compound document support for
multi/hyper-mediadocuments; and on-line help facilities; among others.



























































March, 1989
PROGRAMMING PARADIGMS


Is Multiple Inheritance Necessary?




Michael Swaine


Amoebas at the start Were not complex; They tore themselves apart And started
sex.
--Arthur Guiterman
In my last column in February, I reported on my interviews with Chuck Duff of
The Whitewater Group and Jim Anderson of Digitalk, two proponents of a
more-or-less-pure object-oriented programming paradigm. Their loyalty to this
more or less pure paradigm is consistent with the fact that it's the paradigm
to which each of them has hitched the chariot of his reputation and personal
fortune. Both Actor (Duff's baby) and Smalltalk/V (Anderson's) are more or
less pure object-oriented languages. It is plausible that others, whose
chariots are otherwise harnessed, might disagree with some of Duff's and
Anderson's views, and such is the case. That's what makes chariot races.
Both Anderson and Duff express strong reservations about multiple inheritance,
the capability for an object to inherit from more than one ancestor. Actually,
Smalltalk/V has the facility hidden within it, lacking only a user interface
to make it available to the programmer; and there's a multiple-inheriting
Actor in the wings. But Anderson and Duff seem less than enthusiastic about
biting into the apple of multiple inheritance. There's the matter of name
clashes, for instance. As Duff explains it, "there are cases in which two
multiple-inherited classes with instance variables--with the same name
each--may want to preserve their own copy of that instance variable, and there
are equally viable cases in which they want to share it." There is no
algorithm in existence for resolving such conflicts.
When an object begets new objects all by itself, things are simpler. Multiple
inheritance is mysterious and messy. If its lure is strong, there may be
wisdom in resisting temptation. "Dragons lie there," Duff says, expressing a
diffidence perhaps appropriate for a paradigm still in its adolescence.
I don't mean to put down adolescence, or its preoccupations. After all, I just
wrote a book about HyperTalk, the semi-object-oriented language of a product
whose creator has called it a "software erector set."
But there comes an end to adolescence.


The French Have a Word for It


Unlike Duff and Anderson, Bertrand Meyer has gone all the way. Eiffel is the
name of Meyer's object-oriented programming language, which incorporates
multiple inheritance. In fact, Eiffel depends intimately on multiple
inheritance in its own structure. While Duff and Anderson wonder if the
customers who ask them about multiple inheritance are just indulging in
ivory-tower fantasy, for Meyer it is a fact of life.
In the November/December 1988 issue of The Journal of Object-Oriented
Programming, Meyer presents the view from the Eiffel Tower:
"Whenever you talk about multiple inheritance, someone is bound to ask sooner
or later (usually sooner) what happens in the case of name clashes --
identically named features in two or more parent classes. No doubt the
question is legitimate, but the gravity with which it is asked -- as if it
were a deep conceptual issue -- has been an unending source of bewilderment to
me. I believe it is one of these cases in which, if you only take a minute or
two to pose the problem cleanly, the solution follows immediately.
"First, it is purely a syntactical problem, due to conflicting name choices.
It has nothing to do with the fundamental properties of the classes involved.
Second, nothing is wrong with the parents; each is perfectly consistent as it
stands. The 'culprit' is the common heir, that tries to combine two classes
that are incompatible as they stand. So the heir should also be responsible
for the solution."
Eiffel's solution is to reject such ambiguities with a complier error message,
and to require the programmer to resolve them, possibly by renaming one or
both features (my_father's _temper, my_mother's_eyes). Duff has asked, "Do you
ask the programmer on a case-by-case basis to make the resolution?" Meyer
obviously thinks that's a reasonable solution.
Meyer says that users of a class should not have to know its ancestry; the
interface to the class should be complete and consistent on its own terms. The
simple expedient of appropriate renaming is enough, he says, to ensure that
name clashes don't get in the way of this goal.
Renaming is a purely syntactic business, and Meyer admits that "the
improvement it brings...maybe labeled a cosmetic one." But this is not, in his
view, to dismiss it as trivial. Meyer does not disdain cosmetics.
In the same column, Meyer ridicules nonsensical examples of multiple
inheritance, such as the class apple_pie inheriting from apple and pie, or
class airplane inheriting from fuselage and engine, pointing out that an apple
pie is not an apple and an airplane is neither a fuselage nor an engine. And
he presents some real examples in which multiple inheritance seems called for.
One of these is the class window, which in the implementation he describes,
inherits from the classes rect_shape and tree. In this implementation, a
window is a rectangle, but it is also a tree, with properties such as
superwindows and subwindows and facilities for adding and deleting subwindows.
Meyer's description of a lazy programmer putting together a windowing system
in a day by drawing on existing rect_shape and tree classes gives him the
opportunity to show the need for renaming, even when there are no name
clashes.
Without renaming, the window class inherits tree features with all their
arboreal nomenclature clinging to them. A superwindow is called a parent_node,
the method for adding a subwindow is called insert_node. Anyone using this
class would likely find this confusing. The user of a class has a right to
expect it to be complete and consistent on its own terms. Leaving the tree
terminology in the window class is a pointless flaunting of ancestry, Meyer
believes.
Meyer is not an unbiased reporter, but clearly one who has tasted the fruits
of multiple inheritance and cannot go back to the simpler, more innocent world
of single inheritance. As he puts it, "life without multiple inheritance would
be...boring."
The centerspread of that issue of The Journal of Object-Oriented Programming
is a picture of the Eiffel Tower. It looks like something that an adult
erector set would produce. I wonder why Meyer chose that image to symbolize
his product.


A Fig Leaf for Actor


With the end of adolescence comes a desire for privacy. The matter is one
Chuck Duff has been giving a lot of thought to recently.
"There is no privacy provision in SmallTalk," Duff says. There's no privacy
provision in Actor, either, but Duff and the programmers at The Whitewater
Group are working on that. Duff thinks that privacy is important to the future
of object-oriented programming, because without it "once you write a method,
it is visible to all of your descendant classes. The same is true of instance
variables. Without more control over privacy it becomes very difficult to do
things like multiple inheritance well."
I suppose it would. Duff gives details:
"There are really three categories of things in an object-oriented system:
There's an object that is of the class for which a method was originally
written. That's the most local. Then there are descendants of that class;
they're not as local; they're almost like outsiders, but they're privileged
outsiders in SmallTalk. And then there are objects of completely different
classes.
"Any good object-oriented language will make things opaque to outsiders;
that's the whole point, that's the abstract data type layer. So you can't look
into the representation of something that you're not related to. The problem
is that in SmallTalk, there's no distinction between an object of the class
and an object of a descendant class, so you have full visibility to all those
inherited methods and instance variables, and that really isn't appropriate."
The first level is drawing the curtains against the neighbors, the second is
closing the bedroom door against the kids. In the interest of making the code
more maintainable, programmers at The Whitewater Group are currently
considering how to implement that extra layer of privacy, the bedroom door.


Responsible Cox


And where are such changes as multiple inheritance and privacy taking
object-oriented programming? Closer to the ideal of a system of software
components that can be reused to solve problems similar to the one for which
they were first developed, and by programmers other than the developer, and
that can be adapted to new uses without actually being modified? Toward a
system that minimizes the impact of change in software development? That would
be nice.
One person who has thought hard about the idea of reusable software components
is Brad Cox, whose book Object-Oriented Programming: An Evolutionary Approach
gets at it via the concept of the software IC. Although his book claims to be
about object-oriented programming, Cox takes a less pure approach than Meyer,
Anderson, and Duff, presenting all his examples in the hybrid language
Objective-C and advocating what he calls a "hybrid defense" against change. He
uses the word "defense" frequently in discussing software development; for
Cox, some protection is required if we are to do it responsibly.
Cox's idea of software ICs may be a bigger idea than OOP. In spelling out some
of the desiderata of software building blocks that can serve as the base for a
pyramid of software development, he characterizes the pure object-oriented
approach as building "armor-plated objects that communicate by sending
messages." He describes conventional programming as building "efficient but
brittle software systems, surrounded by static defensive structures that
protect them from change." Encapsulation, inheritance, and dynamic binding are
techniques that can overcome the deficiencies of conventional programming when
change is necessary. But encapsulation is the base on which a software IC
approach must be built:
"Encapsulation is the foundation of the whole approach. Its contribution is
restricting the effect of change by placing a wall of code around each piece
of data. All access to the data is handled by the procedures that were put
there to mediate access to the data." Just like IC design, "object-oriented
programming...is a way for suppliers to encapsulate functionality for delivery
to consumers."
But just like ICs, such components have to be bug free. We've all encountered
the programmer folk wisdom that "a fully debugged program is one that hasn't
failed recently." But we all want to believe that this is just cynicism, that
it is possible to build bug-free software components to seal up black boxes.
It's pleasant to imagine that a programming system could be built that meets
these desiderata, that by some reshuffling of the bits we could remold
programming nearer to our hearts' desire. That we could build to last with
solid blocks. At the moment it seems a poetic fancy.
Ah love! could you and I with Him conspire To grasp this sorry scheme of
things entire, Would not we shatter it to bits -- and then Re-mould it nearer
to the Heart's Desire!
I suppose we would. But this was Fitzgerald's third version of the stanza; he
continued to issue updates and bug fixes over a period of 20 years. Cox points
further to the need for assigning responsibility in human-computer systems,
something that makes no sense without bug free, absolutely reliable software
components. Is object-oriented programming the solution? "Absolutely not," Cox
says. Is there a solution?



The German Has a Word for It


"Neuralnetworks."
It was almost the first word I heard when I picked up the phone that morning
at I know not what hour. Pedants will say that "neural networks" is two words,
but it sounded like one word to me, which it may be in German, although, as
the fog cleared, I realized that the speaker, while indeed German, was
speaking English.
It was my friend JurgenFey, an editor for PC Magazin, a German affiliate of
DDJ. Jurgen's English is excellent, but some of what he was saying skimmed
over my befogged head that morning.
I did get it that Jurgen was putting together a special Neural Networks issue
of PC Magazin. None better for the job, I thought, since Jurgen has been
deeply immersed in neural network research for over a year now. He's done
hardware and software development in support of neural net systems, and has
read extensively in the theory of neural nets. Jurgenthinks that neural nets
have a lot of potential, but no, he doesn't see neural nets as the ultimate
answer.
As part of his work in putting together the special issue, he told me, he had
ten calls to make that day, all to the United States. For some reason, he had
started with me. I'm sure the other nine people had to offer him more, and I
have no hesitation in recommending the issue to anyone interested in neural
networks. It should be on the stands in Germany in mid-March, and it can be
ordered through M&T Publishing. "Of course, it's all in German,"
Jurgenapologized, with the usual pause in which he allows me to reflect upon
the linguistic shortcomings of Americans in general and me in particular.
Jurgen had no hesitation in recommending to me an excellent article on new
neuron models, though, of course, it was in German. I think he said that he'd
summarize it for me when he comes over in the spring. I know he said that a
remarkable amount of neural network work (he must have said it better than
that) takes the McCulloch-Pitts neuron model as gospel. In fact, McCulloch and
Pitts didn't present the model very seriously back in 1943, and anyway,
"something must have happened in the last 40 years."
He's right about McCulloch and Pitts.
Warren McCulloch and Walter Pitts published the seminal paper on neural nets
in "The Bulletin of Mathematical Biophysics" in 1943. The paper was titled "A
logical calculus of the ideas immanent in nervous activity," and it really was
about logic, not biophysics. McCulloch and Pitts departed from physiological
concerns to examine what the physiology might be doing -- what the hardware
might be computing. To do so, they presented a mathematical model of the
neuron.
The model they presented described the neuron as a fixed-threshold binary
device. When its inputs exceeded some fixed threshold of activation, the
neuron fired, and that was it. The inputs could be excitatory or inhibitory,
but all excitatory inputs had the same weight and any inhibitory input had
effectively infinite weight. If an inhibitory input was active, the neuron did
not fire. Finally, time was quantized, so that a neuron summed its inputs and
responded during a phase of fixed length; neural activity was not, in the
model, continuous.
This model was sufficient to implement the propositional calculus, which meant
that combinations of neurons could model any finite (propositional) logical
expression. Since aspects of the model were based on actual neural research,
there were the implications that the brain could be understood as we
understand computers, and that computers could be built along the lines of
organization of the brain. A lot of work was predicated on the assumption that
the McCulloch-Pitts model of simple neurons connected in a complex net was a
useful computational model. A lot of work was also predicated on the
assumption that their simple neuronal model was correct.
The McCulloch-Pitts neuron model is now known to be quite wrong. Neurons are
not simple logic elements. Their action is not all-or-none, and they are
closer in function to voltage-to-frequency translators than to logic devices.
The simple model has proved pregnant for computer science, spawning a great
deal of neural nets research and some early application work that seems
promising. But for modeling the brain, and possibly for some purely
computational purposes as well, these new, truer models of neurons need to be
examined.
The new models are not simple binary threshold models. They add parameters to
the old McCulloch-Pitts neuron and more closely model real neural response.
There are even analog models.
The thought of building a complex system out of analog neurons made me
nervous. Analog technology always makes me uncomfortable; if it's ones and
zeros I have a hope of understanding, but analog is inherently inscrutable.
But the idea of analog artificial neurons was making me more uncomfortable,
and in explanation I can only offer Lee Felsenstein's theory. Felsenstein, who
created the Sol and Osborne 1 computers and the Pennywhistle modem, and many
other things, has a theory. The gist of it, as I brought it up through that
morning's fog, is that men build things because they can't have babies.
It's pointless to speculate about whether or not computers will ever be
intelligent. No one knows what "intelligent" means. But it seems to me likely
that computers will one day grow beyond their current tool status to become
entities to be dealt with at a level of interaction now reserved for other
people. I don't suppose we'll see it. If such artificial entities will one day
share the earth with Man, their birth is still a long way in the future.
But I couldn't help wondering that morning if the gestation period had begun.









































March, 1989
C PROGRAMMING


TINYCOMM Begets SMALLCOM




Al Stevens


Last month we added functions to our library of C tools to support serial
ports and modems and explained --ever so briefly --the basics of serial
communications. To illustrate the use of those tools and principles, we built
a communications program called TINYCOMM. That program uses none of the
window, menu, and help functions from our ongoing library collection. Its
presentation was focused instead on a terse demonstration of the use of the
serial and modem functions. This month the TINYCOMM program spawns an
offspring that is named SMALLCOM and that uses the window, menu, and data
entry tools from earlier columns to support the user interface. SMALLCOM has
more of the features found in a commercial communications program --features,
such as uploading and downloading files, a serial port configuration file that
can be changed from the program, an editor, hooks for a phone directory and
scripts, hooks for file transfer protocols and automatic recognition of, and
reaction to, the Hayes modem result codes.


SMALLCOM Source Code


The listings for SMALLCOM are Listing One, smallcom.c, page 131, Listing Two,
smallcom.prj, page 136, Listing Three, smallcom.mak, page 136, and Listing
Four, smallcom.lnk, page 136. Smallcom.c is the source code for the program.
In addition, you will need most of the library source programs published in
this column since September when the project began. Smallcom.prj is the Turbo
C project make file for building the program from the Turbo C environment. Set
the compact memory model and define these global macros either as #define
statements in window.h or within the Compiler Defines option (Alt-O/C/D) of
the Turbo C environment as shown here:
 TURBOC=1;MSOFT=2;COMPILER=TURBOC
Smallcom.mak and smallcom.lnk are the Microsoft C make file and linker command
file to build the program. They assume that Microsoft C is in the DOS
execution path, that the MSC libraries are in the \LIB subdirectory, and that
the LIB and INCLUDE environment variables are properly set.


Hooks


Earlier I mentioned hooks in the program. These hooks will be used in the
coming months to add a phone directory, on-line service scripts, and XModem
and Kermit file transfer protocols. The hooks are function pointers that
initially have NULL values. As we add features, we will initialize the
function pointers with the addresses of the functions for the features we want
to add. This hooking technique allows us to plan for expansion while
preserving most of the existing code.
The phone directory hook is executed by the directory menu selection. The
script processor hook is executed when an originating call makes connection
with the remote processor. We will decide later what a script process really
is. For now it is enough to know that when we need one, it will be there when
the call goes through. Scripts are typically related to specific online
services, so SMALLCOM will associate scripts with phone numbers in the
directory. The file transfer protocol hooks occur at two levels. The higher
hook is a call to a function that will allow the user to select a protocol for
an upload or download. The address of the function will be in the
select_transfer_protocol function pointer hook. That function must return a
subscript into an array of function pointers and will be provided later. There
is an array for uploads called up_protocol and one for downloads called
down_protocol. These two arrays will contain the addresses of the functions
that implement the various protocols. The first entries in both arrays are the
addresses of the ASCII protocol functions, which are included in this first
edition of SMALLCOM. Others will be added later.


More Communications Processes


To use the modem in ways needed for SMALLCOM, we must change its
initialization string from the value used last month. Modify the INITMODEM
definition in modem.h to this value:
AT&C1E-0M1S7=60S11=55V1X3S0=0\r
Look now at the end of Listing One, smallcom.c. There are some additional
functions for managing the modem and serial port. These functions were not
needed in last month's TINYCOMM program. You might want to move them into
serial.c and modem.c as appropriate. I will explain those functions here.
The testcarrier function uses the carrier macro to see if the modem has lost
the carrier detect signal. By testing this signal, the program can determine
that the remote processor has disconnected. First, though, you must configure
your modem for normal operation of the carrier detect signal. Many modems will
optionally assert this signal at all times regardless of the connection. See
if your modem has a dip switch to turn the signal off and use that as the
default option. Remember, not all modems are alike, and not everyone will have
their modems set up the same.
The waitforconnect function is used when the program is waiting for an
incoming call or waiting for an originating call to be answered. When an
incoming call occurs, the function can sense the baud rate of the caller and
adjust the local-baud rate accordingly. This process is made possible by the
result codes returned by the modem when it makes the connection.
The waitforconnect function calls the waitforresult function, which waits for
and returns the modem's result code. The modem returns CONNECT, CONNECT 1200,
or CONNECT 2400 strings depending on the caller's baud rate. For an originated
call, the modem will return NO CARRIER if the called party answers without a
carrier detect signal or NO ANSWER if the called party does not answer. The
waitforresult function translates these strings into integer result codes.
The waitforresult function calls the general-purpose waitforstring function.
This function will be important to us later when we get into script
processing. You pass this function the address of an array of characters, each
of which points to a string. The function watches the serial input stream to
see if the stream matches any of the strings, and returns the offset of the
matching string or -1 if the TIMEOUT value elapses before any match occurs.
This use of the waitforstring function uses an array of pointers to the
modem's result codes. waitforstring allows you to use the backslash as a wild
card character in the string arguments.


Help Windows


In December we introduced help windows into our tool collection and
illustrated their use by including the feature in the TWRP tiny word
processor. Help windows are recorded in a file named by the load_help function
and identified by help window mnemonics found in the data-entry screen FIELD
structures --the MENU structures --and the calls to the set_help function.
SMALLCOM includes these mnemonics, but I am not publishing the text for the
help windows. There is nothing more to be learned from more help text, so they
would only serve to use up valuable space in the magazine. You can customize
new help windows to your preference, or you can omit them. SMALLCOM includes
most of TWRP as its integrated editor, so you can copy December's twrp.hlp to
a file you will name smallcom.hlp and add SMALLCOM-specific help windows to
it. The mnemonics are found in the source code in smallcom.c.


Configuration File


When SMALLCOM is run, it looks for a file named smallcom.cfg. If that file
exists, SMALLCOM reads its contents into the variables that specify the
default serial port and modem parameters -- which port is being used, the
parity, the number of stop bits, the word length, the baud rate, whether pulse
or tone dialing is to be used, and what the default phone number is for calls
originated by SMALLCOM. If the file does not exist, the program uses the
values coded into those variables when SMALLCOM was compiled. The SMALLCOM
menu bar includes a pop-down menu that allows you to change all but the phone
number and to write everything including the phone number to a new copy of the
configuration file. As things stand right now, that allows you to configure
everything except the phone number.


The SMALLCOM Screen Format


SMALLCOM provides a large window for showing the text passed between
processors and uses the menu manager software from our window library. The
menu bar at the top of the screen has selections named "File," "Connect,"
"Parameters," "Editor," and "Directory." You can get to one of these
selections by pressing F10 and using the right and left arrow keys to move
back and forth among the pop-down menus, or you can press the Alt key along
with the first letter of the selection you want. The "File," "Connect," and
"Parameters" selections pop-down menus for further selections. The "Editor"
and "Directory" selections light up the selection on the menu bar and let you
press Enter to execute the text editor or phone directory. The bottom of the
screen has a status bar that shows what is going on. The "On/Off Line" message
tells if you are connected to a remote computer. The "Direct" message says you
have selected a direct, null-modem connection. If you are logging text, the
"Logging" message appears. If you are in the modem's answer mode waiting for a
call, the "Answering" message appears. While you are uploading or downloading
a file the "Uploading" or "Downloading" message appears with as much of the
file's path and name as can fit on the status bar. The current phone number --
the one that will be dialed by the "Place Call" selection on the "Connect"
menu -- is shown.

The "Parameters" pop-down menu allows you to set and change the serial port
and modem parameters. Their initial values are recorded in the smallcom.cfg
file. You use the Enter key to step through the valid settings for each
parameter. Any changes you make on the menu are in effect for as long as the
program is running. If you use the "Write Parameters" selection, the current
settings are written to the smallcom.cfg file and will be the default values
the next time you run the program.
You cannot communicate through SMALLCOM until you have connected with another
computer. The other computer can be running SMALLCOM, it can be an online
service or bulletin board system, or it can be a different communications
program such as Procomm. The "Connect" pop-down menu has selections for making
and breaking connections. When you select "Place Call," SMALLCOM dials the
current phone number (shown in the status line at the bottom of the screen on
the right side) and waits for the remote computer to answer. The "Answer Call"
selection puts the modem into answer mode. When a call comes in, the modem
will answer the phone and return a status message that tells the baud rate of
the caller. The "Hang Up" selection breaks the connection. The "Direct
Connection" command assumes that the connection is direct with a null modem
cable and no modem commands are involved.
Once one of these connections has been made, the cursor is in the data window,
and you can type messages and read the messages that arrive from the remote
computer. You can use the "File" pop-down menu to upload and download files
and to turn the system log on and off. The system log is a file named
smallcom.log that records everything sent and received by the program. If
smallcom.log exists when you turn the option on, new text is appended to the
file.


Echoes


When two computers converse across phone lines, one of them has placed a call
and the other has answered. The two roles are somewhat different. The caller
will expect the called system to echo any characters that the caller sends and
will not display characters locally as they are being typed. The called system
does not expect the caller to echo and thus displays its own characters as
they are typed. Therefore, SMALLCOM must remember whether it originated or
answered the call and not echo or echo accordingly. (An echo is the return of
the character just received. I derived this behavior empirically by observing
the behavior of other communications programs.) If you call a computer and
upload a text file, the answering computer will echo each character because it
does not know that you are not typing. If, however, the called computer is
told to download the file, it does not echo the characters you sent, because
it assumes that a file transfer is underway. These are the rules that SMALLCOM
obeys. Therefore, if the called computer is downloading and the caller is
typing, the caller will not display the characters on its screen because no
echo is being sent. Conversely, if the caller is uploading and the called
system is not downloading, the text is displayed at both locations during the
transfer. If the caller is uploading and the called system downloading,
neither computer displays the text. When you select the "Direct Connection"
mode, each computer displays its own keystrokes and does not echo anything
back to the other. Confused? So was I when I worked all this out.
In communication jargon these procedures are called half and full duplex
transmissions, and some communications programs let you configure for one or
the other. My objective was to let the program determine the proper mode based
on its recognition of the circumstances at hand.
These echo problems pertain to typing, and they pertain as well to the ASCII
file-transfer protocol because that protocol looks to the receiver just like
typing. If you don't tell the other computer that you are sending a file, it
doesn't see any difference. The ASCII transfer protocol is usually used to
send text messages that were prepared off line, but it can also be used to
upload files that do not have critical content. When you do that, the
receiving program knows a file transfer is in progress because you tell it to
download a file. When we get into the binary file-transfer protocols, no such
echo concerns will bother us because both computers must be fully aware of
what is going on.


Editor


The editor selection on the menu bar calls the SMALLCOM text editor, which is
an integrated version of the TWRP tiny word processor from December. All the
TWRP commands are available, and you can edit a text file of up to 800 lines.
You can call this editor while you are on or off line. It can be used to
browse messages that you downloaded or to compose answers. Usually you will
use it while you are off line to save connect charges.


Phone Directory


If you select the directory entry on the SMALLCOM menu bar, nothing happens
because that feature is stubbed out for now with a NULL in its hook function
pointer. Next month we will add the phone directory. It will allow to add,
change, and delete entries and select an entry as the current one to be
dialed. Until we have that feature, you must hard-code the phone number into
the PHONENO string in February's modem.c.


Discussions with Readers


Some readers find time to write me at the magazine or leave messages for me on
CompuServe. (My CIS ID is 71101,1262.) When a reader's question or comment
raises an issue or provokes a thought that might interest others, I will
address it here.
A reader asked why I was reinventing the wheel. Why another window package,
another help package, another menu manager, another editor, another
communications program? Why any of this indeed? My answer to him was that the
point of the "C Programming" column project is first to bring to you, the
readers-at-large, a collection of C language tools that you can use in your
applications, and second show by example how these tools are programmed in C.
I am not trying to replace any programs that you might already be using,
programs that do all and more of what this software does. If all you want is
what Procomm does, you should get Procomm. It's a good program, and it costs
much less than the time you will devote working with and learning the software
tools in this column. If, on the other hand, you want to learn how programs
like Procomm are developed and, at the same time, collect the tools that go
into such developments, then you are in the right place. Such lessons and
software tool collections are the backbone of this column and are consistent
with the 12-year legacy of DDJ.
One reader cleverly matched one of my crotchets with one of my programs and
suggested I practice what I preach. Being neither preacher nor teacher, I
practice what I practice and offer those practices as examples of things that
might benefit programmers. Sometimes my programming practices stray from the
disciplines held by the distant gurus as proper programming habits. Sometimes
I violate my own rules to get the job done. I do not attach as well to
dogmatism as I do to pragmatism.

_C PROGRAMMING COLUMN_
by Al Stevens


[LISTING ONE]

/* ------ smallcom.c ---------- */
#include <conio.h>
#include <stdio.h>
#include <mem.h>
#include <string.h>
#include <ctype.h>
#include <dos.h>
#include <stdlib.h>
#include "window.h"
#include "editor.h"
#include "menu.h"
#include "entry.h"
#include "serial.h"
#include "modem.h"
#include "help.h"

#define ANSWERTIMEOUT 60
#define MAXSTRINGS 15
#define carrier() (inp(MODEMSTATUS) & 0x80)
#define LOGFILE "smallcom.log"
#define HELPFILE "smallcom.hlp"

#define CFGFILE "smallcom.cfg"
#define ALT_P 153
#define ALT_C 174
#define CTRL_C 3
#define WILDCARD '?'
static union REGS rg;
static FILE *logfp, *uploadfp, *downloadfp, *cfg;
static int running=1,connected,answering,savebaud;
int filecount;
extern int direct_connection, TIMEOUT, inserting;
extern char spaces[];
extern struct wn wkw;
extern MENU *mn;
/* ---------- prototypes ----------- */
void fileedit(char *);
static void displaycount(void);
static void smallmenu(int);
static void logserial(int);
static int upload(int, int);
static int download(int, int);
static int call(int, int);
static int directory(int, int);
static int comeditor(int, int);
static answer(int, int);
static directcon(int, int);
static int loginput(int, int);
static int hangup(int, int);
static int quit(int, int);
static int prm(int, int);
static void loadp(void);
static int savep(int, int);
static void set_parameters(void);
static int get_filename(char *);
static void notice(char *);
static void statusline(void);
static void putch_window(int);
void upload_ASCII(FILE *);
void download_ASCII(FILE *);
int keyhit(void);
char *prompt_line(char *, int, char *);
void reset_prompt(char *, int);
static int testcarrier(void);
static int waitforconnect(void);
static void initcom(void);
int waitforresult(void);
int waitforstring(char **, int, int);
static void waitforcall(void);
static void resetline(void);
/* ------- the hook to the phone directory ---------- */
static void (*phone_directory)(void) = NULL;
/* ------- the hook to script processors ---------- */
void (*script_processor)(void); /* filled in by directory */
/* ------- hooks to file transfer protocols --------- */
static int (*select_transfer_protocol)(void) = NULL;
/* ----- up to five upload function pointers ----- */
static void (*up_protocol[5])(FILE *file_pointer) = {
 upload_ASCII, NULL, NULL, NULL, NULL
};
/* ----- up to five download function pointers ----- */

static void (*down_protocol[5])(FILE *file_pointer) = {
 download_ASCII, NULL, NULL, NULL, NULL
};
/* --------- Files menu ------------ */
static char *fselcs[] = {
 "Log Input On/Off",
 "Upload a File",
 "Download a File",
 "Quit",
 NULL
};
static char *fhelps[] = {"log","upload","download","quitcom"};
/* ----------- Connect menu -------------- */
static char *cselcs[] = {
 "Place Call",
 "Answer Call",
 "Hang up",
 "Direct Connection",
 NULL
};
static char *chelps[] = {"call","answer","hangup","direct"};
/* ---------- Parameters menu --------------- */
static char *pselcs[] = {
 "Com Port: ",
 "Baud Rate: ",
 "Data Bits: ",
 "Stop Bit(s): ",
 "Parity: ",
 "Mode of Dialing: ",
 "Write Parameters",
 NULL
};
static char *phelps[] = {"port","baud","wordlen","stopbits",
 "parity","dialmode","writecfg"};
/* ---------- menu selection function tables ----------- */
static int (*ffuncs[])() = {loginput,upload,download,quit};
static int (*cfuncs[])() = {call,answer,hangup,directcon};
static int (*pfuncs[])() = {prm,prm,prm,prm,prm,prm,savep};
static int (*efuncs[])() = {comeditor};
static int (*dfuncs[])() = {directory};
/* ------ horizontal prompt messages ---------- */
char fdesc[]="Message File Operations";
char cdesc[]="Connections to Remote Processor";
char pdesc[]="Set Communications Parameters for Program Start";
char edesc[]="Edit a Text File";
char ddesc[]="The SMALLCOM Telephone Directory";
/* ------- horizontal menu bar ----------- */
static MENU cmn [] = {
 {"File", fdesc, fselcs, fhelps, "ludq", ffuncs, 0},
 {"Connect", cdesc, cselcs, chelps, "pahd", cfuncs, 0},
 {"Parameters", pdesc, pselcs, phelps, "cbdspmw", pfuncs, 0},
 {"Editor", edesc, NULL, NULL, "e", efuncs, 0},
 {"Directory", ddesc, NULL, NULL, "d", dfuncs, 0},
 {NULL}
};
/* ------ filename data entry template and buffer ------- */
static char filename[65], filemask[65];
static FIELD fn_template[] = {
 {2,14,1,filename,filemask,NULL},

 {0}
};
/* ------ modem result codes ------- */
static char *results[] = {
 "\r\nOK\r\n",
 "\r\nCONNECT\r\n",
 "\r\nRING\r\n",
 "\r\nNO CARRIER\r\n",
 "\r\nERROR\r\n",
 "\r\nCONNECT 1200\r\n",
 "\r\nNO DIALTONE\r\n",
 "\r\nBUSY\r\n",
 "\r\nNO ANSWER\r\n",
 "\r\n\r\n",
 "\r\nCONNECT 2400\r\n",
 NULL
};
extern int COMPORT,PARITY,STOPBITS,WORDLEN,BAUD;
extern char DIAL[], PHONENO[];
/* ================ MAIN ================== */
void main(void)
{
 int c;
 char *mb;
 inserting = FALSE;
 load_help(HELPFILE);
 loadp();
 savebaud = BAUD;
 set_parameters();
 clear_screen();
 mb = display_menubar(cmn);
 establish_window(1,2,80,24,TEXTFG,TEXTBG,TRUE);
 statusline();
 initcom();
 gotoxy(2,2);
 while (running) {
 set_help("smallcom");
 testcarrier();
 if (keyhit()) {
 switch (c = getkey()) {
 case F10: smallmenu(0); break;
 case ALT_F: smallmenu(1); break;
 case ALT_C: smallmenu(2); break;
 case ALT_P: smallmenu(3); break;
 case ALT_E: smallmenu(4); break;
 case ALT_D: smallmenu(5); break;
 case CTRL_C:clear_window();
 wkw.wx = wkw.wy = 0;
 gotoxy(2,2);
 break;
 case ESC: quit(1,1);
 break;
 default: if (!(c & 0x80) && connected) {
 if (answering
 direct_connection)
 logserial(c=='\r'?'\n':c);
 writecomm(c);
 if (c == '\r')
 writecomm('\n');

 }
 break;
 }
 }
 if (input_char_ready()) {
 logserial(c = readcomm());
 if (answering)
 writecomm(c);
 }
 }
 if (connected)
 hangup(1,1);
 release_modem();
 restore_menubar(mb);
 delete_window();
 clear_screen();
}
/* ---------- execute the SMALLCOM menu --------- */
static void smallmenu(int n)
{
 window(1,25,80,25);
 gotoxy(1,1);
 cprintf(spaces);
 putch(' ');
 current_window();
 menu_select(cmn, n);
 set_parameters();
 statusline();
 gotoxy(wkw.wx+2, wkw.wy+2);
}
/* ------ Call menu command ------ */
static int call(hs, vs)
{
 if (!connected) {
 notice("Dialing");
 placecall();
 sleep(4);
 delete_window();
 if ((connected = waitforconnect()) == FALSE) {
 statusline();
 initmodem();
 }
 else if (script_processor)
 (*script_processor)();
 }
 return TRUE;
}
/* --------- Direct Connection menu command --------- */
static int directcon(hs, vs)
{
 direct_connection ^= 1;
 connected = direct_connection;
 return TRUE;
}
/* ------- Hangup menu command ------- */
static int hangup(hs, vs)
{
 if (connected) {
 notice("Hanging up");

 resetline();
 delete_window();
 }
 return TRUE;
}
/* --------- Quit menu command --------- */
static int quit(hs, vs)
{
 int c = 0;
 notice("Exit to DOS? ");
 c = getkey();
 delete_window();
 running = (tolower(c) != 'y');
 return TRUE;
}
/* -------- Log Input menu command -------- */
static int loginput(hs, vs)
{
 if (logfp == NULL)
 logfp = fopen(LOGFILE, "ab");
 else {
 fclose(logfp);
 logfp = NULL;
 }
 return TRUE;
}
/* ---------- Upload file menu command ---------- */
static int upload(hs, vs)
{
 int pr = 0;
 if (!connected) {
 error_message("Not connected");
 return FALSE;
 }
 if (uploadfp == NULL) {
 setmem(filename, sizeof filename - 1, ' ');
 setmem(filemask, sizeof filemask - 1, '_');
 if (get_filename(" Upload what file? ") != ESC) {
 if ((uploadfp = fopen(filename, "rb")) == NULL)
 error_message("Cannot open file");
 else {
 statusline();
 if (select_transfer_protocol)
 pr = (*select_transfer_protocol)();
 (*up_protocol[pr])(uploadfp);
 fclose(uploadfp);
 uploadfp = NULL;
 }
 }
 }
 return TRUE;
}
/* ------ upload a file with ASCII transfer protocol ----- */
void upload_ASCII(FILE *fp)
{
 int c;
 while ((c = fgetc(fp)) != EOF) {
 writecomm(c);
 displaycount();

 if (input_char_ready())
 logserial(readcomm());
 if (keyhit())
 if (getch() == ESC)
 break;
 if (!testcarrier())
 break;
 }
 filecount = 0;
 if (connected)
 writecomm(EOF);
}
/* ---------- Download file menu command ---------- */
static int download(hs, vs)
{
 int pr = 0, save_timeout;
 if (!connected) {
 error_message("Not connected");
 return FALSE;
 }
 setmem(filename, sizeof filename - 1, ' ');
 setmem(filemask, sizeof filemask - 1, '_');
 if (get_filename(" Download what file? ") != ESC) {
 downloadfp = fopen(filename, "wb");
 statusline();
 if (select_transfer_protocol)
 pr = (*select_transfer_protocol)();
 save_timeout = TIMEOUT;
 TIMEOUT = 60;
 (*down_protocol[pr])(downloadfp);
 TIMEOUT = save_timeout;
 fclose(downloadfp);
 downloadfp = NULL;
 }
 return TRUE;
}
/* ----- download a file with ASCII transfer protocol ----- */
void download_ASCII(FILE *fp)
{
 int c = 0;
 while (TRUE) {
 if (keyhit()) {
 if ((c = getkey()) == ESC)
 break;
 writecomm(c);
 if (!answering)
 logserial(readcomm());
 }
 c = readcomm() & 127;
 if (c == 0 c == 0x7f)
 break;
 fputc(c, fp);
 displaycount();
 if (!testcarrier())
 break;
 }
}
/* --- echo modem input and write to the log if selected --- */
static void logserial(int c)

{
 putch_window(c);
 if (logfp)
 fputc(c, logfp);
}
/* -------- read a file name ------------- */
static int get_filename(char *ttl)
{
 int rtn;
 establish_window(1,23,80,25,ENTRYFG,ENTRYBG,TRUE);
 window_title(ttl);
 gotoxy(3,2);
 cputs("File name:");
 rtn = data_entry(fn_template, TRUE, 1);
 delete_window();
 return rtn;
}
/* -------- small message ------------ */
static void notice(char *s)
{
 int lf = (80-strlen(s))/2-1;
 int rt = lf+strlen(s)+2;
 establish_window(lf,11,rt,13,HELPFG,HELPBG,TRUE);
 gotoxy(2,2);
 cputs(s);
}
/* ---- comm and modem parameter menu commands ----- */
static int prm(hs, vs)
{
 switch (vs) {
 case 1: COMPORT ^= 3; /* flip between 1 and 2 */
 break;
 case 2: BAUD *= 2; /* 110,150,300, */
 if (BAUD == 220) /* 600,1200,2400 */
 BAUD = 150;
 if (BAUD == 4800)
 BAUD = 110;
 break;
 case 3: WORDLEN ^= 0xf; /* flip between 7 and 8 */
 break;
 case 4: STOPBITS ^= 3; /* flip between 1 and 2 */
 break;
 case 5: if (++PARITY == 3) /* 0, 1, 2 */
 PARITY = 0;
 break;
 case 6: DIAL[3] = DIAL[3] == 'T' ? 'P' : 'T';
 break;
 default:
 break;
 }
 set_parameters();
 return FALSE;
}
/* ------ post the parameters into the menu display ------- */
static void set_parameters(void)
{
 static char *pars[] = {"None", " Odd", "Even"};
 static char *mode[] = {"Pulse", " Tone"};
 pselcs[0][strlen(pselcs[0])-1] = '0' + COMPORT;

 sprintf(&pselcs[1][strlen(pselcs[1])-4],"%4d",BAUD);
 pselcs[2][strlen(pselcs[2])-1] = '0' + WORDLEN;
 pselcs[3][strlen(pselcs[3])-1] = '0' + STOPBITS;
 sprintf(&pselcs[4][strlen(pselcs[4])-4],"%s",pars[PARITY]);
 sprintf(&pselcs[5][strlen(pselcs[5])-5],"%s",
 mode[DIAL[3]=='T']);
}
/* ------- load the configuration file ---------- */
static void loadp(void)
{
 if ((cfg = fopen(CFGFILE, "r")) != NULL) {
 fscanf(cfg,"%d %d %d %d %d %c %s",
 &COMPORT,&PARITY,&STOPBITS,&WORDLEN,&BAUD,&DIAL[3],
 &PHONENO[0]);
 fclose(cfg);
 }
}
/* ---------- Write Parameters menu command ---------- */
static int savep(hs, vs)
{
 cfg = fopen(CFGFILE, "w");
 fprintf(cfg, "%d %d %d %d %d %c %s",
 COMPORT,PARITY,STOPBITS,WORDLEN,BAUD,DIAL[3],PHONENO);
 fclose(cfg);
 initcom();
 return FALSE;
}
/* --------- Editor menu command --------------- */
static int comeditor(hs, vs)
{
 extern int MAXLINES, inserting;
 MAXLINES = 800;
 mn = NULL;
 fileedit("");
 inserting = FALSE;
 insert_line();
 return TRUE;
}
/* --------- Directory menu command --------------- */
static int directory(hs, vs)
{
 if (phone_directory) {
 mn = NULL;
 (*phone_directory)();
 }
 return TRUE;
}
/* ----------- display a status line ----------- */
static void statusline(void)
{
 char stat[81];
 static char *st = NULL;
 sprintf(stat,
 " %s Line %s %s %s %-12.12s %-14.14s F10:Menu",
 (connected ? " On" : "Off"),
 (direct_connection ? "Direct" : " "),
 (logfp ? "Logging" : " "),
 ((answering & !connected)
 ? "Answering " :

 uploadfp ? "Uploading " :
 downloadfp ? "Downloading" : " "),
 (uploadfpdownloadfp ? filename : " "),
 *PHONENO ? PHONENO : "No Phone #");
 st = prompt_line(stat, 25, st);
}
/* ------- write the file count into the status line ------- */
static void displaycount(void)
{
 filecount++;
 if ((filecount % 10) == 0) {
 window(1,25,80,25);
 textcolor(MENUFG);
 textbackground(MENUBG);
 gotoxy(50,1);
 cprintf("%5d", filecount);
 current_window();
 gotoxy(wkw.wx+2, wkw.wy+2);
 }
}
/* ----- write a one-liner prompt saving video memory ----- */
char *prompt_line(char *s, int y, char *t)
{
 if (t == NULL)
 if ((t = malloc(160)) != NULL)
 gettext(1,y,80,y,t);
 window(1,y,80,y);
 textcolor(MENUFG);
 textbackground(MENUBG);
 gotoxy(1,1);
 cprintf(spaces);
 putch(' ');
 gotoxy(1,1);
 cprintf(s);
 current_window();
 return t;
}
/* ------- reset the one-liner prompt line --------- */
void reset_prompt(char *s, int y)
{
 puttext(1,y,80,y,s);
 free(s);
}
/* -------- write a character to the user's window -------- */
static void putch_window(int c)
{
 gotoxy(wkw.wx+2, wkw.wy+2);
 switch (c) {
 case '\t': while (wkw.wx % 4)
 putch_window(' ');
 break;
 case '\b': if (wkw.wx)
 --wkw.wx;
 break;
 default: putch(c);
 wkw.wx++;
 if (wkw.wx < wkw.wd-2)
 break;
 case '\n': if (wkw.wy < wkw.ht-1)

 wkw.wy++;
 else {
 scroll_window(1);
 writeline(2, wkw.wy+2, spaces+1);
 }
 case '\r': wkw.wx = 0;
 break;
 }
 gotoxy(wkw.wx+2, wkw.wy+2);
}
/* ------------ wait for a call ------------ */
static void waitforcall(void)
{
 answercall();
 if ((connected = answering = waitforconnect()) == FALSE) {
 statusline();
 initmodem();
 }
}
/* ---- wait for a line connection, reset baud rate ---- */
static int waitforconnect(void)
{
 extern int BAUD;
 int baud = 0;
 while (baud == 0)
 switch (waitforresult()) {
 case 1: baud = 300; break; /* CONNECT */
 case 5: baud = 1200; break; /* CONNECT 1200 */
 case 10: baud = 2400; break; /* CONNECT 2400 */
 case 0: /* OK */
 case 2: break; /* RING */
 case 3: /* NO CARRIER */
 case 4: /* ERROR */
 case 7: /* BUSY */
 case 8: /* NO ANSWER */
 case -1: baud = -1; break; /* time-out */
 default: break; /* anything else */
 }
 if (baud != -1 && baud != BAUD) {
 savebaud = BAUD;
 BAUD = baud;
 initcomport();
 }
 return (baud != -1);
}
/* ---- wait for a modem result (0-10). -1 if timed out ---- */
int waitforresult(void)
{
 return waitforstring(results, ANSWERTIMEOUT, 0);
}
/* --------- wait for a string from the serial port -------- */
int waitforstring(char *tbl[], int wait, int wildcard)
{
 int c, i, done = FALSE;
 char *sr[MAXSTRINGS];
 for (i = 0; tbl[i] != NULL; i++)
 sr[i] = tbl[i];
 while (!done) {
 set_timer(wait);

 while (!input_char_ready()) {
 if (timed_out())
 return -1;
 if (keyhit())
 if ((c = getkey()) == ESC)
 return -1;
 }
 logserial(c = readcomm());
 for (i = 0; tbl[i] != NULL; i++) {
 if (c==*(sr[i]) 
 (wildcard && *(sr[i])==wildcard)) {
 if (*(++(sr[i])) == '\0') {
 done = TRUE;
 break;
 }
 }
 else
 sr[i] = tbl[i];
 }
 }
 return i;
}
/* ----- initialize from serial and modem parameters ----- */
static void initcom(void)
{
 notice("Initializing Modem");
 initmodem();
 delete_window();
}
/* ----- test carrier detect -------- */
static int testcarrier(void)
{
 if (!direct_connection && connected && carrier() == FALSE)
 resetline();
 return connected;
}
/* ------ disconnect and reestablish the serial port ------ */
static void resetline(void)
{
 answering = connected = FALSE;
 statusline();
 disconnect();
 BAUD = savebaud;
 initcomport();
}
/* --------- answer a call ----------- */
static int answer(hs, vs)
{
 answering = 1;
 statusline();
 gotoxy(wkw.wx+2, wkw.wy+2);
 waitforcall();
 return TRUE;
}
#if COMPILER==TURBOC
/* --------- use bios to test for a keystroke -------- */
int keyhit(void)
{
 rg.h.ah = 1;

 int86(0x16, &rg, &rg);
 return ((rg.x.flags & 0x40) == 0);
}
#endif





[LISTING TWO]

smallcom (serial.h,modem.h,editor.h,window.h,menu.h,entry.h,help.h)
editshel (editor.h, menu.h, entry.h, help.h, window.h)
editor (editor.h, window.h)
help (help.h, window.h)
modem (serial.h, modem.h)
serial (serial.h)
entry (entry.h, window.h)
menu (menu.h, window.h)
window (window.h)







[LISTING THREE]

#
# SMALLCOM.MAK: make file for SMALLCOM.EXE with Microsoft C/MASM
#

.c.obj:
 cl /DMSOFT=1 /DTURBOC=2 /DCOMPILER=MSOFT -c -W3 -Gs -AC $*.c

smallcom.obj : smallcom.c serial.h modem.h menu.h entry.h \
 help.h window.h

modem.obj : modem.c serial.h modem.h

serial.obj : serial.c serial.h

entry.obj : entry.c entry.h window.h

menu.obj : menu.c menu.h window.h

help.obj : help.c help.h window.h

editshel.obj : editshel.c editor.h menu.h entry.h help.h \
 window.h

editor.obj : editor.c editor.h window.h

window.obj : window.c window.h

microsft.obj : microsft.c

vpeek.obj : vpeek.asm

 masm /MX vpeek;

keyhit.obj : keyhit.asm
 masm /MX keyhit;

smallcom.exe : smallcom.obj modem.obj serial.obj editor.obj \
 editshel.obj entry.obj menu.obj help.obj \
 window.obj keyhit.obj vpeek.obj microsft.obj
 link @smallcom.lnk








[LISTING FOUR]

smallcom+
modem+
serial+
entry+
menu+
editor+
editshel+
help+
window+
vpeek+
keyhit+
microsft
smallcom
nul
\lib\clibce




























March, 1989
GRAPHICS PROGRAMMING


Lines Galore




Kent Porter


If the lowly pixel is the foundation of computer graphics, the line is its
cornerstone. Now that we've figured out how to draw a pixel on the EGA/VGA
screen efficiently, it would seem that combining them to form lines is no big
deal. Not so, as we'll see this month in developing a couple of ways to draw
lines and some combinations thereof.
Here's the problem. A line moves between two end points in any conceivable
direction. Its path varies continuously, passing through an infinite number of
points. The number and positions of points on a display screen, however, are
fixed. Therefore the challenge in drawing a line is to find, for any given
point along its path, the nearest fixed point on the display. A line is thus
represented by lighting the series of pixels lying closest to its path, as
Figure 1 illustrates.
The simplest way to approximate a line is to calculate the series of points
using the linear equation
 y = mx + b
where m is the slope (the amount of vertical change per x increment) and b is
the y- intercept (the point where the line crosses the y axis). When m and b
are known, you plug in any x and round the result to get the nearest y.
Unfortunately, this method yields unacceptable performance because m is a
floating-point value.
You have to use integer arithmetic if you want fast line-drawing. In 1965, IBM
researcher J.E. Bresenham published an integer-based method that has become a
stock item in computer graphics.
The Bresenham algorithm represents the slope with two integers, called the
diagonal and nondiagonal increments, which have opposite signs. As the line
progresses, they are applied to a pixel selection variable generally
represented by the symbol d. The nondiagonal increment is applied when the
nearest pixel is in the current row, the diagonal increment is applied when
it's in an adjacent row. The sign of d indicates when the row changes, and
thus selects the closest pixel. When d is negative, the pixel above is
selected and the row doesn't change, and when d is positive, diagonal movement
occurs to the row below.
The terms "row," "above," and "below" are only conceptual. The signs of the
control variables allow the Bresenham algorithm to draw right to left, thus
mathematically turning the display upside down. And when the major axis of
motion is along the y axis, Bresenham mathematically rotates the display 90
degrees, one way or the other, according to the drawing direction.
That's the purpose of the code between lines 13 and 38 in Listing One, page
137, which implements the Bresenham algorithm. The result is an unbroken line
representation consisting of linearly-and diagonally-adjacent pixels, drawn
using only three integer addition operations in the loop starting on line 40.
Add the draw_line( ) function from Listing One to your copy of GRAFIX.C and
the prototype to GRAFIX.H. In order to use the function, you must recompile
GRAFIX.C and replace it in the GRAFIX.LIB file. (Note: if you purchase the DDJ
companion diskette or download the code for this article from CompuServe, you
will receive complete new copies of the source files containing this and other
routines added this month.)
Listing Two (SPOKES.C), page 137, is a sample program that puts the Bresenham
routine to work. SPOKES.C draws multicolor lines emanating from the center of
the EGA display.
Programming in general develops functionality much as a brick layer builds a
wall: by stacking layers of bricks (lower-level functions) according to a plan
and fastening them together with the mortar of program logic. Nowhere is this
more prevalent than in graphics. So far, we've made away to write pixels and
used that routine to draw lines. Now we'll add another level to the hierarchy
with a couple of functions that draw multiple lines.
The first is the rectangle, an object occurring so often that it needs its own
function. We can describe a rectangle in terms of its width and height
relative to the upper left corner. Thus the rectangle function in Listing
Three page 137, is called with draw_rect (left, top, width, height). The logic
of the routine combines the arguments in various ways and passes them along to
the four invocations of draw_line( ) needed to draw the edges.
The other high-level function, also shown in Listing Three, is more complex.
It draws a series of joined line segments and for that reason has the name
polyline( ). You can use it to draw closed object such as a star or an open
object such as the zigzag data representation of a line chart.
If a polyline has n edges, you need n+1 vertices to represent it. For example,
a line from A to B to C consists of two segments or edges: A to B, and B to C.
The vertices are the end points A, B, and C. The polyline( ) function thus
requires two arguments. The first is the number of edges, and the other is an
array of vertices. The vertex array consists of coordinate pairs, where v[O] =
xA, v[1] = yA, v[2] = xB, v[3] = yB, and so on. Consequently, the size of the
array is (n+1)*2 integers: in the case of polyline ABC, six elements.
The internal flow of polyline( ) advances along the vertices like an inchworm,
drawing from the previous vertex to the current vertex. When the current
vertex exceeds the number of edges plus one, the routine is finished.
Add the two routines from Listing Three to GRAFIX.C and their prototypes to
GRAFIX.H, then recompile and replace the module in GRAFIX.LIB. You can then
run the two small demo programs from Listings Four (BOXES.C) and Five
(STAR.C), page 137, which exercise the new functions by drawing a series of
rectangles (BOXES.C) and a five-pointed star.
The Bresenham algorithm --venerated though it is for its efficiency --still
draws only one pixel at a time. There isn't an alternative when drawing lines
that move in any direction. For strictly horizontal lines, however, we can use
a special hardware feature to write up to eight pixels at a time.
The 6845 video controller's Write Mode 0 enables you to update an entire byte
on each plane --that is, eight pixels --with one operation. The only
stipulation is that all pixels must receive the same index value filtered by
the same bitmask. In other words, the eight pixels are treated identically in
a single operation. Updating eight pixels simultaneously is actually more
efficient than updating one using Write Mode 2, used in the draw_point( )
routine introduced last month. This is because it's not necessary to shift the
data mask in order to identify the individual pixel being written. In fact,
Write Mode 0 incurs almost exactly the same overhead for eight pixels that
Write Mode 2 incurs for one. The result is blazing throughput.
The hline( ) routine in Listing Six (HLINE.ASM), page 138, uses Write Mode 0
to deliver over 580,000 pixels per second on a 10-MHz AT clone when all lines
completely fill each byte. The routine is slowed slightly if there are odd
pixels on each end, because hline( ) must construct the masks required to
update only the affected pixels. Still, worst-case performance is in the area
of 450,000 pixels per second, which amounts to an 18X to 23X improvement over
Bresenham.
Like draw_point( ), hline( ) is written in assembly language for maximum
efficiency. Its public symbol has an underscore prefix to comply with C
linkage conventions. Add the prototype
void far hline (int x, int y, int length);
to your copy of GRAFIX.H, then assemble this routine and add it to GRAFIX.LIB
using a librarian.
Note that hline( ) uses several auto variables allocated on the stack.
Arguments are at positive offsets from the BP register; for local variables,
subtract the sum of their sizes from the SP register, then offset negatively
from BP to reach each variable, as illustrated by the EQUate directives. This
is precisely the same method used by most language compilers for allocating
variables local to subroutines. When exit processing moves the BP register
contents back into SP, the auto variables are automatically removed from the
stack.
A horizontal line lacks flexibility, of course, but it's very useful in
creating solids. A case in point is a filled rectangle; simply loop through a
sequence of Y coordinates, repeatedly drawing a line of the same length.
Listing Seven page 141, gives the GRAFIX function fill_rect( ) for doing this.
Add it to GRAFIX.C and its prototype to 7.
The program COLORS.C in Listing Eight, page 141, uses fill_rect( ) to display
the default 16-color palette of the EGA and VGA. This program fills 90 percent
of the 224,000-pixel screen, yet thanks to hline( ), it takes less than half a
second on the AT. As you see, drawing a line --a fundamental object in
computer graphics --isn't quite as trivial as it seems at first glance. Now
that we've developed the basic algorithms and the beginnings of a hierarchy of
graphics primitives, we can start writing serious graphics programs, and
that's what we'll continue doing over the coming months.

_GRAPHICS PROGRAMMING COLUMN_
by Kent Porter


[LISTING ONE]


 1 void far draw_line (int x1, int y1, int x2, int y2)
 2 /* Bresenham line drawing algorithm */
 3 /* x1, y1 and x2, y2 are end points */
 4 {
 5 int w, h, d, dxd, dyd, dxn, dyn, dinc, ndinc, p;
 6 register x, y;
 7
 8 /* Set up */
 9 x = x1; y = y1; /* start of line */
 10 w = x2 - x1; /* width domain of line */
 11 h = y2 - y1; /* height domain of line */
 12

 13 /* Determine drawing direction */
 14 if (w < 0) { /* drawing right to left */
 15 w = -w; /* absolute width */
 16 dxd = -1; /* x increment is negative */
 17 } else /* drawing left to right */
 18 dxd = 1; /* so x incr is positive */
 19 if (h < 0) { /* drawing bottom to top */
 20 h = -h; /* so get absolute height */
 21 dyd = -1; /* y incr is negative */
 22 } else /* drawing top to bottom */
 23 dyd = 1; /* so y incr is positive */
 24
 25 /* Determine major axis of motion */
 26 if (w < h) { /* major axis is Y */
 27 p = h, h = w, w = p; /* exchange height and width */
 28 dxn = 0;
 29 dyn = dyd;
 30 } else { /* major axis is X */
 31 dxn = dxd;
 32 dyn = 0;
 33 }
 34
 35 /* Set control variables */
 36 ndinc = h * 2; /* Non-diagonal increment */
 37 d = ndinc - w; /* pixel selection variable */
 38 dinc = d - w; /* Diagonal increment */
 39
 40 /* Loop to draw the line */
 41 for (p = 0; p <= w; p++) {
 42 draw_point (x, y);
 43 if (d < 0) { /* step non-diagonally */
 44 x += dxn;
 45 y += dyn;
 46 d += ndinc;
 47 } else { /* step diagonally */
 48 x += dxd;
 49 y += dyd;
 50 d += dinc;
 51 }
 52 }
 53 }





[LISTING TWO]

/* SPOKES.C: Bresenham demo */
/* Multicolored spokes emanate from center of screen */

#include <conio.h>
#include <stdio.h>
#include "grafix.h"

#define CX 320 /* Center of screen */
#define CY 175

void main()

{
int x, y, color = 1, next (int);

 if (init_video (EGA)) {

 /* Spokes from center to top and bottom */
 for (x = 0; x <= 640; x += 80) {
 color = next (color);
 draw_line (x, 0, CX, CY);
 color = next (color);
 draw_line (x, 349, CX, CY);
 }

 /* Spokes from center to sides */
 for (y = 70; y < 350; y += 70) {
 color = next (color);
 draw_line (639, y, CX, CY);
 color = next (color);
 draw_line (0, y, CX, CY);
 }
 getch(); /* Wait for a keystroke */
 } else
 puts ("EGA not present in system: program ended");
} /* ------------------------ */

int next (int hue) /* set/return next color */
{
 set_color1 (hue++);
 return ((hue > 15) ? 1 : hue); /* wrap around */
}







[LISTING THREE]

 1 void far draw_rect (int xleft, int ytop, int w, int h)
 2 /* Draw outline rectangle in color1 from top left corner */
 3 /* w and h are width and height */
 4 /* xleft and ytop are top left corner */
 5 {
 6 draw_line (xleft, ytop, xleft+w, ytop); /* top */
 7 draw_line (xleft+w, ytop, xleft+w, ytop+h); /* right */
 8 draw_line (xleft+w, ytop+h, xleft, ytop+h); /* bottom */
 9 draw_line (xleft, ytop+h, xleft, ytop); /* left */
 10 } /* ------------------------------------------------------ */
 11
 12 void far polyline (int edges, int vertex[])
 13 /* Draw multipoint line of n edges from n+1 vertices where: */
 14 /* vertex [0] = x0 vertex [1] = y0 */
 15 /* vertex [2] = x1 vertex [3] = y1 */
 16 /* etc. */
 17 {
 18 int x1, y1, x2, y2, v;
 19
 20 x1 = vertex[0];

 21 y1 = vertex[1];
 22 for (v = 2; v < (edges+1)*2; v+= 2) {
 23 x2 = vertex[v];
 24 y2 = vertex[v+1];
 25 draw_line (x1, y1, x2, y2);
 26 x1 = x2;
 27 y1 = y2;
 28 }
 29 } /* ------------------------------------------------------ */






[LISTING FOUR]

/* BOXES.C: Demo of draw_rect() in GRAFIX.LIB */
/* Draws a big rectangle and four smaller ones */
/* K. Porter, DDJ Graphics Programming Column, March 89 */
/* ---------------------------------------------------- */

#include <conio.h>
#include "grafix.h"

main ()
{
 if (init_video (EGA)) {
 set_color1 (15);
 draw_rect (100, 100, 440, 230);

 set_color1 (14);
 draw_rect (110, 110, 420, 30);

 set_color1 (13);
 draw_rect (110, 105, 220, 220);

 set_color1 (12);
 draw_rect (340, 150, 190, 100);

 set_color1 (11);
 draw_rect (340, 260, 190, 60);

 getch(); /* wait for keystroke */
 }
}






[LISTING FIVE]

/* STAR.C: Draws a star using polyline */

#include <conio.h>
#include "grafix.h"


int vert [] = { /* vertices of star */
 320, 60, 420,280, 150,140,
 490,140, 220,280, 320, 60
};

void main ()
{
 if (init_video (EGA)) {
 polyline (5, vert); /* draw */
 getch(); /* wait for key */
 }
}






[LISTING SIX]

; HLINE.ASM: Fast horizontal line drawing routine
; Uses 6845 Write Mode 0 to update 8 pixels at a time on EGA/VGA
; C prototype is
; void far hline (int x, int y, int length_in_pixels);
; Writes in current color1 from GRAFIX.LIB
; Microsoft MASM 5.1
; K. Porter, DDJ Graphics Programming Column, March 89

.MODEL LARGE
.CODE
 PUBLIC _hline
 EXTRN _color1 : BYTE ; Current palette reg for pixel
 EXTRN _draw_point : PROC ; Pixel writing routine
 EXTRN _vuport : WORD ; far ptr to vuport structure

; Declare arguments passed by caller
 x EQU [bp+6]
 y EQU [bp+8]
 len EQU [bp+10]

; Declare auto variables
 last EQU [bp- 2] ; Last byte to write
 solbits EQU [bp- 4] ; Mask for start of line
 oddsol EQU [bp- 6] ; # odd bits at start of line
 eolbits EQU [bp- 8] ; Mask for end of line
 oddeol EQU [bp-10] ; # odd bits at end of line
; ----------------------------

_hline PROC FAR ; ENTRY POINT TO PROC
 push bp ; entry processing
 mov bp, sp
 sub sp, 10 ; make room for auto variables
 xor ax, ax ; initialize auto variables
 mov last, ax
 mov solbits, ax
 mov oddsol, ax
 mov eolbits, ax
 mov oddeol, ax


; Do nothing if line length is zero
 mov bx, len ; get line length
 cmp bx, 0 ; length = 0?
 jnz chlen ; if not, go on
 jmp quit ; else nothing to draw

; Call draw_point() with a loop if line length < 8
chlen: cmp bx, 8
 jnb getvp ; go if len >= 8
 mov ax, y ; get args
 mov cx, x
drpt: push bx ; push remaining length
 push ax ; push args to draw_point()
 push cx
 call _draw_point ; draw next pixel
 pop cx ; clear args from stack
 pop ax
 pop bx ; fetch remaining length
 inc cx ; next x
 dec bx ; count pixel drawn
 jnz drpt ; loop until thru
 jmp quit ; then exit

; Point ES:[BX] to vuport structure
getvp: mov ax, _vuport+2 ; get pointer segment
 mov es, ax
 mov bx, _vuport ; get offset

; Clip if starting coordinates outside viewport
 mov cx, y ; get y
 cmp cx, es:[bx+6] ; is y within viewport?
 jl checkx ; ok if so
 jmp quit ; else quit
checkx: mov ax, x ; get x
 cmp ax, es:[bx+4] ; is x within viewport?
 jl remap ; ok if so
 jmp quit ; else quit

; Map starting coordinates to current viewport
remap: add ax, es:[bx] ; offset x by vuport.left
 mov x, ax ; save remapped X
 add cx, es:[bx+2] ; offset y by vuport.top
 mov y, cx ; save remapped Y

; Clip line length to viewport width
 mov ax, es:[bx+4] ; get vuport.width
 sub ax, x ; maxlength = width - starting x
 add ax, es:[bx] ; + vuport.left
 cmp ax, len ; if maxlength > length
 jg wm0 ; length is ok
 mov len, ax ; else length = maxlength

; Set 6845 for write mode 0, all planes enabled, color selected
wm0: mov dx, 03CEh
 mov ax, 0005h ; Set write mode
 out dx, ax
 mov ax, 0FF00h ; Set/Reset reg, enable all planes
 out dx, ax
 mov ax, 0FF01h ; Enable set/reset reg, all planes

 out dx, ax
 mov dx, 03C4h ; 6845 address reg
 mov al, 2 ; Data reg
 mov ah, _color1 ; Palette reg planes enabled
 out dx, ax ; Set color code

; Compute x coord for last byte to be written
 mov bx, x ; get start of line
 add bx, len ; end = start + length
 mov cx, bx
 and cx, 0FFF8h ; x coordinate where odd bits
 mov last, cx ; at end of line begin

; Compute number of odd pixels at end of line
 sub bx, cx
 mov oddeol, bx ; save it

; Construct pixel mask for last byte of line
 cmp bx, 0
 jz bsol ; go if no odd pixels
 xor ax, ax
eolb: shr ax, 1 ; shift right and
 or ax, 80h ; set H/O bit
 dec bl ; until mask is built
 jnz eolb
 mov eolbits, ax ; then save mask

; Compute number of odd pixels at start of line
bsol: mov cx, x ; get starting X again
 and cx, 7 ; # of pixels from start of byte
 jz saddr ; go if none
 mov bx, 8
 sub bx, cx ; # of pixels to write
 mov oddsol, bx ; save

; Construct pixel mask for first byte of line
 xor ax, ax
solb: shl ax, 1 ; shift left and
 or ax, 1 ; set L/O bit
 dec bl ; until mask is built
 jnz solb
 mov solbits, ax ; then save mask

; Translate last byte X into an address
saddr: mov ax, 0A000h
 mov es, ax ; ES ==> video buffer
 mov bx, y ; get row
 mov ax, 80
 mul bx
 mov bx, ax ; BX = row offset = row * 80
 push bx ; save row offset
 mov ax, last ; get last byte X
 mov cl, 3
 shr ax, cl ; shift for col offset
 add bx, ax ; last offs = row offs + col offs
 mov last, bx

; Compute address of first byte (ES:[BX])
 pop bx ; fetch row offset

 mov ax, x ; get col offset
 mov cl, 3
 shr ax, cl ; shift right 3 for col offset
 add bx, ax ; offset = row offs + col offs
 cmp bx, last ; is first byte also last?
 jz weol ; skip to end mask if so

; Write start of line
 mov dx, 03CEh ; 6845 port
 mov ah, solbits ; start-of-line mask
 cmp ah, 0
 jz w8 ; go if empty mask
 mov al, 8 ; set bit mask reg
 out dx, ax
 mov cl, es:[bx] ; load 6845 latches
 mov ax, solbits
 neg al ; complement
 dec al ; for reversed bit mask
 and al, cl ; filter previously unset pixels
 mov es:[bx], al ; clear affected bits
 mov al, _color1
 mov es:[bx], al ; set affected bits
 inc bx ; next byte
 cmp bx, last ; ready for end of line yet?
 jae weol ; go if so

; Write 8 pixels at a time until last byte in line
w8: mov ax, 0FF08h ; update all pixels in byte
 out dx, ax ; set bit mask reg
 mov al, es:[bx] ; load 6845 latches
 xor al, al
 mov es:[bx], al ; clear all pixels
 mov al, _color1
 mov es:[bx], al ; set all bits
 inc bx ; next byte
 cmp bx, last ; thru?
 jnz w8 ; loop if not

; Write end of line
weol: mov dx, 03CEh ; 6845 port
 mov ah, eolbits ; end-of-line mask
 cmp ah, 0
 jz rvc ; go if empty mask
 mov al, 8 ; set bit mask reg
 out dx, ax
 mov cl, es:[bx] ; load 6845 latches
 mov ax, eolbits
 neg al ; complement
 dec al ; for reversed bit mask
 and al, cl ; filter previously unset pixels
 mov es:[bx], al ; clear affected bits
 mov al, _color1
 mov es:[bx], al ; set affected bits

; Restore video controller to default state
rvc: mov dx, 03CEh
 mov ax, 0005h ; write mode 0, read mode 0
 out dx, ax
 mov ax, 0FF08h ; default bit mask

 out dx, ax
 mov ax, 0003h ; default function select
 out dx, ax
 xor ax, ax ; zero Set/Reset
 out dx, ax
 mov ax, 0001h ; zero Enable Set/Reset
 out dx, ax
 mov dx, 03C4h ; 6845 address reg
 mov ax, 0F02h ; Data reg, enable all planes
 out dx, ax

; End of routine
quit: mov sp, bp
 pop bp
 retf
_hline ENDP
 END






[LISTING SEVEN]

void far fill_rect (int xleft, int ytop, int w, int h)
/* Draw solid rectangle in color1 from top left corner */
{
register y;

 for (y = ytop; y < ytop+h; y++)
 hline (xleft, y, w);
} /* ------------------------------------------------------ */






[LISTING EIGHT]

/* COLORS.C: Shows all colors in default palette */

#include "grafix.h"
#include <conio.h>

void main ()
{
int r, c, color;

 if (init_video (EGA)) {
 for (r = 0; r < 4; r++)
 for (c = 0; c < 4; c++) {
 color = (r * 4) + c; /* next color */
 set_color1 (color);
 fill_rect ((c*160), (r*80), 158, 79);
 }
 getch(); /* wait for keypress */
 }

}





























































March, 1989
STRUCTURED PROGRAMMING


Text Screen Metrics




Jeff Duntemann, K16RA


Ahhh, Comdex. Recall the classic Rocky & Bullwinkle episode in which
Bullwinkle is interrogated by a crew of trenchcoated FBI types who speak only
in a jack Webb monotone? After a few questions, Bullwinkle begins to adopt the
same monotone, and the spooks take umbrage.
"You're trying to make fun of the way we talk?" one asks.
"No," replies Bullwinkle, "but it's catching!"
I thought of Bullwinkle after trudging past the eleven-hundredth booth full of
wool-suited Asian gentlemen selling power supplies and clone cases, each of
whom greeted me by asking, "Are you dealer?"
After awhile I had to choke back the impulse to reply, "No, I writer."


The Invasion of the Pie-Expanders


Comdex has always been primarily a hardware show, and in recent years has
become an increasingly Asian hardware show, hence my subliminal impulse to
deal with our Pacific Rim partners in their own syntax. Against constant
protectionist pressures from both left and right, I have to keep reminding
people that the Asian manufacturers keep making the pie bigger. The more
entry-level people who buy cheap Asian clones in 1989, the more experienced
users will buy higher-cost American (and generally more powerful) 80486
machines in 1991. Growth in our industry requires bringing 'em in at the
bottom and spreading computing skills throughout society. American companies
prefer to priceskim at the top, and go belly-up when the cream is exhausted
--leaving us no recourse but to keep generating cream. By doing us the favor
of manufacturing our industry's loss-leaders, our Asian partners are keeping
Compaq and Apple alive.
At Comdex, the hardware to gladden a programmer's heart was almost universally
American --and expensive. ATT Bell Labs was showing a research monitor --not
even a pre-prototype --with a monochrome resolution of 4096 by 4096 and a
16-Mbyte refresh buffer. 300 DPI video to match 300 DPI lasers. Maybe 1994.
International Meta Systems showed their Max2 30 MIPS accelerator board for
Digitalk's Smalltalk/V. The 4-Mbyte $8,000 system makes the Xerox Dorado
workstation I played with in 1983 look pretty sick --and doesn't lock you out
of mainstream DOS and OS/2 applications. (I'll have a lot more to say about
Smalltalk in future columns.)
I was astonished to see a Xerox-blessed implementation of Alan Kay's Dynabook
product there, offered by Scenario Inc. of Somerville Mass. The Dynabook, of
course, was the book-shaped, 100 percent flat-screen computer Xerox PARC
proposed in 1976 as the future of computing. You curl up in your cushy chair
and read it like a book, thumbing the screen to flip pages and touching
hot-link keywords to traverse a hypertext thread. No keyboard, although
Xerox's original design included one. Scenario's Dynabook has a CD-ROM reader
and will be positioned as a research tool, with a giant effort to get low-cost
($40) reference works and classics on CD-ROM. An amazing thing -- and only
$5,000.
My own personal Comdex favorites (perhaps because they're a realizable
fantasy) are Brier Technology's 20-Mbyte and 40-Mbyte 3.5-inch 70ms diskette
drives. The $30 diskettes are mechanically identical to PS/2 and Mac
diskettes, and the drives are no larger. Furthermore, the drives can detect
and read both 720K and 1.4-Mbyte diskettes. Two of these critters, plus a fast
internal 40-Mbyte hard drive are all a programmer's workstation would need for
backup, archiving, and multiple media compatibility. Brier showed at the
Datavue booth --which might give you a hint as to where the drives will first
appear. No price yet. Available '89.
Products like these hint at what our programmer's workstation will be in 1992.
I call it the Red Hot Shoebox, and the machine itself will be a minimal
swelling on the cable between the keyboard and the screen. But oh, what a
swelling....


Comdex Software


Hiding behind a mountain of clone motherboards were a few important programmer
tools. The best kept secret had to be GoScript, an impressive $195 PostScript
clone from LaserGo Inc. Feed it a megabyte of RAM and any supported
graphics-capable printer can print like a LaserWriter. With everybody else
skimming the cream, GoScript could become the Turbo Pascal of the PostScript
world. Sell Adobe, quick.
In the same general category are Maxem's Cause and Clarion Software's Clarion,
two application generators slanted toward vertical-market developers. Those of
you who make your living creating integrated dog-kennel management systems had
better abandon both C and Pascal --neither is necessary in the
transaction-dominated world of vertical markets, where everything's diskbound
to begin with and performance is not a compelling consideration. What either
product can do in a weekend could take weeks in traditional languages, with
results that are solid and attractive if not blazingly fast. Cause provides an
intriguing portability link between the PC and the Mac --probably the best
I've seen.
The sharpest tool I encountered at Comdex was not even "at" Comdex --the
company was not exhibiting and was only giving private demos to the programmer
press. SoftTools of Atlanta demonstrated CASE:W, and I can only characterize
it as the sword that will cut the Gordian knot of Microsoft's Windows/PM API.
Basically, CASE:W is a prototyping tool that does for Windows and PM what
Bricklin's Demo does for DOS text screens. Once you have interactively created
(via point-and-drag) your menus and menu trees, icons, and dialog boxes,
CASE:W compiles the prototype to a C source file containing all the hairy
stuff of a Windows application. Windows is an object-friendly environment that
dispatches event messages to an application, which must parse those events and
take action. Creating the event handler is the tough part, and CASE:W does all
that. The output source file is crisply arranged and well-commented. Even I,
who laid C aside years ago as the sorry mess it is, could pick up and read the
skeletal event handler.
CASE:W is a C tool for now, but nothing would prevent SoftTools (or some other
firm) from doing the same tool for Modula-2 or Pascal. Time saved: Awesome.
This is the future, guys. Bone up on CASE techniques and keep your eyes open.
In the thick of a recession, it could save your skin.


The Text Mode Point Spread


Last month I explained how to use BIOS calls to determine which of the several
IBM video adapters was installed in a given machine. Why is this useful? The
obvious reason is to determine which of the numerous graphics modes are legal
for any given board. But graphics entirely aside, there are numerous
text-oriented reasons for knowing which adapter is on the bus.
First and foremost is the matter of screen size. Most people who have
graduated to EGAs or VGAs now know that PC screens are not immutably limited
to 25 lines. The EGA can display 25-and 43-line screens. The VGA can display
25-, 28-, and 50-line screens. Most good programming environments (including
Turbo Pascal, TopSpeed Modula-2, and QuickBasic) can make use of the larger
screens, and with some care, your applications can too.
Screen size a la PC is a peculiar concept in that it is not mode-dependent but
font-dependent. Understand PC text fonts, and you'll get an understanding of
screen size along for free.
The first-generation video adapters, the CGA and MDA, have only one font in
ROM. The CGA font is an 8 x 8 font, meaning that each character in the font is
created from pixels in a matrix eight characters high by eight characters
wide. To leave space between horizontally adjacent characters, and to allow
for lower-case descenders on characters like p and q, the character patterns
are limited to a 7 x 7 pixel subset of the 8 x 8 pixel matrix.
The MDA font is an 8 x 14 font with a twist: The rightmost vertical column of
pixels within the font is duplicated to its own right, making the font look
like a 9 x 14 font. Since in all but the line-draw characters the rightmost
column is empty, the result is that the line-draw characters are "stretched"
so that they can touch one another in the horizontal direction, while allowing
a little extra horizontal space between normal characters. This is done with
electrical smoke and mirrors within the MDA's circuitry, and since the MDA's
font can't be modified there isn't much more to be said about it.
The fun starts when you consider the EGA. The EGA's default font is also 8 x
14, but without the MDA's ninth-inning stretch. In truth, the ASCII characters
in the EGA's default font are limited to a 7 x 9 pixel subset of the 8 x 14
matrix. There are three dead pixels below the font's baseline, and two dead
pixels above. This allows true 2-pixel descenders to be used on lower-case
characters, and also provides comfortable vertical spacing between character
rows on the screen. The characters themselves, however, are not much better
formed than the CGA's crude 7 x 7 patterns.
The EGA contains another font in ROM. This alternate font was included to
allow the EGA to fully emulate the CGA, and is thus identical to the CGA's 8 x
8 font. This font can be loaded in place of the 8 x 14 font, and when loaded,
whammo! Your screen suddenly contains 43 lines rather than 25.
How so? It's a question of allocating scan lines. In text mode, the EGA places
350-scan lines on the screen. If it allocates these scan lines to character
rows at 14-scan lines per row, you can fit 25 lines on the screen. On the
other hand, if you allocate the scan lines to character rows at 8-scan lines
per row, you get 43 lines and change. The "change" amounts to 6-scan lines --
about all the EGA ever exhibits of "overscan," what we used to call the "text
border" on the CGA. In the default mode, there is essentially no overscan. (So
everybody puh-leez stop asking me how to set the text border color on the
EGA!)
Changing fonts trips some additional logic within the EGA that tells the BIOS
at what line to scroll and all, but in the large it's only a question of
allocating a fixed number of scan lines in two different ways. The fonts are
the same width, so we differentiate them by their pixel height. This is
sometimes called the fonts' point size, but it has little to do with
traditional print measurement by points.


PS/2 Complications


In keeping with IBM's "extend but don't replace" policy, the VGA has
everything that the CGA and EGA have, and then some. In addition to both the
8- and 14-pixel fonts, the VGA has a 16-pixel font, which is its default font.
The VGA puts 400 scan lines on the screen in text mode. Allocate those lines
to a 16-pixel font and you get 25 lines. Allocate 400 lines to a 14-pixel
font, and you get 28 lines. Finally, allocate 400 lines to an 8-pixel font,
and you get 50 lines.
The MCGA, which is the VGA's developmentally-handicapped little brother, has a
stubborn BIOS that understands only the 16-pixel font and will refuse to
display other than a 25-line screen. There are rumored methods to club the
BIOS from behind and do the work yourself, but having neither an MCGA nor much
respect for it, I haven't explored further.

The secret to determining the number of lines currently on the screen (and
hence the size of the current screen buffer) is to determine what font is
current loaded. Once you've identified the resident adapter, that part is a
snap.


Numerous Enumerations


In my last column, I defined an enumerated type encompassing all the PC
display adapters.
AdapterType = (None,MDA,CGA,EGAMono, EGAColor,VGAMono,VGAColor,
MCGAMono,MCGAColor);
Given that there are only three legal font sizes, it makes sense to enumerate
them as well.
FontSize = (Font8, Font14, Font16);
Enumerated types help your code to document itself, and being 8-bit
quantities, they are small and fast compared to hanging string tags on things
like display adapters. The function we'll use to detect the current font,
GetFontSize, will return a value of type FontSize.
Most display adapters only support one font size, which can be reliably
hard-coded as part of a CASE statement. For the EGA and VGA, font size is
found through a BIOS call. Video interrupt 10H has a service that returns
various bits of information about the current video state in the registers.
The service is selected with 11H in AH and 30H in AL. On return, the number of
pixels in the current font will be found GetFontSize is contained within the
Turbo Pascal unit TextInfo in Listing One (textinfo.pas), page 142.
The number of lines on a screen is a function of the adapter type and the font
size. This suggests that a two-dimensional array of integers can be defined
such that indexing into the array with the adapter type and the font size will
yield the number of lines on a screen for that combination of adapter type and
font size. Both factors are present as enumerated types, and enumerated types
may index arrays in both Pascal and Modula-2.
This is the algorithm for the procedure GetTextBufferStats, also in Listing
One. Line counts are arranged in a table that comprises the beef in an array
constant. With some comments to show what the table rows and columns
represent, it's an example of creating code that is truly self-documenting, in
that the documentation is the code. The alternative is to thread through a
multiplynested IF statement that would admittedly save 54 bytes of data
segment but (in addition to being slower) would be a great deal harder to
read.


Override!


There's something else a little different about GetTextBufferStats. Its
CheckFor-Override parameter is a procedural type. Procedural types have always
existed in Modula-2 and have only recently been added to Turbo Pascal with
version 5.0. What happens inside GetTextBufferStats is this: The number of
characters in a line is returned by BIOS VIDEO service $F. The procedure then
uses Query-AdapterType and GetFontSize to derive a screen-line count through
BIOS-based methods that assume one of the IBM display adapters or an exact
clone.
The nature of the PC text video architecture, however, allows stretching
screen extents considerably. Assuming that a refresh buffer begins at either
$B000 (monochrome) or $B800 (color), it may be mapped any convenient way, say,
132 x 43 or 80 x 40. My own display is the absolutely stunning Genius VHR 401
display, which was a desktop publishing display (728 x 1004, portrait-style)
long before there was any such thing as desktop publishing. It has something
the other DTP displays don't have: a large-format monochrome text mode that
may be configured as 80 columns by the standard letter-paper-sized 66 lines,
or by 70, 76, or 83 lines as desired. Ten minutes with that tube will make you
wonder why we've settled for 25-line brain-damaged television sets for so long
--but don't get me started on that particular rant. (You'll hear it in this
column often enough in the future.)
The point is that there may be displays that support more than the
IBM-supported 80 columns and 50 rows -- or some oddball values in between.
132-column text displays exist for spreadsheet jockeys. And lord only knows
what can be done with a PCjr --a machine so bizarre that I don't even consider
it IBM-compatible. The extended text video devices that I've seen (including
the Genius) always "look like" some standard IBM board from a BIOS and control
level to keep some measure of compatibility. Only the character-to-refresh
buffer mapping differs. To support such creatures, you need to be able to
override the logic in GetTextBufferStats with custom logic supplied by you or
someone else. This can be done simply by executing an override procedure after
executing GetTextBufferStats but allowing GetTextBufferStats to call the
override procedure itself ensures that the job will be done.
The override procedure should perform its own display detection specific to
the oddball target display, and if the oddball display is found, the override
routine can replace the IBM-supported X and Y extents with the oddball
display's X and Y extents. GetTextBufferStats then calculates the screen size
using the overriding values. The override procedure NullOverride provided with
the TextInfo unit does nothing at all -- it simply lets the IBM-supported
values pass unchanged. I'll provide an example of a real override procedure
next month, specific to the Genius VHR display.
Note that its use of procedural parameters forces TextInfo to require
compilation with Turbo Pascal 5.0.


Odds 'n Ends


TextInfo exists to tell us important things about the machine's display in
text mode. As I alluded to earlier, there is the matter of the starting
address of the video refresh buffer. A simple rule applies here: Any video
board connected to a monochrome display keeps its buffer at $B000, and any
board connected to a color display keeps its buffer at $B800. The exceptions
are things like (blecch!) black-and-white TV sets, or those awful $69
composite video monochrome monitors from the dawn of PC time. Both connect
only to CGAs, however, so the buffer remains at $B800, monochrome
notwithstanding. Once your video board has been correctly detected, the
GetTextBufferOrigin function can return the buffer's address without fail.
Similarly, GetBIOSTextMode returns the current text mode with a single BIOS
call. This is handy when you want to move into a specific video mode (perhaps
one of the multitude of graphics modes) and still return to the text mode that
was in force when your application began running.
One thing TextInfo does not address is how many pages of display buffer memory
are available in the installed video board. The VGA has as many as 32, while
the poor MDA has only one. Some clones have more than IBM boards, some less.
The infuriating thing is that I have yet to discover a reliable means of
determining how many pages are available in the installed board by inspection.
You can assume that a CGA will have four pages ... and you could be wrong. The
clone business is like that. Any clues? This boy would like to know.
Listing Two (texttest.pas), page 143, is a short program that exercises
TextInfo. Listings Three and Four (textinfo.def and textinfo.mod), pages 143
and 144, are the TopSpeed Modula-2 implementation of TextInfo. (Both the
definition and implementation modules, which Turbo Pascal combines in one unit
file.) Listing Five (texttest.mod), page 146, is the test program in Modula-2.


Part of the Plan


One of the reasons I picked up this column is that my compatriot Kent Porter
has stepped aside to kick off his graphics column. In pondering that, it
occurred to me that while we spend a great deal of effort making our graphics
screens flashy and dynamic, with exploding icons and lord knows what else, our
text screens are content to look like something scraped off a 3278 mainframe
terminal. My first big effort here, consequently, will be to provide you with
tools to allow you to put some flash in your text displays as well. The fun's
only beginning. In the coming months I'll be tossing some INLINE directives,
machine code externals, and virtualized screen machines your way.
In the meantime, Mr. Byte and I are going to set the big scope out in the
driveway and catch the opposition of Jupiter, who always looks better with a
few belts. Drink in the night when you can, my friends --the stars are the
very best antidote for a frantic life.


Products Mentioned


GoScript --LaserGo Inc. 9235 Trade Pl, Ste. A Sand Diego, CA 92126
619-530-2400 $195
Dynabook --Scenario Inc. 235 Holland St. Somerville, MA 02144 617-625-1818
$4,995
Cause --Maxem Corp. 1550 E University Ave. Mesa, AZ 85203 602-827-8181 $495
CASE:W --SofTools Inc. One Dunwoody Park, Ste. 130 Atlanta, GA 30338
404-399-6236 $1,495
Clarion Prof. Developer --Clarion Software 150 E Sample Rd. Pompano Beach, FL
33064 305-785-4555 $695
Max2 30 MIPS --Accelerator board International Meta Systems 23844 Hawthorne
Blvd., Ste. 200 Torrance, CA 90505 213-375-4700 $5,695
BR3020 20-Mbyte, 3.5-inch diskette drive Brier Technology 2363 Bering Dr. San
Jose, CA 95131 408-435-8463 (price not set)

_STRUCTURED PROGRAMMING COLUMN_
by Jeff Duntemann


[LISTING ONE]



{--------------------------------------------------------------}
{ TextInfo }
{ }
{ Text video information library }
{ }
{ by Jeff Duntemann }
{ Turbo Pascal V5.0 }
{ Last update 11/20/88 }
{--------------------------------------------------------------}

UNIT TextInfo;

INTERFACE

USES DOS;


TYPE
 AdapterType = (None,MDA,CGA,EGAMono,EGAColor,VGAMono,
 VGAColor,MCGAMono,MCGAColor);

 FontSize = (Font8,Font14,Font16);

 { The following type definition *requires* Turbo Pascal 5.0! }
 OverrideProc = PROCEDURE(VAR ForceX : Byte; VAR ForceY : Byte);


VAR
 TextBufferOrigin : Pointer;
 TextBufferSize : Word;
 VisibleX,VisibleY : Byte;


FUNCTION GetBIOSTextMode : Byte; { Returns BIOS text mode }

FUNCTION GetFontSize : FontSize; { Returns font height code }

FUNCTION GetTextBufferOrigin : Pointer; { Returns pointer to text buffer }

{ Returns visible X and Y extent plus buffer size in bytes: }

PROCEDURE GetTextBufferStats(VAR BX : Byte;
 VAR BY : Byte;
 VAR BuffSize : Word;
 CheckForOverride : OverrideProc);

PROCEDURE NullOverride(VAR ForceX : Byte; VAR ForceY : Byte);

FUNCTION QueryAdapterType : AdapterType; { Returns installed display }

FUNCTION FontCode(Height : Byte) : FontSize; { Returns font height code }

FUNCTION FontHeight(Code : FontSize) : Byte; { Returns font height value}



IMPLEMENTATION



FUNCTION GetBIOSTextMode : Byte;

VAR
 Regs : Registers; { Type Registers is exported by the DOS unit }

BEGIN
 Regs.AH := $0F; { BIOS VIDEO Service $F: Get Current Video Mode }
 Intr($10,Regs);
 GetBIOSTextMode := Regs.AL; { Mode is returned in AL }
END;



FUNCTION QueryAdapterType : AdapterType;

VAR
 Regs : Registers; { Type Registers is exported by the DOS unit }
 Code : Byte;

BEGIN
 Regs.AH := $1A; { Attempt to call VGA Identify Adapter Function }
 Regs.AL := $00; { Must clear AL to 0 ... }
 Intr($10,Regs);
 IF Regs.AL = $1A THEN { ...so that if $1A comes back in AL... }
 BEGIN { ...we know a PS/2 video BIOS is out there. }
 CASE Regs.BL OF { Code comes back in BL }
 $00 : QueryAdapterType := None;
 $01 : QueryAdapterType := MDA;
 $02 : QueryAdapterType := CGA;
 $04 : QueryAdapterType := EGAColor;
 $05 : QueryAdapterType := EGAMono;
 $07 : QueryAdapterType := VGAMono;
 $08 : QueryAdapterType := VGAColor;
 $0A,$0C : QueryAdapterType := MCGAColor;
 $0B : QueryAdapterType := MCGAMono;
 ELSE QueryAdapterType := CGA
 END { CASE }
 END
 ELSE
 { If it's not PS/2 we have to check for the presence of an EGA BIOS: }
 BEGIN
 Regs.AH := $12; { Select Alternate Function service }
 Regs.BX := $10; { BL=$10 means return EGA information }
 Intr($10,Regs); { Call BIOS VIDEO }
 IF Regs.BX <> $10 THEN { BX unchanged means EGA is NOT there...}
 BEGIN
 Regs.AH := $12; { Once we know Alt Function exists... }
 Regs.BL := $10; { ...we call it again to see if it's... }
 Intr($10,Regs); { ...EGA color or EGA monochrome. }
 IF (Regs.BH = 0) THEN QueryAdapterType := EGAColor
 ELSE QueryAdapterType := EGAMono
 END
 ELSE { Now we know we have an CGA or MDA; let's see which: }
 BEGIN
 Intr($11,Regs); { Equipment determination service }
 Code := (Regs.AL AND $30) SHR 4;
 CASE Code of
 1 : QueryAdapterType := CGA;

 2 : QueryAdapterType := CGA;
 3 : QueryAdapterType := MDA
 ELSE QueryAdapterType := None
 END { Case }
 END
 END;
END;


{ All we're doing here is converting numeric font heights }
{ to their corresponding values of type FontSize. }

FUNCTION FontCode(Height : Byte) : FontSize;

BEGIN
 CASE Height OF
 8 : FontCode := Font8;
 14 : FontCode := Font14;
 16 : FontCode := Font16;
 END { CASE }
END;


{ Likewise, this function converts values of type FontSize }
{ to their corresponding numeriuc values. }

FUNCTION FontHeight(Code : FontSize) : Byte;

BEGIN
 CASE Code OF
 Font8 : FontHeight := 8;
 Font14 : FontHeight := 14;
 Font16 : FontHeight := 16;
 END { CASE }
END;



FUNCTION GetFontSize : FontSize;

VAR
 Regs : Registers; { Type Registers is exported by the DOS unit }

BEGIN
 CASE QueryAdapterType OF
 CGA : GetFontSize := Font8;
 MDA : GetFontSize := Font14;
 MCGAMono,
 MCGAColor : GetFontSize := Font16; { Wretched thing knows but 1 font! }
 EGAMono, { These adapters may be using any of several different }
 EGAColor, { font cell heights, so we need to query the BIOS to }
 VGAMono, { find out which is currently in use. }
 VGAColor : BEGIN
 WITH Regs DO
 BEGIN
 AH := $11; { EGA/VGA Information Call }
 AL := $30;
 BH := 0;
 END;

 Intr($10,Regs); { On return, CX contains the font height }
 GetFontSize := FontCode(Regs.CX);
 END
 END { CASE }
END;



FUNCTION GetTextBufferOrigin : Pointer;

{ The rule is: For boards attached to monochrome monitors, the buffer }
{ origin is $B000:0; for boards attached to color monitors (including }
{ all composite monitors and TV's) the buffer origin is $B800:0. }

BEGIN
 CASE QueryAdapterType OF
 CGA,MCGAColor,EGAColor,VGAColor : GetTextBufferOrigin := Ptr($B800,0);
 MDA,MCGAMono, EGAMono, VGAMono : GetTextBufferOrigin := Ptr($B000,0);
 END { CASE }
END;


{ This proc provides initial values for the dimensions of the visible }
{ display and (hence) the size of the visible refresh buffer. It is }
{ called by the initialization section during startup *BUT* you must }
{ call it again after any mode change or font change to be sure of }
{ having accurate values in the three variables! }

PROCEDURE GetTextBufferStats(VAR BX : Byte; { Visible X dimension }
 VAR BY : Byte; { Visible Y dimension }
 VAR BuffSize : Word; { Refresh buffer size }
{ This requires TP5.0! } CheckForOverride : OverrideProc);

CONST
 ScreenLinesMatrix : ARRAY[AdapterType,FontSize] OF Integer =
 { Font8: Font14: Font16: }
 { None: } ((25, 25, 25),
 { MDA: } (-1, 25, -1),
 { CGA: } (25, -1, -1),
 { EGAMono: } (43, 25, -1),
 { EGAColor: } (43, 25, -1),
 { VGAMono: } (50, 28, 25),
 { VGAColor: } (50, 28, 25),
 { MCGAMono: } (-1, -1, 25),
 { MCGAColor: } (-1, -1, 25));

VAR
 Regs : Registers; { Type Registers is exported by the DOS unit }

BEGIN
 Regs.AH := $0F; { BIOS VIDEO Service $F: Get Current Video Mode }
 Intr($10,Regs);
 BX := Regs.AH; { Number of characters in a line returned in AH }

 BY := ScreenLinesMatrix[QueryAdapterType,GetFontSize];
 IF BY > 0 THEN
 BEGIN
 CheckForOverride(BX,BY); { See if something weird is on the bus... }
 BuffSize := (BX * 2) * BY { Calculate the buffer size in bytes }

 END
 ELSE BuffSize := 0;
END;

{ This is the default override proc, and is called anytime you're }
{ not concerned about finding a nonstandard text adapter on the }
{ bus. (Funny graphics cards with normal text modes don't matter }
{ to this library.) If you want to capture any weird cards, you }
{ must provide your own override proc that can detect the card }
{ and return correct values for the visible X and Y dimensions. }

PROCEDURE NullOverride(VAR ForceX : Byte; VAR ForceY : Byte);

BEGIN
 { Like I said; Null... }
END;


{ The initialization section provides some initial values for the }
{ exported variables TextBufferOrigin, VisibleX, VisibleY, and }
{ TextBufferSize, so that you can use the variables without further }
{ kafeuthering. }

BEGIN
 TextBufferOrigin := GetTextBufferOrigin;
 GetTextBufferStats(VisibleX,VisibleY,TextBufferSize,NullOverride);
END.






[LISTING TWO]


PROGRAM TextTest;

USES TextInfo;

BEGIN
 Write('The installed adapter is ');
 CASE QueryAdapterType OF
 None : Writeln('nothing I''ve ever seen.');
 MDA : Writeln('an MDA .');
 CGA : Writeln('a CGA.');
 EGAMono,EGAColor : Writeln('an EGA.');
 VGAMono,VGAColor : Writeln('a VGA.');
 MCGAMono,MCGAColor : Writeln('an MCGA.');
 END; { CASE }
 Writeln('The current font height is ',FontHeight(GetFontSize),'.');
 Writeln('The current BIOS text mode is ',GetBIOSTextMode,'.');
 Writeln('The current screen is ',VisibleX,' character wide',
 ' and ',VisibleY,' characters wide;');

 Writeln(' and occupies ',TextBufferSize,' bytes in memory.');
END.







[LISTING THREE]


(*--------------------------------------------------------------*)
(* TEXTINFO *)
(* *)
(* Text video information library -- Definition module *)
(* *)
(* by Jeff Duntemann *)
(* TopSpeed Modula 2 V1.12 *)
(* Last update 12/7/88 *)
(*--------------------------------------------------------------*)

DEFINITION MODULE TextInfo;

TYPE
 AdapterType = (None,MDA,CGA,EGAMono,EGAColor,VGAMono,
 VGAColor,MCGAMono,MCGAColor);

 FontSize = (Font8,Font14,Font16);

 OverrideProc = PROCEDURE(VAR BYTE,VAR BYTE);

VAR
 TextBufferOrigin : ADDRESS; (* Address of video refresh buffer *)
 TextBufferSize : CARDINAL; (* Bytes contained in refresh buffer *)
 VisibleX,VisibleY : SHORTCARD; (* Dimensions of the visible display *)


PROCEDURE GetBIOSTextMode() : SHORTCARD;

PROCEDURE GetTextBufferOrigin() : ADDRESS;

PROCEDURE GetTextBufferStats(VAR BufX : BYTE; (* Visible X dimension *)
 VAR BufY : BYTE; (* Visible Y dimension *)
 VAR BuffSize : CARDINAL; (* Refresh buffer size *)
 CheckForOverride : OverrideProc);

PROCEDURE QueryAdapterType() : AdapterType;

PROCEDURE FontCode(Height : SHORTCARD) : FontSize;

PROCEDURE FontHeight(Code : FontSize) : SHORTCARD;

PROCEDURE GetFontSize() : FontSize;

PROCEDURE NullOverride(VAR ForceX : BYTE; VAR ForceY : BYTE);

END TextInfo.







[LISTING FOUR]


(*--------------------------------------------------------------*)
(* TEXTINFO *)
(* *)
(* Text video information library -- Implementation module *)
(* *)
(* by Jeff Duntemann *)
(* TopSpeed Modula 2 V1.12 *)
(* Last update 12/7/88 *)
(*--------------------------------------------------------------*)

IMPLEMENTATION MODULE TextInfo;

FROM SYSTEM IMPORT Registers;
FROM Lib IMPORT Intr;

VAR
 ColorBufOrg [0B800H:0] : WORD; (* First word in color refresh buffer *)
 MonoBufOrg [0B000H:0] : WORD; (* First word in mono refresh buffer *)


PROCEDURE GetBIOSTextMode() : SHORTCARD;

VAR
 Regs : Registers;

BEGIN
 Regs.AH := 0FH; (* VIDEO service 0FH *)
 Intr(Regs,10H);
 RETURN Regs.AL (* AL contains current text mode on return *)
END GetBIOSTextMode;


PROCEDURE QueryAdapterType() : AdapterType;

VAR
 Regs : Registers;
 Code : SHORTCARD;


BEGIN
 Regs.AH := 1AH; (* Attempt to call VGA Identify Adapter Function *)
 Regs.AL := 0; (* Must clear AL to 0 ... *)
 Intr(Regs,10H);
 IF Regs.AL = 1AH THEN (* ...so that if $1A comes back in AL... *)
 (* ...we know a PS/2 video BIOS is out there. *)
 CASE Regs.BL OF (* Code comes back in BL *)
 0 : RETURN None 
 1 : RETURN MDA; 
 2 : RETURN CGA; 
 4 : RETURN EGAColor; 
 5 : RETURN EGAMono; 
 7 : RETURN VGAMono; 
 8 : RETURN VGAColor; 
 0AH,0CH : RETURN MCGAColor; 
 0BH : RETURN MCGAMono; 
 ELSE RETURN CGA

 END (* CASE *)
 ELSE
 (* If it's not PS/2 we have to check for the presence of an EGA BIOS: *)
 Regs.AH := 12H; (* Select Alternate Function service *)
 Regs.BX := 10H; (* BL=$10 means return EGA information *)
 Intr(Regs,10H); (* Call BIOS VIDEO *)
 IF Regs.BX <> 10H THEN (* BX unchanged means EGA is NOT there...*)
 Regs.AH := 12H; (* Once we know Alt Function exists... *)
 Regs.BL := 10H; (* ...we call it again to see if it's... *)
 Intr(Regs,10H); (* ...EGA color or EGA monochrome. *)
 IF (Regs.BH = 0) THEN RETURN EGAColor
 ELSE RETURN EGAMono
 END
 ELSE (* Now we know we have an CGA or MDA; let's see which: *)
 Intr(Regs,11H); (* Equipment determination service *)
 Code := SHORTCARD(BITSET(Regs.AL) * BITSET{4..5}) >> 4;
 CASE Code OF
 1 : RETURN CGA 
 2 : RETURN CGA 
 3 : RETURN MDA
 ELSE RETURN None
 END (* Case *)
 END
 END
END QueryAdapterType;


(* This is a simple "clean conversion" function for relating the *)
(* enumerated font size constants to SHORTCARD numeric font size *)
(* values. *)

PROCEDURE FontCode(Height : SHORTCARD) : FontSize;

BEGIN
 CASE Height OF
 8 : RETURN Font8 
 14 : RETURN Font14 
 16 : RETURN Font16
 ELSE RETURN Font8
 END (* CASE *)
END FontCode;


(* This is a simple "clean conversion" function for relating the *)
(* SHORTCARD numeric font size values to the enumerated font size *)
(* constants *)

PROCEDURE FontHeight(Code : FontSize) : SHORTCARD;

BEGIN
 CASE Code OF
 Font8 : RETURN 8 
 Font14 : RETURN 14 
 Font16 : RETURN 16
 END (* CASE *)
END FontHeight;




PROCEDURE GetFontSize() : FontSize;

VAR
 Regs : Registers;

BEGIN
 CASE QueryAdapterType() OF
 CGA : RETURN Font8 
 MDA : RETURN Font14 
 MCGAMono,
 MCGAColor : RETURN Font16 
 EGAMono, (* These adapters may be using any of several *)
 EGAColor, (* different font cell heights, so we need to query *)
 VGAMono, (* BIOS to find out which is currently in use. *)
 VGAColor : WITH Regs DO
 AH := 11H; (* EGA/VGA Information Call *)
 AL := 30H;
 BL := 0;
 END;
 Intr(Regs,10H);
 RETURN FontCode(SHORTCARD(Regs.CX))
 END (* CASE *)
END GetFontSize;


PROCEDURE GetTextBufferOrigin() : ADDRESS;

(* The rule is: For boards attached to monochrome monitors, the buffer *)
(* origin is $B000:0; for boards attached to color monitors (including *)
(* all composite monitors and TV's) the buffer origin is $B800:0. *)

BEGIN
 CASE QueryAdapterType() OF
 CGA,MCGAColor,EGAColor,VGAColor : RETURN ADR(ColorBufOrg) 
 MDA,MCGAMono, EGAMono, VGAMono : RETURN ADR(MonoBufOrg)
 END (* CASE *)
END GetTextBufferOrigin;


(* This one function returns essential screen/buffer size information. *)
(* It is called by the initializing body of this module but should be *)
(* called again after *any* mode change or font change! *)

PROCEDURE GetTextBufferStats(VAR BufX : BYTE; (* Visible X dimension *)
 VAR BufY : BYTE; (* Visible Y dimension *)
 VAR BuffSize : CARDINAL; (* Refresh buffer size *)
 CheckForOverride : OverrideProc);

TYPE
 FontPoints = ARRAY[Font8..Font16] OF INTEGER;
 PointsArray = ARRAY[None..MCGAColor] OF FontPoints;

VAR
 Regs : Registers; (* Type Registers is exported by the DOS unit *)
 ScreenLinesMatrix : PointsArray;
 Adapter : AdapterType;
 Font : FontSize;

(* TopSpeed can't do two-dimensional array aggregates, Turbo Pascal *)

(* style (arrgh) so we have to make it an array of arrays: *)

BEGIN
 ScreenLinesMatrix := PointsArray(
 (* None: *) FontPoints(25, 25, 25),
 (* MDA: *) FontPoints(-1, 25, -1),
 (* CGA: *) FontPoints(25, -1, -1),
 (* EGAMono: *) FontPoints(43, 25, -1),
 (* EGAColor: *) FontPoints(43, 25, -1),
 (* VGAMono: *) FontPoints(50, 28, 25),
 (* VGAColor: *) FontPoints(50, 28, 25),
 (* MCGAMono: *) FontPoints(-1, -1, 25),
 (* MCGAColor: *) FontPoints(-1, -1, 25));

 Regs.AH := 0FH; (* BIOS VIDEO Service $F: Get Current Video Mode *)
 Intr(Regs,10H);
 BufX := Regs.AH; (* Number of characters in a line returned in AH *)

 BufY := SHORTCARD(ScreenLinesMatrix[QueryAdapterType(),GetFontSize()]);
 IF SHORTCARD(BufY) > 0 THEN
 CheckForOverride(BufX,BufY); (* See if odd adapter is on the bus... *)
 (* Calculate the buffer size in bytes: *)
 BuffSize := (CARDINAL(BufX) * 2) * CARDINAL(BufY)
 ELSE BuffSize := 0
 END
END GetTextBufferStats;


(* This is the "default" override proc, called when there is no *)
(* suspicion of anything nonstandard on the bus. Replace with *)
(* a custom proc that looks for any nonstandard video adapter. *)

PROCEDURE NullOverride(VAR ForceX : BYTE; VAR ForceY : BYTE);

BEGIN
 (* Like I said; Null... *)
END NullOverride;


(* The module body, like a Pascal unit initialization section, is *)
(* executed before the client program that imports this module or *)
(* any part of it. *)

BEGIN
 TextBufferOrigin := GetTextBufferOrigin();
 GetTextBufferStats(VisibleX,VisibleY,TextBufferSize,NullOverride);
END TextInfo.





[LISTING FIVE]


MODULE TextTest;

FROM IO IMPORT WrStr,WrLn,WrCard,WrShtCard;
FROM TextInfo IMPORT AdapterType,QueryAdapterType,GetFontSize,

 FontHeight,GetBIOSTextMode,VisibleX,VisibleY,
 TextBufferSize;

BEGIN
 WrStr("The installed adapter is ");
 CASE QueryAdapterType() OF
 None : WrStr("nothing I've ever seen.") 
 MDA : WrStr("an MDA.") 
 CGA : WrStr("a CGA.") 
 EGAMono,EGAColor : WrStr("an EGA.") 
 VGAMono,VGAColor : WrStr("a VGA.") 
 MCGAMono,MCGAColor : WrStr("an MCGA.");
 END; (* CASE *)
 WrLn;
 WrStr('The current font height is ');
 WrShtCard(FontHeight(GetFontSize()),2);
 WrStr("."); WrLn;
 WrStr("The current BIOS text mode is ");
 WrShtCard(GetBIOSTextMode(),2);
 WrStr("."); WrLn;
 (* VisibleX and VisibleY are initialized by TextInfo module body *)
 WrStr("The current screen is ");
 WrShtCard(VisibleX,2);
 WrStr(" character wide and ");
 WrShtCard(VisibleY,2);
 WrStr(" characters high;");
 WrLn;
 WrStr(" and occupies ");
 (* TextBufferSize is initialized by TextInfo module body *)
 WrCard(TextBufferSize,6);
 WrStr(" bytes in memory."); WrLn;
END TextTest.






























March, 1989
OF INTEREST





White Pine Software will soon release eXodus, an X-Windows system display
server for the Macintosh developed in cooperation with Digital Equipment Corp.
According to White Pine, eXodus is the first implementation of the X11
standard under the Macintosh OS. X11 is a network transparent window system
developed at MIT and supported by vendors such as Apple Computer Inc. and
Digital Equipment Corp.
eXodus conforms to the Macintosh user interface standards and is compatible
with Apple's Multifinder system. Macintosh Plus, SE, and II are supported, as
well as network communication protocols such as Appletalk, DECnet, and TCP/IP.
eXodus provides an X-Windows system front end to client applications running
on a variety of host systems. By handling requests for client connections and
disconnections, receiving and processing client requests, and sending
Macintosh events from user to host, the Macintosh is integrated into the
client application.
Version 1.0 supports monochrome systems; color support will be provided in a
later release. A font compiler is provided that converts X distribution (BDF)
fonts, as well as Macintosh fonts, to an eXodus compatible format. White Pine
also plans to upgrade eXodus to conform to DECwindows, Digital's user
interface.
Cost of eXodus, Version 1.0, is $499 per server. Educational pricing and site
licenses are available. Reader Service No. 20.
White Pine Software 94 Route 101A P.O. Box 1108 Amherst, NH 03031 603-886-9050
Powerline Software's Source Print lists one or more source files with page
headings and optional line numbers and includes an index (cross-reference
list), structure outlining, and automatic indentation of source code and
listings. Source Print also generates a table of contents that lists functions
and procedures.
Tree Diagrammer, another Powerline Software product, creates an organization
chart of programs showing the hierarchy of calls to functions, procedures, and
subroutines. Recursive calls are indicated, and designated comments in the
source code appear on the chart.
Source Print sells for $97; Tree Diagrammer is $77. Reader Service No. 21.
Powerline Software Inc. 2531 Baker St. San Francisco, CA 94123 415-346-8325
800-257-5773 (Dept. M-4)
Microtec Research has released XRAY86, the newest version of its XRAY
high-level debugger to support the Intel 8086/186/286 microprocessors. The
XRAY high-level debugger family supports emulators from applied Microsystems
Corp., Hewlett-Packard, Microcase Inc., Microcosm, Microtek International, and
ZAX Corp.
XRAY86 provides a programmable window interface called Viewports, which
permits users to scan source code, monitor program variables and expressions,
trace procedure calls, and set simple and complex conditional breakpoints.
With XRAY debugger software, programmers can use either Microtec Research's
integrated C compiler and assembler tools or Intel's compilers and assemblers.
In addition, Microtec Research claims that XRAY86 maintains the same user
interface, regardless of the execution environment or the host computer.
XRAY86 supports Microtec Research C and Pascal compilers, as well as C,
Pascal, and PL/M compilers from Intel. XRAY86 and its language development
toolkits are available on VAX, IBM PC, and workstations such as Sun, HP, and
Apollo.
Prices for a toolkit that includes compiler, assembler package, and XRAY86
debugger range from $3,500 on a PC to $5,000 on a workstation and $14,000 on a
VAX. Components of the toolkit may be purchased separately. Reader Service No.
22.
Microtec Research Inc. P.O. Box 60337 Sunnyvale, CA 94088 408-733-2919
Layout, by Matrix Software Technology, is a menu-based software development
system that features object-oriented programming, a graphical interface, CASE
tools, and hypertext.
Layout contains three levels of software objects: elements, procedures, and
black boxes. Elements contain a single preprogrammed command (such as "draw a
window"). Procedures are more complex, consisting of two to 500 elements.
Black boxes are externally developed software objects; rather than containing
elements built into Layout, black boxes are units programmed in code to run
with Layout.
Layout is based on an open architecture, which allows the use of black box
elements developed by third parties in C or assembler, as well as those
provided by Matrix.
Layout's file box opens a window on the screen, displays the current directory
in the disk, displays the file names, allows the user to select and change
drives and folders, and associates screen icons with different file types.
Layout also contains underlying inference engines, an artificial intelligence
technology that enables information to be interpreted as a simulation or
compiled into a program. Layout creates code in Turbo Pascal, Turbo C,
Microsoft C, QuickBasic, or Lattice C. Layout also generates executable DOS
files.
Priced at $149.95, Layout supports a mouse and runs on the IBM PC and
compatibles. Reader Service No. 23.
Matrix Software Technology Corp. 1 Massachusetts Technology Center Harborside
Dr. Boston, MA 02128 617-567-0037 800-533-5644
Grammar Engine has introduced a desk accessory (DA) software tool called
LoadROM, which is used for microprocessor cross development on the Macintosh.
The LoadROM software accepts binary and popular hex record formats for loading
ROM code from the Macintosh to the ROMulator, Grammar Engine's in-circuit ROM
emulator. The ROMulator package consists of hardware, cables, LoadROM host
software, and instruction manual.
LoadROM provides editing capabilities to allow ROM software patching. LoadROM
is also a Macintosh Programmer's Workbench (MPW) tool. In the command-line
version of the software, it is intended to be invoked by the make command of
MPW to load the ROMulator as part of the build process.
After downloading from the Macintosh, the ROMulator software is available for
access from the target computer. Bi-directional models make it possible to use
the target debugger to patch ROM code. A multi-drop interface allows up to
eight modules with different identities to be daisy-chained and loaded from a
single Macintosh printer or modem port.
The ROMulator allows in-circuit emulation of ROMs in 8-, 16-, or 32-bit target
systems, such as the Z80, 80386, and 68020. Standard ROMulator models are
priced from $375 to $1,645, depending on total RAM capacity and other options.
Reader Service No. 24.
Grammar Engine Inc. 3314 Morse Rd. Columbus, OH 43231 614-471-1113
Computer Control Systems has released several products: FABS Plus OS/2,
Autosort OS/2, and DB-FABS/ DABL, Version 3.
Written in assembly language, FABS Plus is a BTree subroutine designed to
maintain index files. FABS Plus supports duplicate and variable length keys.
Multiple primary keys can be maintained in a single tree providing access to
data files on more than one key. The key files are independent of the data
files and do not need to be resequenced. Generic searches and multilevel
sequencing are supported; supported interfaces are Microsoft's Basic compiler,
Pascal, Fortran, and C.
Autosort OS/2 is a sort/merge/select utility written in assembly language. It
sorts on 15 sort keys (ascending or descending) and provides 15 select keys so
that records may be deleted, retained, or retained-if-not when the select key
is less than, equal to, or greater than the select field in the record.
Autosort sorts and selects on string, integer, single precision, and double
precision fields. It also supports paths to files. Sort parameters can be
specified during run time or from parameter files on the disk.
Version 3 of DB-FABS, written in assembly language for MS-DOS systems, is a
data, screen, and report manager program designed to help users manipulate and
control data files. DABL is a programming language designed to be used with
DB-FABS. DB-FABS consists of two modes of operation: the stand-alone mode and
the run-time mode, with which users can create data files and screen forms and
handle the file I/O, indexing, sorting, screen management, and reports.
The single-user version of FABS Plus OS/2 sells for $195, and the network
version is $295. Autosort OS/2 is $150, and DB-FABS/DABL, Version 3, sells for
$195 (single user) and $295 (network version). Reader Service No. 25.
Computer Control Systems Inc. Rt. 3, Box 168 Lake City, FL 32055 904-752-0912
The Ohio Scientific 720, a member of the OSI series 700 family of 32-bit
single board microcomputers based on Motorola 68OxO processors, has been
released by Consolidated Computer Systems Inc. (CCSI).
The Ohio Scientific 720 runs under RTIX, an operating system compatible with
Unix's System V Interface Definition (SVID) at both the kernel and base
extension levels. RTIX incorporates real-time capability within the kernel.
The real-time features of the RTIX kernel include NO_WAIT system calls and
request and event queues.
In its standard configuration, the 720 has 12 RS-232 ports. Through
intelligent terminal concentrators and Ethernet, the multiuser, multitasking
system can accommodate as many as 60 users. If all users are doing processor
or disk-intensive tasks, additional 68020 CPUs (with supporting FPPs and
static RAM cache) can be added in a parallel arrangement with dynamic load
balancing.
Standard RAM is 4 Mbytes, expandable to 64 Mbytes. The paged memory management
unit (PMMU) provides demand-paged virtual memory.
The 720 offers multiple hard disks ranging in size from 91 Mbytes to 1.2
gigabytes for total storage capacity to 16 gigabytes. OSI 720 is integrated
with MIMER, a relational DBMS that offers 4GL tools and Uniplex II+, Version
5.04b, an integrated word processor, spreadsheet, database, and office
automation system.
Prices for the series 700 computers begin at $6,350. Reader Service No. 26.
Consolidated Computer Systems Inc. 2150-D W 6th Ave. Broomfield, CO 80020
303-460-0444
Hawlett-Packard has announced that it will use the Motorola 68030 (030)
microprocessor in an engineering workstation. The HP 9000 Model 340, which
also incorporates Motorola's 68882 (882) math coprocessor, is priced at
$5,495. It offers up to 4 MIPS of processing power.
Motorola's 68000 microprocessor line currently has four members: the 68000,
68101, 68020, and 68030. The 68040 is being developed. New generations of the
68000 are compatible with earlier 68000-based products, and, according to the
company, software written for one chip runs with no modification on the
others. Reader Service No. 27.
Motorola Inc. Microprocessor Products Group 6501 William Cannon Drive W
Austin, TX 78735-8598 512-440-2000
AST Research is now shipping the Mac86, an 8086-based coprocessor board that
allows users of the Macintosh SE to run MS-DOS applications. The board
supports several drives for loading and saving MS-DOS applications, including
the Apple PC 5.25-inch drive, the DaynaFile, or the IBM 3.5-inch drive. Users
can access MS-DOS programs via local area networks, such as AppleShare or
Tops.
Mac86 provides multitasking under Apple's MultiFinder, allowing an MS-DOS
application to process in the Mac86 window while a Macintosh application is
running in another window. It offers the ability to copy and paste text and
graphics from MS-DOS applications to Macintosh applications, or text from
Macintosh to MS-DOS applications. Microsoft mouse emulation using the
Macintosh mouse is provided as well.
The Mac86 was jointly developed by AST Research, Apple Computer Inc., and
Phoenix Technologies. It sells for $599. Reader Service No. 28.
AST Research Inc. 2121 Alton Ave. Irvine, CA 92714 714-863-1333
Oasys has announced the Version 5.1 release of Microsoft C, Microsoft
Assembler (MASM), Microsoft Linker, and Oasys' Microsoft Embedded Kit (MEK86)
hosted on VAX, Sun, and other Unix systems. The Oasys port of Microsoft's
PC-based C development tools, known as the Oasys Microsoft Cross C Development
System, allows embedded systems software developers to run executable output
on Intel 8086/286 embedded microprocessors, as well as MS-DOS or OS/2 target
systems.
Each version of the Oasys Microsoft Cross C Development System is compatible
with Microsoft C. This new release includes MEK86, which contains Microsoft C
run-time library source, as well as 80x86 initialization code.

According to Oasys, the Cross C Development system produces high-speed
executables and optimized code by elimination of common subexpressions. The
compiler also implements register variables. The system offers several memory
models (small, compact, medium, large, huge) and pointers (near, far, huge).
Library routines implement most of the Unix System V C library. Users can
choose from three math libraries and generate in-line 8087/80287 instructions
or floating point calls.
The package, selling for $4,250 to $15,500 (depending on the host system), is
available on VAX/VMS, VAX/Unix V, Sun-3 and Sun-386i, Apollo, and Pyramid
systems. Reader Service No. 34.
Oasys Inc. 230 Second Ave. Waltham, MA 02154 617-890-7889
C2PS, a compiler that converts C source code into PostScript code, has been
released by UniPress Software. C2PS was developed for use with Sun
Microsystems' NeWS window system and other environments requiring the
production of PostScript code, such as Adobe Systems' Display PostScript,
x11/NeWs, and PostScript printer applications.
Developers write NeWS programs in two parts: The first part, the application,
is written in C, while the graphic interface is written in PostScript. With
C2PS, the graphic interface can be written in C and then translated.
C2PS is part of UniPress' PostScript working environment. When it is used with
the UniPress Emacs editor, users can edit and view C code in one window while
viewing the produced PostScript in another. The graphic output can then be
shown in a NeWS window or on a printer.
C2PS is available to commercial users for $2,995 with binary code, and $14,995
with source code. University prices for C2PS are $995 for binary code and
$4,995 for source code. Reader Service No. 30.
UniPress Software Inc. 2025 Lincoln Hwy. Edison, NJ 08817 201-985-8000
TransWare Enterprises (TWE), a software development company, has signed an
agreement with Lahey Computer Systems to implement the Lahey Fortran language
system under Digital Research Inc.'s (DRI) concurrent DOS and FlexOS 386
operating systems. The new Fortran language systems are scheduled to be
available this month.
Lahey's Fortran compilers conform to the ANSI Fortran 77 language standard.
DRI's operating systems are multitasking and multiuser capable for IBM, IBM
compatibles, and Compaq microcomputers.
Concurrent DOS is a DOS-compatible operating system that runs in the real-mode
state of the Intel 80x86 microprocessors. FlexOS 386 is a realtime operating
system that runs in the protected mode state of the 80386 microprocessor.
Reader Service No. 31.
TransWare Enterprises Inc. 5091 Durango Ct. San Jose, CA 95118 408-723-2102
The 88open Consortium Ltd. has announced that NCR Corp. has joined the
organization to establish market acceptance of Motorola's 88000 RISC
microprocessor architecture. NCR joins 39 other software and hardware vendors
as a members of 88open.
NCR is currently active in two 88open software development groups: the 88open
binary compatibility standards (BCS) program and the software initiative
committee. The BCS program focuses on the development and promotion of the
88open's BCS interface standard, and the software initiative committee works
with developing applications for 88000 systems.
NCR plans to incorporate the 88000 chip set in future computer systems. Other
88open members --such as Icon International Inc., Jet Propulsion Laboratories,
Data General, and Convergent Technologies --plan to use the 88000 in future
products.
The 88open is a nonprofit organization with more than 45 worldwide members.
Reader Service No 32.
88open Consortium Ltd. 8560 SW Salish Ln., Ste. 500 Wilsonville, OR 97070
503-682-5703
Using SAS Institute's SAS/CPE software, systems analysts can collect, analyze,
and report current usage of their VAX hardware. SAS/CPE software is a
component of the SAS System, which includes integrated modules for data entry,
retrieval, and management; report writing and graphics; statistical and
mathematical analysis; business planning; operations research and project
management; and applications development.
With SAS/CPE software, systems managers can collect performance and usage data
from data-gathering facilities such as the VMS Monitor Utility, VAX System
Performance Monitor (SPM), the VMS Accounting Utility, and SAS/CPE software's
DISKQUOTA facility, which gathers disk usage data over time.
Users can also convert the data collected by these utilities to SAS data sets
where they can be analyzed and presented using SAS/CPE software's built-in,
menu- or command-driven reporting programs.
Another feature is that users can create tabular reports, charts, line printer
graphs, and high-resolution graphs that detail major areas of resource usage
and system performance.
The SAS System is licensed on an annual basis with fees based on machine
classification. The first-year license fee for SAS/CPE software ranges from
$850 to $2,900, with renewals available at a lower rate. Degree-granting
discounts are also available. To use SAS/CPE software, sites need base SAS
software. To modify the menu-driven portion of the system, sites need SAS/AF
software, the SAS System's interactive applications development tool. Reader
Service No. 33.
SAS Institute Inc. Software Sales Dept. SAS Circle Box 8000 Cary, NC
27512-8000 919-467-8000
Ashton-Tate has released Step IVward (pronounced "Step Forward"), a conversion
program that allows developers with Clipper, FoxBASE, and Quick-silver
applications to convert them to dBase IV. It translates most of the
functionality in these products and includes files tree processing, in-code
commenting, and printing capabilities.
Suggested retail price is $89.95. Reader Service No. 34.
Ashton-Tate Corp. 20101 Hamilton Ave. Torrance, CA 90502-1319 213-329-8000
800-227-6900
An interface that serves as a bridge from ProKit*Workbench to PRO-IV has been
released by McDonnell Douglas. The interface provides a CASE environment that
supports applications developed in IBM, DEC, and Unix environments.
According to McDonnell Douglas, system developers can move from strategic
planning, analysis, and design in ProKit*Workbench to application development,
implementation, maintenance, and documentation in PRO-IV without rekeying
design specifications. The information developed and managed within
ProKit*Workbench during analysis and design is electronically transferred
through the interface.
Prokit*Workbench features include support for process and data modeling,
active prototyping, and design techniques that help software engineers produce
life cycle deliverables for development and maintenance of application code.
Applications developed in PRO-IV are hardware, operating system, and database
independent. Reader Service No. 35.
McDonnell Douglas 600 McDonnell Blvd. Hazelwood, MO 63042 800-325-1087
Software Development Systems Inc. (SDSI) has introduced cross-development
utilities licensed for Unix systems (CLAUS), which includes CrossCode C for
the 68000 microprocessor family, the UniWare Z80 C compiler, and a line
UniWare cross-assembler packages.
For Unix systems that read standard 9-track reel tapes or standard cartridge
tapes in the Unix tar format, SDSI provides the cross development software for
computers running Berkeley Unix, AT&T Unix, Sun's SunOS, Apollo's Domain/IX,
DEC's Ultrix, HP's HP-UX, Sequent's Dynix, Xenix, and others.
SDSI has also implemented a testing suite that is run as part of each port.
This suite tests the cross development package to help ensure that it works
the same way on every machine and adheres to the documentation provided.
Reader Service No. 36.
Software Development Systems Inc. 4248 Belle Aire Lane Downers Grove, IL 60515
800-448-7733
Recently released by Micro Computer Control (MCC) is MICRO/SLD-51, a PC-hosted
source language debugger program that provides C and assembly language
debugging capabilities for the 8051 family of single-chip microcontrollers.
This software development tool executes 8051 or 8052 code on a PC. It includes
support for on-chip devices, such as the UART, timers, and interrupts. With
MICRO/SLD-51, an 8051 program written in C, assembly, or a combination of both
can be loaded into the debugger for testing and evaluation. Another feature of
the debugger is that it single-steps through an 8051 program line by line,
displaying the original C or assembly source code as the program executes on
the host PC.
Also included are debugging tools such as a single-line assembler and
disassembler, breakpoints on code or data, symbolic access to program
variables, and dynamic display of any hardware register or data variable. The
program also includes context sensitive help windows that the detail use
program features.
MICRO/SLD-51 works with program files created with the company's MS-DOS based
MICRO/C-51 C cross-compiler and assembler development package or 8051
assemblers that can generate standard Intel Hex files.
This product is priced at $295 and operates on an IBM PC or compatible with
256K memory and one floppy diskette. It supports monochrome, CGA, VGA, or
Hercules video adapters and is compatible with laptop computers. Reader
Service No. 37.
Micro Computer Control Corp. P.O. Box 275 Hopewell, NJ 08525 609-466-1751
Microcompatibles has announced completion of its latest update to GRAFMATIC
Fortran-callable screen graphics: GRAPHMATIC now supports the VGA graphics
board. Also included in the latest release are Microsoft serial mouse support,
animation, shaded surface plots, solid model shaded images, true clipping,
ability to save and restore a graphics screen, and more.
The $135 price includes support for one Fortran compiler and one type of
graphics board (EGA/CGA or Hercules Mono). Additional libraries for other
Fortran compiler or graphics boards are $35 each. Reader Service No. 38.
Microcompatibles Inc. 301 Prelude Dr. Silver Spring, MD 20901 301-593-0683



















March, 1989
SWAINE'S FLAMES


Words & Figures




Michael Swaine


We verbiage vendors get customer complaints, too. Two recent errors of mine
prompted readers to write.
One reader reminded me that misusing a word damages its ability to communicate
even when the intent behind the particular misuse can be discerned.
"Momentarily," he pointed out, means "for a moment," not "in a moment." Not
all authorities agree, but unfortunately for me, I do.
As to the "wont/won't" error, my first reaction is to say that of course I
know better. But a professional writer is no more literate than his published
work, and I wasn't earning my pay when I let that one get by.
Because we all use language every day, we tend to overlook misuses of language
even by those who are paid to use it effectively. We need not be so generous:
People who make their living by putting words together should be held to high
standards, just as professional programmers are held responsible for every
little bug. Having delivered my mea culpa, I intend to point out here, from
time to time, some of the more awful or hilarious violations of sense and
logic in the computer press.
I also plan to look at pictures: Charts, tables, and figures can lie. I'll be
doing this for more than entertainment value. If you read DDJ, you sometimes
get treated as an expert on things quantitative, and it can't hurt to be
reminded, from time to time, of some of the misleading ways in which
quantitative data can be presented.
Take Apple's latest annual report. The report opens with a bank of four bar
charts, slightly three-dimensional and tilted, showing sales, earnings per
share, and so forth, all zooming skyward. The scale is fair, but the tilting
has the effect that increases are represented by heavy, steep bands of color
and decreases by a lack of ink on the page. The drops almost have to be
deduced -- you don't see them at all at first.
It's a clever trick.


Copyright Protection


Those of us who have hounded copy-protection into its present state of
disrepute and declining employment should now be enjoined to exert equal
effort on behalf of copyright protection for software authors.
Copy protection may have been a bad attempt to solve the problem of copyright
violation, but the problem remains and grows more severe, with no solution in
sight.
I don't think that educational efforts are the solution. Although there are
consumers selfless enough to forego copying for the good of the author or of
the software industry, in general no amount of education is going to make
consumers act against their apparent financial interests. An immediate saving
of several hundred dollars is a hard argument to refute. Shareware works to
the extent that the vendor can convince the consumer that their relationship
is something other than a vendor-consumer relationship, but when big-ticket
vendors try to do the same through guilt trip advertisements and shrinkwrap
licenses, the consumer is justly skeptical.
The analogy of book publishing doesn't point to a solution, because the
problem hasn't been solved there, either. If books were as easy to copy as
software, book buyers would make copies for their friends as casually as they
copy tapes and disks.
The October 1988 issue of Appledirect, Apple's monthly magazine for
developers, carries a relevant viewpoint by Lionshead Software president Chris
Gillette. Gillette challenges Apple to address the problem by, for example,
providing Macintosh developers with a ROM call to get the computer's serial
number. This would let vendors permit unlimited copying of distribution disks,
while tying disks to one machine, and is a scheme that has been used by Sun
and Hewlett-Packard, as well as by Apple itself in the Lisa.
Strategic placement of the serial number check could turn a program into a
crippled demo version of itself when run on an unauthorized computer, so that
when users created copies for friends, they would be promoting sales. It's
possible to defeat such a copyright protection scheme, but as Gillette points
out, it's not the professional who's the problem, but the amateur, who copies
because it's so easy.
Whether or not Gillette's proposal is a solution for copyright protection,
it's deserving of discussion. Maybe it could serve as a starting point for a
debate on ethical and effective means to protect software authors' rights.
Then again, it is possible that software copying is just the consumer's way of
adjusting the price to an acceptable level. If that's the case, the consumer
will eventually win, quality software will sell for $19.95, and casual copying
will not cease, but will cease to be a concern.





























April, 1989
April, 1989
EDITORIAL


Unlike Old Soldiers, Memory Problems Don't Fade Away




Jonathan Erickson


I can see it all now. No longer will future generations have to put up with
the sad litany, "When I was your age, I had to walk two miles through the snow
...." Instead, kids will be hearing something like, "Well, when I was your
age, I only had 64K of memory to work with." And instead of "Tell me, what did
you do in the war, Dad?" parents will be asked "Tell me Dad, what was it like
to write programs in 256K of memory?"
Yes, having only 64K (or even 256K for that matter) of memory is pretty much a
thing of the past, although veterans of the PC wars (those guys with the hint
of gray in their hair and the melancholy smile on their face) still reminisce
about the good old days of limited memory, slow CPUs, and 8-inch disk drives.
What's particularly interesting about the current memory situation is that
even though the problem of limited memory has --to some degree --gone away,
the problems of efficiently using whatever memory is available remains. What
with larger, more complex programs, TSRs, and a host of other constraints
(don't forget that OS/2 requires nearly 2 Mbytes just to boot!), memory
management problems haven't even begun to go away. And therein lies one of the
reasons we're taking a look at memory management this month.
Two of our articles --"More Memory for DOS Exec" and "SWAP" --take different
approaches to a similar problem: how to swap programs in and out of expanded
memory (or disk) so that you can free up main memory to use more than one
program at a time. In the first of these articles, Kim Kokkonen shares with us
some of the actual techniques he uses in the development of his TurboPower
tools, proving that he's a good writer, as well as a good programmer.
Two other of this month's articles --"Demand Page Virtual Memory" and
"Advanced 80386 Memory Management" --also look at similar topics. In the first
instance, Kent Dahlgren examines the general subject of demand page memory and
discusses how different CPUs handle paging. In the second, Neal Margulis looks
in depth at demand paging on the 80386.
These won't be the only articles we'll be running that discuss the problem of
memory management. The topic is too important to simply touch on and forget.
The last time we asked for articles was in October and we heard from a lot of
you who had all kinds of good ideas for articles. You've already seen some of
those articles in print and you'll see more in the next few months. But don't
let up. If you have an idea for an article, get it (the idea or the article)
to us.
What kind of articles are we looking for? Well, how about one that discusses
your approach to memory management. What kind of articles would you like to
see in the magazine? What kind of articles would be useful to you? If you have
a useful utility or technique, you can bet that other readers would find it
just as useful.
The editorial calendar for the rest of the year looks like this: May,
Structured Languages; June, Operating Systems; July, Graphics; August, the
Annual C Issue; September, Modeling and Simulation; October,
Telecommunications; November, Parallel Processing; and December,
Object-Oriented Programming. While we're looking for articles on these
subjects, we'll consider just about any task-specific, code-intensive article.
If you miss one of our deadlines (say you have a graphics article but don't
get it done in time for our July issue), don't worry. If it's good, we'll run
it at anytime.
If you have any ideas for articles on any of the above topics --or others
--give Kent, Mike, or me a call at 415-366-3600, or drop us a note at DDJ, 501
Galveston Drive, Redwood City, CA 94063. We can also be reached on CompuServe
(#76704,50), MCI Mail ("DDJ"), or BIX ("jerickson"). We'll look forward to
hearing from you and publishing your articles over the coming months.






































April, 1989
LETTERS







Conversion Confusion


Dear DDJ,
I enjoyed Rob Moore's article on "Mapping DOS Memory Allocation" (November
1988) very much; it made a lot of sense. I did have some problems converting
it over to Microsoft C, though. The one change I discovered was that <dos.h>
in Microsoft C does not contain a MK_FP (segment, offset) macro, so I borrowed
the macro from a friend who uses Turbo C. The macro left shifts the segment
into the next word and ors the offset in.
This one macro, however, was not enough. I am not convinced that my program,
called ptrmcb, is accurately finding memory control blocks. The first five
bytes of the first memory-control block found are: 0973:0000 4D 08 00 08 02.
Referring to the structure of "struct mcb": chain = 'M'; pid = 2048; psize =
2050.
I am using QuickC and MSC 4.0 with MS-DOS 3.3. Example 1 shows a run of both
mcb and mapmem. Any thoughts on completing the conversion from Turbo C to
Microsoft C would be appreciated.
Bruce Koivu
Orlando, Fla.
Example 1: Allocated Memory Map by TurboPower Software Version 1.8
 PSP bytes owner command line hooked vectors
------------------------------------------------------------------------------
 0008 8320 N/A
 0B7D 4176 N/A 22 24 2E
 0C99 603760 free

==============================================================================
 MCB MCB ID PID MB PARENT ENV OWNER
 NO. SEG SIZE BLK?
------------------------------------------------------------------------------
 01 0973 M 0800 28704 068C N
 02 0973 0007 7808 022B N UNKNOWN OWNER
==============================================================================

Robert responds:
You are correct that Microsoft C (MSC) provides no macro MK_FP (seg,off) to
construct a far pointer composed of segment seg and offset off. Other DDJ
readers have pointed this out to me, notably Ron Sonntag (of Analytical
Software Consulting in Seattle, Wash.), who found it necessary to replace the
macro FP_SEG with the Turbo C version and make other related macro definition
adjustments when converting MCB.C to run under MSC 5.1. (I won't give any
further details because I currently do not have access to a copy of MSC and
cannot personally vouch for the results.)
Mr. Koivu's erroneous output from my MCB program is almost certainly due to
compilation of the program with a compiler option set to word alignment. My
program will not execute properly under any C compiler unless byte alignment
is used. I worked out the examples after reader Edward Rippy of Oakland,
Calif. informed me of similar problems he encountered when he tried word
alignment under Turco C.
When compiled under byte alignment (this is the default in Turbo C), MCB.EXE
reports the results, shown in Example 2, when invoked on my current system
(true blue IBM PC running under PC-DOS 3.2).
Example 2: MCB.C compiled with byte alignment (the default)
==============================================================================
 MCB MCB ID PID MB PARENT ENV OWNER
 NO. SEG SIZE BLK?
------------------------------------------------------------------------------
 01 0974 M 0008 134768 0291 N IBMDOS.COM/MSDOS.SYS
 02 2A5C M 2A5D 3120 2A5D N COMMAND.COM COPY #1
 03 2B20 M 2A5D 160 2A5D Y COMMAND.COM COPY #1
 . .
 . (22 MCBs not relevant to discussion omitted) .
 . .
 25 331B M 3327 160 2A5D Y C:\UTIL\MCB.EXE
 26 3326 M 3327 71264 2A5D N C:\UTIL\MCB.EXE
 27 448D Z 0000 374560 F000 N FREE MEMORY CONTROL BLOCK
==============================================================================

When I compile under word alignment (Options/Compiler/Code
Generation/Alignment Word in the Turbo C integrated environment or the command
line-a option) and run the new MCB.EXE I get the results that are shown in
Example 3.
Example 3: MCB.C compiled with word alignment
==============================================================================
 MCB MCB ID PID MB PARENT ENV OWNER
 NO. SEG SIZE BLK?

------------------------------------------------------------------------------
 01 0974 M E700 512 FF7E N *************
 02 0999 * BCDO 11568 0720 N UNKNOWN OWNER
==============================================================================

Note: All characters marked * (asterisk) are various nonprintable characters,
which confirms all the more that there is a problem.
This latter mess is similar to that reported by reader Koivu. Let me explain
what has gone wrong. The relevant contents of the first MCB located at
0974:0000 (hex) in my case are 4D 08 00 E7 20 00 determined by using DOS
DEBUG. A pointer to struct MCB is used as a template to overlay and retrieve
the contents of the MCBs as they already exist in memory. From my MCB.C
listing, struct MCB is defined as:
struct MCB
{
char chain;
unsigned pid;
unsigned psize;
char unused[11]'
};
With byte alignment, the compiler does not adjust user-defined structures, so,
if a structure of this type overlays the first MCB in memory we get:
chain = 4D ('M'interpreted as char)
pid = 08 00 (0x0008 interpreted as unsigned)
psize = E7 20 (0x20EF interpreted as unsigned)
This occurs because the contents of offset 0 are assigned to chain, offsets 1
and 2 to pid, and offsets 3 and 4 to psize, which you would expect upon
examination of the definition of struct MCB. (When interpreting the results
remember that Intel processors use "backwords" storage.)
Furthermore, the size of struct MCB is 0x10 (16 decimal), so each unit
increment of a pointer to such a structure (ptrmcb in my published MCB.C
listing) increases the address of the pointer by 0x10 bytes. The next MCB is
thus located at 0x974:0 + 0x20EF.* 0x10 + 1 = 0x2A5C:0 and agrees with MCB NO.
02 in Example 2.
With word alignment, the compiler generates code to start each field of a
structure on an even address boundary, which provides for quicker execution on
the 80286 and 80386 processors. Usually such adjustments by the compiler make
no difference. However, in a case where a structure is used as a template
(i.e., used to overlay and retrieve existing data in memory written by some
other process), such compiler adjustments are generally unwanted. Let's see
what goes wrong in the case at hand.
With word alignment we get chain = 4D ('M' interpreted as char) pid = 00 E7
(0x0008 interpreted as unsigned) psize = 20 00 (0x20 interpreted as unsigned)
This is because the compiler has padded the byte after the chain field with an
extra byte so that the remaining structure fields pid, psize, and unused[11]
all start on word boundaries. The unused[11] field is also padded on the end
with an extra byte so that the structure holds an even number of bytes (0x12 =
18 in this case). The vales of pid and psize are then read from the contents
of offsets 2-3 and 4-5, respectively. You can imagine the problems this
creates since the proper values are contained in offsets 1-2 and 3-4 instead.
Furthermore, each increment of a pointer to this word aligned structure
increases the pointer's address by 0x12 bytes, not by 0x10. It is no wonder
the main logic of the MCB program (chain successively from the first to the
last MCB) is immediately derailed leading to the print out of garbage.
Reader Sonntag informs me that the statement #pragma pac(1) (an MSC compiler
directive) placed at the top of the MCB.C source file will cause MSC 5.1 to
compile with byte alignment. I would imagine that an equivalent command line
switch (for MSC) or a menu selection (for Quick C) exists to accomplish the
same thing. In any case, I strongly recommend that Codeview be used to step
through MCB.EXE to drive home the ideas I have presented.


Answers Anyone?


Dear DDJ,
Tandy Corp. offers a 20-Mbyte hard disk for its mode 1400 LT portable
computer, which I eagerly purchased hoping to finally have a gem of a machine.
But alas ... problems with the power supply destroyed my aspirations.
It seems to me that the AC power adapter supplied with the 1400 LT does not
produce enough current to sustain the 1400 LT and cannot operate the computer
(and hard disk) on AC only. My question is as follows: Is it possible to
substitute the AC power adapters supplied with the unit with a more powerful
one? If so, how can I obtain or build one?
The following information could possibly be of help in solving my problem:
Tandy AC adapter (supplied with unit); class 2 transformer; input 120V AC,
60Hz, 20W; and output 15V DC, 700mA. Also, I have a LiteDrive-II hard disk
subsystem by CMS Enhancements.
A solution to this problem would greatly benefit me and my fellow 1400 LT
users.
Wilberto Garcia
456 Beach 63rd St.
Arverne, NY 11692


Ecstatic About Al


Dear DDJ,
I am writing the first letter to a magazine that I have ever written in my (47
years) life, in response to the recent series by Al Stevens. In a nutshell, I
think the series has been fantastic!
I started programming 25 years ago in Cobol; I have written in Assembler,
Fortran, Basic, Easycoder, and Autocoder, and I have thoroughly enjoyed the "C
Column." As a youngster I used to enjoy saving my nickels and buying Popular
Mechanics magazine (for 25 cents), taking it home, building something based on
an article, and then showing it off to my family and friends. Thanks to Al
Stevens' column, I can enjoy the same pleasure and satisfaction that I did so
many years ago.
I am a programmer: I write computer software (for a firm in Silicon Valley),
and I still enjoy tinkering. Thanks to Al's great column, I look forward to
Dr. Dobb's Journal every month like I used to look forward to the latest
Popular Mechanics.
Don't let the flak from the Microsoft C folks derail a great project. The
serious programmer doesn't want to hear the politics about which C compiler is
better or why. Let the Dvoraks handle that, and keep Al Stevens busy with the
projects that are so much fun and so rewarding. Thanks a million!
Richard M. Linder
Los Gatos, Calif.


Mad About Modula-2


Dear DDJ,
I take umbrage with your (Kent Porter's) complaints on the wordiness of
Modula-2 library procedures. Even the most primitive of text editors (and the
JPI environment) will perform global substitutions; thus, one need only type
some small acronym until one is finished and replace them later. Actually, I
tend to place reassignments such as
 CONST WRiteChar = IO.WrChar;
in my modules because I personally find the abbreviation "Wr" somewhat
damaging to the eye.
The aspect of Modula-2 that I take dearest to heart is the necessity of
explicitly stating the location of the procedures used. In contrast, C will
freely accept IOUs in lieu of code, and it's not until the linker hauls you up
before the bench that you know something is missing. Modula-2 demands cash on
the barrelhead (or at least in the definition modules), or one goes nowhere.

The notion of type transfer allows you to do as much damage in this language
as in C. Simply assign an arbitrary code location to a PROC-type variable and
then call it to see this. Perhaps one could convert this into some sort of
inline code for JPI (by assigning the location of a constant aggregate), but
it doesn't seem worth it, for their assembler meshes so nicely with their
compiler.
I hope your successor (Jeff Duntemann) will not forgo Modula-2 for Turbo
Pascal.
John O. Goyo



























































April, 1989
MORE MEMORY FOR DOS EXEC


Swapping calling programs in and out of expanded memory is useful when
programs invoke each other




Kim Kokkonen


Kim Kokkonen is the president of TurboPower Software and the author of many
public domain Turbo Pascal tools. He can be reached at P.O. Box 66747, Scotts
Valley, CA 95066.


As many have lamented, the 640K of memory available to DOS programs is looking
smaller every year. With TSRs gobbling up memory on one end, and applications
growing larger on the other, it is easy to use up all the space. Of course,
necessity is the mother of invention, so desperate DOS programmers have
devised a number of ad hoc methods --using expanded and extended memory,
overlays, and so on --to cram more functions into the same space.
This article describes another such method. I've enhanced the DOS Exec
function by swapping most of the calling program into expanded memory or to
disk, and giving all that free memory to the child process. When the
subprocess is complete, the calling program is swapped back into place and
continues normally. This technique is especially valuable for menuing
environments that must Execute other large programs, or modern programming
editors that are expected to spawn huge compilations at the touch of a key. In
fact, it's useful for any program that must invoke another.
The swapping Exec function is implemented in a Turbo Pascal 5.0 unit called
ExecSwap. The meat of the code is written in assembly language, however, and
with some changes could be linked into other languages.


Turbo Pascal Program Organization


In order for me to explain how ExecSwap works, we'll need to delve into the
organization of a Turbo Pascal program. Let's examine this program called X,
shown in Example 1.
Example 1: An examination of program X

 program X;
 uses {System,} Dos, Crt;
 begin
 ClrScr;
 Exec('C:\COMMAND.COM',");
 end.

What this program does isn't important. I'll use it just to show the
arrangement of memory. X uses two of Turbo's standard units, Crt and Dos. It
also implicitly uses the System unit, as does every Turbo Pascal program.
Table 1 maps out the various segments. (You can see a similar map of a real
program by having the compiler create a MAP file and inspecting the segment
map at the beginning of that file.) It's important to note that each Pascal
unit has its own code segment (denoted by CS_CRT, etc. in Table 1), and that
the code segments are arranged in what might seem like reverse order. That is,
the unit appearing first in the USES statement is linked at the highest memory
address, and the main program has the lowest code segment. Also, if the
program doesn't need to use the heap, the memory above the heap base may not
be allocated.
Table 1: Memory map of example program

 PSP: program segment prefix lower addresses
 CS_X: X code 
 CS_Crt: Crt code 
 CS_Dos: Dos code v
 CS_System: System code higher addresses
 DS: initialized data 
 uninitialized data 
 SS: stack v
 HeapOrg: heap base
 HeapPtr: heap high water mark
 available heap space
 FreePtr: free list
 FreePtr+1000h: top of program
 available DOS memory
 xxxx: top of memory

ExecSwap's goal is to copy most of the memory used by the program to secondary
storage and then to deallocate that memory. ExecSwap needs to leave only
enough of itself behind to call DOS Exec and restore the image when the child
process returns.
By this criterion the best place for ExecSwap's code would be in the main body
of the program. This way it could start swapping memory at the lowest possible
code segment and free the most memory for the child process. In Table 1's
terms it would start swapping at code segment CS_X and continue to the top of
the program. After deallocating memory, the only overhead would be the program
segment prefix (256 bytes) plus the portion of segment CS_X required to undo
the swap. Example 2 shows what memory might look like while the child process
was active. The rest of program X would have been stored in EMS memory if
available, or in a disk file if not.
Example 2: Memory map while child process is active

 PSP: program segment prefix ExecSwap
 CS_X: X code (partial) overhead

 .---------------------------------------------------
 child program program segment prefix
 ...
 xxxx: top of memory

There's another factor to consider, though. ExecSwap should be convenient to
use in more than just one program. Hence, I've made it a self-contained unit
which is available just by adding it to the main program's USES statement.
Considering Table 1 again, it's clear that when you USE ExecSwap you want to
add it at the end of the list. In that case, the memory map will look like
Example 3. The memory that remains allocated during the Exec is the PSP, the
code in the main program X, and whatever part of ExecSwap must remain
resident.
Example 3: Memory map after using ExecSwap

 PSP: program segment prefix
 CS_X: X code
 CS_ExecSwap: ExecSwap code <------
 CS_Crt: Crt code
 CS_Dos: Dos code
 CS_System: System code
 ...
 xxxx: top of memory

The main program's code segment need not be large, of course. In the extreme
case, the main program would consist of nothing but a USES statement and a
single procedure call to another unit. This reduces the overhead of the Exec
call to essentially just the PSP plus ExecSwap itself. And that's not much:
ExecSwap's resident portion consumes less than 2,000 bytes.


Using ExecSwap


Before we plunge into the mechanics of ExecSwap, I'll describe how it is used
by an application. The unit interfaces three routines, shown in Example 4 .
Before performing an Exec call, the program must call InitExecSwap. This
routine computes how many bytes to swap and allocates space to store the
swapped region.
Example 4: ExecSwap routines

 function InitExecSwap (LastToSave : Pointer; SwapFileName:
 String) : Boolean;
 {-Initialize for swapping, returning TRUE if successful}

 function ExecWithSwap (Path, CmdLine : String) : Word;
 {-DOS Exec supporting swap to EMS or disk}

 procedure ShutdownExecSwap;
 {-Deallocate swap area}

The swapped region of memory starts just beyond the resident portion of
ExecSwap. The programmer must specify the end of the region with the parameter
LastToSave because the choice depends on how the program uses the heap. What
you choose for LastToSave affects only the size of the swap file or the amount
of EMS memory needed; it has no effect on resident overhead during the Exec
call.
There are three reasonable values for LastToSave. Passing the System variable
HeapOrg tells ExecSwap not to save any part of the heap; this is the correct
option for programs that make no use of the heap. Passing the system variable
HeapPtr causes ExecSwap to save all allocated portions of the heap. Only the
free list is ignored, so this is a good choice for programs that don't
fragment the heap. Passing the expression Ptr(Seg (FreePtr^)+$1000, 0) tells
ExecSwap to save the entire heap, including the free list. This is the most
conservative option, but it may lead to swap files approaching 640K bytes in
size.
InitExecSwap's second parameter, SwapFileName, specifies the name and location
of the swap file. If EMS memory is available, this name won't be used;
otherwise InitExecSwap will create a new file. InitExecSwap assures that
sufficient EMS or disk space exists for the swap; if not, it returns FALSE. To
minimize swap times, it's a good idea to put the swap file on the fastest
drive that will hold it. It's also prudent to avoid a floppy drive because the
user may change disks while the child process is active. The swap file remains
open, using a file handle, until ShutdownExecSwap is called or the program
ends. InitExecSwap marks the file with the Hidden and System attributes so
that the user of the child process won't be tempted to delete it.
ExecWithSwap is analogous to the standard Exec procedure in Turbo's Dos unit.
Its first parameter is the path name of the program to Execute, and its second
is the command line to pass to it. The only difference from Exec is that
ExecWithSwap is a function, returning the status of the call in a Word. The
function returns DOS error codes, with one exception. The most common error
codes are:
 0 success
 1 swap error (no swap storage, disk error, EMS error)
 2 file not found
 3 path not found
 8 insufficient memory
You may never need to call ShutdownExecSwap, since ExecSwap sets up an exit
handler that automatically calls it when the program ends. In some cases,
however, you may want to close and erase the swap file or regain EMS space
before continuing.
There's a conundrum here. ExecSwap should be last in the USES list, but the
main program should do as little as possible. So where do you place calls to
the ExecSwap routines? It's easiest to call them from the main program and
take the hit in overhead. Turbo Pascal provides a better key to the puzzle,
though. Version 5 supports procedure variables, and Version 4 makes it easy to
fake them. So do this: In the main program, assign the address of each
ExecSwap procedure to a procedure variable declared in a unit used early in
the USES list. Then call ExecSwap's routines in any later unit by referring to
the procedure variables.
One caution about using ExecSwap: Because most of your program's code isn't in
memory while the child process runs, it's essential that the program's
interrupt handlers be deactivated first. Turbo Pascal 5 provides a handy
procedure called SwapVectors that does this for all the System interrupt
handlers. Call SwapVectors just before and after ExecWithSwap, and treat any
of your own handlers in a similar fashion.
Listing One, page 66, offers a simple example of using ExecSwap. You can
assemble ExecSwap.ASM (Listing Three, page 66) using MASM 4.0 or later, or any
compatible assembler. Then compile the test program to an EXE file and run it,
and you'll enter a DOS shell. If you have a DOS memory mapping utility, you'll
see that the TEST program is using less than 3K of memory. The swap file uses
about 20K, most of that for the 16K stack, which is Turbo's default. If the
swap goes to EMS, the EMS block will be 32K bytes, because EMS is allocated in
16K chunks. Type Exit to leave the shell, and the test program will regain
control.
A real program provides more impressive results. We developed ExecSwap for use
in our Turbo Analyst product, which offers an integrated environment where the
programmer can edit source files, then Exec the compiler, debugger, or any of
many other programming utilities. Without benefit of ExecSwap, the environment
keeps about 25OK of memory during the Exec. With ExecSwap, the overhead is
only about 4K. That 246K makes a difference!


How It's Done


ExecSwap's Pascal source file, ExecSwap.PAS, is given in Listing Two, page 66.
It's little more than a shell for the assembly language routines in
ExecSwap.ASM, Listing Three.
Looking at InitExecSwap in Listing Two, you'll see that it checks first for
EMS memory (any version of EMS will do). If that is available, it is used in
preference to disk storage. If not, InitExecSwap goes on to assure that
there's enough space on the specified drive to hold the swap area. The
production version of ExecSwap (trimmed here for the sake of brevity), checks
that the drive doesn't hold removable media. InitExecSwap also stores several
items in global variables, where they're easily accessible by the assembly
language routines, and installs an exit handler to clean up after itself in
case the program halts unexpectedly.
The tricky stuff is in ExecSwap.ASM. The file starts with the standard
boilerplate needed for linking to Turbo Pascal. A number of temporary
variables in the code segment are declared; these are essential because the
entire data segment is gone during critical portions of ExecWithSwap. One of
these variables is a temporary stack. It's a small one, only 128 bytes, but it
is required because the normal Turbo Pascal stack is also swapped out. Macro
definitions follow; more than the usual number of macros are used to keep the
Listing to a reasonable length.
ExecWithSwap starts by copying a number of variables into the code segment.
Then it checks to see whether swapping will go to EMS or disk. If neither has
been activated, ExecWithSwap exits immediately, returning error code 1.
Otherwise, ExecWithSwap processes one of four similar loops: one each to swap
to or from disk or EMS storage. Let's trace the "swap to EMS" loop in detail,
at label WriteE. The sequence for swapping to disk is so similar that I won't
need to describe it here.

First map EMS memory, making the first 16K page of the EMS swap area
accessible through the page window at FrameSeg:0. (Note that ExecSwap doesn't
save the EMS context; if your application uses EMS for other storage, be sure
to remap EMS after returning from ExecWithSwap.) The macro SetSwapCount then
computes how many bytes to copy into the first page, returning a full 16K
bytes unless it's also the last page. The first location to save is at label
FirstToSave, which immediately follows the ExecWithSwap routine. The MoveFast
macro copies the first swap block into the EMS window. BX is then incremented
to select the next logical EMS page, and the DS register is adjusted to point
to the next swap block, 16K bytes higher in memory. The loop continues until
all the bytes have been copied.
Next, modify the DOS memory allocation, so that the space just swapped out is
available to the child process. First, save the current allocated size so you
can restore it later. Then switch to the small temporary stack, which is
safely nestled in the code segment. Finally, call the DOS SetBlock function to
shrink the memory to just beyond the end of the ExecWithSwap routine.
The actual DOS Exec call follows. The implementation here is similar to the
one in Borland's Dos unit. It validates and formats the program path and
command line, parses file control blocks (FCBs) from the command line in case
the child expects them, and calls the DOS Exec function. The error code
returned by Exec is stored unit the reverse swap is complete.
The reverse swap is just that: It reallocates memory from DOS and copies the
parent program back into place. There is, however, one critical difference
from the fist swap. Errors that occur during the reverse swap are fatal.
Because the program to return to no longer exists, the only recourse is to
halt. The most likely reason for such an error is the inability to reallocate
the initial memory block. This occurs whenever the Exec call (or the user) has
installed a memory resident program while in the shell. Be sure to warn your
users not to do this! ExecSwap could write an error message before halting; to
save space here, it just sets the ErrorLevel, which can be checked within a
batch file:
 0FFh can't reallocate memory 0FEh disk error 0FDh EMS error
ExecWithSwap is done after it switches back to the original stack, restores
the DS register, and returns the status code.
The remainder of ExecSwap.ASM is a collection of small utility routines, some
of which may find for general use.


In Summary


ExecSwap seems quite reliable. It doesn't depend on any newly discovered
undocumented features of DOS, and has been tested by thousands of users.
There are a few additional features it might have. The production version
writes status messages while swapping, so nervous users don't think their hard
disks are being formatted. It might also support direct swapping to extended
memory; this hasn't been attempted because experience indicates that using
extended memory in a DOS application is a compatibility nightmare, and RAM
disks seem quite adequate for swapping. If the remainder of ExecSwap were
converted to assembly language, Turbo Pascal's link order conventions (within
a unit) could be circumvented, and another 500 bytes or so of Exec overhead
would be saved. With a few more DOS memory management calls, it would be
possible for the parent and child processes to share a common data area. An
extension of the ExecSwap concept allows TSR programs to leave just a core of
interrupt handlers in memory, and swap the application code in when they pop
up (SideKick Plus apparently does this).
The ExecSwap unit has become a very useful item in my bag of tricks. With an
ExecSwap-based DOS shell in the programming editor I use, I can achieve the
kind of multitasking I need ("interruption-based" multitasking). ExecSwap
should make it easier for you to squeeze more functionality into that 640K box
as well.


Acknowledgment


Special thanks to Chris Franzen of West Germany, who added disk swapping
capability to the original unit, which had supported only EMS.


[LISTING ONE]

program TestExecSwap;
uses
 Dos,ExecSwap; {Keep ExecSwap last}
const
 SwapLoc : array[Boolean] of String[7] = ('on disk', 'in EMS');
var
 Status : Word;
begin
 if not InitExecSwap(HeapPtr, 'SWAP.$$$') then
 WriteLn('Unable to allocate swap space')
 else begin
 WriteLn('Allocated ', BytesSwapped, ' bytes ', SwapLoc[EmsAllocated]);
 SwapVectors;
 Status := ExecWithSwap(GetEnv('COMSPEC'), '');
 SwapVectors;
 WriteLn('Execu status; ', Status);
 end;
end.





[LISTING TWO]

{Copyright (c) 1988 TurboPower Software}
{May be used freely as long as due credit is given}
{$R-,S-}
unit ExecSwap;
 {-Memory-efficient DOS EXEC call}
interface

const
 BytesSwapped : LongInt = 0; {Bytes to swap to EMS/disk}

 EmsAllocated : Boolean = False; {True when EMS allocated for swap}
 FileAllocated : Boolean = False; {True when file allocated for swap}

function ExecWithSwap(Path, CmdLine : String) : Word;
 {-DOS EXEC supporting swap to EMS or disk}

function InitExecSwap(LastToSave : Pointer; SwapFileName : String) : Boolean;
 {-Initialize for swapping, returning TRUE if successful}

procedure ShutdownExecSwap;
 {-Deallocate swap area}

implementation

var
 EmsHandle : Word; {Handle of EMS allocation block}
 FrameSeg : Word; {Segment of EMS page frame}
 FileHandle : Word; {DOS handle of swap file}
 SwapName : String[80]; {ASCIIZ name of swap file}
 SaveExit : Pointer; {Exit chain pointer}

 {$L EXECSWAP}
 function ExecWithSwap(Path, CmdLine : String) : Word; external;
 procedure FirstToSave; external;
 function AllocateSwapFile : Boolean; external;
 procedure DeallocateSwapFile; external;

 {$F+} {These routines could be interfaced for general use}
 function EmsInstalled : Boolean; external;
 function EmsPageFrame : Word; external;
 function AllocateEmsPages(NumPages : Word) : Word; external;
 procedure DeallocateEmsHandle(Handle : Word); external;
 function DefaultDrive : Char; external;
 function DiskFree(Drive : Byte) : LongInt; external;

 procedure ExecSwapExit;
 begin
 ExitProc := SaveExit;
 ShutdownExecSwap;
 end;
 {$F-}

 procedure ShutdownExecSwap;
 begin
 if EmsAllocated then begin
 DeallocateEmsHandle(EmsHandle);
 EmsAllocated := False;
 end else if FileAllocated then begin
 DeallocateSwapFile;
 FileAllocated := False;
 end;
 end;

 function PtrDiff(H, L : Pointer) : LongInt;
 type
 OS = record O, S : Word; end; {Convenient typecast}
 begin
 PtrDiff := (LongInt(OS(H).S) shl 4+OS(H).O)-
 (LongInt(OS(L).S) shl 4+OS(L).O);

 end;

 function InitExecSwap(LastToSave : Pointer;
 SwapFileName : String) : Boolean;
 const
 EmsPageSize = 16384; {Bytes in a standard EMS page}
 var
 PagesInEms : Word; {Pages needed in EMS}
 BytesFree : LongInt; {Bytes free on swap file drive}
 DriveChar : Char; {Drive letter for swap file}
 begin
 InitExecSwap := False;

 if EmsAllocated or FileAllocated then
 Exit;
 BytesSwapped := PtrDiff(LastToSave, @FirstToSave);
 if BytesSwapped <= 0 then
 Exit;
 SaveExit := ExitProc;
 ExitProc := @ExecSwapExit;

 if EmsInstalled then begin
 PagesInEms := (BytesSwapped+EmsPageSize-1) div EmsPageSize;
 EmsHandle := AllocateEmsPages(PagesInEms);
 if EmsHandle <> $FFFF then begin
 EmsAllocated := True;
 FrameSeg := EmsPageFrame;
 if FrameSeg <> 0 then begin
 InitExecSwap := True;
 Exit;
 end;
 end;
 end;
 if Length(SwapFileName) <> 0 then begin
 SwapName := SwapFileName+#0;
 if Pos(':', SwapFileName) = 2 then
 DriveChar := Upcase(SwapFileName[1])
 else
 DriveChar := DefaultDrive;
 BytesFree := DiskFree(Byte(DriveChar)-$40);
 FileAllocated := (BytesFree > BytesSwapped) and AllocateSwapFile;
 if FileAllocated then
 InitExecSwap := True;
 end;
 end;
end.





[LISTING THREE]

;EXECSWAP.ASM
; Swap memory and exec another program
; Copyright (c) 1988 TurboPower Software
; May be used freely as long as due credit is given
;-----------------------------------------------------------------------------
DATA SEGMENT BYTE PUBLIC

 EXTRN BytesSwapped:DWORD ;Bytes to swap to EMS/disk
 EXTRN EmsAllocated:BYTE ;True when EMS allocated for swap
 EXTRN FileAllocated:BYTE ;True when file allocated for swap
 EXTRN EmsHandle:WORD ;Handle of EMS allocation block
 EXTRN FrameSeg:WORD ;Segment of EMS page frame
 EXTRN FileHandle:WORD ;Handle of DOS swap file
 EXTRN SwapName:BYTE ;ASCIIZ name of swap file
 EXTRN PrefixSeg:WORD ;Base segment of program
DATA ENDS
;-----------------------------------------------------------------------------
CODE SEGMENT BYTE PUBLIC
 ASSUME CS:CODE,DS:DATA
 PUBLIC ExecWithSwap,FirstToSave
 PUBLIC AllocateSwapFile,DeallocateSwapFile
 PUBLIC DefaultDrive,DiskFree
 PUBLIC EmsInstalled,EmsPageFrame
 PUBLIC AllocateEmsPages,DeallocateEmsHandle
;-----------------------------------------------------------------------------
FileAttr EQU 6 ;Swap file attribute (hidden+system)
EmsPageSize EQU 16384 ;Size of EMS page
FileBlockSize EQU 32768 ;Size of a file block
StkSize EQU 128 ;Bytes in temporary stack
lo EQU (WORD PTR 0) ;Convenient typecasts
hi EQU (WORD PTR 2)
ofst EQU (WORD PTR 0)
segm EQU (WORD PTR 2)
;-----------------------------------------------------------------------------
;Variables in CS
EmsDevice DB 'EMMXXXX0',0 ;Name of EMS device driver
UsedEms DB 0 ;1 if swapping to EMS, 0 if to file
BytesSwappedCS DD 0 ;Bytes to move during a swap
EmsHandleCS DW 0 ;EMS handle
FrameSegCS DW 0 ;Segment of EMS page window
FileHandleCS DW 0 ;DOS file handle
PrefixSegCS DW 0 ;Segment of base of program
Status DW 0 ;ExecSwap status code
LeftToSwap DD 0 ;Bytes left to move
SaveSP DW 0 ;Original stack pointer
SaveSS DW 0 ;Original stack segment
PathPtr DD 0 ;Pointer to program to execute
CmdPtr DD 0 ;Pointer to command line to execute
ParasWeHave DW 0 ;Paragraphs allocated to process
CmdLine DB 128 DUP(0) ;Terminated command line passed to DOS
Path DB 64 DUP(0) ;Terminated path name passed to DOS
FileBlock1 DB 16 DUP(0) ;FCB passed to DOS
FileBlock2 DB 16 DUP(0) ;FCB passed to DOS
EnvironSeg DW 0 ;Segment of environment for child
CmdLinePtr DD 0 ;Pointer to terminated command line
FilePtr1 DD 0 ;Pointer to FCB file
FilePtr2 DD 0 ;Pointer to FCB file
TempStack DB StkSize DUP(0) ;Temporary stack
StackTop LABEL WORD ;Initial top of stack
;-----------------------------------------------------------------------------
;Macros
MovSeg MACRO Dest,Src ;Set one segment register to another
 PUSH Src
 POP Dest
 ENDM


MovMem MACRO Dest,Src ;Move from memory to memory via AX
 MOV AX,Src
 MOV Dest,AX
 ENDM

InitSwapCount MACRO ;Initialize counter for bytes to swap
 MovMem LeftToSwap.lo,BytesSwappedCS.lo
 MovMem LeftToSwap.hi,BytesSwappedCS.hi
 ENDM

SetSwapCount MACRO BlkSize ;Return CX = bytes to move this block
 LOCAL FullBlk ;...and reduce total bytes left to move
 MOV CX,BlkSize ;Assume we'll write a full block
 CMP LeftToSwap.hi,0 ;Is high word still non-zero?
 JNZ FullBlk ;Jump if so
 CMP LeftToSwap.lo,BlkSize ;Low word still a block or more?
 JAE FullBlk ;Jump if so
 MOV CX,LeftToSwap.lo ;Otherwise, move what's left
FullBlk:SUB LeftToSwap.lo,CX ;Reduce number left to move
 SBB LeftToSwap.hi,0
 ENDM

NextBlock MACRO SegReg, BlkSize ;Point SegReg to next block to move
 MOV AX,SegReg
 ADD AX,BlkSize/16 ;Add paragraphs to next segment
 MOV SegReg,AX ;Next block to move
 MOV AX,LeftToSwap.lo
 OR AX,LeftToSwap.hi ;Bytes left to move?
 ENDM

EmsCall MACRO FuncAH ;Call EMM and prepare to check result
 MOV AH,FuncAH ;Set up function
 INT 67h
 OR AH,AH ;Error code in AH
 ENDM

DosCallAH MACRO FuncAH ;Call DOS subfunction AH
 MOV AH,FuncAH
 INT 21h
 ENDM

DosCallAX MACRO FuncAX ;Call DOS subfunction AX
 MOV AX,FuncAX
 INT 21h
 ENDM

InitSwapFile MACRO
 MOV BX,FileHandleCS ;BX = handle of swap file
 XOR CX,CX
 XOR DX,DX ;Start of file
 DosCallAX 4200h ;DOS file seek
 ENDM

HaltWithError MACRO Level ;Halt if non-recoverable error occurs
 MOV AL,Level ;Set errorlevel
 DosCallAH 4Ch
 ENDM

MoveFast MACRO ;Move CX bytes from DS:SI to ES:DI

 CLD ;Forward
 RCR CX,1 ;Convert to words
 REP MOVSW ;Move the words
 RCL CX,1 ;Get the odd byte, if any
 REP MOVSB ;Move it
 ENDM

SetTempStack MACRO ;Switch to temporary stack
 MOV AX,OFFSET StackTop ;Point to top of stack
 MOV BX,CS ;Temporary stack in this code segment
 CLI ;Interrupts off
 MOV SS,BX ;Change stack
 MOV SP,AX
 STI ;Interrupts on
 ENDM
;-----------------------------------------------------------------------------
;function ExecWithSwap(Path, CmdLine : string) : Word;
ExecWithSwap PROC FAR
 PUSH BP
 MOV BP,SP ;Set up stack frame

;Move variables to CS where we can easily access them later
 MOV Status,1 ;Assume failure
 LES DI,[BP+6] ;ES:DI -> CmdLine
 MOV CmdPtr.ofst,DI
 MOV CmdPtr.segm,ES ;CmdPtr -> command line string
 LES DI,[BP+10] ;ES:DI -> Path
 MOV PathPtr.ofst,DI
 MOV PathPtr.segm,ES ;PathPtr -> path to execute
 MOV SaveSP,SP ;Save stack position
 MOV SaveSS,SS
 MovMem BytesSwappedCS.lo,BytesSwapped.lo
 MovMem BytesSwappedCS.hi,BytesSwapped.hi
 MovMem EmsHandleCS,EmsHandle
 MovMem FrameSegCS,FrameSeg
 MovMem FileHandleCS,FileHandle
 MovMem PrefixSegCS,PrefixSeg
 InitSwapCount ;Initialize bytes LeftToSwap

;Check for swapping to EMS or file
 CMP EmsAllocated,0 ;Check flag for EMS method
 JZ NotEms ;Jump if EMS not used
 JMP WriteE ;Swap to EMS
NotEms: CMP FileAllocated,0 ;Check flag for swap file method
 JNZ WriteF ;Swap to file
 JMP ESDone ;Exit if no swapping method set

;Write to swap file
WriteF: MovSeg DS,CS ;DS = CS
 InitSwapFile ;Seek to start of swap file
 JNC EF0 ;Jump if success
 JMP ESDone ;Exit if error
EF0: SetSwapCount FileBlockSize ;CX = bytes to write
 MOV DX,OFFSET FirstToSave ;DS:DX -> start of region to save
 DosCallAH 40h ;File write
 JC EF1 ;Jump if write error
 CMP AX,CX ;All bytes written?
 JZ EF2 ;Jump if so
EF1: JMP ESDone ;Exit if error

EF2: NextBlock DS,FileBlockSize ;Point DS to next block to write
 JNZ EF0 ;Loop if bytes left to write
 MOV UsedEms,0 ;Flag we used swap file for swapping
 JMP SwapDone ;Done swapping out

;Write to EMS
WriteE: MOV ES,FrameSeg ;ES -> page window
 MOV DX,EmsHandle ;DX = handle of our EMS block
 XOR BX,BX ;BX = initial logical page
 MovSeg DS,CS ;DS = CS
EE0: XOR AL,AL ;Physical page 0
 EmsCall 44h ;Map physical page
 JZ EE1 ;Jump if success
 JMP ESDone ;Exit if error
EE1: SetSwapCount EmsPageSize ;CX = Bytes to move
 XOR DI,DI ;ES:DI -> base of EMS page
 MOV SI,OFFSET FirstToSave ;DS:SI -> region to save
 MoveFast ;Move CX bytes from DS:SI to ES:DI
 INC BX ;Next logical page
 NextBlock DS,EmsPageSize ;Point DS to next page to move
 JNZ EE0 ;Loop if bytes left to move
 MOV UsedEms,1 ;Flag we used EMS for swapping

;Shrink memory allocated to this process
SwapDone:MOV AX,PrefixSegCS
 MOV ES,AX ;ES = segment of our memory block
 DEC AX
 MOV DS,AX ;DS = segment of memory control block
 MOV CX,DS:[0003h] ;CX = current paragraphs owned
 MOV ParasWeHave,CX ;Save current paragraphs owned
 SetTempStack ;Switch to temporary stack
 MOV AX,OFFSET FirstToSave+15
 MOV CL,4
 SHR AX,CL ;Convert offset to paragraphs
 ADD BX,AX
 SUB BX,PrefixSegCS ;BX = new paragraphs to keep
 DosCallAH 4Ah ;SetBlock
 JNC EX0 ;Jump if successful
 JMP EX5 ;Swap back and exit

;Set up parameters and call DOS Exec
EX0: MOV AX,ES:[002Ch] ;Get environment segment
 MOV EnvironSeg,AX
 MovSeg ES,CS ;ES = CS
 LDS SI,PathPtr ;DS:SI -> path to execute
 MOV DI,OFFSET Path ;ES:DI -> local ASCIIZ copy
 CLD
 LODSB ;Read current length
 CMP AL,63 ;Truncate if exceeds space set aside
 JB EX1
 MOV AL,63
EX1: MOV CL,AL
 XOR CH,CH ;CX = bytes to copy
 REP MOVSB
 XOR AL,AL
 STOSB ;ASCIIZ terminate
 LDS SI,CmdPtr ;DS:SI -> Command line to pass
 MOV DI,OFFSET CmdLine ;ES:DI -> Local terminated copy
 LODSB

 CMP AL,126 ;Truncate command if exceeds space
 JB EX2
 MOV AL,126
EX2: STOSB
 MOV CL,AL
 XOR CH,CH ;CX = bytes to copy
 REP MOVSB
 MOV AL,0DH ;Terminate with ^M
 STOSB
 MovSeg DS,CS ;DS = CS
 MOV SI,OFFSET CmdLine
 MOV CmdLinePtr.ofst,SI
 MOV CmdLinePtr.segm,DS ;Store pointer to command line
 INC SI
 MOV DI,OFFSET FileBlock1
 MOV FilePtr1.ofst,DI
 MOV FilePtr1.segm,ES ;Store pointer to filename 1, if any
 DosCallAX 2901h ;Parse FCB
 MOV DI,OFFSET FileBlock2
 MOV FilePtr2.ofst,DI
 MOV FilePtr2.segm,ES ;Store pointer to filename 2, if any
 DosCallAX 2901h ;Parse FCB
 MOV DX,OFFSET Path
 MOV BX,OFFSET EnvironSeg
 DosCallAX 4B00h ;Exec
 JC EX3 ;Jump if error in DOS call
 XOR AX,AX ;Return zero for success
EX3: MOV Status,AX ;Save DOS error code

;Set up temporary stack and reallocate original memory block
 SetTempStack ;Set up temporary stack
 MOV ES,PrefixSegCS
 MOV BX,ParasWeHave
 DosCallAH 4Ah ;SetBlock
 JNC EX4 ;Jump if no error
 HaltWithError 0FFh ;Must halt if failure here
EX4: InitSwapCount ;Initialize LeftToSwap

;Check which swap method is in use
EX5: CMP UsedEms,0
 JZ ReadF ;Jump to read back from file
 JMP ReadE ;Read back from EMS

;Read back from swap file
ReadF: MovSeg DS,CS ;DS = CS
 InitSwapFile ;Seek to start of swap file
 JNC EF3 ;Jump if we succeeded
 HaltWithError 0FEh ;Must halt if failure here
EF3: SetSwapCount FileBlockSize ;CX = bytes to read
 MOV DX,OFFSET FirstToSave ;DS:DX -> start of region to restore
 DosCallAH 3Fh ;Read file
 JNC EF4 ;Jump if no error
 HaltWithError 0FEh ;Must halt if failure here
EF4: CMP AX,CX
 JZ EF5 ;Jump if full block read
 HaltWithError 0FEh ;Must halt if failure here
EF5: NextBlock DS,FileBlockSize ;Point DS to next page to read
 JNZ EF3 ;Jump if bytes left to read
 JMP ESDone ;We're done


;Copy back from EMS
ReadE: MOV DS,FrameSegCS ;DS -> page window
 MOV DX,EmsHandleCS ;DX = handle of our EMS block
 XOR BX,BX ;BX = initial logical page
 MovSeg ES,CS ;ES = CS
EE3: XOR AL,AL ;Physical page 0
 EmsCall 44h ;Map physical page
 JZ EE4 ;Jump if success
 HaltWithError 0FDh ;Must halt if failure here
EE4: SetSwapCount EmsPageSize ;CX = Bytes to move
 XOR SI,SI ;DS:SI -> base of EMS page
 MOV DI,OFFSET FirstToSave ;ES:DI -> region to restore
 MoveFast ;Move CX bytes from DS:SI to ES:DI
 INC BX ;Next logical page
 NextBlock ES,EmsPageSize ;Point ES to next page to move
 JNZ EE3 ;Jump if so

ESDone: CLI ;Switch back to original stack
 MOV SS,SaveSS
 MOV SP,SaveSP
 STI
 MOV AX,SEG DATA
 MOV DS,AX ;Restore DS
 MOV AX,Status ;Return status
 POP BP
 RET 8 ;Remove parameters and return
ExecWithSwap ENDP
;-----------------------------------------------------------------------------
;Label marks first location to swap
FirstToSave:
;-----------------------------------------------------------------------------
;function AllocateSwapFile : Boolean;
AllocateSwapFile PROC NEAR
 MOV CX,FileAttr ;Attribute for swap file
 MOV DX,OFFSET SwapName+1 ;DS:DX -> ASCIIZ swap name
 DosCallAH 3Ch ;Create file
 MOV FileHandle,AX ;Save handle assuming success
 MOV AL,0 ;Assume failure
 JC ASDone ;Failed if carry set
 INC AL ;Return true for success
ASDone: RET
AllocateSwapFile ENDP
;-----------------------------------------------------------------------------
;procedure DeallocateSwapFile;
DeallocateSwapFile PROC NEAR
 MOV BX,FileHandle ;Handle of swap file
 DosCallAH 3Eh ;Close file
 XOR CX,CX ;Normal attribute
 MOV DX,OFFSET SwapName+1 ;DS:DX -> ASCIIZ swap name
 DosCallAX 4301h ;Set file attribute
 DosCallAH 41h ;Delete file
 RET
DeallocateSwapFile ENDP
;-----------------------------------------------------------------------------
;function EmsInstalled : Boolean;
EmsInstalled PROC FAR
 PUSH DS
 MovSeg DS,CS ;DS = CS

 MOV DX,OFFSET EmsDevice ;DS:DX -> EMS driver name
 DosCallAX 3D02h ;Open for read/write
 POP DS
 MOV BX,AX ;Save handle in case one returned
 MOV AL,0 ;Assume FALSE
 JC EIDone
 DosCallAH 3Eh ;Close file
 MOV AL,1 ;Return TRUE
EIDone: RET
EmsInstalled ENDP
;-----------------------------------------------------------------------------
;function EmsPageFrame : Word;
EmsPageFrame PROC FAR
 EmsCall 41h ;Get page frame
 MOV AX,BX ;AX = segment
 JZ EPDone ;Done if Error = 0
 XOR AX,AX ;Else segment = 0
EPDone: RET
EmsPageFrame ENDP
;-----------------------------------------------------------------------------
;function AllocateEmsPages(NumPages : Word) : Word;
AllocateEmsPages PROC FAR
 MOV BX,SP ;Set up stack frame
 MOV BX,SS:[BX+4] ;BX = NumPages
 EmsCall 43h ;Allocate EMS
 MOV AX,DX ;Assume success
 JZ APDone ;Done if not 0
 MOV AX,0FFFFh ;$FFFF for failure
APDone: RET 2 ;Remove parameter and return
AllocateEmsPages ENDP
;-----------------------------------------------------------------------------
;procedure DeallocateEmsHandle(Handle : Word);
DeallocateEmsHandle PROC FAR
 MOV BX,SP ;Set up stack frame
 MOV DX,SS:[BX+4] ;DX = Handle
 EmsCall 45h ;Deallocate EMS
 RET 2 ;Remove parameter and return
DeallocateEmsHandle ENDP
;-----------------------------------------------------------------------------
;function DefaultDrive : Char;
DefaultDrive PROC FAR
 DosCallAH 19h ;Get default drive
 ADD AL,'A' ;Convert to character
 RET
DefaultDrive ENDP
;-----------------------------------------------------------------------------
;function DiskFree(Drive : Byte) : LongInt;
DiskFree PROC FAR
 MOV BX,SP ;Set up stack frame
 MOV DL,SS:[BX+4] ;DL = Drive to check
 DosCallAH 36h ;Get disk space
 MOV DX,AX ;Return 0FFFFFFFFh for failure
 CMP AX,0FFFFh ;Bad drive number?
 JZ DFDone ;Jump if so
 MUL CX ;AX = bytes/cluster
 MUL BX ;DX:AX = bytes free
DFDone: RET 2 ;Remove parameter and return
DiskFree ENDP
;-----------------------------------------------------------------------------

CODE ENDS
 END




























































April, 1989
ADVANCED 80386 MEMORY MANAGEMENT


Paging is the 80386's answer to the memory management challenges for today's
multitasking operating systems




Neal Margulis


Neal is an applications engineer for Intel Corp. and can be reached at 2625
Walsh Ave., SC4-40, Santa Clara, CA 95051.


Memory management is a challenge for multitasking operating systems. To combat
this difficulty, the Intel 80386 architecture has a method for managing memory
called "paging," which is in addition to the segmentation features of the
80286. Paging can increase efficiency of virtual memory multitasking operating
systems that run 8086, 80286, and 80386 microprocessor software. This article
explains how paging increases the performance for multitasking operating
systems and why paging is a requirement for multitasking 8086 and 32-bit 80386
microprocessor applications. In order to make use of the information in this
article, you should have basic knowledge of protected mode and segmentation on
the 80286 or 80386 microprocessor.
Both the 8086 and the 80286 address memory with a linear address. For systems
that use these processors, or the 386 CPU without paging, the linear address
is equivalent to the physical address. Address translation on the 386 CPU is
shown in Figure 1. Notice that the paging unit comes after the linear address
calculation. The paging unit translates the logical address seen by the
programs into the physical address that goes out on the bus, which allows
paging to be performed by the operating system but does not impact
applications in any manner.


The Logical Basis


A segment is used to define the task's logical address space, which consists
of one or more segments. The 80286 allows segments of up to 64K, and the 386
microprocessors allow segments up to 4 gigabytes long. As any experienced
programmer knows, segments are visible to the application programmer, although
less so on the 386 due to its larger size.
Microsoft's OS/2 currently uses segments as the basis for its virtual memory
management. The 80286's maximum segment size of 64K makes segment-based
physical memory allocation possible. With the 386 microprocessor, in which
segments can be up to 4 gigabytes, allocating memory on such a large basis is
not practical.
Segmentation-based memory management has additional shortcomings. When
variable sized segments are used for physical memory allocation, for example,
memory fragmentation often occurs. Fragmentation occurs when the free memory
in a system consists of discontinuous small sections. Then when the operating
system needs to load a large segment, it must perform a costly rearranging of
memory. To overcome fragmentation, some segmentation-based schemes allocate
the maximum segment size when any size segment is loaded. Although this
overcomes the fragmentation problem, it wastes memory. Clearly a new method
that overcomes these inefficiencies and works with 32-bit code is needed. The
new method for virtual memory management is paging.


Paging


While paging is enabled, the processor translates a linear address to a
physical address with the aid of page tables. Like mainframe computers, the
page tables are arranged in a two-level hierarchy, as shown in Figure 2. The
page table directory base, which is the control register CR3, points to the
page table directory. The directory is one page long and contains entries for
1,024 page tables. Page tables are also one page long, and the entries in a
page table describe 1,024 pages; each page is 4K in size. As an option, tasks
can have their own page table directory, for there is a page table directory
base associated with each task.
The processor uses the upper 10 bits of the linear address as an index into
the directory. Each directory entry holds 20 bits of addressing information,
which contain the address of a page table. The processor uses these 20 bits
and the middle 10 bits of the linear address to form the page table address.
The address contents of the page table entry and the lower 12 bits of the
linear address form the 32-bit physical address.


The Translation Lookaside Buffer (TLB)


Paging information is stored in the on-chip TLB and in memory. If the
processor had to access these page tables in memory each time a reference was
made, performance would suffer. To save the overhead of the page table
lookups, the processor automatically caches mapping information for 32
recently used pages in an on-chip translation lookaside buffer. The TLB's 32
entries cover 4K, each providing a total cover of 128K of memory addresses.
The TLB is flushed by changing the value of CR3, which is commonly
accomplished by a privileged MOV instruction or a task switch.
As shown in Figure 3, only when the processor does not find the mapping
information for a page in the TLB does it perform a page table lookup from
information stored in memory. To improve hit rates, the TLB is four-way set
associative, meaning each translation can be stored in one of four locations
in the TLB. The result is that for typical systems, 97-99 percent of the
address references are TLB hits, requiring no memory references to translate.
When a TLB miss occurs, the processor replaces an older TLB entry with the new
entry that is likely to be used again. This replacement, called TLB miss
processing, is performed entirely in hardware.


Using Paging for Virtual Memory Management


Virtual memory allows large or multiple programs to be executed as if the
entire program were in memory, even though portions are still on disk. In the
case of a large program that has 20 Mbytes of data, for example, and a
computer that has only 2 Mbytes of memory, the operating system can load and
run the program. An operating system that uses demand paging can multitask
more applications in less physical memory than an operating system that uses
segmentation for memory allocation. The information for efficiently
accomplishing memory management lies within the page directory entries and
page table entries. In Figure 4, in the lower 12 bits of each of these entries
there are several control bits used by the operating system for keeping track
of which pages are in memory, which pages are on disk, information for
deciding which page should be swapped out in favor of a new page, and if the
swapped page needs to be written back to disk or merely discarded.
If set to 1, the P (present) bit indicates that the entry is present in
memory. If the P bit is 0, any attempt to access this page will cause a page
fault (exception 14) prior to the memory access. When a page fault occurs, the
processor passes control to the interrupt 14 handler, part of the operating
system that must read the needed page into memory and return execution to the
program. The handler reads the contents of CR2 to decide which page is
required. If there is no more room in physical memory to load in another page,
the handler must decide which presently loaded page should be discarded.
Although the operating system cannot be sure which pages will not be needed in
the future, it can make a very good guess based on the least recently used
pages. The A (accessed) bit and the bits reserved for operating-system use
determine which pages have not been used recently. The processor's hardware
automatically sets the A bit to 1 whenever the processor accesses the page;
the bit can only be cleared by software. By periodically clearing the A bits,
the operating system can keep track of pages not recently used. The A bit,
combined with managing the operating system reserved bits, allows an accurate
"least recently used" algorithm to be implemented for page management.
More Details.
Once the operating system determines the page that will be discarded from RAM,
it must then decide if the page needs to be written back to disk. The D
(dirty) bit indicates if the page has been written to. If the D bit is set,
then the operating system knows that it must be written back to disk. If the D
bit is not set, then the copy of the page that is currently on the disk is the
most recent version. The U/S (user/supervisor) and R/W (read/write) bits are
described in the page-based protection section later on in this article.
The method of swapping pages in and out of memory when needed is called demand
paging. Unix System V for the 386 microprocessor has always offered this
feature, and in September 1988, PharLap Software announced that 386/VMM, which
runs on top of 386/DOS-Extender, will also support demand paging. With
PharLap's development tools and a compiler, such as 80386 High C Compiler from
Metaware, users can develop large applications that can take advantage of
demand paging.
In addition to virtual memory management, paging has another useful feature:
It can be used to do a simple remapping of memory, a feature used in some DOS
control programs. Programs such as Compaq's CEMM, Quarterdeck's QEMM-386, and
Qualitas's 386-to-the-MAX use remapping ability to implement various features.
Application programs addresses that use the LIM (Lotus, Intel, Microsoft)
specification for accessing expanded memory, for instance, are remapped in
software to use fast extended memory, thus eliminating the need for special
memory board's external mapping hardware. This type of program also allows
extended memory to be mapped into the DOS-accessible 512K - 640K range on 386
microprocessor-based PCs that have 512K of memory on the motherboard. In
addition, it allows relocation of terminate-and-stay-resident utilities
outside of DOS's 640K.


Protection


The 80386 provides many protection mechanisms that operating systems can
selectively employ to fit their needs. Segmentation provides the basis for
much of the task-based protection and multilevel protection schemes. Level 3
of segmentation-based protection corresponds to user level for paging, and
levels 0, 1, and 2 correspond to supervisor level.
Paging has a separate protection mechanism that is sufficient for most
operating systems. Paging protects supervisor memory and allows for write
protecting of user pages. The U/S and R/W bits are found in each page
directory entry and page table entry. Their presence in both levels allows
more selective control over the access to page groups and individual pages.
The operating system can allow user programs to have read only, read and
write, or no access to a given page or page group. If a memory access violates
the page protection attributes, such as user level code writing a read only
page, an exception 14 will be generated.
Exception 14 is used for reporting page access violations and page faults. To
distinguish the cause of an exception 14, the operating system examines a
16-bit error code that is pushed as part of the page fault handler. From the
error code and the faulting linear address, stored in CR2, the operating
system can correctly handle the fault and resume execution.



Virtual 8086 Environment


For running existing code, one of the biggest advantages of the 386
microprocessor is in the support for multitasking DOS applications.
Microsoft's current 80286-based OS/2 does not allow multitasking of DOS
applications, but the 386 can multitask DOS applications with virtual 86 mode.
IGC's VM/ 386, Microsoft's Windows/386, Quarterdeck's DESQview, and several
software vendors offer multitasking of DOS applications as a major feature of
their software. Other products, like MERGE and VPIX, use virtual 86 mode for
running DOS applications under Unix.
The 386 can execute 8086 applications in both real mode and virtual 86 mode.
Virtual 86 mode allows the execution of 8086 applications while still allowing
use of paging. The main difference between real and protected mode, however,
is how the segment selectors are interpreted. When the processor is executing
in virtual mode, the segment registers are used as in real mode. The contents
of the segment register are shifted left 4 bits and added to the offset to
form the segment linear address. When running in protected mode, the processor
determines which applications are protected mode and which are 8086
applications. The 8086 applications require their segments to be interpreted
as in real mode.
Paging is crucial to multitasking 8086 applications. With paging, the
applications can be executed anywhere in memory, not just the lower 1 Mbyte.
Without this ability, only 640K of applications would be multitaskable. The
paging hardware allows the 20-bit linear address produced by a virtual mode
program to be divided into up to 256 pages. Each one of the pages can be
relocated anywhere in physical memory. The operating system is able to treat
memory for virtual 86 applications as it does memory for 32-bit applications.
Because CR3, the page directory base register, is different for each task,
each task can use a different mapping scheme to map pages to different
physical locations. Of course entries can appear in both tables to allow
sharing of operating system code between applications.


Summary


Paging is the 80386's answer to the challenges of memory management for
today's multitasking operating systems. Whether a task's logical address space
consists of one segment or many, an operating system can subdivide the linear
address space into pages. To an operating system, pages are more convenient
units than are segments for allocation because pages are all the same size,
which prevents common problems that occur while using segments for memory
allocation. Additionally, page-based swapping is better tuned to disk drives
than segment-based swapping.
Paging is the only mechanism for virtual memory management of 32-bit
applications (in which segments can be up to 4 gigabytes in length) and for
virtual 8086 applications, which all exist in the lowest Mbyte of the linear
address space. The fast on-chip TLB and hardware TLB miss processing allow the
386 microprocessor to perform these advanced memory management techniques
without reducing the application's processing power.


Enabling and Disabling Paging on the 80386


When the 80386 microprocessor is brought out of reset, it first executes in
real mode. To use paging, the processor must be executing in protected mode.
The steps for entering and exiting protected mode are described in the DDJ
article "80386 Protected Mode Initialization" by Neal Margulis (October 1988).
To enable paging, follow these steps:
1. Set up the page directory and the page tables in memory with the desired
values.
2. Load CR3, the page directory base, with the base address of the page
directory. Loading CR3 also invalidates any information stored in the TLB.
3. Execute a MOV CRO, EAX instruction where bit 31 is set to 1, and the other
bits are unchanged. This can be accomplished with the sequence MOV EAX, CRO;
OR EAX, 80000000H. It is possible to enable paging at the same time protected
mode is entered. The instruction sequence in which the transition to paging
will occur must have its linear address mapped to its physical address.
4. The instruction prefetch queue should be flushed by performing a JMP $2
instruction.
Once paging is turned on, all linear addresses are paged to the correct
physical address. The address translation information is then automatically
cached into the TLB each time the hardware performs a page translation from
the tables in memory. Should you change any of the page table information in
memory, or decide to use a different set of page tables, you must perform a
MOV CR3, xxxxxxxx to invalidate the current TLB entries so that the new paging
information will be used.
When disabling paging you must do the following:
1. Locate the instruction that performs the translation on a page whose linear
address is the same as the physical address. This prevents an unpredictable
instruction prefetch from occurring between changing the paging status and the
next instruction.
2. A MOV CRO, EAX where bit 31 is forced to 0 is executed. This can be
accomplished with the sequence MOV EAX, CRO: AND EAX, 7FFFFFFFH.
3. MOV CR3, EAX to invalidate the TLB entries.

After paging is disabled, the linear address and physical address are the
same. --N. M.





























April, 1989
DEMAND PAGED VIRTUAL MEMORY


Putting a mainframe environment on a micro




Kent Dahlgren


Kent Dahlgren is the strategic planner for graphics products at Paradise
Systems, a manufacturer of graphics devices. He can be reached at 800 E.
Middlefield Rd., Mountain View, CA 94043.


Once an exotic large-system technology, Demand Paged Virtual Memory has come
to the world of small computers. The new generation of 32-bit microprocessors,
with blazing instruction throughput and advanced on-chip memory management,
brings mainframe performance to the desktop. With it comes the ability to run
several programs at the same time, each requiring more memory than the
computer physically possesses. How? That's what this article is about.
So what is Demand Paged Virtual Memory, or DPVM? In a nutshell, it's the
partitioning of a program's linear-address space into smaller units called
"pages". Even very large programs confine their activities at any given
instant to small regions of memory: a loop in executable code, a table in data
space, some variables here and there. Thus, there is never a need for all the
code and data to be in memory at once.
This observation, often referred to as locality of reference, is the basis for
DPVM. As programs run, the pages they need are moved into memory, replacing
pages no longer required. If a page containing changes is to be replaced, it's
saved on disk to prevent data loss and for access later. Other jobs can run
while this process, called swapping, is going on. In this way, DPVM supports
concurrent execution of several programs, any one (or all) of which "thinks"
its linear memory space is unlimited.
Naturally all this doesn't occur by magic. It takes a combination of hardware
and software called a Virtual Memory Manager, or VMM. The CPU must provide
memory management and the high-instruction rate required to run the
sophisticated operating system that makes it happen. The system must also have
a high-performance disk to handle swapping efficiently. And of course there
have to be enough cycles left after the overhead to do productive work.
The key element in DPVM, then, is the CPU. Heretofore, microprocessors simply
didn't have enough horsepower or features. Now they do.
So let's look at the workings and issues of DPVM, and how three
current-generation microprocessors handle them.


How Does It Work?


Paged memory management uses a dynamic, relocatable partitioning scheme to
manage main memory. The partitions, referred to as pages, are of uniform size,
typically 4K. Unlike the segmented memory familiar to users of Intel
processors, the existence and manipulation of pages are transparent to
processes running on the system. This is a result of taking the addresses
generated by the CPU (referred to as logical or virtual addresses), and
translating them into a real memory address. Consequently a process always
sees an uninterrupted linear address space, even though the pages allocated to
it are probably not contiguous in main memory.
The logical to physical translation is accomplished by breaking addresses into
two components. The high-order bits represent the page number, with the
low-order bits giving the offset within the page. Logical addresses are mapped
onto the physical addresses of page frames located in main memory using a data
structure known as the page map table, which contains the current logical to
physical mapping. The page offset bits are unmapped and therefore simply
passed through as the physical page offset. The page map table also stores bit
fields describing protection and cachability. Having such information for each
page is another benefit of paged memory management. Figure 1 illustrates the
mapping procedure.
The power of this mapping methodology makes the most compelling benefit of
paged memory management possible. Because a complete memory image of the
process is kept in secondary storage and only the portion that is currently
being executed in main memory, the process can run in an address space much
smaller than its total image. As the program refers to new sections (for
example pages) of the image, they're brought into main storage. Hence the term
"demand paged virtual memory". Newly arriving pages replace pages that are not
currently being referenced. To support this scheme an additional data
structure is required in order to find logical pages in secondary storage.
This structure, the file map table, parallels the page map table.
Often, of course, a process refers to a page that is not physically present in
main memory. This situation is called a page fault, and is treated as an
exception. It requires that the CPU abort the memory cycle (data or
instruction fetch) and save its state. The system must then find the required
page and bring it into memory, after which the CPU can restart the aborted
access.
Support for DPVM address translation and access validation are the core
requirements for the Memory Management Unit (MMU). The scheme depends on its
ability to translate each memory access transparently to the executing
process. This is not a trivial task; mapping a logical address space of 4
gigabytes requires over a million 4K pages. Even when a process requires only
4 Mbytes, the MMU must translate a thousand pages to support it. This leads to
extremely large map tables: so large that many systems swap portions of the
map tables themselves in and out of main memory.
As a result of this swapping requirement, most systems don't use simple linear
translation tables as shown in Figure 2. Instead the tables are arranged in a
tree structure, in which the leaves contain page tables --arrays of logical to
physical translation data --and intermediate nodes are hierarchical tables of
pointers --or page directories --to the next lower level. The number of levels
is a function of system requirements. More levels allow the memory manager to
add new address translation information in smaller increments, but costs more
memory cycles during a page map table search.
The arrangement used in the 80386 represents a compromise between pay-off and
penalty. The page size is fixed at 4K with the remaining 20 bits of the
logical page address split into two 10-bit indexes. This results in a
1,024-entry page directory consisting of 32-bit pointers to page tables, each
of which contains 1,024 32-bit address translation entries. The beauty of this
scheme is that both the page tables and page directory are 4K in size,
simplifying the paging of these entries themselves.


But What About Performance?


The elegance of DPVM extracts a price: Managing the memory can consume a
significant portion of CPU time. This is reflected in the fact that the bulk
of the papers written about this topic deal with various performance
optimization issues. The key areas are address translation and page
replacement.
Two strategies can enhance address translation performance. The first is to
include registers in the MMU for caching recent address translations. These
caches are referred to as translation lookaside buffers (TLBs) or address
translation caches (ATCs). A well-designed TLB can eliminate table searches
for well over 90 percent of memory references, and for this reason all the
available MMUs include them. The architecture of TLBs falls into two broad
categories, fully associative and set associative.
Fully associative TLBs offer the most flexibility. Here, any of the entries
can hold a given address translation, since logical addresses are compared
with all of the logical address tags in the TLB. The drawback is that this
requires address comparators for each entry and uses up area on the chip. In
addition, the hardware required for keeping track of replacement information
is significantly complex.
The goal of a set associative TLB design is to simplify the hardware. This is
done by using a portion of the logical page address to select what is referred
to as a line in the cache. Each line consists of one TLB entry for each set.
The problem with this approach is that it effectively reduces the number of
available TLB entries for mapping specific sets of address ranges.
Another strategy to improve translation performance is providing hardware or
microcode in the CPU to search the page map table for a new translation when
it cannot be found in the cache. This situation is called a TLB miss. This
hardware also takes care of determining which entry to replace when a new
entry is to be loaded. Not only does it improve performance, but it also means
the programmer doesn't have to write code to handle this critical function.
Improvements in page handling performance are similar to those for address
translation. The key issue here involves swapping: deciding which page to
remove from main memory when a new one must be loaded. The goal is to remove
pages that will not be needed again soon. Because it is not possible to
predict future program behavior, past referencing patterns are used to guess
which page is least likely to be needed. The two most popular techniques are
FIFO and LRU. These algorithms have been the focus of much research over the
past 20 years. The most preferable (in terms of performance) is LRU, which is
achieved at the cost of algorithmic complexity (see sidebar). Hardware support
for paging comes in the form of intelligent I/O controllers that relieve the
CPU of some of the work involved in swapping pages in and out of memory. No
doubt this approach, common in mainframes, will eventually migrate to small
systems as multitasking operating systems become more prevalent.
Thrashing is a term describing situations in which the CPU spends more time
managing memory (paging, searching tables, and so on) than executing process
code. In some cases the situation can get so bad that only one percent of
instructions executed are those of processes. The simplest way to reduce the
problem is to add more real memory. Unfortunately, that costs real money. The
alternative is to add sophisticated performance analysis and tuning software
to the VMM. For example, the VMM in a multitasking environment can continually
monitor the page faulting behavior of executing tasks and dynamically adjust
their page allotments to optimize performance. Tasks creating large numbers of
page faults have their allocation of available pages increased.
More Details.


And Then There's Real Time


The key problem with DPVM in real-time systems is the performance degradation
and indeterminacy resulting from page faulting and, to a lesser extent, TLB
misses. There are two approaches to this problem.
One is to lock pages containing memory and translation table entries into the
TLB for time-critical routines. The only performance penalty is the address
translation overhead, which is usually small in single chip CPU/MMU
implementations. It's not difficult to lock pages in memory because page
swapping is handled by system software. All that is required is an additional
bit in the page map tables' address translation entries to indicate whether
the page is swappable. In addition, the page replacement algorithm must
eliminate such pages from the pool of available swapping candidates.
Locking TLB entries is another matter, because TLB reloading is usually
handled by the CPU's microcode. In such cases, the CPU must directly furnish
the ability to lock TLB entries. The Motorola 68851, the companion MMU for the
68020, includes such support.
Perhaps the best approach for enhancing real-time performance is to set aside
regions of the virtual address space that are not mapped by the MMU at all.
This completely eliminates address translation overhead and results in
portions of the memory map where virtual and real addresses are the same. This
is the approach taken by the designers of the 68030's MMU.


Virtual Address Spaces



An important decision in the design of a virtual memory manager is how many
address spaces to support and how to arrange processes within them. This can
go as far as running multiple operating systems concurrently, each in its own
virtual machine. The most visible example of this type of implementation is
Microsoft Windows/386, which uses the paging and protection capabilities of
the 80386 to create multiple 640K DOS environments.
Other applications include high-reliability, fault-tolerant systems where it
is essential that failing tasks not destroy data belonging to others. Putting
each task in its own address space guarantees this.
Multiple address spaces can be created entirely via software in the virtual
memory manager and/or with aid of hardware support in the MMU. The software
approach involves setting up separate mapping tables for each address space
and changing the pointer to the table area when activating a process that
resides in a different address space.
Most MMUs support multiple address spaces in hardware with pseudo address
bits: Status codes that may be combined with the regular address bits during
address translation. Unfortunately, the pseudoaddress bits can confuse the VMM
if it wants to move data between spaces. This can be handled by mapping
addresses in both spaces to a common page in memory, but this circumvents the
protection that was the goal of separate spaces in the first place.
Consequently, most CPUs that use pseudo address bits also provide special
privileged instructions that allow the VMM to read and write physical
addresses.


Sizing Up The Players


There are currently three widely available 32-bit CPUs with on chip MMUs: The
Motorola 68030, the Intel 80386, and the AMD Am29000. The latter is one of the
new RISC processors. The MMU architecture of each reflects its target market
and historical background. Table 1 gives a comparison of their features.
Table 1: MMU architectures compared

 CPU TLB
 Size/Type TLB Reload Page Sizes Translation Real-Time
 Supported Tree Support
 Levels
 Supported
 ----------------------------------------------------------------------

 68030 22 Entry,
 Fully Microcode 256, 512,
 1K, 2K, 4K, 1 to 5 Transparent
 Associative 8K, 16K, 32K Mapping

 80386 32 Entry, 4
 Way Set Microcode 4K 1 None
 Associative

 29000 64 Entry, 2
 Way Set Software 1K, 2K, 4K, any number None
 Associative 8K

The 68030 has the most sophisticated MMU of the three. Based on the 68851 MMU
used with the 68020, it emphasizes flexibility and maximum hardware support.
The reason for this approach can be traced to the 68000's success in the
technical workstation market. Each of Motorola's customers had different
system requirements; Silicon Graphics with a high-performance graphics
environment; MassComp's real-time Unix implementation; Sun and Apollo with
general-purpose workstations.
Flexibility is apparent in the wide range of page sizes as well as the ability
to select from 1- to 5-level translation trees. The three-bit function code
generated by the 68020 and 68030 supports up to eight separate 32-bit logical
address spaces. The TLB is a fully associative 22 entry design with control
fields for protection and memory caching suppression.
Real-time support is provided in the form of transparent translation (that is,
no translation) for up to two regions. The regions can be as small as 16
Mbytes, with each allocated to a specific address space. The main drawback to
this scheme is that it bypasses normal page protection methods, which are
applied to the region as a whole.
Compared to the 68030, the 80386's paged memory management capabilities are
minimal. This is the first member of the 80X86 family with any support. Intel
also retained the segmented memory architecture of the earlier chips in the
design, and made it possible to use both schemes simultaneously. These
backward compatibility requirements, combined with the need to keep the chip
size within producible limits, dictated the simplicity of the 80386's page
management support. Page size is fixed at 4K and only two-level translation
tables are supported. Despite these limitations, the 80386's ability to
support a virtual machine environment for DOS applications makes it a strong
contender for the high-end personal computer market.
The AMD 29000 is the first commercial 32-bit RISC microprocessor with an
on-chip MMU. In keeping with the bare-bones, high-performance goals of RISC,
the 29000's MMU provides the minimum hardware needed to support DPVM. Systems
software for the 29000 must therefore perform some functions that are done by
hardware in other architectures: TLB reloads are an example. While this adds
complexity to the job of writing a VMM, it does yield the ultimate in system
flexibility. That's a major benefit in the embedded real-time controller
market for which the chip is targeted.
The 29000 MMU supports up to 256 32-bit address spaces through an 8-bit
process identifier (PID). Unlike the 68000 family's function codes, which the
processor automatically generates, the PID is loaded into a register in the
MMU by system software. This gives the system designer more control over the
allocation of address spaces. In a multi-OS virtual machine, the PIDs separate
the address spaces of each guest OS. In high-reliability, multitasking
environments, each task can be allocated its own address space using the PIDs.
Figure 3 shows the structure of the 29000's MMU.
The trend is clearly toward CPUs, such as the 68030 and 80386 for both
mid-range and high-end desktop systems. The incorporation of the MMU means
support for DPVM, is a given in systems using these CPUs [HELP]. The major
remaining requirement for DPVM is a reasonably fast fixed disk, and these are
already widely available at low cost. As a result you will begin to see more
and more systems featuring the multitasking and unlimited address spaces that
DPVM makes possible on small computers.


References


Advanced Micro Devices Corporation. Am29000 32-Bit Streamlined Instruction
Processor User's Manual. Sunnyvale, Calif.: Advanced Micro Devices, 1988.
Bate, S.F.; and Kenah, L. J. 1984. VMS Internals and Data Structures.
Burlington, Mass.: Digital Press.
Donovan J.J.; Madnick, S.E. Operating Systems. New York: McGraw-Hill, 1974.
Intel Corporation. 80386 Programmer's Reference Manual. Santa Clara, Calif.
1988.
Milenkovic, M. Operating Systems, Concepts and Design. New York: McGraw-Hill,
1987.
Motorola Corporation. MC68851 Paged Memory Management Unit User's Manual.
Englewood Cliffs, NJ: Prentice-Hall, 1987.
Motorola Corporation. MC68030 Enhanced 32-Bit Microprocessor User's Manual,
Second Edition. Englewood Cliffs, NJ: Prentice-Hall, 1989.


Implementing the LRU Algorithm


Tom Gettys
Tom Gettys is a principle software engineer for GDP Technologies and can be
reached at 700 Snowberry, Lafayette, CO 80026.

The design of a Demand Paged Virtual Memory system revolves around two key
decisions: the buffer write policy and the page replacement algorithm.
The two main write policies are write through and copy-back. Both involve
keeping two copies of each page image, one in main memory, the other in
secondary storage such as a disk. The write-through policy updates the copy in
secondary storage every time its image in main memory is altered. This is the
safest, since both always contain the same data. Its effectiveness is
significantly degraded, however, if the number of read operations does not
greatly exceed the number of writes.
The copy-back strategy updates secondary storage only when a buffer in main
memory is to be overwritten. The algorithm knows whether to do this by
maintaining a TRUE/FALSE value known as a "dirty bit" for each page buffer.
The dirty bit is set to FALSE (0) when the page is first loaded into memory.
The bit is then changed to TRUE whenever a write occurs in the page's address
space. If this bit is still FALSE when it's time to overwrite the page buffer,
the image is unchanged so there's no need to copy it back to secondary
storage.
A page fault occurs when a process refers to a page that is not currently in
memory. If there are no free page buffers in main memory, a page replacement
algorithm must determine which buffer to overwrite. Clearly, it's most
efficient to use a method requiring the fewest accesses to secondary storage.
The optimum strategy is to overlay the page that will not be used again for
the longest time. Because this requires predicting the future, though, it's
hard to implement, so we must find another algorithm that yields something
close to the optimum. The most popular of such methods is called "least
recently used," or LRU.
LRU replaces the page that has not been accessed for the longest period of
time. It performs remarkably well due to the principle of locality: Processes
tend to access storage in nonuniform, highly-localized patterns clustered
together both in time and space. Consequently, a page that has not been
referenced recently is unlikely to be referenced again in the near future.
With the proper choice of a data structure, the LRU algorithm is
straightforward and efficient. It requires that the buffers be ordered
according to their most recent access in time. This leads us directly to the
ubiquitous FIFO (first-in, first-out) queue. When a page is referenced it is
placed at the back of the queue. The page at the head of the queue is thus the
least recently used, and its buffer is the one the algorithm can overlay. The
size of this queue corresponds to the fixed number of existing page buffers,
so -- because the queue doesn't shrink and grow -- it's not necessary to incur
the overhead of dynamic allocation to manage it. Instead we can use an array.
When an existing page is accessed, its dirty bit is set and the node moves to
the back of the queue. Similarly, a page fault grabs the head of the queue,
sets its dirty bit, and moves the node to the back. From the standpoint of
queue management, then, these operations are identical. A doubly linked list
provides the necessary flexibility and, by designating one of the nodes as the
list head, there is no distinction between the two cases.
Listing One shows the structure of a queue node and the declaration of the
queue as an array of these nodes. The elements pred and succ are the backward
and forward pointers, respectively. Note that they are just array indices and
not pointer types. This is due to the non-dynamic nature of the queue. The
succ of the list head is the oldest (LRU) page, and the pred of the list head
is the youngest (MRU) page. The other two elements are the dirty bit (actually
a byte here, because there are no other bit fields with which it can share
memory) and the identifier of the page associated with the node. In this
example the buffer number associated with each queue node is equal to its
array index, so no additional data is required to provide page-to-buffer
mapping.
Listing Two is the queue initialization routine, which executes once to set up
the forward and backward links and clear the dirty bit. It also sets the page
ID to an invalid value, thus pre-loading the queue with phantom pages. As a
new page request is received it will not match any of the current entries, and
because the dirty bit is not set, the memory manager bypasses the copy-back
step. The buffer associated with the head of the queue will hold the new page,
and its node moves to the back of the queue. This small bit of software
sleight of hand avoids special case checking that would otherwise be necessary
with every page request.
Listing Three manipulates the pointers to delete the specified node and relink
it to the back of the queue. The existence of the list head node allows this
to work without case checking. This is not obvious, it is a valuable exercise
to prove this by assuming instead a separate variable which points to the head
of the queue and developing the equivalent function. You will find that it
becomes more complex.
The rest of the code shows what else is needed to put a complete Memory
Management System in place. The details of the MMS front end will vary since
the definition of a page depends on the application, and the exact nature of
the interface between the MMS and the application will, of course, vary also.
--T.G.


_IMPLEMENTING THE LRU ALGORITHM_
by Tom Gettys

[SIDEBAR TO _DEMAND PAGED VIRTUAL MEMORY_ BY KENT DAHLGREN]


[LISTING ONE]

 struct queue_node
 {
 WORD succ; /* index of next younger node */
 WORD pred; /* index of next older node */
 WORD page_id; /* ID of page associated w/this node */
 BYTE dirty_bit; /* TRUE if page has been written to */
 } lru_fifo[BFRS + 1];






[LISTING TWO]

 void init_fifo(void)
 {
 WORD i;

 for (i = 0; i <= BFRS; i++)
 {
 lru_fifo[i].succ = i + 1;
 lru_fifo[i].pred = i - 1;
 lru_fifo[i].page_id = BAD_PAGE;
 lru_fifo[i].dirty_bit = FALSE;
 }
 lru_fifo[0].pred = BFRS;
 lru_fifo[BFRS].succ = 0;
 }






[LISTING THREE]

 void requeue(WORD node)

 {
 lru_fifo[lru_fifo[node].pred].succ = lru_fifo[node].succ;
 lru_fifo[lru_fifo[node].succ].pred = lru_fifo[node].pred;

 lru_fifo[lru_fifo[LIST_HEAD].pred].succ = node;
 lru_fifo[node].pred = lru_fifo[LIST_HEAD].pred;

 lru_fifo[LIST_HEAD].pred = node;
 lru_fifo[node].succ = LIST_HEAD;
 }




















































April, 1989
SWAP


An application-independent method for one MS-DOS application to run another




Nico Mak


Nico Mak is a software developer for Mansfield Software Group in Storrs, Conn.
He can be reached at 70056,241 on CompuServe, or as Nico_Mak on BIX.


Many application programs have "shell-to-DOS" capabilities that let you run
applications or execute DOS commands from within your resident (current)
program. The problem with many of these programs, however, is that once you've
enabled the shell-to-DOS capabilities, you often don't have enough memory left
to run the additional programs you need. In this article I'll present a
program that provides an application-independent method for one MS-DOS
application to run another, even if both programs would normally not fit in
memory at the same time. The program, which I've named SWAP (see Listing One ,
page 72), eliminates the need for an application to have its own memory
management routines. A typical use is to temporarily swap out a text editor so
that you can run memory-hungry compilers, linkers, or even debuggers without
losing your place in an editing session. The box on page 46 discusses some
typical uses of SWAP. The amount of extra memory you gain by using SWAP can be
significant, but it depends on the size of the application that is swapped
out. When run from dBase III Plus, for example, SWAP frees approximately 220K
of memory.
SWAP works by copying conventional memory used by the currently running
application to expanded memory or to a disk file, thereby freeing conventional
memory used by the application. SWAP then runs the desired program before
restoring the original environment.
For convenience sake, I refer to the program that shells to DOS and gets
swapped out as the swappee. Expanded memory and EMS refer to memory that
conforms to Version 3.2 or later of the Lotus/Intel/Microsoft Expanded Memory
Specification. Numbers suffixed by an H are in hexadecimal notation (for
example, 21H).
SWAP is compatible with most MS-DOS programs. I've tested it with many popular
programs, including dBase III Plus, Lotus 1-2-3, and Mansfield Software
Group's KEDIT. Before relying on it, however, you should test SWAP with your
own configuration in case of conflicts with terminate-and-stay-resident (TSR)
software and other programs.
The program is most likely to fail when used with multitasking software such
as DESQview and Windows. This is because these programs, like SWAP, modify DOS
memory control blocks. However, because they have their own facilities for
overcoming memory constraints, you generally would not use SWAP with these
products.
Though SWAP is coded in 8088 assembly language, you don't need to know
assembly language to understand how it works. The description provided here
will make more sense, however, if you have had some experience with the IBM PC
interrupt system and some of the more common DOS (interrupt 21H) functions.


Using the SWAP Program


The SWAP command (see Figure 1 for the command syntax) can be entered at the
DOS prompt, or it can be part of the command string that your application
sends to DOS to run another program. When the -C option is used, SWAP copies
everything written to the standard output device (stdout) to the console
before it is written to stdout. This option is useful when standard output is
redirected. It can save considerable time in certain situations --for example,
when a long-running program goes astray, you normally don't find out about the
problem until the program completes and you view the file containing the
redirected output. The -C option gives you an opportunity to abort the program
immediately by pressing Ctrl-Break because output is displayed on the console
at the same time as it is written to stdout.
Figure 1: SWAP's command syntax

 SWAP [options] command [command-parameters] [ > fileid]
 Brackets indicate optional parameters.
 Valid SWAP options are:

 -C Copy redirected output to the console
 -D Disk file C:\SWAP.DAT is used instead of expanded memory.
 -F Forces SWAP to continue even if an interrupt vector points to the Swappee.
 -Q Quiet operation. Informational messages are not displayed.

 Command is any command that can be issued at the DOS
 prompt. command-parameters are parameters or options for the
 command. fileid is a DOS file or device to which output from the
 command can be directed.

The -C option is particularly useful when compiling programs from inside an
editor. I use an editor macro that redirects compiler output to a disk file,
examines the output for error messages, and automatically locates any source
lines that generated compiler errors (similar macros are available for most
editors). Occasionally, a simple mistake confuses a compiler enough so that
every source line generates an error. If the program is large, it can be a
while before the compiler and editor macro run to completion. Because SWAP
copies each error message to the console as it is generated, I can terminate a
long compile immediately rather than waiting for unnecessary and lengthy
processing by the compiler and editor macro.
Normally, SWAP will not operate on a program that it thinks has taken control
of any of the machine's interrupt vectors. Because removing interrupt-handling
code would probably result in a crashed machine. Examples of programs that
take control of interrupt vectors are SideKick, The Norton Guides, and many
communications products.
To determine whether the swappee has hooked an interrupt vector, SWAP issues
the Get Vector function (interrupt 21H, function 35H) to obtain the addresses
of the interrupt handlers for interrupts 0 through 80H. If one of the
addresses points to memory owned by the swappee, SWAP assumes that the program
deliberately took control of the interrupt vector.
Because stray unused vectors might, by chance, point at the program you want
to swap out, the -F option is provided to override this default behavior. This
option should be used only if you are sure that the Swappee does not hook
interrupt vectors.


MCBs, PSPs, and PIDs


Before going into a step-by-step description of SWAP's operation, I should
discuss three aspects of DOS internals that are not covered by the IBM and
Microsoftdocumentation -- memorycontrol blocks (MCBs), the program segment
prefix (PSP), and process identifiers (PIDs).
SWAP's operation depends on modifying DOS MCBs. Although the following
information is not documented by Microsoft or IBM, it is consistent in DOS
Versions 2.0 through 4.0. Two references for information on MCBs are the
"16-Bit Software Toolbox" columns in the October 1986 and February 1987 issues
of DDJ.
An MCB immediately precedes each block of memory allocated by DOS or by a user
program. MCBs are one paragraph (16 bytes) long, but only the first 5 bytes
are used. (See Figure 2 for the format of an MCB.) The id field contains a hex
5A for the MCB describing the last block of memory in the system and hex 4D
for all other MCBs. The owner field contains the address of the program
segment prefix (PSP) of the program that allocated the memory or for which DOS
allocated the memory. A 0 in this field indicates that the memory is free, and
an 8 in this field marks memory allocated before the first user program is
loaded. The length field contains the number of paragraphs in the block of
memory (note that this field does not include the length of the MCB itself).
DOS allocates two MCBs when it runs a program: one for the program's
environment and one for the program itself. Additional MCBs are built whenever
a program allocates memory with interrupt 21H function 48H.
In the simplest case, when SWAP is started, the system will contain MCBs for
the following blocks of memory: DOS system memory (owner address 8), the copy
of COMMAND.COM loaded during DOS initialization (three MCBs), the swappee
environment, the swappee's PSP and code, the copy of COMMAND.COM loaded by the
swappee (three MCBs), SWAP's environment, SWAP's PSP and code, and unallocated
memory (owner address 0).
Note that this list includes three MCBs for each copy of COMMAND.COM. DOS
builds two MCBs when it loads COMMAND.COM, just as when any other program is
loaded. The third MCB is for memory allocated by COMMAND.COM for an additional
(larger) copy of the environment. There may be additional MCBs for any TSR
programs. MCBs may not be in this order if memory is fragmented.

Most of the PSP is well documented in the DOS Technical Reference, but several
useful fields are described as "reserved" in IBM and Microsoft manuals. See
Figure 3 for the location and lengths of these fields.
In DOS Versions 2.0 through 4.0, the parent pointer contains the segment
address of the program's parent. COMMAND.COM is always its own parent, so this
field is also useful in determining whether a PSP belongs to COMMAND.COM.
The fields related to the file handle array are valid only in DOS Versions 3.0
through 4.0. The file handle array contains 20 1-byte entries, one for each
possible file handle. For example, the entry at offset 0 into the file handle
array corresponds to file handle 0. Entries are indexes into another table DOS
uses internally to keep track of file handles. An entry of hex FF indicates
that the corresponding file handle is not open. The file-handle-array pointer
holds the address of the file handle array. The handle array size field
contains the number of entries in the file handle array.
A program can enlarge the file handle array by changing the pointer and handle
array size fields and by copying the original file handle array. In this case
the default file handle array is no longer used. Under DOS Versions 3.3 and
4.0, applications can use interrupt 21Hfunction 67Hto perform this function.
You can read more about the file-handle-array-related fields in the "16-Bit
Software Toolbox" columns in the May 1986, September 1986, and December 1986
issues of DDJ.
The process ID (PID) for a program running under DOS Versions 3.0 through 4.0
is the address of its PSP. When a program asks DOS to perform functions that
use fields in the PSP, DOS uses the PID to locate the current PSP. The
undocumented interrupt 21H functions 51H and 52H respectively get and set the
current PID. See the "16-Bit Software Toolbox" column in the May 1986 issue of
DDJ for more on these functions.
SWAP uses the Get/Set PID functions in two places: the interrupt 21H handling
routine and the subroutines that are copied on top of the Swappee. These
routines are described in detail in the following sections.


How SWAP Works


SWAP performs miscellaneous initialization, including displaying the copyright
notice and checking the command-line options. It checks the parent pointer
field in its PSP to verify that its parent is COMMAND.COM. It then builds a
table containing the relevant information from all MCBs, identifies the MCB
for its own PSP, and works backward through the table of MCBs to determine how
much memory can be swapped out. This memory consists of its own environment,
memory belonging to the parent COMMAND.COM, and memory allocated in all MCBs
belonging to the swappee (except the MCB for the swappee's environment).
SWAP verifies that no interrupt vectors point to memory allocated to the
Swappee. It copies memory used by itself, COMMAND.COM, and the swappee to
expanded memory or to a disk file. The program copies the necessary
subroutines to the first few kilobytes of memory allocated to the swappee,
thereby overlaying the swappee. It also overlays the swappee's file handle
array with its own file handle array and ensures that the file-handle-array
pointer in the swappee's PSP points to this new file handle array.
SWAP reduces the amount of memory allocated to the Swappee by altering the
length field in the MCB for the swappee's PSP, building an MCB for the
remainder of the storage it swapped out, and marking this new MCB as
unallocated. SWAP then transfers control to the code that overlaid the
swappee. This subroutine uses the Set PID function to set the current PID to
the swappee's PSP. Then it runs COMMAND.COM to execute the user's command.
When the user's command completes, all but the first 16K that was swapped out
is swapped back in (the first 16K can't be swapped in at this point because it
would overlay the code for this step). SWAP then transfers control back to the
area of memory in which it was first run.
SWAP issues the Set PID function to reset the process ID. The first 16K of the
Swappee are swapped in, thus restoring the machine to the previous state. SWAP
then releases its EMS handle, or if the -D option was used, erases the work
file.


Interrupt 21H Handler


When the -C option is used, SWAP installs an interrupt 21H handler to copy
everything written to the standard output device to the console. Standard
output is presumably redirected away from the console, so SWAP also opens
another file handle for the console device. Every time an interrupt 21H
occurs, the interrupt handler checks for functions that write to standard
output. When one of these functions is used, SWAP copies the data headed for
standard output to the console. These functions are:
02H --Display Output (outputs the character in DL)
06H --Direct Console I/O (unless DL = hex FF, same as function 02H)
09H --Print String (outputs a dollar-sign-delimited string)
40H --Write to a File or Handle (if writing to stdout)
Output written to any file handle is copied if the file-handle-array entry for
that handle is the same as the entry for standard output.
More Details.
When the interrupt 21H handler determines it should copy data to the console,
it issues the Get PID function to save the caller's PID, then issues the Set
PID function specifying the interrupt 21H handler's PSP. This is done so that
DOS uses the appropriate file handle array to write a copy of the data to the
handle opened for the console device. SWAP then issues the Set PID function to
restore the caller's PID and jumps to the original interrupt 21H handler. Note
that the original interrupt 21H handler does not know that SWAP has
"front-ended" the interrupt.


Conclusion


You'll like SWAP if you've ever tried to run one application from within
another and got the message "Program too big to fit in memory." SWAP is
particularly useful to programmers because it can be used as the basis for an
integrated environment that makes it easy to compile, link, correct errors,
and even debug without leaving your editor.
If you make enhancements to SWAP, I encourage you to upload the modified code
to the DDJ Forum on CompuServe. I have already posted a version that makes it
easier to use SWAP from Mansfield Software Group's Personal REXX interpreter.
Comments and suggestions are welcome. I can be reached at 70056,241 on
CompuServe, or as Nico_Mak on Bix.


Typical Uses of SWAP


To understand how SWAP can be used, assume for the moment that you have
entered DOS from within your editor in order to assemble a program. If the
command you normally use to assemble your program is:

 MASM filename;

then the command to run your assembler with SWAP becomes:

 SWAP MASM filename;

If you typically redirect the assembler output to a disk file, you can still
see a copy of the assembler output on the screen using:

 SWAP -C MASM filename; > output.fil

Of course, you would do this in your editor's macro language or from a .BAT
file.

To use SWAP to swap out dBase and load your favorite editor, you could enter
the dBase command:

 RUN SWAP KEDIT filename

Alternatively, you could add the following line to your CONFIG.DB file:


 TEDIT = SWAP.COM -F KEDIT.EXE

and subsequently enter the following dBase command to edit a file:

 MODIFY COMMAND filename

--NM


_SWAP_
by Nico Mak


[LISTING ONE]


; SWAP - (c) Copyright 1988 Nico Mak and Mansfield Software Group
; All rights reserved
;
; To rebuild SWAP.COM use the following instructions:
; masm swap;
; link swap;
; exe2bin swap swap.com
;
cr equ 13
lf equ 10

error macro message ;; macro to display an error message
 local around, msg, msglen ;; and to jump to error_exit routine
 jmp around
msg db &message,cr,lf ;; define error message
msglen equ $-msg
around:
 mov dx,offset msg ;; get address of error message
 mov cx,msglen ;; get length
 jmp error_exit ;; jump to error exit routine
 endm

; -------------------------------------------------------------------
; the following is copied over the swappee
; -------------------------------------------------------------------
code segment 'code'
 assume cs:code,ds:code
 org 100h ; org past psp
swap proc near
 jmp begin
 db 20 dup('STACK')
stack equ $

flag db 0 ; option flag
flag_copy equ 80h ; copy stdout to 'con'
flag_force equ 40h ; swap even if vector points to swappee
flag_quiet equ 20h ; don't print hello message
flag_disk equ 10h ; swap to disk

emsg_ems db "SWAP EMS Error "
ems_rc db 'xx function '
ems_func db 'xx',cr,lf

emsg_ems_len equ $-emsg_ems

my_psp dw ? ; segment of SWAP's original psp
swappee_psp dw ? ; segment of swappee's psp

; variables used when swapping to expanded memory
ems_handle dw ? ; emm handle
swap_pages dw ? ; number of pages for ems_handle
ems_frame dw ? ; ems page frame
last_ems_func db ? ; last emm function issued by swap

; variables used when swapping to disk
swap_fid db "c:\swap.dat",0 ; asciiz string to open swap file
swap_handle dw ? ; handle while swap file is open

; fields for int 21 function 4b (exec)
commandcom_addr dd ? ; address of program to exec (command.com)
exec_sp dw ? ; save area for reg clobbered by exec function
command_line db ?,"/c" ; command line for command.com
command_text db 130 dup (0) ; command line continued
blank_fcb db 36 dup (0) ; dummy fcb for exec function
exec_parm_block equ $ ; exec parameter block
exec_env dw ? ; segment addr of environment
cmdline_addr dw offset command_line ; address of command line
cmdline_seg dw ?
 dw offset blank_fcb ; address of fcb
fcb1_seg dw ?
 dw offset blank_fcb ; address of fcb
fcb2_seg dw ?

; fields used by int 21 handler
save_pid dw ? ; pid at time int 21 handler received control
int21_vector dd ? ; original int 21 vector owner
con db "con",0 ; asciiz string to open console
handle dw ? ; handle while "con" is open
char_buf db ? ; buffer for int 21 function 2 and 6 handlers
save_ax dw ? ; register save areas for int 21 handler
save_bx dw ?
save_cx dw ?
save_dx dw ?
save_ds dw ?

; -------------------------------------------------------------------
; run_command - the following code is copied over the swappee
; -------------------------------------------------------------------
run_command:
 call copy_start ; start copying stdout to the console
 call exec_user_cmd ; execute the user's command
 call copy_stop ; stop copying stdout to the console
 call swap_in ; swap in all but first 16k
 retf

; -------------------------------------------------------------------
; subroutines for run_command follow
; -------------------------------------------------------------------

; -----
; copy_start - if -c option specified, open handle for console and hook int 21
; -----

copy_start:
 test flag,flag_copy ; will we copy stdout to display?
 jz copy_start_ret ; no
; ----- open a handle that points to "con"
 mov dx,offset con ; address of asciiz file name
 mov ax,3d01h ; code to open handle for writing
 int 21h ; open the file
 mov handle,ax ; remember handle
 jnc open_worked ; did open succeed?
 and flag,255-flag_copy ; no, then we won't copy stdout ...
 jmp short copy_start_ret ; ... and won't hook int 21
open_worked:
; ----- hook int 21 vector
 mov ax,3521h ; code to get interrupt 21 vector
 int 21h ; ask dos for address in vector
 mov word ptr int21_vector,bx; save offset
 mov word ptr int21_vector[2],es ; save segment
 mov dx,offset int21_handler ; address of our int 21 handler
 mov ax,2521h ; code to set interrupt 21 address
 int 21h ; tell dos to set int 21 vector
; ----- ensure that standard error is redirected and copied
 mov al,cs:[19h] ; get stdout file handle array entry
 mov cs:[1ah],al ; use stdout entry for stderr entry
copy_start_ret:
 ret

; -----
; exec_user_cmd - set up and issue the int 21 function 4b (exec)
; -----
exec_user_cmd:
 mov cs:exec_sp,sp ; save register
 mov ax,cs:[2ch] ; pass address of our environment
 mov exec_env,ax ; to exec function
 mov word ptr cmdline_seg,ds ; address of command line
 mov word ptr fcb1_seg,ds ; fill in segments for fcbs
 mov word ptr fcb2_seg,ds
 push cs
 pop es
 mov bx,offset exec_parm_block ; bx = exec parameter block
 lds dx,commandcom_addr ; es:bx = asciiz string of command.com
 mov ax,4b00h ; code to load and execute a program
 int 21h ; tell dos to execute the user's program
 mov ds,cs:swappee_psp ; restore ds addressability
 cli ; turn off interrupts
 mov ss,swappee_psp ; restore stack
 mov sp,exec_sp
 sti ; allow interrupts
 ret

; -----
; copy_stop - close handle and restore original int 21 vector
; -----
copy_stop:
 test cs:flag,flag_copy ; did we copy stdout to display?
 jz copy_stop_ret ; no
; ----- close handle for console
 mov bx,handle ; close handle for 'con'
 mov ah,3eh ; dos function = close handle
 int 21h ; tell dos to close 'con'

; ----- restore original int 21 vector
 push ds ; ds gets clobbered, so save it
 lds dx,int21_vector ; get address of old int 21 vector
 mov ax,2521h ; code to set interrupt 21 address
 int 21h ; tell dos to change it
 pop ds ; restore ds addressability
copy_stop_ret:
 ret

; -----
; swap_in - swap in all but the first page of swappee
; -----
swap_in:
 mov bx,cs ; bx = swappee's psp
 add bx,3ffh ; first page to swap in over
 mov es,bx
 test flag,flag_disk
 jnz swap_in_disk
; ----- swap in from expanded memory
 mov cx,1 ; start with second logical page
 cld
swap_in_page: ; loop to swap 16K
 mov bx,cx ; logical page
 call map_page
 push ds ; save ds
 mov ds,ems_frame ; ds = where to swap from
 mov si,0
 mov di,0
 push cx
 mov cx,4000h ; copy 16K
 rep movsb
 pop cx
 pop ds ; restore ds
 mov bx,es
 add bx,400h
 mov es,bx ; es = next place to swap to
 inc cx
 cmp cx,swap_pages
 jl swap_in_page
 ret
; ----- swap in from disk
swap_in_disk: ; es = first page to swap over
 call open_swap_file ; open the swap file
 mov cx,0 ; high order part of offset
 mov dx,4000h ; file pointer to start + 16k
 mov bx,swap_handle ; get swap file handle
 mov ax,4201h ; code to lseek from current location
 int 21h ; tell dos to lseek to 2nd page
 jnc lseek_done
 error "LSEEK on swap file failed"
lseek_done:
 mov cx,1 ; start with second logical page
swap_in_disk_page: ; loop to swap 16K
 call read_swap_file ; read 16k from swap file
 mov bx,es
 add bx,400h
 mov es,bx ; es = next place to swap to
 inc cx
 cmp cx,swap_pages

 jl swap_in_disk_page
 call close_swap_file
 ret

; -------------------------------------------------------------------
; int_21_handler and its subroutines follow
; -------------------------------------------------------------------
 assume ds:nothing
int21_handler:
; ----- decide whether we will front-end this int 21 function
 cmp ah,02h
 je func02
 cmp ah,06h
 je func06
 cmp ah,09h
 je func09
 cmp ah,40h
 je func40
; ----- call the original int 21 vector owner
do_real_thing:
 jmp cs:int21_vector

; -----
; handle int 21 function 9 (print dollar-sign delimited string)
; -----
func09:
 call front_start
 push di
 push es
 mov di,dx
 mov es,save_ds ; address of string at es:di
 mov al,'$' ; scan for $
 mov cx,-1 ; max bytes to scan
 cld ; scan in forward direction
 repne scasb ; find the $
 sub di,dx
 mov cx,di ; length to write
 dec cx ; don't write the $
 pop es
 pop di
 mov ds,save_ds ; ds addressability is blown
 call write_to_con ; write buffer to display
 mov ds,cs:swappee_psp ; restore ds addressability
 jmp front_done

; -----
; handle int 21 function 6 (direct console i/o)
; -----
func06:
 cmp dl,0ffh ; get input characters?
 je do_real_thing ; yes, then there is no output to copy

; -----
; handle int 21 function 2 (display character in dl register)
; -----
func02:
 call front_start
 mov char_buf,dl ; put character to write in buffer
 mov dx,offset char_buf ; get address of buffer

 mov cx,1 ; get length
 call write_to_con ; write buffer to display
 jmp front_done

; -----
; handle int 21 function 40 (write to file handle)
; -----
func40:
 call front_start
; ----- verify that file handle array entry for this handle == stdout entry
 push di
 push es
 mov bx,save_bx ; get caller's handle
 mov es,save_pid ; psp for process issuing int 21
 les di,es:34h ; address of caller's file handle array
 mov ah,es:[di+1] ; file handle array entry for stdout
 cmp ah,es:[di+bx] ; does handle entry == stdout entry?
 pop es
 pop di
 jne func40_done ; no, don't copy to console
; ----- call real int 21 handler with handle opened for 'con'
 mov ds,save_ds ; ds addressability blown
 call write_to_con ; write buffer to display
 mov ds,cs:swappee_psp ; restore ds addressability
func40_done:
 jmp front_done

; -----
; front_start - start front-ending int 21
; -----
front_start:
 assume ds:nothing
; ----- establish ds addressability and save registers
 mov save_ds,ds
 mov ds,cs:swappee_psp ; establish ds addressability
 assume ds:code ; tell assembler
 mov save_ax,ax ; save registers
 mov save_bx,bx
 mov save_cx,cx
 mov save_dx,dx
; ----- remember caller's pid
 mov ah,51h ; dos function = get pid
 int 21h ; tell dos to get pid
 mov save_pid,bx ; remember pid
; ----- set pid so our file handle array is used
 mov bx,cs ; pid = my cs register
 mov ah,50h ; dos function = set pid
 int 21h ; tell dos to set pid
 ret

; -----
; write_to_con - call original int 21H handler to write buffer to display
; -----
write_to_con:
 assume ds:nothing
 mov bx,cs:handle ; handle opened for 'con'
 mov ah,40h ; dos function = write to handle
 pushf
 call dword ptr cs:int21_vector ; call dos

 ret

; -----
; front_done - almost done front-ending int 21
; -----
front_done:
 assume ds:code
; ----- restore caller's pid
 mov bx,save_pid ; get pid of process that issued int 21
 mov ah,50h ; dos function = set pid
 int 21h ; set pid
; ----- restore registers & go jump to previous int 21 handler
 mov ax,save_ax
 mov bx,save_bx
 mov cx,save_cx
 mov dx,save_dx
 mov ds,save_ds ; ds addressability blown
 jmp do_real_thing

; -------------------------------------------------------------------
; the following routines are used by both parts of the program
; -------------------------------------------------------------------

; -----
; emm - remember emm function in case of error and issue int 67
; -----
emm:
 mov last_ems_func,ah
 int 67h ; call expanded memory manager
 or ah,ah
 ret

; -----
; ems_error - handle ems errors
; -----
ems_error:
 mov di,offset ems_rc
 call hex_to_ascii ; make ems error code printable
 mov ah,last_ems_func
 mov di,offset ems_func
 call hex_to_ascii ; make last ems function printable
 mov cx,emsg_ems_len
 mov dx,offset emsg_ems
 jmp error_exit ; go display error message and exit

; ------
; hex_to_ascii - convert ah register contents to ascii hexadecimal at ds:di
; ------
hex_to_ascii:
 mov dl,ah
 mov cx,2
hex_char:
 push cx
 mov cl,4
 rol dl,cl
 mov al,dl
 and al,00fh
 daa
 add al,0f0h

 adc al,040h
 mov [di],al
 inc di
 pop cx
 loop hex_char
 ret

; -----
; error_exit - display error message and exit
; ds:dx point to error message, cx has the length
; -----
error_exit:
 push cx
 push dx
 mov dx,offset emsg_start
 mov cx,emsg_start_len
 mov bx,2 ; handle for stderr
 mov ah,40h ; dos function = handle write
 int 21h ; output error message to stderr
 pop dx
 pop cx
 mov bx,2 ; handle for stderr
 mov ah,40h ; dos function = handle write
 int 21h ; output error message to stderr
 jmp return

; -----
; routines to open, read from, and close the swap file
; -----
open_swap_file:
 mov dx,offset swap_fid ; address of fileid to open
 mov ax,3d00h ; open file in read-only mode
 int 21h
 jnc open_exit
 error "Could not open swap file"
open_exit:
 mov swap_handle,ax
 ret

; read_swap_file - read 16K from swap file to address in es:0
; saves cx
read_swap_file:
 push cx
 mov bx,swap_handle ; get swap file handle
 mov cx,4000h ; read 16k
 mov dx,0 ; buffer offset
 push ds
 push es
 pop ds ; buffer segment
 mov ah,3fh ; dos function = handle read
 int 21h
 pop ds
 pop cx
 jnc read_exit
 error "Error reading swap file"
read_exit:
 ret

close_swap_file:

 mov bx,swap_handle ; get swap file handle
 mov ah,3eh ; dos function = close file
 int 21h
 ret

; -----
; return - return to DOS
; -----
return:
 mov ax,4c00h ; dos function to terminate
 int 21h ; back to dos

; -----
; map_page - map EMS logical page in bx into physical page 0
; -----
map_page:
 mov al,0 ; physical page
 mov dx,ems_handle ; ems handle
 mov ah,44h ; map handle page
 call emm
 jz map_page_exit
 jmp ems_error
map_page_exit:
 ret

lowend equ $ ; end of code copied to lower memory

; -------------------------------------------------------------------
; the following is *not* copied on top of the swappee
; -------------------------------------------------------------------

hello db "SWAP Version 1.0 (c) Copyright 1988 Nico Mak"
 db " and Mansfield Software Group", cr, lf
hello_len equ $-hello
emsg_start db "SWAP Error: "
emsg_start_len equ $-emsg_start
run_addr dw offset run_command ; offset of run_command
run_seg dw ? ; segment of run_command
swappee_mcb dw ? ; segment of mcb for swappee psp
swappee_end dw ? ; segment of mcb after swappee
my_mcb_size dw ?
next_mcb dw ? ; address of next mcb
next_code db ? ; M/Z code in next MCB
next_owner dw ? ; etc
next_size dw ? ;
ems_device_name db "EMMXXXX0",0 ; expanded memory manager signature
comspec db 'COMSPEC=' ; environment variable name
comspec_len equ $-comspec

mcb_info struc ; important memory control block info
addr dw ? ; address of mcb
owner dw ? ; psp of owner
len dw ? ; length of mcb
mcb_info ends

max_mcbs equ 100
mcbs mcb_info <>
mcb_length equ $-mcbs
 db (max_mcbs-1)*mcb_length dup (?)


; -------------------------------------------------------------------
; mainline code run from system prompt
; -------------------------------------------------------------------
begin:
 assume ds:code,es:code
 mov sp,offset stack ; set up new stack pointer
 call process_cmdline ; check options, set up 'exec' cmdline
 call say_hello ; print copyright message
 call check_dos_version ; ensure we have dos 3.0 or later
 call find_comspec ; find comspec= in environment
 call shrink_ourself ; free unneeded memory
 call get_mcb_info ; get relevant info about mcbs
 call check_mcbs ; ensure mcbs are in expected order
 call vector_check ; ensure swappee has not hooked vectors
 call figure_pages ; determine how many ems pages we need
 call init_ems ; ems initialization, allocation, etc
 call swap_out ; swap out swappee, command.com, and us
 call muck_with_memory ; copy swap over swappee & set up mcbs
 mov ss,swappee_psp ; switch to stack in low memory
 call run_user_command ; go call run_command rtn in low memory
 mov ss,my_psp ; switch back to original stack
 call swap_first ; swap in first 16K
 call clean_up ; restore original environment
exit:
 jmp return ; leave SWAP

; -------------------------------------------------------------------
; subroutines for code that is not copied to low memory follow
; -------------------------------------------------------------------

; -----
; process_cmdline - process options, set up command line for exec function
; -----
process_cmdline:
 mov bx,80h
option_check:
 inc bx
 cmp byte ptr [bx],cr ; carriage return?
 jne option_check2 ; no
 error "No command to execute"
option_check2:
 cmp byte ptr [bx],' ' ; blank?
 je option_check
 cmp byte ptr [bx],'/' ; option signal?
 je got_option
 cmp byte ptr [bx],'-' ; option signal?
 jne copy_command_line
got_option:
 mov byte ptr [bx],' ' ; blank out character on command line
 inc bx ; point at option
 mov al,byte ptr [bx] ; get option
 mov byte ptr [bx],' ' ; blank out character on command line
 or al,' ' ; convert option to lower case
 cmp al,'c' ; option 'c'?
 jne check_option_q
 or flag,flag_copy
 jmp option_check
check_option_q:

 cmp al,'q' ; option 'q'?
 jne check_option_f
 or flag,flag_quiet
 jmp option_check
check_option_f:
 cmp al,'f' ; option 'f'?
 jne check_option_d
 or flag,flag_force
 jmp option_check
check_option_d:
 cmp al,'d' ; option 'd'?
 jne bad_option
 or flag,flag_disk
 jmp option_check
bad_option:
 error "Invalid option"
; ----- copy remainder of our command line to command line for command.com
copy_command_line:
 mov cl,ds:[80h] ; length of my command line
 inc cl ; add one for cr
 mov si,81h ; address of my command line
 mov di,offset command_text ; address of where to put it
 xor ch,ch ; zero uninitialized part of count
 cld ; scan in forward direction
 rep movsb ; copy command line
; set length of new command line
 mov cl,ds:[80h] ; length of my command line
 add cl,2 ; add 2 for "/c"
 mov command_line,cl ; save new length
 ret

; -----
; say_hello - print hello message
; -----
say_hello:
 test flag,flag_quiet ; was -q option used?
 jnz say_hello_exit ; yes, skip this
 mov dx,offset hello ; get address of message
 mov cx,hello_len ; get length of message
 mov bx,2 ; handle for stderr
 mov ah,40h ; dos function = write to handle
 int 21h ; write copyright message
say_hello_exit:
 ret

; -----
; check_dos_version - be sure this is dos 3.0 or higher
; -----
check_dos_version:
 mov ah,30h ; dos function = get version
 int 21h ; get dos version
 cmp al,3 ; ok?
 jae dos_version_ret
 error "DOS version must be 3.0 or higher"
dos_version_ret:
 ret

; -----
; find_comspec - find fileid for exec function

; -----
find_comspec:
 mov es,es:2ch ; es = environment segment
 xor di,di ; point to start of env in es:di
 cld ; scan in forward direction
; ----- loop thru environment strings one by one, beginning here
find_string:
 test byte ptr es:[di],-1 ; end of environment?
 jnz check_string ; nope, continue
 error "Could not find COMSPEC= in environment" ; very unlikely
; ----- compare current env string to 'COMSPEC='
check_string:
 mov si,offset comspec ; point to 'COMSPEC=' string
 mov bx,di ; save ptr to start of env string
 mov cx,comspec_len ; length of 'COMSPEC='
 repe cmpsb ; compare
 je found_comspec ; found it
 mov di,bx ; restore ptr to start of env string
 xor al,al ; scan for end of string
 mov cx,-1
 repne scasb
 jmp find_string ; go back for next string
; ----- found COMSPEC=
found_comspec:
 mov word ptr commandcom_addr[0],di ; remember address of ...
 mov word ptr commandcom_addr[2],es ; ... asciiz "command.com"
 ret

; -----
; shrink_ourself - release unneeded memory
; -----
shrink_ourself:
 push cs
 pop es ; address of start of SWAP memory
 mov bx,offset endcode+15 ; address of end of SWAP code
 mov cl,4
 shr bx,cl ; convert to paragraphs
 mov ah,4ah ; dos function = SETBLOCK
 int 21h ; shrink ourselves
 ret

; -----
; get_mcb_info - get relevant info from mcb chain
; -----
get_mcb_info:
 mov my_psp,cs ; remember address of our PSP
 mov ah,52h ; undocumented function
 int 21h ; get base of memory chain
 mov es,es:[bx]-2 ; this is it
 mov bx,offset mcbs
 mov dx,0 ; count of MCBs
mem_loop:
 mov [bx].addr,es
 mov cx,word ptr es:1 ; owner of mcb
 mov [bx].owner,cx
 mov cx,word ptr es:3 ; length of mcb
 mov [bx].len,cx
 inc dx ; increment count of MCBs
 cmp dx,max_mcbs

 jle mem_loop1
 error "Over 100 Memory Control Blocks in system"
mem_loop1:
 cmp byte ptr es:0,'Z' ; last memory block?
 jne mem_next
 error "Could not find SWAP's PSP"
mem_next:
 mov cx,es ; copy seg addr of mcb
 inc cx ; next paragraph
 cmp cx,my_psp ; is this our psp?
 je found_our_psp ; yes
 add cx,[bx].len ; add length of this mcb
 mov es,cx ; this is next memory block
 add bx,mcb_length ; where next mcb goes
 jmp mem_loop ; proceed
found_our_psp: ; have found our psp
 mov dx,[bx].len
 mov my_mcb_size,dx ; remember length of our mcb
 add cx,[bx].len ; add length of memory
 mov next_mcb,cx ; this is next memory block
; ----- remember information about the next mcb
 mov es,cx
 mov dl,es:0
 mov next_code,dl
 mov dx,es:1
 mov next_owner,dx
 mov dx,es:3
 mov next_size,dx
 ret

; -----
; check_mcbs - ensure mcbs are in expected order
; verify that our parent is command.com, find swappee psp, etc.
; -----
check_mcbs:
 mov cx,cs:16h ; our parent's address
 mov es,cx
 mov ax,es:16h ; and our grandparent's address
 cmp ax,cx ; better be equal
 jne unknown_parent
 mov ax,cs:10h ; our ctrl-break handler
 cmp ax,cx ; better equal our parent's address
 je skip_our_env
unknown_parent:
 error "SWAP not directly run from COMMAND.COM"
; ----- back up to find swappee's mcb. bx still points at entry for our mcb
skip_our_env:
 mov cx,cs
 call prev_mcb
 cmp [bx].owner,cx ; is this mcb for our environment?
 jne skip_command
 call prev_mcb
; ----- back up over all mcb's owned by command.com (es == command.com psp)
skip_command:
 mov cx,es ; address of command.com psp
 cmp [bx].owner,cx ; is this mcb owned by command.com?
 je command_loop ; yes
 error "COMMAND.COM must immediately precede SWAP in memory"
command_loop:

 mov dx,[bx].addr ; remember address of mcb in case
 mov swappee_end,dx ; it is the one above swappee
 call prev_mcb ; back up one mcb
 cmp [bx].owner,cx ; is this mcb owned by command.com?
 je command_loop ; yes, skip it
; ----- assume we have one of swappee's mcbs
; back up over all it's mcb's till we reach psp
 mov cx,[bx].owner ; cx = swappee's psp
find_swappee_psp:
 mov dx,[bx].addr ; address of this mcb
 inc dx ; address of memory
 cmp dx,cx ; is this swappee's psp?
 je found_swappee_psp ; yes
 call prev_mcb ; check previous psp
 cmp [bx].owner,cx ; still owned by swappee?
 je find_swappee_psp ; yes continue
 error "Unexpected MCB while looking for PSP of swappee"
; ----- we've found swappee's psp - bx points at mcb entry for swappee
found_swappee_psp:
 mov es,[bx].owner ; es = swappee's psp
 mov swappee_psp,es ; remember swappee's psp
 cmp word ptr es:2ch,0 ; swappee must have an environment
 jne check_mcbs_ret
 error "Swappee does not have an environment"
check_mcbs_ret:
 ret

; -----
; unless the -f option was specified, check whether vectors point at swappee
; note: only interrupts 1-79h (inclusive) are checked
; -----
vector_check:
 test flag,flag_force
 jnz vector_check_ret
 mov cx,0 ; start at the beginning
next_vector:
 inc cx ; next vector
 cmp cx,80h ; all done?
 jae vector_check_ret ; yes, no vectors hooked
 mov ah,35h ; get vector function
 mov al,cl ; vector number
 int 21h ; call dos to get vector address
 mov dx,es ; get segment addr
 push cx
 mov cl,4 ; shift count
 add bx,15 ; round up
 shr bx,cl ; divide offset by 16
 pop cx
 add dx,bx ; compute segment
 cmp swappee_psp,dx ; compare to start of swappee
 jae next_vector ; no problem, keep looking
 cmp dx,swappee_end ; compare to end of swappee
 jae next_vector ; no problem either
 error "Swappee has hooked an interrupt vector"
vector_check_ret:
 ret

; -----
; figure_pages - figure how many 16K pages of EMS we need

; -----
figure_pages:
 mov cx,swappee_psp
 dec cx ; cx = swappee's mcb
 mov swappee_mcb,cx ; remember address of mcb
 mov dx,next_mcb ; dx = mcb after swap.com
 sub dx,cx ; dx = difference in paragraphs
 mov cx,10
 shr dx,cl ; convert paragraphs to 16k pages
 or dx,dx
 jnz figure2
 error "Less than 16K to swap"
figure2:
 inc dx
 mov swap_pages,dx
 ret

; -----
; init_ems - ensure ems is up to par, allocate pages, and save page map
; -----
init_ems:
 test flag,flag_disk
 jz find_emm
 jmp init_ems_exit
; ----- determine whether ems is installed
find_emm:
 mov ax,3567h ; code to get int 67 handler address
 int 21h ; get interrupt vector
 mov di,0ah ; offset to name string
 mov si,offset ems_device_name ; correct ems name
 mov cx,8 ; length of name
 cld ; scan in forward direction
 repe cmpsb ; do the compare
 jz test_status ; ems not loaded
 error "Could not find Expanded Memory Manager"
; ----- test ems status
test_status:
 mov ah,40h ; code to test status
 call emm
 jz check_ems_version
 jmp ems_error
; ----- ensure that we have ems version 3.2 or later
check_ems_version:
 mov ah,46h ; get version
 call emm
 jz got_ems_version
 jmp ems_error
got_ems_version:
 cmp al,32h
 jnb get_page_frame
 error "Expanded Memory Manager version must be 3.2 or higher"
; ----- get page frame address
get_page_frame:
 mov ah,41h ; code to get page frame addr
 call emm
 mov ems_frame,bx ; where ems memory starts
 jz alloc_pages
 jmp ems_error
; ----- allocate ems pages

alloc_pages:
 mov ah,43h
 mov bx,swap_pages
 call emm
 mov ems_handle,dx
 jz save_page_map
 error "Not enough free expanded memory"
; ----- save ems page map
save_page_map:
 mov ah,47h ; save page map
 mov dx,ems_handle
 call emm
 jz init_ems_exit
 jmp ems_error
init_ems_exit:
 ret

; -----
; swap_out - swap out swappee, command.com, and ourself
; -----
swap_out:
 mov es,swappee_mcb
 test flag,flag_disk ; swap to disk?
 jnz swap_out_disk ; yes
; ----- swap out to expanded memory
 mov cx,0
 cld
swap_out_page: ; loop to swap 16K
 mov bx,cx ; logical page = loop count
 call map_page
 mov bx,ems_frame
 assume ds:nothing
 push es
 pop ds ; ds = where to swap from
 mov es,bx ; es = ems_frame
 mov si,0
 mov di,0
 push cx
 mov cx,4000h ; copy 16K
 rep movsb
 pop cx
 mov bx,ds ; where to swap from
 add bx,400h ; add 16K
 mov es,bx ; es = next place to swap from
 push cs
 pop ds
 assume ds:code
 inc cx
 cmp cx,swap_pages ; done swapping?
 jl swap_out_page ; no, swap the next page
 ret
; ----- swap out to disk
swap_out_disk: ; es = swappee's mcb
 mov cx,0 ; attribute
 mov dx,offset swap_fid
 mov ah,3ch ; dos function = create a file
 int 21h
 jnc create_done
 error "Could not create swap file"

create_done:
 mov swap_handle,ax
 mov cx,0 ; number of pages swapped
swap_out_disk_page: ; loop to swap 16K
 push cx ; remember number pages swapped
 mov bx,swap_handle ; handle to write to
 mov cx,04000h ; write 16k
 xor dx,dx ; offset to write from
 push ds
 push es
 pop ds ; segment to write from
 mov ah,40h ; dos function = write to handle
 int 21h
 pop ds
 jnc write_worked1
 error "Error writing to swap file"
write_worked1:
 mov bx,es ; where to swap from
 add bx,400h ; add 16K
 mov es,bx ; es = next place to swap from
 pop cx ; remember number of pages swapped
 inc cx ; now we've swapped one more page
 cmp cx,swap_pages ; done swapping?
 jl swap_out_disk_page ; no, swap the next page
 call close_swap_file
 ret

; -----
; muck_with_memory - copy part of SWAP over swappee's psp, set up mcbs, etc
; -----
muck_with_memory:
 mov es,swappee_psp
; ----- copy code over swappee's psp
 cld ; copy in forward direction
 mov cx,offset lowend ; length of code to copy
 mov si,100h ; start copying after psp
 mov di,100h ; where to copy
 rep movsb ; copy code over swappee's psp
; ----- copy our file handle array down to swappee's psp
 mov cx,20 ; length of file handle table
 mov si,18h ; address of our file handle table
 mov di,18h ; where to put file handle table
 rep movsb ; copy file handle table to swappee psp
; ----- set the file handle array size and offset in swappee's psp
 mov word ptr es:32h,20 ; length of file handle table
 mov word ptr es:34h,18h ; offset of file handle table
 mov word ptr es:36h,es ; segment of file handle table
; ----- now fix up the swappee's mcb (still has an M)
 mov es,swappee_mcb ; address of swappee's mcb
 mov dx,offset lowend+15 ; offset to end of SWAP code
 mov cx,4
 shr dx,cl ; convert to paragraphs
 mov word ptr es:3,dx ; put result in swappee's mcb
; ----- find address of mcb for memory that was freed up
 mov bx,swappee_psp ; address of swappee's psp
 add bx,dx ; add paragraphs in swappee's mcb
 mov es,bx ; this is where mcb for free mem goes
; ----- fill in new mcb
 mov dx,next_mcb ; address of mcb after original swap

 sub dx,bx ; compute paragraphs of free space
 add dx,next_size ; add paragraphs for next mcb
 mov word ptr es:3,dx ; fill in size
 mov dl,next_code ; get id from next mcb
 mov byte ptr es:0,dl ; copy id (M or Z)
 mov word ptr es:1,0 ; mark block as free
 ret

; -----
; run_user_command - call run_command routine in low memory
; -----
run_user_command:
; ----- put swappee segment address into pointer to run_command
 mov bx,swappee_psp
 mov word ptr run_seg,bx ; segment of swappee psp
; ----- set pid to address of swappee psp
 mov ah,50h ; dos function = set pid
 int 21h ; set process id
; ----- call run_command in low memory
 mov ds,bx
 assume ds:nothing
 call dword ptr cs:run_addr ; call run_command
 mov ds,cs:my_psp
 assume ds:code
; ----- restore pid to SWAP's psp
 mov bx,cs ; pid = my cs register
 mov ah,50h ; code to set pid
 int 21h
 ret

; -----
; swap_first - swap in first page that was swapped out
; -----
swap_first:
 mov es,swappee_mcb
 test flag,flag_disk ; swapping in from disk?
 jnz swap_first_disk ; yes
; ----- swap in from expanded memory
 mov bx,0 ; logical page = 0
 call map_page
 push ds ; save ds
 mov ds,ems_frame ; ds = where to swap from
 mov si,0
 mov di,0
 mov cx,4000h ; copy 16K
 cld
 rep movsb
 pop ds ; restore ds
 ret
; ----- swap in from disk
swap_first_disk:
 call open_swap_file
 call read_swap_file
 call close_swap_file
 ret

; -----
; clean_up - restore ems or delete swap file
; -----

clean_up:
 test flag,flag_disk
 jnz clean_up_disk
; ----- restore ems page map
 mov ah,48h ; restore page map
 mov dx,ems_handle
 call emm
 jz deallocate
 jmp ems_error
; ----- deallocate the ems pages
deallocate:
 mov ah,45h ; deallocate pages
 mov dx,ems_handle
 call emm
 jz clean_up_exit
 jmp ems_error
; ----- delete swap disk file
clean_up_disk:
 mov dx,offset swap_fid ; file handle for swap file
 mov ah,41h ; code to delete a file
 int 21h
clean_up_exit:
 ret

; -----
; prev_mcb - back up one entry in table of MCBs
; -----
prev_mcb:
 sub bx,mcb_length
 cmp bx,offset mcbs
 jae prev_mcb_ret
 error "Memory Control Blocks not in expected order"
prev_mcb_ret:
 ret

endcode equ $
 align 16
 db 16 dup(0) ; so that at least on mcb follows swap
swap endp
code ends
 end swap





















April, 1989
A MEMORY ALLOCATION COMPACTION SYSTEM


Here's a compaction technique to solve your fragmented memory problems




Steve Peterson


Steve Peterson is a consultant with FOURTH SHIFT Corp. in Minneapolis. He can
be reached at 3100 Boone Ave., North, Minneapolis, MN 55427.


With the popularity of the C programming language, the use of dynamic memory
allocation in PC software has become common. Graphical interfaces and
object-oriented programming techniques both rely heavily upon the ability to
dynamically create and delete blocks of memory.
A natural by-product of the memory allocation tools provided by the standard C
library is heap fragmentation. In an operating environment with virtual
memory, fragmentation is a concern only to the extent that it degrades
performance. Unfortunately, under MS-DOS, memory can become so fragmented that
the program fails for lack of contiguous free memory. Microcomputer operating
systems developed after MS-DOS provide solutions to this problem. Both
Microsoft Windows and the Macintosh Toolbox provide support for compaction of
memory. In addition, Microsoft OS/2 provides support for virtual memory.
This article describes a compacting memory management scheme that is
implemented in C. The scheme's features include the allocation of memory
blocks that can move in memory to minimize fragmentation, the ability to mark
individual memory blocks as deleteable or nondeleteable and as fixed or
moveable, and the availability of a function that can be associated with a
block that is called whenever the block is moved or deleted. Furthermore, this
scheme can easily be ported to other environments where an ANSI-standard C
compiler is available.


Memory Compaction Strategies


For a memory compaction system to be useful, it must compress memory without
disturbing pointers in the user program. Two approaches to this process are in
general use. Both approaches maintain a master pointer table, which contains
the master pointers to every allocated block. These approaches differ in the
way that the user accesses the master pointer.
Under Microsoft Windows, the user program is given a handle to an allocated
block of memory. To use the handle, a function must be called to lock the
block in place and to retrieve a pointer to the block. The lock should be
released when the program no longer needs to reference that block of memory.
The advantage of this method is that a block will not move in memory while the
block is being used. The disadvantage of this method is that you must call a
function in order to get a pointer to a block.
The method implemented in this article is similar to that used on the Apple
Macintosh. When a block of memory is allocated, the user program is given a
pointer to the master pointer table entry that corresponds to the block. The
user program can either reference the block by dereferencing the pointer
twice, or it can copy the address from the master pointer table to a local
pointer. This implementation requires careful programming. Consider the
following example of a function that reads a record from a database where Rec
is a double-indirect pointer to a block.
 GetDbRec(BlockNum, *Rec)
If GetDbRec causes memory to be compacted, and Rec is moved to a different
location, *Rec is no longer a valid pointer, and anything that GetDbRec writes
to *Rec will overwrite other data. To overcome this problem, do one of the
following:
Ensure that any function passed a dereferenced pointer cannot cause memory to
be compacted.
Write your code so that the double-indirect pointer, rather than a
dereferenced pointer, is passed.
Lock the block in memory if a dereferenced pointer is passed to a routine that
can cause memory to be compacted.
By adhering to these rules, you can avoid memory corruption.


Data Structures


When the system is initialized, it creates one or more segments, a master
segment table (MST), and a master pointer table (MPT). (See Figure 1.) Each
segment has an entry in the MST. An MST entry contains the physical address of
the segment, the number of blocks allocated, the amount of free space in the
segment, and pointers to the first free block and to the most recent free
block accessed. The pointer to the most recent free block is used to allocate
blocks evenly throughout the segment.
Each segment contains one or more blocks, as shown in Figure 2. When the
segment is created, it contains one block that is marked as free and
encompasses the entire segment. As the user program requests memory, this free
block is broken into smaller blocks that are marked as they are allocated.
Each block has a header that is stored immediately prior to the block in
memory. This header contains the length of the block, a pointer to the
previous block in the segment, flags, the segment in which this block is
located, the master pointer associated with this block, and the address of the
block function. The meaning of the bits in the flag byte are shown in Table 1.
Figure 2: Each segment contains one or more blocks.

 Block Structure

 data of previous block

 check byte flags

 segment -----

 previous block pointer

 block length

 master pointer number

 block data

Table 1: Flags contained in the header block


 FLAG FUNCTION
 ---------------------------------------------------------------

 BLK_INUSE Block is allocated
 BLK_DELETED Block contents have been deleted
 BLK_DELETABLE Contents of block can be deleted if necessary
 BLK_LOCKED Block cannot be moved in memory
 BLK_FUNCDELETE Call block function before deleting block
 BLK_FUNCMOVE Call block function before moving block
 BLK_LAST Block is last one in the segment

When a block is allocated, the system places a pointer to the block into a
free entry in the MPT. The user program receives a pointer to this table
entry. The use of double pointer references lets the block move in memory
without the user program's knowledge. Unused MPT entries point to NULL. The
MPT is created when the system is initialized, and is not stored in a segment.
The compaction algorithm coalesces free space in the segment until a request
for space is satisfied. If the amount of freed space is not sufficient, this
algorithm begins removing deletable blocks until the request is satisfied.
When a block is deleted, the header portion of the block remains in memory.
This allows the application to determine that the block has been deleted.


The Memory Compaction Code


Listing One, page 82, contains the memory manager header file. Listing Two,
page 82, contains the code that implements the module, including functions to
initialize the module, allocate and deallocate memory, manually compact
memory, and test the state of the module.
int MemInit(long Size, int Num MPTEntries) --This function initializes the
module. Size specifies the number of bytes to be allocated to segments.
NumMPTEntries is the number of entries to be allocated in the master pointer
table. The function returns TRUE after successful initialization, and FALSE if
not enough memory was available. For example, MemInit(2621441, 512); allocates
256K under the system and allows 512 master pointers.
The function first allocates an array large enough to hold the entire master
pointer list, and it then allocates segments. The code tries to allocate
segments of 64K - 1 bytes long. If the last segment is less than 16K bytes,
its size is increased by 16K, and the size of the next-to-last segment is
decreased correspondingly. This step helps avoid the allocation of segments
that are too small to be useful.
Once the segments are created, a single free block is allocated inside of each
one.
void **P = MemAlloc(unsigned int Size) --This function allocates a block of
memory in the system. Size is the number of bytes to be allocated in the
block. P points to the master pointer for the block, or to NULL if the block
could not be allocated.
The memory allocator first searches the free list, starting at the last free
block referenced, in order to find a free block that is large enough to hold
the block to be allocated. If no free block is sufficiently large, the
function compacts the segment that contains the most remaining free space,
until that segment has a free block that can hold the block to be allocated.
When a block is created, it is marked non-deletable and moveable.
void **P = MemAllocFunc (unsigned int Size, void (*Func)(void *b, unsigned int
f, unsigned int Size, void *c), unsigned int Flags) --This function is similar
to MemAlloc, except that you can specify a function (the "block function") to
be called when the block is moved or deleted. Passing the flag BLK_FUNCMOVE
causes the function to be called when the block is moved. Similarly, passing
the flag BLK_FUNCDELETE causes the function to be called when the block is
deleted. These flags can be ORed together to indicate both options.
The block function takes as parameters the block to be deleted, a flag that
indicates if the block is being moved or deleted, and the length of the block.
If BLK_FUNCMOVE is set, C points to the new location of the block. This
function returns VOID.
void MemFree(void **P) --This function deallocates a block of memory. P points
to the MPT entry for the block, and the deallocation routine combines the
block with adjacent free blocks. The block function is not called when a block
is deallocated.
MemCompact(void) --This function lets you manually compact memory. By calling
CompactSeg( ) for each segment, memory is compacted completely.
CompactSeg(byte Segment, unsigned int Size, int Deletable) --The CompactSeg
function performs the actual compaction of memory. Segment indicates the
segment to be compacted. Size indicates how many bytes of contiguous space are
required. Deleteable is TRUE if deletable blocks are to be deleted.
The function proceeds through memory, examining each block in turn, and looks
for a free block where coalescing can begin. Once the first free block is
found, the function begins the compaction of subsequent blocks. If a block is
deletable, it is marked as deleted, and the function moves the block header to
its new location. If a block is in use and moveable, that block is moved to
its new location.
In MemAlloc( ), the function is called twice. It is called with Deletable
FALSE, then called again with Deletable TRUE if there is still not enough free
space.
MemLocked(void **P) -- The MemLocked function returns TRUE if a block is
locked; otherwise it returns FALSE.
MemLock(void **P) -- locks a block into a location in memory.
MemUnlock(void **P) -- The function MemUnlock releases a lock on a block.
MemDeleted(void **P) -- This function returns TRUE if a block is deletable, or
FALSE if the block is not deletable.
MemDeletable(void **P) -- This function makes a block deletable.
MemUndeletable(void **P) -- This function makes a block undeletable.
MemGetLargest(void) -- This function returns the size of the largest block
that could be allocated.
MemGetCurrentLargest(void) -- The function MemGetCurrentLargest returns the
size of the largest block that could be allocated without compaction.


Using the Memory Compaction System


Listing Three, page 89, shows a simple program designed to demonstrate the
features of the module.
As in any module, many enhancements and improvements can be made, depending
upon your needs. For example, if space is a critical consideration, you can
decrease the block overhead. One way to do so is by replacing the pointer to
the previous block with an offset, saving two bytes in the large memory model.
Another way is to eliminate the block function, thus saving four bytes. Yet
another tactic is to store the segment number in the unused bits of the flags
field, or to compute the segment number from the master segment table entries.
Finally, the master pointer number can be determined by searching the master
pointer table.
If time is more important, you can copy contiguous allocated blocks in one
operation during the compaction process, rather than copying them separately.
Since the current compaction algorithm eliminates deletable blocks in a
first-come, first-served order, another optimization is to add a mechanism for
assigning priority to deleteable blocks in order to allow the application to
control the order of deletion. You can also write the block configuration
functions (MemLocked, MemLock, Mem Unlock, MemDeleted, MemDeletable, and
MemUndeletable) as macros. Finally, under Microsoft C 5.1, in certain
situations you can use the memcpy( ) function, which is more efficient.
Additional functions that you can implement are MemDelete( ), which allows a
block to be deleted under program control, and MemReallocate( ), which
reallocates a block that has been deleted.

_A MEMORY ALLOCATION COMPACTION SYSTEM_
by Steve Peterson


[LISTING ONE]


/**** mem.h data structures for memory manager

 S. Peterson programmer 12/88
*/
typedef unsigned char byte;
typedef unsigned int uint;

/* This structure is the header of a memory block. It lies before
 the actual memory block. */
struct memBlkHdr {
 char checkByte; /* Header validation */
 byte flags; /* Flags (see below) */
 byte segment;/* Segment of this block */
 struct memBlkHdr *prev; /* Pointer-previous block */
 uint size; /* size of block */
 int pointerNum; /* Block master pointer */
 void (*func)(void *, byte, uint, void *);
 } ;

typedef struct memBlkHdr MEMBLK;

/* These are the definitions of each of the flags in the flags byte
 of memBlkHdr. */

#define BLK_INUSE 0x01 /* Block is allocated */
#define BLK_DELETABLE 0x02 /* Block can be deleted */
#define BLK_LOCKED 0x04 /* Block is locked */
#define BLK_FUNCDELETE 0x08 /* Call block function on delete */
#define BLK_FUNCMOVE 0x10 /* Call block function on move */
#define BLK_DELETED 0x20 /* Block has been deleted */
#define BLK_LAST 0x40 /* Last block in segment */

/* freePtr is stored at the beginning of the user part of a
 free block. It contains the links to next and previous free blocks
 in the segment. prev is NULL if this is the first free block, and
 next is NULL if this is the last block */

struct freePtr {
 MEMBLK *prev; /* Pointer to previous free block */
 MEMBLK *next; /* Pointer to next free block */
 } ;

typedef struct freePtr FREEBLK;

/* Macro to return address of data area give block header address */

#define DATALOC(xx) ((byte *) (xx) + sizeof(MEMBLK))

/* Macro to return address of header given data area address */

#define HEADERLOC(xx) ((byte *) (xx) - sizeof(MEMBLK))

/* This is an entry in the master block table. */

struct masterSegmentEntry {
 void *block; /* Address, associated block */
 uint size; /* Size of block */
 uint freeSpace; /* Amount of free space */
 struct memBlkHdr *free; /* First free block */
 struct memBlkHdr *last; /* Last reference free block */
 } ;


#define DEFSEGSIZE 4096 /* Default segment size */
#define CB 'S'

/* MINREMAINDER is the smallest free block that can remain after
 allocation. Making this larger reduces fragmentation but wastes
 more space. */

#define MINREMAINDER 20

/* MINBLOCKDATA is the smallest block data area that can be created. This
 must be at least sizeof(FREEBLK) bytes so there is space for the
 next and prev pointers stored in a free block. */

#define MINBLOCKDATA (sizeof(FREEBLK))

/* Function prototypes */

#ifndef NOPROTO

int MemInit(long int, int);
int MemLocked(void **);
void MemLock(void **);
void MemUnlock(void **);
void MemDeletable(void **);
void MemUndeletable(void **);
int MemDeleted(void **);
void **MemAlloc(uint);
void **MemAllocFunc(uint,void (*)(void *,byte, uint, void *), uint);
void MemFree(void **);
uint MemGetLargest(void);
uint MemGetCurrentLargest(void);
void MemCompact(void);

#endif





[LISTING TWO]

/***** mem.c Compactible memory manager
 S. Peterson programmer 12/88, 1/89
 This module provides a memory management system with the following
 features:
 . Relocatable memory blocks
 . Able to collapse fragemented free space
 . Disposable blocks
 . Lockable blocks
*/

#include <stdio.h>
#include <stddef.h>
#include <malloc.h>
#include <string.h>
#include <process.h>
#include <stdlib.h>
#include <conio.h>

#include "mem.h"

/* Static module data */

struct masterSegmentEntry *mst = NULL; /* Master segment table */

byte numSegs = -1; /* Number of whole & partial segments */
byte lastSeg = -1; /* Index of last segment used */

void **mpt = NULL; /* Master pointer table */
int numMP = -1; /* Number of master pointers allocated */
int mpFree = -1; /* Number of master pointers free */
int lastMP = -1; /* Last master pointer used */

/* Local function declarations */

#ifndef NOPROTO

static int FindFreeBlock(int, uint, void **);
static int AllocBlock(uint, void **, byte);
static int GetFreeMP(void);
static void ReleaseMP(int);
static void CompactSeg(byte, uint, int);

#endif

/** MemInit -- initalize memory manager
 Entry lSize size of requested memory in bytes
 nHandles number of block handles to allocate
 Exit none
 Returns TRUE worked
 FALSE failed
**/
int
MemInit(lSize, nHandles)
long int lSize;
int nHandles;
{
 byte numWhole; /* Number of whole segments to create */
 uint numBytes; /* Size of partial segment */
 uint createSize; /* Size to create */
 MEMBLK *mbh; /* Work block header */
 int i; /* Work */
 FREEBLK *f; /* Free links */

 /* Allocate handle list */

 if ((mpt = (void **) malloc(sizeof(void *)*nHandles)) == NULL)
 return FALSE;

 numMP = nHandles;
 for (i = 0; i < numMP; mpt[i] = NULL, i++) ;
 mpFree = numMP;
 lastMP = 0;

 /* Determine size of segments to create */

 numWhole = (byte) (lSize / (long) DEFSEGSIZE);
 numBytes = (uint) (lSize % (long) DEFSEGSIZE);

 numSegs = numWhole + 1;
 lastSeg = 0;

 if ((mst = (struct masterSegmentEntry *) malloc(sizeof
 (struct masterSegmentEntry)*(numSegs))) == NULL)
 return FALSE;

 /* Allocate segments */

 for(i = 0; i < numSegs; i++) {
 if (i == numSegs - 2) { /* Second to last */
 if (numBytes < DEFSEGSIZE / 4) { /* Last is small */
 numBytes += DEFSEGSIZE / 4;
 createSize = DEFSEGSIZE - (DEFSEGSIZE / 4);
 } else {
 createSize = DEFSEGSIZE;
 }
 } else if (i == numSegs - 1) { /* Last */
 createSize = numBytes;
 } else { /* Whole segment */
 createSize = DEFSEGSIZE;
 }
 if (createSize < sizeof(MEMBLK) + 10)
 return FALSE;
 if ((mst[i].block = (void *) malloc(createSize)) == NULL)
 return FALSE;
 mst[i].size = createSize;
 mst[i].freeSpace = createSize;

 /* Allocate one block in segment */

 mbh = (MEMBLK *) mst[i].block;
 mst[i].free = mbh;
 mst[i].last = mbh;

 mbh->checkByte = CB;
 mbh->prev = NULL;
 mbh->flags = BLK_LAST;
 mbh->size = mst[i].size;
 mbh->segment = (byte) i;

 /* Clear next and prev pointer area */

 f = (FREEBLK *) DATALOC(mbh);
 f->prev = NULL;
 f->next = NULL;
 }
 return TRUE;
}

/** MemLocked -- test whether a block is locked
 Entry block address of a block
 Exit none
 Returns TRUE block is locked
 FALSE not locked
**/
int
MemLocked(block)
void **block;

{
 return !(((MEMBLK *) HEADERLOC(*block))->flags & BLK_LOCKED);
}

/** MemLock -- locks a block into a particular location in memory
 Entry block address of block to lock
 Exit none
 Returns void
**/
void
MemLock(block)
void **block;
{
 ((MEMBLK *) HEADERLOC(*block))->flags = BLK_LOCKED;
}

/** MemUnlock -- unlocks a block
 Entry block address of block to unlock
 Exit none
 Returns void
**/
void
MemUnlock(block)
void **block;
{
 ((MEMBLK *) HEADERLOC(*block))->flags &= ~BLK_LOCKED;
}

/** MemDeletable -- make a block deletable
 Entry block address of block to mark as deletable
 Exit none
 Returns void
**/
void
MemDeletable(block)
void **block;
{
 ((MEMBLK *) HEADERLOC(*block))->flags = BLK_DELETABLE;
}

/** MemUndeletable -- mark a block as undeletable
 Entry block address of block to mark
 Exit none
 Returns void
**/
void
MemUndeletable(block)
void **block;
{
 ((MEMBLK *) HEADERLOC(*block))->flags &= ~BLK_LOCKED;
}

/** MemIsDeleted -- test whether a block has been deleted
 Entry block address of a block
 Exit none
 Returns TRUE block is available
 FALSE has been deleted
**/
int

MemDeleted(block)
void **block;
{
 return (((MEMBLK *) HEADERLOC(*block))->flags & BLK_DELETED) != 0;
}


/** MemAlloc -- allocate memory block
 Entry size size of block to allocate
 Exit none
 Returns pointer to created block, or NULL if not enough room to create
 Notes The function operates by first examining the current segment
 for a first fit to the requested size. It then proceeds to
 the remaining blocks looking for a free block of adequate
 size.
 If no block exists that is large enough, it examines the
 segment list looking for a segment that can be compacted
 to produce enough room. The segment with the fewest
 allocated blocks is favored for compaction.
 Possible enhancement: if no segment has enough room, shuffle
 blocks between segments.
 If there is not enough room in any segment, the function
 fails.
**/
void **
MemAlloc(size)
uint size;
{
 byte curSeg; /* Current segment */
 byte segIndex; /* Index of segment in list */
 long ltotalFree = 0l; /* Total free space */
 byte maxFreeSeg = 255; /* Segment, most free space */
 uint maxFreeSize = 0; /* Segment space, most space */
 MEMBLK *b; /* Block address to allocate */
 int created = FALSE; /* Block created */
 int mp; /* Master pointer */

 if ((mp = GetFreeMP()) < 0)
 return NULL;

 if (size > DEFSEGSIZE - sizeof(MEMBLK))
 return NULL;

 if (size < MINBLOCKDATA) /* Smallest allocatable block */
 size = MINBLOCKDATA;
 size += sizeof(MEMBLK); /* Add header to block */

 /* First pass -- try to allocate from current block structure */

 for (segIndex = 0; (segIndex < numSegs) && (!created); segIndex++) {

 curSeg = (lastSeg + segIndex) % numSegs;

 /* Get stats */

 ltotalFree += (long) mst[curSeg].freeSpace;

 /* Is there enough space in the current segment to allocate? */


 if (mst[curSeg].freeSpace >= size) {

 if (maxFreeSize < mst[curSeg].freeSpace) {
 maxFreeSize = mst[curSeg].freeSpace;
 maxFreeSeg = curSeg;
 }

 /* Search free list for first fit */

 if (FindFreeBlock(curSeg, size, &b)) {

 /* Allocate */

 if (!(created = AllocBlock(size, &b, curSeg)))
 return FALSE;
 }
 }
 }

 /* Which segment could be compacted to create a block large enough?
 We kept track of the segment with the most free space. This should
 compact easily. */

 if (!created && (maxFreeSeg != 255)) {
 /* Compact to produce needed free space */

 curSeg = maxFreeSeg;
 CompactSeg(curSeg, size, FALSE);

 if (!FindFreeBlock(curSeg, size, &b)) {
 CompactSeg(curSeg, size, TRUE);

 if (!FindFreeBlock(curSeg, size, &b))
 return NULL;
 }

 if (!(created = AllocBlock(size, &b, curSeg)))
 return FALSE;
 }

 if (created) {
 mst[curSeg].freeSpace -= b->size;
 mpt[mp] = DATALOC(b);
 b->pointerNum = mp;
 b->func = NULL;
 lastSeg = curSeg;
 return (void *) &(mpt[mp]);
 } else {
 ReleaseMP(mp);
 return NULL;
 }
}

/** MemAllocFunc -- allocate a memory block with an associated function
 Entry size size of block to allocate
 func function to call
 flags following constants:
 BLK_FUNCDELETE call function when block deleted
 BLK_FUNMOVE call function when block moved

 Exit none
 Returns pointer to allocated block, or NULL if no space available
 Notes Calls memAlloc to allocate memory
**/
void **
MemAllocFunc(size, func, flags)
uint size;
void (*func)(void *, byte, uint, void *);
uint flags;
{
 void **aBlock; /* Allocated block */
 MEMBLK *b; /* Dereferenced block header */

 if ((aBlock = MemAlloc(size)) != NULL) {
 b = (MEMBLK *) HEADERLOC(*aBlock);
 b->flags = flags & (BLK_FUNCMOVE BLK_FUNCDELETE);
 b->func = func;
 return aBlock;
 } else {
 return NULL;
 }
}

/** FindFreeBlock -- locate a free block in a segment
 Entry seg segment to search
 size bytes required for block including header
 Exit b address of block header (if found)
 Returns TRUE block found
 FALSE no space
 Notes Finds a free block in the current segment. Starts
 with the last-referenced free block in the segment. This
 spreads the allocations through the segment and combats
 fragmentation.
 The caller can save some time by first checking to see if
 the segment has enough free space to allocate the block.
 This algorithm uses the first-fit method, which is fast but
 promotes fragmentation.
**/
static int
FindFreeBlock(seg, size, b)
int seg;
uint size;
void **b;
{
 MEMBLK *m; /* Current memory block */

 m = mst[seg].last;

 if (m != NULL) {
 while (m->size < size) {
 if ((m = ((FREEBLK *) DATALOC(m))->next) == NULL) {
 /* End of free list */
 m = mst[seg].free;
 }

 if (m == mst[seg].last)
 break;
 }


 if (m->size < size) { /* No block large enough */
 *b = NULL;
 return FALSE;
 } else { /* Block found */
 *b = m;
 return TRUE;
 }
 } else { /* No block */
 *b = NULL;
 return FALSE;
 }
}

/** AllocBlock -- allocate a block
 Entry size size of block (including header) to allocate
 b address of block where it will be allocated
 Exit b Address of allocated block header (this is
 different that the b input parameter)
 Returns TRUE allocated
 FALSE memory structure corrupt
 Notes This function takes a block and splits it in two. If the
 remaining free block is small, the entire block is allocated
 and no free portion is created. This can waste some memory,
 but avoids extreme fragmentation.
**/

static int
AllocBlock(size, b, segment)
uint size;
void **b;
byte segment;
{
 MEMBLK *freeBlock; /* Free block */
 MEMBLK *nextBlock; /* Next block */
 MEMBLK *aBlock; /* Allocated block */
 FREEBLK *curFree; /* Free pointers in block we are allocating */
 FREEBLK *t; /* Pointer to free block we are adjusting */

 freeBlock = (MEMBLK *) *b;

 if ((freeBlock->size - MINREMAINDER < size)) {

 /* Whole block will be allocated */

 aBlock = freeBlock;
 size = aBlock->size;
 aBlock->flags = BLK_INUSE;

 curFree = (FREEBLK *) DATALOC(aBlock);

 if (mst[segment].free == aBlock) {
 /* Implicit: aBlock is first in list */
 mst[segment].free = curFree->next;
 }

 if (curFree->next != NULL) {
 t = (FREEBLK *) DATALOC(curFree->next);
 t->prev = curFree->prev;
 mst[segment].last = curFree->next;

 } else if (curFree->prev != NULL) {
 t = (FREEBLK *) DATALOC(curFree->prev);
 t->next = curFree->next;
 mst[segment].last = curFree->prev;
 } else { /* No free block */
 mst[segment].last = NULL;
 mst[segment].free = NULL;
 }

 } else { /* Block will be split */

 aBlock = (MEMBLK *) ((byte *) freeBlock +
 (freeBlock->size - size));
 aBlock->checkByte = CB;
 aBlock->prev = freeBlock;
 aBlock->flags = BLK_INUSE;
 aBlock->size = size;
 aBlock->segment = segment;

 freeBlock->size -= size;

 /* Is block last in segment? */

 if ((freeBlock->flags & BLK_LAST) != 0) {
 freeBlock->flags &= ~BLK_LAST;
 aBlock->flags = BLK_LAST;
 } else {
 nextBlock = (MEMBLK *) ((byte *) aBlock + (aBlock->size));
 nextBlock->prev = aBlock;
 }
 }

 *b = aBlock;
 return TRUE;
}

/** GetFreeMP -- return next free master pointer
 Entry none
 Exit none
 Returns index of next free master pointer
**/
static int
GetFreeMP()
{
 int i;

 if (mpFree == 0)
 return -1;

 for (i = 0; i < numMP; i++, lastMP++) {
 if (lastMP >= numMP)
 lastMP = 0;
 if (mpt[lastMP] == 0) {
 mpFree--;
 return lastMP;
 }
 }

 return -1;

}

/** ReleaseMP -- release a master pointer to free pool
 Entry mp master pointer to release
 Exit none
 Returns void
**/
static void
ReleaseMP(mp)
int mp;
{
 mpFree++;
 mpt[mp] = NULL;
}

/** MemFree -- free a block of memory
 Entry blockPtr pointer to mpt entry for block
 Exit none
 Returns none
**/
void
MemFree(blockPtr)
void **blockPtr;
{
 MEMBLK *cur; /* Actual block */
 FREEBLK *curFree; /* Free block in current block */
 MEMBLK *workMem; /* Work block in memory */
 FREEBLK *workFree; /* Work block in next block */
 int combined = FALSE; /* Has been combined */

 cur = (MEMBLK *) HEADERLOC(*blockPtr);

 if (cur->checkByte != CB)
 return;

 ReleaseMP(cur->pointerNum); /* blockPtr no longer valid */

 curFree = (FREEBLK *) DATALOC(cur);

 /* Mark block as unallocated */

 cur->flags &= ~BLK_INUSE;
 mst[cur->segment].freeSpace += cur->size;

 /* Combine with subsequent block? */

 if ((cur->flags & BLK_LAST) == 0) {
 workMem = (MEMBLK *) ((byte *) cur + cur->size);

 if ((workMem->flags & BLK_INUSE) == 0) { /* unallocated */
 combined = TRUE;

 if ((workMem->flags & BLK_LAST) != 0)
 cur->flags = BLK_LAST;

 workFree = (FREEBLK *) DATALOC(workMem);
 curFree->next = workFree->next;
 curFree->prev = workFree->prev;
 cur->size += workMem->size;


 /* New top of free list? */

 if (workMem == mst[cur->segment].free)
 mst[cur->segment].free = cur;

 if (workMem == mst[cur->segment].last)
 mst[cur->segment].last = cur;

 /* Adjust pointers in free chain. Point to new block */

 if (curFree->next != NULL) {
 workFree = (FREEBLK *) DATALOC(curFree->next);
 workFree->prev = cur;
 }

 if (curFree->prev != NULL) {
 workFree = (FREEBLK *) DATALOC(curFree->prev);
 workFree->next = cur;
 }

 /* Adjust previous block chain */

 if ((cur->flags & BLK_LAST) == 0) {
 workMem = (MEMBLK *) ((byte *) workMem +
 workMem->size);
 workMem->prev = cur;
 }
 }
 }

 /* Combine with previous block? */

 if (cur->prev != NULL) {
 workMem = cur->prev;

 if ((workMem->flags & BLK_INUSE) == 0) { /* unallocated */
 workMem->size += cur->size;

 if ((cur->flags & BLK_LAST) != 0)
 workMem->flags = BLK_LAST;
 cur->checkByte = '\0';

 if (combined) {

 /* Eliminate free block from links */

 workFree = (FREEBLK *) DATALOC(workMem);
 workFree->next = curFree->next;

 if (workFree->next != NULL) {
 workFree = (FREEBLK *)
 DATALOC(workFree->next);
 workFree->prev = workMem;
 }

 } else {

 /* Free block already linked */


 combined = TRUE;
 }

 /* Connect block chain */

 if ((cur->flags & BLK_LAST) == 0) {
 workMem = (MEMBLK *) ((byte *) cur + cur->size);
 workMem->prev = cur->prev;
 }

 if (cur == mst[cur->segment].free)
 mst[cur->segment].free = workMem;

 if (cur == mst[cur->segment].last)
 mst[cur->segment].last = workMem;

 }
 }

 /* If not combined, link into free chain */

 if (!combined) {

 /* We scan backwards looking for the last block that
 is free */

 workMem = cur->prev;

 while ((workMem != NULL) && ((workMem->flags &
 BLK_INUSE) != 0)) {
 workMem = workMem->prev;
 }

 /* A block is prior to the free block in memory */

 if (workMem != NULL) {
 workFree = (FREEBLK *) DATALOC(workMem);
 curFree->prev = workMem;
 curFree->next = workFree->next;
 workFree->next = cur;

 if (curFree->next != NULL) {
 workFree = (FREEBLK *) DATALOC(curFree->next);
 workFree->prev = cur;
 }
 } else { /* Place at beginning of list */
 curFree->prev = NULL;
 curFree->next = mst[cur->segment].free;

 if (mst[cur->segment].free != NULL) {
 workFree=(FREEBLK *) DATALOC(mst[cur->segment].free);
 workFree->prev = cur;
 }
 mst[cur->segment].free = cur;
 }
 }

 if (mst[cur->segment].last == NULL) {

 mst[cur->segment].last = cur;
 mst[cur->segment].free = cur;
 }
}

/** MemGetLargest -- returns size of largest block available after
 compaction
 Entry none
 Exit none
 Returns Free space in segment with most free space, less the size of
 one block header.
**/
uint
MemGetLargest()
{
 byte seg; /* Current segment */
 uint largest = 0; /* Free space */

 for (seg = 0; seg < numSegs; seg++)
 if (largest < mst[seg].freeSpace)
 largest = mst[seg].freeSpace;

 if (largest < sizeof(MEMBLK))
 return 0;
 else
 return largest - sizeof(MEMBLK);
}

/** MemGetCurrentLargest -- returns size of largest available block
 Entry none
 Exit none
 Returns size of largest available block
**/
uint
MemGetCurrentLargest()
{
 byte seg; /* Current segment */
 MEMBLK *curBlock; /* Current block */
 FREEBLK *curFree; /* Current free block */
 uint largest = 0; /* Largest block found */

 for (seg = 0; seg < numSegs; seg++) {
 curBlock = mst[seg].free;
 while (curBlock != NULL) {
 if (curBlock->size > largest) {
 largest = curBlock->size;
 }
 curFree = (FREEBLK *) DATALOC(curBlock);
 curBlock = curFree->next;
 }
 }

 if (largest < sizeof(MEMBLK))
 return 0;
 else
 return largest - sizeof(MEMBLK);
}

/** MemCompact -- invoke compaction routine

 Entry none
 Exit none
 Returns none
**/
void
MemCompact()
{
 byte seg;

 for (seg = 0; seg < numSegs; seg++) {
 CompactSeg(seg, DEFSEGSIZE, FALSE);
 }
}

/** CompactSeg -- compact a segment
 Entry seg segment to compact
 size desired free block size
 deletable TRUE if deletable segments should be deleted
 FALSE otherwise
 Exit none
 Returns none
**/
static void
CompactSeg(seg, size, deletable)
byte seg;
uint size;
int deletable;
{
 MEMBLK *copyLoc = NULL; /* Next place to copy */
 FREEBLK *copyFree; /* Free pointers */
 MEMBLK *curBlk; /* First block */
 MEMBLK *saveBlk; /* Saved block location */
 MEMBLK *prevBlk = NULL; /* Previous block */
 MEMBLK *nextFree = NULL; /* Saved next free block */
 MEMBLK *prevFree = NULL; /* Saved prev. free block */
 FREEBLK *tmpFree; /* Temporary free pointer */
 uint spaceRecovered = 0; /* Amount space recovered */
 int last = FALSE; /* Last record found */

 curBlk = mst[seg].block;

 while ((spaceRecovered < size) && (!last)) {

 last = (curBlk->flags & BLK_LAST) != 0;

 if (((curBlk->flags & BLK_DELETABLE) != 0) && (deletable) &&
 (curBlk->size > sizeof(MEMBLK) + MINBLOCKDATA)) {
 spaceRecovered += curBlk->size - sizeof(MEMBLK) -
 MINBLOCKDATA;

 if (copyLoc != NULL) {

 if ((curBlk->flags & BLK_FUNCDELETE) != 0)
 (*(curBlk->func))(curBlk, BLK_FUNCDELETE
 BLK_FUNCMOVE, curBlk->size, copyLoc);

 /* Save location of block */

 saveBlk = curBlk;


 /* Move block header to new location */
 memmove(copyLoc, curBlk, sizeof(MEMBLK));

 /* Set up new pointer to previous block */

 copyLoc->prev = prevBlk;
 copyLoc->flags &= ~BLK_LAST;
 copyLoc->flags = BLK_DELETED;
 copyLoc->size = sizeof(MEMBLK) + MINBLOCKDATA;
 mpt[copyLoc->pointerNum] = DATALOC(copyLoc);
 prevBlk = copyLoc;
 curBlk = (MEMBLK *) ((byte *) saveBlk +
 copyLoc->size);
 copyLoc = (MEMBLK *) ((byte *) copyLoc +
 copyLoc->size);

 } else {
 if ((curBlk->flags & BLK_FUNCDELETE) != 0)
 (*(curBlk->func))(curBlk,
 BLK_FUNCDELETE, curBlk->size, NULL);

 /* Initialize free area after block */

 curBlk->flags &= ~BLK_LAST;
 curBlk->flags = BLK_DELETED;
 saveBlk = curBlk;
 prevBlk = curBlk;
 curBlk = (MEMBLK *) ((byte *) curBlk +
 curBlk->size);
 saveBlk->size = sizeof(MEMBLK) + MINBLOCKDATA;
 copyLoc = (MEMBLK *) DATALOC(saveBlk);

 }

 } else if ((curBlk->flags & BLK_INUSE) != 0) {
 if (copyLoc == NULL) {
 prevBlk = curBlk;
 curBlk = (MEMBLK *) ((byte *) curBlk +
 curBlk->size);
 } else {
 if ((curBlk->flags & BLK_LOCKED) == 0) {
 if ((curBlk->flags & BLK_FUNCMOVE) != 0)
 (*(curBlk->func))(curBlk,
 BLK_FUNCMOVE, curBlk->size,
 copyLoc);

 /* Move block to new location */

 saveBlk = curBlk;
 memmove(copyLoc, curBlk, curBlk->size);

 copyLoc->prev = prevBlk;
 copyLoc->flags &= ~BLK_LAST;
 prevBlk = copyLoc;

 mpt[copyLoc->pointerNum] =
 DATALOC(copyLoc);


 /* Move pointer to next block */

 curBlk = (MEMBLK *) ((byte *)saveBlk +
 copyLoc->size);
 copyLoc = (MEMBLK *) ((byte *)copyLoc +
 copyLoc->size);

 } else { /* Close up current free block */

 curBlk = (MEMBLK *) ((byte *) curBlk +
 curBlk->size);

 copyLoc->flags = 0;
 copyLoc->checkByte = CB;
 copyLoc->prev = prevBlk;
 copyLoc->size = spaceRecovered;
 copyLoc->pointerNum = 0;
 copyLoc->segment = seg;
 saveBlk =(MEMBLK *) ((byte *)copyLoc +
 copyLoc->size);
 saveBlk->prev = copyLoc;
 spaceRecovered = 0;

 copyFree =(FREEBLK *)DATALOC(copyLoc);
 copyFree->prev = prevFree;
 if (prevFree != NULL) {
 tmpFree = (FREEBLK *)
 DATALOC(prevFree);
 tmpFree->next = copyLoc;
 }

 copyFree->next = nextFree;
 if (nextFree != NULL) {
 tmpFree = (FREEBLK *)
 DATALOC(nextFree);
 tmpFree->prev = copyLoc;
 }

 prevBlk = copyLoc;
 prevFree = copyLoc;
 copyLoc = NULL;
 }
 }
 } else { /* Free */
 if (copyLoc == NULL)
 copyLoc = curBlk;

 spaceRecovered += curBlk->size;

 nextFree = (MEMBLK *) (((FREEBLK *)
 DATALOC(curBlk))->next);
 curBlk = (MEMBLK *) ((byte *) curBlk + curBlk->size);
 }
 }

 /* At this point copyLoc points to the beginning of the free area,
 spaceRecovered is the size of the new free block, and saveBlk
 points to the next free block in the segment */


 if (copyLoc != NULL) {
 copyLoc->checkByte = CB;
 copyLoc->prev = prevBlk;
 copyLoc->size = spaceRecovered;
 copyLoc->pointerNum = 0;
 copyLoc->segment = seg;

 if (last) {
 copyLoc->flags = BLK_LAST;
 } else {
 copyLoc->flags = 0;
 saveBlk = (MEMBLK *)((byte *) copyLoc + copyLoc->size);
 saveBlk->prev = copyLoc;
 }

 copyFree = (FREEBLK *) DATALOC(copyLoc);
 copyFree->prev = prevFree;
 if (prevFree != NULL) {
 tmpFree = (FREEBLK *) DATALOC(prevFree);
 tmpFree->next = copyLoc;
 }
 copyFree->next = nextFree;
 if (nextFree != NULL) {
 tmpFree = (FREEBLK *) DATALOC(nextFree);
 tmpFree->prev = copyLoc;
 }
 mst[seg].free = copyLoc;
 mst[seg].last = copyLoc;
 }
}





[LISTING THREE]

/***** demo.c Demonstration program for memory manager
 S. Peterson programmer 1/89
 This program demonstrates the features of the memory manager.
**/
#include <stdio.h>
#include <stddef.h>
#include <process.h>
#include <memory.h>
#include "mem.h"

void blockFunc(void *, byte, uint, void *); /* Prototype, block function */
int main(void);

int
main()
{
 void **p; /* Double pointer to block */
 void *q; /* Single pointer to block */

 /* Allocate a 30K buffer and 100 master pointers */

 if (MemInit(30000l, 100) != TRUE)

 exit(1);

 /* Allocate a buffer of 1024 bytes. It is created as moveable and
 undeletable */

 if ((p = MemAlloc(1024)) == NULL)
 printf("Error allocating 1024 bytes.\n");

 MemLock(p); /* Make the block unmoveable */
 q = *p; /* Get pointer to memory block */
 memset(q, 0, 1024); /* Initialize block to 0 */

 MemUnlock(p); /* Make the block moveable */
 /* Note: q no longer valid */

 MemCompact(); /* Cause memory to be compacted */

 MemFree(p); /* Deallocate p */

 /* Allocate a block with an associated function. */

 if ((p = MemAllocFunc(1024, blockFunc, BLK_FUNCDELETE 
 BLK_FUNCMOVE)) == NULL)
 exit(2);

 MemFree(p); /* Deallocate p */

 return 0;
}

void
blockFunc(blockPtr, flags, length, newLoc)
void *blockPtr;
byte flags;
uint length;
void *newLoc;
{
 if ((flags & BLK_FUNCMOVE) != 0) {
 /* Perform some action when block moves */
 }

 if ((flags & BLK_FUNCDELETE) != 0) {
 /* Perform some action when block is deleted */
 }
}

















April, 1989
A CLASS ACT


Object-oriented programming makes a case for software recycling




Michael Floyd


Mike is a technical editor for DDJ and can be reached through CompuServe at
76703,4057 or through MCI: MFLOYD.


An object in the real world is something with mass, an associated weight, and
volume; it has other attributes that further refine its description and, in
some cases, there may be some procedure or action that describes its function.
Our world is filled with objects, so it seems only natural to describe and
solve problems in terms of objects as well.
This idea is the basis for object-oriented programming (OOP). As you'll soon
see, however, a programming language requires more than just objects to be
called object-oriented.
This article begins with a tour of the concepts in object-oriented
programming. I'll cover the basic concepts including objects, classes,
inheritance, and polymorphism. Next, I'll discuss how these concepts are
brought together to do what is called "object-oriented programming." In
addition, I'll examine some of the reasons you might consider OOP. Finally,
I'll cover some of the popular object-oriented languages such as SmallTalk and
Actor, as well as some of the traditional languages, like C (C++), that have
object-oriented extensions.


Objects


As in the real world, an object is something with a set of attributes and
features that describe its nature and functionality. Objects are represented
in most OOP languages as data structures. The degree of success that an object
can be described depends, in part, on how well those data structures can be
implemented. This is a key point when considering the various OOP languages.
At the program level, objects can be thought of as Pascal records or C
structures. Just as a record can contain other records, an object can contain
other objects. Messages are used to communicate with objects. When a message
is sent to an object, the object uses a set of procedures (called methods) to
respond to that message. Messages are analogous to function calls in other
languages. As an example, consider the following statement:
 3 factorial
In this example, 3 is an object to which the message factorial is sent. The
extent to which messages are supported depends on the implementation, as
you'll soon see.


Classes


In concept, objects belong to categories or classes of objects. These classes
can in turn be subclasses of broader classes. Looking at it another way, an
object is an instance of a particular class. Classes, then, provide a way to
categorize a given set of objects.
As an example, consider Figure 1, which describes a class of objects known as
"aircraft." Within the aircraft class are subclasses of objects corresponding
to particular types of aircraft. For instance, "private plane" is a subclass
of the "aircraft" class. Continuing down the graph you'll note that ""Cessna
182" is a subclass of "private plane" and that "Flying Lady" is an object that
describes a particular Cessna 182.
Note that each subclass is connected to its "parent" class by a link. Since a
private plane is an aircraft, the link between these two is called an IS-A
link. It is this network of links (called a semantic network) that allows us
to deduce that the Flying Lady is also an aircraft.
A class definition is used to describe a new class of objects. This is
analogous to type definitions in other languages. Defining a new class of
objects is referred to as data abstraction. Objects are therefore described as
implementations of abstract data types.
As I mentioned earlier, OOP treats data structures as primary. At this level,
we are talking about classes of objects. Indeed, OOP may well have been called
COP (class oriented programming) since it is better to think in terms of
classes of objects rather than individual objects (instances).
Just as messages are specific to objects, methods are specific to classes. A
method describes how a particular class of objects will behave. Methods are
analogous to program code in other languages.
The difference between objects and classes is often confused. Keep in mind
that objects are created at run time and are the instances of a class, while
classes are a static description of an object set that exists in the program.
Another concept that confuses traditional programmers is the notion that, at
least in "pure" OOP, a module and a type are the same. We can therefore state
the following identity:
 module = type
What this implies is that the class description includes both the defining
attributes of an object and the services it provides. Indeed, this identity
implies the dual nature of classes.


Inheritance and Polymorphism


Inheritance is a mechanism that allows an object to inherit properties from a
class of objects. For instance, consider Figure 2, which adds some
descriptions to the various classes from Figure 1. The description for
aircraft, for instance, states that all aircraft have landing gear. From this,
you can deduce that the Flying Lady must have landing gear since it is a
Cessna 182 and a Cessna 182 is an aircraft. In fact, the description of
"Flying Lady" refines the notion by stating that this particular aircraft has
fixed landing gear. As presented, this is known as single inheritance since an
object inherits its attributes from a single parentclass (or ancestor).
There are, of course, extensions to the basic inheritance concept. Multiple
inheritance, for instance, allows you to declare a class as heir to more than
one parent class. Repeated inheritance, on the other hand, allows you to
declare a class as heir to more than once to the same class.
Polymorphism allows program entities to refer to objects in more than one
class, and through dynamic binding the ability for those objects to respond
uniquely to the same message. This is an important feature since it allows us
to group dissimilar classes (analogous to an array of mixed types).


Object-Oriented Programming


So, what is OOP and what makes a language object-oriented? You might say that
object-oriented programming is the combination of data abstraction,
inheritance, and polymorphism. OOP treats data as primary, therefore,
object-oriented programming is a style that modularizes a program on the basis
of its data structures.
An object-oriented language, then, must support abstract data typing for the
creation of new classes, some degree of inheritance so that new objects
created at run time can take on the properties of predefined classes, and
polymorphism so that a single message can elicit different responses from
different object classes.
A true OOP language should also support garbage collection. The development
system should automatically deallocate memory for objects that will no longer
be used in the system. This feature exists in most object oriented languages,
but not in many of the extended languages such as C++.



Why OOP?


A colleague of mine once noted that, "if an object is comparable to a C
structure, why do I need another paradigm?" Supporters of object-oriented
programming point out that typical top-down approaches to software development
lead to narrowly defined procedures that are specific to a given problem. Such
procedures must be changed as the problem at hand changes. OOP on the other
hand, promotes a modular approach to programming. Because objects stand on
their own as individual components, they can be readily extended to handle new
features. In addition, objects can be reused, either by other parts of the
system, or in other applications.
Inheritance in particular promotes reusability because modules (classes) are
extensions of existing modules (parent classes). Polymorphism, on the other
hand, promotes extendability and flexibility of the system.
Besides reusability and extendibility, there are other programming issues that
are solved by object-oriented programming. For instance, earlier I mentioned
that objects, just as a Pascal records or C structures, can contain other
objects. But there are some characteristics of objects that are beyond the
scope of procedural languages. For instance, objects must be able to "share"
objects with other objects (comparable to sharing a field of a record with
another record). This is not possible with Pascal.
Another problem difficult to solve in procedural languages is the ability to
create new objects on the fly (comparable to creating structures dynamically
at run time). Indeed, this is one of the features that allows the creation of
more "intelligent" applications. This allows the system to change its behavior
without programmer intervention, and has important consequences in data base
and knowledge-base applications.


Object-Oriented Languages


As you might guess, there are a number of object-oriented programming
languages with disparate philosophies on how these features should be handled.
Just as a hammer is better than a screwdriver at pounding a nail, each of the
object-oriented languages caters to particular needs. By surveying some of the
popular object-oriented languages, you may be able to come to grips with which
language is most suitable for your purposes:
SmallTalk --Implemented as an interpreted environment, SmallTalk is a typeless
language that favors dynamic binding; because no type checking is performed,
binding of structures takes place at run time. SmallTalk was developed at
Xerox by Alan Kay and his associates in the mid 1970s. SmallTalk has since
then been implemented for a number of platforms and Xerox has formed a
subsidiary called ParcPlace Systems to fully support the SmallTalk
environment.
Everything in SmallTalk is an object including the environment that SmallTalk
runs in. Therefore, SmallTalk does not distinguish between objects and
classes. SmallTalk can be described as a tree of classes where object is the
root class, and the leaves of the tree are considered subclasses. In this
scenario, a subclass is an instance of a higher-level class called a
metaclass.
SmallTalk supports most of the concepts developed in this article including
data abstraction, inheritance, polymorphism, and garbage collection. Actor
--Actor, which is implemented as an interpreter, still uses a more procedural
approach with control and block structures. The Actor language set supports
messages and methods, is strongly typed, and includes a built-in library of
classes similar to SmallTalk.
Actor, like SmallTalk, does not distinguish between objects and classes in the
sense that object is the "ultimate" class that all other class will be built
from. Actor supports all the "pure" object-oriented concepts described in this
article including data abstraction, message sending, single inheritance, and
polymorphism.
Eiffel --Eiffel is a compiler that provides a complete object oriented
programming environment. Eiffel supports most of the concepts described in
this article including multiple inheritance, dynamic binding, and garbage
collection. Eiffel provides a complete library of traditional tools as
classes. These classes include windowing and graphics based on X Windows,
lists, hash tables, binary trees, and more.
An interesting feature of Eiffel is that it generates C as intermediate code.
This allows for portability as well as opening the door to cross-development
in the C environment.


Extended Languages


C++ --C++, developed by Bjarne Stroustrup at AT&T, offers a number of
extensions to the C language. C++ by nature distinguishes objects from
classes. As with C structures, classes are declared and defined separately.
C++ supports the notion of "friend" routines that make it possible to call C++
routines from C. C++ also supports operator overloading, which allows you to
use the same name for different operations.
C++ handles the issue of dynamic binding by forcing the programmer to declare
classes as virtual. Defaulting to static binding means that the compiler can
implement calls efficiently. This, of course, has an effect on the polymorphic
nature of the declared class.
Current implementations of C++ only support single inheritance. A C++ standard
is expected from AT&T sometime this year. Some of the issues still to be
addressed include multiple inheritance.
Of course, all memory management and garbage collection is left to the
programmer. In keeping with traditional C, the C++ programmer merely defines
Create and Destroy procedures.
Objective-C --Objective-C departs from C++ in the sense that the
object-oriented portion of the language is untyped. Objective-C provides a
library of classes that roughly parallel the classes built into SmallTalk.
Providing untyped classes allows Objective-C to emphasize dynamic binding and
polymorphism, although it only supports single inheritance. Again, allocation
and deallocation of memory is in the hands of the programmer.
Lisp --Because of the interest in the artificial intelligence community,
several object-oriented extensions to Lisp exist. Probably the most prominent
in this category is Loops (developed at Xerox). Most Lisp implementations have
well supported environments that provide for multiple and repeated
inheritance, polymorphism and dynamic binding, and garbage collection. As with
Lisp proper, types are not specified in the language.
Simula --Simula, an object-oriented extension of Algol 60, was developed in
1967 by Ole-Johan Dahl and Krysten Nygaard. Simula is a strongly typed
language that supports program control blocks, coroutines, and the notion of a
main program.
The major benefit brought to OOP by Simula is its set of built-in primitives
for simulation modeling. In particular, Simula focuses on discrete event
simulation where the model can be thought of as a state machine and the system
consists of individual events. Each event is in a given state and the system
evolves in response to these changing states.
Most implementations of Simula include a well-supported development
environment and source-level debugging facilities. In addition, Simula
supports the necessary garbage collection. In contrast with languages such as
SmallTalk, however, Simula does not include a standard class library.


The Carpenter's Wrench


Although not a new idea, OOP has gained a great deal of popularity in recent
years. One reason is that as hardware technology improves, systems can better
handle the heavy demand on resources. The down side is that features such as
multiple inheritance and dynamic binding can still slow down an application to
a crawl. Waiting for improved hardware technology, then, is not the answer as
we must look for new ways to optimize search strategies.
Remember too that every tool serves a specific purpose. Just as the carpenter
has no use for a mechanic's wrench, you may find little use for
object-oriented programming. But if you're involved in the development of
large scale systems that must evolve over time, you'll want to take a look at
the various object-oriented languages. In the process, however, keep in mind
OOP's prime directive; reusability.


Reference


"Object-oriented Software Construction," Bertrand Meyer, Prentice Hall, 1988.














April, 1989
SHERLOCK HOMES IN


This month the Doctor examines Edward K. Ream's "Sherlock" debugger, TSR
System's "C"erious Toolkit," Crescent Software's "Basic Quickpak
Professional," and a book called "The Puzzling Adventures of Dr. Ecco."




Alex Lane


Alex Lane is a knowledge engineer for Technology Applications Inc. of
Jacksonville, Fla. He can be reached as a lane on BIX and as ALANE on MCI
Mail.




Product Information


Sherlock; Edward K. Ream, 1617 Monroe St., Madison, WI 53711. IBM PC/XT/AT or
compatible with 2 floppy disks or hard-disk MS-DOS. Microsoft C or Turbo C
compiler. (Note: Microsoft MASM macro assembler required to change interrupt
handler.) Price: $195.
It's funny how C programmers get caught in the rut of debugging programs by
peppering them with printf( ) statements. Sure, some master the intricacies of
source-level debuggers like CodeView, which offers both slick features and a
formidable learning curve; others, influenced by books like Debugging C
(Robert Ward, Que, 1986), try various ad hoc combinations of compile-time
variables and macros. The rest, however, lacking time or budget, uneasily
curse the darkness and continue to rely on printf( ) statements to flush bugs
out into the open.
Rejoice, programmers! That darkness may be short-lived. Sherlock is a
reasonably priced, well thought-out tool for debugging, tracing, and profiling
programs written in C. The package was written by Edward K. Ream, who's
developed a reputation for putting out quality software and whose name is no
stranger to these pages. Sherlock runs on IBM PC/XT/AT machines under MS-DOS
with either Microsoft C or Turbo C. (Because source code is provided for the
macros and support routines, minor modifications should allow Sherlock to be
used with other compilers as well.) The software comes on two 5 1/4-inch
low-density floppies, and is accompanied by a 66-page spiral-bound manual.


Features


I can appreciate the work that Ed Ream did on Sherlock because I did something
similar (but nowhere near as polished) for a C project a while back. By
enabling and disabling various trace-points, you can quickly see what's
happening in the program without either being overrun with spurious debugging
information or having to continually stop, edit, recompile, and relink the
code. This is particularly important when trying to track down pointer bugs,
which tend to behave differently as printf( ) statements are added to and
removed from the code. Ream discusses such problems in the manual, and refers
readers to Ward's Debugging C, mentioned earlier.
The typical statement executed from a Sherlock macro is a printf( ) statement,
although any executable C statement (including blocks of statements) may be
used in a macro. This lets you create really complicated debugging functions
(display of data structures, for example) that are only called when you need
them. I found Sherlock's tracing ability useful in selectively viewing the
input from a serial port during program execution. This allowed me to quickly
ell whether my program was misbehaving, or whether the serial input was
corrupt.
Leaving the Sherlock object code in the application effectively embeds an
extensive diagnostic capability that's easy to use and simple to document.
Sherlock is also a profiling tool, which means it can gather frequency and
timing statistics and report them at any time. This feature can be of
particular value when trying to track down the 20 percent of the code that
uses 80 percent of the CPU's time, or when trying to track down a
boundary-value bug. While I normally don't need this capability, I was
pleasantly surprised to see how easy it was to implement.
Typically, Sherlock macros are enabled or disabled from the command line by
including the macro name with a special prefix (like '+ +' or '--',
respectively). These command-line arguments are removed by the SL_PARSE()
macro, which passes the remaining arguments to the subsequent code. The scope
of Sherlock tracepoints can also be controlled from other macros, particularly
the SL_ON() and SL_OFF() macros.
Defining or not defining the compile-time variable SHERLOCK determines whether
the macros in the code are expanded or whether they disappear without a trace.
Because Sherlock macros are unobtrusive in source code, there's really no need
to remove them before generating production versions of executable code. This
capability is valuable for code that undergoes periodic revision or
enhancement.


Utilities


You can get a head start on mastering Sherlock by using the Sherlock
preprocessor (SPP). This program automatically inserts macros to trace the
entry and exit of all functions in a file and inserts initialization macros in
the main() function. Studying such a "Sherlocked" file with the manual at hand
speeds the learning process. Another Sherlock utility, SDEL reverses the
process should you decide to remove macros.
A third utility, SDIF, is a file comparison program that is designed to
compare an "un-Sherlocked" file with its "Sherlocked" cousin. The output of
SDIF lets you immediately see which macros are in the file.
The C source code for SDEL and SDIF is included with the package. In addition,
the source to CPP, a C preprocessor compatible with the draft ANSI standard of
1-11-88, is included as an extensive example of how Sherlock macros are used.


No Nonsense


Despite the sophistication of the Sherlock package as a whole, there are some
things to watch out for. Sherlock is intended for intermediate-to-advanced C
programmers, and thus, doesn't go into the level of startup detail one might
expect in a general-purpose package. For example, some things are dropped in
the user's lap with no-nonsense explanations, such as the existence of two
sets of macros (the second, in case your compiler does not allow multiple
static variables to have the same name even if declared in separate blocks).
Another such example is a warning to the effect that the timing interrupt
handler (supplied in assembler source and in small- and large-model object
file formats) may have to be twiddled for non-100 percent IBM compatibles.
Sherlock is not copy-protected and Ed Ream's no-nonsense license agreement
asks the user to treat the software just like a book. Just as no-nonsense is
the author's offer of a satisfaction-or-money-back guarantee.
Sherlock offers excellent value and utility for the price, and belongs in the
toolkit of every serious C programmer.











April, 1989
...BUT CERIOUSLY FOLKS


This month the Doctor examines Edward K. Ream's "Sherlock" debugger, TSR
System's "C"erious Toolkit," Crescent Software's "Basic Quickpak
Professional," and a book called "The Puzzling Adventures of Dr. Ecco."




Keith Weiskamp


Keith Weiskamp is the author of numerous computer books including Advanced
Turbo C Programming.




Product Information


"C"erious Toolkit; TSR Systems Ltd., 1600 B Main Street, Port Jefferson, N.Y.
11777; 516-331-6336. IBM PC or PS/2 and compatibles. Requires: DOS 2.0 or
later; Turbo C 2.0, Microsoft 5.0, or Watcom C 6.5. Price: libraries and
documentation $99; libraries, documentation, and source code $199.
If you take your C programming seriously, you're going to need a set of
serious tools. Let's face it, writing the same pop-up window interface or
string-handling routine over and over is a complete waste of time. To help you
write more powerful C programs, TSR Systems has developed a toolkit of useful
functions for the major PC-based C compilers -- Microsoft C, Turbo C, and
Watcom C.
TSR's offering reduces the difficulty of adding pop-up windows and menus to
your programs. In addition, you'll find the tools for accessing your PC's
hardware including the screen, disk drive, keyboard, speaker, and printer a
real productivity booster. For a slight additional expense ($50), you can
obtain the "C"erious Toolkit Plus, which includes a set of terminate and stay
resident (TSR), and hot-key routines.


What You Get


The standard "C"erious Toolkit comes complete with a library of over 100
functions and a 111-page manual. The first part of the manual is devoted to
background information, such as how the tools are linked with different
compilers and a brief presentation of the data structures required to support
the tools. For some strange reason the Turbo C compiler is not covered. The
second part of the manual provides a quick reference for each of the
functions. Here, you'll find short examples that demonstrate how parameters
and return values are used. Overall, I was disappointed with the manual,
especially the reference section, because the examples were incomplete and
hard to read. The best way to learn how to use the functions is by
trial-and-error or by studying the sample programs that are included with the
toolkit.
The functions are grouped into 11 categories as shown in Table 1.
Table 1: Categories of functions included with the toolkit

 Cursor Routines for reading, moving, and displaying cursors
 Disk Reading the status of disk drives
 Keyboard Low-level keyboard access
 Printing Initialize the printer, print characters and screen
 images
 Rectangle Fills, moves, copies, erases, and saves rectangular
 screen regions
 Screen I/O Set attributes, restore and save screen images, read
 and write characters, strings, and tokens
 Sound Control the PC's speaker
 String processing Insert, delete, and search for characters; justify
 strings
 Window Everything you need to create dynamic pop-up windows
 Video Control video modes
 Miscellaneous Internal functions, determine RAM size, read the light
 pen, and other goodies

The tools are packed into a library that fits on one disk. I didn't review the
source code version of the product, but I assume that the source files are
provided on a second disk. The tools are coded in assembler and C. In general,
you'll find that they're optimized for size and speed. In fact, TSR provides a
menu program that was constructed with their tools and weighs in at only 7,116
bytes. A similar version of the program constructed with the popular Blaise C
tools required over 35K.
With the standard version, TSR includes a demo disk that provides a set of
sample programs. By working with the sample programs, I discovered some
powerful applications of the tools that were not apparent from reading the
manual. For example, at first glance I didn't think that features, such as
pop-up menu interfaces, could be generated without a lot of work. After
exploring the sample programs, however, I realized that with a little
creativity and a few switch statements it could be done. You'll also find
sample programs for displaying the status of disk drives, formatting strings,
displaying windows, and many other goodies.
In most cases, the functions are relatively easy to use. Four header files are
provided, although, most of the functions only require tproto.h.


Some Improvements


The greatest defect is the manual. Although the text seems accurate and
examples are provided for each function, the type is hard to read because it's
so small. I'd suggest that TSR either provide a magnifying glass with the
product or reprint the manual. Also, you won't find a table of contents or an
index. If you spend half of your programming time looking things up in the
manual, like I do, you'll really miss the index.

The design emphasis of the tools is on speed and size. In some cases, however,
the tools are too low level for my preference. That is, you need too many
separate tools to construct something useful. For example, if you want to
display a window with a border and a title, you'll have to call at least five
different functions:
 setWind( ... );
 setAttr( ... );
 setBord( ... );
 setTitle( ... );
 strtWind( ... );
If you're displaying a lot of different windows in a program, you might want
to combine some of these functions so that you can actually display a window
with one or two calls.























































April, 1989
QUICK LOOK AT QUICKPAK


This month the Doctor examines Edward K. Ream's "Sherlock" debugger, TSR
System's "C"erious Toolkit," Crescent Software's "Basic Quickpak
Professional," and a book called "The Puzzling Adventures of Dr. Ecco."




Bruce Tonkin


Bruce Tonkin develops and sells software for TRS-80 and MS-DOS/PC-DOS
computers. You may reach him at T.N.T. Software Inc., 34069 Hainesville Rd.,
Round Lake, IL 60073.




Product Information


QuickPak Professional; Crescent Software, 11 Grandview Ave., Stamford, CT
06905; 203-846-2500. IBM PC and compatibles. DOS 2.0 or later; Microsoft Basic
and Quick Basic compilers, Versions 1.0 or higher, or Turbo Basic. Price:
$149.
QuickPak Professional is a set of routines (many in assembler) intended for
use with the Microsoft Basic and Quick Basic compilers, versions 1.0 or later,
and with Borland's Turbo Basic.
I counted 276 routines (there are a few more on disk, documented in a text
file) and saw just a handful I couldn't use; if only a dozen were worthwhile,
QuickPak would still be worth the money. I give this package an unqualified
recommendation on the basis of its value and utility.


Documentation


Before talking about the routines, you should know the downside: There is no
quick reference, summary, or index. The manual is not numbered by page but by
section and page within section; I'd prefer the former. There is a decent
table of contents, though, and the routines are usually named clearly; that
makes finding references easier than you might think. Still, an index would be
worthwhile.
It would be even nicer to have a reference card or section that listed
routines by function (for example directory routines or file routines). The
present manual does provide section headings in the table of contents, but
those headings are a bit too general. Further, there is no explanation of the
routines except those given in the body of the manual itself. The table of
contents lists a routine named "DOSVer." It's easy to guess what that one
does, but what about "FGetRT?" From the table of contents, we know only that
it's a DOS service routine. (It's similar to a random file GET, but avoids ON
ERROR handling.) Memory joggers are always useful, particularly when there are
more routines in QuickPak than keywords in Quick Basic or Turbo Basic.
I was told that a new version of the manual is in preparation. I hope these
shortcomings are addressed, because the explanations and examples in the
manual are very good --both clear and detailed. The hints and discussions of
various programming techniques are valuable, too.


Down to Business


I could probably make a career out of writing my own versions of each of the
routines in QuickPak and writing a short article about each for publication. I
haven't done that, and won't, of course. There are routines to insert and
delete elements from string and numeric arrays, sort arrays, create and
manipulate bit arrays, parse strings in downright useful ways, check for file
existence, find space available on a disk (and determine disk parameters),
determine or set the current drive, run a batch file from Basic, read or write
absolute disk sectors, manipulate moving bar menus, use a mouse, check or set
keyboard status, find the day of the week for any date, determine whether a
printer is ready, encrypt or decrypt a file, create, clear, or print to a
window (even draw a box around it), save and restore screens (even for EGA),
scan files, use 8-byte integers, speed up "slow" native functions, and
hundreds more at least as useful.
If that's not enough, Crescent also includes a workable full-screen editor
with word wrap, block and column moves, and the usual editing functions,
together with a sample spreadsheet program. Of course, functions or routines
to do the usual financial or statistical calculations (present value, annuity,
internal rate of return, depreciation, standard deviation, maximum, minimum,
and so forth) are included in the QuickPak package. Either the editor or
spread-sheet programs (all or part) can be included in code you write for your
own applications. How many times have you wanted to include a small editor or
spreadsheet in one of your programs?
Finally, complete source code is available for everything. Be warned: There's
more than one megabyte, so it's hardly an afternoon's browse. It's well
commented and worth getting.
Buy QuickPak and enjoy! You won't go wrong. It's just not possible.





















April, 1989
PUZZLING ADVENTURES


This month the Doctor examines Edward K. Ream's "Sherlock" debugger, TSR
System's "C"erious Toolkit," Crescent Software's "Basic Quickpak
Professional," and a book called "The Puzzling Adventures of Dr. Ecco."




Jonathan Amsterdam


Jonathan Amsterdam is a graduate student at MIT's artificial intelligence
laboratory. He has articles published in several magazines. He can be reached
at Room 814, 545 Technology Square, Cambridge, MA 02139.




Product Information


The Puzzling Adventures of Dr. Ecco by Dennis Shasha. W.H. Freeman and
Company, New York, 1988. Price: $9.95 (paperback).
The best puzzles have always been showcases for beautiful mathematics. Until
recently, however, one field of mathematics -- computer science -- has been
conspicuous by its absence from puzzledom. You might think that nothing could
be farther from math's abstract sparklings than the inexorable bit-by-bit
crunching of our favorite machines, but a short course in the theory of
computation would change that misguided opinion. Some of the most elegant
mathematics of our century has been inspired by computers, and it would come
as no surprise to any student of computer science that the field would provide
material for some new and wonderful puzzles.
But NYU computer scientist Dennis Shasha is the first to actually write such
puzzles -- or at least, the first to write a book of them. The book, called
The Puzzling Adventures of Dr. Ecco, contains some 40 problems, nearly all of
them drawn from the mathematics of computers, and nearly all of them fresh and
exciting. You won't find any of the tired old standbys that ask you to move
matchsticks or pennies around; instead, you'll construct digital circuits,
design a communication network that still works in the presence of failures,
and invent cryptographic protocols for the secure transmission of data. Along
the way, Shasha covers many of the key ideas of modern computer science in an
elegant, Gardneresque prose. Sometimes the text contains the explanation; but
often, the puzzles themselves lead you to discovery. For instance, solving a
seemingly innocuous puzzle about ranking tennis players can lead you to a
beautiful sorting algorithm. In the solutions at the end of the book, Shasha
often reveals the inspiration for the puzzle and refers the interested reader
to a paper or textbook. Occasionally, however, no good explanation is provided
--the beautiful sorting algorithm is named but never described explicitly, for
example --and this is a weakness of the book. On the other hand, it is a book
of puzzles, not a teaching text, and judged by those standards provides a fine
layman's introduction to the field.
The book relates the adventures of one Dr. Jacob Ecco, omniheurist. An
omniheurist, as the book's narrator, Professor Scarlet, explains, is a person
who solves all problems. Scarlet first meets Ecco in a bakery when they are
both children, where he watches Ecco win a cake from the baker by solving a
clever puzzle. The episode sticks in Scarlet's mind, and many years later,
upon discovering that Ecco lives not far from his own home, Scarlet seeks him
out, befriends him, and learns his story.
After his brilliant doctoral work, Ecco tired of academia moved to MacDougal
Street in Greenwich Village, where he earns a living by solving interesting
puzzles brought to him by tycoons, generals, engineers, and presidents in
various states of desperation. Ecco listens to their problems, presses them
for more details, and invariably asks Scarlet, "What do you make of it,
Professor?" Scarlet, who plays Watson to Ecco's Holmes, will sometimes offer a
suggestion or present a way of understanding a problem, sometimes just comment
on the problem's difficulty. Ecco then thinks for a few minutes, scribbles a
solution, and hands it to his client. While he is occupied, we, the readers,
have the opportunity to solve the puzzle. Ecco's clients are almost always
pleased by his solution and usually ask a couple of follow-up questions, which
amount to variations on the original puzzle. In solving successively more
difficult versions of the puzzle, we are often led to more general ways of
looking at the problem.
In between puzzle-solving sessions, there is plenty of time for Ecco and
Scarlet to play innumerable games of chess and to discuss topics such as the
aesthetics of mathematics, the neural basis of our passion for design, and the
foibles of human nature. There's also a bit of jetsetting: Ecco and Scarlet
travel to India, and, accompanied by non-monotonic logician Evangeline Goode,
they go windsurfing at the Columbia Gorge in Oregon. Shasha's Ecco is a
three-dimensional man, not a mere placeholder for genius --we learn about his
past, his need for privacy, his love for chess, knowledge and, of course,
puzzles, his well-justified fear that he is under surveillance, and his
constant striving for elegance and simplicity.
The book even has something of a plot: it gradually becomes apparent that
someone or some organization is out to get Ecco. They break into his
apartment, leave coded messages that are cryptic, even when decoded, and, in
the end, perhaps do something more. Just what has happened at the story's
conclusion and who is responsible are the final, unresolved mysteries of the
book.
Oh yes, there's also a contest (which runs to the middle of this month):
decode 10 messages sprinkled throughout the book and solve the puzzles they
describe and you'll be entered in a drawing for a hand-carved chess set.
You'll also get a free T-shirt proclaiming you to have joined the great Dr.
Ecco in the ranks of that most distinguished and unusual of professions, the
omniheurists.

































April, 1989
PROGRAMMING PARADIGMS


Apple Acquires Lisp Company




Michael Swaine


This first item is for those of you who lust after a Lisp workstation. You
know who you are. Others can skip ahead to this month's excursion into
superlinear speedup in parallel algorithms or my radical proposal regarding
the teaching of programming.
Early this year, Apple acquired the assets of Coral Software, a Cambridge,
Mass., software company specializing in programming languages and artificial
intelligence tools. Five Coral engineers have been installed as the core of a
Cambridge-based research lab for Larry Tesler's Advanced Technology Group.
Apart from the questions that this acquisition raises or puts to rest
regarding the kinds of software that Apple finds it appropriate to sell, it
focuses attention on the Coral's product line, and particularly on its Allegro
Common Lisp. Coral has been selling a Lisp for the Macintosh since early on,
and currently has three language products: Allegro Common Lisp, the
entry-level Pearl Lisp, and Object Logo. Although Logo is often regarded as a
language for children, Object Logo reputedly offers nearly complete access to
the toolbox and is surprisingly powerful. But it is Allegro Common Lisp that
Apple is initially distributing through the recently-ingested APDA.
Franz Inc. contributed to the original development of Allegro Common Lisp, and
will continue to offer a version of the product for the Mac running under
A/UX. Because Franz is the company supplying the Lisp implementation being
bundled with the NeXT computer, one expects smooth portage of Lisp between
these environments. Because Mathematica is also available on both machines, it
looks as though certain kinds of scientific programming could live comfortably
on either machine. Common Lisp itself was designed for portability, as Guy
Steele points out in The Book, Common Lisp: The Language.


Superlinearity Revisited


I have discussed superlinearity in this column before (August 1988). A recent
paper by V. Nageshwara Rao and Vipin Kumar casts more light on the phenomenon.
In a parallel architecture, you hope to get greater processing speed from
adding more processors, distributing a problem across them so that they can
attack it in parallel. The more processors, the greater the speed, it is fair
to expect. If you can reduce sequential running time by a factor of n through
using n processors, you are getting a linear speedup, and that's very good.
Linearity is, in fact, a kind of logical limit on the speedup you can expect
from parallelizing a problem. It's clear why this should be so: If a parallel
algorithm can solve a problem in n seconds using m processors, there is a
sequential algorithm that can solve the problem in m*n seconds, namely, the
algorithm that consists of the m components of the parallel algorithm,
executed sequentially by the single processor. In practice, full linearity is
rarely achieved, because there is always some overhead involved in
distributing the problem to the processors and combining their outputs.
According to this logic, superlinearity ought not to be possible. If you can
speed up an algorithm by a factor greater than m by parallelizing it across m
processors, it would seem that you just didn't have the fastest sequential
algorithm to begin with. And it would seem that it would be trivial to
construct a better sequential algorithm from the parallel one, as described in
the preceding paragraph.
That's why the existence of superlinear speedup in a number of parallel
algorithms is surprising.
Typically, when apparent superlinearity is encountered in the parallelization
of an algorithm, one of two situations exists. In one situation, the original
sequential algorithm can be shown to have been improved in parallelizing it,
so that resequentializing it yields a better sequential algorithm and
eliminates the superlinearity. In the other situation, the speedup is
superlinear in isolated cases, but still linear or sublinear on the average.
The former is useful if it really leads to improving the sequential algorithm,
and this has occurred.
The latter is not uncommon, and is real superlinearity, in a sense. This kind
of superlinearity is not so surprising once you examine it. After all, it's
like a local decrease in entropy: it's more than offset by increases elsewhere
in the system. And while this isolated-case superlinearity may be useful
(especially if you have some knowledge of the kinds of cases you are likely to
encounter in practice), it is more than offset in the long run by the
sublinear cases, resulting in a sublinear average speedup, which is what we
are led to expect by the logical argument against superlinearity.
Unfortunately for the argument, there are cases in which it is possible to
achieve an average speedup that is superlinear.
Rao and Kumar have found, for state-space search problems, superlinearity on
the order of a 17-fold speedup with nine processors, on the average.
The kind of problem to which Rao and Kumar address themselves fits a simple
model of depth-first search of a complete binary tree of m leaf nodes, with an
equal static partitioning of the tree among n processors.
Depth-first search is a fundamental AI algorithm, used when the problem can be
cast as "the search for a path in a directed graph from an initial node to a
goal node.... [It] is used quite frequently if good problem-specific
evaluation functions are not available." (I'm quoting Stuart Shapiro from page
1004 of his Encyclopedia of Artificial Intelligence, Volume 2, John Wiley &
Sons, 1987.) The goal is to find a solution node in a tree, and the basic
algorithm, adapted from Shapiro, looks like this:
 Put the initial node on a list called OPEN.

 While OPEN is nonempty:

 Remove the first node from OPEN and call it n.

 If n is a solution, return n and quit.

 Generate the immediate successors of n.

 Put them at the beginning of OPEN.

 End while.
That is, the algorithm searches all the way down the leftmost branch, then
backs up to the first unsearched branch and searches it, until it finds a
solution node or runs out of tree.
Solution nodes, in Rao and Kumar's model, are uniformly distributed in a
randomly located region, possibly over the whole tree (solutions live at the
leaf nodes). The assumption regarding the distribution of solutions is a good
one for problems for which no good heuristic exists. (If there is a good
heuristic, you won't even be able to get linearity, because the heuristic will
reorder the branches, driving solutions to the left part of the tree, where a
sequential depth-first algorithm will find them quickly.) The algorithm
(sequential or parallel) stops when it finds the first solution.
The parallel algorithm Rao and Kumar use slices the tree vertically into n
subtrees, and runs the sequential algorithm on each, in parallel on n
processors.
In this model, parallel depth-first search is essentially linear if there is
only one solution to find, and also linear if there are many solutions
distributed uniformly over the entire tree (more precisely, over the entire
set of leaf nodes).
Superlinearity enters the picture when there are several solutions and the
(unknown) region in which they lie is smaller than the entire tree. Maximal
superlinearity occurs when the region in which solutions lie is equal to the
size of the partition of the search space allocated to a processor; i.e., m/n
leaf nodes. If s represents the number of solutions, the maximum efficiency,
or degree of superlinearity, is given by:
 (s+1)/2-(s-1)/(2*n),
or for a large number of processors, essentially
 (s+1)/2.
Theoretically, that can be a very large number, but even the more modest 17/9
speedup the authors actually found is impressive.
This superlinearity seems to be the result of the imbalance in the solution
density over the entire solution space, which lets one of the processors get a
sizable chunk of the region in which the solutions lie. This gives that
processor an excellent chance to find a solution, and, because the first
solution found ends the search, this gives the parallel algorithm a
superlinear edge over the identical sequential algorithm.
Rao and Kumar think that there ought to be many real-world cases in which this
kind of superlinearity can be expected. There are many problems in which a
"good" non-goal node often leads to many solutions, while "bad" non-goal nodes
lead to no solutions. The solutions headed by a "good" node will lie close
together, leading to the kind of restricted solution region the authors have
modelled. They cite the Eight Queens problem as an example, and their
experiments focused on the 15 Puzzle and the Hacker's problem.
Personally, I still think superlinearity looks like magic. I can't help a
touch of skepticism about these results. The superlinearity effect Rao and
Kumar describe rests on certain information about the distribution of
solutions, namely that the solutions are clustered in a region smaller than
the entire search space. Because I don't see that information being used by
the sequential depth-first algorithm, I can't help wondering if the deck isn't
stacked against it, and if there isn't some sequential algorithm that
incorporates this information and brings depth-first search back to linear
reality. I dunno.



Paradigms for Beginners


I'd like to advance a radical proposal: That programming paradigms be used as
an entry into the study of computer science and programming. I'd like to see
programming paradigms studied in interdisciplinary courses that serve as the
student's first and only course in computer science. I'd also like to see the
study of paradigms introduced into the core computer science curriculum at the
very beginning of the computer science student's coursework.
I'm not saying that I don't believe there are any prerequisites for the study
of computer science. The student needs some elementary mathematical knowledge
that any eighth grader ought to have, but that colleges often have to provide.
And a computer science program within an EE department will naturally have
elementary electronics and computer hardware prerequisites. For such a
program, that's certainly the right way to begin. But there is always a first
course in programming, and that's the course I have in mind. That course marks
the student's first practical experience with controlling the machine.


A Bad Introduction May Mark the Student For Life


I don't want to overstate the case. Those of us who learned Fortran or Basic
during that impressionable phase were probably not ruined forever by the
experience. But when we learned to program in Fortran or Basic, we were also
learning that what we were doing was programming. Our later experiences with
such alternative paradigms as functional programming, object-oriented
programming, or logic programming were confusing and frustrating, and at first
we found ourselves resisting the experience. We were unlearning what
programming is, and learning a new, broader idea of programming that
encompassed the new paradigm. We thought that programming involved
implementing good algorithms to solve problems, but discovered that sometimes
it's selecting representative classes to model the system. We thought that
programming was all about writing code to manipulate data, but we had to
accept that sometimes the distinction between code and data just got in the
way. Well, I don't want ever to find myself resisting learning something new;
I want to be open to new ideas. I believe that learning about programming in
the usual way leads to such resistance, by implanting fixed notions about what
programming is, notions that are just plain wrong.
What would I cover in this imaginary Computer Science 101? I can only list
some elements that seem appropriate; I don't have a syllabus. I'd present many
example programs for the students to read and study; I'd ask the students to
interpret the code, explaining what they think it does. There would be no
keying and running of programs. I'd present alternative paradigms, identified
as such, and get the students to see the differences in approach of the
different paradigms. Ideally, by the end of the course, the students would
have some ability to criticize code in several languages from the point of
view of clarity and readability (though not efficiency). I could see the
course being titled Computer Programming for Reading Knowledge.
There are several obvious arguments against using such a course in programming
paradigms as the introduction to programming.
One runs like this: 1. The differences among programming paradigms only make
sense after you've acquired one paradigm. Students needs at least one concrete
example of what programming is before they can deal with abstractions about
what programming can be. Programming is an abstract activity anyway, and it
will confuse students to introduce it at an abstract level, offering
alternative visions of the activity from the start.
Or there's this argument: 2. A course such as I describe will leave the
student still unable to write a computer program. What I propose is tantamount
to requiring a course in linguistics as prerequisite to learning a foreign
language. Learning a new language --natural or computer --is difficult, and
the student ought to have a chance to come out of the first encounter with a
feeling of success.
Or this: 3. Students of programming may need help learning the first language,
but after that, they can teach themselves. We need programmers. Many students
will take no course after the first. In the traditional scheme, these people
will be equipped to go forth and program, despite the fact that they'll have
much to learn on the job. Students who have been subjected to my proposed
Computer Science 101 will not be so equipped.
I have an answer for each.
1. I don't buy this. I've never had a college physics course, yet I have read
and understood, for recreation, dozens of books on relativity, cosmology, and
quantum physics. I think I have a sense of the issues, a feeling for what it
is like to work in the fields, and if I were to go back to school to study
physics, I think I could choose a field of study more wisely for having read
these books. While there are difficult abstractions in programming, the basic
concepts of computer programming itself are nowhere near as ethereal as those
of quantum physics. People who know nothing about programming think that it
means writing instructions to make the computer do what they want it to do,
and that's a workable first approximation.
2. I guess I don't hold the ability to write syntactically-correct Basic code
to be at college-level skill. I also don't think that it is a particularly
important skill for the great majority of people who have some interest in
computer science, but who will never become professional software developers.
The ability to read a block of code is a very different skill, is useful to
more people, and is a skill that professional programmers need but are never
explicitly taught. I do believe that it is possible to teach a basic reading
knowledge of several programming languages, exemplifying radically different
paradigms within the course of a college term. I don't mean proofreading or
finding errors in code; I mean discerning what a correct and well-written page
of code does. My CS 101 would not be like a linguistics course; it would be a
course in what computer programming is. There's no analog in natural languages
because we don't need a course telling us what a natural language is.
Weeks of student time in introductory programming courses is taken up in
identifying and correcting syntax errors, honing typing skills, letting the
problems that happen to arise dictate what the student happens to learn. These
things have to be dealt with eventually if the student is to learn to write
correct programs, but they are getting in the way of getting the idea. Those
who remember that far back should know that first programming courses don't
give a sense of success; they give a sense of accomplishment, acquired from
finally breaking through the brick wall you'd been beating your head against.
I propose sending the student to the wall equipped with an intellectual
hammer.
3. Students who are introduced to programming in the way I propose will have
to take programming courses or deal with some frustration in learning the
mechanics of programming by themselves. I think that a basic reading knowledge
of several programming languages will seriously reduce the frustration of
learning where to put the semicolons, but in any case, the students will be
coming to the process of writing code with a broader understanding. It's
possible that the approach I propose would reduce the number of
poorly-educated people passing themselves off as programmers. I guess that's a
drawback if you think that bad software is better than no software at all.


References


V. Nageshwara Rao and Vipin Kumar, "Superlinear Speedup in Parallel
State-Space Search," in Foundations of Software Technology and Theoretical
Computer Science, Nori, K.V. and Kumar, S, eds., Springer-Verlag, 1988.
Shapiro, Stuart, Encyclopedia of Artificial Intelligence, Volume 2, John Wiley
& Sons, 1987.
Steele, Guy, Jr., Common Lisp: The Language, Digital Press, 1984.
Swaine, Michael, "Programming Paradigms," Dr. Dobb's Journal, (August, 1988).































April, 1989
C PROGRAMMING


A Phone Directory and XMODEM for SMALLCOM




Al Stevens


This month adds two features to the SMALLCOM communications program introduced
in this column last month. The features are a telephone directory and the
XModem file transfer protocols. Most communications programs include support
for both capabilities. As this project proceeds from month to month, I add
features to the program by initializing hooking function pointers. Whenever
adding features this way, I will provide the lines of code that you must
change in order to turn on the hooks.
Those of you who download the code from the CompuServe online service should
be alerted to my methods. Each source code file that you download is written
just as it appears in the issue where the code is originally published. I use
the column to document minor program changes required for subsequent months
and do not change the uploaded code. Why do it this way? There are two
reasons: First, the management of those files is a significant administrative
burden borne by the already overworked and loyal staff at DDJ. Second, I
planned this project so that anyone could bail out at any time and still have
usable software. Each month presents a finished tool or a complete program.
The tools and programs are built upon those of the previous months.
You might not need all the enhancements of the programs as they progress, and
I might get run over by a pie wagon. The code you download should therefore
not be dependent on the requirements of later months. It follows then that the
programs and the columns that describe them are a matched set. You need both.


The Hooks


Listing One, page 132, hooks.c, is a replacement for a small block of code
near the beginning of smallcom.c. Its location and what it replaces should be
obvious from the comments. I do not give line number references because you
might have typed the code in, and your numbers and mine would probably not
agree. Hooks.c provides the function prototypes for the new features and
initializes the hooking function pointers. You will see hooks to Kermit
protocols in the upload and download function pointers. These are stubbed
functions. Kermit is not in this issue.


The Phone Directory


Listing Two, page 132, is phonedir.c. This source file adds the phone
directory to SMALLCOM. A phone directory is just what you might expect: a
directory of numbers that you call frequently. Because different online
services and electronic bulletin boards use different connection protocols
(baud rate, stop bits, and so on), the phone directory allows you to specify
those parameters for each entry.
The phone directory is called by the Directory selection on the SMALLCOM menu.
By initializing the phone_directory function pointer in smallcom.c to the
phdirectory function in phone-dir.c, we enable the feature. The phdirectory
function opens a full-screen window, reads the directory from the file named
phone.dir, and fills the window with text from the directory records by
calling the text_window function. The phone.dir file does not exist until you
build a directory with at least one entry. Each record in the file is a
null-terminated string with fixed position fields. The fields are separated by
spaces. This way the strings are given to the text_window function in the file
format. The get_directory function allocates a block of memory for each string
and builds the array of string pointers expected by the text_window function.
Once the phone directory is displayed, the program calls the select_window
function to allow a menu-like cursor to select an entry from the directory.
You move the cursor up and down with the arrow keys and use other keys to
select what you want to do with the entry pointed to by the cursor.
select_window returns with the number of the entry selected or zero if you
press the Esc key. The call to select_window includes the address of the
dirproc function. select_window will, therefore, call the dirproc function
when you press a key other than the Help key (F1), the Enter key, an arrow
key, or the Esc key. The dirproc function processes the Del key, which deletes
an entry; the Ins key, which inserts a new entry; the F2 key, which writes the
modified directory to the phone.dir file; and the F3 key, which modifies the
directory entry where the cursor points.
Inserting or modifying a directory entry uses the data entry window template
functions from the window library. A window is popped up over the directory,
and you modify its contents by using the data entry features.
When you press the Enter key, the selected directory record updates the
current phone number and line protocols, and the program returns to the
SMALLCOM screen where you will see the newly selected phone number at the
bottom of the screen. If you pop down the Parameters menu, you will see that
the parameters associated with the selection are now selected. You can make
these values the startup default by choosing the Write Parameters selection
from the Parameters menu.
The SMALLCOM phone directory feature is an example of a simple application of
the data entry library where the values in a flat file database are maintained
with a data entry template. Notice that there are help window mnemonics in the
directory_template FIELD array. As with last month's code, I am not
republishing the smallcom.hlp text file to add help text for the phone
directory (or for the file transfer protocol data entry template explained
later). You may compose help windows that please you for these templates. The
directory and its data fields should be obvious to you without help windows,
but you might want to add help text if you give the executable SMALLCOM
program to less sophisticated users than yourself.
Each directory record includes a name for a script file. Scripts are sequences
of interpreted commands that a communications program uses to automate routine
dialogues with online services. Each script would be specific to the service
it supports because the services use different command languages. The
directory process posts the script's name into a global string variable named
scriptfile. Later, when we add the script processor, we will use that variable
to retrieve the script file.


File Transfer Protocols


Here's the problem. When you send data across phone lines, errors called "line
hits" sometimes occur. A line hit might add a bogus character, skip a good
character, or change the binary configuration of a character. Line hits can
occur as a natural byproduct of noisy, voice-grade circuits. These errors
cannot always be detected by the serial hardware in the computers. So, if your
data cannot endure errors, you must put error-detecting and error-correcting
logic into the file transfer software. The handshaking and data formats that
manage error detection and correction in serial communications are called
"file transfer protocols."
What kind of data cannot endure errors? Three examples are executable computer
programs, compressed files, and some encrypted files. These are binary files
that must be received intact in order for them to be properly used at their
destinations. Binary files must be transmitted with a protocol that either
supports an 8-bit word length as the XModem protocol does or translates byte
values greater than 127 into a pair of 7-bit bytes as the Kermit protocol can
do. ASCII text files that contain the source code for sizable programs should
always be transmitted with an error-correcting protocol. A garbled character
in a program's source code can be difficult to spot, can go undetected by the
compiler, and can wreak havoc when the program is run.
What kind of data can endure some margin of error? Text messages can usually
be sent without error correction because their readers are humans who can
perform their own visual error correction. You can read the garbled message
"Climatv is what wi expect, weath$r is whit we gt" and know what was sent.
That example is extreme. Most transmissions get off without a hitch. The ASCII
protocol is usually acceptable for text that is meant for people to read. When
the hits get too severe for you to read the message, hang up the phone and try
another time. Don't bother switching to an error-correcting protocol. If the
line hits are so bad that you can't read the message, no error-correcting
protocol would work either.
Most communications programs allow you to select from a set of file-transfer
protocols so that they are compatible with the various online services
throughout the world. We will add that capability to SMALLCOM with a menu that
selects from the ASCII, XModem, and Kermit protocols.
Listing Three, page 133, is protocol.c. It lets you select a specific
file-transfer protocol when you are about to upload or download a file. It
opens a small window and writes the selections for three protocols: ASCII,
XModem, and Kermit. ASCII is the only protocol that was included in the
SMALLCOM program last month. XModem is added this month. I might address
Kermit in a later month.
If you want to add still another protocol, you would add an entry to the
window menu's display, code corresponding function addresses to the
up_protocol and down_protocol function pointer arrays in hooks.c, and write
the functions to implement the new protocol. The protocol name should start
with a unique letter. I've built the protocol selection menu to respond to the
usual menu cursor selection and also to select a protocol from the menu when
you press the first letter of the protocol's name. If you add a protocol, you
must insert a keystroke test for the correct first letter into the prot_key
function toward the beginning of protocol.c.
Suppose, for example, that you wanted to add the CompuServe B protocol to
SMALLCOM. Users of CompuServe might want to do that, and the way to do it is
paved by an article in the June 1986 issue of DDJ. The article, "The
CompuServe B Protocol: A Better Way to Send Files" by Levi Thomas and Nick
Turner, briefly describes the protocol. Their description is supported by a C
program written by Steve Wilhite. I have not used this code, but it implements
the CompuServe B protocol in a way that looks as if it would be readily
adaptable to the SMALLCOM architecture. The authors of the article say the
code is in the public domain, but Wilhite put a copyright notice in the source
file. Rather than mess with that contradiction and attempt to incorporate the
code into SMALLCOM, I will retreat in temerity to the usual reference book
position and leave the more difficult task as an exercise for the reader.


XModem


Almost anyone who uses modems and online services knows about XModem. It is a
file-transfer protocol designed in the early eighties by Ward Christensen, who
placed it --and CP/M programs that implemented it --into the public domain. It
is a simple and effective protocol that provides a measure of error correction
during the transmission of critical files between computers. For comprehensive
discussions of the XModem protocol, see the books on serial communications I
cited in the last two months or get a copy of the June 1985 DDJ and read
"Christensen Protocols in C" by Donald Krantz. (The editorial and article
content of all past issues are available in bound volumes, by year, from M&T
Publishing.)
Writers who tackle XModem are quick to criticize it and then hasten to
congratulate Christensen who generously gave it to the world. Discussions of
its deficiencies are a favorite pastime with some writers who do not
adequately acknowledge XModem's original purpose, which was to get around the
line hit problems people were having trying to push binary data around between
CP/M microcomputers. There was nothing about XModem that pushed the technology
when it was published. It is a simple protocol similar to ones that were being
used in asynchronous and bisynchronous communications applications in the
mini-computers of the middle seventies. It was destined to become a standard,
not because it revolutionized anything or was the absolutely best way to solve
the problem but because it was there --working, reliable, and available to
anyone who needed and wanted it. If Christensen had not put it into the public
domain, we might never have heard of it. Because he did, it became a roaring
success and remains in use everywhere.
In a nutshell (a big nutshell), here is how the XModem protocol works. The
sending XModem program transmits a file to a receiving XModem program. To
manage error detection, the sending program adds something to the data that
relates to the data records themselves. The original XModem computes and
transmits a simple 8-bit checksum of the data characters. The receiving
program calculates the checksum too, and if the calculated checksum is not the
same as the one transmitted by the sender, an error exists in either the data
characters or the checksum itself. A later improvement to XModem substitutes a
Cyclic (or Cyclical --the authors disagree. One author, Campbell, disagrees
with himself) Redundancy Check (CRC) value for the checksum, thus tightening
up the error detection.
To manage error correction, the receiving program tells the sending program
whether or not the data were received correctly. The sending program then
either sends more data or resends the data record that was in error. This
handshake would not be efficient if it were used on the entire file at one
clip. So, the XModem protocol breaks the file into fixed length, 128-byte
blocks and sends them one at a time.
The sending program cannot begin sending until it knows that the receiver is
ready to receive. The sender waits for the receiver to send a special
character that gives the go-ahead. The sender and receiver might be two
different implementations of XModem. Older versions did not support the CRC,
so newer ones must be able to support either the CRC or the checksum. If the
sender is an older version, it simply waits for a NAK (an international
transmission control protocol) (0x15) from the receiver. If the sender is a
newer version it waits for either the NAK or a 'C.' If it gets a NAK, it uses
the checksum. If it gets the 'C,' it uses the CRC. If the receiver is an older
version, it sends the NAK when it is ready to receive. If it is a newer
version, it does not know yet whether the sender can support the CRC or not.
So it sends the 'C' hoping for the best. If it does not receive the SOH (0x01)
character, which identifies the start of a block, it tries a few more times
and then sends the NAK.
When the sender knows that the receiver is ready, the sender sends a block.
Each block contains the SOH, the 1-byte block number (a serial number relative
to one), the ones-complement of the block number, 128 bytes of data, and
either the 1-byte checksum or the 2-byte CRC. Then it waits for a response
from the receiver. After the last block, the sender sends the single EOT
(0x04) character to the receiver.
The receiver reads all this stuff and looks at it. The first byte must be the
SOH or EOT character. If the EOT character is received, the receiver sends the
ACK (0x06) character, closes the file it is building, and assumes the
transmission is complete. Otherwise the receiver looks at the rest of the
packet. The block number must be either the previous block number plus one or,
in the case of a retransmission, the previous block number. The next byte must
be the ones-complement of the block number. The 128-data bytes can be anything
at all. If the checksum mode is being used, the checksum must be the 8-bit sum
of the bytes in the 128-data field. If the CRC mode is being used, the two CRC
bytes must be the CRC that is computed from the 128-data bytes plus two more
bytes of a zero value. If all this is true, the receiver writes the data block
to the file it is building and sends the ACK character to the sender. If any
of this is not true, or if the receiver times out while waiting for any of the
above from the sender, the receiver sends the NAK (0x15) character to the
sender.

If the sender sees the ACK character, it proceeds with the next block.
Otherwise, it resends the block that was just NAK'd.
The receiver works with timeouts. While it is waiting for the SOH, it sets a
ten second timeout. While waiting for each of the other bytes of the packet,
it sets a one second timeout. Timeouts and error detections are all handled
with NAKs. After so many consecutive NAKs, both the sender and receiver give
up and declare the transmission a bust. The one-second timeout can be too
short for use with services that go through a network. The network's packet
overhead on a busy day can exceed one second, causing the receiver to time
out. If this gets to be a problem, increase the timeout to some larger value,
perhaps five seconds.
Listing Four, page 134, is xmodem.c, the functions that implement the XModem
upload and download file-transfer protocols. Notice the handling of the
external xonxoff_enabled variable. While XModem is sending and receiving data,
the XON and XOFF protocols must be disabled. The data being transmitted can
consist of any bytes of any value, any of which could look like the XOFF code.
If XON/XOFF protocols were in force and a block number or checksum looked like
the XOFF, the serial transmission function would think it was being told by
the other side to quit transmitting. It would then wait, perhaps forever, for
the XON signal.
Notice also the test_wordlen function. For XModem to operate properly, the
serial port must be programmed for a word length of eight data bits. This
function prevents the upload or download from starting if eight data bits are
not programmed. Christensen specifies that the protocol can use seven bits by
clearing the most significant bit of the block numbers and checksum at both
ends of the line, but this would work only with ASCII files. Other files need
the full eight bits to be transmitted.
I am not going to attempt to explain why the CRC algorithm works because I do
not know, but here is the procedure it uses. The algorithm builds a 16-bit CRC
from a string of 8-bit bytes. The CRC starts with a zero value. The following
steps occur for each byte in the string. The byte is concatenated to the right
of the CRC to form a 24-bit value. This value is shifted left one bit eight
times. For each one-bit shift, if the most significant bit of the 24-bit value
before the shift is a one, the value 0x1021 is XORed with the leftmost 16 bits
(the CRC part) of the new 24-bit value. When all bytes have been processed
this way, the leftmost 16 bits are the CRC. This algorithm works with XModem
and should work with Kermit as well.
The CRC is built by the compute_crc function. You pass the function --a string
pointer --and the number of bytes in the string. A better and more
mathematical explanation of the CRC can be found in most of the references
already cited. Perhaps you will understand it, although that is not necessary.
The algorithm works and we know how to build it. I looked at how others coded
the algorithm and wrote the one you see here. Then I tested it by transmitting
files back and forth between SMALLCOM and another computer that was running
ProComm.
The XModem protocol was originally described in a paper published by
Christensen in the fall of 1982. In January of 1985 John Byrns published a
description of the logic that replaced the checksum with a CRC. That paper
included a C language CRC algorithm that does not work with today's compilers.
From the looks of it the code would not work with many older compilers either.
Both papers can be found on BBSs and online services around the country. Other
descriptions are found in the books and articles I've already mentioned.
To compile and build the phone directory and XModem into SMALLCOM, add
phonedir, prototype, and XModem entries to the smallcom.prj file for Turbo C,
and smallcom.mak and smallcom.lnk for Microsoft C. These files were in last
month's column.


Not Just Another Turbo C Book


Most of the C books in the bookstore have "Turbo C" in their titles. Many of
these books are about the C language itself with little information specific
to Turbo C. (Modesty prevents me from pointing out the one outstanding
exception.) Now my "boss" at DDJ, Kent Porter, has written a book that has
Turbo C in its title, and is about C, Turbo C, the PC, and good programming in
general. There are chapters on how DOS disks and files work, BIOS, the TC
video library, popup windows, menus, mice, graphics, expanded memory
management, data structures, interrupt functions, and plenty more. There is
lots of C code to explain the lessons, and it all works by using the unique
features of Turbo C.
I saw the manuscript before publication. I really like the book, and I am not
saying that just because Kent is my editor. As I write this column, the book
is not yet available but by the time you read the column, the book should be
available. Its title is Stretching Turbo C 2.0 and it is published by Brady
Books.

_C PROGRAMMING COLUMN_
by Al Stevens


[LISTING ONE]

/* ------- the hook to the phone directory ---------- */
extern void phdirectory(void);
static void (*phone_directory)(void) = phdirectory;
/* ------- the hook to script processors ---------- */
void (*script_processor)(void);
/* ------- hooks to file transfer protocols --------- */
extern int select_protocol(void);
static int (*select_transfer_protocol)(void) = select_protocol;
/* ----- up to five upload function pointers ----- */
void upload_xmodem(FILE *);
void upload_kermit(FILE *);
static void (*up_protocol[5])(FILE *file_pointer) = {
 upload_ASCII, upload_xmodem, upload_kermit,NULL,NULL
};
/* ----- up to five download function pointers ----- */
void download_xmodem(FILE *);
void download_kermit(FILE *);
static void (*down_protocol[5])(FILE *file_pointer) = {
 download_ASCII, download_xmodem, download_kermit,NULL,NULL
};






[LISTING TWO]

/* --------- phonedir.c ---------- */
#include <stdio.h>
#include <stdlib.h>
#include <conio.h>
#include <string.h>
#include <mem.h>
#include <ctype.h>
#include "window.h"
#include "entry.h"

#include "help.h"
#include "modem.h"

#define DIRECTORY "phone.dir"
#define MAX_ENTRIES 50
#define WRITEDIR F2
#define MODIFYDIR F3

void phdirectory(void);
char scriptfile[13];
static void get_directory(void);
static void put_directory(void);
static int dirproc(int, int);
static void bld_default(int);
static void build_dir(int);
static int enter_directory(int);
static void select_directory(int);
char *prompt_line(char *, int, char *);
void reset_prompt(char *, int);
static int edit_directory(void);
static int direrror(int);
static void field_terminate(FIELD *fld, int termchar);

extern int PARITY,STOPBITS,WORDLEN,BAUD;
extern char PHONENO[];
extern char spaces[];
extern struct wn wkw; /* the directory window structure */
extern void (*phone_directory)() = phdirectory;
/* -------- phone directory file record ------------ */
struct {
 char ol_name[21]; /* callee's name */
 char ol_phone[24]; /* phone number */
 char ol_parity[8]; /* none/odd/even */
 char ol_stopbits[4]; /* 1 or 2 */
 char ol_wordlen[3]; /* 7 or 8 */
 char ol_baud[6]; /* 110,150,300,600,1200,2400 */
 char ol_script[9]; /* name of script file */
} pd;
static char hdr[] =
 " Name "
 "Phone Number "
 "Parity Stop Len Baud Script";
static char select_prompt[] =
 "\030\031\021\304\331:Select Esc:Return "
 "F2:Write Directory F3:Modify "
 "Ins:Insert Del:Delete";
static char enter_prompt[] =
 " F2:Write Changes to Directory "
 " Esc:Ignore Entry F1:Help";
static char *pds[MAX_ENTRIES+1];
static int pct;
static FILE *fp;
static char nmmask[] = "____________________";
static char phmask[] = "____________________";
static char prmask[] = "____";
static char sbmask[] = "_";
static char wlmask[] = "_";
static char bdmask[] = "____";
static char scmask[] = "________";

/* ------- data entry template for the directory ------- */
FIELD directory_template[] = {
 {3, 16, 1, pd.ol_name, nmmask, "name"},
 {4, 16, 1, pd.ol_phone, phmask, "phone"},
 {5, 16, 1, pd.ol_parity, prmask, "parity"},
 {6, 16, 1, pd.ol_stopbits, sbmask, "stopbits"},
 {7, 16, 1, pd.ol_wordlen, wlmask, "wordlen"},
 {8, 16, 1, pd.ol_baud, bdmask, "baud"},
 {9, 16, 1, pd.ol_script, scmask, "script"},
 {0}
};
/* -------- data entry error messages --------- */
static char *ermsgs[] = {
 "Parity must be None, Odd, or Even",
 "Stop Bits must 1 or 2",
 "Word Length must be 7 or 8",
 "Baud Rate must be 110,150,300,600,1200,2400"
};
/* ------ manage the telephone directory ------ */
void phdirectory(void)
{
 int s = 1;
 char *ttl, *sel;
 set_help("directry");
 ttl = prompt_line(hdr, 1, NULL);
 sel = prompt_line(select_prompt, 25, NULL);
 establish_window(1,2,80,24,TEXTFG,TEXTBG,TRUE);
 get_directory();
 text_window(pds, 1);
 while (pct &&
 (s=select_window(s,SELECTFG,SELECTBG,dirproc))!=0)
 if (pct && pds[s-1] != spaces+1) {
 select_directory(s-1);
 break;
 }
 delete_window();
 reset_prompt(sel, 25);
 reset_prompt(ttl, 1);
}
/* -------- select the directory entry for the dialer ------- */
static void select_directory(int n)
{
 char *cp = scriptfile;
 movmem(pds[n], &pd, sizeof pd);
 strncpy(PHONENO, pd.ol_phone, 20);
 BAUD = atoi(pd.ol_baud);
 STOPBITS = *pd.ol_stopbits - '0';
 WORDLEN = *pd.ol_wordlen - '0';
 PARITY = (*pd.ol_wordlen == 'N' ? 0 :
 *pd.ol_wordlen == 'O' ? 1 : 2);
 establish_window(30,11,50,13,HELPFG,HELPBG,TRUE);
 gotoxy(2,2);
 cputs("Initializing Modem");
 initmodem();
 delete_window();
 setmem(scriptfile, sizeof scriptfile, '\0');
 strncpy(scriptfile, pd.ol_script, 8);
 while (*cp && *cp != ' ')
 cp++;

 strcpy(cp, ".scr");
}
/* ------ read the phone directory ----------- */
static void get_directory(void)
{
 if (pct == 0 && (fp = fopen(DIRECTORY, "r")) != NULL) {
 while (fread(&pd, sizeof pd, 1, fp) != 0) {
 build_dir(pct++);
 if (pct == MAX_ENTRIES)
 break;
 }
 pds[pct++] = spaces+1;
 pds[pct] = NULL;
 fclose(fp);
 }
 if (pct == 0)
 dirproc(INS, 1);
}
/* ------- build a default phone directory entry -------- */
static void bld_default(int n)
{
 static char *prs[] = {"None", "Odd ", "Even"};
 setmem(&pd, sizeof pd-1, ' ');
 strncpy(pd.ol_parity, prs[PARITY], 4);
 *pd.ol_stopbits = STOPBITS + '0';
 *pd.ol_wordlen = WORDLEN + '0';
 sprintf(pd.ol_baud, "%4d", BAUD);
 pd.ol_baud[4] = ' ';
 build_dir(n);
}
/* --------- build a directory entry for display ----------- */
static void build_dir(int n)
{
 if ((pds[n] = malloc(sizeof pd)) != NULL)
 movmem(&pd, pds[n], sizeof pd);
}
/* ------- write the phone directory ---------- */
static void put_directory(void)
{
 int i;
 fp = fopen(DIRECTORY, "w");
 for (i = 0; i < pct; i++)
 if (pds[i] != spaces+1)
 fwrite(pds[i], sizeof pd, 1, fp);
 fclose(fp);
}
/* ---------- process a directory entry ------------- */
static int dirproc(int c, int lineno)
{
 int i, j;
 switch (c) {
 case DEL:
 if (pds[lineno-1] != spaces+1) {
 free(pds[lineno-1]);
 for (j = lineno-1; j < pct; j++)
 pds[j] = pds[j+1];
 if (--pct) {
 text_window(pds, wkw.wtop);
 for (i = pct+2; i <= wkw.wtop+wkw.ht; i++)

 writeline(2, i, spaces+1);
 if (lineno-1 == pct)
 --lineno;
 }
 else
 clear_window();
 }
 break;
 case INS:
 if (pct == MAX_ENTRIES)
 break;
 i = pct;
 if (i)
 while (i >= lineno) {
 pds[i] = pds[i-1];
 --i;
 }
 bld_default(i);
 pct++;
 case MODIFYDIR:
 if (pds[lineno-1] != spaces+1) {
 movmem(pds[lineno-1], &pd, sizeof pd);
 enter_directory(lineno-1);
 }
 break;
 case WRITEDIR:
 put_directory();
 break;
 }
 wkw.wy = lineno - wkw.wtop + 1;
 return (pct == 0);
}
/* ------- data entry for a directory record ---------- */
static int enter_directory(int lineno)
{
 int s = 1;
 char *p = prompt_line(enter_prompt, 25, NULL);
 establish_window(20,5,56,15,ENTRYFG,ENTRYBG,TRUE);
 window_title(" Telephone Directory Entry ");
 gotoxy(3,3), cputs("Name:");
 gotoxy(3,4), cputs("Phone:");
 gotoxy(3,5), cputs("Parity:");
 gotoxy(3,6), cputs("Stop Bits:");
 gotoxy(3,7), cputs("Word Length:");
 gotoxy(3,8), cputs("Baud Rate:");
 gotoxy(3,9), cputs("Script:");
 field_terminate(directory_template, '\0');
 while (s != WRITEDIR && s != ESC) {
 s = data_entry(directory_template, FALSE, s);
 if (s == WRITEDIR)
 s = edit_directory();
 }
 field_terminate(directory_template, ' ');
 *(((char *)(&pd)) + sizeof pd - 1) = '\0';
 delete_window();
 reset_prompt(p, 25);
 if (s == WRITEDIR) {
 movmem(&pd, pds[lineno], sizeof pd);
 put_directory();

 }
 text_window(pds,wkw.wtop ? wkw.wtop : 1);
 return (s != ESC);
}
/* -------- validate the directory entry -------- */
static int edit_directory(void)
{
 int i;
 static int bds[] = {110,150,300,600,1200,2400};
 *pd.ol_parity = toupper(*pd.ol_parity);
 if (*pd.ol_parity != 'N' &&
 *pd.ol_parity != 'O' &&
 *pd.ol_parity != 'E')
 return direrror(3);
 if (*pd.ol_stopbits != '1' && *pd.ol_stopbits != '2')
 return direrror(4);
 if (*pd.ol_wordlen != '7' && *pd.ol_wordlen != '8')
 return direrror(5);
 for (i = 0; i < 6; i++)
 if (atoi(pd.ol_baud) == bds[i])
 break;
 if (i == 6)
 return direrror(6);
 return WRITEDIR;
}
/* ------- post a directory entry error ---------- */
static int direrror(int n)
{
 error_message(ermsgs[n-3]);
 return n;
}
/* -------- set field terminators to null or space ------- */
static void field_terminate(FIELD *fld, int termchar)
{
 for (;fld->frow;fld++)
 *(fld->fbuff+strlen(fld->fmask)) = termchar;
}






[LISTING THREE]

/* ------------- protocol.c --------------- */

#include <stdio.h>
#include <conio.h>
#include <ctype.h>
#include "window.h"
#include "help.h"
#include "menu.h"

static char *prots[] = {
 " ASCII",
 " Xmodem",
 " Kermit",
 NULL

};

/* ----- translate A,X,K keystrokes for protocol menu ----- */
static int protkey(int ky, int lnno)
{
 ky = tolower(ky);
 return ky=='a' ? 1 : ky=='x' ? 2 : ky=='k' ? 3 : ERROR;
}

/* --- file transfer protocol for uploads and downloads --- */
int select_protocol(void)
{
 extern MENU *mn;
 MENU *holdmn;
 static int rtn = 0;
 holdmn = mn;
 mn = NULL;
 set_help("protocol");
 establish_window(25,7,55,11,MENUFG,MENUBG,TRUE);
 window_title(" Select Transfer Protocol ");
 text_window(prots, 1);
 rtn = select_window(rtn?rtn:1,SELECTFG,SELECTBG,protkey);
 delete_window();
 mn = holdmn;
 return rtn ? rtn-1 : 0;
}

/* ---- These are stubs, to be replaced later ---- */
void upload_kermit(FILE *fd)
{
 error_message("Upload KERMIT not implemented");
}

void download_kermit(FILE *fd)
{
 error_message("Download KERMIT not implemented");
}





[LISTING FOUR]

/* -------------- xmodem.c --------------- */
#include <stdio.h>
#include <conio.h>
#include <stdlib.h>
#include <mem.h>
#include "window.h"
#include "serial.h"

#define RETRIES 12
#define CRCTRIES 2
#define PADCHAR 0x1a
#define SOH 1
#define EOT 4
#define ACK 6
#define NAK 0x15

#define CAN 0x18
#define CRC 'C'
/* -------- external data ---------- */
extern int TIMEOUT;
extern int WORDLEN;
extern int xonxoff_enabled;
/* --------- local data ------------ */
static int tries; /* retry counter */
static char bf [130]; /* i/o buffer */
/* -------- prototypes ------------- */
extern int keyhit(void);
static void receive_error(int, int);
static void xmodem_msg(char *);
static void test_wordlen(void);
unsigned compute_crc(char *, int);
/* --------- error messages ----------- */
static char *errs[] = {
 "Timed Out ",
 "Invalid SOH ",
 "Invalid block # ",
 "Invalid chksum/crc"
};
/* ---------- upload with xmodem protocol ------------- */
void upload_xmodem(FILE *fd)
{
 int i, chksum, eof = FALSE, ans = 0, ln, crcout = 0;
 unsigned crc;
 char bno = 1;
 xonxoff_enabled = FALSE;
 establish_window(20,10,52,14,MENUFG,MENUBG,TRUE);
 window_title("XMODEM Upload (CHKSUM)");
 tries = 0;
 test_wordlen();
 /* ----- wait for the go-ahead from the receiver ------ */
 TIMEOUT = 6;
 while (tries++ < RETRIES && crcout != NAK && crcout != CRC)
 crcout = readcomm();
 if (crcout == CRC)
 window_title(" XMODEM Upload (CRC) ");
 TIMEOUT = 10;
 /* -------- send the file to the receiver ----------- */
 while (tries < RETRIES &&
 !eof && ans != CAN && !timed_out()) {
 /* ---- read the next data block ----- */
 setmem(bf, 128, PADCHAR);
 if ((ln = fread(bf, 1, 128, fd)) < 128)
 eof = TRUE;
 if (ln == 0)
 break;
 gotoxy(2, 2);
 cprintf("Block %d ",bno);
 chksum = 0;
 if (keyhit())
 if (getch() == ESC) {
 writecomm(CAN);
 break;
 }
 writecomm(SOH); /* SOH */
 writecomm(bno); /* block number */

 writecomm(~bno); /* 1s complement */
 /* ------- send the data block ------ */
 for (i = 0; i < 128; i++) {
 writecomm(bf[i]);
 chksum += bf[i]; /* checksum calculation */
 }
 /* -- send error-correcting value (chksum or crc) -- */
 if (crcout == NAK)
 writecomm(chksum & 255);
 else {
 crc = compute_crc(bf, 130);
 writecomm((crc >> 8) & 255);
 writecomm(crc & 255);
 }
 /* ----- read ACK, NAK, or CAN from receiver ----- */
 ans = readcomm();
 if (ans == ACK) {
 bno++;
 tries = 0;
 gotoxy(2, 4);
 cprintf(" ");
 }
 if (ans == NAK) {
 eof = FALSE;
 gotoxy(2, 4);
 cprintf("%2d tries", ++tries);
 /* ---- position to previous block ----- */
 if (fseek(fd, -128L, 1) == -1)
 fseek(fd, 0L, 0);
 }
 }
 if (eof) {
 writecomm(EOT); /* send the EOT */
 readcomm(); /* wait for an ACK */
 xmodem_msg("Transfer Completed");
 }
 else
 xmodem_msg("Transfer Aborted");
 xonxoff_enabled = TRUE;
}
/* ---------- download with xmodem protocol ------------- */
void download_xmodem(FILE *fd)
{
 int blk=0, soh= 0, bn, nbn, i, crcin = TRUE, fst = TRUE;
 unsigned chksum, cs, cs1;
 xonxoff_enabled = FALSE;
 establish_window(20,10,52,14,MENUFG,MENUBG,TRUE);
 window_title("XMODEM Download (CHKSUM)");
 /* - send Cs then NAKs until the sender starts sending - */
 tries = 0;
 test_wordlen();
 TIMEOUT = 6;
 while (soh != SOH && tries < RETRIES) {
 crcin = (tries++ < CRCTRIES);
 writecomm(crcin ? CRC : NAK);
 soh = readcomm();
 if (!timed_out() && soh != SOH)
 sleep(6);
 }

 if (crcin)
 window_title(" XMODEM Download (CRC) ");
 while (tries < RETRIES) {
 if (timed_out())
 receive_error(0, NAK);
 /* -- Receive the data and build the file -- */
 gotoxy(2,2);
 cprintf("Block %d ", blk + 1);
 if (!fst) {
 TIMEOUT = 10;
 soh = readcomm();
 if (timed_out())
 continue;
 if (soh == CAN)
 break;
 if (soh == EOT) {
 writecomm(ACK);
 break;
 }
 }
 fst = FALSE;
 TIMEOUT = 1;
 bn = readcomm(); /* block number */
 nbn = readcomm(); /* 1's complement */
 chksum = 0;
 /* ---- data block ----- */
 for (i = 0; i < 128; i++) {
 *(bf + i) = readcomm();
 if (timed_out())
 break;
 chksum = (chksum + (*(bf + i)) & 255) & 255;
 }
 if (timed_out())
 continue;
 /* ---- checksum or crc from sender ---- */
 cs = readcomm() & 255;
 if (crcin) {
 cs1 = readcomm() & 255;
 cs = (cs << 8) + cs1;
 }
 if (timed_out())
 continue;
 if (soh != SOH) { /* check the SOH */
 receive_error(1, NAK);
 continue;
 }
 /* --- same as previous block number? --- */
 if (bn == blk)
 fseek(fd, -128L, 1);
 /* --- no, next sequential block number? --- */
 else if (bn != blk + 1) {
 receive_error(2, CAN);
 break;
 }
 blk = bn;
 /* --- test the block # 1s complement --- */
 if ((nbn & 255) != (~blk & 255)) {
 receive_error(2, NAK);
 continue;

 }
 if (crcin)
 chksum = compute_crc(bf, 130);
 /* --- test chksum or crc vs one sent --- */
 if (cs != chksum) {
 receive_error(6, NAK);
 continue;
 }
 soh = bn = nbn = cs = 0;
 tries = 0;
 /* --- write the block to disk --- */
 fwrite(bf, 128, 1, fd);
 if (keyhit())
 if (getch() == ESC) {
 writecomm(CAN);
 break;
 }
 writecomm(ACK);
 }
 if (soh == EOT)
 xmodem_msg("Transfer Complete");
 else
 xmodem_msg("Transfer Aborted");
 TIMEOUT = 10;
 xonxoff_enabled = TRUE;
}
/* ------------- send a nak ------------ */
static void receive_error(erno, rtn)
{
 ++tries;
 if (TIMEOUT == 1) {
 gotoxy(2,4);
 cprintf("%s (%d tries)", errs[erno], tries);
 }
 writecomm(rtn);
}
/* ------ test for valid word length -------- */
static void test_wordlen(void)
{
 if (WORDLEN != 8) {
 gotoxy(2,4);
 cprintf("Must be 8 Data Bits");
 tries = RETRIES;
 }
}
/* --------- final message about xmodem transfer -------- */
static void xmodem_msg(char *s)
{
 gotoxy(2,3);
 cprintf(s);
 putch(BELL);
 sleep(3);
 delete_window();
}
/* --------- compute the crc ------------ */
unsigned compute_crc(char *bf, int len)
{
 int i;
 long crc = 0;

 while (len--) {
 crc = (*bf++) & 255;
 for (i = 0; i < 8; i++) {
 crc <<= 1;
 if (crc & 0x1000000L)
 crc ^= 0x102100L;
 }
 }
 return (unsigned) (crc >> 8);
}




















































April, 1989
GRAPHICS PROGRAMMING


Matching My Wife's Wet Washcloth




Kent Porter


My wife and I have an arrangement regarding home decor. We didn't plan it this
way, it's just how it happens. She decides when a room needs redecorating and
what needs to be done. Then I do all the work. Afterwards she shows it to
friends and says, "Look what we did." That plural pronoun always bothers me
because, after all, she was probably out shopping most of the time I was
toiling. On the other hand, the "we" might be fair inasmuch as, if left solely
up to me, nothing would get done around the house beyond replacing dead light
bulbs.
Anyway (and there is a point to this), some years ago as she was preparing to
put me to work, my wife took a washcloth to the paint store and told the guy,
"Wet this and mix a paint that's the same color." Amazingly, he was able to do
it. Starting with white, he added two squirts of this pigment, three of that,
one of another, then stirred it up. And there it was, a perfect match.
We might not be able to exactly duplicate the color of my wife's wet washcloth
on the EGA screen, but we can probably come pretty close. And the process is
almost the same as the salesman used to blend the paint. We probably can match
it exactly in VGA 256 color mode; however, that's significantly more
complicated than EGA colors, so we'll put it off until later. As pointed out
in earlier columns, the VGA behaves just like the EGA when in an EGA graphics
mode or 640X480 16-color mode, so this discussion applies to both.
Graphics images are made from pixels, which in turn are represented by bit
patterns in programs. The color that a pixel assumes on the screen is a
function of the bit pattern placed in the video buffer's corresponding
location. Therefore a pixel's data representation value determines its color,
right?
Well, not really. There's a data structure between the video buffer and the
screen itself, the contents of which governs the mapping of pixel values to
actual colors. This data structure is called the color palette.
The name no doubt derives from an analogy to the artist's palette, a board
containing an array of colors from which the artist selects as he or she
paints. That's apt, because the EGA palette is an electronic equivalent. It
holds an array of 16 colors in elements called registers. A pixel value is
simply an index into this array. When you stuff pixel value 2 into the video
buffer, in effect you say, "I want this pixel to have the color contained in
palette register 2."
As it refreshes the screen, the video controller grabs a pixel value from the
video buffer, then uses that value to locate the actual color in the palette.
That's how color mapping occurs, and why EGA pixel values must be in the range
0 - 15.
So how does a palette register control the actual color? Through a rather
oddly arranged 6-bit number that governs the intensities of the three basic
colors --red, green, and blue --from which all displayable hues are derived.
Because a 6-bit number can represent 64 different values, the EGA is capable
of displaying 64 colors. However, there can be a maximum of only 16 at a time
due to the size of the palette. You can instantly change the color of all
instances of a given pixel value by changing the contents of its corresponding
palette register, but you can't do anything on an EGA to get more than 16 out
of 64 colors at one time; it's a hardware limitation. If you gotta have more,
get a VGA, which does 256 colors at a time from a total selection of 262,144.
Still, 16 out of 64 colors at a resolution of 640x350 is enough to do some
pretty respectable graphics.
The EGA initializes its palette to the same color sequence as the CGA's fixed
text colors. This provides the default pixel value-to-color mapping shown in
Table 1. The program COLORS.C in Listing One, page 138, displays the default
EGA palette. The program must be linked with GRAFIX.LIB developed in earlier
installments of this column.
Table 1: Default color palette for the EGA and VGA

 Value Color
 -----------------------

 0 Black
 1 Blue
 2 Green
 3 Cyan
 4 Red
 5 Magenta
 6 Brown
 7 Light gray
 8 Dark gray
 9 Light blue
 10 Light green
 11 Light cyan
 12 Light red
 13 Light magenta
 14 Yellow
 15 White

If you've done much color programming in text, you've probably got the default
values so engraved in your mind that 14 and yellow have become synonymous. The
ability to change palette registers to any color you want means you'll have to
break an old habit. From now on, pixel value 14 refers to whatever is in
palette register 14 at the time.


Squirts of Pigment


The guy that matched my wife's washcloth made the color by squirting various
quantities of one pigment and another into a white base. The process of
creating EGA colors is similar, except that we start from a base of black.
Display colors are hues blended by combining various measures of three primary
colors: red, green, and blue. These measures are actually intensities ranging
from none (black) through bright. There are four intensities of each primary
color, which is how we get 64 possible colors on the EGA (4{3}3). Here's how
it works.
A byte contains eight bits, but a palette register is a 6-bit value.
Consequently the two high-order bits are "don't care" and the remaining low
six bits convey color information. It doesn't take a degree in math to figure
out that, with three colors and six bits, there are two bits per primary
color, and that each set of two bits can represent any of four intensities.
What is not so obvious is how those bits are arranged. The designers of the
EGA no doubt had their reasons for doing it as they did, but I don't know what
those reasons were. And whether it makes sense or not, it's how it is.
The palette bits are arranged in the order rgbRGB, where lower case indicates
lower intensity. Thus the palette value xx100000 (binary) is a very dark shade
of red, and palette value xx000011 is light cyan (a combination of bright blue
and bright green). Turning all bits on yields pure white, while resetting all
bits produces black, or the absence of any color component.
The default palette colors are grouped according to "normal" and "intense"
hues. Thus registers 0 - 7 hold a sequence of colors with normal intensity,
and 8 - 15 contain their brighter counterparts in the same sequence. The
primary colors (red, green, and blue) in the lower range are actually of
intensity level 2, or one level away from brightest. The default palette
doesn't display any low-intensity primary colors, which tend to be quite dim
on the display and are thus mostly useful for mixing to create composite
colors.
An example is brown. It's composed of red level 2 and green level 1. To get
this mixture, set the bits as follows:
 rgbRGB ------


 10100
This yields the value 14h. Notice that brown contains no blue component, so
bits b and B are both 0. This corresponds to blue level 0, or in other words
the absence of blue.
Adding blue level 1 to the mix (011100b) is the same as squirting a little
blue pigment into tan paint. The outcome is mauve, which is a sort of
washed-out purple. If instead we take brown and "promote" both its colors
upward one level (to green 2 and red 3, represented as 100110b) we get bright
brown, or in other words yellow.
Because each of the primary color levels is represented by a fixed bit
pattern, we can give them symbolic names to improve program readability. Table
2 shows the mapping of names to bit patterns. Using them, we can construct
color palette values with expressions such as
 brown = GRN1 RED2;
My colleagues in the publishing business have a name for this technique of
forming hues by mixing measures of three primary colors: They call it color
separation. To me that's backwards. It's really color combination. Whatever
you call it, it works for the painter, the printer, and the programmer. Later
in this article we'll look at a program that you can use to experiment with
color mixing on the EGA.
Table 2: Symbolic names for the primary color intensities

 Name Binary Hex
 ------------------

 RED0 000000 0x00
 RED1 100000 0x20
 RED2 000100 0x04
 RED3 100100 0x24
 GRN0 000000 0x00
 GRN1 010000 0x10
 GRN2 000010 0x02
 GRN3 010010 0x12
 BLU0 000000 0x00
 BLU1 001000 0x08
 BLU2 000001 0x01
 BLU3 001001 0x09



Adding EGA Colors to the GRAFIX Library


In order to support EGA colors, we need to modify the GRAFIX.H header file
that we're building and add some functions to the library.
Each month we'll be adding something to the header file, so I'll list that
entire file with the additions set apart and labeled by a heading indicating
the month they appeared. As you can see from Listing Two, page 138, this month
we're adding color constants and the definition of a byte type to the top of
GRAFIX.H, and four functions to the bottom.
The functions themselves are defined in EGAPALET.C (Listing Three, page 138).
You should compile this file to an .OBJ, then add it to the GRAFIX library
with the DOS command
 LIB GRAFIX +EGAPALET;
Let's look at what this program module does.
Because it's declared outside the scope of any function, the ega_palette array
is a static visible to the four functions in this compile unit. Furthermore,
it's initialized to the default colors of the real palette. Note that this is
our copy of what's in the palette known to the video controller; it's not the
palette. Because it carries the same values at initialization as the video
controller's, and because the palette-setting function maintains it, we can
safely assume that the local copy always accurately reflects the colors
appearing on the display.
The ega_palreg( ) function returns the color value in a given palette
register. Pass it the register number (that is, pixel value) and the function
returns the 6-bit composite color pattern from that register.
Note that the ROM BIOS video services furnish function 10h, subfunction 7, to
read a hardware palette register directly. If you put 10h in register AH, 7 in
register AL, and the register number in BL, then execute interrupt 10h, you
get the color pattern back in register BH. This is an alternative that is
slightly safer than returning a value from the phantom software palette, but
it incurs the heavy overhead of calling the ROM BIOS. Consequently we use the
method given here in the interest of performance.
The next function, ega_blend( ), is a color pattern constructor. It combines
the primary components into a byte that you can then load into a palette
register or use for other purposes. For example, to make brown, write
 brown = ega_blend (RED2, GRN1, BLU0);
Note that the order of arguments with respect to color sequence doesn't
matter. The function simply combines bits. Thus,
 brown = ega_blend (BLU0, GRN1, RED2);
produces exactly the same result.
The get_ega_colormix( ) function is the opposite of ega_blend( ). It explodes
the content of a given palette register into its three primary color
intensities. Here the sequence of color arguments is significant, and
additionally the arguments must be addresses of receiving variables (since a
function can return only one value directly). The function derives the bit
patterns by ANDing the register contents with each of the most intense primary
values, thus, isolating the appropriate bits.
The final function is set ega_palreg( ), which places a new color pattern into
the given palette register. The function updates the local copy of the
palette, then uses the ROM BIOS video services-unavoidable in this case-to
change the hardware setting. The effect of this function is to instantly
change all instances of the corresponding pixel to the new color on the
display.
Listing Four, page 138, is a utility program that you can use to experiment
with EGA color combinations. To get it up and running, compile MIX.C, then
link with GRAFIX.LIB.
This program displays a 4x3 array of filled rectangles showing the four
intensities each of red, green, and blue (the leftmost is black, the lowest
possible intensity corresponding to RED0, GRN0, and BLU0). A large filled
rectangle at the bottom of the screen contains the current color mixture; it
starts out bright white, because the most intense of all three primary shades
is selected.
A white rectangle surrounds one box in each color row. This is an indicator
showing which intensity is currently a component of the color at the bottom.
To move the indicator for red to a lower intensity, type r. The indicator
shifts left and the color bar at the bottom immediately changes to pale cyan.
To move the red indicator to its brighter neighbor, type uppercase R. You can
similarly control green with g and G, and blue with b and B. Quit by pressing
Esc; the program then shows the breakdown of the color most recently
constructed.
The local functions in MIX.C reveal how it works. The setup_palette( )
function loads reds into palette registers 1 - 4, greens into 5 - 8, and blues
into 9 - 12. Palette register 13 drives the color sample at the bottom of the
screen. setup_screen( ) constructs the work display.
Most of the program's work is done by mix_colors( ), a loop that repeats until
you press Esc. The loop gets a keystroke, then acts on it via a switch( )
whose cases vary depending on the keystroke. The RAISE and LOWER macros ensure
that the indicator box remains within the four possible intensity selections.
After the switch( ) completes, the new color mixture is computed and this
value updates palette register 13, instantly changing the color sample box.
The main( ) function itself does little besides reporting the results after
mix_colors( ) completes. The results make MIX more than just a toy or an
example of how to use the graphics library we're constructing. Use it as a
utility for designing the colors you want to use elsewhere, and the results
report to code your program.
Who knows? One of the colors you mix might match my wife's wet washcloth.

_GRAPHICS PROGRAMMING COLUMN_
by Kent Porter


[LISTING ONE]



/* COLORS.C: Shows all colors in default palette */

#include "grafix.h"
#include <conio.h>

void main ()
{
int r, c, color;

 if (init_video (EGA)) {
 for (r = 0; r < 4; r++)
 for (c = 0; c < 4; c++) {
 color = (r * 4) + c; /* next color */
 set_color1 (color);
 fill_rect ((c*160), (r*80), 158, 79);
 }
 getch(); /* wait for keypress */
 }
}






[LISTING TWO]


/* Include file for GRAFIX.LIB */
/* EGA/VGA graphics subsystem */
/* K. Porter, DDJ Graphics Programming Column */
/* ------------------------------------------ */

/* Color constants from April, 89 */
#define Black 0 /* standard colors */
#define Blue 1
#define Green 2
#define Cyan 3
#define Red 4
#define Magenta 5
#define Brown 0x14
#define LtGray 7
#define DkGray 0x38
#define LtBlue 0x39
#define LtGreen 0x3A
#define LtCyan 0x3B
#define LtRed 0x3C
#define LtMagenta 0x3D
#define Yellow 0x3E
#define White 0x3F

#define RED0 0x00 /* basic hues for mixing */
#define RED1 0x20
#define RED2 0x04
#define RED3 0x24
#define GRN0 0x00
#define GRN1 0x10
#define GRN2 0x02

#define GRN3 0x12
#define BLU0 0x00
#define BLU1 0x08
#define BLU2 0x01
#define BLU3 0x09

#if !defined byte
#define byte unsigned char
#endif

/* Supported video modes */
#define EGA 0x10 /* EGA 640 x 350, 16/64 colors */
#define VGA16 0x11 /* VGA 640 x 480, 16/64 colors */

/* Function prototypes */
/* From February, '89 */
/* ------------------ */
int far init_video (int mode); /* init display in video mode */

void far pc_textmode (void); /* PC text mode */

void far draw_point (int x, int y); /* write pixel in color1 */

void far set_color1 (int palette_reg); /* set foreground color */

/* From March, '89 */
/* --------------- */
void far draw_line (int x1, int y1, int x2, int y2);
 /* Bresenham line drawing algorithm */

void far draw_rect (int left, int top, int width, int height);
 /* draw rectangle from top left corner */

void far polyline (int edges, int vertices[]); /* draw polyline */

void far hline (int x, int y, int len); /* horizontal line */

void far fill_rect (int left, int top, int width, int height);
 /* draw solid rectangle in color1 starting at top left corner */

/* From April, '89 */
/* --------------- */
byte far ega_palreg (int preg); /* color in EGA palette reg */

void far set_ega_palreg (int reg, int color); /* set palette reg */

byte far ega_blend (byte c1, byte c2, byte c3); /* blend primaries */

void far get_ega_colormix (int preg, int *r, int *g, int *b);
 /* get mix of red, green, and blue in EGA pal register preg */






[LISTING THREE]

/* EGAPALET.C: Palette control for EGA/VGA 16-color modes */

/* Compile separately, then add to GRAFIX.LIB */
/* K. Porter, DDJ Graphics Programming Column, April '89 */
/* ----------------------------------------------------------- */

#include "grafix.h"
#include <dos.h>

byte ega_palette [16] = { /* current palette contents */
 Black, Blue, Green, Cyan, Red, Magenta, Brown, LtGray,
 DkGray, LtBlue, LtGreen, LtCyan, LtRed, LtMagenta, Yellow,
 White
};
extern int grafixmode; /* from GRAFIX.C */
/* ----------------------------------------------------------- */

byte far ega_palreg (int preg)
 /* return contents of palette register preg */
{
 return ega_palette [preg];
} /* --------------------------------------------------------- */

byte far ega_blend (byte c1, byte c2, byte c3)
 /* blend three primary colors, return color pattern */
{
 return c1 c2 c3;
} /* --------------------------------------------------------- */

void far get_ega_colormix (int preg, int *r, int *g, int *b)
 /* break color in palette reg preg into its components */
{
 *r = ega_palette [preg] & RED3;
 *g = ega_palette [preg] & GRN3;
 *b = ega_palette [preg] & BLU3;
} /* --------------------------------------------------------- */

void far set_ega_palreg (int preg, int color)
 /* change palette register preg to new color */
 /* use only for EGA 16-color mode */
{
union REGS r;

 if (grafixmode == EGA) {
 ega_palette [preg] = color; /* update our copy of palette */
 r.h.ah = 0x10; /* use ROM BIOS video services */
 r.h.al = 0; /* to update the real palette */
 r.h.bh = color;
 r.h.bl = preg;
 int86 (0x10, &r, &r);
 }
} /* --------------------------------------------------------- */






[LISTING FOUR]

/* MIX.C: Utility for mixing colors with EGA */


#include "grafix.h"
#include <conio.h>
#include <stdio.h>

#define ESC 27 /* Esc char */
#define ERASE 0
#define SHOW 15

int col[] = {30, 190, 350, 510}; /* box horiz locations */
int row[] = {30, 110, 190}; /* box vert locations */
int rr = 1, gr = 5, br = 9; /* reg index for color set */
int base_red[] = {RED0, RED1, RED2, RED3};
int base_grn[] = {GRN0, GRN1, GRN2, GRN3};
int base_blu[] = {BLU0, BLU1, BLU2, BLU3};

void main ()
{
int r, g, b;

void setup_palette (void), setup_screen (void),
 mix_colors (void);

 if (init_video (EGA)) {
 setup_palette();
 setup_screen();
 mix_colors();
 pc_textmode();
 }

 /* Report results */
 get_ega_colormix (13, &r, &g, &b);
 printf ("\nMixed color has value 0x%02X", ega_palreg (13));
 printf ("\nComponent breakdown:");
 printf ("\n Red: 0x%02X ", r);
 switch (r) {
 case RED0: puts ("(RED0)"); break;
 case RED1: puts ("(RED1)"); break;
 case RED2: puts ("(RED2)"); break;
 case RED3: puts ("(RED3)"); break;
 }
 printf (" Green: 0x%02X ", g);
 switch (g) {
 case GRN0: puts ("(GRN0)"); break;
 case GRN1: puts ("(GRN1)"); break;
 case GRN2: puts ("(GRN2)"); break;
 case GRN3: puts ("(GRN3)"); break;
 }
 printf (" Blue: 0x%02X ", b);
 switch (b) {
 case BLU0: puts ("(BLU0)"); break;
 case BLU1: puts ("(BLU1)"); break;
 case BLU2: puts ("(BLU2)"); break;
 case BLU3: puts ("(BLU3)"); break;
 }
} /* ----------------------------------------------------- */

void setup_palette (void) /* put basic colors in palette */
{

 set_ega_palreg ( 1, RED0); set_ega_palreg ( 2, RED1);
 set_ega_palreg ( 3, RED2); set_ega_palreg ( 4, RED3);

 set_ega_palreg ( 5, GRN0); set_ega_palreg ( 6, GRN1);
 set_ega_palreg ( 7, GRN2); set_ega_palreg ( 8, GRN3);

 set_ega_palreg ( 9, BLU0); set_ega_palreg (10, BLU1);
 set_ega_palreg (11, BLU2); set_ega_palreg (12, BLU3);

 set_ega_palreg (13, White); /* used for mixed colors */
} /* ----------------------------------------------------- */

void draw_box (int reg, int c, int r) /* box around color */
{
 set_color1 (reg);
 draw_rect (col[c]-5, row[r]-5, 110, 70);
} /* ----------------------------------------------------- */

void setup_screen (void) /* construct work screen */
{
int r, c, reg = 1;

 for (r = 0; r < 3; r++)
 for (c = 0; c < 4; c++) {
 set_color1 (reg++); /* select color reg */
 fill_rect (col[c], row[r], 100, 60); /* fill */
 }
 set_color1 (13);
 fill_rect (col[0], 280, 580, 60); /* mixed color area */

 for (r = 0; r < 3; r++) /* rubberband most intense */
 draw_box (SHOW, 3, r);
} /* ----------------------------------------------------- */

void mix_colors (void) /* interactive portion */
{
#define RAISE(col) col = (col < 3) ? ++col : 3
#define LOWER(col) col = (col > 0) ? --col : 0

char reply;
int r, rc = 3, gc = 3, bc = 3, mixture;

 do {
 reply = getch();
 switch (reply) {
 case ESC : break; /* quit program */
 case 'r' : r = 0; /* lower red */
 draw_box (ERASE, rc, r);
 LOWER (rc);
 draw_box (SHOW, rc, r);
 break;
 case 'R' : r = 0; /* raise red */
 draw_box (ERASE, rc, r);
 RAISE (rc);
 draw_box (SHOW, rc, r);
 break;
 case 'g' : r = 1; /* lower green */
 draw_box (ERASE, gc, r);
 LOWER (gc);

 draw_box (SHOW, gc, r);
 break;
 case 'G' : r = 1; /* raise green */
 draw_box (ERASE, gc, r);
 RAISE (gc);
 draw_box (SHOW, gc, r);
 break;
 case 'b' : r = 2; /* lower blue */
 draw_box (ERASE, bc, r);
 LOWER (bc);
 draw_box (SHOW, bc, r);
 break;
 case 'B' : r = 2; /* raise blue */
 draw_box (ERASE, bc, r);
 RAISE (bc);
 draw_box (SHOW, bc, r);
 break;
 }
 mixture = ega_blend (base_red[rc], base_grn[gc],
 base_blu[bc]);
 set_ega_palreg (13, mixture); /* update color mix */
 } while (reply != ESC);
}







































April, 1989
STRUCTURED PROGRAMMING


Three Little Words




Jeff Duntemann, K16RA


The publisher of the London Star (a sort of National Enquirer for
scone-eaters) has taken a survey indicating that the three most
attention-getting words for the British audience are (in order) "sex," "win,"
and "free." The chap said that if he could run a contest with a banner
reading, "Win free sex!" he'd be the only newspaper left in London. After
twenty years of trying to avoid seeing the tabloids down at the Piggly Wiggly
checkout counter, I'd have to say the three equivalent words to American eyes
are "murder," "UFOs," and "Elvis." (Which makes one long to run a story with
the headline, "UFOs murder Elvis!" but alas, it didn't happen that way.)
Given that real programmers keep their pants on and listen to Boz Scaggs, what
would you imagine are the three most attention-getting words on the covers of
computer magazines? My bet would fall to "windows," "speed," and "graphics."
("Speed windows graphics!" is a cry Microsoft has been ignoring for years.
...)
These three words make for strange bedfellows, given that:
Our machines are not fast enough to do graphics at a speed I find acceptable.
My benchmark is this: Slow enough to watch is way too slow. Last weekend I was
down at Fry's, shaking my head behind the yokels who were marveling at
Macintosh II graphics. The II was taking seconds to redraw a window. Landfill.
And that's supposedly the fastest machine you can buy south of ten grand. ...
Our screens are not big enough to divide even once, much less several times.
The smallest screen I would consider dividable is 19-inch diagonal, with 1024
x 1200 graphics and/or 132 x 66 text. A century of work with the 66-line
typewritten page has trained us to see such a page as a "glanceable unit," and
we are usually capable of spotting keywords on such a page without close
reading. Most of us think in terms of 8 1/2-inch by 11-inch pages. A 25-line
screen is an infuriating straitjacket.
I have often grimaced, seeing magazines present code for windowing systems
targeted at a 25-line or 350-pixel screen. This is the wrong way to be moving,
folks. We should be modeling full typewritten pages on our screens, not
postage stamps.
So, if you've been wondering what I've been leading up to in the last two
columns, it's this: I'm building a sort of anti-windowing system. Rather than
specifying a rectangular subset of the screen as a window, I'll be specifying
the hardware display as a window into a he-man's 66-line virtual screen. This
will have to do us until the rest of you guys catch on, and all buy 66-line
text screens. (The Micro Display Systems' Genius VHR has been out there for
four years, and you can buy one from 47th Street Computer in Nuh Yawk for
$999.)


Anatomy of a V Screen


A virtual screen is virtual for the same reason that virtual memory is
virtual: The hardware display doesn't have enough screen to show a full page,
so you have to put part of the page elsewhere and bring pieces of the full
page onto the display as needed.
Our "elsewhere" is the heap, and the whole thing is held together with
pointers in a highly memory-efficient fashion. (Turbo Pascal is every bit as
clever with pointers as C is. Maybe more.) Figure 1 is a schematic of the
memory architecture of a virtual screen. Follow along on the figure during the
following discussion.
What should a virtual screen allow us to do? The key is this: We are modeling
a piece of paper. Ideally, we should be able to write text to any location on
the virtual screen without worrying about whether any given portion of the
screen is currently displayed or not. Independently, we should be able to pan
the visible window into the virtual screen at will, by one line or any other
number of lines at a time.
The important thing is that we abandon the visible display as a reference
frame. Line 1 is the first line of the virtual screen, whether visible or not.
The visible display becomes a window that we slide up and down the virtual
screen, inspecting it at will.
Right off, this forces us to abandon the PC's hardware cursor, which is a
prisoner of the current visible display. Abandoning the hardware cursor is
probably a good idea. The PC's cursor management firmware is bug ridden and
quirky, and I'd just as soon turn the sorry thing off and leave it off.
(Turning it off is easy. Turning it back on again is not-but those are tales
for another time.)
Physically, the virtual screen is a collection of blocks of memory on the
heap, with one block allocated for each line on the virtual screen. The
virtual screen is held together with a descriptor record having the structure
shown in Listing One, page 142.
Here, HEIGHT is a constant that specifies the number of lines (counting from
1) in the virtual screen. I use 66, modeling our typical letter-size paper
sheet at the traditional six lines of type per inch. You can, however, change
HEIGHT to some other value to model legal-size paper (84 lines) or one of the
European paper sizes. For now, don't probe the internal structure of
individual lines (I'll get to that later), but simply think of them as storage
arrays for text.
For simplicity's sake, I have designed the virtual screen system so that the
size of the screens is fixed at compile time, and only one size of screen
(that specified by the constants HEIGHT and WIDTH) is supported at once. If
fields were added to the record screen to describe the screen's dimensions,
the size of a screen could be specified at screen initialization time; that
is, the time when a screen is created on the heap with the InitScreen
procedure. Some complication would have to be added to the supporting
procedures and functions, but it's not impossible or even especially
difficult, and might be worth it to be able to support the modeling of both
letter and legal paper in the same application at the same time.
The heart of the virtual screen record is a pair of arrays of pointers,
ShowPtrs and StorePtrs. The differences between their purposes are subtle but
important: StorePtrs points to where the virtual screen information is stored.
The pointers in StorePtrs are initialized at screen initialization time and
never change until the screen is disposed of. ShowPtrs, by contrast, contains
pointers that are used to direct display output; that is, where the
information is to be shown, hence the name. Some of ShowPtrs, pointers point
into the display adapter's visible refresh buffer. The rest point to lines in
the virtual screen on the heap. Which pointers point where is the key to this
whole virtual business.
When the virtual screen is initialized with InitScreen, the first Visible Y
pointers are set to point into successive 160-byte regions in the display
adapter refresh buffer, starting at the address given in TextBufferOrigin.
(Both VisibleY and TextBufferOrigin are initialized and exported by the
TextInfo unit I described last month. VisibleY contains the number of lines
currently displayed on the screen, and TextBufferOrigin is a pointer pointing
to the first byte of the display adapter refresh buffer.)
The field TopLine in the descriptor record stores the number of the virtual
screen line shown at the top of the visible display. In Figure 1, this is line
7; note that ShowPtrs[7] is the first pointer in the array that points into
the physical display refresh buffer, rather than into the virtual screen
itself, Starting with the index stored in TopLine, a number of pointers
corresponding to the number of visible lines (VisibleY) point into the visible
refresh buffer.
Writing to the screen is easy. We don't have to know whether the line to which
we wish to write is currently visible. If we want to write to line 8, we
specify line 8 with a custom GotoXY procedure, and then the WriteTo procedure
will send the output to whatever line is pointed to by ShowPtrs[8]. If
ShowPtrs[8] points to a line on the visible display, we see the output
immediately; if ShowPtrs[8]instead points to a line in the virtual screen, we
won't see the output and will have to pan the visible display until the newly
written line comes into view.


Panning the Screen


Sliding the visible display up and down as a "window" into the virtual screen
is by far the trickiest part of SCREENS.PAS (Listing Two, page 142).
Regardless of which direction the pan operation takes, the work is all done
within procedure pan. Understanding pan is best done by looking to Figure 1
during the following discussion.
Accomplishing a pan involves a number of separate tasks, each of which must be
accomplished in a specific order. It's possible to perform both an upward and
a downward pan using the same run of code, but the algorithm is more
straightforward by testing for the direction of the pan and using a separate
sequence of statements for each of the two directions. The following sequence
is for a downward pan; that is, sliding the visible window such that first
line 1 is at the top, then line 2, then line 3, and so on.


By the Numbers


1. Make sure that the pan will not take the visible display beyond the array
bounds of the virtual screen. To do so would trigger a runtime range error if
range error trapping is enabled. If range error trapping is not enabled, the
program will behave erratically and probably abort to DOS. If a multiple-line
pan would take the visible display out of range, but some number of lines
remain between the limit of the visible display and the end of the virtual
screen, the number of lines in the pan (By Lines) is adjusted so that the pan
"just makes it."
2. Copy the newly hidden line or lines from the visible display to the virtual
screen. Remember that I/O to a virtual screen may be sent to either the
refresh buffer or to the virtual screen lines allocated on the heap. A visible
line may contain data that does not exist in the visible line's virtual
counterpart on the heap. Therefore, each time a visible line or lines "roll
off" the top or bottom of the visible screen, those lines must be copied into
their corresponding virtual lines.
3. Glitch the portion of ShowPtrs that points into the visible refresh buffer
down by the number of lines in the pan. If the top line being displayed is 7
(as shown in Figure 1) the series of pointers beginning at ShowPtrs[7] must be
moved down the array by 4 bytes (the size of a Turbo Pascal pointer) so that
what was ShowPtrs[7] becomes ShowPtrs[8] and so on. The number of pointers
involved depends on the number of lines in the visible display, which is given
by global variable VisibleY. The glitch is done with a simple call to Turbo
Pascal's Move statement.
4. Repoint ShowPtrs' pointers to the newly-hidden lines back into the virtual
screen. Step 3 copied the data in those lines from the visible refresh buffer
into their corresponding lines on the heap. Here we must now repoint the
pointers that used to point into the visible refresh buffer back into the
heap. This is done simply by assigning the corresponding pointers from
StorePtrs (which always point onto the heap) into their twins in ShowPtrs.
5. Glitch the visible display upward by the number of lines in the pan. It may
seem counterintuitive at first, but to slide the window downward you must
glitch the display upward. Think of it this way: When you pass a cow on the
Interstate, you're moving forward, but the cow appears to move backward. As
Einstein would say, Everything's relative. The glitch itself is done by a call
to the firmware scrolling routines in ROM BIOS video interrupt 10H.
6. Copy the newly-visible line or lines from the virtual screen on the heap
into the visible refresh buffer. Glitching the display leaves a blank line or
lines, which must be filled with lines from the virtual screen. This is the
mirror-image to step 2, and is done quickly with Turbo Pascal's Move
procedure.
7. Finally, update the TopLine counter in the virtual screen's descriptor
record. The top line shown in the visible display has changed, and TopLine
must always reflect the number of that top line.


To Virtualize or Not to Virtualize



I can hear the screams already: That sliding a window up and down all the time
is a monstrous violation of good ergonomic software design. And that's true
... but the key is the phrase "all the time." Simply having virtual screens
does not require that you use them in all cases. I might also add that you can
use them well or you can use them badly, just as with a paring knife or false
eyelashes.
A little cleverness can help a lot. A 66-line form might be designed in three
major portions: The top 25 lines as one portion, the next 18 lines as the
second section, and the remaining 23 lines as the third section. Why? A person
with a 25-line screen gets the top section at a glance, pans down to pick up
the middle 18 lines, and then pans again to get the final 23 lines. A person
with an EGA or VGA can read the top two sections (totalling 43 lines) at a
glance, with one pan down to see the last 23 lines.
A person with a Genius tube, of course, gets the whole meatball in one glance
and doesn't need to pan at all. If you have enough fields on a form to fill 66
lines, you're going to have to manage multiple forms for 25, 43, and 50 line
screens anyway. Designing a form in this fashion manages multiple forms for
you without any change in the software.
And then, once the form is completed in the virtual screen, you can print the
virtual screen to paper as a unit --again, without any additional fussing to
combine multiple forms onto one sheet for printing.
Virtual screens also make report previewing easy and convenient. Reports must
be formatted for paper, not for the screen, so if the user wants to preview a
printed report on the screen, he or she must be able to pan around the sheet
in order to inspect the whole sheet on a sub-66-line screen. My infamous
Little Black Book program JBOOK.PAS (which I will someday release as
shareware) prints address information on Day Runner-style memo book sheets,
many of which accept over 50 lines in the Laserjet II's 16.66 pitch Line
Printer font. Using SCREENS.PAS, JBOOK builds each sheet of the paper report
in a virtual screen, and then prints directly from the virtual screen.
Listing Two is the Turbo Pascal code for the virtual display management unit.
Listing Three, page 144, is a short demo program, SCRNTEST.PAS. SCRNTEST
allows you to load the first 66 lines of any text file into a virtual screen
and then pan through it. Pressing Del will clear the center line of the
visible display, which can then be scrolled onto and off of the screen as
proof that the screen really is virtual. (SCRNTEST cannot write to disk, so
your text file will not be corrupted.)
SCREENS.PAS is complete and usable as it stands, but much could be added to
it. I'll be presenting additional procedures for the unit in future columns.
These will include a box-drawing procedure, as well as code to allow the
creation of "ghost" screens stored entirely on the heap, and flashed in and
out of view with a little external assembly language magic. Also, from time to
time I'll present screen-oriented procedures showing how different user
interface concepts are implemented, and they'll be built on the virtual
screens platform shown here.
And if I can score the loan of a video board that displays more than 80
columns horizontally, I'll take a crack at adding horizontal panning to the
system. People who have display boards that support 132 column text, write to
me. Let me know what's good.


Off the Shelf Video Information


I've learned most of the information on text displays presented in these last
few columns the hard way, but more and more of it is being gathered up into
reference books specifically devoted to video topics.
Let me call attention to two recent titles that I've found very useful.
Programmer's Guide to PC and PS/2 Video Systems by Richard Wilton is the most
broad ranging, in that it covers the full spectrum of PC video, including the
CGA, MDA, Hercules, and something called the InColor card that I've never
heard of. Programmer's Guide to the EGA and VGA Cards by Richard F. Ferraro is
more focused, but covers its territory in greater depth. The books distinguish
themselves quite clearly in several ways. The Wilton book is a software book,
that contains tremendous quantities of C and assembler code, including what
looks like a fast graphics BITBLT in assembler. Sadly, the source is printed
in an eye-crossing pale green that suggests an intention to present source
code in 3-D, only they forgot to bind the little glasses into the book.
(Microsoft Press recognizes the error and will print it more clearly in future
press runs.) The Ferraro book is a hardware book, and tells you much more
about what goes on at a register level. Wilton presents his great quantities
of data in reference manual fashion, and doesn't provide as much in the line
of example code.
The two dovetail nicely with one another, and I consider both of them
essential. I'll let Kent pass on how useful they are to the graphics
programmer, but they have not failed so far to answer any questions I've had
on text video.


Carol of the BELs


I seem to be making a habit of finishing up these columns on major holidays.
(In the Santa Cruz area, Halloween is major.) A little later on this drippy
Christmas Eve, Carol and I will be packing off in the Magic Van to have a
quiet dinner with friends on the slopes of Loma Prieta.
A good night, I think, to shut the machines down and look for the best that is
human somewhere else. Give somebody who counts a hug, and it'll be right
there.
Promise.


Products Mentioned


Programmer's Guide to the PC and PS/2 Video Systems by Richard Wilton
Microsoft Press, 1987 ISBN 1-55615-103-9 531 pp. $24.95 Source code disk
$21.95
Programmer's Guide to the EGA and VGA Cards by Richard F. Ferraro
Addison-Wesley, 1988 ISBN 0-201-12692-3 606 pp. $26.95 Source code disk $24.99
Genius VHR Video Subsystems Micro Display Systems 1310 Vermillion Str.
Hastings, MN 55033 612-437-2233 Board and monitor $1,395


Errata


A bug has been discoverd in the listings for the March 1989 "Structured
Programming" column. On line 175 of TEXTINFO.PAS, (within procedure GetFont
Size) replace the line
 BL:= 0;
with the line
 BH:= 0;
The bug apparently affects VGA display boards only, and then only under
certain circumstances. Please update your copy of TEXTINFO.PAS accordingly.

_STRUCTURED PROGRAMMING COLUMN_
by Jeff Duntemann


[LISTING ONE]

Screen = RECORD
 ShowPtrs : ARRAY[1..HEIGHT] OF LinePtr;
 StorePtrs : ARRAY[1..HEIGHT] OF LinePtr;
 X,Y : Byte;
 TopLine : 1..HEIGHT;
 FollowCursor : Boolean
 END;






[LISTING TWO]

{--------------------------------------------------------------}
{ SCREENS }
{ Virtual screen management unit }
{ }
{ by Jeff Duntemann KI6RA }
{ Turbo Pascal 5.0 }
{ Last modified 12/24/88 }
{--------------------------------------------------------------}

UNIT Screens;

INTERFACE

USES DOS, { Standard Borland unit }
 TextInfo; { Given last issue; DDJ 3/89 }

CONST
 WIDTH = 80; { These are the character sizes of the virtual screens }
 HEIGHT = 66; { KEEP IN MIND THAT THIS IS A 1-ORIGIN SYSTEM!!!!!!!!! }
 { I.e., we count rows and columns from *1*, not 0. }
 UP = True; { Constants for glitching and panning }
 DOWN = False;


TYPE
 String5 = STRING[5];
 String10 = STRING[10];
 String80 = STRING[80];

 { Lines are made of these; helps us mix characters and attributes: }
 ScreenAtom = RECORD
 CASE Boolean OF
 True : (Ch : Char;
 Attr : Byte);
 False : (Atom : Word);
 END;

 LinePtr = ^Line;
 Line = ARRAY[1..WIDTH] OF ScreenAtom;

 ScreenPtr = ^Screen;
 Screen = RECORD
 ShowPtrs : ARRAY[1..HEIGHT] OF LinePtr;
 StorePtrs : ARRAY[1..HEIGHT] OF LinePtr;
 X,Y : Byte;
 TopLine : 1..HEIGHT;
 FollowCursor : Boolean
 END;

CONST
 ClearAtom : ScreenAtom = (Ch : ' '; { ASCII space char }
 Attr : $07); { "Normal" screen attribute }


VAR
 CurrentAttr : Byte; { Exported global, *not* a function! }


PROCEDURE ClearLine(LineTarget : LinePtr;
 VisibleX : Byte;
 ClearAtom : ScreenAtom);

INLINE
($58/ { POP AX } { Pop filler char/attribute into AX }
 $59/ { POP CX } { Pop line length (repeat count) into CX }
 $5F/ { POP ES } { Pop line address segment into ES }
 $07/ { POP DI } { Pop line address offset into DI }
 $8C/$C2/ { MOV DX,ES } { Move ES into DX for test against 0 }
 $81/$FA/0/0/ { CMP DX,0000 } { Compare ES value (in DX) against 0 }
 $74/$02/ { JE 2 } { If Equal, jump ahead 2 bytes }
 $F3/$AB); { REP STOSW } { Otherwise, blast that line to atoms! }


FUNCTION BooStr(BooleanValue : Boolean) : String5;
PROCEDURE ClrScreen(Target : ScreenPtr; ClearAtom : ScreenAtom);
PROCEDURE DisposeOfScreen(VAR Target : ScreenPtr);
PROCEDURE GotoXY(Target : ScreenPtr; NewX,NewY : Byte);
PROCEDURE InitScreen(Target : ScreenPtr; Visible : Boolean);
FUNCTION IntStr(IntegerValue,FieldWidth : Integer) : String10;
PROCEDURE Pan(Target : ScreenPtr; PanUp : Boolean; ByLines : Integer);
FUNCTION RealStr(RealValue : Real; Exponential : Boolean;
 FieldWidth,DecimalWidth : Integer) : String80;
PROCEDURE WriteTo(Target : ScreenPtr; S : String);
PROCEDURE WritelnTo(Target : ScreenPtr; S : String);


IMPLEMENTATION

{ Private to SCREENS--make it public if you need it. }

PROCEDURE GlitchDisplay(Up : Boolean; ByLines : Integer);

VAR
 Service : Byte;
 Regs : Registers;

BEGIN
 IF Up THEN Service := $06 ELSE Service := $07;
 WITH Regs DO
 BEGIN
 AH := Service;
 AL := ByLines;
 BH := CurrentAttr; { Attribute for blanked line(s) }
 CH := 0; { CX & DX: Glitch the full display }
 CL := 0;
 DH := VisibleY-1;
 DL := VisibleX-1;
 END;
 Intr($10,Regs);
END;


{ Returns string equivalent of RealValue: }


FUNCTION RealStr(RealValue : Real; Exponential : Boolean;
 FieldWidth,DecimalWidth : Integer) : String80;

VAR
 Dummy : String80;

BEGIN
 IF Exponential THEN
 Str(RealValue : FieldWidth,Dummy)
 ELSE
 Str(RealValue : FieldWidth : DecimalWidth,Dummy);
 RealStr := Dummy
END;


{ Returns string equivalent of BooleanValue: }

FUNCTION BooStr(BooleanValue : Boolean) : String5;

BEGIN
 IF BooleanValue THEN BooStr := 'TRUE'
 ELSE BooStr := 'FALSE'
END;


{ Returns string equivalent of IntegerValue: }

FUNCTION IntStr(IntegerValue,FieldWidth : Integer) : String10;

VAR
 Dummy : String10;

BEGIN
 Str(IntegerValue : FieldWidth,Dummy);
 IntStr := Dummy
END;


{ Clears Target to the atom passed in ClearAtom: }

PROCEDURE ClrScreen(Target : ScreenPtr; ClearAtom : ScreenAtom);

VAR
 I : Integer;

BEGIN
 WITH Target^ DO
 BEGIN
 { Brute force: Clear all lines at the ends of pointer }
 { referents, even though non-visible lines are cleared twice }
 FOR I := 1 TO HEIGHT DO
 ClearLine(ShowPtrs[I],VisibleX,ClearAtom);
 FOR I := 1 TO HEIGHT DO
 ClearLine(StorePtrs[I],VisibleX,ClearAtom);
 X := 1; Y := 1;
 END
END;



{ Moves logical (*not* hardware!) cursor to NewX,NewY: }

PROCEDURE GotoXY(Target : ScreenPtr; NewX,NewY : Byte);

{ Simply places new values in descriptor record's X & Y fields }
BEGIN
 WITH Target^ DO
 BEGIN
 X := NewX;
 Y := NewY
 END
END;


{ V-Screen equivalent of Write: }

PROCEDURE WriteTo(Target : ScreenPtr; S : String);

VAR
 I,K : Integer;
 TX : Byte;
 ShiftedAttr : Word;

BEGIN
 { Put attribute in the high byte of a word: }
 ShiftedAttr := CurrentAttr SHL 8;
 WITH Target^ DO
 BEGIN
 TX := X;
 K := 0;
 FOR I := 0 TO Length(S)-1 DO
 BEGIN
 IF X+I > VisibleX THEN { If string goes past end of line: }
 BEGIN
 Inc(Y); { Increment Y value }
 X := 1; TX := 1; { Reset X and temp X value to 1 }
 K := 0; { K is the line-offset counter }
 END;
 { Here we combine the character from the string and the }
 { current attribute via OR, and assign it to its location }
 { on the screen: }
 Word(ShowPtrs[Y]^[X+K]) := Word(S[I+1]) OR ShiftedAttr;
 Inc(TX); Inc(K);
 END;
 X := TX; { Update X value in descriptor record }
 END
END;


{ V-Screen equivalent of Writeln: }

PROCEDURE WritelnTo(Target : ScreenPtr; S : String);

BEGIN
 WriteTo(Target,S);
 Inc(Target^.Y); { These 2 lines are the equivalent of CR/LF }
 Target^.X := 1
END;



{ Moves the visible display as a window onto a full-page virtual screen: }

PROCEDURE Pan(Target : ScreenPtr; PanUp : Boolean; ByLines : Integer);

VAR
 I : Integer;
 YOffset : byte;

BEGIN
 YOffset := VisibleY-1; { Compensates for 1-based line numbering }
 WITH Target^ DO
 IF PanUp THEN { If we want to pan the display up the screen }
 BEGIN
 { Don't do anything if we're at the top of the V-screen: }
 IF TopLine > 1 THEN
 BEGIN
 { If we're not at the top but ByLines would take us out of }
 { legal range, adjust ByLines to scroll the rest of the way: }
 IF TopLine - ByLines < 1 THEN ByLines := TopLine - 1;
 { Move newly-hidden lines into virtual screen buffer: }
 FOR I := TopLine + YOffset DOWNTO
 TopLine + YOffset - (ByLines-1) DO
 Move(ShowPtrs[I]^,StorePtrs[I]^,VisibleX * 2);
 { Glitch the display pointer array up: }
 Move(ShowPtrs[TopLine],ShowPtrs[TopLine-ByLines],VisibleY * 4);
 { Repoint affected line pointers into virtual screen: }
 FOR I := TopLine + YOffset DOWNTO
 TopLine + YOffset - (ByLines-1) DO
 ShowPtrs[I] := StorePtrs[I];
 { Glitch the display buffer down: }
 GlitchDisplay(False,ByLines);
 { Update virtual screen's TopLine counter: }
 TopLine := TopLine - ByLines;
 { Move newly-visible lines to display from virtual screen: }
 FOR I := TopLine TO TopLine + (ByLines-1) DO
 Move(StorePtrs[I]^,ShowPtrs[I]^,VisibleX * 2);
 END
 END
 ELSE { If we want to pan the display down the screen }
 BEGIN
 { First check if the pan would take us out of legal line range: }
 IF TopLine + YOffset < Height THEN
 BEGIN
 { If we're not at bottom but ByLines would take us out of }
 { legal range, adjust ByLines to scroll the rest of the way: }
 IF TopLine + YOffset + ByLines > HEIGHT THEN
 ByLines := HEIGHT - (TopLine + YOffset);
 { Move newly-hidden lines into virtual screen buffer: }
 FOR I := TopLine TO TopLine + (ByLines-1) DO
 Move(ShowPtrs[I]^,StorePtrs[I]^,VisibleX * 2);
 { Glitch the display pointer array down: }
 Move(ShowPtrs[TopLine],ShowPtrs[TopLine+ByLines],VisibleY * 4);
 { Repoint affected line pointers into virtual screen: }
 FOR I := TopLine TO TopLine + (ByLines-1) DO
 ShowPtrs[I] := StorePtrs[I];
 { Glitch the display buffer up }
 GlitchDisplay(True,ByLines);

 { Move newly-visible lines to display from virtual screen: }
 FOR I := TopLine + VisibleY TO TopLine + VisibleY + (ByLines-1) DO
 Move(StorePtrs[I]^,ShowPtrs[I]^,VisibleX * 2);
 { And finally, update virtual screen's TopLine counter: }
 TopLine := TopLine + ByLines
 END
 END
END;


{ You *must* init a V-Screen through this proc before using it: }

PROCEDURE InitScreen(Target : ScreenPtr; Visible : Boolean);

VAR
 I : Integer;

BEGIN
 WITH Target^ DO
 BEGIN
 FOR I := 1 TO HEIGHT DO
 BEGIN
 New(ShowPtrs[I]); { Allocate a line on the heap }
 StorePtrs[I] := ShowPtrs[I] { Duplicate pointer }
 END;
 X := 1;
 Y := 1;
 TopLine := 1;
 FollowCursor := True;
 IF Visible THEN { As opposed to a "ghost" screen on the heap }
 FOR I := 0 TO VisibleY-1 DO
 ShowPtrs[I+1] := { Repoint pointers into refresh buffer }
 Ptr(Seg(TextBufferOrigin^),
 Ofs(TextBufferOrigin^) + (I * (VisibleX * 2)))
 END
END;


{ Frees up heapspace occupied by Target. DON'T use if Target is the }
{ address of a statically declared-record obtained with @ or Addr()!! }

PROCEDURE DisposeOfScreen(VAR Target : ScreenPtr);

VAR
 I : Integer;

BEGIN
 FOR I := 1 TO Height DO Dispose(Target^.ShowPtrs[I]);
 Dispose(Target);
 Target := NIL
END;


{ SCREENS Initialization Section: }

BEGIN
 CurrentAttr := $07; { $07 is the "normal" video attribute }
END.






[LISTING THREE]


{--------------------------------------------------------------}
{ SCREENTEST }
{ Virtual screen demo program }
{ }
{ by Jeff Duntemann KI6RA }
{ Turbo Pascal 5.0 }
{ Last modified 12/24/88 }
{--------------------------------------------------------------}

PROGRAM ScreenTest;

USES DOS, { Standard Borland unit }
 TextInfo, { Given last issue; DDJ 3/89 }
 Screens; { Given this issue; DDJ 4/89 }

CONST
 PanBy = 1; { Specifies # of lines to pan at once }

VAR
 I : Integer;
 Check : Integer;
 Ch : Char;
 Extended : Boolean;
 Scancode : Byte;
 Shifts : Byte;
 TestScreen : Screen;
 MyScreen : ScreenPtr;
 FileName : String80;
 TestFile : Text;
 HalftoneAtom : ScreenAtom;
 InString : String80;


{->>>>GetKey<<<<-----------------------------------------------}
{ }
{ Filename: GETKEY.SRC -- Last modified 7/23/88 }
{ }
{ This routine uses ROM BIOS services to test for the presence }
{ of a character waiting in the keyboard buffer and, if one is }
{ waiting, return it. The function itself returns a TRUE }
{ if a character has been read. The character is returned in }
{ Ch. If the key pressed was a "special" (non-ASCII) key, the }
{ Boolean variable Extended will be set to TRUE and the scan }
{ code of the special key will be returned in Scan. In }
{ addition, GETKEY returns shift status each time it is called }
{ regardless of whether or not a character was read. Shift }
{ status is returned as eight flag bits in byte Shifts, }
{ according to the bitmap below: }
{ }
{ BITS }
{ 7 6 5 4 3 2 1 0 }
{ 1 . . . . . . . INSERT (1=Active) }

{ . 1 . . . . . . CAPS LOCK (1=Active) }
{ . . 1 . . . . . NUM LOCK (1=Active) }
{ . . . 1 . . . . SCROLL LOCK (1=Active) }
{ . . . . 1 . . . ALT (1=Depressed) }
{ . . . . . 1 . . CTRL (1=Depressed) }
{ . . . . . . 1 . LEFT SHIFT (1=Depressed) }
{ . . . . . . . 1 RIGHT SHIFT (1=Depressed) }
{ }
{ Test for individual bits using masks and the AND operator: }
{ }
{ IF (Shifts AND $0A) = $0A THEN CtrlAndAltArePressed; }
{ }
{ From: COMPLETE TURBO PASCAL 5.0 by Jeff Duntemann }
{ Scott, Foresman & Co., Inc. 1988 ISBN 0-673-38355-5 }
{--------------------------------------------------------------}

FUNCTION GetKey(VAR Ch : Char;
 VAR Extended : Boolean;
 VAR Scan : Byte;
 Var Shifts : Byte) : Boolean;

VAR Regs : Registers;
 Ready : Boolean;

BEGIN
 Extended := False; Scan := 0;
 Regs.AH := $01; { AH=1: Check for keystroke }
 Intr($16,Regs); { Interrupt $16: Keyboard services}
 Ready := (Regs.Flags AND $40) = 0;
 IF Ready THEN
 BEGIN
 Regs.AH := 0; { Char is ready; go read it... }
 Intr($16,Regs); { ...using AH = 0: Read Char }
 Ch := Chr(Regs.AL); { The char is returned in AL }
 Scan := Regs.AH; { ...and scan code in AH. }
 IF Ch = Chr(0) THEN Extended := True ELSE Extended := False;
 END;
 Regs.AH := $02; { AH=2: Get shift/alt/ctrl status }
 Intr($16,Regs);
 Shifts := Regs.AL;
 GetKey := Ready
END;



BEGIN
 IF ParamCount < 1 THEN { No file-ee, no work-ee }
 BEGIN
 Writeln('>>>SCRNTEST by Jeff Duntemann ');
 Writeln(' Virtual screen demo program');
 Writeln(' Version of 12/24/88 -- Turbo Pascal 5.0');
 Writeln(' Invoke: SCRNTEST <textfile> <CR>');
 Writeln(' Use up/down arrows to pan window;');
 Writeln(' the DEL key to blank out a line.');
 Writeln(' Press "Q" or "q" to quit...');
 END
 ELSE
 BEGIN
 FileName := ParamStr(1); { See if named file can be opened }

 Assign(TestFile,FileName);
 {$I-} Reset(TestFile); {$I+}
 Check := IOResult;
 IF Check <> 0 THEN { If not, complain: }
 BEGIN
 Writeln('>>Test file ',FileName,' Cannot be opened.');
 Writeln(' Please invoke again with a valid file name.');
 END
 ELSE
 BEGIN { File can be opened; let's read it into a V-screen }
 HalftoneAtom.Ch := Chr(177); HalftoneAtom.Attr := $07;
 MyScreen := @TestScreen;
 InitScreen(MyScreen,True); { Allocate & init the screen }
 ClrScreen(MyScreen,ClearAtom); { Clear the screen }

 IF NOT EOF(TestFile) THEN { If the file isn't empty... }
 BEGIN
 I := 1; { Start from line 1 }
 WHILE (NOT EOF(TestFile)) AND (I <= HEIGHT) DO
 BEGIN
 Readln(TestFile,InString);
 { Truncate each line at 70 columns: }
 InString := Copy(InString,1,70);
 { Write line number to the V-Screen: }
 WriteTo(MyScreen,IntStr(I,5));
 { Write the data line to the V-Screen: }
 WritelnTo(MyScreen,': '+InString);
 Inc(I) { Increment the line counter }
 END;

 { Up to 66 lines of the file are on the screen. }
 { Here we pan up on the up arrow, and down on }
 { the down arrow. 'Q' quits the program. }
 Extended := False;
 REPEAT
 IF Extended THEN
 CASE Scancode OF
 { DEL } 83 : WITH MyScreen^ DO
 ClearLine(ShowPtrs[TopLine + (VisibleY DIV 2)],
 VisibleX,HalftoneAtom);
 { Up Arrow } 72 : Pan(MyScreen,Up,PanBy);
 { Down arrow } 80 : Pan(MyScreen,Down,PanBy);
 END; { CASE }
 REPEAT UNTIL GetKey(Ch,Extended,Scancode,Shifts);
 UNTIL Ch IN ['Q','q'];
 END
 END
 END
END.













April, 1989
OF INTEREST





Database Applications has announced a business and scientific graphics product
called dR&G, which is driven by 4GL statements. This software package combines
graphics with tabular reporting, mail merge, and SQL-like queries of
relational databases.
Chart styles include bar, pie, line, area filled, histogram and scatter, as
well as fitted curves with forward or backward moving averages, regression
lines, and exponential/logarithmic curves with prediction and confidence
limits displayed. Up to six data sets may be plotted at once, with automatic
scaling to fit the chart in vertical and horizontal formats.
Charts and tabular reports are specified as 4GL statements, as in "Graph as
Bars, Sales by Day Across Region," which yields multilevel bars summarizing
daily transaction data from a dBase file or 16 related dBase files. 4GL
statements are stored in one or many program files and may be called by name
or from a menu, or they can be entered as ad hoc queries.
dR&G produces charts, linear regression statistics for up to six dependent
variables, new template files, tabular data reports, cross-table matrix
reports, custom format reports, letter reports, mail-merge, and mailing
labels, as well as program files to reproduce all of the above. The user has
control over chart labelling, the treatment of missing data values, the color
palette, the page size and position, and the use of various type fonts. 33
hatch and line patterns, up to 128 colors, and over 30 type styles are
supported.
dR&G integrates G-RUN software of Mightysoft Corp. of Seattle, Wash., with
nonprocedural language (NPL) of the dNPL/Reporter report writer to provide
access of 16 related databases, sorting, summarizing, computation, and chart
plotting.
dR&G is priced at $179; the upgrade prepaid price for registered dNPL/Reporter
users is $89. The company will also upgrade NPL/R database application systems
with dR&G capability at the same upgrade price. Reader Service No. 26.
Database Applications Inc. 400 Wall St. Princeton, NJ 08540 609-924-2900
Glockenspiel's C+ + 1.2, the AT&T designed superset of ANSI C, will be
released by Imagesoft. Encapsulation, one of the product's features, helps
users develop pretested invulnerable classes. C+ + 1.2 also features reusing
stable code --inheritance helps users reuse their pretested classes in new
circumstances.
Also included is type checking, an optimized code generator in Microsoft C 5.x
that supports mixed model keywords, a CL-compatible seamless compilation
command, and support for both real and protected mode development and delivery
environments on OS/2.
Glockenspiel C+ + is grammar compatible with designer C+ + on Unix and VMS, as
well as Domain C+ + on Apollo; it also implements the standard AT&T 1.2
grammar for C+ +.
Containing over 600 pages of documentation that covers topics such as mixed
language programming, implementing dynalink libraries in C+ +, debugging with
CodeView, and mixed model development, Glockenspiel C+ + sells for $3,995.
Reader Service No. 27.
Imagesoft 6-57 158th St. Beechhurst, NY 11357 718-746-9069
Lattice has released Version 3.4 of the Lattice C compiler for DOS and OS/2.
This latest upgrade includes CodeProbe, a full-screen, source-level debugger
and also features an integrated editor, linker, and librarian.
Lattice's debugger can be used with a mouse and offers three ways to display
source code. A high-level view shows only the C source lines, a low-level view
shows only the assembler code, and the mixed view shows both. Depending on
which view is used, breakpoints and other program references can be made by
way of source line numbers and either absolute or relative addresses.
In Version 3.4, CodeProbe operates under DOS or OS/2 real mode. Support is
provided for debugging family-mode programs that use the OS/2 EXE format but
run under DOS. Registered users receive a free upgrade, which extends
CodeProbe to OS/2 protected mode. The upgraded version supports multithread
and multiprocess debugging and includes features for testing Presentation
Manager programs.
Lattice C for DOS and OS/2, Version 3.4, sells for $450. The package includes
the compiler, linker, CodeProbe and C-SPRITE debuggers, integrated LSE editor,
bind utility, object module librarian, disassembler, API, graphics, standard
Lattice C libraries, and documentation. Registered users of Version 3.3
receive Version 3.4 at no charge; users of earlier versions may upgrade for
$75. Reader Service No. 28.
Lattice Inc. 2500 S Highland Ave. Lombard, IL 60148 312-916-1600
Softaid has introduced a version of MTBASIC Multitasking Basic Compiler
targeted to Intel's Wildcard 88. MTBASIC is an interactive compiler designed
to be included in ROM on systems designed around the Wildcard.
MTBASIC includes built-in multitasking that supports up to ten concurrent
tasks; a windowing environment is also supplied. MTBASIC for the Wildcard is
provided as a set of source codes so that it can be customized to suit the
OEM's requirements. The compiler is burned into a ROM that resides on the
Wildcard.
MTBASIC is designed for the diskless, ROM-based equipment typically designed
with the Wildcard. Selling for $6,500, this product includes source code.
Reader Service No. 29.
Softaid Inc. 8930 Rte. 108 Columbia, MD 21045 800-433-8812
The DOS version (2.1) of the True Basic Language System includes VGA graphics
support, new manuals, trace utility, and compatibility between machine types
supported by True Basic (IBM, Macintosh, Amiga, and Atari). Current users can
upgrade for $29.95.
A Unix version, targeted to the higher-education and government markets, will
be released during the fourth quarter of this year. Reader Service No. 30.
True Basic Inc. 45 Theodore Fremd Ave. Rye, NY 10580 914-921-1630 800-TRBASIC
Version 3.0 of the PC-MOS multiuser DOS-compatible operating system from The
Software Link (TSL) has been announced. The new release supports Windows/286,
bringing multiuser capabilities to the windows environment through devices
that support highspeed, bit-mapped graphics, such as TSL's VNA and SunRiver's
Cygna/386 workstation.
Release 3.0 hardware support additions include DCA's Irma2 and the Intel
Inboard 386 and 386/PC. Also added are support of Compaq's 386/20e, the IBM
PS/2 mouse, and Everex's Quick 02 controller card. Support for several tape
backup systems --such as Priam, Everex's Excell Streaming 60 Tape Drive, the
Wangtek tape backup system, and the Techmar 60E for the IBM PS/2 --has also
been added.
Software compatibility includes DataFlex 2.3, Clarion Professional Developer,
Wordperfect 5.0, dBaseIV 1.0, Excel 2.0, Sidekick Plus 1.0, ACCPAC Plus
Accounting 5.0, Q&A 2.0, and Advanced Revelation 1.0.
PC-MOS includes an option for delayed write caching; users also can define
cache sizes and determine how often read requests are written to the disk.
Also, PC-MOS allows the PC-MOS kernel to be relocated to noncontiguous areas
of upper memory.
This product includes support for TSR programs such as Sidekick and SideKick
Plus. Spanish and German keyboard drivers and language translations are
supported. PC-MOS, Version 3.0, sells for $195 (single user), $595 (five
user), and $995 (25 user). Reader Service No. 31.
The Software Link 3577 Parkway Lane Norcross, GA 30092 404-448-5465
Howard W. Sams & Co. has released three new books: The Waite Group's OS/2
Programmer's Reference by Asael Dror, The Waite Group's Turbo C Bible by Naba
Barkakati, and Turbo C Programming for the PC, Revised Edition by Robert
Lafore.
The Waite Group's OS/2 Programmer's Reference is a debugged reference to the
set of over 200 OS/2 API function calls. This distillation of the OS/2
Software Developer's Kit is divided into categories, each with its own
tutorial. Within each category, functions are presented alphabetically, each
with its purpose, syntax, and example, including explanations of error codes,
uses, cautions, and references to related APIs. Book no. 22645, it is 848
pages and sells for $24.95.
For programmers using Borland's Turbo C compiler, The Waite Group's Turbo C
Bible includes tutorials and explanations to point out the different purposes
and uses of each function. Compatibility check boxes show portability with
Microsoft C, Versions 3.0, 4.0, and 5.0; Microsoft QuickC; and the Unix System
V compilers. This book, no. 22631, sells for $24.95.
Turbo C Programming for the PC, Revised Edition includes graphics and debugger
features. This book moves through the fundamentals of the language with a
structured step-by-step approach. Expanded to include Version 2.0 (but still
compatible with Turbo C 1.0 and 1.5), it covers the graphics library, graphics
modes, the Debugging Tracer, and it highlights ANSI C features. This book
sells for $22.95 and is no. 22660. Reader Service No. 32.
Howard W. Sams & Co. 4300 W 62nd St. Indianapolis, IN 46268 800-428-7267
Oregon Software has released Version 1.2a of Oregon C+ + for the Sun-3. This
update is highlighted by the introduction of a mouse-driven, window interface
to the C+ + source-level debugger.
The window interface consists of five windows: a status window that displays
file name, line number, function name, and scope information; a source window
that displays the source text; a button-panel window that allows mouse-click
access to the debugger's commands; a dialog window for the debugger's I/0; and
an application window for the application's I/0. The separate windows for
application and debugger I/O mean that the graphical layout of an
application's I/0 is not scrambled by the debugger's I/0. The source, dialog,
and application windows are scrolling.
Oregon C+ +, Version 1.2a, sells for $1,900 for a single-user license, $4,750
for 1 - 6 node network on a single file server, and $12,000 for an unlimited
node network on a one file server. Reader Service No. 36.
Oregon Software 6915 SW Macadam Ave. Portland, OR 97219-2397 503-245-2202















April, 1989
SWAINE'S FLAMES


Words & Figures, and a Programmer's Word Processor




Michael Swaine


Puzzle: Who said the following, when, and where? Hint: It was not Ted Nelson
in "Those Unforgettable Next Two Years" at the Second West Coast Computer
Faire in San Jose in 1977. The answer will be revealed at the bottom of the
page. Don't peek.
"Information technology represents a radical discontinuity in industrial
history.... It means changing your basic assumptions about what business
organizations are supposed to look like.
"For instance, you have to abandon the assumption that managers are different
from the people they manage. One of the things that made them appear different
in the past was that they had information that the people they managed didn't
have. Managers, we said, were the people uniquely equipped to deal with
information, and for that reason we granted them authority.... If you used to
think that what separated managers from workers was information, you have to
abandon that assumption. Workers will have information too."
Shortly after I transmitted last month's column in which I waxed wroth over
misuses of words and figures in the computer press, Jon Erickson showed me the
cover of the January 17, 1989 issue of PC Magazine, with its award for
technichal (sic) excellence. Then MacUser features editor, John Anderson,
passed me a copy of the January 1 New York Times Book Review section, with a
delightfully relevant article by John Allen Paulos.
Paulos decries, as I did, American innumeracy, the analog to illiteracy in the
realm of numbers. He points out that most people have no real feeling for the
difference between a million and a trillion, and lack the simple skills
necessary to assess risks to life, limb, and freedom. (Canceling a European
trip for fear of terrorists is silly; the expected value of the payoff in a
lottery is always less than the cost of a ticket; even for very accurate
tests, mandatory AIDS testing would probably produce vastly more false
positives than correct identifications.)
It matters that the public not be quite such numerically numbskulls, Paulos
claims, because innumeracy makes people suckers for simple scams and makes
them poor citizens in a world where numbers count. I decided to discuss the
subject on this page from time to time because Dr. Dobb's readers are as good
a group as I can imagine to take the message forth, and even they (you, [we])
need an occasional reminder. Even a generally numerate book such as Alexander
Hellemans and Bryan H. Bunch's The Timetables of Science (Simon and Schuster,
1988) can slip up: "correct to n decimal places" means rounded to n places,
not truncated at the nth place. The expression "n times more" has been so
thoroughly corrupted to mean "n times as" that it's probably unsalvageable.
Paulos's forthcoming book, Innumeracy: Mathematical Illiteracy and Its
Consequences, should be one such reminder. So are Darrell Huff's classic
little How to Lie with Statistics (W.W. Norton & Co., 1954) and The Mismeasure
of Man by Stephen Jay Gould, an impressive debunking of one of the hoariest
and most malignant of statistical fictions, IQ.
Paragon's QUED/M, a text editor for the Macintosh, is highly regarded by Mac
programmers. Now Paragon has introduced a full word processor based on QUED/M,
and as soon as I saw the feature list, I requested the product. Still a sucker
for a well-written press release, I guess.
The features of Nisus, though, look like Paragon has been peeking at my word
processor feature wish list. The search-and-replace feature is a GREP that
knows about fonts and styles and even graphics. You can draw directly into the
text, overlap text and pictures, and wrap text around art, and can place
pictures in headers or footers (of which you can have any number). Like
QUED/M, Nisus lets you record, write, and edit macros; and gives you
essentially unlimited undoing of almost any changes.
Nisus has a modeless catalog function for opening files, searching through
unopened files, and other file operations. Paragon claims that Nisus can
locate a word or phrase in unopened files faster than other word processors
can search one open document. It'll balance quotation marks and parentheses
and add line numbers on screen, and will compare files.
It seems to me that anybody who writes code and English on a Macintosh would
be intrigued by what Paragon claims for Nisus, and as soon as I've worked with
the product, I'll give you my report on how well Paragon delivered.
Answer to puzzle: Harvard Business School Associate Professor Shoshana Zuboff,
in Inc. magazine January, 1989. Computer Lib comes to HBS. What on earth does
Zuboff think could drive managers to lose their grip on the manager's edge --
information?
Competition. Companies that fail to put the information in the hands of those
who carry out policy will not be able to compete with those that do.




































May, 1989
May, 1989
EDITORIAL


The Open Editorial




Jonathan Erickson


A few weeks back, Mike, Kent, and I spent a couple of days at Microsoft's
annual systems seminar hearing about the future as Microsoft sees it. It
should come as little surprise that MS sees OS/2 as part of everyone's future.
Actually, I feel better about OS/2 and its chances of survivability after
hearing some of MS's strategies. Nevertheless, as I've said before, if OS/2 is
to be accepted, then MS needs to get out the 80386 version that will provide
access to the 32-bit linear address space, allow multiple DOS sessions,
include the high-performance file system, and more. But it isn't a trivial
task to rewrite the several hundred thousand lines of assembly code that make
up OS/2. Even though MS says this is one of their top priorities, it bothered
me that we didn't see a technology demonstration of OS/2-386 at the seminar
(MS is generally open about technology demos when they are making progress)
and it makes me nervous about MS hitting their mid 1990 target for getting the
386 version out to market.
I was also interested in hearing more about what Microsoft intends on doing
with object-oriented technology. When the words "C++ compiler" flashed on the
overhead screen, I thought that we might be onto something. However, it turned
out they weren't too specific on their plans because, it seemed, they had'nt
really made up their mind yet. The internal debate seems to concern whether or
not they should put out a stand-alone C++ compiler or add C++ extensions to
existing C compilers. As best as I could tell, the current preference is to
add extensions to the Microsoft C compiler sometime before the end of this
year. But C++ isn't the only object-oriented path MS will be pursuing. They
also talked about a "Visual" Basic which, among other things, will let people
write graphic user-interface applications without having to do a lot of
coding.
One thing that I found particularly interesting is that long-time language
guru Greg Whitten, who has worked with all language implementations in the MS
systems group over the past 10 years, recently moved over to the application
development side of the company. His mission there seems to be two-fold: to
direct the development of new object-oriented applications and to apply
object-oriented technology to the applications development. According to
Whitten, Microsoft's various applications make up several million lines of
code and the object-oriented paradigm is one way for MS to keep things under
control and on schedule. It's worth noting that the recent dramatic drop in MS
earnings was attributed to the inability to get application products out in
time, and they apparently believe 0OPs can prevent this in the future.
As if two days of OS/2 wasn't enough, the UniForum conference, which followed
close on the heels of the MS seminar, left me with several impressions: 1. The
Unix market is as fragmented as ever. 2. The widespread acceptance of the X
Window protocol is one of the few consistent things that ran throughout the
show; and 3. I'm tired of consortiums and "open" anything. Let's see ...
there's the 88open consortium, the SPARC International consortium, the OSF
consortium, and the Unix International consortium. Then there's Open Look,
Open Systems, Open Desktop, Open Font, and X Open.
Here's an idea. Someone should go out and get the rights to the name Open
Window because, considering the proliferation of windowing systems, some
outfit is bound to come out with a product by this name and if you have the
rights to the name, you might be able to make a couple of bucks. But then,
somebody has probably already tied it up anyway.
Actually, the real future of the prefix "open" may be in vertical market
applications. How about a package for dentists called Open Mouth, or a program
for duck hunters called Open Season. Or maybe even a product for trouser
manufacturers called Open Fly. Hey, will someone please open the door so I can
get out of here.










































May, 1989
LETTERS







SEI Reactions


Dear DDJ,
I read with interest your editorial "Swaine's Flames" in the November 1988
issue of Dr. Dobb's Journal. I regret that your experience has given you a
negative impression of the Software Engineering Institute (SEI) and would like
to explain that situation from my perspective.
According to your account, the SEI did not meet a commitment to produce an
article for your September issue. As best as I can trace your telephone
odyssey through the SEI, your request for an article was initially turned down
because of the prior commitments of those staff members who could have either
written the article or committed resources to having it written.
Unfortunately, the person who finally accepted your request thought he was
making a personal commitment, but you thought you were getting an SEI
commitment. We did not perceive that problem until a draft of the article went
to review. This is where SEI communication failed, and we apologize. Reviewers
found the draft incomplete -- not ready for publication without significant
effort. Meanwhile, your deadline was rapidly approaching.
At this point, one of our program directors called to let you know that the
draft did not meet our standards and that we could not revise it in time to
meet your deadline. In later conversations with you, SEI staff agreed to
produce an article sometime in the future but made no specific commitment, or
so they thought. Obviously, you felt otherwise.
I am surprised by the inference you draw from this experience about software
engineering and bureaucratic delay. I see the situation quite differently. An
important characteristic of a successful software engineering organization is
that it meets commitments with quality results on time and within budget. At
the SEI, we take commitments seriously: We don't make one unless we intend to
honor it. Ironically, this caution in making commitments is what prompted
staff members to turn down your request in the first place.
We undoubtedly have made and will in the future make mistakes at the SEI. I
regret that this instance of miscommunication left you with a poor opinion of
us. If you are still interested in publishing an article from the SEI, please
call.
Larry E. Druffel
SEI Director
Pittsburgh, Penn.
Jon responds: Thanks for the offer Larry and, yes, I'd still like to see the
article. Believe it or not, I do think the SEI serves a useful function and
that many fine people are associated with it. I wish the organization well.
However, I stand by my description of my experiences with the SEI, experiences
that were substantiated by readers who have also had to deal with the
organization's bureaucracy.
Dear DDJ,
I was entertained by your page in the November 1988 Dr. Dobb's ("Swaine's
Flames" by Jon Erickson) about your difficulties with the Software Engineering
Institute (SEI). I don't know much about the SEI but, yes, the government
software world is that bad. I've been dealing with it for 25 years. It's
always bad, and it's been getting worse lately. The last project I worked on
had maybe five technical people trying to modify code, 10 or 15 people trying
to test it, and maybe 30 people running around to meetings and stumbling over
each other making decisions whose quality is best left to the imagination.
Aside from an abnormally large number of test people -- which for reasons too
complicated to go into here made some sense -- this show is not atypical.
If you'd like some real entertainment, obtain a copy of DOD standard 2167A,
which describes how software is to be developed for mission critical software.
This gem and the descriptions of the associated documentation include
virtually every idea (good or bad) ever espoused about the software production
process. Now try to imagine what will happen when a bureaucracy gets a hold of
it.
As for software engineering: Are you sure it exists? I'd submit that the
difference between engineers and tinkers (used in the original non-pejorative
sense) is that engineers know pretty much what should happen when they fire
things up, whereas tinkers know what they hope will happen. Engineers
understand why what they are building/fixing works, whereas tinkers understand
what to do to make things work a lot of the time. From what I've seen, most
"software engineering" is really software tinkering. To pick just one example,
the people I've seen "engineering" real-time software rarely seem to
understand the difference between synchronous and asynchronous systems, and do
not understand enough about queuing systems to predit the performance of
asynchronous systems. Not surprisingly, the performance characteristics of the
systems they build often comes as an unpleasant surprise to them.
It seems to me that a process that can't predict size, performance, or cost
with any precision isn't engineering. Don't get me wrong: Tinkering is an
honorable art and a necessary one. But there are a lot of situations where I'd
prefer a little more determinism in the results.
It particularly disturbs me that I don't know where to send someone who
honestly wants to learn about software engineering. For algorithms, I can send
them to Knuth's books. For information on testing, I can send them to Boris
Beizer's excellent books. But where can I send someone for information on how
to design a system that will work and how to predict how well it will work? I
missed your SE issue. I'll dig up a copy and read it, and I'm ordering a copy
of your SE Source-book. Perhaps it will help, but I'm not optimistic.
Donald Kenney
San Diego, Calif.


XREF Examination


Dear DDJ,
Kent Porter's December 1988 "Structured Programming" column, with the XREF
program, was entertaining and instructive, as his articles usually are. Some
of the material calls for a little further discussion.
All line number references are to Kent's Listing Three.
1. In describing the "binary B+ tree" as a combination of a binary tree and a
B+ tree, the article may have confused data with links. The structure appears
to me to be a simple binary tree with nodes containing, among other data,
pointers to singly linked lists of line numbers. The fact that data points to
a linked list in no way affects the nature of the tree.
2. XREF's opening comments declare that the program "uses binary trees and
doubly-linked lists to effect B-Tree." Neither the article nor the program
deals with doubly-linked lists or B-Trees. I often use such comments to
describe what I'm going to do, then do something different. One of Al Steven's
recent crotchets addresses that type of program comment.
3. Because XREF operates on Modula-2 as well as Pascal files, line 137 should
test for DQuote as well as Quote, and line 139 should set ScanChar : = ch. The
program logic appears to successfully prevent ch from containing DQuote when
processing a syntactically correct Pascal file, so Kent would be justified in
leaving PComment as declared in line 35, rather than manipulating it in
response to a command line switch for file type.
4. If Token (line 119) finds an opening comment or quotation marker, it calls
procedure FindEndOfComment (line 48). FindEndOfComment then searches for a
matching closing comment or quotation marker. If it fails to find the matching
marker, it will read beyond the end of the file.
5. If the tests in lines 268 through 282 are to "weed out nuisances" reliably,
they should be made against UpShift(Symbol) rather than just Symbol. As
written, they fail to weed out While, at line 323 when making a case-sensitive
listing.
6. When making a case-sensitive listing, XREF avoids placing the nodes in
their final order until just before making the report. A call to the
Alphabetize procedure (line 342) then builds a new tree, disposing of the old
one as it goes. Kent is to be commended for making his case sensitive ordering
follow a human collating sequence, but this approach seems redundant. I am
tempted to build the tree in its final order on the first pass.
The tree only needs be built once if the symbols are compared according to the
case sensitivity flag rather just testing "if s1 > s2," and so forth. Comp,
shown in my Listing will do the job. My Listing also contains BNode, modified
to use Comp.
Building the tree in a single pass, the Alphabetize procedure (line 342) can
be eliminated, and a few other places can be simplified. But does this
approach really accomplish anything useful? Maybe, maybe not.
Results from sample runs of the original and modified XREF programs, using the
original XREF.PAS as an input file, are in Table 1. The modified version
performs fewer string comparisons than the original version does: 52 percent
fewer without case sensitivity, and 70 percent fewer with case sensitivity.
The modified version performs as expected regarding node creation. The
original version, however, makes 47 percent fewer calls to the UpShift
function in the case-sensitive trial.
Table 1: Sample runs of the original and modified XREF programs

 XREF Case String{1} Calls to Calls to{2}
 Version Sensitivity Compares NewNode UpShift
 -------------------------------------------------------

 Original insensitive 11,660 143 1,462
 Modified insensitive 5,560 143 1,462
 Original sensitive 18,921 296 443
 Modified sensitive 5,638 148 831



1. "String Compares" is a count of the number of times that symbols are
compared for for equality or inequality. 2. The modifications made some of the
original calls to UpShift pointless. They were removed from the modified
version and are not included in this table.
Clearly, getting reliable benefit from the modifications would require
finetuning UpShift. Then I would fine-tune Comp. Then I would ... well,
instead of considering changes to dealing with case sensitivity, I am
considering details of optimization -- not the present point. The most I can
say about the benefits of the modifications is that they justify further
study.
As usual, Kent has provided material to stimulate his readers' thinking. My
thanks to him and to DDJ for that. Please keep it up!
Karl Brendel
Hutchinson, Kansas


Columnist's Comedy


Dear DDJ,
I bought the December 1988 issue of DDJ at a software store and was delighted
with Al Stevens' column ("C Programming"). It's refreshing to see his droll
treatment of the C language's imaginary portability and Intel 8X86
architecture in the "C Tool Set Errata," and then his sarcastic declarations
that C is "language of choice," to be "the best programming language ever
devised," and that he is "C purist." Naming a word processor "twerp" is cute
too: What does it really do? Play the Mickey Mouse Club theme song with a
brightly colored screen and bouncing ball?
So many microcomputer magazines and columnists seem to be overly serious.
Thanks for bringing a bit of humor to this often drab and tedious business.
Jim Miller
Nashua, New Hampshire


"Find That Function" Revisited


Dear DDJ,
It made me smile to read all of the "mine-is-better-than-yours" letters
regarding "Find that Function!" So far we have been shown "better" ways of
doing this task using AWK, GREP, LEX, and C. All of the examples showed
improvement on the original, given one assumption. The programmer must know
AWK, GREP, or LEX.
Perhaps my mind is not as powerful as those of some of the AWK wizards in the
world, but it would take me much longer to do this job in AWK than in C
because I don't know AWK.
The list of helpful programmer's tools is long, and it grows daily. These
tools can be fantastically helpful for tasks such as "Find That Function!" but
most programmers don't have the time necessary to get over the learning curve
of such tools. The only programmers who know them all are the gurus whose jobs
are not to write code but to know stuff.
If I took the time to learn every useful tool that is available to me, I would
be much less productive than I am today. This is because I would spend a
significant portion of every day piddling with the latest tool that came
across my development platform.
It is up to every one of us to determine where to draw the line on
"productivity" tools. I prefer to invest my time in management, database
management, and communications. I am able to save myself weeks and even months
by learning and using tools such as these.
Many of the other productivity tools seem to me to be little more than typing
tools. And because I type well, I don't spend much time using this type of
tool. A good editor, a source code control system, and my trusty Norton
Utilities do everything I expect out of my typing tools. Perhaps I would spend
a minute or two less a day if I knew AWK, LEX, GREP, YACC, and so forth, but
it is a long return on investment for the time I must put in to learn them.
My "mine-is-better-than-yours" solution to the "Find That Function!" problem
is to write programmer's documentation that includes the source file for the
function. Then when I need to find the function, I just RTFM (Read The [Darn]
Manual). When that won't work, I use a simple text search utility and my
eyeballs.
Tim Berens
Dayton, Ohio

_Letters to the Editor_


[LISTING ONE, FROM KARL BRENDEL]



TYPE CompType = (LessThan,Equal,GreaterThan);

FUNCTION Comp(VAR s1, s2, sh1, sh2 : SymString;
 CaseSen : BOOLEAN) : CompType;
 { sh1/sh2 are UpShift(s1/s2) }
BEGIN
 IF CaseSen THEN BEGIN
 IF sh1 < sh2 THEN
 Comp := LessThan
 ELSE IF sh1 > sh2 THEN
 Comp := GreaterThan
 ELSE IF s1 < s2 THEN
 Comp := LessThan
 ELSE IF s1 > s2 THEN
 Comp := GreaterThan
 ELSE
 Comp := Equal
 END
 ELSE BEGIN {not CaseSen}

 IF sh1 < sh2 THEN
 Comp := LessThan
 ELSE IF sh1 > sh2 THEN
 Comp := GreaterThan
 ELSE
 Comp := Equal
 END
END;

The BNode function (line 186) can then be rewritten as

FUNCTION BNode(VAR sym : SymString) : SymTreePtr;
 { Find sym's node in binary tree, or add it if it doesn't exist }

VAR Node, Parent : SymTreePtr;
 UCsym : SymString;
 co, parentco : CompType;

BEGIN
 Node := Head;
 IF Case_Sensitive THEN
 UCsym := UpShift(sym)
 ELSE
 UCsym := sym;

 { Note that calls to NewNode have initialized all Node.UCSymbol }

 co := Comp(sym,Node^.Symbol,UCsym,Node^.UCsymbol,Case_Sensitive);
 WHILE (Node <> NIL) AND (co <> Equal) DO BEGIN
 Parent := Node;
 IF co = LessThan THEN
 Node := Node^.LLink
 ELSE
 Node := Node^.RLink;
 parentco := co;
 co := Comp(sym,Node^.Symbol,UCsym,Node^.UCsymbol,Case_Sensitive);
 END;
 IF Node <> NIL THEN { Node exists for this symbol }
 INC (Node^.Count)
 ELSE BEGIN { Else add new node to binary tree }
 Node := NewNode (sym);
 IF parentco = LessThan THEN { Update parent's pointer }
 Parent^.LLink := Node
 ELSE
 Parent^.RLink := Node
 END;
 BNode := Node;
END;














May, 1989
 CREATING TSRS PROGRAMS WITH TURBO PASCAL: PART 1


Ken L. Pottebaum


Ken Pottebaum is a professional mechanical engineer for the Small Disk
Division of Imprimis, a subsidiary of Control Data Corporation. He can be
reached at 321 Redbud. Yukon. OK 73099.


Can you write TSR (terminate and stay resident) programs in Turbo Pascal? The
answer is yes, and this two-part series shows how. This month I'll talk about
the basic requirements for TSR programs and present the tools (see TSRUNIT.PAS
in Listing One ) to create them. Next month I'll develop an example program
that illustrates installing the TSR program and changing its hot keys, using
files, and doing other things that real-world TSR programs need to do.
The primary requirement for a TSR program is for it to be "invisible" to the
operating system and other software. The operation of such a program can be
separated into two parts; its background mode, in which it waits to be popped
up, and its activated mode. In their background mode. TSR programs intercept
one or more interrupt vectors and monitor their calls to determine when to pop
up. These intercept routines are normally written in assembly language in
order to be efficient and to preserve the contents of the registers. Being
invisible in the popped-up mode means preserving the computer's state when the
TSR program pops up, and restoring its state when the program pops back down.
The main problem encountered when popping up a TSR program is dealing with
nonreentrant routines.
Because nonreentrant routines cannot be reentered safely, two general
approaches are used to eliminate the problem. The simplest method is to avoid
using any of the nonreentrant routines. As many DOS I/O and file functions are
nonreentrant, this approach would severely restrict the tasks that the TSR
program could perform; however, it does lend itself to the creation of small,
special-purpose TSR programs. An approach that allows more versatile TSR
programs is to avoid popping up the program while a nonreentrant routine is
being executed --this is the method TSRUnit uses.
While waiting for their cue to pop up. TSR programs normally monitor the
keyboard input and then pop up when their hot keys are detected. Either the
BIOS hardware keyboard interrupt 09h or the BIOS software keyboard interrupt
16h can be intercepted and used to monitor the keyboard input. Using interrupt
09h provides the advantages of quickly detecting the hot keys and of using key
combinations that are not normally recognized. Its quick response to the hot
keys is because this is the hardware interrupt that is executed whenever a key
is pressed or released. Its immediate response is also a drawback because the
computer could be executing a nonreentrant routine when the TSR program's
activation keys are pressed. Using interrupt 16h reduces the likelihood of the
hot keys being detected while an unsafe routine is being executed because this
software interrupt must be called by another routine; therefore it cannot be
executed at any time, as can a hardware interrupt. Unfortunately, it does not
completely prevent the TSR program from being popped up during unsafe or
nonreentrant routines. The disadvantages of using interrupt 16h are that the
Shift-key combinations used to make up the hot keys are reduced (they must be
a combination that is normally interpreted by the keyboard input routines) and
that the TSR program cannot be popped up until the software interrupt is
called.
Recent versions of BIOS firmware contain a service function. 4Fh, of interrupt
15h that is intended to be intercepted by software programs. This service
provides a clean way of monitoring the keyboard input and modifying the input.
Making use of this feature would restrict the TSR programs created using
TSRUnit to computers with the new BIOS function, however, so TSRUnit does not
use interrupt 15h.
In order to have flexibility in hot-key combinations, quick detection of the
hot keys, and the safety provided by the software interrupt, the method used
in TSRUnit involves both keyboard interrupt vectors.
Although the specific maintenance tasks required for a general-purpose TSR
program can best be seen by examining the tasks performed inside TSRUnit, the
guidelines for intercepting interrupt vectors are explained in the following
section.


Intercepting Interrupt Vectors


Routines to intercept interrupt calls are a vital part of a TSR program. As
with TSR programs, intercept routines must preserve the contents of the stack
and the registers --including the flag register. Interrupt routines are
normally called by executing an Int instruction, which pushes the flags and
then a two-word return address onto the stack. The instruction also clears the
interrupt bit of the flags, which disables further interrupts.
Although several methods of emulating an interrupt call exist, the simple and
efficient method used in TSRUnit is to push the flags onto the stack and then
perform a far indirect call. Using the indirect call allows the register
contents to be intact when the far call is executed --the call destination
address is contained at the memory location specified in the call instruction.
With the exception of interrupts 25h and 26h, all interrupt routines leave the
stack clean; that is, they exit by removing the return address and flags from
the stack as they return to the calling routine. The return can be performed
by executing either an interrupt return, IRET, or a far return, RETF 2, that
discards 2 bytes from the stack.
As previously mentioned, interrupts 25h and 26h --DOS absolute physical sector
read and write routines --are atypical. They leave the old flags on the stack,
and so all routines that call them must clean up the stack by discarding the
old flags after calling either interrupt. Furthermore, intercept routines for
them must leave the old flags on the stack when they exit.


Inside TSRUnit


TSRUnit performs three types of tasks. One routine, TSRInstall, installs the
TSR program. Once installed, the maintenance of the program is performed by
INLINE code routines in ASM and the Pascal routine PopUpCode. The remaining
four routines provide auxiliary services for checking the printer status and
for accessing lines of the saved text image. For additional information on
these auxiliary service routines see Listing One.
TSRInstall is a straightforward routine that provides the TSR intercept
routines with the information they will require. Rather than storing all the
data in the data segment, the TSR program stores the information required by
the INLINE code routines in the code segment. Having their data in the code
segment simplifies the INLINE routines by allowing them to access the data
without using an additional segment register. The following items are saved by
the installation routine:
1. The complete pointer to the top of the TSR program stack (obtained from the
current stack pointer and then adjusted to its value prior to the call to
TSRInstall)
2. The address of the routine to be called from PopUpCode
3. The original interrupt vectors for interrupts 09h, 16h, 21h, 25h, and 26h
4. The current video mode and the size of the cursor
5. Whether or not a math coprocessor is present
6. The size and address of the buffer for saving the screen image
7. The Shift-key combination code and the scan code of the activation key.
Although the command-line parameters are accessed and evaluated to determine
the hot keys to be used, this is not critical to the TSR program. The
conversion of the activation character to a keyboard scan is performed by
searching for the character in an array of allowed characters --the index of
the array is the scan code.
After saving the required items and displaying an installation message, the
interrupt vectors for the five routines being intercepted are replaced. The
SwapVectors command is used to save Turbo Pascal's interrupt vectors and to
restore previous vectors. Before issuing the Keep command to exit and stay
resident, the adjacent flag bytes, UnSafe and Flg, are cleared to enable the
TSR program to be popped up. Because their memory location is coincident with
the first 2 bytes of the ASM procedure, their default values are nonzero and
therefore prevent the TSR program from popping up while it is being installed.
The critical part of TSRUnit is contained in the INLINE code of procedure ASM.
Although the Turbo Pascal compiler sees ASM as one INTERRUPT procedure, it
actually contains five intercept routines and some data-storage space. The
standard entry and exit codes to ASM are used only as locations for storing
data --an additional block of space is also reserved for data storage at the
beginning of the routine. For programming convenience, ASM must be located at
offset 0 in TSRUnit. The untyped constants preceding the procedure then
provide the offsets to data locations and INLINE code routines in TSRUnit.
OurIntr21, the first routine in ASM, is an intercept routine for the DOS
function interrupt (interrupt 21h). The routine checks whether the function is
to be treated as nonreentrant by using the function number as the index to the
array DosTab --array values of 1 indicate nonreentrant routines. If it is
considered unsafe to pop up the TSR program while the function is being
executed, the UnSafe flag is set and then cleared upon returning from a far
call to the intercepted routine. Rather than toggling a single bit to indicate
safe and unsafe conditions, a byte value is incremented and decremented; this
allows the intercept routine to be reentrant. If the function is considered
safe, the registers are restored and a far jump to the intercepted routine is
used to exit from OurIntr21.
The next two routines, OurIntr25 and OurIntr26, intercept the DOS absolute
physical sector read and write interrupts in order to set the UnSafe flag
while they are executed. Notice that the two interrupts leave the old flags on
the stack so the intercept routines discard the old flags from the stack after
their call to the original interrupts by adding 2 to the stack pointer. Also,
they end with a RETF instruction that leaves the old flags on the stack.
Although the concept behind the intercept routine for BIOS 09h, OurIntr9, is
simple, its implementation is not as simple as that of the previous intercept
routines. In effect, the intercept routine checks for the activation key
combination and sets the pop-up bit in the flag Flg when it is detected
--provided that it or the in-use bits are not already set. All repeated
hot-key scan codes --obtained by holding the hot key down --are discarded, as
well as the initial one that causes the pop-up bit to be set. The scan codes
for repeated presses of the hot key while Flg is set are not discarded.
The actual implementation of OurIntr9 involves checking for multibyte and
buffer-full scan codes, as well as keeping track of the previous scan code in
order to discard repeated hot keys. All the scan codes, except the hot-key
code that set the pop-up flag and all repeated hot-key codes, are passed on to
the normal interrupt 09h routine. The multibyte scan codes are detected by
checking for the scan codes 0E0h and 0F0h; whenever one of them is detected,
the flag Flg9 is set and then cleared after the next byte is obtained. Because
buffer-full scan codes can separate repeated hot-key codes, the buffer-full
scan codes 00h and 0FFh do not cause Prev --containing the previous scan code
--to be updated.
OurIntr16, the intercept routine for the software keyboard interrupt,
essentially checks Flg to determine whether the TSR program is to be popped up
and whether any characters are to be inserted into the keyboard input steam.
It then executes the appropriate block of code. As with the OurIntr9 routine,
its implementation is not simple. The main services provided by interrupt 16h
are to read a character (services 00h and 10h) and to report whether a
character is ready (services 01h and 11h). The read-character services do not
return until a character is available. Because OurIntr9 discards the hot-key
scan code, it does not provide the read-character service with a character to
return. If a read-character request is being executed and the hot key is
pressed. OurIntr9 will set the pop-up flag, but OurIntr16 will not be able to
respond to it until another key is pressed --one that can be returned by the
read-character service. The solution to this problem is to emulate the
read-character service with a loop of report-character-ready requests and
allow a read request to be performed only when the report function indicates
that a character is ready. The loop contains separate checks for the pop-up
flag and the insert-character flag. When the pop-up signal is detected inside
the loop, the interface routine ToPopUp is called to allow the TSR program to
pop up. Because program execution returns to the read-character emulation loop
when the TSR program is popped back down, the only two ways out of the loop
are for the insert-character flag to be set or for a character to be reported
ready.
Inserting characters into the input stream involves moving them into the AL
register, setting the AH register to 0 (for a zero scan code), and exiting
from the intercept routine. Naturally, the "characters to insert" counter is
decremented and the character pointer incremented unless the service request
is report character ready instead of read character. The report service still
returns the character and scan code in AX, but the character should not be
"removed" from the buffer and the zero flag must be cleared to indicate that a
character is ready. In order to return the modified flags, the old flags must
be discarded instead of restored from the stack when the routine is exited.
In order to pop up the TSR program, OurIntr16 calls ToPopUp, which serves as
an interface to PopUpCode. If the UnSafe flag is set, ToPopUp immediately
returns to OurIntr16. If it is safe to pop up the TSR program, it adjusts the
in-use flag and pop-up flags, changes to the TSR program stack, and emulates
an interrupt call to PopUpCode. Upon returning from PopUpCode, it restores the
stack pointers and clears the in-use flag. Simple disable- and
enable-interrupt commands before and after the stack-changing commands are
required to prevent interrupts from occurring while the stack pointer is being
changed.
PopUpCode was written as an INTERRUPT routine so that the compiler would
automatically insert the code to save all the registers and to load the DS
register with the TSR program's data segment. The code generated also restores
the registers when PopUpCode is exited. The first and last tasks performed in
PopUpCode are calls to SwapVectors. The first call to SwapVectors restores all
of Turbo Pascal's interrupt vectors --these were saved when the TSR program
was installed. The second call restores the interrupt vectors to the settings
they had when PopUpCode was called.
The code between the beginning and ending SwapVectors statements is divided
into three sections. The first section saves various information about the
computer's current state and, if necessary, changes the video mode to the one
in use when the TSR program was installed. If the program is unable to save
the screen image, PopUpCode will be exited without popping up the TSR program.
In order to keep the memory requirements down, PopUpCode was written to handle
only text and CGA video modes. The code in the second section performs a far
call to the function address obtained when the TSR program was installed --the
popping up of the program is completed. This section also handles the number
of characters to insert value, which is returned from the function call. The
final section restores everything that may have been changed. Although a Copy
command was used to obtain some of the video information, all the video
information is restored using BIOS interrupt calls.
For a demonstration program illustrating how to use TSRUnit, check the next
issue of DDJ.


References Used


Naturally, I used Borland's Turbo Pascal Reference Guide for information on
Turbo Pascal. For information on DOS and BIOS interrupt routines, I used The
New Peter Norton Programmer's Guide to the IBM PC & PS/2, published by
Microsoft Press, 1988. For additional information on keyboard modes and their
scan codes, I consulted a copy of Compaq DeskPro 386/20 Technical Reference
Guide.



Availability


All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063; or call 800-356-2002 (from inside
Calif.) or 800-533-4372 (from outside Calif.). Please specify the issue number
and format (MS-DOS, Macintosh, Kaypro).


_Creating TSRs With Turbo Pascal, Part I_
by Ken L. Pottebaum


[LISTING ONE]

UNIT TSRUnit; {Create TSR programs with Turbo Pascal 5.0 & TSRUnit}
(*
The author and any distributor of this software assume no responsi-
bility for damages resulting from this software or its use due to
errors, omissions, incompatibility with other software or with
hardware, or misuse; and specifically disclaim any implied warranty
of fitness for any particular purpose or application.
*)

{$B-,F-,I+,R-,S+} {Set compiler directives to normal values.}

INTERFACE {=======================================================}
USES DOS, CRT;
CONST
{*** Shift key combination codes. }
 AltKey = 8; CtrlKey = 4; LeftKey = 2; RightKey = 1;

 TSRVersion : WORD = $0202; {Low byte.High byte = 2.02 }

TYPE
 String80 = STRING[80];
 ChrWords = RECORD CASE INTEGER OF
 1: ( W: WORD );
 2: ( C: CHAR; A: BYTE );
 END;
 LineWords = ARRAY[1..80] OF ChrWords;

VAR
 TSRScrPtr : POINTER; {Pointer to saved screen image. }
 TSRChrPtr : POINTER; {Pointer to first character to insert. }
 TSRMode : BYTE; {Video mode --------- before TSR popped up.}
 TSRWidth : BYTE; {Number of screen columns-- " " " " .}
 TSRPage : BYTE; {Active video page number-- " " " " .}
 TSRColumn : BYTE; {Cursor column number ----- " " " " .}
 TSRRow : BYTE; {Cursor row number -------- " " " " .}
{
** Procedure for installing the TSR program. }
PROCEDURE TSRInstall( PgmName, {Ptr to a char. string. }
 PgmPtr : POINTER; {Ptr to FUNCTION to call}
 ShiftComb: BYTE; {Hot key--shift key comb}
 KeyChr : CHAR ); {Hot Key--character key.}
{
 ShiftComb and KeyChr specify the default hot keys for the TSR.
 ShiftComb may be created by adding or ORing the constants AltKey,
 CtrlKey, LeftKey, and RightKey together. KeyChr may be

 characters 0-9 and A-Z.

 The default hot keys may be overridden when the TSR is installed
 by specifying optional parameters on the command line. The
 parameter format is:
 [/A] [/C] [/R] [/L] [/"[K["]]]
 The square brackets surround optional items--do not include them.
 Any characters between parameters are ignored. The order of the
 characters does not matter; however, the shift keys specified are
 cummulative and the last character key "K" specified is the used.
}
{
** Functions for checking status of printer LPT1. }
FUNCTION PrinterStatus: BYTE; {Returns status of printer. }
FUNCTION PrinterOkay: BOOLEAN; {Returns TRUE if printer is okay.}

{
** Routines for obtaining one row of screen characters. }
FUNCTION ScreenLineStr( Row: BYTE ): String80; {Returns char. str.}
PROCEDURE ScreenLine( Row: BYTE; VAR Line: LineWords; {Returns }
 VAR Words: BYTE ); {chr & color}

IMPLEMENTATION {==================================================}
VAR
 BuffSize, InitCMode : WORD;
 NpxFlag : BOOLEAN;
 Buffer : ARRAY[0..8191] OF WORD;
 NpxState : ARRAY[0..93] OF BYTE;
 RetrnVal, InitVideo : BYTE;

CONST {Offsets to items contained in PROCEDURE Asm. }
 UnSafe = 0; Flg = 1; Key = 2; Shft = 3;
 StkOfs = 4; StkSs = 6; DosSp = 8; DosSs = 10;
 Prev = 12; Flg9 = 13; InsNumb = 14;
 Dos21 = $10; Dos25 = Dos21+4; Dos26 = Dos25+4;
 Bios9 = Dos26+4; Bios16 = Bios9+4; DosTab = Bios16+4;
 Our21 = DosTab+99; Our25 = Our21+51; Our26 = Our25+24;
 Our09 = Our26+24; Our16 = Our09+127+8; InsChr = Our16+180-8;
 PopUp = InsChr+4; Pgm = PopUp+4;

PROCEDURE Asm; {Inline code--data storage and intercept routines. }
INTERRUPT;
BEGIN
INLINE(
{*** Storage for interrupt vectors. }
 {Dos21: } >0/>0/ {DOS func. intr vector. }
 {Dos25: } >0/>0/ {DOS abs. disk read intr. vector. }
 {Dos26: } >0/>0/ {DOS abs. sector write intr.vector. }
 {Bios9: } >0/>0/ {BIOS key stroke intr. vector. }
 {Bios16: } >0/>0/ {BIOS buffered keybd. input intr.vect.}

 {DosTab: ARRAY[0..98] OF BYTE = {Non-reetrant DOS functions.}
 0/0/0/0/0/0/0/0/ 0/0/0/0/0/1/1/1/ 1/1/1/1/1/1/1/1/
 1/1/1/1/1/1/1/1/ 1/1/1/1/1/1/0/1/ 1/1/1/1/1/1/1/0/
 1/0/0/0/0/0/1/1/ 1/1/1/1/1/1/1/1/ 1/1/1/1/1/1/1/1/
 0/0/0/0/0/0/1/1/ 0/0/0/0/1/0/1/1/ 0/1/1/1/1/0/0/0/ 0/0/0/

{*** OurIntr21 ******* Intercept routine for DOS Function Intr.***}
{ 0} $9C/ { PUSHF ;Save flags. }

{ 1} $FB/ { STI ;Enable interrupts. }
{ 2} $80/$FC/$63/ { CMP AH,63H ;Assume unsafe if new }
{ 5} $73/<22-7/ { JNB IncF ;function--skip table.}
{ 7} $50/ { PUSH AX ;Save registers. }
{ 8} $53/ { PUSH BX ;Load offset to table.}
{ 9} $BB/>DosTab/ { MOV BX,[DosTab] }
{ 12} $8A/$C4/ { MOV AL,AH ;Load table entry }
{ 14} $2E/ { CS: ;index. }
{ 15} $D7/ { XLAT ;Get value from table.}
{ 16} $3C/$00/ { CMP AL,0 ;If TRUE then set flag}
{ 18} $5B/ { POP BX ;Restore registers. }
{ 19} $58/ { POP AX ; }
{ 20} $74/$17/ { JZ JmpDos21 ;Jump to orig. intr. }
{ 22} $2E/ {IncF: CS: ; }
{ 23} $FE/$06/>UnSafe/ { INC [UnSafe] ;Set UnSafe flag. }
{ 27} $9D/ { POPF ;Restore flags. }
{ 28} $9C/ { PUSHF ; }
{ 29} $2E/ { CS: ; }
{ 30} $FF/$1E/>Dos21/ { CALL FAR [Dos21] ;Call orig. intr. }
{ 34} $FB/ { STI ;Enable interrupts. }
{ 35} $9C/ { PUSHF ;Save flags. }
{ 36} $2E/ { CS: ; }
{ 37} $FE/$0E/>UnSafe/ { DEC [UnSafe] ;Clear UnSafe flag. }
{ 41} $9D/ { POPF ;Restore flags. }
{ 42} $CA/$02/$00/ { RETF 2 ;Return & remove flag.}

{ 45} $9D/ {JmpDos21: POPF ;Restore flags. }
{ 46} $2E/ { CS: ; }
{ 47} $FF/$2E/>Dos21/ { JMP FAR [Dos21] ;Jump to orig. intr. }
{ 51}
{*** OurIntr25 ********** Intercept routine for DOS Abs. Read *** }
{ 0} $9C/ { PUSHF ;Save flags. }
{ 1} $2E/ { CS: ; }
{ 2} $FE/$06/>UnSafe/ { INC [UnSafe] ;Set UnSafe flag. }
{ 6} $9D/ { POPF ;Restore flags. }
{ 7} $9C/ { PUSHF ; }
{ 8} $2E/ { CS: ; }
{ 9} $FF/$1E/>Dos25/ { CALL FAR [Dos25] ;Call DOS abs. read. }
{ 13} $83/$C4/$02/ { ADD SP,2 ;Clean up stack. }
{ 16} $9C/ { PUSHF ;Save flags. }
{ 17} $2E/ { CS: ; }
{ 18} $FE/$0E/>UnSafe/ { DEC [UnSafe] ;Clear UnSafe flag. }
{ 22} $9D/ { POPF ;Restore flags. Leave}
{ 23} $CB/ { RETF ;old flags on the stk.}
{ 24}
{*** OurIntr26 ********** Intercept routine for DOS Abs. Write ***}
{ 0} $9C/ { PUSHF ;Save flags. }
{ 1} $2E/ { CS: ; }
{ 2} $FE/$06/>UnSafe/ { INC [UnSafe] ;Set UnSafe flag. }
{ 6} $9D/ { POPF ;Restore flags. }
{ 7} $9C/ { PUSHF ; }
{ 8} $2E/ { CS: ; }
{ 9} $FF/$1E/>Dos26/ { CALL FAR [Dos26] ;Call DOS abs. write. }
{ 13} $83/$C4/$02/ { ADD SP,2 ;Clean up stack. }
{ 16} $9C/ { PUSHF ;Save flags. }
{ 17} $2E/ { CS: ; }
{ 18} $FE/$0E/>UnSafe/ { DEC [UnSafe] ;Clear UnSafe flag. }
{ 22} $9D/ { POPF ;Restore flags. Leave}
{ 23} $CB/ { RETF ;old flags on the stk.}

{ 24}

{*** OurIntr9 ********** Intercept for BIOS Hardware Keyboard Intr}
{ 0} $9C/ { PUSHF ;Entry point. }
{ 1} $FB/ { STI ;Enable interrupts. }
{ 2} $1E/ { PUSH DS ; }
{ 3} $0E/ { PUSH CS ;DS := CS; }
{ 4} $1F/ { POP DS ; }
{ 5} $50/ { PUSH AX ;Preserve AX on stack.}
{ 6} $31/$C0/ { XOR AX,AX ;Set AH to 0. }
{ 8} $E4/$60/ { IN AL,60h ;Read byte from keybd }
{ 10} $3C/$E0/ { CMP AL,0E0h ;If multi-byte codes, }
{ 12} $74/<75-14/ { JE Sfx ;then jump and set }
{ 14} $3C/$F0/ { CMP AL,0F0h ;multi-byte flag, Flg9}
{ 16} $74/<75-18/ { JE Sfx ; }
{ 18} $80/$3E/>Flg9/$00/ { CMP [Flg9],0 ;Exit if part of }
{ 23} $75/<77-25/ { JNZ Cfx ;multi-byte code. }
{ 25} $3A/$06/>Key/ { CMP AL,[Key] ;Exit if key pressed }
{ 29} $75/<88-31/ { JNE PreExit ;is not hot key. }

{ 31} $50/ { PUSH AX ;Hot key was pressed, }
{ 32} $06/ { PUSH ES ;check shift key }
{ 33} $B8/$40/$00/ { MOV AX,0040h ;status byte. First }
{ 36} $8E/$C0/ { MOV ES,AX ;load BIOS segment. }
{ 38} $26/ { ES: ; }
{ 39} $A0/>$0017/ { MOV AL,[0017h] ;AL:= Shift key status}
{ 42} $07/ { POP ES ;Restore ES register. }
{ 43} $24/$0F/ { AND AL,0Fh ;Clear unwanted bits. }
{ 45} $3A/$06/>Shft/ { CMP AL,[Shft] ;Exit if not hot key }
{ 49} $58/ { POP AX ;shift key combination}
{ 50} $75/<88-52/ { JNE PreExit ;(Restore AX first). }

 { ;Hot Keys encountered.}
{ 52} $3A/$06/>Prev/ { CMP AL,[Prev] ;Discard repeated hot }
{ 56} $74/<107-58/ { JE Discard ;key codes. }
{ 58} $A2/>Prev/ { MOV [Prev],AL ;Update Prev. }
{ 61} $F6/$06/>Flg/3/ { TEST [Flg],3 ;If Flg set, keep key }
{ 66} $75/<99-68/ { JNZ JmpBios9 ;& exit to orig. BIOS }
{ 68} $80/$0E/>Flg/1/ { OR [Flg],1 ;9. Else set flag and}
{ 73} $EB/<107-75/ { JMP SHORT Discard;discard key stroke. }

{ 75} $B4/$01/ {Sfx: MOV AH,1 ;Load AH with set flag}
{ 77} $88/$26/>Flg9/ {Cfx: MOV [Flg9],AH ;Save multi-byte flag.}
{ 81} $C6/$06/>Prev/$FF/ { MOV [Prev],0FFh ;Change prev key byte.}
{ 86} $EB/<99-88/ { JMP SHORT JmpBios9 }

{ 88} $3C/$FF/ {PreExit: CMP AL,0FFh ;Update previous key }
{ 90} $74/<99-92/ { JE JmpBios9 ;unless key is buffer-}
{ 92} $3C/$00/ { CMP AL,0 ;full code--a 00h }
{ 94} $74/<99-96/ { JZ JmpBios9 ;0FFh }
{ 96} $A2/>Prev/ { MOV [Prev],AL ;Update previous key. }

{ 99} $58/ {JmpBios9: POP AX ;Restore registers and}
{100} $1F/ { POP DS ;flags. }
{101} $9D/ { POPF ; }
{102} $2E/ { CS: ; }
{103} $FF/$2E/>Bios9/ { JMP [Bios9] ;Exit to orig. intr 9.}

{107} $E4/$61/ {Discard: IN AL,61h ;Clear key from buffer}

{109} $8A/$E0/ { MOV AH,AL ;by resetting keyboard}
{111} $0C/$80/ { OR AL,80h ;port and sending EOI }
{113} $E6/$61/ { OUT 61h,AL ;to intr. handler }
{115} $86/$E0/ { XCHG AH,AL ;telling it that the }
{117} $E6/$61/ { OUT 61h,AL ;key has been }
{119} $B0/$20/ { MOV AL,20h ;processed. }
{121} $E6/$20/ { OUT 20h,AL ; }
{123} $58/ { POP AX ;Restore registers and}
{124} $1F/ { POP DS ;flags. }
{125} $9D/ { POPF ; }
{126} $CF/ { IRET ;Return from interrupt}
{127}

{*** OurIntr16 ***** Intercept routine for Buffered Keyboard Input}
{ 0} $58/ {JmpBios16: POP AX ;Restore AX, DS, and }
{ 1} $1F/ { POP DS ;FLAGS registers then }
{ 2} $9D/ { POPF ;exit to orig. BIOS }
{ 3} $2E/ { CS: ;intr. 16h routine. }
{ 4} $FF/$2E/>Bios16/ { JMP [Bios16] ; }

{ 8} $9C/ {OurIntr16: PUSHF ;Preserve FLAGS. }
{ 9} $FB/ { STI ;Enable interrupts. }
{ 10} $1E/ { PUSH DS ;Preserve DS and AX }
{ 11} $50/ { PUSH AX ;registers. }
{ 12} $0E/ { PUSH CS ;DS := CS; }
{ 13} $1F/ { POP DS ; }
{ 14} $F6/$C4/$EF/ { TEST AH,EFh ;Jmp if not read char.}
{ 17} $75/<48-19/ { JNZ C3 ;request. }

 {*** Intercept loop for Read Key service.}
{ 19} $F6/$06/>Flg/1/ {C1: TEST [Flg],1 ;If pop up Flg bit is }
{ 24} $74/<29-26/ { JZ C2 ;set then call INLINE }
{ 26} $E8/>122-29/ { CALL ToPopUp ;pop up routine. }
{ 29} $F6/$06/>Flg/16/{C2: TEST [Flg],10h ;Jmp if insert flg set}
{ 34} $75/<48-36/ { JNZ C3 ; }
{ 36} $FE/$C4/ { INC AH ;Use orig. BIOS }
{ 38} $9C/ { PUSHF ;service to check for }
{ 39} $FA/ { CLI ;character ready. }
{ 40} $FF/$1E/>Bios16/ { CALL FAR [Bios16];Disable interrupts. }
{ 44} $58/ { POP AX ;Restore AX and save }
{ 45} $50/ { PUSH AX ;it again. }
{ 46} $74/<19-48/ { JZ C1 ;Loop until chr. ready}

{ 48} $F6/$06/>Flg/17/{C3: TEST [Flg],11h ;Exit if neither bit }
{ 53} $74/<-55/ { JZ JmpBios16 ;of Flg is set. }
{ 55} $F6/$06/>Flg/$01/ { TEST [Flg],1 ;If pop up Flg bit is }
{ 60} $74/<65-62/ { JZ C4 ;set then call INLINE }
{ 62} $E8/>122-65/ { CALL ToPopUp ;pop up routine. }
{ 65} $F6/$06/>Flg/$10/{C4:TEST [Flg],10h ;Exit unless have }
{ 70} $74/<-72/ { JZ JmpBios16 ;characters to insert.}
{ 72} $F6/$C4/$EE/ { TEST AH,0EEh ;If request is not a }
{ 75} $75/<-77/ { JNZ JmpBios16 ;chr. request, exit. }

 {*** Insert a character. }
{ 77} $58/ { POP AX ;AX := BIOS service no}
{ 78} $53/ { PUSH BX ;Save BX and ES. }
{ 79} $06/ { PUSH ES ; }
{ 80} $C4/$1E/>InsChr/ { LES BX,[InsChr] ;PTR(ES,BX) := InsChr;}
{ 84} $26/ { ES: ;AL := InsChr^; }

{ 85} $8A/$07/ { MOV AL,[BX] ; }
{ 87} $07/ { POP ES ;Restore ES and BX. }
{ 88} $5B/ { POP BX ; }
{ 89} $F6/$C4/$01/ { TEST AH,01h ;IF AH IN [$01,$11] }
{ 92} $B4/$00/ { MOV AH,00h ; THEN ReportOnly; }
{ 94} $75/<114-96/ { JNZ ReportOnly ;Set Scan code to 0. }
{ 96} $FE/$06/>InsChr/ { INC [InsChr] ;Inc( InsChr ); }
{100} $FF/$0E/>InsNumb/ { DEC [InsNumb] ;Dec( InsNumb ); }
{104} $75/<111-106/ { JNZ SkipReset ;IF InsNumb = 0 THEN }
{106} $80/$26/>Flg/$EF/ { AND [Flg],0EFh ; Clear insert chr flg}
{111} $1F/ {SkipReset: POP DS ;Restore BX, DS, and }
{112} $9D/ { POPF ;FLAGS, then return }
{113} $CF/ { IRET ;from interrupt. }

{114} $1F/ {ReportOnly: POP DS ;Report char. ready. }
{115} $9D/ { POPF ;Restore DS and FLAGS.}
{116} $50/ { PUSH AX ;Clear zero flag bit }
{117} $40/ { INC AX ;to indicate a }
{118} $58/ { POP AX ;character ready. }
{119} $CA/>0002/ { RETF 2 ;Exit & discard FLAGS }

 {*** Interface to PopUpCode Routine. }
{122} $50/ {ToPopUp: PUSH AX ;Save AX. }
{123} $FA/ { CLI ;Disable interrupts. }
{124} $F6/$06/>UnSafe/$FF/{TEST [UnSafe],0FFh ;IF UnSafe <> 0 }
{129} $75/<177-131/ { JNZ PP2 ; THEN Return. }
{131} $A0/>Flg/ { MOV AL,[Flg] ;Set in-use bit; clear}
{134} $24/$FE/ { AND AL,0FEh ;pop up bit of Flg. }
{136} $0C/$02/ { OR AL,2 ;Flg := (Flg AND $FE) }
{138} $A2/>Flg/ { MOV [Flg],AL ; OR 2; }
 { ;**Switch to our stack}
{141} $A1/>StkOfs/ { MOV AX,[StkOfs] ;Load top of our stack}
{144} $87/$C4/ { XCHG AX,SP ;Exchange it with }
{146} $A3/>DosSp/ { MOV [DosSp],AX ;stk.ptr, save old SP.}
{149} $8C/$16/>DosSs/ { MOV [DosSs],SS ;Save old SS. }
{153} $8E/$16/>StkSs/ { MOV SS,[StkSs] ;Replace SS with our }
{157} $FB/ { STI ;SS. Enable interrupts}

{158} $9C/ { PUSHF ;Interrupt call to pop}
{159} $FF/$1E/>PopUp/ { CALL FAR [PopUp] ;up TSR routine. }

{163} $FA/ { CLI ;Disable interrupts. }
{164} $8B/$26/>DosSp/ { MOV SP,[DosSp] ;Restore stack ptr }
{168} $8E/$16/>DosSs/ { MOV SS,[DosSs] ;SS:SP. Clear in-use }
{172} $80/$26/>Flg/$FD/ { AND [Flg],0FDh ;bit of Flg. }

{177} $FB/ {PP2: STI ;Enable interrupts. }
{178} $58/ { POP AX ;Restore AX. }
{179} $C3 ); { RET ;Return. }
{180}
END; {Asm.} {END corresponds to 12 bytes of code--used for storage}

PROCEDURE PopUpCode; {Interface between the BIOS intercept }
INTERRUPT; {routines and your TSR function. }
CONST BSeg = $0040; VBiosOfs = $49;
TYPE
 VideoRecs = RECORD
 VideoMode : BYTE;
 NumbCol, ScreenSize, MemoryOfs : WORD;

 CursorArea : ARRAY[0..7] OF WORD;
 CursorMode : WORD;
 CurrentPage : BYTE;
 VideoBoardAddr : WORD;
 CurrentMode, CurrentColor : BYTE;
 END;
VAR
 Regs : Registers;
 VideoRec : VideoRecs;
 KeyLock : BYTE;
 ScrnSeg : WORD;
BEGIN
 SwapVectors; {Set T.P. intr. vectors.}
 Move( Ptr(BSeg,VBiosOfs)^, VideoRec, {Get Video BIOS info. }
 SizeOf(VideoRec) );
 WITH VideoRec, Regs DO BEGIN
 IF (VideoMode > 7) OR {Abort pop up if unable}
 (ScreenSize > BuffSize) THEN BEGIN {to save screen image. }
 SwapVectors; {Restore intr. vectors.}
 Exit;
 END;
 KeyLock := Mem[BSeg:$0017]; {Save lock key states. }
 IF VideoMode = 7 THEN ScrnSeg := $B000 {Save screen--supports }
 ELSE ScrnSeg := $B800; {text, MGA & CGA modes.}
 Move( PTR( ScrnSeg, MemoryOfs )^, Buffer, ScreenSize );
 AX := InitVideo; {If in graphics mode, }
 IF (VideoMode >=4) {switch to text mode. }
 AND (VideoMode <= 6) THEN Intr( $10, Regs );
 AX := $0500; {Select display page 0.}
 Intr( $10, Regs );
 CX := InitCMode; {Set cursor size. }
 AH := 1;
 Intr( $10, Regs );

 TSRMode := VideoMode; {Fill global variables }
 TSRWidth := NumbCol; {with current information}
 TSRPage := CurrentPage;
 TSRColumn := Succ( Lo( CursorArea[CurrentPage] ) );
 TSRRow := Succ( Hi( CursorArea[CurrentPage] ) );

 IF NpxFlag THEN {Save co-processor state.}
 INLINE( $98/ $DD/$36/>NpxState ); {WAIT FSAVE [NpxState] }
{
*** Call user's program and save return code--no. char. to insert.
}
 INLINE( $2E/$FF/$1E/>Pgm/ { CALL FAR CS:[Pgm] }
 $2E/$A3/>InsNumb ); { MOV CS:[InsNumb],AX }

 IF Mem[CSeg:InsNumb] > 0 THEN BEGIN {Have char. to insert. }
 MemL[CSeg:InsChr] := LONGINT( TSRChrPtr );
 Mem[CSeg:Flg] := Mem[CSeg:Flg] OR $10;
 END;
{
*** Pop TSR back down--Restore computer to previous state.
}
 IF NpxFlag THEN {Restore co-prcssr state.}
 INLINE( $98/ $DD/$36/>NpxState ); {WAIT FSAVE [NpxState] }

 Mem[BSeg:$17] := {Restore key lock status.}

 (Mem[BSeg:$17] AND $0F) OR (KeyLock AND $F0);

 IF Mem[BSeg:VBiosOfs] <> VideoMode THEN BEGIN
 AX := VideoMode; {Restore video mode. }
 Intr( $10, Regs );
 END;
 AH := 1; CX := CursorMode; {Restore cursor size. }
 Intr( $10, Regs );
 AH := 5; AL := CurrentPage; {Restore active page. }
 Intr( $10, Regs );
 AH := 2; BH := CurrentPage; {Restore cursor positon. }
 DX := CursorArea[CurrentPage];
 Intr( $10, Regs ); {Restore screen image. }
 Move( Buffer, PTR( ScrnSeg, MemoryOfs )^, ScreenSize );

 SwapVectors; {Restore non-T.P. vectors.}
 END;
END; {PopUp.}
{
***** Printer Functions:
}
FUNCTION PrinterStatus: BYTE; {Returns status of LPT1.}
{ Definition of status byte bits (1 & 2 are not used), if set then:
 Bit: -- 7 --- ---- 6 ---- -- 5 --- -- 4 --- -- 3 -- --- 0 ---
 Not busy Acknowledge No paper Selected I/O Err. Timed-out
}
VAR Regs : Registers;
BEGIN
 WITH Regs DO BEGIN
 AH := 2; DX := 0; {Load BIOS function and printer number. }
 Intr( $17, Regs ); {Call BIOS printer services. }
 PrinterStatus := AH; {Return with printer status byte. }
 END;
END; {PrinterStatus.}

FUNCTION PrinterOkay: BOOLEAN; {Returns TRUE if printer is okay. }
VAR S : BYTE;
BEGIN
 S := PrinterStatus;
 IF ((S AND $10) <> 0) AND ((S AND $29) = 0) THEN
 PrinterOkay := TRUE
 ELSE PrinterOkay := FALSE;
END; {PrinterOkay.}
{
***** Procedures to obtain contents of saved screen image.
}
PROCEDURE ScreenLine( Row: BYTE; VAR Line: LineWords;
 VAR Words: BYTE );
BEGIN
 Words := 40; {Determine screen line size.}
 IF TSRMode > 1 THEN Words := Words*2; {Get line's }
 Move( Buffer[Pred(Row)*Words], Line, Words*2 ); {characters and }
END; {ScreenLine.} {colors. }

FUNCTION ScreenLineStr( Row: BYTE ): String80; {Returns just chars}
VAR
 Words, i : BYTE;
 LineWord : LineWords;
 Line : String80;

BEGIN
 ScreenLine( Row, LineWord, Words ); {Get chars & attributes. }
 Line := ''; {Move characters to string}
 FOR i := 1 TO Words DO Insert( LineWord[i].C, Line, i );
 ScreenLineStr := Line;
END; {ScreenString.}
{
***** TSR Installation procedure.
}
PROCEDURE TSRInstall( PgmName, PgmPtr: POINTER;
 ShiftComb: BYTE; KeyChr: CHAR );
CONST
 ScanChr = '+1234567890++++QWERTYUIOP++++ASDFGHJKL+++++ZXCVBNM';
 CombChr = 'RLCA"';
VAR
 PgmNamePtr, PlistPtr: ^STRING;
 i, j, k : WORD;
 Regs : Registers;
 Comb, ScanCode : BYTE;
BEGIN
 IF Ofs( Asm ) <> 0 THEN EXIT; {Offset of Asm must be 0}
 MemW[CSeg:StkSs] := SSeg; {Save pointer to top of }
 MemW[CSeg:StkOfs] := Sptr + 308; {TSR's stack. }
 MemL[CSeg:PopUp] := LONGINT(@PopUpCode); {Save PopUpCode addr. }
 MemL[CSeg:Pgm] := LONGINT(PgmPtr); {Save PgmPtr. }
 PgmNamePtr := PgmName; {Convert ptr to string ptr.}

 Writeln('Installing Stay-Resident program: ',PgmNamePtr^);
{
***** Save intercepted interrupt vectors: $09, $16, $21, $25, $26.
}
 GetIntVec( $09, POINTER( MemL[CSeg:Bios9] ) );
 GetIntVec( $16, POINTER( MemL[CSeg:Bios16] ) );
 GetIntVec( $21, POINTER( MemL[CSeg:Dos21] ) );
 GetIntVec( $25, POINTER( MemL[CSeg:Dos25] ) );
 GetIntVec( $26, POINTER( MemL[CSeg:Dos26] ) );
{
***** Get equipment list and video mode.
}
 WITH Regs DO BEGIN
 Intr( $11, Regs ); {Check equipment list for }
 NpxFlag := (AL AND 2) = 2; {math co-processor. }
 AH := 15; {Get current video mode }
 Intr( $10, Regs ); {and save it for when TSR }
 InitVideo := AL; {is activated. }
 AH := 3; BH := 0; {Get current cursor size }
 Intr( $10, Regs ); {and save it for when TSR }
 InitCMode := CX; {is activated. }
 END; {WITH Regs}
{
***** Get info. on buffer for saving screen image.
}
 BuffSize := SizeOf( Buffer );
 TSRScrPtr := @Buffer;
{
*** Determine activation key combination.
}
 Comb := 0; i := 1; {Create ptr to }
 PlistPtr := Ptr( PrefixSeg, $80 ); {parameter list. }

 WHILE i < Length( PlistPtr^ ) DO BEGIN {Check for parameters.}
 IF PlistPtr^[i] = '/' THEN BEGIN {Process parameter. }
 Inc( i );
 j := Pos( UpCase( PlistPtr^[i] ), CombChr );
 IF (j > 0) AND (j < 5) THEN Comb := Comb OR (1 SHL Pred(j))
 ELSE IF j <> 0 THEN BEGIN {New activation char. }
 Inc( i ); k := Succ( i );
 IF i > Length(PlistPtr^) THEN KeyChr := #0
 ELSE BEGIN
 IF ((k <= Length(PlistPtr^)) AND (PlistPtr^[k] = '"'))
 OR (PlistPtr^[i] <> '"') THEN KeyChr := PlistPtr^[i]
 ELSE KeyChr := #0;
 END; {ELSE BEGIN}
 END; {ELSE IF ... BEGIN}
 END; {IF PlistPtr^[i] = '/'}
 Inc( i );
 END; {WHILE ...}
 IF Comb = 0 THEN Comb := ShiftComb; {Use default combination. }
 IF Comb = 0 THEN Comb := AltKey; {No default, use [Alt] key.}
 ScanCode := Pos( UpCase( KeyChr ), ScanChr ); {Convert char. to}
 IF ScanCode < 2 THEN BEGIN {scan code. }
 ScanCode := 2; KeyChr := '1';
 END;
 Mem[CSeg:Shft] := Comb; {Store shift key combination}
 Mem[CSeg:Key] := ScanCode; {and scan code. }
{
*** Output an installation message: Memory used & activation code.
}
 Writeln( 'Memory used is approximately ',
 ( ($1000 + Seg(FreePtr^) - PrefixSeg)/64.0):7:1,' K (K=1024).');
 Writeln(
'Activate program by pressing the following keys simultaneously:');
 IF (Comb AND 1) <> 0 THEN Write(' [Right Shift]');
 IF (Comb AND 2) <> 0 THEN Write(' [Left Shift]');
 IF (Comb AND 4) <> 0 THEN Write(' [Ctrl]');
 IF (Comb AND 8) <> 0 THEN Write(' [Alt]');
 Writeln(' and "', KeyChr, '".');
{
*** Intercept orig. interrupt vectors; Then exit and stay-resident.
}
 SetIntVec( $21, Ptr( CSeg, Our21 ) );
 SetIntVec( $25, Ptr( CSeg, Our25 ) );
 SetIntVec( $26, Ptr( CSeg, Our26 ) );
 SetIntVec( $16, Ptr( CSeg, Our16 ) );
 SetIntVec( $09, Ptr( CSeg, Our09 ) );
 SwapVectors; {Save turbo intr.vectors.}
 MemW[CSeg:UnSafe] := 0; {Allow TSR to pop up. }
 Keep( 0 ); {Exit and stay-resident. }
END; {TSRInstall.}
END. {TSRUnit.}












May, 1989
KERMIT MEETS MODULA-2


The modularity of Modula-2 makes it well-suited for communications projects




Brian R. Anderson


Brian is an instructor in the computer systems technology department of the
British Columbia Institute of Technology. He can be reached at BCIT, Burnaby
Campus, 3700 Willingdon Ave., Burnaby, BC V5G 3H2 Canada.


Kermit (named after the famous frog) is a general-purpose file transfer
protocol that was developed at Columbia University in 1981. What is innovative
about Kermit is that it makes minimal assumptions about the hardware upon
which it operates. While this innovation allows Kermit to be used on virtually
any computer, Kermit is typically used for communications between
microcomputers and larger computers (minicomputers or mainframes).
This article describes a minimal implementation of Kermit that uses Logitech
Modula-2. In order to provide a framework for my discussion of the design and
implementation of the program, I will first discuss the Kermit protocol in
some detail. I will use the layered protocol approach to describe how I
employed a Yourdon-style Data Flow Diagram (DFD) for the overall design of the
program. Finally, I will discuss how the use of Modula-2 allowed me to
implement each layer of the protocol as a separately compiled module.


The Need For Data Communications Protocols


On the surface, the process of data communications appears to be very simple:
Data is converted into a form that matches the transmission medium (the
MODulator part of the Modem handles this step); the data is sent on its way;
and then the data is converted back to the original data at the other end
(this step is the task of the DEModulator part of the Modem). Unfortunately,
the actual process is seldom that easy.
If the transmission path used to move data were perfect, no communication
protocol would be necessary (or at least, the protocol could be very simple),
and the data could merely be sent as electrical impulses from one system to
another. All real transmissions paths suffer from problems such as noise,
dropout, and distortion (including phase distortion, frequency distortion, and
amplitude distortion). Despite the use of the best possible hardware, any of
these impairments can cause data errors -- such that the received message does
not match the transmitted message. To be effective, a file transfer protocol
must detect such errors and take corrective action.
The receiver and the transmitter may not have the same timing requirements. If
the receiver is not ready for data when the transmitter starts sending,
information will be lost. A file transfer protocol must also allow the
receiver and the transmitter to synchronize so that such data loss does not
happen.


The Kermit Protocol


Kermit is a point-to-point (as opposed to multipoint) communications protocol
that uses stop-and-wait Automatic Retransmission reQuest (ARQ) for error
control. (XModem, which is widely used for communications between
microcomputers, is also a point-to-point, stop-and-wait protocol.) After
sending some data, Kermit stops and waits for an acknowledgement of the data
that it just sent. If no acknowledgement (or a negative acknowledgement)
occurs, Kermit resends any data that had been sent since the last
acknowledgement.
Kermit handles all communications via packets (unlike XModem, which uses
single characters for acknowledgement). A packet is a stream of bytes that
fits a particular pattern. Figure 1 shows a block diagram of the Kermit
packet.
The Mark field uniquely identifies the beginning of the Kermit packet. This
field contains an ASCII start-of-header character, which is the only
nonprintable character anywhere in the packet. (If nonprintable characters are
encountered elsewhere, they are converted into printable characters. This
process is explained shortly.)
The Length field contains a count of the number of characters in the rest of
the packet; this count ranges from 3 (to indicate an empty data field) to 94
(the maximum size of a data field). The Sequence field indicates the packet
sequence number. The range of sequence numbers is from zero to 63. (After 63,
the sequence repeats.) The Type field identifies the type of packet. Packet
types include D (data), Y (yes, acknowledge), N (negative acknowledge), S
(send initiate), F (filename), Z (end of file), E (error), and B (break
transmission, end of transmit). (There are other packet types that are not
used in this implementation; see the references given at the end of this
article for further details.)
The Data field in the Kermit packet contains just what its name implies --
data. (Due to the constraint that the packet contain only printable
characters, some data must be converted to printable form and/or encoded.)
Finally, the Check field (which is usually an arithmetic CheckSum) allows
error control.
The features of Kermit that allow this protocol to be used on nearly any
computer system are the small packet size (96 characters maximum) and Kermit's
reliance upon printable characters. The latter is necessary because many
systems interpret nonprintable characters in special ways. (In Unix, for
example, Control-D means end-of-file; in CP/M and MS-DOS, Control-Z may be
used for the same function.) In addition, many mainframes can deal with only
7-bit characters -- these mainframes cannot use the parity bit for data. In
order to avoid the problems caused by control and binary characters, the
designers of Kermit choose to convert any suspect characters into normal
printable characters. In this context, suspect characters include any control
characters that have a binary 1 in the eighth bit.


Initialization


Before transmission can begin, the receiver and the transmitter must agree
upon certain parameters. Some of these parameters (transmission rate, parity,
and the number of stop bits) must be set up in hardware before Kermit can
begin. Other parameters are handled via a limited negotiation that occurs
during transmission of the first packet. The data field of the first packet
contains several fields for this purpose. This negotiation process is very
simple: If there is no agreement about a feature where agreement is necessary,
an automatic fallback to defaults occurs. When agreement about a feature is
not necessary, each end complies with the other end's wishes.
Some of the fields within the initialization field may contain values that do
not result in printable ASCII characters. Each of these values must be
converted by adding 20H to the value. This process, called "character-izing,"
is performed on the MAXL, TIME, NPAD, and EOL fields. The PADCfield is treated
differently. This field is "control-ified," which means that bit 6 is
inverted, by XORing it with 01000000B. Other fields are sent as literal
characters.
Figure 2 shows the contents of the initialization packets. (One packet is sent
from each end.) During initialization, each end initializes the other. (The
pronouns "I" and "you" are often used to describe each end of the
communications link.) Table 1 describes each of the fields in the
initialization packet.
Table 1: Fields in the initialization packet

 FIELD FUNCTION
 ---------------

 MAXL The maximum packet size that I can accept; you will also
 indicate your maximum packet length.
 TIME The maximum time (in seconds) that you should wait before timing
 me out.
 NPAD The number of padding characters that you should send ahead of
 each packet. (Default is zero.)
 PADC The padding character that you should use.
 EOL The character that you should send to terminate each packet
 (usually none or ASCII <cr>).

 QCTL The character that I will use to control the process of
 character quoting (usually #).
 QBIN The character that I will use for quoting binary bytes
 (usually &.
 CHKT The type of check character used: 1 means a one-byte CheckSum;
 2 means a two-byte CheckSum; and 3 means a three-byte CRC.
 This version uses only the one-byte CheckSum.
 REPT The character that I will use for repeat-count encoding. A blank
 (ASCII <sp>) means that this feature is not used (as is the case
 with this implementation).
 CAPA Advanced capabilities. Each bit has a separate function, and
 several bytes may be linked together (bit zero is a linkage
 bit). These features are not used in this implementation.



File Transfer


After initialization, the process of file transfer consists of sending or
receiving any number of files. Each of these files is prefixed with a packet
that contains the filename and terminates with an end-of-file packet. After
all files have been transferred, an end-of-transmit packet is sent.
While this implementation cannot transmit multiple files (each file must be
transmitted with a separate Send command), it can receive multiple files.
Figure 3 and Figure 4 show state diagrams for the processes of receiving and
sending, respectively (adapted from reference 1 in the bibliography following
this article). These simplified diagrams are meant only to present an overview
of the sequence of events that occur during file transfer.
As mentioned earlier, Kermit requires that all data be sent as printable
characters. To accomplish this, and at the same time to allow any type of data
to be transmitted, control codes (ASCII 00H - 19H and 7FH plus binary codes
outside of the range of ASCII 80H - FFH) must be handled in a special way. To
do so, the data is modified so that it is a printable character, and a
"prefix" (so-called "quote") character is used to advise the other side that
this modification has been performed.
Control characters are XORed with 01000000B (which inverts bit 6), and then
prefixed with #. For example, a <cr> <lf> sequence (which is Control-M and
Control-J) becomes #M#J.
When a character has a binary 1 in the most significant bit position, this bit
is inverted and the resulting character is prefixed with &. For example,
11000001B is converted to &A.
In some cases, both of these quoting schemes must be used: 10000001B becomes
&#A. The two prefix (quoting) characters can be sent by prefixing them with #.
# becomes ##, while & becomes #&. The use of quoting schemes can add
considerable overhead. The control characters make up 26.6 percent of the
ASCII character set, while bytes with the most significant bit high make up 50
percent of random binary files. In contrast, text files can be transmitted
quite efficiently because they contain few control characters and no nonASCII
characters.


Layered Communications Protocols


Communications protocols are often described as having several layers (or
levels). Each of the layers is responsible for certain aspects of the steps
involved in establishing and maintaining a connection, moving the data, or
ensuring data integrity. Layered protocols allow a standard to be flexible
enough to be widely adopted. The inherent modularity of layered protocols also
makes the process of implementing standards easier.


The ISO-OSI Seven-Level Model


The most notable layered protocol is the International Standards
Organization's Open Systems Interconnect Seven-Level Reference Model, usually
called the OSI model. This was meant to be a universal model for large-scale
international networks, although many of the principles also apply to more
limited applications (such as Kermit).
The diagram in Figure 5 compares the OSI model with Kermit. Because Kermit is
a simple point-to-point protocol where the connection process is under the
manual control of the user, not all of the OSI layers are required.


Software Design


By following the data flow through the protocol layers, design of the program
becomes straightforward. As mentioned at the beginning of this article, I used
a data flow diagram (DFD) as an initial design tool. The DFD, which is shown
as Figure 6, provides a clear picture of the overall design of this Kermit
implementation.
When designing a program in Modula-2, an early step (and the next step here)
is to design the main module and the definition modules. The main module,
called the PCKermit, calls the other modules. Definition modules contain no
code, and describe the actions the implementation modules will perform. Four
definition modules were designed from the DFD: Shell, PAD, Files, and
DataLink. The physical layer module is provided by the Modula-2 compiler that
I used (Logitech), so some effort was saved here. The compiler-supplied module
that implements the physical layer is named RS232Int (for "interrupt-driven
access to the RS-232 serial port"). The main module and the definition modules
are shown as Listings One through Five respectively.
The definition modules closely match the DFD, with the exception that the
packet assembler and the packet disassembler are combined into a single
module, called PAD (for Packet Assembler/Disassembler). Although many
procedures are hidden within the implementation of PAD, only two are imported
and used by the main module: Send and Receive. The main module imports several
procedures from Shell, including dispOpts, Options, Dir, Connect, eXit, and
MainHelp. The purpose of many of these procedures is obvious from their names:
dispOpts and Options refer to the communications options (Baud rate, parity,
and so forth). Connect provides terminal emulation.
The Files, DataLink, and RS232Int modules follow the layering concept and are
not accessed from the main module. These lower-level modules are accessed only
from within PAD.
Although examination of the main module and the definition modules reveals few
(if any) details of the underlying implementation, it does provide a good
picture of the overall design of the program.


Implementation Details


Each of the modules mentioned above is divided into several procedures, and
each of these procedures has one well-defined purpose or task. Here are
descriptions of a few of the key parts of the code:


The Shell Module



In Modula-2, each implementation module can have an initialization section,
which is also called the "module body." The module body looks like a main part
of a program module. It is executed once, when the program first starts up, in
order to initialize the module. In the case of the Shell module, the module
body sets the initial conditions of the communications hardware (that is, the
Baud rate, parity, and so on).
The Shell implementation module (Listing Six) contains several local
procedures for setting the Baud rate, parity, word length, and number of stop
bits used by the communications hardware. In Modula-2, local procedures that
are not mentioned in the EXPORT list of the definition module are available
only within the module in which these procedures are defined. In this case,
these procedures are accessed by the Options procedure, which is exported and
used by the main module, PCKermit.
I had a bit of fun with the Dir procedure -- I wanted a single command that
could either display the current directory, or else change to another
subdirectory, and then display the contents of that subdirectory. This
required some minor parsing of user input in order to separate the directory
name from the file name. For example, if you entered FOO\*.EXE, the Dir
procedure would try to switch to a subdirectory named FOO, and then display
all of the .EXE files. Dir is smart enough to know the difference between
FOO\*.EXE and \FOO\*.EXE. A useful enhancement would be the ability to log
onto a different disk drive; this enhancement would just require a bit more
parsing.
The Connect command emulates a glass teletype by scanning the keyboard for
input and the sending any input that it receives it out to the RS-232 serial
port. Connect also scans the RS-232 serial input port, and then sends anything
that it receives to the screen. Some interpretation is necessary in order to
prevent control characters from printing, to allow for backspaces, and to
switch echo modes on and off. The most useful addition that could be made here
would be full terminal emulation (to make the screen and keyboard act like a
common video terminal, such as the Televideo 950 or the DEC VT100).


The Pocket Assembler/Disassembler (PAD) Module


Although the PAD module (Listing Seven) contains over 20 procedures, only two
procedures (Send and Receive) are exported for the use of other modules. The
balance of the procedures (some of which are only a few line of code) help the
exported procedures get the job done.
Several of the procedures construct or decipher the various control packets,
such as the initialization, response (ACK/NAK), and error packets.
The Send procedure reads data from a file and uses that data to construct a
packet. Before a character is added to a packet, the character must be checked
to see if it is a control character or if it has the most significant bit set.
If either condition exists, the character is altered, and a quote character (#
or &) is added. When the packet is nearly full, the count, sequence, and type
bytes are added before the packet is passed on to the DataLink module.
The Receive procedure contains a nested loop. (Example 1 shows the pseudocode
for this loop.) As required by the protocol, each successfully received packet
is acknowledged by a Y packet. If no packet is received, or if the received
packet contains errors, then an N packet is sent instead. After several errors
in a row (errors = MAXtrys), file transfer is abandoned and an error message
is issued to the user. If all goes as it should, each good packet is processed
to remove both control-quoting and binary-quoting (e.g., #M#J is changed to
<cr><lf> again, and so on) before the data is stored to the output file.
Example 1: Pseudocode for the Receive procedure

 WHILE <"end of transmit" packet not received> DO
 (* receive a file *)
 WHILE <"end of file" packet not received> DO
 (* receive a packet *)
 END;
 END;

Two small procedures, Char and UnChar, are used throughout this module to
convert packet service bytes to/from their "characterized" format. (This is
necessary to ensure that the sequence and length fields contain only printable
characters, and that these characters are later correctly interpreted as
numbers.)


The Files Module


The purpose of the Files module (Listing Eight) is two-fold: to provide a
"nicer" interface to the underlying file system, and to gain full control of
disk buffering.
Wirth's FileSystem module uses a single procedure, called Lookup, to open
existing files or to create new files. (A Boolean parameter determines which
action Lookup performs.) I have used Lookup to construct separate Open and
Create procedures. The Open procedure returns a Status Error if the file does
not exist. If the Create procedure determines that the requested file already
exists, this procedure advises the user and asks if the file should be
overwritten before it continues.
One of the major problems encountered when developing communications programs
is timing -- specifically, it is possible to miss incoming data if the program
is doing something that is time-consuming (such as disk I/O) at the "wrong"
time. The "right" time to output to the disk is after a packet has been
received, but before the packet has been acknowledged. The stop-and-wait
protocol then ensures that no more data is received until after the disk write
is complete.
The Files module writes to the disk in two stages. The Put procedure puts the
character into a buffer. The DoWrite procedure outputs the buffer to the disk
only if the buffer is nearly full (that is, if it is too full to fit into
another packet). The Receive procedure in PAD calls Put for each character in
the packet, and then calls DoWrite at the end of the packet. (At first glance,
you might be tempted to use a block write command to store the entire packet
that is received. This is not possible, however, because the packet contains a
great deal of information that should not be stored to the file -- the service
fields [Mark, Length, Sequence, Type, and Check] and the control/binary
quoting characters must be stripped from the packet before it is saved.)
Finally, the CloseFile procedure must ensure that the buffer is flushed to
disk before actually closing an output file.


The DataLink Module


The DataLink module (Listing Nine) receives and transmits packets. The Send
Packet procedure accepts a packet from PAD and writes the packet to the serial
port, one character at a time (via the RS232Int module). The ReceivePacket
procedure reads characters from the serial port (via RS232Int) and assembles
them into packets, which are then passed on to PAD. SendPacket is by far the
simpler of these two procedures -- it outputs the characters in a loop,
calculates the CheckSum as it goes, and then outputs the CheckSum at the end.
Very little can go wrong with Send Packet.
By contrast, the Receive Packet procedure has to be able to handle several
potential problems: timeouts (no data received), packet format, and CheckSum
errors. First of all, ReceivePacket looks for the Mark character (ASCII SOH)
until one of the following three events occurs: ReceivePacket reads 100
characters without finding SOH; about 10 seconds elapses; or SOH is read. If
SOH is not encountered, an error message is output to the screen and error
status is returned to PAD, which decides whether or not to try again. If SOH
is encountered, ReceivePacket then receives the rest of the packet.
ReceivePacket also assumes that the next byte is the Length field, which is
used to determine how many characters should be read before expecting the
CheckSum field. If that byte is not the Length field, the CheckSum field will
be wrong, and the error will be recognized. As the characters are being
received, a local CheckSum is calculated for comparison to the CheckSum field
that is received from the remote end. If the CheckSum do not match, an error
is reported to the screen and to PAD. As always, it is vital that the Receive
routines time out if no data is received.
Notice that although the CheckSum is a simple arithmetic sum of the ordinal
value of the characters, some rather bizarre calculations are performed upon
the sum before it is used. These calculations ensure that the resulting
CheckSum is a printable ASCII character, and make all of the bits of the
original sum significant (so that all of the bits have some effect upon the
CheckSum field).


Performance


Kermit is most useful if no other file transfer method is available, as is
often the case when dissimilar computers are involved. Kermit is relatively
efficient when transferring text files. When Kermit is used to transfer binary
files, its quoting schemes can add more than 70 percent redundancy.
I have used this implementation of Kermit to upload and download files to the
IBM 3083 mainframe at the British Columbia Institute of Technology. This
implementation has also been tested successfully with the Kermit section of
Procomm, using a direct link (NULL MODEM) at up to 9600 Baud.


Portability


I have ported this implementation to the Atari ST, and now frequently use the
two Kermits to transfer files between the ST and the PC. The only significant
change in porting over to the ST involved writing an RS232Int module for the
ST in order to handle differences in the interface to the serial
communications hardware on the ST.
The process of moving Kermit to other platforms should involve minimal effort
if a reasonably complete Modula-2 compiler is available. Of course, conversion
to a different language is also possible, although that approach would involve
considerably more effort. Several C implementations of Kermit are available on
various bulletin boards and directly from Columbia University.


Limitations and Problems



This was meant to be a minimal implementation of the Kermit protocol, so none
of the advanced features were incorporated. Anyone interested in adding these
features is referred to the bibliography at the end of this article. (The Joe
Campbell book includes some recent enhancements to the protocol that are not
mentioned in the earlier Byte article, but the article is more detailed and
complete.)


Conclusion


When implementing a communications protocol, it is vital to understand and to
apply the concept of layering. Each layer has in the protocol a small number
of well-defined tasks. If each layer is handled by an independent module, the
protocol's overall complexity is broken down to a manageable level. The
ISO-OSI Seven-Level Reference Model provides a useful model, even for simpler
protocols such as Kermit.
The most subtle problems that can arise with a communications protocol are due
to timing constraints -- one end will not quit sending just because the other
end is busy doing something other than receiving. It is important to properly
handle such potential problems in order to prevent data loss.
The Modula-2 programming language proved to be well able to handle this
communications project. The design and implementation were completed in about
a week. Due to the modular nature of the program, alterations and enhancements
will not be difficult.


Bibliography


1. de Cruz, Frank; and Catchings, Bill. "Kermit: A File-Transfer Protocol For
Universities." Byte Magazine June/July 1984).
2. Campbell, Joe. C Programmer's Guide to Serial Communications. Indianapolis,
Ind.: Howard W. Sams & Company, 1987.
3. McGovern, Tom. Data Communications. Concepts and Applications. Englewood
Cliffs, N.J.: Prentice-Hall, 1988.


Availability


All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063, or call 800-356-2002 (inside Calif.) or
800-533-4372 (outside Calif.). Please specify the issue number and format
(MS-DOS, Macintosh, Kaypro).


_Kermit Meets Modula-2_
by Brian Anderson


[LISTING ONE]

MODULE PCKermit;

 FROM Break IMPORT
 DisableBreak, EnableBreak;

 FROM Terminal IMPORT
 WriteString, WriteLn, Read;

 FROM Shell IMPORT
 dispOpts, Options, Dir, Connect, eXit, MainHelp;

 FROM PAD IMPORT
 Send, Receive;


 VAR
 Quit : BOOLEAN;
 ch : CHAR;


BEGIN (* main program *)
 DisableBreak; (* don't recognize Control-C *)
 WriteLn; WriteLn;
 WriteString ("Welcome to PCKermit -- Mainframe to Micro Communications");
 WriteLn;
 dispOpts;
 Quit := FALSE;

 REPEAT
 WriteLn; WriteLn;
 WriteString ("PCKermit [O, C, D, S, R, X, ?]: ");
 LOOP
 Read (ch);
 CASE CAP (ch) OF
 'O' : Options; EXIT;
 'C' : Connect; EXIT;
 'D' : Dir; EXIT;
 'S' : Send; EXIT;
 'R' : Receive; EXIT;
 'X' : eXit (Quit); EXIT;
 '?' : MainHelp; EXIT;
 ELSE
 (* ignore *)
 END;
 END;
 UNTIL Quit;
 EnableBreak;
END PCKermit.





[LISTING TWO]

DEFINITION MODULE Shell; (* User interface for Kermit *)

 EXPORT QUALIFIED
 dispOpts, Options, Dir, Connect, eXit, MainHelp;

 PROCEDURE dispOpts;
 (* Display communications parameters for the user *)

 PROCEDURE Options;
 (* set communications options *)

 PROCEDURE Dir;
 (* Displays a directory *)

 PROCEDURE Connect;
 (* Terminal mode allows connection to host (possibly through MODEM) *)

 PROCEDURE eXit (VAR q : BOOLEAN);
 (* Allow user to exit program after prompting for confirmation *)

 PROCEDURE MainHelp;
 (* help menu for main program loop *)

END Shell.






[LISTING THREE]



DEFINITION MODULE PAD; (* Packet Assembler/Disassembler for Kermit *)

 EXPORT QUALIFIED
 PacketType, yourNPAD, yourPADC, yourEOL, Send, Receive;

 TYPE
 (* PacketType used in both PAD and DataLink modules *)
 PacketType = ARRAY [1..100] OF CHAR;

 VAR
 (* yourNPAD, yourPADC, and yourEOL used in both PAD and DataLink *)
 yourNPAD : CARDINAL; (* number of padding characters *)
 yourPADC : CHAR; (* padding characters *)
 yourEOL : CHAR; (* End Of Line -- terminator *)

 PROCEDURE Send;
 (* Sends a file after prompting for filename *)

 PROCEDURE Receive;
 (* Receives a file (or files) *)

END PAD.






[LISTING FOUR]


DEFINITION MODULE Files; (* File I/O for Kermit *)

 FROM FileSystem IMPORT
 File;

 EXPORT QUALIFIED
 Status, FileType, Open, Create, CloseFile, Get, Put, DoWrite;

 TYPE
 Status = (Done, Error, EOF);
 FileType = (Input, Output);

 PROCEDURE Open (VAR f : File; name : ARRAY OF CHAR) : Status;
 (* opens an existing file for reading, returns status *)

 PROCEDURE Create (VAR f : File; name : ARRAY OF CHAR) : Status;
 (* creates a new file for writing, returns status *)

 PROCEDURE CloseFile (VAR f : File; Which : FileType) : Status;
 (* closes a file after reading or writing *)

 PROCEDURE Get (VAR f : File; VAR ch : CHAR) : Status;
 (* Reads one character from the file, returns status *)

 PROCEDURE Put (ch : CHAR);
 (* Writes one character to the file buffer *)


 PROCEDURE DoWrite (VAR f : File) : Status;
 (* Writes buffer to disk only if nearly full *)

END Files.






[LISTING FIVE]

DEFINITION MODULE DataLink; (* Sends and Receives Packets for PCKermit *)

 FROM PAD IMPORT
 PacketType;

 EXPORT QUALIFIED
 FlushUART, SendPacket, ReceivePacket;

 PROCEDURE FlushUART;
 (* ensure no characters left in UART holding registers *)

 PROCEDURE SendPacket (s : PacketType);
 (* Adds SOH and CheckSum to packet *)

 PROCEDURE ReceivePacket (VAR r : PacketType) : BOOLEAN;
 (* strips SOH and checksum -- return FALSE if timed out or bad checksum *)

END DataLink.






[LISTING SIX]


IMPLEMENTATION MODULE Shell; (* User interface for Kermit *)

 FROM SYSTEM IMPORT
 AX, BX, CX, DX, SETREG, SWI;

 FROM Exec IMPORT
 DosCommand;

 FROM Terminal IMPORT
 WriteString, WriteLn, KeyPressed, ReadString;

 IMPORT Terminal; (* for Terminal.Write and Terminal.Read *)

 FROM InOut IMPORT
 WriteCard;

 FROM RS232Int IMPORT
 Init, StartReading, StopReading;

 IMPORT RS232Int; (* for RS232Int.Write and RS232Int.BusyRead *)


 FROM Strings IMPORT
 Length, Concat;

 FROM NumberConversion IMPORT
 StringToCard;

 IMPORT ASCII;


 VAR
 baudRate : CARDINAL;
 stopBits : CARDINAL;
 parityBit : BOOLEAN;
 evenParity : BOOLEAN;
 nbrOfBits : CARDINAL;
 OK : BOOLEAN;
 echo : (Off, Local, On);
 ch : CHAR;
 str : ARRAY [0..10] OF CHAR;
 n : CARDINAL;


 PROCEDURE Initialize;
 BEGIN
 Init (baudRate, stopBits, parityBit, evenParity, nbrOfBits, OK);
 END Initialize;


 PROCEDURE ClrScr;
 (* Clear the screen, and home the cursor *)
 BEGIN
 SETREG (AX, 0600H); (* function 6 = scroll or clear window *)
 SETREG (BX, 0700H); (* 7 = normal screen attribute *)
 SETREG (CX, 0000H); (* top LH of screen *)
 SETREG (DX, 184FH); (* bottom RH of screen *)
 SWI (10H); (* call bios *)
 SETREG (AX, 0200h); (* function 2 = position cursor *)
 SETREG (BX, 0000H); (* page 0 *)
 SETREG (DX, 0000H); (* home position *)
 SWI (10H); (* call bios *)
 END ClrScr;


 PROCEDURE CommHelp;
 (* help menu for communications options *)
 BEGIN
 ClrScr;
 WriteString (" C o m m u n i c a t i o n s O p t i o n s");
 WriteLn;
 WriteString (" H e l p M e n u");
 WriteLn; WriteLn;
 WriteString ("set Baud rate ................................ B");
 WriteLn;
 WriteString ("set Parity ................................... P");
 WriteLn;
 WriteString ("set Word length .............................. W");
 WriteLn;
 WriteString ("set Stop bits ................................ S");

 WriteLn;
 WriteString ("eXit ......................................... X");
 WriteLn;
 END CommHelp;


 PROCEDURE dispOpts;
 (* Display communications parameters for the user *)
 BEGIN
 WriteLn;
 WriteString ("Baud rate = "); WriteCard (baudRate, 0);
 WriteString ("; ");
 IF parityBit THEN
 IF evenParity THEN
 WriteString ("Even ");
 ELSE
 WriteString ("Odd ");
 END;
 ELSE
 WriteString ("No ");
 END;
 WriteString ("parity; ");
 WriteCard (nbrOfBits, 0);
 WriteString (" Data bits; ");
 IF stopBits = 1 THEN
 WriteString ("One stop bit.");
 ELSE
 WriteString ("Two stop bits.");
 END;
 WriteLn;
 END dispOpts;


 PROCEDURE Options;
 (* set communications options *)

 VAR
 Quit : BOOLEAN;

 BEGIN
 ClrScr;
 Quit := FALSE;
 dispOpts;

 REPEAT
 WriteLn; WriteLn;
 WriteString ("Set Communications Options [B, P, W, S, X, ?]: ");
 LOOP
 Terminal.Read (ch);
 CASE CAP (ch) OF
 'B' : Baud; EXIT;
 'P' : Parity; EXIT;
 'W' : Word; EXIT;
 'S' : Stops; EXIT;
 '?' : CommHelp; EXIT;
 'X' : Quit := TRUE; EXIT;
 ELSE
 (* ignore *)
 END;

 END;
 IF Quit THEN
 ClrScr;
 ELSE
 Initialize;
 dispOpts;
 END;
 UNTIL Quit;
 END Options;


 PROCEDURE Baud;
 (* Allow user to change the bit rate of the communications port *)
 BEGIN
 WriteString ("Baud Rate? [110 - 9600]: ");
 ReadString (str);
 IF Length (str) # 0 THEN
 StringToCard (str, n, OK);
 IF OK THEN
 CASE n OF
 110, 150, 300, 600, 1200, 2400, 4800, 9600 : baudRate := n;
 ELSE
 (* do nothing *)
 END;
 END;
 END;
 END Baud;


 PROCEDURE Word;
 (* Allow user to change the word length of the communications port *)
 BEGIN
 WriteString ("Word Length? [7, 8]: ");
 ReadString (str);
 IF Length (str) # 0 THEN
 StringToCard (str, n, OK);
 IF OK AND (n IN {7, 8}) THEN
 nbrOfBits := n;
 END;
 END;
 END Word;


 PROCEDURE Parity;
 (* Allow user to change the parity bit of the communications port *)
 BEGIN
 WriteString ("Parity? [None, Even, Odd]: ");
 ReadString (str);
 IF Length (str) # 0 THEN
 CASE CAP (str[0]) OF
 'N' : parityBit := FALSE;
 'E' : parityBit := TRUE; evenParity := TRUE;
 'O' : parityBit := TRUE; evenParity := FALSE;
 ELSE
 (* no action *)
 END;
 END;
 END Parity;



 PROCEDURE Stops;
 (* Allow user to change the number of stop bits *)
 BEGIN
 WriteString ("Stop Bits? [1, 2]: ");
 ReadString (str);
 IF Length (str) # 0 THEN
 StringToCard (str, n, OK);
 IF OK AND (n IN {1, 2}) THEN
 stopBits := n;
 END;
 END;
 END Stops;


 PROCEDURE Dir;

 VAR
 done, gotFN : BOOLEAN;
 path : ARRAY [0..60] OF CHAR;
 filename : ARRAY [0..20] OF CHAR;
 i, j, k : INTEGER;

 BEGIN
 filename := ""; (* in case no directory change *)
 WriteString ("Path? (*.*): ");
 ReadString (path);
 i := Length (path);
 IF i # 0 THEN
 gotFN := FALSE;
 WHILE (i >= 0) AND (path[i] # '\') DO
 IF path[i] = '.' THEN
 gotFN := TRUE;
 END;
 DEC (i);
 END;
 IF gotFN THEN
 j := i + 1;
 k := 0;
 WHILE path[j] # 0C DO
 filename[k] := path[j];
 INC (k); INC (j);
 END;
 filename[k] := 0C;
 IF (i = -1) OR (i = 0) AND (path[0] = '\')) THEN
 INC (i);
 END;
 path[i] := 0C;
 END;
 END;
 IF Length (path) # 0 THEN
 DosCommand ("CHDIR", path, done);
 END;
 IF Length (filename) = 0 THEN
 filename := "*.*";
 END;
 Concat (filename, "/w", filename);
 ClrScr;
 DosCommand ("DIR", filename, done);

 END Dir;


 PROCEDURE ConnectHelp;
 (* provide help while in connect mode *)
 BEGIN
 ClrScr;
 WriteString ("LOCAL COMMANDS:"); WriteLn;
 WriteString ("^E = Echo mode"); WriteLn;
 WriteString ("^L = Local echo mode"); WriteLn;
 WriteString ("^T = Terminal mode (no echo)"); WriteLn;
 WriteString ("^X = eXit from connect"); WriteLn;
 WriteLn; WriteLn;
 END ConnectHelp;


 PROCEDURE Connect;
 (* Terminal mode allows connection to host (possibly through MODEM) *)

 VAR
 Input : BOOLEAN;

 BEGIN
 ConnectHelp;
 REPEAT
 RS232Int.BusyRead (ch, Input);
 IF Input THEN
 IF ((ch >= 40C) AND (ch < 177C))
 OR (ch = ASCII.cr) OR (ch = ASCII.lf) OR (ch = ASCII.bs) THEN
 Terminal.Write (ch);
 END;
 IF echo = On THEN
 RS232Int.Write (ch);
 END;
 END;

 IF KeyPressed() THEN
 Terminal.Read (ch);
 IF ch = ASCII.enq THEN (* Control-E *)
 echo := On;
 ELSIF ch = ASCII.ff THEN (* Control-L *)
 echo := Local;
 ELSIF ch = ASCII.dc4 THEN (* Control-T *)
 echo := Off;
 ELSIF ((ch >= 40C) AND (ch < 177C))
 OR (ch = ASCII.EOL) OR (ch = ASCII.bs) THEN
 IF ch = ASCII.EOL THEN
 RS232Int.Write (ASCII.cr);
 RS232Int.Write (ASCII.lf);
 ELSE
 RS232Int.Write (ch);
 END;
 IF (echo = On) OR (echo = Local) THEN
 Terminal.Write (ch);
 END;
 END;
 END;
 UNTIL ch = ASCII.can; (* Control-X *)
 END Connect;



 PROCEDURE eXit (VAR q : BOOLEAN);
 (* Allow user to exit program after prompting for confirmation *)
 BEGIN
 WriteString ("Exit PCKermit? [Y/N]: ");
 Terminal.Read (ch);
 IF CAP (ch) = 'Y' THEN
 Terminal.Write ('Y');
 StopReading; (* turn off the serial port *)
 q := TRUE;
 ELSE
 Terminal.Write ('N');
 END;
 WriteLn;
 END eXit;


 PROCEDURE MainHelp;
 (* help menu for main program loop *)
 BEGIN
 ClrScr;
 WriteString (" P C K e r m i t H e l p M e n u"); WriteLn;
 WriteLn;
 WriteString ("set communications Options ............. O");
 WriteLn;
 WriteString ("Connect to host ........................ C");
 WriteLn;
 WriteString ("Directory .............................. D");
 WriteLn;
 WriteString ("Send a file ............................ S");
 WriteLn;
 WriteString ("Receive a file ......................... R");
 WriteLn;
 WriteString ("eXit ................................... X");
 WriteLn; WriteLn;
 WriteString ("To establish connection to Host:"); WriteLn;
 WriteString (" -Use Connect Mode"); WriteLn;
 WriteString (" -Dial Host (AT command set?)"); WriteLn;
 WriteString (" -Log On to Host"); WriteLn;
 WriteString (" -Issue Send (or Receive) command"); WriteLn;
 WriteString (" -Return to main menu (^X)"); WriteLn;
 WriteString (" -Issue Receive (or Send) command"); WriteLn;
 WriteLn;
 END MainHelp;


BEGIN (* module initialization *)
 ClrScr;
 baudRate := 1200;
 stopBits := 1;
 parityBit := TRUE;
 evenParity := TRUE;
 nbrOfBits := 7;
 Initialize;
 StartReading; (* turn on the serial port *)
 echo := Off;
END Shell.







[LISTING SEVEN]

IMPLEMENTATION MODULE PAD; (* Packet Assembler/Disassembler for Kermit *)

 FROM InOut IMPORT
 Write, WriteString, WriteInt, WriteHex, WriteLn;

 FROM Terminal IMPORT
 ReadString, Read, KeyPressed;

 FROM Strings IMPORT
 Length;

 FROM BitByteOps IMPORT
 ByteXor;

 FROM FileSystem IMPORT
 File;

 FROM Files IMPORT
 Status, FileType, Open, Create, CloseFile, Get, Put, DoWrite;

 FROM DataLink IMPORT
 FlushUART, SendPacket, ReceivePacket;

 IMPORT ASCII;


 CONST
 myMAXL = 94;
 myTIME = 10;
 myNPAD = 0;
 myPADC = 0C;
 myEOL = 0C;
 myQCTL = '#';
 myQBIN = '&';
 myCHKT = '1'; (* one character checksum *)
 MAXtrys = 5;

 TYPE
 (* From Definition Module:
 PacketType = ARRAY [1..100] OF CHAR;
 *)
 PathnameType = ARRAY [0..40] OF CHAR;

 VAR
 yourMAXL : INTEGER; (* maximum packet length -- up to 94 *)
 yourTIME : INTEGER; (* time out -- seconds *)
 (* From Definition Module
 yourNPAD : INTEGER; (* number of padding characters *)
 yourPADC : CHAR; (* padding characters *)
 yourEOL : CHAR; (* End Of Line -- terminator *)
 *)
 yourQCTL : CHAR; (* character for quoting controls '#' *)

 yourQBIN : CHAR; (* character for quoting binary '&' *)
 yourCHKT : CHAR; (* check type -- 1 = checksum, etc. *)
 sF, rF : File; (* files being sent/received *)
 sFname, rFname : PathnameType;
 sP, rP : PacketType; (* packets sent/received *)
 sSeq, rSeq : INTEGER; (* sequence numbers *)
 PktNbr : INTEGER; (* actual packet number -- no repeats up to 32,000 *)


 PROCEDURE Char (c : INTEGER) : CHAR;
 (* converts a number 0-94 into a printable character *)
 BEGIN
 RETURN (CHR (CARDINAL (ABS (c) + 32)));
 END Char;


 PROCEDURE UnChar (c : CHAR) : INTEGER;
 (* converts a character into its corresponding number *)
 BEGIN
 RETURN (ABS (INTEGER (ORD (c)) - 32));
 END UnChar;


 PROCEDURE Aborted() : BOOLEAN;

 VAR
 ch : CHAR;

 BEGIN
 IF KeyPressed() THEN
 Read (ch);
 IF ch = 033C THEN (* Escape *)
 RETURN TRUE;
 END;
 END;
 RETURN FALSE;
 END Aborted;


 PROCEDURE TellError (Seq : INTEGER);
 (* Send error packet *)
 BEGIN
 sP[1] := Char (15);
 sP[2] := Char (Seq);
 sP[3] := 'E'; (* E-type packet *)
 sP[4] := 'R'; (* error message starts *)
 sP[5] := 'e';
 sP[6] := 'm';
 sP[7] := 'o';
 sP[8] := 't';
 sP[9] := 'e';
 sP[10] := ' ';
 sP[11] := 'A';
 sP[12] := 'b';
 sP[13] := 'o';
 sP[14] := 'r';
 sP[15] := 't';
 sP[16] := 0C;
 SendPacket (sP);

 END TellError;


 PROCEDURE ShowError (p : PacketType);
 (* Output contents of error packet to the screen *)

 VAR
 i : INTEGER;

 BEGIN
 FOR i := 4 TO UnChar (p[1]) DO
 Write (p[i]);
 END;
 WriteLn;
 END ShowError;


 PROCEDURE youInit (type : CHAR);
 (* I initialization YOU for Send and Receive *)
 BEGIN
 sP[1] := Char (11); (* Length *)
 sP[2] := Char (0); (* Sequence *)
 sP[3] := type;
 sP[4] := Char (myMAXL);
 sP[5] := Char (myTIME);
 sP[6] := Char (myNPAD);
 sP[7] := CHAR (ByteXor (myPADC, 100C));
 sP[8] := Char (ORD (myEOL));
 sP[9] := myQCTL;
 sP[10] := myQBIN;
 sP[11] := myCHKT;
 sP[12] := 0C; (* terminator *)
 SendPacket (sP);
 END youInit;


 PROCEDURE myInit;
 (* YOU initialize ME for Send and Receive *)

 VAR
 len : INTEGER;

 BEGIN
 len := UnChar (rP[1]);
 IF len >= 4 THEN
 yourMAXL := UnChar (rP[4]);
 ELSE
 yourMAXL := 94;
 END;
 IF len >= 5 THEN
 yourTIME := UnChar (rP[5]);
 ELSE
 yourTIME := 10;
 END;
 IF len >= 6 THEN
 yourNPAD := UnChar (rP[6]);
 ELSE
 yourNPAD := 0;
 END;

 IF len >= 7 THEN
 yourPADC := CHAR (ByteXor (rP[7], 100C));
 ELSE
 yourPADC := 0C;
 END;
 IF len >= 8 THEN
 yourEOL := CHR (UnChar (rP[8]));
 ELSE
 yourEOL := 0C;
 END;
 IF len >= 9 THEN
 yourQCTL := rP[9];
 ELSE
 yourQCTL := 0C;
 END;
 IF len >= 10 THEN
 yourQBIN := rP[10];
 ELSE
 yourQBIN := 0C;
 END;
 IF len >= 11 THEN
 yourCHKT := rP[11];
 IF yourCHKT # myCHKT THEN
 yourCHKT := '1';
 END;
 ELSE
 yourCHKT := '1';
 END;
 END myInit;


 PROCEDURE SendInit;
 BEGIN
 youInit ('S');
 END SendInit;


 PROCEDURE SendFileName;

 VAR
 i, j : INTEGER;

 BEGIN
 (* send file name *)
 i := 4; j := 0;
 WHILE sFname[j] # 0C DO
 sP[i] := sFname[j];
 INC (i); INC (j);
 END;
 sP[1] := Char (j + 3);
 sP[2] := Char (sSeq);
 sP[3] := 'F'; (* filename packet *)
 sP[i] := 0C;
 SendPacket (sP);
 END SendFileName;


 PROCEDURE SendEOF;
 BEGIN

 sP[1] := Char (3);
 sP[2] := Char (sSeq);
 sP[3] := 'Z'; (* end of file *)
 sP[4] := 0C;
 SendPacket (sP);
 END SendEOF;


 PROCEDURE SendEOT;
 BEGIN
 sP[1] := Char (3);
 sP[2] := Char (sSeq);
 sP[3] := 'B'; (* break -- end of transmit *)
 sP[4] := 0C;
 SendPacket (sP);
 END SendEOT;


 PROCEDURE GetAck() : BOOLEAN;
 (* Look for acknowledgement -- retry on timeouts or NAKs *)

 VAR
 Type : CHAR;
 Seq : INTEGER;
 retrys : INTEGER;
 AckOK : BOOLEAN;

 BEGIN
 WriteString ("Sent Packet #");
 WriteInt (PktNbr, 5);
 WriteString (" (ID: "); WriteHex (sSeq, 4);
 WriteString ("h)");
 WriteLn;

 retrys := MAXtrys;
 LOOP
 IF Aborted() THEN
 TellError (sSeq);
 RETURN FALSE;
 END;
 IF (ReceivePacket (rP)) THEN
 Seq := UnChar (rP[2]);
 Type := rP[3];
 IF (Seq = sSeq) AND (Type = 'Y') THEN
 AckOK := TRUE;
 ELSIF (Seq = (sSeq + 1) MOD 64) AND (Type = 'N') THEN
 AckOK := TRUE; (* NAK for (n + 1) taken as ACK for n *)
 ELSIF Type = 'E' THEN
 ShowError (rP);
 AckOK := FALSE;
 retrys := 0;
 ELSE
 AckOK := FALSE;
 END;
 ELSE
 AckOK := FALSE;
 END;
 IF AckOK OR (retrys = 0) THEN
 EXIT;

 ELSE
 WriteString ("Resending Packet #");
 WriteInt (PktNbr, 5);
 WriteString (" (ID: "); WriteHex (sSeq, 4);
 WriteString ("h)");
 WriteLn;
 DEC (retrys);
 FlushUART;
 SendPacket (sP);
 END;
 END;

 IF AckOK THEN
 INC (PktNbr);
 sSeq := (sSeq + 1) MOD 64;
 RETURN TRUE;
 ELSE
 RETURN FALSE;
 END;
 END GetAck;


 PROCEDURE GetInitAck() : BOOLEAN;
 (* configuration for remote station *)
 BEGIN
 IF GetAck() THEN
 myInit;
 RETURN TRUE;
 ELSE
 RETURN FALSE;
 END;
 END GetInitAck;


 PROCEDURE Send;
 (* Sends a file after prompting for filename *)

 VAR
 ch : CHAR;
 i : INTEGER;

 BEGIN
 WriteString ("Send: (filename?): ");
 ReadString (sFname);
 WriteLn;
 IF Length (sFname) = 0 THEN
 RETURN;
 END;
 IF Open (sF, sFname) # Done THEN
 WriteString ("No such file: "); WriteString (sFname);
 WriteLn;
 RETURN;
 END;
 WriteString ("(<ESC> to abort file transfer.)");
 WriteLn; WriteLn;
 FlushUART;
 sSeq := 0; PktNbr := 0;
 SendInit; (* my configuration information *)
 IF NOT GetInitAck() THEN (* get your configuration information *)

 WriteString ("Excessive Errors..."); WriteLn;
 RETURN;
 END;

 SendFileName;
 IF NOT GetAck() THEN
 WriteString ("Excessive Errors..."); WriteLn;
 RETURN;
 END;

 (* send file *)
 i := 4;
 LOOP
 IF Aborted() THEN
 TellError (sSeq);
 RETURN;
 END;
 IF Get (sF, ch) = EOF THEN (* send current packet & terminate *)
 sP[1] := Char (i - 1);
 sP[2] := Char (sSeq);
 sP[3] := 'D'; (* data packet *)
 sP[i] := 0C; (* indicate end of packet *)
 SendPacket (sP);
 IF NOT GetAck() THEN
 WriteString ("Excessive Errors..."); WriteLn;
 RETURN;
 END;
 SendEOF;
 IF NOT GetAck() THEN
 WriteString ("Excessive Errors..."); WriteLn;
 RETURN;
 END;
 SendEOT;
 IF NOT GetAck() THEN
 WriteString ("Excessive Errors..."); WriteLn;
 RETURN;
 END;
 EXIT;
 END;

 IF i >= (yourMAXL - 4) THEN (* send current packet *)
 sP[1] := Char (i - 1);
 sP[2] := Char (sSeq);
 sP[3] := 'D';
 sP[i] := 0C;
 SendPacket (sP);
 IF NOT GetAck() THEN
 WriteString ("Excessive Errors..."); WriteLn;
 RETURN;
 END;
 i := 4;
 END;

 (* add character to current packet -- update count *)
 IF ch > 177C THEN (* must be quoted (QBIN) and altered *)
 (* toggle bit 7 to turn it off *)
 ch := CHAR (ByteXor (ch, 200C));
 sP[i] := myQBIN; INC (i);
 END;

 IF (ch < 40C) OR (ch = 177C) THEN (* quote (QCTL) and alter *)
 (* toggle bit 6 to turn it on *)
 ch := CHAR (ByteXor (ch, 100C));
 sP[i] := myQCTL; INC (i);
 END;
 IF (ch = myQCTL) OR (ch = myQBIN) THEN (* must send it quoted *)
 sP[i] := myQCTL; INC (i);
 END;
 sP[i] := ch; INC (i);
 END; (* loop *)

 IF CloseFile (sF, Input) # Done THEN
 WriteString ("Problem closing source file..."); WriteLn;
 END;
 END Send;


 PROCEDURE ReceiveInit() : BOOLEAN;
 (* receive my initialization information from you *)

 VAR
 RecOK : BOOLEAN;
 errors : INTEGER;

 BEGIN
 errors := 0;
 LOOP
 IF Aborted() THEN
 TellError (rSeq);
 RETURN FALSE;
 END;
 RecOK := (ReceivePacket (rP)) AND (rP[3] = 'S');
 IF RecOK OR (errors = MAXtrys) THEN
 EXIT;
 ELSE
 INC (errors);
 SendNak;
 END;
 END;

 IF RecOK THEN
 myInit;
 RETURN TRUE;
 ELSE
 RETURN FALSE;
 END;
 END ReceiveInit;


 PROCEDURE SendInitAck;
 (* acknowledge your initialization of ME and send mine for YOU *)
 BEGIN
 WriteString ("Received Packet #");
 WriteInt (PktNbr, 5);
 WriteString (" (ID: "); WriteHex (rSeq, 4);
 WriteString ("h)");
 WriteLn;
 INC (PktNbr);
 rSeq := (rSeq + 1) MOD 64;

 youInit ('Y');
 END SendInitAck;


 PROCEDURE ValidFileChar (VAR ch : CHAR) : BOOLEAN;
 (* checks if character is one of 'A'..'Z', '0'..'9', makes upper case *)
 BEGIN
 ch := CAP (ch);
 RETURN ((ch >= 'A') AND (ch <= 'Z')) OR ((ch >= '0') AND (ch <= '9'));
 END ValidFileChar;


 TYPE
 HeaderType = (name, eot, fail);

 PROCEDURE ReceiveHeader() : HeaderType;
 (* receive the filename -- alter for local conditions, if necessary *)

 VAR
 i, j, k : INTEGER;
 RecOK : BOOLEAN;
 errors : INTEGER;

 BEGIN
 errors := 0;
 LOOP
 RecOK := ReceivePacket (rP) AND ((rP[3] = 'F') OR (rP[3] = 'B'));
 IF errors = MAXtrys THEN
 RETURN fail;
 ELSIF RecOK AND (rP[3] = 'F') THEN
 i := 4; (* data starts here *)
 j := 0; (* beginning of filename string *)
 WHILE (ValidFileChar (rP[i])) AND (j < 8) DO
 rFname[j] := rP[i];
 INC (i); INC (j);
 END;
 REPEAT
 INC (i);
 UNTIL (ValidFileChar (rP[i])) OR (rP[i] = 0C);
 rFname[j] := '.'; INC (j);
 k := 0;
 WHILE (ValidFileChar (rP[i])) AND (k < 3) DO
 rFname[j + k] := rP[i];
 INC (i); INC (k);
 END;
 rFname[j + k] := 0C;
 WriteString ("Filename = "); WriteString (rFname); WriteLn;
 RETURN name;
 ELSIF RecOK AND (rP[3] = 'B') THEN
 RETURN eot;
 ELSE
 INC (errors);
 SendNak;
 END;
 END;
 END ReceiveHeader;


 PROCEDURE SendNak;

 BEGIN
 WriteString ("Requesting Repeat of Packet #");
 WriteInt (PktNbr, 5);
 WriteString (" (ID: "); WriteHex (rSeq, 4);
 WriteString ("h)");
 WriteLn;
 FlushUART;
 sP[1] := Char (3); (* LEN *)
 sP[2] := Char (rSeq);
 sP[3] := 'N'; (* negative acknowledgement *)
 sP[4] := 0C;
 SendPacket (sP);
 END SendNak;


 PROCEDURE SendAck (Seq : INTEGER);
 BEGIN
 IF Seq # rSeq THEN
 WriteString ("Duplicate Packet ");
 ELSE
 WriteString ("Received Packet #"); WriteInt (PktNbr, 5);
 rSeq := (rSeq + 1) MOD 64;
 INC (PktNbr);
 END;
 WriteString (" (ID: "); WriteHex (Seq, 4);
 WriteString ("h)");
 WriteLn;
 sP[1] := Char (3);
 sP[2] := Char (Seq);
 sP[3] := 'Y'; (* acknowledgement *)
 sP[4] := 0C;
 SendPacket (sP);
 END SendAck;


 PROCEDURE Receive;
 (* Receives a file (or files) *)

 VAR
 ch, Type : CHAR;
 Seq : INTEGER;
 i : INTEGER;
 EOF, EOT, QBIN : BOOLEAN;
 errors : INTEGER;

 BEGIN
 WriteString ("Ready to receive file(s)..."); WriteLn;
 WriteString ("(<ESC> to abort file transfer.)");
 WriteLn; WriteLn;
 FlushUART;
 rSeq := 0; PktNbr := 0;
 IF NOT ReceiveInit() THEN (* your configuration information *)
 WriteString ("Excessive Errors..."); WriteLn;
 RETURN;
 END;
 SendInitAck; (* send my configuration information *)
 EOT := FALSE;
 WHILE NOT EOT DO
 IF Aborted() THEN

 TellError (rSeq);
 RETURN;
 END;
 CASE ReceiveHeader() OF
 eot : EOT := TRUE; EOF := TRUE;
 name : IF Create (rF, rFname) # Done THEN
 WriteString ("Unable to open file: ");
 WriteString (rFname); WriteLn;
 RETURN;
 ELSE
 PktNbr := 1;
 EOF := FALSE;
 END;
 fail : WriteString ("Excessive Errors..."); WriteLn;
 RETURN;
 END;
 SendAck (rSeq); (* acknowledge for name or eot *)
 WHILE NOT EOF DO
 IF Aborted() THEN
 TellError (rSeq);
 RETURN;
 END;
 IF ReceivePacket (rP) THEN
 Seq := UnChar (rP[2]);
 Type := rP[3];
 IF Type = 'Z' THEN
 EOF := TRUE;
 IF CloseFile (rF, Output) # Done THEN
 WriteString ("Error closing file: ");
 WriteString (rFname); WriteLn;
 RETURN;
 END;
 SendAck (rSeq);
 ELSIF Type = 'E' THEN
 ShowError (rP);
 RETURN;
 ELSIF (Type = 'D') AND ((Seq + 1) MOD 64 = rSeq) THEN
 (* discard duplicate packet, and Ack anyway *)
 SendAck (Seq);
 ELSIF (Type = 'D') AND (Seq = rSeq) THEN
 (* put packet into file buffer *)
 i := 4; (* first data in packet *)
 WHILE rP[i] # 0C DO
 ch := rP[i]; INC (i);
 IF ch = yourQBIN THEN
 ch := rP[i]; INC (i);
 QBIN := TRUE;
 ELSE
 QBIN := FALSE;
 END;
 IF ch = yourQCTL THEN
 ch := rP[i]; INC (i);
 IF (ch # yourQCTL) AND (ch # yourQBIN) THEN
 ch := CHAR (ByteXor (ch, 100C));
 END;
 END;
 IF QBIN THEN
 ch := CHAR (ByteXor (ch, 200C));
 END;

 Put (ch);
 END;

 (* write file buffer to disk *)
 IF DoWrite (rF) # Done THEN
 WriteString ("Error writing to file: ");
 WriteString (rFname); WriteLn;
 RETURN;
 END;
 errors := 0;
 SendAck (rSeq);
 ELSE
 INC (errors);
 IF errors = MAXtrys THEN
 WriteString ("Excessive errors..."); WriteLn;
 RETURN;
 ELSE
 SendNak;
 END;
 END;
 ELSE
 INC (errors);
 IF errors = MAXtrys THEN
 WriteString ("Excessive errors..."); WriteLn;
 RETURN;
 ELSE
 SendNak;
 END;
 END;
 END;
 END;
 END Receive;


BEGIN (* module initialization *)
 yourEOL := ASCII.cr;
 yourNPAD := 0;
 yourPADC := 0C;
END PAD.






[LISTING EIGHT]


IMPLEMENTATION MODULE Files; (* File I/O for Kermit *)

 FROM FileSystem IMPORT
 File, Response, Delete, Lookup, Close, ReadNBytes, WriteNBytes;

 FROM InOut IMPORT
 Read, WriteString, WriteLn, Write;

 FROM SYSTEM IMPORT
 ADR, SIZE;



 TYPE
 buffer = ARRAY [1..512] OF CHAR;

 VAR
 inBuf, outBuf : buffer;
 inP, outP : CARDINAL; (* buffer pointers *)
 read, written : CARDINAL; (* number of bytes read or written *)
 (* by ReadNBytes or WriteNBytes *)


 PROCEDURE Open (VAR f : File; name : ARRAY OF CHAR) : Status;
 (* opens an existing file for reading, returns status *)
 BEGIN
 Lookup (f, name, FALSE);
 IF f.res = done THEN
 inP := 0; read := 0;
 RETURN Done;
 ELSE
 RETURN Error;
 END;
 END Open;


 PROCEDURE Create (VAR f : File; name : ARRAY OF CHAR) : Status;
 (* creates a new file for writing, returns status *)

 VAR
 ch : CHAR;

 BEGIN
 Lookup (f, name, FALSE); (* check to see if file exists *)
 IF f.res = done THEN
 Close (f);
 WriteString ("File exists! Overwrite? (Y/N): ");
 Read (ch); Write (ch); WriteLn;
 IF CAP (ch) = 'Y' THEN
 Delete (name, f);
 Close (f);
 ELSE
 RETURN Error;
 END;
 END;
 Lookup (f, name, TRUE);
 IF f.res = done THEN
 outP := 0;
 RETURN Done;
 ELSE
 RETURN Error;
 END;
 END Create;


 PROCEDURE CloseFile (VAR f : File; Which : FileType) : Status;
 (* closes a file after reading or writing *)
 BEGIN
 written := outP;
 IF (Which = Output) AND (outP > 0) THEN
 WriteNBytes (f, ADR (outBuf), outP, written);

 END;
 Close (f);
 IF (written = outP) AND (f.res = done) THEN
 RETURN Done;
 ELSE
 RETURN Error;
 END;
 END CloseFile;


 PROCEDURE Get (VAR f : File; VAR ch : CHAR) : Status;
 (* Reads one character from the file, returns status *)
 BEGIN
 IF inP = read THEN
 ReadNBytes (f, ADR (inBuf), SIZE (inBuf), read);
 inP := 0;
 END;
 IF read = 0 THEN
 RETURN EOF;
 ELSE
 INC (inP);
 ch := inBuf[inP];
 RETURN Done;
 END;
 END Get;


 PROCEDURE Put (ch : CHAR);
 (* Writes one character to the file buffer *)
 BEGIN
 INC (outP);
 outBuf[outP] := ch;
 END Put;


 PROCEDURE DoWrite (VAR f : File) : Status;
 (* Writes buffer to disk only if nearly full *)
 BEGIN
 IF outP < 400 THEN (* still room in buffer *)
 RETURN Done;
 ELSE
 WriteNBytes (f, ADR (outBuf), outP, written);
 IF (written = outP) AND (f.res = done) THEN
 outP := 0;
 RETURN Done;
 ELSE
 RETURN Error;
 END;
 END;
 END DoWrite;

END Files.






[LISTING NINE]



IMPLEMENTATION MODULE DataLink; (* Sends and Receives Packets for PCKermit *)

 FROM InOut IMPORT
 WriteString, WriteLn;

 FROM Delay IMPORT
 Delay; (* delay is in milliseconds *)

 FROM BitByteOps IMPORT
 ByteAnd;

 IMPORT RS232Int; (* for RS232Int.BusyRead, RS232Int.Write *)

 FROM PAD IMPORT
 PacketType, yourNPAD, yourPADC, yourEOL;

 IMPORT ASCII;


 CONST
 MAXtime = 10000;
 MAXsohtrys = 100;

 VAR
 ch : CHAR;
 GotChar : BOOLEAN;


 PROCEDURE Char (c : INTEGER) : CHAR;
 (* converts a number 0-95 into a printable character *)
 BEGIN
 RETURN (CHR (CARDINAL (ABS (c) + 32)));
 END Char;


 PROCEDURE UnChar (c : CHAR) : INTEGER;
 (* converts a character into its corresponding number *)
 BEGIN
 RETURN (ABS (INTEGER (ORD (c)) - 32));
 END UnChar;


 PROCEDURE FlushUART;
 (* ensure no characters left in UART holding registers *)
 BEGIN
 Delay (500);
 REPEAT
 RS232Int.BusyRead (ch, GotChar);
 UNTIL NOT GotChar;
 END FlushUART;


 PROCEDURE SendPacket (s : PacketType);
 (* Adds SOH and CheckSum to packet *)

 VAR
 i : INTEGER;

 checksum : INTEGER;

 BEGIN
 Delay (10); (* give host a chance to catch its breath *)
 FOR i := 1 TO yourNPAD DO
 RS232Int.Write (yourPADC);
 END;
 RS232Int.Write (ASCII.soh);
 i := 1;
 checksum := 0;
 WHILE s[i] # 0C DO
 INC (checksum, ORD (s[i]));
 RS232Int.Write (s[i]);
 INC (i);
 END;
 checksum := checksum + (INTEGER (BITSET (checksum) * {7, 6}) DIV 64);
 checksum := INTEGER (BITSET (checksum) * {5, 4, 3, 2, 1, 0});
 RS232Int.Write (Char (checksum));
 IF yourEOL # 0C THEN
 RS232Int.Write (yourEOL);
 END;
 END SendPacket;


 PROCEDURE ReceivePacket (VAR r : PacketType) : BOOLEAN;
 (* strips SOH and checksum -- return FALSE if timed out or bad checksum *)

 VAR
 sohtrys, time : INTEGER;
 i, len : INTEGER;
 ch : CHAR;
 checksum : INTEGER;
 mycheck, yourcheck : CHAR;

 BEGIN
 sohtrys := MAXsohtrys;
 REPEAT
 time := MAXtime;
 REPEAT
 DEC (time);
 RS232Int.BusyRead (ch, GotChar);
 UNTIL GotChar OR (time = 0);
 ch := CHAR (ByteAnd (ch, 177C)); (* mask off MSB *)
 (* skip over up to MAXsohtrys padding characters, *)
 (* but allow only MAXsohtrys/10 timeouts *)
 IF GotChar THEN
 DEC (sohtrys);
 ELSE
 DEC (sohtrys, 10);
 END;
 UNTIL (ch = ASCII.soh) OR (sohtrys <= 0);

 IF ch = ASCII.soh THEN
 (* receive rest of packet *)
 time := MAXtime;
 REPEAT
 DEC (time);
 RS232Int.BusyRead (ch, GotChar);
 UNTIL GotChar OR (time = 0);

 ch := CHAR (ByteAnd (ch, 177C));
 len := UnChar (ch);
 r[1] := ch;
 checksum := ORD (ch);
 i := 2; (* on to second character in packet -- after LEN *)
 REPEAT
 time := MAXtime;
 REPEAT
 DEC (time);
 RS232Int.BusyRead (ch, GotChar);
 UNTIL GotChar OR (time = 0);
 ch := CHAR (ByteAnd (ch, 177C));
 r[i] := ch; INC (i);
 INC (checksum, (ORD (ch)));
 UNTIL (i > len);
 time := MAXtime;
 REPEAT
 DEC (time);
 RS232Int.BusyRead (ch, GotChar);
 UNTIL GotChar OR (time = 0); (* get checksum character *)
 ch := CHAR (ByteAnd (ch, 177C));
 yourcheck := ch;
 r[i] := 0C;
 checksum := checksum +
 (INTEGER (BITSET (checksum) * {7, 6}) DIV 64);
 checksum := INTEGER (BITSET (checksum) * {5, 4, 3, 2, 1, 0});
 mycheck := Char (checksum);
 IF mycheck = yourcheck THEN (* checksum OK *)
 RETURN TRUE;
 ELSE (* ERROR!!! *)
 WriteString ("Bad Checksum"); WriteLn;
 RETURN FALSE;
 END;
 ELSE
 WriteString ("No SOH"); WriteLn;
 RETURN FALSE;
 END;
 END ReceivePacket;

END DataLink.






















May, 1989
 LANGUAGE-INDEPENDENT DYNAMIC PSEUDOSTRUCTURES


Simple data conversions can pay big dividends




Bruce W. Tonkin


Bruce develops and sells software for TRS-80 and MS-DOS/PC-DOS computers. You
may reach him at T.N.T. Software Inc., 34069 Hainesville Rd., Round Lake, IL
60073.


A structure may be defined as a grouping of data items (characters, numbers,
pointers, or other types) that can be treated as an entity by the programmer.
Structures (or the largely equivalent Pascal records and Basic user-defined
types) are enormously useful because they simplify and generalize the
underlying program logic. Without structures, programs that could have been
written cleanly and elegantly must be written with more of a "brute force"
approach.
Still, a structure is commonly limited in one important way: The structure's
components (and even the order of those components) must be defined at compile
time, not at run time. This shortcoming often causes problems.
Consider, for example, a general sort program. The program should be able to
handle an arbitrary number of keys, to work with all reasonable data types of
whatever lengths, and to sort files far larger than available memory. Because
the amount of available memory is not known in advance and will vary, the sort
should use the fastest comparison algorithm(s) possible for the sake of
efficiency.
If the programmer knew in advance about the order and the types of the keys in
the file to be sorted, a structure that contains all of the keys would be easy
to define. A comparison algorithm could then be written to take full advantage
of the structure, and space could be allocated for as large an array as
necessary. When no formal structure can be defined in advance, that approach
is not possible.
In practice, most programmers allocate a block of memory at run time instead.
That block of memory will be filled with the data to be sorted (all or part of
each record). Another block of memory may hold pointers to the collection of
keys for each of the records. Individual key values that belong to each record
can be accessed by a fixed offset from that value.
In effect, the programmer gets the effect of a structure, without many of its
benefits. To see how severe this disadvantage can become, consider the
pseudocode for a sort on K keys in Example 1. The actual sorting algorithm is
not important, so I've kept things as general as possible. I've assumed that
two records are compared at some point, but that the action taken (swapping
pointers, moving memory, and so on) does not matter.
Example 1: Pseudocode for a sort on K keys

 loop while still records to sort
 get new record
 set x=1
 loop while x <= K
 determine the data type of key x
 call the appropriate comparison
 compare record a to record b on key x
 if record a is less than record b
 return result 'less than'
 else if record a is greater than record b
 return result 'greater than'
 else return result 'equal'
 end if
 if result is 'less than'
 take action and set x=K+1
 else if result is 'greater than'
 take action and set x=K+1
 else increment x
 end if
 end loop (compare keys)
 end loop (sort records)

For each key that is compared, the program looks up the data type and then
calls the correct comparison. The comparison is passed either the two values
or else the pointers to those values, and will probably return an integer
value that will be examined in order to determine what action should be
subsequently taken. If necessary, the process is repeated for each key.
If the structure is known in advance, only a pointer to the structure needs to
be passed to the comparison algorithm (or the comparison can be done inline).
The comparison does not need to look up the type of each key. Substantially
fewer parameters need to be passed.
Suppose that there are ten 1-byte keys for two records, and that all but the
last key are identical. In this case, the general method will require ten type
comparisons, ten calls with two passed parameters and a return value, and nine
incremented key pointers to compare each of two records.
In this scenario, a specific solution will pass two pointers, with one pointer
passed to each of the two key structure members. Assuming that each key is of
a different type, the ten type comparisons can be omitted. If all keys are of
the same type, then nine calls (with two passed parameters and a return value)
can also be omitted. Furthermore, if all keys are either a character byte or
an unsigned byte, there is no need to explicitly increment any key pointers.
Although it is not likely that all keys will be either a character byte or an
unsigned byte, it is common for two or more adjacent keys to be one of these
bytes. The resulting smaller overhead can offer a substantial advantage, since
many sorts require a large number of comparisons -- and, therefore, a large
number of passed parameters. If one or more merge passes are required, any
advantage is further magnified because even more comparisons will occur.


Converting to a Common Format


Nearly every modern processor contains a string comparison instruction, or at
least a small set of instructions that can perform a string comparison at high
speed. This capability is an obvious optimization, and all common compilers
take advantage of it.
No common CPU-level instruction for the comparison of floating-point numbers,
long integers, or other special data types exists. Such comparison routines
are supplied by the compiler vendor. For some types of data, the routines can
be quite slow.
To take advantage of efficient string comparisons, an ideal general-purpose
sort might convert all data into ASCII string format. If that conversion can
be made faster than the sum of 1) the type comparisons, 2) the additional
calls to the comparison algorithms (and the corresponding stack usage), and 3)
the time required to increment the key pointers (in this example, x=K+1), then
the sort will run faster.

Let me emphasize that I'm not talking about converting floating-point numbers
into strings of digits. That would waste memory and be time-consuming. By
"converting to string format," I mean something much simpler--converting the
various data types into a string of characters, so that when the characters
are sorted in ASCII order, the data from which the string was derived is also
sorted.
As an example of the technique, consider a 16-bit signed integer stored on an
IBM PC-type machine. In that format, the most significant byte is on the
right, and the sign is stored in the high-order bit (on = negative number) of
that byte.
If integers were stored with the bytes reversed and with the sign bit
complemented, then a string comparison would work. An ASCII-order sort of the
converted data would create the same relative order as an integer sort.
There wouldn't be any merit to the conversion if the sort key were a single
integer field, because integer comparisons are fast. If the sort were required
to process several integer keys per record, however, there might well be an
advantage. The conversion would take time, but afterwards, any comparisons
would require only two parameters. The more comparisons, keys, and different
data types, the stronger the incentive to convert.
If the data is converted, then the original data (or some kind of pointer to
the original data) needs to be retained in order to output a sorted file at
the end. In most cases, that step is not necessary--only the original record
number is needed in order to output an index to the sorted records. Note, too,
that if the record number is converted and included as the last sort key, and
then unconverted at the end of the sort, the sort preserves the order of the
original data where possible. This can be important when a file is read
through an index, because disk head movement and drive wear are minimized, and
throughput is improved. Whether or not this conversion is performed, the step
of appending the record number to each collection of keys makes it impossible
for any records to be "equal," and makes the sort logic even faster.


The Number of Comparisons


In most sorts, all but a few records are compared to another record at least
twice (more if a merge pass occurs). Computational and hash-based sorts can
beat this process, but only at the price of added complexity and additional
assumptions about the data. Often, there are far more comparisons: A bubble
sort of N records makes N-1 comparisons on the first pass alone, and can (in
the worst case) make N(N-1)/2 comparisons before it finishes. That is an
average of (N-1)/2 comparisons per record!
A sort that uses a binary search technique makes approximately log(N)
comparisons to insert the Nth record, log (N-1) comparisons to insert the
N-1st record, and log(1) comparisons to insert the first record. (All
logarithms in this article are to the base 2.) For files of more than eight
records, sorts based on a binary search average more than two comparisons per
record. As a rough approximation, a sort of N records based on a binary search
requires an average of log(N)-1 comparisons per record. For a larger number of
records (from about 20 to at least 100,000), a better approximation is
log(.737N)-1. Table 1 lists an approximate number of comparisons (computed by
summing the logs of all values of N to 16 significant digits) and the improved
estimate. Listing One shows the program that was used to generate the table.
Table 1: Approximate number of comparisons per record for a sort using a
binary search

 Number of Comparisons Log(.737N)-1
 Records Per Record
 ------------------------------------

 1 0 -1.4402630
 2 0.5 -0.4402636
 3 0.861654 0.1446990
 4 1.146241 0.5597370
 5 1.381378 0.8816650
 6 1.581976 1.1446996
 7 1.757030 1.3670914
 8 1.912401 1.5597370
 9 2.052126 1.7296623
 10 2.179106 1.8816653
 100 5.247650 5.2035935
 1000 8.529398 8.5255217
 2000 9.526494 9.5255217
 3000 10.110419 10.110483
 4000 10.524916 10.525521
 5000 10.846511 10.847449
 6000 11.109319 11.110483
 7000 11.331546 11.332876
 8000 11.524065 11.525521
 9000 11.693891 11.695446
 10000 11.845814 11.847449
 20000 12.845441 12.847449
 30000 13.430272 13.432411
 40000 13.845242 13.847449
 50000 14.167128 14.169377
 6000 14.430134 14.432411
 70000 14.652506 14.654804
 80000 14.845136 14.847449
 90000 15.015049 15.017374

If there are N records, K keys per record, and all keys need to be compared in
order to resolve matters, then a very good sort needs to make about 2KN
individual key comparisons. A bubble sort could require about (N/2)KN
comparisons. A sort based on a binary search needs roughly [log(.737N)-1]KN
comparisons. Table 2 may make the results easier to comprehend.
Table 2: Number of comparisons for a complete sort (based on three keys per
record)

 # Records Very Good Sort Bubble Sort Binary Sort
 ---------------------------------------------------------

 16 48 360 123
 128 768 24,384 2,135
 1,024 6,144 1,571,328 26,296
 16,384 98,304 402,628,608 617,336
 131,072 786,432 25,769,607,168 6,118,339

 1,048,576 6,291,456 1,649,265,868,800 58,383,894

An attempt to sort a million records with a bubble sort would be a ridiculous
exercise. If each comparison took 100 nanoseconds, the sort would take nearly
two days, even with no other overhead. The binary search method would finish
in about six seconds!


Sort Overhead


In practice there is a lot of overhead. The actual comparison involves memory
accesses, register increment and decrement operations, stack usage, and
various other instruction processing.
The relative time required to perform comparisons of different kinds varies by
compiler, processor, data type, and possibly the memory locations involved and
the memory model used. String comparisons are usually among the fastest
available comparisons. (Integer-to-integer or byte-to-byte comparisons are
normally the fastest.) That being the case, it is difficult to state precisely
what advantage will be gained by converting raw data into strings. The only
possible answer is, "it depends."
Generally speaking, though, if a string comparison is even slightly faster
than an explicit comparison, then there is a point after which it becomes more
efficient to convert. This point is reached when the number of comparisons
vastly outweighs the number of conversions necessary for any reasonably large
file.
Remember that conversion saves some time when comparisons are done. If nothing
else, the fact that fewer separate calls are made, and fewer parameters are
passed, makes the string comparison more efficient. This efficiency increases
as more keys are added --this factor is quite important for a general-purpose
sort. Table 3 shows a sort of 10,000 records on two keys, where the key-by-key
sort calls a comparison procedure.
Table 3: Operations required for a two-key sort of 10,000 records

 Conversion Sort Key-by-key Sort
 ---------------------------------------------------

 Conversions 20,000 0
 Comparisons 118,458 118,458 to 236,916
 Type checks 20,000 one per comparison
 Procedure calls 0 one per comparison

From this, it's clear that a key-by-key sort suffers in performance unless a
relatively long time is needed in order to perform a conversion, few keys and
few records exist, and the time required to do type checks and procedure calls
is negligible. As we shall see, none of these conditions are likely to hold.
Regardless of the amount of time saved, the savings in code complexity are
very real and obvious. Once the data is converted to string format, all
subsequent comparisons are string-to-string, and are fast and easy to
optimize. The savings in complexity can be even easier to appreciate if one or
more of the keys must be sorted in descending order.
When a full "brute force" method is used to compare each key individually, a
general sort must accommodate an extra flag for each key. The sort must also
invert the result returned by the affected comparison algorithm, or else
supply a different comparison (doubling the number of algorithms and
lengthening the sort time) in order to return a correct result.
If the data is converted to strings, then a simple and fast XOR (with decimal
255) can be performed to invert the bits in each affected key. This step
allows the use of a straight string comparison thereafter. Example 2
illustrates some pseudocode for a sort that uses conversion.
Example 2: Pseudocode for a sort that uses conversion

 loop while still records to sort
 get new record
 convert keys for new record and append record number
 compare record a to record b
 if record a is less than record b
 do appropriate action
 else do something else
 end if
 end loop (sort records)

The bare logic is certainly simpler than the previous example, but the sort
depends heavily upon the step of data conversion. In order to obtain the best
performance, the conversion routines should be written in assembler.
I've written a set of such assembler routines for Microsoft data types. The
routines are in Listing Two (CONVINT, to convert integers), Listing Three
(CONVLONG, to convert long integers), Listing Four (CONVOF, to convert old
floating-points), Listing Five (CONVNF, to convert IEEE floating-points), and
Listing Six (INVERT, to XOR all bytes in a key with ASCII 255 for
reverse-order sorts). Each of these conversion routines alters the actual data
in memory. The size of the key data does not change.
Though the routines were written for use with Quick Basic 4.0, there should be
little problem in altering them for use with other languages. A word of
warning: the converted data may contain ASCII null characters, which will
cause problems in the standard C strcmp library routine. C programmers should
use a different (perhaps assembler-based) string comparison.


Test Data in Support of Conversion


To test the worth of the conversion approach, I wrote a simple series of
benchmarks in Quick Basic 4.5 and ran the resulting stand-alone .EXE file on
my Tandy 4000 (80386, 16 MHz, no math co-processor). The benchmark program
(shown in Listing Seven) is tedious but straightforward: It consists of a
series of timing loops for each of a number of elementary operations. The time
required to perform a bare loop was subtracted from each test. The times are
presented in seconds per 100,000 operations, and averaged over five runs. The
results are shown in Table 4.
Table 4: Benchmark results

 TEST RESULTS
 Operation Test Time (sec.)
 -------------------------------------------------

 Raw integer loop 0.11
 Integer conversion 0.88
 Long integer conversion 1.05
 Old single float conversion 2.33
 Old double float conversion 3.01
 IEEE single float conversion 1.98

 IEEE double float conversion 3.66
 Invert 8 bytes 2.27
 Compare two-byte strings 3.00
 Compare two integers 0.34
 Compare two four-byte strings 3.13
 Compare two single floats 39.73
 Compare two double floats 40.78
 Compare two 8-byte strings 3.40
 Compare two long integers 1.33
 Compare two 20-byte strings 4.22
 Compare two 20-byte strings (best?) 2.91
 Three-integer-parameter call{*} 2.26

 {*} Dummy call with three parameters passed and one
 integer (constant) assignment in the subprogram.
 All string comparisons were for strings that differed
 in only the final byte, except for the "20-byte (best?)"
 operation, where the strings differed in the first byte.

From this table, there seems to be only one case where data should not be
converted: when the key or keys are integers or long integers. As we shall
see, that's not necessarily true.
If the sort ever includes any floating-point keys, then a conversion is nearly
mandatory--the time required for a conversion and a string comparison ranges
from 5.11 to 7.06 seconds per 100,000 operations, while the time required for
a floating-point comparison alone (even coded in-line) can be nearly eight
times as large.


Some Examples


Other combinations are not so clear-cut, but are more instructive. If an
integer comparison is not coded in-line (meaning that the comparison is a
procedure call), then it takes at least 2.60 seconds per 100,000 operations
(2.26 seconds for the procedure calls). 100,000 conversions and 100,000
comparisons take 3.88 seconds. As a result, if the key is a single integer,
you might think it better not to convert. Again, though, the sheer number of
comparisons may dominate.
Suppose that you need to sort 100,000 records, and you intend to use a method
that relies upon a binary search. If there are two integer keys, then you will
need to perform 200,000 conversions and about 1,516,704 comparisons, which
together will take 47.26 seconds. A simple integer comparison on the first key
(not coded in-line) will take at least 39.43 seconds. If the first keys are
always identical, then a second integer comparison will take another 39.43
seconds. If the second comparison is required only 19.9 percent of the time,
then the two methods will be equally fast. If the second comparison is needed
more often, then the conversion method will save time.
These times are quite conservative. For unconverted integer variables, the
actual comparison operation would never be as simple as shown in the
benchmarks--rather than only one logical operation, there could be two
(compare for less than, compare for greater than, and drop through for
equality). The number of necessary operations depends upon the data. If two
logical comparisons are necessary for each integer key (all data is in exactly
the wrong order for the first logical test), then the simple comparisons will
take an additional 5.16 seconds. This additional time makes the process of
conversion and string comparisons nearly equivalent to an integer comparison,
even for a single key. In addition, each unconverted comparison requires a
type check, which is not necessary for converted data. The type check will
take at least another 5.16 seconds (one logical operation per comparison), and
possibly make a sort on a single integer key faster if the data is converted
into string format first.
From the results determined in the previous table, it seems that a string
comparison takes about 2.9 seconds, plus approximately .066 seconds per byte
(per 100,000 comparisons). No single data comparison takes less time than the
time required per byte, whether or not the comparison is coded in-line. The
greater the number of bytes of keys, and the more comparisons performed, the
more efficient a conversion to ASCII strings will be.
To see how beneficial conversion could be, consider the case of ten integer
keys. Assume that the tenth key is always necessary, and that there are
100,000 records to sort. I'll compare a sort that uses data conversion against
a sort that is coded to use in-line comparisons, no type checks, and two
logical integer comparisons per key (no fair testing for key equality first).
In other words, I'll test a hard-coded special sort written specifically for
the integer data against a general sort that uses data conversion.
The conversion sort requires 1,000,000 conversions (time: 8.8 seconds),
1,000,000 type checks (time: 3.40 seconds, if the integer test is first), and
about 1,516,704 20-byte string comparisons (64.0 seconds) for a total of 76.20
seconds. The hard-coded integer sort requires no conversions or type checks,
but performs 30,334,084 logical comparisons (two comparisons per key for each
of the ten keys for each record comparison) for a total of 103.1
seconds--about 62 percent more time. If the integer sort had used procedure
calls, the sort time would have been increased by the performance of an
additional 15,167,040 procedure calls (one call per key per comparison) to a
total of 445.9 seconds--nearly seven times as long as the time required with
conversion!
These results are interesting because they show that:
1. Sort times are greatly dominated by the time required to perform a
comparison;
2. Sorts that use data conversion can be more efficient at comparison because
string comparisons are faster, byte for byte;
3. Sorts that do not use conversion must avoid procedure calls; but if
procedure calls are avoided, all comparisons (at each key level) must be coded
in-line, which greatly increases code size and complexity;
4. General-purpose sorts that don't use conversion must determine the type of
each key each time that a comparison is invoked. Sorts that do use conversion
must determine the type of each key only once, when the data is converted.


Conclusions


Data conversion can pay rather large dividends, especially for a
general-purpose sort, but also for any other program that must compare data in
a variety of formats. If procedure calls are used to perform a comparison, any
algorithm that does not use conversion might be (at best) only slightly faster
than an algorithm that does use conversion. If there are multiple comparisons
per record, as well as procedure calls for each comparison, it is virtually
certain that conversion will be faster. In any case that involves the
comparison of floating-point formats, the process of conversion plus a string
comparison will be far faster than a straight floating-point comparison.
Apparently, compiler writers have missed a chance to optimize floating-point
comparisons. (Borland's Turbo Basic floating-point and string comparison times
are not especially different from those of Microsoft's QuickBasic, and the
long-integer comparison times are much longer). I hope that future Basic
compilers offer more optimization, but that won't change the overall value of
conversions when multiple keys are used--string comparison operations are just
too fast per byte.
Try extending these results and conversion routines to other languages,
non-Intel-based machines, and to other data types. I'd be interested to hear
about any cases where data conversion does not prove effective in a general
sort.


Availability


All source code for articles in this issue is available on a single disk. To
order, send $14 95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063, or call 800-356-2002 (from inside
Calif.) or 800-533-4372 (from outside Calif.). Please specify the issue number
and format (MS-DOS, Macintosh, Kaypro).


_Language-Independent Dynamic Pseudostructures_
by Bruce Tonkin


[LISTING ONE]



DEFDBL A-Z
DIM i AS LONG
k = 1# / LOG(2#) 'convert to base 2
OPEN "o", 1, "c:temp"
FOR i = 1 TO 100000
 t = t + LOG(i) * k
 IF i MOD 10000 = 0 OR i < 11 OR i = 100 OR
 (i < 10000 AND i MOD 1000 = 0) THEN
 PRINT #1, i, t / i, (LOG(i * .737) * k) - 1
NEXT i






[LISTING TWO]

.MODEL MEDIUM
.CODE
PUBLIC convint
;convint written by Bruce W. Tonkin on 8-14-88 for use with QB 4.0 & MASM 5.0
;convint will convert a binary 2-byte integer passed as a string into
;a string with the bytes re-ordered so an ASCII-order sort will sort in
;numeric order. It is called with:
;call convint (x$)
;where x$ is the string to convint (in the current data segment).
;The routine does not check for a zero length of the passed string.
convint PROC
 push bp ;save old BP
 mov bp,sp ;Set framepointer to old stack
 mov bx,[bp+6] ;bx points to string length, which we don't need
 mov bx,[bx]+2 ;move the string address to bx
 mov dh,byte ptr [bx] ;get first byte
 inc bx ;point to next byte
 mov dl,byte ptr [bx] ;get second byte
 xor dl,080h ;invert the sign bit
 mov byte ptr [bx],dh ;store first byte where second was
 dec bx ;now for modified second byte
 mov byte ptr [bx],dl ;store where first byte went
 pop bp ;restore old base pointer
 ret 2 ;clear 2 bytes of parameters on return
convint ENDP
 END






[LISTING THREE]

.MODEL MEDIUM
.CODE
PUBLIC convlong
;convlong written by Bruce W. Tonkin on 8-14-88 for use with QB 4.0 & MASM 5.0
;convlong will convert a long integer passed as a packed 4-byte string into
;a string with the bytes re-ordered so an ASCII-order sort will sort in

;numeric order. It is called with:
;call convlong (x$)
;where x$ is the string to convlong (in the current data segment).
;The routine does not check for a zero length of the passed string.
convlong PROC
 push bp ;save old BP
 mov bp,sp ;Set framepointer to old stack
 mov bx,[bp+6] ;address of string length isn't needed
 mov bx,[bx]+2 ;move the string address to bx
 mov dh,byte ptr [bx] ;get first byte
 inc bx ;point to next byte
 mov dl,byte ptr [bx] ;get second byte
 inc bx ;point to third byte
 mov ah,byte ptr [bx] ;get third byte
 inc bx ;point to last byte
 mov al,byte ptr [bx] ;get fourth and last byte
 mov byte ptr [bx],dh ;store first byte in fourth spot
 dec bx ;point to third spot
 mov byte ptr [bx],dl ;store former second byte
 dec bx ;point to second spot
 mov byte ptr [bx],ah ;store former third byte
 dec bx ;point to former first byte spot
 xor al,080h ;invert the sign bit
 mov byte ptr [bx],al ;and store the fourth byte where first was
 pop bp ;restore old base pointer
 ret 2 ;clear 2 bytes of parameters on return
convlong ENDP
 END






[LISTING FOUR]


.MODEL MEDIUM
.CODE
PUBLIC convof
;convof written by Bruce W. Tonkin on 8-14-88 for use with QB 4.0 & MASM 5.0
;convof will convert a Microsoft Binary format floating-point number passed as
;a string into a string with the bytes re-ordered so an ASCII-order sort will
;sort in numeric order. It is called with:
;call convof (x$)
;where x$ is the string to convof (in the current data segment).
;The routine does not check for a zero length of the passed string.
convof PROC
 push bp ;save old BP
 mov bp,sp ;Set framepointer to old stack
 mov bx,[bp+6] ;move the string length to cx
 mov cx,[bx]
 push si ;save si--used by routine
 mov ax,cx ;copy cx into ax
 dec ax ;subtract one from ax
 shr cx,1 ;divide cx by two
 mov bx,[bx]+2 ;move the string address to bx
 push bx ;save that address
 add bx,ax ;look at the end of the string

 cmp byte ptr [bx],0 ;check the first byte
 pop bx ;restore string pointer
 jnz va ;last byte was not zero
 mov byte ptr [bx],081h ;it was zero, so make first byte 129
 dec cx ;and make all other bytes zero
 inc bx ;point to next byte
vt: mov byte ptr [bx],0 ;clear it
 inc bx ;point to next byte
 loop vt ;decrement cx and loop until done
 jmp vi ;then go to the end and restore registers
va: mov si,bx ;set up si to point to bytes to swap
 add si,ax ;points to last byte of string
iv: mov dl,[bx] ;first byte of string to dl
 mov dh,[si] ;second byte to dh
 mov [bx],dh ;and save it
 mov [si],dl ;swap two bytes to reverse order
 inc bx ;point to next byte
 dec si ;and get ready for corresponding byte to move
 loop iv ;dec cx and repeat until all bytes were swapped
 mov bx,[bp+6] ;restore the original string pointer
 mov cx,[bx] ;length to cx
 mov bx,[bx]+2 ;location in bx
;at this point, all the bytes in the string have been put in reverse order
 mov ah,[bx] ;save first string byte into ah
 inc bx ;point to second byte
 mov al,[bx] ;second byte into al
 dec bx ;now point to first byte again
 mov dh,ah ;save copies
 mov dl,al ;of both bytes
 push cx ;save cx=length
 mov cl,7 ;get ready to rotate
 shl ah,cl ;move low bit left 7 positions for first byte
 shr al,cl ;move high bit right 7 positions for second byte
 pop cx ;restore count in cx
 and dl,07fh ;mask high bit for byte two
 add dl,ah ;low bit of byte one to high bit of byte two
 shr dh,1 ;shift byte one right one bit
 or dh,080h ;and turn high bit on for byte one
 cmp al,1 ;check status of former high bit on byte two
 jnz v ;high bit wasn't set
 push bx ;save string pointer
 xor dx,0ffffh ;invert first two bytes
 inc bx ;point to second byte
 inc bx ;point to third byte
 dec cx ;decrement counter accordingly
 dec cx
vv: xor byte ptr [bx],0ffh ;invert successive bytes three to end
 inc bx ;point to next byte
 loop vv ;decrement cx and repeat until done
 pop bx ;restore string pointer
v: mov byte ptr [bx],dh ;save altered byte one
 inc bx
 mov byte ptr [bx],dl ;and byte two
vi: pop si ;restore si
 pop bp ;restore old base pointer
 ret 2 ;clear 2 bytes of parameters on return
convof ENDP
 END







[LISTING FIVE]



.MODEL MEDIUM
.CODE
PUBLIC convnf
;convnf written by Bruce W. Tonkin on 8-13-88 for use with QB 4.0 & MASM 5.0
;convnf will convert an IEEE floating-point number passed as a string into
;a string with the bytes re-ordered so an ASCII-order sort will sort in
;numeric order. It is called with:
;call convnf (x$)
;where x$ is the string to convnf (in the current data segment).
;The routine does not check for a zero length of the passed string.
convnf PROC
 push bp ;save old BP
 mov bp,sp ;Set framepointer to old stack
 mov bx,[bp+6] ;move the string length to cx
 mov cx,[bx]
 push si ;save si--used by routine
 mov ax,cx ;copy cx into ax
 dec ax ;subtract one from ax
 shr cx,1 ;divide cx by two
 mov bx,[bx]+2 ;move the string address to bx
 mov si,bx ;set up si to point to bytes to swap
 add si,ax ;points to last byte of string
iv: mov dl,[bx] ;first byte of string to dl
 mov dh,[si] ;second byte to dh
 mov [bx],dh ;and save it
 mov [si],dl ;swap two bytes to reverse order
 inc bx ;point to next byte
 dec si ;and get ready for corresponding byte to move
 loop iv ;dec cx and repeat until all bytes were swapped
 mov bx,[bp+6] ;restore the original string pointer
 mov cx,[bx] ;length to cx
 mov bx,[bx]+2 ;location in bx
 test byte ptr [bx],080h ;check the high-order bit of the first byte
 jnz v ;high-order bit was set
 xor byte ptr [bx],080h ;fix the first byte
 jmp vi ;and done
v: xor byte ptr [bx],0ffh ;invert all the bytes in the string
 inc bx ;next location
 loop v ;dec cx and repeat until all bytes have been inverted
vi: pop si ;restore si
 pop bp ;restore old base pointer
 ret 2 ;clear 2 bytes of parameters on return
convnf ENDP
 END







[LISTING SIX]


.MODEL MEDIUM
.CODE
PUBLIC INVERT
;INVERT written by Bruce W. Tonkin on 8-13-88 for use with QB 4.0 & MASM 5.0
;INVERT will xor all the bytes in a string, thus allowing it to be sorted in
;descending order. It is called with:
;call INVERT (x$)
;where x$ is the string to invert (in the current data segment).
;The routine does not check for a zero length of the passed string.
INVERT PROC
 push bp ;save old BP
 mov bp,sp ;Set framepointer to old stack
 mov bx,[bp+6] ;move the string length to cx
 mov cx,[bx]
 mov bx,[bx]+2 ;put the string address into bx
iv: xor byte ptr [bx],0ffh ;convert the first byte
 inc bx
 loop iv ;decrement cx and repeat until done
 pop bp ;restore old base pointer
 ret 2 ;clear 2 bytes of parameters on return
INVERT ENDP
 END






[LISTING SEVEN]


defint a-z
dim t!(17)
dim t$(17)
open"o",1,"bench.dat"
cls
t!(0)=timer
for i=1 to 100
 for j=1 to 1000
 next j
next i
t!(0)=timer-t!(0) 'time for bare loop
t$(0)="Raw integer loop"

d$=mki$(-13)
t!(1)=timer
for i=1 to 100
 for j=1 to 1000
 CALL convint(d$)
 next j
next i
t!(1)=timer-t!(1) 'time for integer conversions
t$(1)="Integer conversion"

d$=mkl$(-130000)
t!(2)=timer

for i=1 to 100
 for j=1 to 1000
 CALL convlong(d$)
 next j
next i
t!(2)=timer-t!(2) 'time for long integer conversions
t$(2)="Long integer conversion"

d$=string$(4,204)
t!(3)=timer
for i=1 to 100
 for j=1 to 1000
 CALL convof(d$)
 next j
next i
t!(3)=timer-t!(3) 'time for old single-precision float conversion
t$(3)="Old single float conversion"

d$=string$(8,204)
t!(4)=timer
for i=1 to 100
 for j=1 to 1000
 CALL convof(d$)
 next j
next i
t!(4)=timer-t!(4) 'time for old double-precision float conversion
t$(4)="Old double float conversion"

d$=mks$(-13.0405)
t!(5)=timer
for i=1 to 100
 for j=1 to 1000
 CALL convnf(d$)
 next j
next i
t!(5)=timer-t!(5) 'time for IEEE single-precision float conversion
t$(5)="IEEE single float conversion"

d$=mkd$(-13.04050607)
t!(6)=timer
for i=1 to 100
 for j=1 to 1000
 CALL convnf(d$)
 next j
next i
t!(6)=timer-t!(6) 'time for IEEE double-precision float conversion
t$(6)="IEEE double float conversion"

t!(7)=timer
for i=1 to 100
 for j=1 to 1000
 CALL invert(d$)
 next j
next i
t!(7)=timer-t!(7) 'time for inverting 8 bytes
t$(7)="Invert 8 bytes"

a$="AB"
b$="AX"

t!(8)=timer
for i=1 to 100
 for j=1 to 1000
 if a$>b$ then j=j+1
 next j
next i
t!(8)=timer-t!(8) 'time to compare two-byte strings
t$(8)="Compare two-byte strings"

a=100
b=200
t!(9)=timer
for i=1 to 100
 for j=1 to 1000
 if a>b then j=j+1
 next j
next i
t!(9)=timer-t!(9) 'time to compare two integers
t$(9)="Compare two integers"

a$="ABCD"
b$="ABCX"
t!(10)=timer
for i=1 to 100
 for j=1 to 1000
 if a$>b$ then j=j+1
 next j
next i
t!(10)=timer-t!(10) 'time to compare two 4-byte strings
t$(10)="Compare two four-byte strings"

a!=100.01!
b!=200.01!
t!(11)=timer
for i=1 to 100
 for j=1 to 1000
 if a!>b! then j=j+1
 next j
next i
t!(11)=timer-t!(11) 'time to compare two single floats
t$(11)="Compare two single floats"

a#=100.01#
b#=200.01#
t!(12)=timer
for i=1 to 100
 for j=1 to 1000
 if a#>b# then j=j+1
 next j
next i
t!(12)=timer-t!(12) 'time to compare two double floats
t$(12)="Compare two double floats"

a$="ABCDEFGH"
b$="ABCDEFGX"
t!(13)=timer
for i=1 to 100
 for j=1 to 1000
 if a$>b$ then j=j+1

 next j
next i
t!(13)=timer-t!(13) 'time to compare two 8-byte strings
t$(13)="Compare two 8-byte strings"

a&=123456&
b&=123457&
t!(14)=timer
for i=1 to 100
 for j=1 to 1000
 if a&>b& then j=j+1
 next j
next i
t!(14)=timer-t!(14) 'time to compare two long integers
t$(14)="Compare two long integers"

a$="ABCDEFGHIJKLMNOPQRST"
b$="ABCDEFGHIJKLMNOPQRSX"
t!(15)=timer
for i=1 to 100
 for j=1 to 1000
 if a$>b$ then j=j+1
 next j
next i
t!(15)=timer-t!(15) 'time to compare two 20-byte strings
t$(15)="Compare two 20-byte strings"

a$="ABCDEFGHIJKLMNOPQRST"
b$="XBCDEFGHIJKLMNOPQRST"
t!(16)=timer
for i=1 to 100
 for j=1 to 1000
 if a$>b$ then j=j+1
 next j
next i
t!(16)=timer-t!(16) 'best? time to compare two 20-byte strings
t$(16)="Compare two 20-byte strings (best?)"

t!(17)=timer
for i=1 to 100
 for j=1 to 1000
 call dummy(a,b,c)
 next j
next i
t!(17)=timer-t!(17) 'time to make a call with three parameters
t$(17)="Three-integer-parameter call"

print t$(0),t!(0)
print #1,t$(0),t!(0)
for i=1 to 17
 t!(i)=t!(i)-t!(0)
 print t$(i),t!(i)
 print #1,t$(i),t!(i)
next i

sub dummy(a,b,c) static
 c=1
end sub
































































May, 1989
TAWK: A SIMPLE INTERPRETER IN C++


The data-encapsulation features of C++ prove useful when reading and writing
records




Bruce Eckel


Bruce Eckel is a C++ consultant and owner of Eisys Consulting. He has been
writing for Micro Cornucopia for two and a half years. This article is adapted
from his book Using C++ (Osborne/ McGraw-Hill, 1989). Bruce may be contacted
at Eisys Consulting, 501 N. 36th St., Ste. 163, Seattle, WA 98103.


Most microcomputer database management systems (DBMSs) read and write records
in a "comma-separated ASCII" format. This is probably an artifact from the
days when Basic (which uses that format) was the only common tongue on
microcomputers. Comma-separated ASCII files are useful not only because they
allow the records from one DBMS to be moved to another, but also because they
can be manipulated by using programming languages.
While Basic automatically reads and writes these records, other languages must
be programmed to do so. In C++, this tedious task can be encapsulated into
several classes; the user of the class doesn't need to worry about the
details. In the first part of this article, two classes are created. The
first, class field, reads a single quoted and comma-separated field and makes
an object from that field. The second, class csascii, opens a comma-separated
ASCII file and reads records (as arrays of field objects) one at a time, until
the file ends. A simple application that uses the classes to search through a
data-base file for a last name is presented.
Database files must often be manipulated or output in an organized way as a
"report." It becomes tedious and problematic to write and compile code for
each different report since nonprogrammers must often design reports. A common
solution to a problem such as this is the creation of a "scripting language"
specifically tailored to the task at hand. The second part of this article is
the creation of a simple language that outputs the records (to standard
output) in a comma-separated ASCII file according to a script in a separate
file.
The program is called TAWK for "tiny awk," since the problem it solves is
vaguely reminiscent of the "awk" pattern-matching language found on Unix
(versions have also been created for DOS). It demonstrates one of the thornier
problems in computer science: parsing and executing a programming language.
The data-encapsulation features of C++ prove most useful here, and a
recursive-descent technique is used to read arbitrarily long fields and
records.
The code was developed and tested on a DOS system. It compiles with Zortech
C++ or the Glockenspiel/Advantage translator used with Microsoft C. The
programs should also work on Unix, because all library calls are ANSI C and
the only class used that is not defined here is the streams class (which is
included with every C++ package). The simple screen-manipulation commands
(clear screen, reverse video) assume an ANSI terminal or a PC with ANSI.SYS
loaded.


Object-Oriented Terminology


When discussing object-oriented programming, it is helpful to review some of
the terminology. Object-oriented programming means "sending messages to
objects." In traditional languages, data is declared and functions act
directly on the data. In object-oriented programming, objects are created,
messages are sent to the objects, and the objects decide what to do with the
message (in other words, how to act on their internal data).
An object is an entity with internal state (data) and external operations,
called member functions in C++. "Sending a message" means calling one of these
member functions.
An important reason for organizing a program into distinct objects is the
concept of encapsulation. When data is encapsulated in an object, it is hidden
away (private) and can only be accessed by way of a clearly defined interface.
Only class member functions and friend functions may modify private data. Data
encapsulation can clarify code by combining data in a single package with
specific legal operations (member functions). Data encapsulation is also
useful in preventing bugs -- a working class doesn't break simply because it
is used in a new program.


Virtual Functions


Object-oriented purists will notice this program does not use late binding (by
way of C++ virtual functions) and thus is not object-oriented in the Smalltalk
sense. When a message is sent to an object in Smalltalk, the object always
decides what to do with the message (that is, the specific function to call)
at run time, so the function address isn't bound to the function call until
the call is actually made. Because most compilers bind function calls during
compilation, run-time binding is often called late binding.
C++ always performs binding at compile time, unless the programmer
specifically instructs the compiler to wait until run time by using the
virtual keyword. This feature allows subclasses (all inherited from the same
base class) to have identical function calls that are executed differently. A
collection of generic objects (all of the same base class) can be maintained
and all the "legal" messages for the base class may be sent to any of the
objects in that collection. The object figures out what to do with the message
according to what specific subclass it is. This is object-oriented programming
in its true sense.
Most problems can benefit from the data-encapsulation features of C++. It
seems, however, that not every problem demands virtual functions. The project
presented here is one of those cases. For an example of the use of virtual
functions, see my article "Building MicroCAD" in the November/December 1988
issue of Micro Cornucopia (also available as part of C++ source code library
disk #1, available from Eisys Consulting for $15).


Reading C++ Code


If you are a C programmer, here's a simple way to think about C++ while you
are reading the code for this article: Objects look like structures, with a
few bells and whistles. One bell is that you can hide some of the structure
members--members are automatically hidden (private) unless you explicitly
state they are public. A whistle is the ability to include functions as
members of the struct. Members of a class (a user-defined type) are accessed
just as you would access members of a struct--with a dot (or an arrow, if you
have a pointer to an object). One more whistle is that the programmer can
define the way the objects are initialized (by using the constructor) when
they come into scope, and cleaned up (by using the destructor) when they go
out of scope.
Example 1 shows a tiny class to introduce you to the basics of C++
programming. class declarations are generally contained in header files with
the extension .hxx. Definitions are generally contained in files with the
extension .cxx. The AT&T Unix C++ chose the unfortunate extension of .C for
definitions, and .h for declarations. This is fine on Unix, which is
case-sensitive, but causes problems while in DOS. Walter Bright's Zortech C++
compiler originally used .cpp. He later modified it to allow .cxx, which is
the style the Glockenspiel translator (previously marketed by Lifeboat as
Advantage C++) uses. I use the .cxx format because it works with both
products.
Example 1: A C++ class

 class tiny {
 // private stuff here (this is a comment)
 int i;
 public: // public stuff here:
 print () { // an "in-line" function
 printf ("i = %d\n", i);
 }
 tiny (int j); // constructors have the class name
 tiny() {} // destructors use a tilde
 }; // classes end with a brace and a semicolon

 tiny::tiny (int j) { // non inline definition
 i = j;

 }

 main() {
 tiny A(2); // implicit constructor call
 // A.i = 30; // error! private member
 A.print (); // calling a member function
 // implicit destructor call at end of scope
 }



The Streams Class


The streams class used here is an extremely useful class developed by Bjarne
Stroustrup (the inventor of the language) to handle input/output. It defaults
to standard input and standard output (the cin and cout objects, automatically
defined when you include the stream.hxx header file), but can also be used to
read and write files. A buffer can even be made into a stream object, and the
same operations can be performed on that object.
The most complete written reference available for the streams class is chapter
8 of Stroustrup's The C++ Programming Language (Addison-Wesley, 1986). This
chapter is not exactly an exhaustive example of streams. One of the beauties
of C++ is that you always have access to a description (often admittedly
terse) of the operations available for that particular class -- the header
file. By studying the header file, you can often get ideas for new ways to use
an object. Zortech C++ also has library source code available, which includes
valuable comments on the use of certain functions (that's how I figured out
many features).
Output in streams is accomplished with the operator you know from C as left
shift. C++ allows you to overload functions and operators to give them
different meanings depending on their arguments. When left shift is used with
a stream object, it means "put this stuff out to the stream." Example 2 lists
a short program to show the use of streams. Notice how streams allow you to
string together a series of output statements.
Example 2: The use of streams

 # include <stream.hxx> // cout automatically defined
 main() {
 cout << "Hello, world!\n" << "I am"
 << 6 << "today!\n";



Recursive Descent


A recursive descent algorithm is useful if you don't know how long or
complicated a statement will be when you start looking at it. In programming
languages, for example, recursive descent parsers are often used in expression
evaluation, because expressions can contain other expressions. In this
project, the expressions aren't particularly complicated, but we don't know
how long a string of text is going to be.
A central function is used when scanning an expression using recursive
descent. This function munches along and absorbs input until it runs into a
delimiter that indicates a change in the type of input (white space, for
example, or a number). At this point, it might call another function to eat
the white space or to get a string of digits and turn it into a number. Then,
if the expression is finished, it will just return. If the expression isn't
finished (and here's the tricky part), it calls itself (that is, it recurses).
Every time it encounters a new expression within the one it's evaluating, it
recurses to evaluate the expression.
When solving more complex problems (such as a programming language), a set of
functions is used. Each function may call any of the others during expression
evaluation.
At some point, the evaluation must bottom out. When this happens, the function
performs some termination activities and then returns. As the stack unwinds
from all the recursive calls, the tail end of each function call performs some
operation to store the information it was able to glean, and then it returns.
When the function finally completes, the expression has been evaluated.
Recursive descent is used in three places in this project. The field class,
which creates an object containing a single quote-delimited field, has a
recursive function field::getfield() (shown in Listing Two) to read one
character at a time, keeping track of the number of characters encountered,
until the end of the field. When the closing quotation mark is encountered,
memory is allocated for exactly the right number of characters and the
function returns. As it unwinds, characters are placed in the object's data
buffer. Using recursive descent means no restrictions are imposed on the field
size (other than stack space).
The token class uses recursive descent in a more sophisticated way. When a
token object is created by handing it an input stream (by way of the
constructor function token::token(istream & input)), it reads the input stream
until it has scanned a complete token. When the constructor completes, a new
token has been created.
A token is a group of symbols that represent a single concept. A C++ compiler
uses a large number of tokens: { means begin a scope, for means start a for
loop, foo means a variable. TAWK has a much smaller number of tokens. All
tokens in TAWK are delimited by the "@" sign, which starts a new command. When
"@" is encountered, it is pushed back onto the input stream (for use in the
next token object) and the current token is completed. The central
recursive-descent function for token is token::get_token(), shown in Listing
Seven.
The class parse_array builds an array of tokens by recursively calling
parse_array::build_array() (shown in Listing Seven). This function makes a new
token, then looks at the token to decide what to do next. The two programs
(LOOKUP and TAWK) are built from several classes. Each of these classes will
be examined.


The Class Field


The declaration of the field class is shown in Listing One and the definitions
are in Listing Two. The field object doesn't control opening or closing files.
It is simply handed an istream from which it takes its input. If it finds the
end of input, it just makes an internal note (by setting its end_of_file flag)
and returns. It's up to the caller to check for end-of-file with the function
field::eof().
The operator<<() is overloaded so that a field object may be put to a stream
output object. When this occurs, the data field is copied to the output.
The field constructor field::field (istream & instream) initializes all the
variables to zero and sets the member istream * input equal to instream. This
allows field::getfield() to treat input as a global variable and to simply get
the next character. The last thing the constructor does is call the
recursive-descent function field::getfield(), which recurses until it reaches
the end of the field. When the constructor finishes, the field is complete.
The function field::getfield() reads a character from the input stream. If it
isn't an end-of-file character, it checks for terminators, which include a
comma if not enclosed by quotation marks (determined by a special flag
infield) or a carriage return, which delimits the entire record. If no
terminator is found, the function counts the current character and calls
itself to get the next character. If a terminator is found, memory is
allocated to hold the string (using the C++ dynamic-memory allocation keyword
new) and the string terminator \ 0 is inserted. As the function returns from
calling itself, each character is inserted, from right to left, into the
buffer.
Memory is not always allocated for a field. The constructor for a field object
sets the data pointer to zero. If memory is never allocated, the destructor
will delete a null pointer, which is defined to have no effect.


The Class csascii


The csascii (for comma-separated-AS-CII) class is shown in Listings Three (the
declaration) and Four (the definition). The constructor opens the input file,
counts the number of fields in a record, and closes the file. It then creates
an array of pointers to field objects, reopens the file and reads in the first
record. Every time csascii::next() is called, a new record is read until the
end of the file.
The operator[]() is overloaded so the individual fields may be selected from
each record. This function checks to ensure that the index is within bounds.
The method of opening files should be examined here. The line istream
infile(new filebuf>open (filename, input)); is a succinct way to create a
buffer and open a file. The new filebuf creates a filebuf object (necessary to
open a file as an istream) on the free store and returns a pointer to this
object. The pointer is used to call a member function, filebuf::open(). The
pointer is also handed to the constructor of istream to create an object
called infile.
This is a clever piece of code, and nice for quick programming -- I got it
from the Glockenspiel/Advantage manual, so I suspect it's something John
Carolan cooked up. Unfortunately, it isn't robust unless you know that the
file exists. If the file doesn't exist on DOS machines, the system locks up.
A more robust way to open the files in this program is to replace the previous
code with the code in Example 3. Notice that in csascii::csascii(), the file
is closed implicitly by putting braces around the first clause in the
constructor where the fields are counted. When the istream object goes out of
scope, the file is closed. This is the only purpose for putting the braces
there. Anytime you want to control the destruction of a local variable, simply
put it in braces.
Example 3: Opening files


 "Ball", "Mike", "Oregon Software C++ Compiler"
 "Bright", "Walter", "Zortech C++ Compiler"
 "Carolan", "John", "Glockenspiel C++ Translator"
 "Stroustrup", "Bjarne", "AT&T, C++ Creator"
 "Tiemann", "Michael", "Free Software Foundation C++ Compiler"



Testing Field and csascii


Listing Five is a short program to show the use of class csascii. The csascii
object file is created by giving it the name of the comma-separated ASCII file
PPQUICK.ASC. (See Example 4 for a sample file.) Then the records are read one
at a time and field 0 is compared to the first argument on the command line
(presumably the last name of the persons in the database). When a record is
found, it is displayed on the screen (notice the use of the ANSI
screen-control codes). A flag called found is set to indicate the least one
record is found. When no more matches occur, the program knows to exit (it is
assumed the file has been sorted by the database manager).
Example 4: A sample comma-separated ASCII file PPQUICK.ASC

 filebuf fl;
 if (fl. open(argv [1], input) == 0) {
 cout << "cannot open" << argv[1] << "\n";
 exit (1);
 }
 istream infile (&fl);

The ANSI C library function strcmp() has been used here for compatibility. To
ignore uppercase or lowercase in the comparisons, Microsoft C provides
strcmpi() and Zortech provides strcmpl().
Notice how easy it is to use a class once it has been created. One of the
advantages of C++ is the ease of use of library functions. (That is, when
library functions become available!)


TAWK


Table 1 provides the complete syntax for the TAWK language. You can see that
each TAWK conmmand consists of an "@" sign and a single character (in the case
of @() and @<>, the commands are @( and @< and the ) and > are used by the
function that reads the number, to find the end).
Table 1: The TAWK syntax

 TAWK: A Tiny database processor, vaguely like AWK

 usage: tawk tawkfile csafile
 where: csafile contains comma-separated ASCII records. Each field in a
 record is contained in quotes, and each record is delimited by a
 newline. These are standard records that can be generated by the
 Basic language and most database management systems.

 tawkfile is a file that contains formatting commands. Each
 record in the csafile is read and fields in the record are
 printed out according to the formatting commands in the
 tawkfile. Everything in the tawkfile (characters, spaces,
 newlines) is printed literally except for the following:

 @(n) Print field number n; @(3) prints field 3 of the
 current record. The first field in a record is
 field 0.

 @<n> Print an ascii character number n; @<27> prints
 the escape character

 @! This line is a comment until the end of the line

 @?nn@: (then statements) @ (else statements) @. An
 if-then-else conditional. If field nn is not
 empty, the then statements are executed, otherwise
 the else statements are executed. A conditional
 must have all three parts, but the statements may

 be empty. Conditionals can be nested.

 @Preamble or When a tawkfile is begun, all statements until
 @P or @p @main are considered to be part of the
 preamble. The preamble is only executed once, at
 the beginning of the program. The preamble must
 be strictly text; it cannot contain field numbers
 or conditionals. The @preamble statement is
 optional; @preamble is assumed until @main.

 @main The main section is executed once for each record
 or @M or @m in the file. All statements between
 @main and @conclusion are part of the main.
 section. @main may contain field numbers and
 conditionals. The @main statement is required.

 @conclusion The conclusion is executed after the last record
 or @C or @c in the database file is read and the file is
 closed. The conclusion,
 like the preamble, may only contain text. All
 other characters on the same line as @preamble,
 @main, or @conclusion are ignored. The
 @conclusion statement is required.

 @end This must be at the end of the tawkfile

 @@ Print an @ sign

 Example tawkfile:
 @! A comment, which isn't printed
 @! The @preamble is optional, but promotes understanding
 @main

 This is field 1:@(1)
 This is field 10:@(10)
 @?4@:@(4) @Field 4 is empty @.
 print an escape: @<27>
 Re-generate comma-separated ASCII record:

The execution of a tawkscript parallels the compilation or interpretation of
other programming languages. The tawkscript is parsed into arrays of tokens
when the program starts up. An execution routine steps through the arrays and
performs actions based on the tokens to run the tawkscript.
Listing Six is the declaration for class token and class parse_array. Listing
Seven contains the definitions. Listing Eight is the main() function for TAWK.
In Listing Eight the tawkscript is parsed into three different parse_arrays,
one each for the @preamble, @main, and @conclusion. These arrays are executed
using the database file as input.


The Class token


Each token must be a particular type. The kind of information a token contains
depends on what type it is. In TAWK, the possible token types are as follows:
a field number (for printing out a field or testing if a field is empty in an
if statement), a string (simple text including nonprintable characters), parts
of a conditional statement (if, else, and endif), or a phase change (which
indicates a transition from @preamble to @main or @main to @conclusion).
Because a phase change is never executed but is simply used to terminate the
creation of a parse_array, it isn't a token in the same sense, but some form
of communication was necessary and this seemed the cleanest.
The different types of tokens and phases are enumerated in the tokentype and
phase declarations. The phase information is kept by the main program, but
each token contains a tokentype identifier. Because a token can never be a
field number and a string at the same time, the data container in a token is
combined into an anonymous union (which is like a regular union only it has no
name). The union is used to save space.
A token also contains information about the level of the if statement. Because
if statements can be nested, each token that is an if, else, or endif must
have information about the nesting level. If the conditional evaluates to
false (that is, the field is empty), the interpreter must hunt through tokens
in the parse_array until it finds the else statement at the same level, and
continue executing statements from there.
While token::get_token() is performing its recursive-descent scanning, it
calls several other functions, which are made private because they aren't
needed by the user. token::get_next() gets a character and tests for
end-of-file (which is an error condition, because an @end statement should
always terminate the tawkfile). token::get_value() is used for the @() and @<>
statements. token::dumpline() is called for comments.
Listing Seven starts with a number of global variables that are declared
static. This means they cannot be accessed outside the file (this use of the
static keyword is called file static). When the constructor is called, it
establishes the source of input characters (token-stream), sets the length of
the string (which has been read so far) to zero, and begins the descent by
calling token::get_token().
The following are three possibilities in token::get_token():
1. The next character in the input stream is an @ and the length is zero. This
means you are at the beginning of a command and the next character will
determine what the command is. In this case, a large switch statement is
executed.
2. The next character is an @ and the length is not zero. This means you are
in the middle of a string and a command is starting. In this case, the @ is
pushed back on the input stream (for use by the next token), space is
allocated for the string, and the unwinding of the stack is started with a
return.
3. The next character is not an @. This means it must be plain text. In this
case, token::get_token() calls itself to get more characters.


The Class parse_array



The class parse_array is a container class, because it is only used to contain
objects of another class (token). There is no way to know how many tokens a
parse_array will contain, so the recursive approach is used again. The
constructor initializes some variables and calls the recursive function
parse_array::build_array(), which keeps getting tokens and calling itself
until a phase change or the end of the input (an @end statement). At this
point, it allocates space to hold all the tokens (which it has been counting
during the descent) and ascends, storing a token on each function return.
The individual tokens in a parse_array can be selected by using brackets ([])
because the bracket operator has been overloaded in parse_array:
:operator[](). Because token has a stream function defined, tokens can be put
directly to cout.


Executing a TAWKscript


Listing Eight shows the main() function for TAWK. After the command-line
arguments are checked, the tawkfile is opened and three parse_arrays are
created: one for the @preamble, one for @main, and one for the @conclusion.
The second command-line argument is used to create a csascii object.
At the beginning and end of the script execution, the preamble and conclusion
parse_arrays are simply sent to standard output (cout). Because they can only
contain text, no other action is necessary.
The central loop executes the statements in the @main phase for each record
the csascii object reads from the database file. After a record is read, the
type of each token in parse_array Amain is used in a switch statement to
choose the proper action. Strings are sent to cout and fieldnumbers send the
selected field to cout.
If an if statement, if the selected field is empty in the current record, the
parse_array index is incremented until the else token at the same level is
found. If the field is not empty, no action is taken (the following statements
are executed). When an else is encountered, it means the if evaluated to true,
so the else clause is skipped over until the endif of the same level is found.
Listing Nine is a make-file to make all the examples in this project.


Example TAWKscripts


Listings Ten and Eleven show examples of tawkscripts. Listing Ten reformats a
file with six fields into one with five fields, combining the last two fields.
If both of the last two fields are not empty, a space is inserted between
them.
Listing Eleven shows the usefulness of the preamble and conclusion. It creates
a tiny telephone list (which I carry in my wallet) on an HP LaserJet printer.
The preamble and conclusion are used to send special control codes to the
printer. The use of nested if-then-else statements is shown here: If field 3
exists, it is printed followed by a carriage return and a test to see if field
4 exists, which is printed with a linefeed if it does (nothing happens if it
isn't). If field 3 doesn't exist, field 4 is tested and printed with a
linefeed (otherwise only a linefeed is printed). When everything is completed
a reset is sent to the LaserJet.
If you want a further challenge, try adding a goto system to TAWK. You will
need to create a label command and a goto command. gotos can be executed from
if-then-else statements.


Conclusion


The main() program for TAWK is actually quite small for what it does. Because
the details are hidden in the csascii and parse_array objects, you can imagine
creating a much more sophisticated program without losing control of the
complexity. This is typical of C++. Indeed, it was designed to allow one
programmer to handle the same amount of code that previously required several
programmers. The compiler supports the creation of large projects by hiding
initialization and cleanup, and by enforcing the correct use of user-defined
types.


Availability


All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063; or call 800-356-2002 (from inside
Calif.) or 800-533-4372 (from outside Calif.). Please specify the issue number
and format (MS-DOS, Macintosh, Kaypro).


Products Mentioned


Zortech C++ -- Zortech Inc. 1165 Massachusetts Ave. Arlington, MA 02174
800-848-8408 $149.95
Glockenspiel C++ -- Glockenspiel (Available for DOS and OS/2) 2 Haven Ave.
Port Washington, NY 11050 800-462-4374 $495 Includes Glockenspiel C++ for DOS,
Glockenspiel C++ for OS/2, CommonView for MS Windows, CommonView for OS/2 PM,
and source code for CommonView.
Microsoft C 5.1 --Microsoft (For use with the Glockenspiel translator.
Discounts available from other retailers) Box 97017 Redmond, WA 98073-9717
206-882-8080 $450


_TAWK, A Simple Interpreter in C++_
by Bruce Eckel


[LISTING ONE]


// FIELD.HXX: used by csascii class to build a single field.
// Fields are collected by csascii to create a record.
// by Bruce Eckel,
#include <stream.hxx>

class field { // one field in a comma-separated ASCII record
 istream * input; // where to get the data
 char * data;

 int length, fsize;
 int end_of_file; // flag to indicate the end of file happened
 void getfield(); // recursive function to read in a field;
 // treats data, length & input as globals
 int infield; // flag used by getfield() to determine whether
 // it's inside a quoted field
 public:
 field(istream & instream);
 ~field();
 friend ostream& operator<<(ostream &s, field & f) {
 s << f.data;
 return s;
 }
 int eof() { return end_of_file; } // to check for end
 int size() { return fsize;}
 int last_length() {return length; }
 char * string() { return data; }
};






[LISTING TWO]

// FIELD.CXX: definitions for class field
// A "recursive descent" scanning scheme is used because field
// length is always unknown.
// by Bruce Eckel
#include "field.hxx"

field::field(istream & instream) {
 input = &instream;
 length = 0;
 end_of_file = 0; // set flag to say "we're not at the end"
 infield = 0; // set flag to say "we're not inside a field"
 data = (char *)0; // to show no memory has been allocated
 getfield(); // recursively get characters until end of field
}

field::~field() {
 delete data; // if no memory has been allocated,
 // data = (char *)0 so this will have no effect.
}

// A Comma-separated ASCII field is contained in quotes to allow
// commas within the field; these quotes must be stripped out
void field::getfield() {
 char c;
 // This happens when DEscending:
 if((input->get(c)).eof() ) {
 end_of_file++; // just say we reached the end...
 return;
 }
 else // watch out for the Unix vs. DOS LF/CR problem here:
 if (((c != ',') infield) && (c != '\n')) {
 if ( (c != '"') && (c != '\r')) // watch for quotes or CR
 length++; // no quotes -- count this character

 else {
 if ( c == '"')
 infield = !infield; // if we weren't inside a field
 // and a quote was encountered, we are now inside
 // a field. If we were inside a field and a quote
 // was found, we're out of the field.
 c = 0; // a quote or CR; mark it so it isn't included
 }
 getfield(); // recursively get characters in field
 // after returning from function call, we jump past
 // the following "else" part to finish the recursion
 }
 else { // This happens once, when the terminator is found:
 fsize = length; // remember how long the string is
 data = new char[length + 1]; // space for null terminator
 data[length] = '\0'; // highest index is "length"
 // when you allocate an array of length + 1
 length--; // notice we don't insert the delimiter
 // Now the first "if" statement evaluates to TRUE and
 // the function rises back up.
 return;
 }
 // This happens when Ascending:
 if ( c ) // if it wasn't a quote or CR,
 data[length--] = c; // put chars in as we rise back up...
}






[LISTING THREE]

// CSASCII.HXX: class to manipulate comma-separated ASCII
// database files.
//by Bruce Eckel
#include <stream.hxx>
#include "field.hxx"

class csascii { // manipulates comma-separated ascii files,
// generated by most database management systems (generated and
// used by the BASIC programming language). Each field
// is separated by a comma; records are separated by newlines.
 int fieldcount;
 field ** data; // an array to hold the entire record
 istream * datafile; // file with comma separated ASCII input
 int readrecord(); // private function to read a record
 public:
 csascii( char * filename ); // Open file, get first record
 ~csascii(); // destructor
 int next(); // get next record, return 0 when EOF
 field & operator[](int index); // select a field
 int number_of_fields() { return fieldcount; }
};







[LISTING FOUR]

// CSASCII.CXX: function definitions for comma-separated
// ascii database manipulation class
// by Bruce Eckel,
#include "csascii.hxx"

int csascii::readrecord() {
 for (int fieldnum = 0; fieldnum < fieldcount; fieldnum++ ) {
 data[fieldnum] = new field(*datafile);
 if (data[fieldnum]->eof()) return 0;
 }
 return 1;
}

csascii::csascii( char * filename ) {
 char c;
 fieldcount = 0;
 int quote = 0;
 // first, determine the number of fields in a record:
 {
 // See text for dangers of opening files this way:
 istream infile(new filebuf->open(filename, input));
 while(infile.get(c), c != '\n') {
 // keep track of being inside a quoted string:
 if (c == '"') quote = !quote;
 // fields are delimited by unquoted commas:
 if ( c == ',' && !quote)
 fieldcount++;
 }
 } // infile goes out of scope; file closed
 fieldcount++; // last field terminated by newline, not comma
 // an array of field pointers:
 data = new field * [ fieldcount ];
 // re-open at start; dynamically allocate so it isn't scoped:
 datafile = new istream(new filebuf->open(filename, input));
 readrecord();
}

csascii::~csascii() {
 delete data;
 delete datafile; // calls istream destructor to close file
}

int csascii::next() {
 for (int i = 0; i < fieldcount; i++ )
 delete data[i]; // free all the data storage
 return readrecord(); // 0 when end of file
}

field & csascii::operator[](int index) {
 if (index >= fieldcount) {
 cerr << "index too large for number of fields in record\n";
 exit(1);
 }
 return *(data[index]);
}







[LISTING FIVE]

// LOOKUP.CXX: simple use of csascii to find name in a database
// by Bruce Eckel,
#include "csascii.hxx"
#include <string.h>

main(int argc, char ** argv) {
 if (argc < 2) {
 cerr << "usage: lookup lastname\n";
 exit(1);
 }
 // This puts the database file in the root directory:
 csascii file("\\ppquick.asc"); // create object & open file
 int found = 0; // indicates one record was found
 do {
 if (strcmp(file[0].string(),argv[1]) == 0) {
 found++; // found one. File is sorted, so if we stop
 // finding them, quit instead of wasting time.
 cout << chr(27) << "[2J"; // ANSI clear screen
 for (int i = 0; i < file.number_of_fields(); i++)
 cout << file[i] << "\n";
 cout << chr(27) << "[7m" << "press any key" <<
 chr(27) << "[0m";
 if( getch() == 27) break;
 } else if (found) exit(0); // quit if that was the last
 } while (file.next());
}






[LISTING SIX]


// PARSE.HXX: class to parse a tawk script file. Creates
// a structure which can be used at run-time to "execute"
// the tawk script.
// by Bruce Eckel,
#include <stream.hxx>

// types of tokens the scanner can find:
enum tokentype {
 fieldnumber, string, if_, else_, endif_, phase_change
};

// preamble and conclusion of the tawk script are only executed
// once, while main is executed once for every data record
enum phase { preamble, tmain, conclusion};

class token {

 tokentype ttype;
 union { // an "anonymous union"
 int fieldnum; // if type is a fieldnumber
 unsigned char * literal; // if type is a string
 };
 int if_level; // if this is an if_, then_, or else_
 // private functions:
 void get_token(); // recursive descent scanner
 // Functions to help in scanning:
 void getnext(char & c); // used by get_token();
 unsigned char get_value(char delimiter, char * msg);
 void dumpline(); // for @! comments
 void error(char * msg = "", char * msg2 = "");
 public:
 token(istream & input);
 ~token();
 friend ostream & operator<<(ostream &s, token &t);
 int field_number() { return fieldnum; }
 int token_type() { return ttype; }
 int nesting_level() { return if_level;}
};

// The following is called a "container class," since its sole
// purpose is to hold a list of objects (tokens, in this case):
class parse_array {
 token ** tokenarray; // an array of token pointers
 istream * parse_stream;
 int token_count;
 int end; // the size of the array
 phase p_section; // of the program (preamble, etc.)
 void build_array(); // another recursive function
 public:
 parse_array(istream & input);
 ~parse_array();
 int size() { return end; } // how big is it?
 token & operator[](int index); // select a token
 phase section() { return p_section; }
};






[LISTING SEVEN]

// PARSE.CXX: class parse function definitions
// by Bruce Eckel,
#include "csascii.hxx"
#include "parse.hxx"
#include <ctype.h>
#include <stdlib.h>

// The following are "file static," which means no one outside
// this file can know about them. This is the meaning when a
// global variable is declared "static."
static istream * tokenstream;
static int length; // to remember size of string
static int line_number = 1; // line counting for errors

static int if_counter = 0; // monitors "if" statement nesting
static phase program_section = preamble; // ... until @main
static int end_of_file = 0; // zero means not end of file

token::token(istream & input) {
 // initialize values and start the descent
 tokenstream = &input;
 length = 0;
 get_token(); // recursively get characters to end of token
}

token::~token() { // delete heap if any has been allocated:
 if (ttype == string)
 delete literal;
}

void token::error(char * msg, char * msg2) {
 cerr << "token error on line " << line_number << ": " <<
 msg << " " << msg2 << "\n";
 exit(1);
}

ostream & operator<<(ostream &s, token &t) {
 switch (t.ttype) {
 case string:
 s << (char *)t.literal;
 break;
 case fieldnumber: // only for testing
 s << " fieldnumber: " << t.fieldnum << "\n";
 }
 return s;
}

// Get a character from the tokenstream, checking for
// end-of-file and newlines
void token::getnext(char & c) {
 if(end_of_file)
 error("attempt to read after @end statement\n",
 "missing @conclusion ?");
 if((tokenstream->get(c)).eof() )
 error("@end statement missing");
 if (c == '\n')
 line_number++; // keep track of the line count
}

// See text for description of tokens
void token::get_token() {
 char c;
 // This happens when DEscending:
 getnext(c);
 if ( c == '@') {
 if (length == 0) { // length 0 means start of token
 getnext(c);
 switch(c) {
 case '!': // comment line
 dumpline(); // dump the comment
 get_token(); // get a real token
 break;
 case 'p' : case 'P' : // preamble statement

 if ( program_section != preamble )
 error("only one preamble allowed");
 dumpline(); // just for looks, ignore it
 get_token(); // get a real token
 break;
 case 'm' : case 'M' : // start of main loop
 dumpline(); // toss rest of line
 program_section = tmain;
 ttype = phase_change;
 return; // very simple token
 case 'c' : case 'C' : // start conclusion
 dumpline();
 program_section = conclusion;
 ttype = phase_change;
 return; // very simple token
 case 'e' : case 'E': // end statement
 end_of_file++; // set flag
 ttype = fieldnumber; // so destructor doesn't
 // delete free store for this token.
 if (if_counter)
 error("unclosed 'if' statement(s)");
 return;
 case '(' :
 if ( program_section == preamble 
 program_section == conclusion )
 error("@() not allowed in preamble or conclusion");
 fieldnum = get_value(')',"@()");
 ttype = fieldnumber;
 // This is a complete token, so quit
 return;
 case '<' :
 c = get_value('>',"@<>");
 length++;
 get_token(); // get more...
 break;
 case '?' : // beginning of an "if" statement
 if ( program_section == preamble 
 program_section == conclusion )
 error("@? not allowed in preamble or conclusion");
 fieldnum = get_value('@',"@?@");
 ttype = if_;
 getnext(c); // just eat the colon
 if(c != ':')
 error("@? must be followed by @: (then)");
 if_level = ++if_counter; // for nesting
 return;
 case '~' : // the "else" part of an "if" statement
 ttype = else_;
 if_level = if_counter;
 return;
 case '.' : // "endif" terminator of an "if" statement
 ttype = endif_;
 if_level = if_counter--;
 if(if_counter < 0)
 error("incorrect nesting of if-then-else clauses");
 return;
 case '@' : // two '@' in a row mean print an '@'
 length++; // just leave '@' as the value of c
 get_token();

 break;
 default:
 error("'@' must be followed by:",
 "'(', '<', '?',':','~','.','p','m','c' or '@'");
 }
 } else { // an '@' in the middle of a string; terminate
 // the string. Putback() is part of the stream class.
 // It is only safe to put one character back on the input
 tokenstream->putback(c); // to be used by the next token
 // allocate space, put the null in and return up the stack
 literal = new unsigned char[length + 1]; // space for '\0'
 literal[length--] = '\0'; // string delimiter
 ttype = string; // what kind of token this is
 return; // back up the stack
 }
 } else { // not an '@', must be plain text
 length++;
 get_token();
 }
 // This occurs on the "tail" of the recursion:
 literal[length--] = c; // put chars in as we rise back up...
}

// This function is used by get_token when it encounters a @(
// or a @< to get a number until it finds "delimiter."
// If an error occurs, msg is used to notify the user what
// kind of statement it is.
unsigned char token::get_value(char delimiter, char * msg) {
 char c;
 char buf[5];
 int i = 0;
 while(getnext(c), c != delimiter) {
 if (!isdigit(c))
 error("must use only digits inside", msg);
 buf[i++] = c;
 }
 buf[i] = 0;
 return atoi(buf);
}

void token::dumpline() { // called when '@!' encountered
 char c;
 while(getnext(c), c != '\n')
 ; // just eat characters until newline
}

// Since there's no way to know how big a parse_array is
// going to be until the entire tawkfile has been tokenized,
// the recursive approach is again used:

parse_array::parse_array(istream & input) {
 parse_stream = &input;
 token_count = 0;
 p_section = program_section; // so we know at run-time
 build_array();
}

void parse_array::build_array() {
 token * tk = new token(*parse_stream);

 if( ! end_of_file && tk->token_type() != phase_change) {
 // normal token, not end of file or phase change:
 token_count++;
 // recursively get tokens until eof or phase change:
 build_array();
 } else { // end of file or phase change
 // only done once per object:
 // allocate memory and return up the stack
 tokenarray = new token * [end = token_count];
 if(token_count) token_count--; // only if non-zero
 return;
 }
 tokenarray[token_count--] = tk; // performed on the "tail"
}


parse_array::~parse_array() {
 for (int i = 0; i < end; i++)
 delete tokenarray[i];
 delete tokenarray;
}

token & parse_array::operator[](int index) {
 if ( index >= end ) {
 cerr << "parse_array error: index " << index
 << " out of bounds\n";
 exit(1);
 }
 return *tokenarray[index];
}






[LISTING EIGHT]

// TAWK.CXX: parses a tawk script and reads an ascii file;
// generates results according to the tawk script.
// by Bruce Eckel,
#include "csascii.hxx"
#include "parse.hxx"

main (int argc, char * argv[]) {
 int screen = 0; // flag set true if screen output desired
 if (argc < 3) {
 cerr << "usage: tawk tawkfile datafile\n" <<
 "trailing -s pages output to screen";
 exit(1);
 }
 if (argc == 4) {
 if (argv[3][0] != '-') {
 cerr << "must use '-' before trailing flag\n";
 exit(1);
 } else
 if (argv[3][1] != 's') {
 cerr << "'s' is only trailing flag allowed";
 exit(1);

 } else
 screen++; // set screen output flag true
 }
 istream tawkfile(new filebuf->open(argv[1], input));
 parse_array Apreamble(tawkfile); // the @preamble
 parse_array Amain(tawkfile); // the @main section
 parse_array Aconclusion(tawkfile); // the @conclusion
 csascii datafile(argv[2]); // make a comma-separated ASCII
 // object from the second arg
 // ------ @preamble ------
 for (int i = 0; i < Apreamble.size(); i++)
 cout << Apreamble[i]; // preamble can only contain strings
 if(screen) {
 // ANSI reverse video sequence:
 cout << chr(27) << "[7m" << "press any key" <<
 chr(27) << "[0m";
 getch();
 }
 // ------ The Central Loop (@main) -------
 do { // for each record in the data file
 if(screen) cout << chr(27) << "[2J"; // ANSI clear screen
 for(int i = 0; i < Amain.size(); i++) {
 switch(Amain[i].token_type()) {
 case fieldnumber:
 cout << datafile[Amain[i].field_number()];
 break;
 case string:
 cout << Amain[i];
 break;
 case if_:
 int fn = Amain[i].field_number();
 if (datafile[fn].size() == 0) { // conditional false
 int level = Amain[i].nesting_level();
 // find the "else" statement on the same level:
 while ( !(Amain[i].token_type() == else_
 && Amain[i].nesting_level() == level))
 i++;
 } // conditional true -- just continue
 break;
 case else_: // an "if" conditional was true so skip
 // all the statements in the "else" clause
 int level = Amain[i].nesting_level();
 // find the "endif" statement on the same level:
 while ( !(Amain[i].token_type() == endif_
 && Amain[i].nesting_level() == level))
 i++;
 break;
 case endif_: // after performing the "else" clause
 break; // ignore it; only used to find the end
 // of the conditional when "if" is true.
 default: // should never happen (caught in parsing)
 cerr << "unknown statement encountered at run-time\n";
 exit(1);
 }
 }
 if(screen) {
 cout << chr(27) << "[7m" <<
 "press a key (ESC quits)" << chr(27) << "[0m";
 if( getch() == 27) break;

 }
 } while (datafile.next()); // matches do { ...
 // ------ @conclusion ------
 for ( i = 0; i < Aconclusion.size(); i++)
 cout << Aconclusion[i]; //conclusion contains only strings
}






[LISTING NINE]

# makefile for tawk.exe & lookup.exe
# Zortech C++:
CPP = ztc
# Glockenspiel C++ w/ MSC 4:
#CPP = ccxx !4

all: tawk.exe lookup.exe

tawk.exe : tawk.obj parse.obj csascii.obj field.obj
 $(CPP) tawk.obj parse.obj csascii.obj field.obj

lookup.exe : lookup.cxx csascii.obj field.obj
 $(CPP) lookup.cxx csascii.obj field.obj

tawk.obj : tawk.cxx parse.hxx csascii.hxx field.hxx
 $(CPP) -c tawk.cxx

parse.obj : parse.cxx parse.hxx
 $(CPP) -c parse.cxx

csascii.obj : csascii.cxx csascii.hxx field.hxx
 $(CPP) -c csascii.cxx

field.obj : field.cxx field.hxx
 $(CPP) -c field.cxx







[LISTING TEN]

@! REFORM.TWK
@! A tawk script to reformat a comma-separated ASCII file
@! with 6 fields. This creates a new CS-ASCII file with
@! fields 4 and 5 combined.
@main
"@(0)","@(1)","@(2)","@(3)","@(4)@?4@: @~@.@(5)"
@conclusion
@end







[LISTING ELEVEN]


@! WALLET.TWK
@! Tawkfile to create a tiny phone listing for a wallet
@! on a Hewlett-Packard Laserjet-compatible printer
@! From a comma-separated ASCII file generated by a DBMS
@preamble
@<27>&l5C@! approximately 10 lines per inch
@<27>(s16.66H@! small typeface, built into Laserjet
@main
@! last, first, (area code) phone1
@(0),@(1)(@(2))@?3@:@(3)
@ phone2, if it exists
@?4@:@(4)
@~@.@~@?4@:@(4)
@~
@.@.@conclusion
@<27>E @! Reset the Laserjet
@end

[EXAMPLE 1]

class tiny {
 // private stuff here (this is a comment)
 int i;
 public: // public stuff here:
 print() { // an "in-line" function
 printf("i = %d\n",i);
 }
 tiny(int j); // constructors have the class name
 ~tiny() {} // destructors use a tilde
}; // classes end with a brace and a semicolon

tiny::tiny(int j) { // non inline definition
 i = j;
}

main() {
 tiny A(2); // implicit constructor call
 // A.i = 30; // error! private member
 A.print(); // calling a member function
 // implicit destructor call at end of scope
}





[EXAMPLE 2]


#include <stream.hxx> // cout automatically defined
main() {
 cout << "Hello, world!\n" << "I am "
 << 6 << "today!\n";

}




[EXAMPLE 3]

filebuf f1;
if (f1.open(argv[1],input) == 0) {
 cout << "cannot open " << argv[1] << "\n";
 exit(1);
}
istream infile(&f1);



[EXAMPLE 4]


"Ball","Mike","Oregon Software C++ Compiler"
"Bright","Walter","Zortech C++ Compiler"
"Carolan","John","Glockenspiel C++ Translator"
"Stroustrup","Bjarne","AT&T, C++ Creator"
"Tiemann","Michael","Free Software Foundation C++ Compiler"






































May, 1989
QUICKDRAWING WITH XCMDS


Don't let HyperTalk slow you down




Jay Martin Anderson


Jay is a professor of computer science in the department of mathematics and
astronomy and can be reached at Franklin & Marshall College, Lancaster, PA
17604-3003. Before joining the F&M faculty, he was a software developer at
Tymlabs Corp., Austin, Tex. and the director of Academic Computing and a
professor of chemistry at Bryn Mawr College in Bryn Mawr, Penn.


It's no secret that XCMDs (eXternal CoMmanDs) can be used to provide access to
the Macintosh Toolbox. Since the introduction of HyperCard, in fact, a variety
of XCMDs that provide access to dialog boxes, the standard Macintosh file
system, and the serial port have been published. In this article, I will
examine a few examples of using external commands to provide access to
QuickDraw.
Why is access to QuickDraw useful? The process of drawing with HyperTalk is
directly tied to the painting tools that are provided with HyperCard. That is,
to draw a straight line, you select the line tool and drag it from the
starting to the ending point. To draw a straight line using one of the
available painting patterns, select a brush tool, then a pattern, and finally
drag the brush tool from the starting to the ending point. The drawing that
has been done appears on the card and, unless the stack has been protected
against writing, will be saved with the card when the stack is closed.
It is clear (even to a casual observer) that something happens in HyperCard
when you select any of the painting tools. There is a noticeable pause, the
cursor turns into the watch, and the menu bar (if visible) expands to include
the paint and options menus. A similar transition takes place in reverse when
you abandon the painting tools for the browse tool. In addition to the time
penalty that you pay when you select a painting tool and then abandon it, the
appearance of the card or stack also suffers. While the painting tools are
being selected, the cursor flickers and bounces from browse to brush and back;
the menubar extends to reveal two additional menus and then contracts when the
painting tools have been abandoned. Although this activity is necessary and
helpful when you are doing the painting, it is unattractive and distracting
when you only expect to watch a drawing appear.
If you could avoid using the painting tools and make calls to QuickDraw
directly, you could avoid both the time penalty in loading and unloading the
painting tools, and the aesthetic penalty you experience while the cursor and
menubar flicker and jump.
Access to QuickDraw can also be dangerous. QuickDraw operations take place on
the screen, not on the card in a HyperCard stack. The person using a stack may
not be able to distinguish between a display drawn with QuickDraw routines and
a display drawn with HyperTalk painting tools; but the drawing made with the
painting tools is (in principle) savable on the card in the stack, whereas the
drawing made with QuickDraw routines is not. Furthermore, mixing QuickDraw
routines with painting tools can lead to undesired results. Loading the
painting tools, for example, refreshes the display so that only the card
contents are visible; any drawing made on the screen with QuickDraw routines
will disappear. Finally, the user of QuickDraw must be careful to confine the
drawing to the active window on the screen, whose dimension is the same as the
HyperCard card. The user of QuickDraw cannot draw on the card and cannot draw
outside the region bounded by the card.
In a HyperCard tutorial, designed to demonstrate and teach the features of
Graphic Session (a commercial terminal emulator), I needed to draw a gray
cross hair on the display. When the mouse was clicked, I needed to erase the
first gray cross hair and to draw another gray cross hair at the mouse
position. Using HyperTalk and painting tools led me to the two scripts shown
in Listing One.
Although these scripts serve to provide the necessary functionality, they are
slow to execute and distracting to watch. Furthermore, this is just the
situation where drawing on the screen and not on the card is useful: I do not
want cross hairs left behind on the card after clicking. The user is supposed
to practice clicking, thereby moving the cross hair; the user is not supposed
to alter the tutorial stack. Consequently, I appealed to QuickDraw directly
and developed two brief XCMDs --for pen movement with and without drawing. The
XCMD that performs pen movement without drawing is called XMoveTo, and it is a
simple "cover" for the QuickDraw routine MoveTo. The XCMD that performs pen
movement with drawing is called XLineTo; besides serving as a "cover" for the
QuickDraw routine LineTo, XLineTo also includes the ability to change the pen
pattern and drawing mode. Therefore, XLineTo can both draw and erase. The
syntax of these two XCMDs is shown in Listing Two. The new HyperTalk scripts
for drawing the initial cross hair and erasing and redrawing the cross hair on
a mouse click are shown in Listing Three.
These scripts are no more complex than the script using the painting tools;
the drawing takes place faster with the XCMD script than with the tools
script, and there is no distracting change to the cursor or menu bar while the
drawing takes place.
The two XCMDs were developed using Think Technology's Lightspeed C (Version
3.0). The C source code for XMoveTo is shown in Listing Four.
The source code for XLineTo, shown in Listing Five, is slightly more complex;
it permits a variable number of parameters (two are required, the third and
fourth are optional) and it must select a QuickDraw pen pattern and pen
drawing mode.
There are four steps to making a HyperTalk script using an XCMD, which is
written in Lightspeed C. Two of these steps are handled within Lightspeed C
itself.
1. To construct a Lightspeed "project," load the appropriate Macintosh
libraries, following the guidelines in the Lightspeed C manual. Then create
and edit the source code file. Finally, build or make the project within
Lightspeed C.
2. Next, use Lightspeed C to construct an XCMD instead of an application. The
XCMD is left -- as a pure code resource -- in a file.
3. The third step in constructing the HyperTalk script using an XCMD requires
a resource editor, such as ResEdit. By using ResEdit, you can move the XCMD
resource, such as XMoveTo, from the file in which Lightspeed C leaves it into
a HyperCard stack. These steps are repeated in order to create and embed each
XCMD in the same stack. The XCMD's XLineTo and XMoveTo are now available
anywhere within the stack, but in no other stack. If I wanted the XCMDs to be
available to other stacks as well, I could have embedded them in the home
stack or in HyperCard itself.
4. The fourth step is to write HyperTalk scripts, as shown earlier, to use
these XCMDs in order to achieve faster, less detracting drawing.
On a Macintosh with sufficiently large memory (probably 2.5 Mbytes or larger),
you can run HyperCard, Lightspeed C, and ResEdit all under MultiFinder, and
quickly bounce from editing and compiling to managing resources to
constructing HyperTalk scripts. On a smaller Macintosh, or without the use of
MultiFinder, you will have to perform these steps separately by quitting each
application before starting another.
The same procedures that have been discussed here for using QuickDraw LineTo
and MoveTo procedures from a HyperCard stack can be applied to other QuickDraw
procedures as well. Although it did not come up in the design of the Graphic
Session tutorial, I might equally well have wanted to demonstrate the
positioning of a conventional box-like alpha cursor by a mouse click. In this
situation, instead of drawing a gray cross hair at the location of the mouse
click, I would draw a small black rectangle at the location of the mouse
click. This leads naturally to the development of an XCMD, called XBox, which
covers the QuickDraw procedure PaintRect, as shown in Listing Six.
Finally, some amusing effects can be achieved by combining QuickDraw
operations on the screen with the painting of similar objects on the card. For
example, the small rectangle painted with the XCMD XBox can be repeated often,
leading to the effect of animation. This animated rectangle can then appear to
come to rest in the form of a rectangle that is actually painted on a card.
The script in Listing Seven illustrates this action: for this example, the
animated rectangle painted by the XCMD XBox appears on the screen in front of
a blank card; after the animation sequence is finished, a card with a painted
rectangle is brought into view.
The construction of XCMDs that cover QuickDraw procedures can be accomplished
easily within either C or Pascal, and can be done largely by rote. The use of
XCMDs for access to QuickDraw allows you access to faster, but transient
drawing.


Notes


1. Graphic Session, software for the Macintosh that emulates an
Hewlett-Packard HP 2393A monochrome graphics terminal, is available from
Tymlabs Corp., 811 Barton Springs Rd., Austin, TX78704; 512-478-0611; the
HyperCard tutorial for this product is available from the author, or for
downloading on GE-nie and CompuServe.
2. Version 3.0 of THINK C (Symantec Corp., 10201 Torre Ave., Cupertino, CA
95014; 408-253-9600) is MultiFinder compatible. Its predecessor, Lightspeed C
Version 2.0x, was not.
3. For well-worked examples in Lightspeed C, see Gary Bond, XCMDs for
HyperCard (MIS Press, 1988).


Availability


All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063; or call 800-356-2002 (from inside
Calif.) or 800-533-4372 (from outside Calif.). Please specify the issue number
and format (MS-DOS, Macintosh, Kaypro).



_Quickdrawing With XCMDs_
by Jay Martin Anderson


[LISTING ONE]



 on openCard
 -- draw the crosshair at 400, 200 when card is
 -- first displayed
 global oldH, oldV
 -- keep location of crosshair in global vars
 put 400 into oldH
 put 200 into oldV
 choose brush tool
 set the brush to 4
 set the pattern to 22
 drag from oldH-8,oldV to oldH+8,oldV
 drag from oldH,oldV-8 to oldH,oldV+8
 choose browse tool
 end openCard

 on mouseUp
 -- erase crosshair where it was
 -- draw crosshair at new location
 global oldH, oldV
 choose eraser tool
 drag from oldH-8,oldV to oldH+8,oldV
 drag from oldH,oldV-8 to oldH,oldV+8
 choose brush tool
 set the brush to 4
 set the pattern to 22
 get the mouseH
 put it in oldH
 get the mouseV
 put it in oldV
 drag from oldH-8,oldV to oldH+8,oldV
 drag from oldH,oldV-8 to oldH,oldV+8
 choose browse tool
 end mouseUp






[LISTING TWO]

 XMoveTo x,y
 -- x and y are coordinates in pixels in the usual
 -- QuickDraw coordinate system.
 XLineTo x,y,pen,mode
 -- x and y are coordinates as for XMoveTo; pen is
 -- the number of a QuickDraw pen pattern, selected
 -- from the system pattern list as shown on page I-474
 -- of Inside Macintosh; mode is 0 to erase and 1 to
 -- draw. If mode is omitted, it is presumed to be 1
 -- (draw); if pen is omitted, it is presumed to be 1
 -- (solid black).






[LISTING THREE]


 on openCard
 -- draw crosshair at 400, 200 when card is
 -- first displayed
 global oldX, oldY
 -- maintain position of crosshair in global var
 put 400 in oldX
 put 200 in oldY
 XMoveTo oldX-8,oldY
 XLineTo oldX+8,oldY,4,1
 XMoveTo oldX,oldY-8
 XLineTo oldX,oldY+8,4,1
 end openCard

 on mouseUp
 -- erase former crosshair and draw new crosshair
 -- at position of mouse-click
 global oldX, oldY
 -- erase former crosshair; pattern 4 (gray);
 -- mode 0 (erase).
 XMoveTo oldX-8, oldY
 XLineTo oldX+8, oldY,4,0
 XMoveTo oldX, oldY-8
 XLineTo oldX, oldY+8,4,0
 -- draw new crosshair; pattern 4 (gray),
 -- mode 1 (draw).
 put the mouseH in oldX
 put the mouseV in oldY
 XMoveTo oldX-8,oldY
 XLineTo oldX+8,oldY,4,1
 XMoveTo oldX,oldY-8
 XLineTo oldX,oldY+8,4,1
 end mouseUp






[LISTING FOUR]

 #include <Types.h>
 #include <MemoryMgr.h>
 #include <OSUtil.h>
 #include <QuickDraw.h>
 #include "XCmd.h"
 pascal void main(paramPtr)
 XCmdBlockPtr paramPtr;
 {
 short x,y;
 Str31 str;
 /* There are two parameters to the XCMD XMoveTo:
 * the x and y coordinates to which to move. These
 * are found as C-strings (null-terminated
 * strings), converted to Pascal-strings, and then
 * to integers. There must be just two parameters;
 * if there are more or less, nothing happens.
 */
 if (paramPtr->paramCount == 2)

 {
 ZeroToPas(paramPtr,
 (char *)*(paramPtr->params[0]),
 (StringPtr)&str);
 x = StrToNum(paramPtr, &str);
 ZeroToPas(paramPtr,
 (char *)*(paramPtr->params[1]),
 (StringPtr)&str);
 y = StrToNum(paramPtr, &str);
 MoveTo(x, y);
 }
 }







[LISTING FIVE]


 #include <Types.h>
 #include <MemoryMgr.h>
 #include <OSUtil.h>
 #include <QuickDraw.h>
 #include "XCmd.h"
 pascal void main(paramPtr)
 XCmdBlockPtr paramPter;
 {
 short x, y, pat, mode;
 Str31 str;
 Pattern thePattern;
 PenState oldPenState;
 /* There are four possible parameters; two are
 * required. The first two (required) are the
 * x and y coordinates to which to draw. The
 * third is the pattern, taken from SysPatList,
 * (see Inside Macintosh I-474); the fourth is
 * the mode (0 = erase, 1 = draw). The four
 * parameters are found as C-strings (null-
 * terminated strings), converted to Pascal-
 * strings, and then to short integers. If the
 * fourth parameter is omitted, it is presumed
 * to be "1" for draw. If the third parameter
 * is also omitted, it is presumed to be "1" for
 * black. If there are fewer than 2 or more than
 * 4 parameters, nothing happens.
 */
 if ((paramPtr->paramCount < 2) 
 (paramPtr->paramCount > 4)) return;
 ZeroToPas(paramPtr, (char *)*(paramPtr->params[0]),
 (StringPtr)&str);
 x = StrToNum(paramPtr, &str);
 ZeroToPas(paramPtr, (char *)*(paramPtr->params[1]),
 (StringPtr)&str);
 y = StrToNum(paramPtr, &str);
 if (paramPtr->paramCount < 3)
 pat = 1;

 else
 {
 ZeroToPas(paramPtr,
 (char *)*(paramPtr->params[2]),(StringPtr)&str);
 pat = StrToNum(paramPtr, &str);
 }
 if (paramPtr->paramCount < 4)
 mode = 1;
 else
 {
 ZeroToPas(paramPtr,
 (char *)*(paramPtr->params[3]),
 (StringPtr)&str);
 mode = StrToNum(paramPtr, &str);
 }

 /* preserve former pen state */
 GetPenState(&oldPenState);

 /* set pen state to our pattern and mode */
 GetIndPattern(&thePattern, 0, pat);
 PenPat(thePattern);
 if (mode) PenMode(patCopy);
 else PenMode(patBic);

 /* draw the line */
 LineTo(x, y);

 /* restore pen state */
 SetPenState(&oldPenState);
 }







[LISTING SIX]
/*
 XBox -- a HyperCard user-defined command in C.
 J. M. Anderson, 1988
 All Rights Reserved.
 This XCMD draws a rectangle.
*/
#include <Types.h>
#include <MemoryMgr.h>
#include <OSUtil.h>
#include <QuickDraw.h>
#include "XCmd.h"

/* **** WARNING: DO NOT USE GLOBAL VARIABLES! **** */

pascal void main(paramPtr)
 XCmdBlockPtr paramPtr;
{
 short x, y, u, v, pat, mode;
 Rect box;
 Str31 str;

 Pattern thePattern;
 PenState oldPenState;

 /* The six parameters are the x and y coordinates of
 * the upper left corner of the box, its width and
 * height,the pattern (taken from the SysPatList)
 * and the mode with which to draw. These are found
 * as C-strings, converted to P-strings, and then to
 * numbers. The "mode" is either 0 to erase or 1 to
 * draw. There may be four, five, or six
 * parameters. If the sixth parameter is omitted, it
 * is presumed to be "1" for draw. If the fifth
 * parameter is omitted,it is presumed to be "1" for
 * black. The x and y coordinates and sizes u and v
 * must be present. If there are fewer than 4 or more
 * than 6 parameters, nothing happens.
 */
 if ((paramPtr->paramCount < 4) 
 (paramPtr->paramCount > 6)) return;
 ZeroToPas(paramPtr,(char *)
 *(paramPtr->params[0]),(StringPtr)&str);
 x = StrToNum(paramPtr,&str);
 ZeroToPas(paramPtr,(char *)
 *(paramPtr->params[1]),(StringPtr)&str);
 y = StrToNum(paramPtr,&str);
 ZeroToPas(paramPtr,(char *)
 *(paramPtr->params[2]),(StringPtr)&str);
 u = StrToNum(paramPtr,&str);
 ZeroToPas(paramPtr,(char *)
 *(paramPtr->params[3]),(StringPtr)&str);
 v = StrToNum(paramPtr,&str);
 if (paramPtr->paramCount < 5)
 pat = 1;
 else
 {
 ZeroToPas(paramPtr,(char *)
 *(paramPtr->params[4]),(StringPtr)&str);
 pat = StrToNum(paramPtr,&str);
 }
 if (paramPtr->paramCount < 6)
 mode = 1;
 else
 {
 ZeroToPas(paramPtr,(char *)
 *(paramPtr->params[5]),(StringPtr)&str);
 mode = StrToNum(paramPtr,&str);
 }

 /* preserve pen state */
 GetPenState(&oldPenState);

 /* set pen state to our pattern and mode */
 GetIndPattern(&thePattern, 0, pat);
 PenPat(thePattern);
 if (mode) PenMode(patCopy);
 else PenMode(patBic);

 /* draw the box */
 SetRect(&box, x, y, x+u, y+v);

 PaintRect(&box);

 /* restore the pen state */
 SetPenState(&oldPenState);
}






[LISTING SEVEN]

on mouseUp
 -- Effect animation of a small black rectangle. This
 -- card is blank; the next card has a small black
 -- rectangle painted on the card at location 298, 167.
 -- draw and erase many times a QuickDraw black
 -- rectangle; its location starts at the left edge
 -- of the card (x = 5), and 167 pixels down. It
 -- moves to x = 298, y = 167.
 put 3 into x
 repeat while x < 298
 XBox x, 167, 10, 10, 1, 1 -- draw
 XBox x-5, 167, 10, 10, 1, 0 -- erase
 put x + 5 into x
 end repeat
 -- now show next card with rectangle painted in place
 go to next card
end mouseUp
































May, 1989
QUICK C VERSUS TURBO C


The competition goes on...




Scott Robert Ladd


Scott Ladd is a full-time freelance computer journalist. You can reach him at
302 N 12th, Gunnison, CO 81230.




Product Information


Turbo C -- Borland International 1800 Green Hills Rd. Scotts Valley, CA
95066-0001 408-438-8400
Quick C -- Microsoft Corp. Box 97017 Redmond, WA 98073-9717 206-882-8080
In the world of software, the war for market share goes on and on. At the
forefront of this conflict are two titans, Microsoft and Borland. It is
inarguable that these two industry giants influence the type and quality of
products available both now and in the future. In particular, their language
products are among the first in line when companies and individuals consider
which products to buy.
This article reviews and compares Borland's Turbo C and Microsoft's QuickC
compilers. Due to their low prices and the name recognition of their vendors,
Turbo C and QuickC (or TC and QC) are often the C compilers chosen by people
who are just learning the C language. One of the most popular questions raised
in user groups and electronic forums is, "Which one is best?" That's not an
easy question to answer.
Part of the problem is that, although similar in many ways, TC and QC are
aimed at different programmer audiences. Microsoft targets QC chiefly at
newcomers and casual users. Experience with Version 1.0 apparently convinced
Microsoft that these are the buyers for QC, and so Version 2.0 is tailored to
their needs, with Microsoft C 5.1 for professionals. Meanwhile, Borland's TC
competes with both Microsoft C offerings, thus aiming higher. This introduces
some subtle, but telling differences.
And precisely because TC goes after the spectrum of Microsoft C products,
we're going to compare it with both in a two-part review: this month with QC,
in August with Microsoft C. That month's installment will include benchmark
results for all three.
TC and QC are very similar in scope and appearance. TC costs $150, while QC
costs $99. In the basic package, each product has an integrated environment
that includes an editor, a compiler, a linker, a debugger, and Make
facilities. Command-line utilities are also provided for programmers who
dislike environments. TC and QC are fast products -- both compile tens of
thousands of lines of source code per minute on an 80286-based PC.
Documentation for both products consists of paperback manuals. With all of
these similarities, it might appear difficult to find true qualitative
differences between the two products. Appearances, however, can be
deceiving....


Installation


Both TC and QC offer automated installation programs. TC's automated
installation program is a bit "prettier," with fancy pop-up windows and
background shading. QC's installation is more austere, but equally adequate.
It is possible to make TC and QC work in a diskette-only environment -- but I
strongly recommend against doing so. The installation programs set the
compilers up on multiple diskettes, and your arm will quickly become tired
swapping disks. The situation improves if you have high-capacity diskettes,
such as the 3 1/2-inch drives found in many newer machines. I had no problems
using both QC and TC on my dual-720K microfloppy laptop. A hard disk is best
for both.
The installation programs leave little room for complaint. The user is given
complete control over which components are installed and which directories the
components are installed into. Each installation program reminds the user to
set the proper environment variables. In keeping with QC's less experienced
target audience, Microsoft's more detailed (but also more verbose) description
of the installation process is a bonus for those unfamiliar with memory models
and compiler installation.
While it's not a "bug" per se QC's installation program does have the annoying
habit of placing the font files into the same directory as the sample
programs. I always think of sample code as something to be deleted (sooner or
later). I would prefer the fonts to be copied into the directory where library
files are stored.
If you use a hard disk drive, you'll need more than 2 Mbytes of free space on
the drive when you install these products. After cleaning up example files and
read-me files, you can easily fit either product into about one and one-half
Mbytes.


Documentation


TC's documentation consists of a 425-page user's guide, and a 612-page
reference guide. The user's guide contains tutorials on the environment, the
debugger, and the C language. The reference guide includes a library function
dictionary and detailed information about the command-line utilities.
Borland has created a good set of tutorials that cover all of the necessary
subjects and include well-designed examples. I especially like the "helpful
hints," such as a section on common C programming pitfalls. These "dos and
don'ts" are valuable to the beginning programmer who has little experience
with C.
Documentation of the TC command-line compiler and utilities is terse but
complete. By placing this information in appendices, Borland seems to indicate
that they expect most programmers to use the environment. Examples that show
how to use the command-line compiler and the linker are virtually nonexistent.
QC's documentation also consists of three manuals --a 64-page Up and Running
booklet, the 336-page toolkit manual, and the 376-page C for Yourself. Up and
Running covers installation and includes some basic details about how to run
the environment. The toolkit manual covers the command-line utilities in
detail. C for Yourself is a C-language tutorial and reference.
Despite the number of its pages, I consider this set of documentation largely
inadequate. Microsoft assumes that the user will run through the interactive
tutorial. While the tutorial is well-designed and informative, it does not
touch on all of the subjects that a complete manual would cover. At least half
of the editor functions are not documented anywhere. Many menu items are
ignored or glossed over. The user must explore the environment in order to
completely learn it. While the tutorial is a general introduction to the
product, it is not comprehensive enough to replace thorough paper
documentation.
C for Yourself is a superb C tutorial that leads the programmer through all of
the fundamentals of the language and into implementation-specific facets of
QC. Unfortunately, this manual falls short of perfection by omitting any
mention of many important library functions. For instance, there is no mention
about the MS-DOS interface functions, such as intdos( ) and int86( ). Several
keywords, including far, near, and interrupt, are also ignored. Information
about these subjects is available in the on-line help (described later), but
that is not a viable solution --a beginning C programmer will not know which
questions to ask in order to be able to look up this missing information
on-line. The solution is to purchase a $25 supplemental manual.
Microsoft's documentation of the command-line utilities is first-class. The
toolkit manual begins with a short section discussing the utilities in
general, and then moves on to chapters that describe each tool in detail.
Liberal examples illustrate the use of the different utilities.


On-line Help


Microsoft has put a great deal of effort into QC's on-line help system, which
it considers an essential part of the documentation. The help system is
context-sensitive and uses some of the concepts of hypertext. You point to an
item with the cursor and then press F1 to get help; this method works even on
help information. All of the library functions, error messages, and C keywords
are documented. "Hotlinks" in each description block can be used to move to
other areas, including an alphabetical index and a table of contents. You can
set "bookmarks" in the help system that let you quickly return to a specific
piece of information. As an added bonus, you can cut and paste information in
the help system (including example programs) into a file that you are editing.
Despite its many strong points, QC's help system has a few holes. The Huge
keyword can be found in the list of C keywords, but not in the alphabetical
index. Also, the help system requires a great deal of hard disk space -- more
than half a megabyte! Because the system is built into the QC environment, it
is impossible to access the help information if you use your own editor. On
the other hand, the QC help has more than 200 working code samples that the
user can copy directly into a program.
TC's help system is less sophisticated but adequate. A general index and/or
table of contents is missing from the help system. To retrieve help
information on the current window, you press F1. To obtain details about the
item that the cursor is currently pointing to, such as a standard library
function or keyword, you press Ctrl-F1. By pointing to a header file name
(such as stdio.h) with the cursor and pressing Ctrl-F1, you can access a list
of the function prototypes and definitions that the file contains. In the long
run, though, TC's help system is less extensive than the help system in QC.
However, TC provides a TSR to make the help information available in an
external editor. This is very handy for those who prefer to use their favorite
editor and the command-line utilities.

Thus we have a trade-off: more and better paper documentation but less on-line
help in TC, more help and less detailed manuals with QC.


Language Implementations


QC and TC follow the ANSI standard as well as, or better than, most other
MS-DOS C compilers on the market. Both products include all of the common
MS-DOS language extensions, such as the near and far keywords. Programmers
will not be disappointed with the languages supported by either of the
products.
While both compilers support inline assembly, QC's implementation is by far
the better. With TC, each line of inline assembly language must be prefaced
with the keyword _asm. TC also requires that you have a command-line
assembler, such as Borland's TASM or Microsoft's MASM, in order to use the
inline assembler. On the other hand, QC includes a complete built-in
assembler. Multiline blocks of inline assembly code can be created within
curly braces following a single instance of the _asm keyword.


Function Libraries


Competing vendors are always looking for ways to spice up their products. With
C compilers, an excellent way to do this is by adding functions to the
library. TC and QC sport some of the largest libraries in existence.
With the exception of multicharacter and international support functions, the
libraries are very complete from an ANSI standpoint. QC and TC provide
additional functions for direct access to the underlying hardware and MS-DOS.
Most of the function names and parameters are similar between the two
products.
Borland added an excellent graphics library with TC, Version 1.5, that was not
significantly changed for Version 2.0. When QC was first introduced, it
included a modest graphics library, which has been dramatically improved in
the most recent version of QC.
QC and TC support all of the basic graphics primitives, along with stroked and
bit-mapped fonts. TC "autodetects" the type of graphics adapter installed; QC
does not, but listings in the manuals provide a "trial and error" loop for
finding and entering the best graphics mode. Borland allows fonts and graphics
device drivers to be loaded at run time or to be made into linkable object
modules. Microsoft's graphics drivers are linkable, but the fonts must be
loaded from disk files at run time.
If you work with charts and tables, QC has as a real treat in store. Its
graphics library includes a set of presentation graphics functions. A special
structure is loaded with information that can be used to draw pie, bar,
stacked-bar, line, or scatter charts. Hatching, borders, legends, and headings
are all handled very nicely. By contrast, TC provides a full complement of
lower-level functions to help you build charts, but Microsoft's package is
more automated and convenient.


Environment


QC and TC are both chiefly integrated environments combining an editor, a
compiler, and a linker into a single, all-in-one program. A menu bar across
the top of the screen provides pull-down menus for various operations, such as
loading and saving files, compiling, and debugging. Program options can be set
through these menus and then saved to disk. Both programs permit such
configurations to be stored in files in different directories, offering you
the flexibility to specify options for different projects. When you start the
environment from a given project directory, the configuration stored there
becomes the default.
In keeping with its target audience of less experienced users, QC offers a
unique two-level menuing system. The lower (default) level provides a "bare
bones" set of menus, presumably intended to keep the novice user from being
intimidated by the environment. As the QC user becomes more confident, he or
she can set an option that activates the more complete menuing system.
The QC and TC editors are not going to excite programmers accustomed to
sophisticated editing programs such as Brief and QEdit. Both editors use an
extended subset of the WordStar command set. By "extended subset," I mean that
they use many of the WordStar commands, and also add several of their own. QC
has a menu option to install an alternate editor. In addition, QC's "notepad"
window allows you to edit more than one file at a time. QC and TC both include
utilities for redefining/adding command keys and otherwise adjusting the
editor to personal preferences.
Neither product supports editor macros or context-sensitive editing. QC
accommodates -- in fact, almost demands -- a mouse, which makes many
operations easier. (Why Borland products lack mouse support is a matter of
ongoing curiosity and annoyance.) The undo facilities of both editors are
virtually nonexistent.
Both products' compilers and linkers are very fast and support a full line of
memory models. TC supports a Tiny model (unavailable from Microsoft) that can
be made into .COM files. Otherwise, the basic code-generation capabilities of
the two products are similar. Errors are tracked and displayed in their own
window.
In QC's favor is its "incremental" compile and link. In effect, the compiler
only recompiles and relinks those parts of a program that have changed. On a
large source program, QC's ability to perform incremental compiles gives this
compiler an amazing speed advantage. This is particularly important during
debugging, when compiles often take place after only a few changes are made to
a program.
Make facilities are built into both environments. TC calls the control
entities "Project Files," while QC names them "Program Lists." Each lists the
files belonging to a multimodule program, and the compiler/linker is
intelligent enough to process only those modules that have changed since the
last program build. I like QC's built-in Make-better, because it creates a
true Make-file on disk for use with the stand-alone Make program. On the other
hand, TC needs differently-formatted project and Make files for the
environment and command-line utilities, even when the same source modules are
involved. There are conversion utilities, but the different formats are a
hassle.
Here's an interesting statistic about memory usage: When no files are loaded
TC uses 318K of memory. By contrast, QC uses only 185K, an amazing improvement
over the previous version that barely fit into 640K. Note, however, that TC
takes advantage of EMS when present, whereas QC does not.


Debuggers


The debuggers included in integrated environments are not as capable as their
stand-alone brethren. This does not mean they're useless; it merely means that
these debuggers are designed to detect the most common kinds of run-time
problems. The debuggers for QC and TC can step through programs line-by-line,
set breakpoints, watch variables while the program runs, and change the values
of variables while the program runs. They can also follow pointers, display
structures, examine the stack and registers, and provide output in several
forms (hex, decimal, and so on).
The TC debugger is an easy-to-use product that is solid and useful for finding
common bugs. Borland devotes a great deal of space in the TC manuals to the
discussion of debugging techniques and strategies. Unfortunately, TC does not
support conditional breakpoints or the "animation" (execution in slow motion)
of programs.
QC's debugger, a subset of Codeview, has several pluses and one surprising
minus. I like the "locals" window, which shows the values of all local
variables for whatever function is currently being traced. The "registers"
window shows the processor registers as if they were variables, and makes the
debugging of inline assembler much easier. The ability to store "history"
information permits you to retrace steps previously executed. User input can
also be recorded and replayed, saving a great deal of time when bugs are
located deep within a program's logic.
The omission in QC's debugger is almost unbelievable --it is not possible to
simply check a variable; instead you have to put the variable into the watch
window! This process involves several keystrokes, where one would be
sufficient. And having seen the value, it takes several more keystrokes to get
rid of it.
In contrast, TC's debugger allows you to see the value of any variable
instantaneously by pressing Ctrl-F4. The variable name under the cursor is
automatically displayed. You can "rip out" further text from the source --for
example, a structure field name --by pressing the right arrow key until the
complete name of the variable you want to see is in the window. If the cursor
location selects the wrong variable name, you simply type the one you want.
Similarly, you can type C expressions and see the result, and also change a
value "on the fly," as for example in finding out how the program deals with
an unexpected data value. You press Esc to make the examination window go
away. TC's integrated debugger also includes a watch window similar to QC's
for more permanently displaying selected variables and their dynamically
updated values.


Command-line Utilities


For those who dislike environments, QC and TC supply utilities for use at the
MS-DOS prompt. Each product includes a command-line compiler, linker, Make
facility, and object-module librarian. Each also furnishes some unique
utilities. For example, TC adds a preprocessor, a grep, an object-module
cross-reference utility, and a program to convert graphics drivers and fonts
to object modules. As for QC, it furnishes a utility to create customized help
files.


Conclusions


TC has few blemishes. For example, documentation of the command-line utilities
could be expanded, and real inline assembly would eliminate the need for an
external assembler. On the plus side, the integrated debugger is more
powerful, documentation is more complete, and the set of command-line
utilities is more extensive than that of QC. All in all, TC is a quality
product with features aimed at a more demanding, competent C programmer than
QC.
Probably the most important area where QC needs improvement is the paper
documentation --more information is needed for beginners who are moving toward
the realm of experts. In spite of this, QC has a slight edge over TC in a
couple of areas. QC cleanly supports inline assembler, has a superior graphics
library, and incorporates some impressive new technology with its incremental
compilation and linking capabilities. One dramatic improvement over the
earlier version is that QC is no longer a "toy" compiler with no selection of
memory models. It now supports all models except Tiny.
I truly like both products --each has good and bad points, and there is no
loser. The winners are the programmers who benefit from the competition that
generates better products. No matter which product you buy, be assured that
you're not wasting your money.
So much for TC versus QC. In the August issue we'll pit TC against QC's "big
brother" and wrap up with a benchmark report that compares the performance of
all three.






May, 1989
PROGRAMMING PARADIGMS


A Decade Later




Michael Swaine


Each year, The Association for Computing Machinery (ACM) gives its Turing
Award to a computer scientist who has earned distinction for technical
contributions of lasting and major importance to the field of computer
science. The recipients through 1985 were Charles Bachman, John Backus, E.F.
Codd, Stephen Cook, Edsger Dijkstra, Robert Floyd, R.W. Hamming, C.A.R. Hoare,
Kenneth Iverson, Richard Karp, Donald Knuth, John McCarthy, Marvin Minsky,
Allen Newell, Alan J. Perlis, Michael O. Rabin, Dennis Ritchie, Dana S. Scott,
Herbert Simon, Ken Thompson, Maurice V. Wilkes, J. H. Wilkinson, and Niklaus
Wirth.
I list them all here not only because they all deserve recognition, but also
to give a sense of the range of contributions honored, from Minsky's AI work
to Codd's creation of the relational database model, from Hoare's foundational
work in axiomatic semantics to Ritchie and Thompson's creation of C and Unix.
Clearly, recognition is not solely restricted to abstract, academic
achievements. And in fact the Turing Award lectures, delivered by the
recipients at annual conferences of the ACM, usually stress-broad issues of
concern to all programmers. These lectures have been collected in a great book
ACM Turing Award Lectures: The First Twenty Years: 1966-1985, Addison Wesley,
1987.
The 1978 Turing Award went to Robert Floyd for his work in the foundations of
the theory of parsing, programming language semantics, the automation of
program synthesis and verification, and the analysis of algorithms.
It was Floyd's Turing Award lecture, The Paradigms of Programming, that
inspired this column. In that lecture, he cited inadequacies in our store of
programming paradigms, in our knowledge of existing paradigms, in the way we
teach paradigms, and in the way our languages support the paradigms of their
user communities. Recently I talked with Professor Floyd about his thinking in
the area of programming paradigms a decade after the lecture.
Swaine: In this column, I've talked about paradigms generally at the level of
the programming language implementation. But you're really not working with
different paradigms at that level, are you?
Floyd: No, I do most of my programming in Pascal. There are paradigms on all
scales.
On one end of the scale you have the paradigm that's embodied in a language:
object-oriented programming and functional programming and logic programming
and all these things. At the other extreme there are things that would be
substantial conveniences right down at the micro level, like having parallel
assignment. Then there's the intermediate level, the paragraph of programming.
Swaine: How would you characterize your work today?
Floyd: My own programming efforts for a very long time have been concerned
with programming in the small rather than in the large. The programs I write
myself seldom go more than four pages of Pascal. Typically, I'm concerned with
getting the very good ten-to-fifty-line program for a mathematically
sharply-defined task. I spend a lot of my time deducing the mathematical
characteristics that the solution has to have and substantially less time
working out how the mathematical objects will be represented as data
structures and how the coding will go. I don't face the software engineering
problems of programming in the large.
Swaine: You work more at the paragraph level? Can you cite some examples of
these paragraph-level paradigms?
Floyd: A couple of those that are reasonably well known that I can mention as
typical are dynamic programming and branch-and-bound algorithms. Then I use
various subject-matter oriented paradigms. Statistical sampling is an example
of that.
Swaine: I think you'll have to explain how statistical sampling can be a
programming paradigm.
Floyd: There are problems that are not in themselves statistical problems but
where you can improve the performance of an algorithm by taking a random
sample of the data set and pre-computing something for that, in effect, gives
you an estimate of what your final result is going to be.
The algorithm that Ron Rivest and I developed for calculating the median and
other quantiles is an example of that. Random sampling is in no way required,
but it turns out to be the way that we got an algorithm that appears to be
asymptotically close to the maximum efficiency, and I don't think anybody has
discovered a non-sampling algorithm that can accomplish that.
Here's another example. I have a mathematical sample that is quite tightly
clustered, where the mean is very large compared to the standard deviation,
and I want to calculate the standard deviation very precisely. The obvious
ways of calculating standard deviation are numerically rather dangerous. Not
only can they be very inaccurate, but the cumulative effect of rounding errors
can very easily lead to your taking the square root of a negative number. One
way to attack that problem would be to take a random subset of the data, using
that get an estimate of the mean, then to run through the entire data set,
subtracting that estimate off all the data. That doesn't change the standard
deviation, but it does scale the data down in magnitude so that the variations
among them are now large compared to the data themselves. And that makes the
problem numerically well conditioned.
Swaine: So in one case you've used sampling to produce an efficient algorithm,
and in another case to improve accuracy?
Floyd: Yes, that kind of use of sampling is a general paradigm. I've used it
in lots of areas that are not necessarily terribly close to one another. It's
obviously not something that's going to be put forward as a way to solve all
the world's problems, the way that functional programming and such are
presented as the universal solvent. But I think that the high-level programmer
is going to need to know a lot of these paradigms on the intermediate scale,
the scale of the sentence and the paragraph rather than the scale of the page
and the book.
Swaine: One paradigm that some have put forth as the universal solvent is
structured programming. You describe it in your Turing Award lecture as "the
dominant paradigm in most current treatments of programming methodology." But
it has its detractors: Rich Gabriel at Software Development 88 last year said
something to the effect that structured programming had done more damage to
programmers' thought processes than the GOTO statement.
Floyd: I don't know why Rich would have said that.
Swaine: Don't take the quote too seriously; I'm misquoting from memory. But
where do you stand on structured programming today?
Floyd: As I said in the lecture: "Its firmest advocates would acknowledge that
it does not by itself suffice to make all hard problems easy." An example
where the top-down structured programming approach doesn't work very
effectively is the eight queens problem. Wirth tackled that in Program
Development by Stepwise Refinement, and he got an answer, but it's a real
kludge. There's an earlier paper of mine that solved it using a
non-deterministic paradigm. You write a program for an imaginary
non-deterministic computer that makes arbitrary choices in deciding where to
put the queens on the chessboard, but also makes tests to see if it's still on
track, to see if it's violated any rules yet, and aborts if it has made any
mistakes. Then that can be macro-expanded into a backtracking program that
backs up wherever the original program would have aborted, and systematically
searches the entire space of choices.
The program that you write as a non-deterministic program is very simple and
is itself within the classical top-down discipline, but the macro expansion
from it produces a program in which the loops don't nest. If you looked
directly at the result from the macro expansion you'd say that there's no
structure here, this is a kludge, this is hacking. But there is a conceptual
structure in the sense that it's the result of a simple transformation from a
simple formulation.
That's the kind of thing that makes me think of the top-down viewpoint as
excessively limited. It has its uses, but within that framework there are some
thoughts that are too complicated to think.
Swaine: In your lecture, you dismiss the idea that automatic programming will
solve the so-called software crisis. Is there any more promise there today?
Floyd: I don't think that there is going to be any universal solvent for
programming. Programming is hard.
There has been some interesting work on automatic program generation. Cordell
Green has concentrated on algorithms for sorting and searching. He estimates
that a program to come close to human performance in this area has to simply
know a lot of facts that a skilled programmer in this area would know. The
number of facts is on the order of five hundred.
So one can't expect to march up to a completely naive computer and give it a
problem and expect it to come out with a really neat program. Because really
net programs are based on a lot of world knowledge.
I am sure that what he found in his subject matter you would find in any
subject matter that people want to apply programming to. I know that if you
want to do sophisticated graphic programming, there's a lot that you have to
know about computational geometry and appropriate representation and
algorithms for triangulation.
These things don't invoke a software crisis. These things are problems
typically at the scale of the programs that I have been talking about, that
fit on a page or so. But they are hard. You have to know a lot and be able to
use that knowledge in order to be able to do these things well.
Was it Euclid who said "There is no royal road to geometry?" There is no royal
road to programming. Programming is hard. On the other hand, there are
problems that look very hard, but if you have the appropriate paradigm they
become easy.
Swaine: For example?
Floyd: In the context of a spelling checker, I've got a misspelled word, and I
want to compare it to a word from the dictionary, to measure how different
they are. I'm looking for the most similar word in the dictionary; which is to
say I want to know the minimum number of editing operations, from a limited
set of editing operations, that will convert the one word into the other.
Years ago I gave that as a final project in a programming course in which I
hadn't talked about dynamic programming, and nobody got anywhere with it.
Those who did it ended up limiting themselves very severely, restricting
themselves to looking at one or two editing operations. But if you formulate
it as a dynamic programming problem where the subproblems are to measure the
difference between every prefix of the one word and every prefix of the other,
and you go through those in increasing length of these prefixes, it's a very
straight-forward problem. Ten or twenty lines does it. I could write it in
half an hour and have time left over to trim my nails.
Swaine: What general paradigm do you find most useful in your work?
Floyd: In my programming, I try to find mathematical characterizations of what
I'm looking for, to limit my search space.
You're familiar with loops that you have to go through n plus one-half times?
For example, you're processing a data set and one of the data is a sentinel.
You have to read all the n+1 data, but the processing part omits the sentinel.
That can be very difficult to explain to a novice programmer.
Swaine: And yet it's something that novice programmers run into immediately.
Floyd: Very definitely. Well, they can deduce from the statement of the
problem some things that tell them that their programs are going to need some
things that they might not guess. It's plausible that the program is going to
use some kind of iteration, and that the processing step is going to be inside
the loop so that they can do it as many times as they need to. But they're
only going to process n items, while they need to read n+1 items. From this
they can deduce that there has to be a read outside the loop or they have to
have some conditional branching inside the loop. And when they argue that the
test that controls the loop is going to have to always have one datum
available to it, that tells them that they need one read outside the loop, and
in particular before the loop. And if they're going to alternate between
reading and processing, that tells them that inside the loop, they need first
to process and then to read, which is contrary to what they'd expect.
So by just some logical reasoning you characterize the kind of program you
need well enough that your search space is starting to get quite reasonable,
even if you are a novice.
Swaine: Speaking of novices, one of the emphases of your Turing Award lecture
was on the way in which we educate, people about programming. It strikes me
that American education generally shortchanges students on the paradigm level
when dealing with technical and quantitative matters. Most people don't even
know how to check results for plausibility.
Floyd: Right. They come out of school never having learned to subject their
answers to plausibility checks. A good many schools seem to adopt an approach
to mathematical problem solving that shows the student how to turn himself
into a machine. And machines don't worry about the plausibility of their
results.
Swaine: I suppose that implementing plausibility checks in a programming
language would be something of a challenge.
Floyd: Well, you could have a language in which every datum had dimension, and
the type checking included dimension checking, so that you couldn't add feet
to square feet and if you added feet to yards an automatic conversion took
place. It would be fairly straightforward to incorporate that in a programming
language if you wanted to do it.
Or you could write a programming language for banks. Banks neither create nor
destroy money, and their programs should embody that fact. The programmer
should not be able to make money disappear, because if there's money
disappearing in the program, the chances are there's real money disappearing.
Swaine: That sounds useful. But to get back to education, where do paradigms
fit into the computer science curriculum?

Floyd: Paradigms are not something you could have a separate course about.
They fit in by getting professors to talk explicitly about the paradigms that
they use in the particular subject discipline in which they're lecturing.
Swaine: It's appealing to believe that we could educate and excite new
programmers about programming through the concepts and paradigms of
programming as effectively as we now do through that sense of power or mastery
or relief or whatever it is that they get when they finally make the machine
do what they want. I assume you're doing that in your courses. How do you go
about it?
Floyd: Right now I'm teaching a course in searching and sorting out of Knuth
Volume 3, which is a huge compendium of great algorithms in that area and
analyses of their performance, but which to my mind lacks a unifying
viewpoint. What I use as the unifying viewpoint is the idea that every problem
in that domain has a certain inherent difficulty that I call its entropy; the
formula from information theory and thermodynamics that gives you that. So I
use an information-and-entropy paradigm to get an overall viewpoint.
Swaine: How do you do that?
Floyd: In these sorting and searching programs, you are extracting information
from data sets. The program makes various queries of the data, and each query
gathers a certain amount of information, and you can derive a formula for
that. Then you can characterize the algorithms by the rate at which they
gather relevant information, and divide that into the entropy of the original
problem to get the time that it's going to take.
A well known case is that you're sorting a data set and the incoming order of
the data is random. Then each of the possible reorderings is equally likely
and has probability 1/n! and the entropy of that problem turns out to be log
of n! and, applying Sterling's approximation you end up with n log n. This
tells you that any method that you apply that only extracts one bit of
information per step will necessarily need time proportional to n log n to
determine the right permutation to reorder the data.
They go up in sophistication from there, but one can view most sorting and
searching problems, and some other problems like problems of large-scale
movement of information in memory, from an information-and-entropy viewpoint.
And you'll end up not only having some insight into whether an algorithm is
efficient as a whole, but you'll also have a microscope with which you can
look at each phase of an algorithm and ask, What's its information-theoretic
efficiency? Is there room in there for potential improvement?
Swaine: Are there books on paradigms or problem solving that you admire?
Floyd: In one of your columns you cite George Polya's How to Solve It, but to
my mind the great one is his two-volume Patterns of Plausible Inference. In
it, he deals with several tough problems from the history of mathematics, such
as the isoperimetric inequality. When you read Polya you not only learn about
paradigms, you learn something about the subject matter.
Educating programmers is only part of the business of educating people about
programming. It strikes me as plausible that the level of paradigms might be
the appropriate level for talking to intelligent non-programmers about what
programming is about. But that doesn't seem to happen at all.
Because of the way in which programmers acquire what they know, they often
don't know what they know; they can't articulate it in any general terms.
Swaine: I just read an article that addresses exactly that. It was the public
lecture that Tony Hoare gave on his induction to one of the professorships he
held. He was describing adapting the Quicksort algorithm to find a particular
quantile, and he illustrated it by hand using cards with numbers printed on
them. The whole thing was nonmathematical and directed at an audience that was
not at all familiar with the shoptalk of computing. It's in a book that just
came out: Lectures in Computer Science, Hoare et al, from Prentice-Hall, 1989.
Floyd: There have only been a few good essays of that sort directed at the
layperson. All too often, if they don't think that the computer is an
electronic brain, they think that it's a glorified adding machine.
Swaine: You certainly staked out the high ground in your Turing Award lecture.
What do you think the impact of the speech has been?
Floyd: The lecture had very little observable effect. It's been cited very
little in the literature; I've checked that through science citation indices,
and most of the citation has been peripheral: citation of specific examples or
problems, rather than concerned with the thesis. My overall feeling is that
the world has not advanced very much in that direction.
Swaine: Perhaps this interview will give the world a tiny nudge in that
direction. It occurs to me that, although we've used the word paradigm
liberally, we haven't really defined it except through example. Maybe that's
the best way to define it. But could you give one more example of a paradigm?
Floyd: In high school you probably learned to solve quadratic equations by
completing the square. The theorem says, You can complete the square. The
paradigm says, In a lot of situations you ought to complete the square.













































May, 1989
C PROGRAMMING


SI: A C-Like Script Interpreter




Al Stevens


This month we begin the construction of a script interpreter for the SMALLCOM
communications program. A script is a command language that allows a
communications program to automatically interact with an online service. The
language we'll use for this project is a subset of C, so the interpreter looks
a lot like a traditional C language interpreter. For this phase of the project
we will build an interpreter "engine," one that we can use to interpret C-like
statements for general purposes. This month we'll concentrate on the engine.
Later we'll integrate it into SMALLCOM as a script interpreter and try out
some scripts.
SMALLCOM manages the connection with remote computers running online services,
bulletin boards, other communications programs, or SMALLCOM itself. Most such
remote programs involve sign-on sequences where the callers identify
themselves, and the remote program grants or denies access. Once the caller
gets in, a command language gives access to electronic mail, conferences,
forums, files to download, and places to upload files. Virtually every such
service is unique with respect to its sign-on sequence and command language.
Most BBSs use one of the popular BBS programs (RBBS, FIDO, and so on), but
have their own file and forum architecture. The online services are usually
made of custom software and have their own proprietary command languages.
The command languages share one trait: They are designed to be understood by
people in an interactive dialogue. This approach makes the services available
to users with terminals and modems as well as to those with personal
computers. The PC users have an advantage. They can use their PCs to automate
routine parts of the dialogue. If you can execute the sign-on, move into the
forum of your choice, send your messages, capture messages from other members,
and all at the speed of the modem, then you have saved connect charges. A
shareware program named TAPCIS automates this procedure for CompuServe. Other
services are supported by scripts in shareware and commercial communications
programs. A general-purpose communications program that uses a script must
have a script processor.
Let's examine the requirements for a communications script. Once you have
connected with a service, the script processor watches the input stream and
waits for one or more character sequences. When one appears, the processor
makes a decision. Perhaps it sends a character sequence to the service. It
might tell the service to execute commands, such as to upload or download
files. When the script processor decides that the online tasks are done, it
disconnects. These actions are the same ones you would take if you signed on
and did the typing yourself. You could hard-code the decisions, character
sequences, and commands into a dedicated script processor such as the one in
TAPCIS, or you could develop a general-purpose script processor that uses an
external script file such as is done by ProComm. Our script processor will use
an external interpreted script language that resembles C. We call the script
language "S" and the script processor program the "S Interpreter," or simply
SI.
SI's operation will be mostly transparent to the rest of SMALLCOM. It will be
invoked when you dial a service that has a script named in the SMALLCOM phone
directory. SI will use the waitforstring function in SMALLCOM to watch for
character sequences, interceding when required. To emulate the user's
keystrokes, SI will stuff the characters it wants to type into the BIOS
keyboard buffer. These characters will then be returned to the program as if
they had been keyed by the user.


The S Language


S is a subset of C. If you know C you know S once you learn what the subset
includes. I chose this approach for three reasons. First, the syntax of C is
easily processed by an interpreter. Second, the decision-making properties of
a communications script processor are well supported by C's structured
programming features. Third, this is the "C Programming" column, and we all
know C to one degree or another. Why wrestle with a new language?
The S syntax differs from C in these ways: S has two data types, the int and
the char, and the two are usually synonymous. Either one can be declared as a
pointer or used as a pointer by reference. A variable works the same whether
you declare it as a pointer or not. If you reference it by name alone, you get
its value. If you reference it with an asterisk in front of its name, then
what the variable points to is what is returned. This is weak typing to the
extreme and follows in the waning spirit of the earliest C compilers. All
pointers are treated as character pointers. You can code a pointer-to-pointer
reference but it will not work. All variables are treated as integers except
in the case where the chartype is used in an array of char pointers like this:
 char*mylist[]={"password?","offline"};
This format is valid only in the global, external scope. Scope rules in S are
the same as those in C.
S has no other arrays, structures, or unions. It supports decimal and hex
constants and lets you code string literals as parameters to functions or
identifiers to ints or chars (which then become implied character pointers).
You build a script consisting of a main function and subordinate functions of
your own. S provides for a library of intrinsic functions that support the
environment the interpreter will support.
All global variables, except function parameters, are by default, static and
extern. All local variables are auto. Because these scopes are defaults, S
does not recognize the keywords. There are no shifts, bitwise logical
operators, continue, do, goto, typedef, switch, casts, prototypes, or function
declarations. The <expr>?<expr>:<expr> operation is not supported. The <op>=
operators (+=, -=, etc.) are not supported. Every function is assumed to
return an int unless it returns nothing. The unary + and - operators and the
ones-complement operator are not supported. There is no preprocessor.
S includes for, while, break, return, if, else, brace surrounded statement
blocks, recursion, the && and operators and the ! (not) and & (address of)
operators. The relational operators = =, !=, <, >, <=, and >= are supported.
Comments are the same as in C.


Intrinsic Functions


An implementation of SI must provide a library of intrinsic functions. These
are not functions such as you would have in an object library or in
interpreted source code, but ones that are built into the interpreter itself.
Intrinsic functions are what give the generic interpreter its intended
functionality. SI is built under a shell program that includes the intrinsic
functions, provides an array of structures that describes the intrinsic
functions, manages the source code file and errors, and executes the three
phases of the interpreter.


The S Interpreter


The SI interpreter code is built to be reusable. We will separate those parts
of SI that are script specific into a shell code module so that the
interpreter can be used for other purposes. For example, we might add a macro
processor to the editor. By isolating the interpreter from its original
purpose, we leave the door open for its use in other areas.
Listing One is interp.h, a header file that an application shell will use to
implement the interpreter. The first several items are configuration global
macros that you set to the values that define limits for the interpreter. The
first such global, TOK-BUFSIZE, defines the number of characters in the token
buffer. Each S keyword and operator is translated into a token. Each
identifier and string is kept in the token buffer. Each newline character in
the source stream is recorded in the buffer with a newline token and a three
character line number. If you get the error that says the token buffer has
overflowed, you will need to increase the TOKBUFSIZE global or shorten the
source file. The MAXSYM-BOLS global controls how many symbols can be in the
symbol table at a given time. This maximum is equal to the total number of
functions and external globals with any local automatics that are in scope. As
locals go out of scope, they are removed from the symbol table and their
entries are reused. MAXSTRINGS is the maximum number of strings that can
appear in external character pointer arrays. MAXPARAMS is the maximum number
of parameters a function can have.
The error codes are defined as an enumerated data type. SI is managed by a
shell program, an example of which is described later. The shell provides all
error processing and must react to these codes. The shell also provides the
table of intrinsic functions. The format for entries in that table is
described by the INTRINSIC structure.
Interp.h defines the format of the symbol table, which is a structure with
four elements. The symbol table has an entry for each function and variable in
the source file. The table entry includes a pointer to the symbol's name. If
the symbol is a function, the entry has a pointer to the location in the token
buffer of the start of the function's tokens. If the symbol is an array of
character pointers, the table has a pointer to the array. If the symbol is an
integer or pointer, the table has an entry that contains the symbol's value.
Pointers in S are always equivalent to integers. Finally, interp.h contains
function prototypes and external references to the critical data items
produced by the lexical scan and linker. These references will allow a shell
to record the pseudocompiled tokens for repeated interpreting later.
Listing Two is interp.c, the core of the interpreter. You would link this
program with a shell that manages the source file and user interface and
provides the intrinsic functions that give SI its purpose. These functions are
described later.
Interpreting the S language involves these three steps. First, the source file
is read and the statements are translated into tokens in the token buffer. In
traditional language processing this step is called the lexical scan. The
load-source function in interp.c starts the lexical scan. If the lexical
scanner finds a character sequence in the source file that it cannot
recognize, it declares an error and quits. During the lexical scan, source
file line number tokens are inserted into the token buffer. These line number
tokens identify the source file lines when errors occur. If anyone ever writes
a debugger for the S language, these line numbers might be useful.
When loadsource has successfully loaded the source file and translated it into
lexical tokens, loadsource calls the linker function. The linker scans the
token buffer and builds the global symbol table, which contains the names and
locations of functions and the names and values of variables. When the linker
finds errors in the S grammar the scan terminates. Following this step the
three data structures, the token buffer,string space, and symbol table could
be written to a file for later interpretation. SI, as we build it here, will
not take that intermediate step but will move directly into the interpreter.
We do, however, provide for that feature in case we should want it later. An
editor macro interpreter would probably use the compile-now-interpret-later
strategy to speed up the process.
After the linker is finished, loadsource returns to its caller, the
applications shell program that must start the interpreter by calling the
interpreter macro defined in interp.h. This macro sets up a pseudotoken buffer
with a tokenized call to the main function of the S language source program.
The macro calls function in the interpreter and gets the main function
underway. This procedure interprets the tokens, executes the language, and
parses the S grammar for errors. Obviously, scripts should be tested before
they are used on an expensive long-distance hookup.
Interpreting begins in function to execute the S main. All subsequent S
function calls come through here, so the process is recursive. First, the
interpreter looks for a parameter list in the function call. Each parameter is
evaluated by the expression function, and its value is stored in an array of
incoming parameters. Then the interpreter searches the table of intrinsic
functions to see if the function is one of them. If so, the interpreter calls
the intrinsic function through its pointer in the INTRINSIC structure array in
the shell. If not, the interpreter searches the global symbol table for the
called function and points the token pointer to the beginning of that
function. The parameters named in the function declaration go into the local
symbol table and the parameter values from the caller go into the value
elements of the new local symbols. The interpreter now begins interpreting the
function's procedure by calling the compound_statement function.
The compound_statement function processes brace surrounded statement blocks.
It adds variables within the block to the local symbol table and then executes
each statement in the block by calling the statements function. The statements
function calls compound_statement recursively if it finds another left brace
in the token stream. Otherwise it calls the statement function.
The statement function processes if, while, for, return, break, and simple
expression statements. The first three operators involve expression evaluation
and executing or skipping blocks of tokenized statements. The return and break
set flags to be sensed by subsequent statements. Simple expressions are
evaluated by the expression function.
SI evaluates expressions with a recursive descent parsing algorithm. The
precedence of operators is controlled by the position of the operators'
processing functions in the recursive descent function chain. You can trace
the progress of this chain by beginning with the expression function. This
algorithm expects the program token pointer variable, tptr, to point to the
first token of an expression. When the expression function returns the
evaluated result, the pointer will point to the first token past the
expression. First the expression function calls the and function to get its
result. Then, as long as there are logical or tokens in the expression, the
result is ORed with subsequent results from the and function. Looking up at
the and function, you can see flat it has a similar relationship with the eq
function above it. All these operator functions trace up the chain of
precedence to the primary function, which evaluates a primary element of an
expression. A primary element is a constant, pointer, or variable. If,
however, the primary function finds a left parenthesis as the next token, the
primary function starts the evaluation all over again by first calling
expression and then requiring a right parenthesis. The primary function
handles the NOT operator, the pointer operator, the address-of operator, and
auto-increment and decrement operators. Each function in the precedence chain
returns its result to the function below it until control is back at the
expression function. If you wanted to change the precedence of operators, you
would change the sequence of these functions in the chain. When two or more
operators have the same precedence, as do multiply and divide, they are
processed together in the same function and have left-to-right precedence
because the token scan is in that direction.


The SI Shell


Listing Three is si.c, a throw-away demonstration shell that lets you test SI.
It illustrates how an application would be integrated with the interpreter and
has three intrinsic functions: printf, getchar, and putchar. An SI shell
program has four responsibilities. First, the shell must provide the intrinsic
functions and the array of INTRINSIC structures that describes them. Second,
the shell must manage the S source code file by providing the functions named
getsource and ungetsource, which the interpreter calls to get and push back
characters from and to the source file. This way the interpreter does not care
whether the code comes from a file or an edit buffer. Third, the shell must
execute the interpreter by calling get-source and interpreter in that order.
Finally, the shell must provide the sierrorfunction to manage the errors found
by the interpreter. This strategy hides the details of error processing from
the interpreter. The interpreter passes to sierroran error code, a string that
might amplify the error such as the word being parsed when the error occurred,
and the source code line number where the error was found.



Another C Interpreter


For a look at a different approach to a subset C interpreter, watch for the
DDJ annual C issue in August. Well-known C author Herbert Schildt has
contributed an article that includes such a program. While our S interpreter
is intended to be an engine for interpreting C-like script languages,
Schildt's is oriented to the execution of C source programs and is offered as
a study in interpreter design and implementation. You will see many
similarities in our approaches, perhaps because interpreter theory is
well-defined and understood. We selected different subsets of C, and some of
our terminology differs, but the principles are similar.


QuickC 2.0: Worth Another Look


After a year on the market, QuickC 1.0 is replaced by QuickC 2.0. QuickC 1.0
was a compiler with plenty of bugs and some design features that hindered the
development environment. Its integrated environment limited you to the medium
memory model. The .OBJ files generated by the environment and the command line
compiler were incompatible. The .EXE files generated to be run by the
environment would not run from the command line. There were known, but
unacknowledged and unattended compiler bugs from the beginning.
QuickC 2.0 is another story. Microsoft has eliminated the dubious features and
seems to have cleared up the bugs. QuickC 2.0 is faster than Microsoft C 5.1
but generates bigger programs. On a 20-MHz 386, QuickC built a 47,481 byte
TWRP.EXE file in 1 minute, 14 seconds. MSC built a 35,507 byte file in 2
minutes, 59 seconds.
The programs in our "C Programming" column project now compile correctly with
QuickC 2.0. You will recall from January that I had a problem with one module.
The problem is gone with 2.0.
Microsoft changed the makefile syntax for their new NMAKE program, which works
more like the traditional make programs that programmers are accustomed to. If
you have the older MAKE program, you can use the make-files I published by
changing the cl command to the qcl command to use the QuickC 2.0 command line
compiler. With minor modifications, the makefiles can be adapted to NMAKE.
QuickC includes a book called C For Yourself that describes the C language at
the introductory level -- at least as well as most books you will find at the
store. The weakness in this book is its treatment (or neglect) of the run-time
library. Microsoft expects you to use the Microsoft Advisor, their on-line
help utility, for complete library reference information. This assumes that
you use the QuickC editor, which many of us will not. QuickC has no equivalent
to Turbo C's THELP program, which pops up the help data base from inside your
own editor. You do get the HELPMAKE utility program that lets you modify and
add to the help texts. Third-party library developers can provide help texts
that users can integrate into their development environments. Microsoft does
not, however, supply the format of the compressed help text database, which a
TSR developer would need to write a THELP clone for QuickC.


ANSI Answers


In January I took Borland to task for conforming to what they said was a new
ANSI rule that allowed the back-slash octal escape sequence to exceed three
digits. The effect of the rule was that existing code would no longer compile
the same way, and there was no warning message to let you know.
No sooner had the issue hit the stands than I found myself reading a letter
from P.J. Plaugher, a member of the committee, and talking to Tom Plum, the
vice chairman. Both members assured me in certain terms and with authority
that the so-called rule was not the rule at all and that tradition prevails
with respect to the octal escape sequence. Discussions with Borland reveal
that they genuinely believed they were tracking the evolving ANSI standard
with accuracy. The misunderstanding resulted from Borland's interpretation of
the committee vote on the matter. They decided to conform to the rule and we
had our original discussions (and I wrote my criticism) before the December
draft was published. Now the draft is out, and Borland will fix the
discrepancy in a future release of Turbo C.
The consequences of the issue will vary from user to user. Those of you using
Turbo C 2.0 for new software are well-advised to avoid the octal escape
sequence until the problem is fixed. The new ANSI hex sequence works as
specified and will do the job. Those of you using Turbo C 2.0 to compile
existing programs are counseled to watch for the offending octal sequences.
The upshot of this episode is that I am now the proud owner of a copy of the
"Draft Proposed American National Standard for Information Systems --
Programming Language C" document dated December 7, 1988 and a card-carrying
member of the X3J11 committee. Just in time -- their work is done.


Availability


All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063, or call 800-356-2002 (from inside
Calif.) or 800-533-4372 (from outside Calif.). Please specify the issue number
and format (MS-DOS, Macintosh, Kaypro).


_C Programming Column_
by Al Stevens


[LISTING ONE]


/* ---------------- interp.h -------------------- */
#define TOKBUFSIZE 4096 /* token buffer size */
#define MAXSYMBOLS 100 /* maximum symbols in table */
#define MAXSTRINGS 50 /* maximum strings in arrays */
#define MAXPARAMS 10 /* maximum parameters in calls */
/* ----------- error codes ----------------- */
enum errs {EARLYEOF,UNRECOGNIZED,DUPL_DECLARE,TABLEOVERFLOW,
 OMERR,UNDECLARED,SYNTAX,BRACERR,PARENERR,MISSING,
 NOTFUNC,BREAKERR,OUTOFPLACE,TOOMANYSTRINGS,BUFFULL,
 DIVIDEERR};
/* ------- intrinsic function table -------- */
typedef struct {
 char *fname;
 int (*fn)(int *);
} INTRINSIC;
/* -------- symbol table ------------ */
typedef struct {
 char *symbol; /* points to symbol name */
 char *location; /* points to function code (NULL if int) */

 char **tblloc; /* points to table array (NULL if func) */
 int ival; /* value of integer */
} SYMBOL;
/* ------- function prototypes -------- */
void loadsource(void);
int function(char *, SYMBOL *);
#define interpret() function("main\0();", symtop);
/* ------ functions provided by the shell -------- */
int getsource(void);
void ungetsource(int);
void sierror(enum errs, char *, int);
/* -------- the compiled (interpretable) S source -------- */
extern SYMBOL globals[];
extern char *tokenbf;
extern char *strings[];
extern SYMBOL *symtop;






[LISTING TWO]

/* ----------- interp.c ----------- */
#include <stdio.h>
#include <conio.h>
#include <string.h>
#include <stdlib.h>
#include <ctype.h>
#include <process.h>
#include "interp.h"
#define TRUE 1
#define FALSE 0
/* --------- the compiled (interpretable) S source ---------- */
SYMBOL globals[MAXSYMBOLS]; /* function/variable symbol table */
char *tokenbf = NULL; /* compiled token buffer */
char *strings[MAXSTRINGS]; /* char *[] string arrays */
SYMBOL *symtop; /* top of symbol table */
/* ---------- the external intrinsic functions ----------- */
extern INTRINSIC *infs; /* initialized by the SI shell */
/* ------ function macros ------------ */
#define bypass() tptr+=strlen(tptr)+1
#define isletter(c) (isalpha(c)isdigit(c)c =='_')
#define iswhite(c) (c==' 'c=='\t')
/* ---- function prototypes ----- */
static void linker(void);
static int gettoken(void);
static int getok(void);
static int iskeyword(void);
static int isident(void);
static int istoken(void);
static int getword(void);
static int getcx(void);
static void compound_statement(SYMBOL *);
static void statement(SYMBOL *);
static void statements(SYMBOL *);
static void skip_statements(SYMBOL *);
static void addsymbol(SYMBOL *, char *, char *, char **);

static SYMBOL *findsymbol(SYMBOL *, char *, SYMBOL *);
static SYMBOL *ifsymbol(SYMBOL *, char *, SYMBOL *);
static void freesymbols(SYMBOL *);
static void error(enum errs, char *);
static int iftoken(int);
static void skippair(int, int);
static void needtoken(int);
static int iftoken(int);
static int nexttoken(void);
static int expression(SYMBOL *);
static int escseq(void);
/* --------------- tokens ------------------ */
#define AUTOINC 'P'
#define AUTODEC 'D'
#define EQUALTO 'E'
#define NOTEQUAL 'N'
#define GE 'G'
#define LE 'L'
#define IF 'f'
#define ELSE 'e'
#define WHILE 'w'
#define FOR 'F'
#define CHAR 'c'
#define INT 'i'
#define STRING 's'
#define COMMENT1 '/'
#define COMMENT2 '*'
#define POINTER '*'
#define PLUS '+'
#define MINUS '-'
#define MULTIPLY '*'
#define DIVIDE '/'
#define EQUAL '='
#define LESS '<'
#define GREATER '>'
#define NOT '!'
#define LPAREN '('
#define RPAREN ')'
#define LBRACE '{'
#define RBRACE '}'
#define LBRACKET '['
#define RBRACKET ']'
#define COMMA ','
#define AND '&'
#define ADDRESS '@'
#define OR ''
#define QUOTES '"'
#define QUOTE '\''
#define UNDERSCORE '_'
#define SEMICOLON ';'
#define IDENT 'I'
#define CONSTANT 'C'
#define LINENO 127
#define RETURN 'r'
#define BREAK 'b'
/* -------- table of key words and their tokens --------- */
static struct keywords {
 char *kw;
 int kwtoken;

} kwds[] = {
 "\n", LINENO,
 "for", FOR,
 "while", WHILE,
 "if", IF,
 "else", ELSE,
 "int", INT,
 "char", CHAR,
 "return",RETURN,
 "break", BREAK,
 NULL, 0
};
/* ------------ table of direct translate tokens -------- */
static int tokens[] = {
 COMMA,LBRACE,RBRACE,LPAREN,RPAREN,EQUAL,NOT,POINTER,
 LESS,GREATER,AND,OR,QUOTES,SEMICOLON,LBRACKET,RBRACKET,
 MULTIPLY,DIVIDE,PLUS,MINUS,EOF,LINENO,0
};
/* --------------------- local data ----------------------- */
static char word[81]; /* word space for source parsing */
static int linenumber; /* current source file line number */
static int frtn; /* return value from a function */
static char *tptr; /* running token pointer */
static int stptr; /* running string allocation offset */
static int breaking, returning, skipping;
static SYMBOL *endglobals;
/* ----------- lexical scan and call linker ------------ */
void loadsource(void)
{
 int tok = 0;
 if (tokenbf == NULL)
 if ((tokenbf = malloc(TOKBUFSIZE+81)) == NULL)
 error(OMERR, "");
 symtop = symtop ? symtop : globals;
 freesymbols(globals);
 memset(tokenbf, '\0', TOKBUFSIZE+81);
 linenumber = 1;
 tptr = tokenbf;
 while (tok != EOF) {
 if (tptr >= tokenbf + TOKBUFSIZE)
 error(BUFFULL, "");
 *tptr++ = tok = gettoken();
 switch (tok) {
 case LINENO:
 sprintf(tptr, "%03d", linenumber);
 tptr += 3;
 break;
 case CONSTANT:
 case IDENT:
 case STRING:
 strcpy(tptr, word);
 bypass();
 break;
 default:
 break;
 }
 }
 linker(); /* link the external variables and functions */
}

/* -- convert a script program to tokens for interpreter,
 return the next token -------- */
static int gettoken(void)
{
 int tok = getword();
 if (tok == 0)
 if ((tok = iskeyword()) == 0)
 if ((tok = istoken()) == 0)
 tok = isident();
 if (tok == 0)
 error(UNRECOGNIZED, word);
 return tok;
}
/* ----- test to see if current word is a token ----- */
static int istoken(void)
{
 int *t = tokens, t2;
 while (*t && word[1] == '\0')
 if (*word == *t++) {
 switch (*word) {
 case EOF:
 break;
 case AND:
 if ((t2 = getcx()) != AND) {
 *word = ADDRESS;
 ungetsource(t2);
 }
 break;
 case OR:
 if (getcx() != OR)
 error(MISSING, word);
 break;
 case PLUS:
 case MINUS:
 if ((t2 = getcx()) == *word)
 *word = *word==PLUS ? AUTOINC : AUTODEC;
 else
 ungetsource(t2);
 break;
 default:
 if ((t2 = getcx()) == EQUAL) {
 switch (*word) {
 case EQUAL: return EQUALTO;
 case NOT: return NOTEQUAL;
 case LESS: return LE;
 case GREATER: return GE;
 default: break;
 }
 }
 ungetsource(t2);
 break;
 }
 return *word;
 }
 return 0;
}
/* -------- test word for a keyword --------- */
static int iskeyword()
{

 struct keywords *k = kwds;
 while (k->kw)
 if (strcmp(k->kw, word) == 0)
 return k->kwtoken;
 else
 k++;
 return 0;
}
/* ------ test for an ident -------- */
static int isident()
{
 char *wd = word;
 int n = 0;
 if (isalpha(*wd) *wd == UNDERSCORE)
 return IDENT;
 if (strlen(wd) <= 6) {
 if (strncmp(wd, "0x", 2) == 0) {
 wd += 2; /* 0x.... hex constant */
 while (*wd) {
 n = (n*16)+(isdigit(*wd) ? *wd-'0' :
 tolower(*wd) - 'a' + 10);
 wd++;
 }
 sprintf(word,"%d", n); /* converted hex constant */
 }
 else /* test for decimal constant */
 while (*wd)
 if (!isdigit(*wd++))
 return 0;
 return CONSTANT;
 }
 return 0;
}
/* -------- get the next word from the input stream ------- */
static int getword(void)
{
 char *wd = word;
 int c = ' ', tok = 0;
 while (iswhite(c)) /* bypass white space */
 c = getok();
 if (c == QUOTE) {
 tok = CONSTANT; /* quoted constant ('x') */
 if ((c = getcx()) == '\\') /* escape sequence (\n) */
 c = escseq();
 sprintf(word, "%d", c); /* build the constant value */
 wd += strlen(word);
 if (getcx() != QUOTE) /* needs the other quote */
 error(MISSING,"'");
 }
 else if (c == QUOTES) {
 tok = STRING; /* quoted string "abc" */
 while ((c = getcx()) != QUOTES)
 *wd++ = c == '\\' ? escseq() : c;
 }
 else {
 *wd++ = c; /* 1st char of word */
 while (isletter(c)) { /* build an ident */
 c = getok();
 if (isletter(c))

 *wd++ = c;
 else
 ungetsource(c);
 }
 }
 *wd = '\0'; /* null terminate the word or token */
 return tok;
}
/* ---- escape sequence in literal constant or string ---- */
static int escseq()
{
 int c = getcx();
 return (c == 'n' ? '\n' :
 c == 't' ? '\t' :
 c == 'f' ? '\f' :
 c == 'a' ? '\a' :
 c == 'b' ? '\b' :
 c == 'r' ? '\r' :
 c == '0' ? '\0' : c);
}
/* ------- get a character from the input stream -------- */
static int getok(void)
{
 int c, c1;
 while ((c = getsource()) == COMMENT1) {
 if ((c1 = getcx()) != COMMENT2) { /* comment */
 ungetsource(c1);
 break;
 }
 while (TRUE) { /* found comment begin token pair */
 while ((c1 = getcx()) != COMMENT2)
 ;
 if ((c1 = getcx()) == COMMENT1)
 break; /* found comment end token pair */
 }
 }
 if (c == '\n') /* count source line numbers */
 linenumber++;
 return c;
}
/* ------- read a character from input, error if EOF ------ */
static int getcx(void)
{
 int c;
 if ((c = getsource()) == EOF)
 error(EARLYEOF, "");
 return c;
}
/* --------- build the global symbol table --------- */
static void linker(void)
{
 int tok = 0;
 char *svtptr;
 INTRINSIC *ff = infs;
 tptr = tokenbf;
 /* --- add intrinsic functions to the symbol table --- */
 while (ff->fname) {
 addsymbol(globals,ff->fname,ff->fname,NULL);
 ff++;

 }
 while (tok != EOF) {
 switch (tok = nexttoken()) {
 case CHAR:
 svtptr = tptr;
 if (iftoken(POINTER)) {
 needtoken(IDENT);
 bypass();
 if (iftoken(LBRACKET)) {
 tptr = svtptr;
 nexttoken();
 nexttoken();
 addsymbol(globals,tptr,NULL,
 strings+stptr);
 bypass();
 needtoken(LBRACKET);
 needtoken(RBRACKET);
 needtoken(EQUAL);
 needtoken(LBRACE);
 while (TRUE) {
 if (!iftoken(STRING))
 break;
 if (stptr == MAXSTRINGS)
 error(TOOMANYSTRINGS, "");
 strings[stptr++] = tptr;
 bypass();
 if (!iftoken(COMMA))
 break;
 }
 strings[stptr++] = NULL;
 needtoken(RBRACE);
 needtoken(SEMICOLON);
 break;
 }
 }
 tptr = svtptr;
 case INT:
 while (TRUE) {
 if (iftoken(POINTER))
 ;
 needtoken(IDENT);
 addsymbol(globals,tptr,NULL,NULL);
 bypass();
 if (iftoken(EQUAL))
 (symtop-1)->ival=expression(globals);
 if (!iftoken(COMMA))
 break;
 }
 needtoken(SEMICOLON);
 break;
 case IDENT:
 addsymbol(globals, tptr, tptr, NULL);
 bypass();
 skippair(LPAREN, RPAREN);
 skippair(LBRACE, RBRACE);
 break;
 case EOF:
 break;
 default:

 error(OUTOFPLACE, (char *) &tok);
 }
 }
 endglobals = symtop;
}
/* --------- a function is called ---------- */
int function(char *fnc, SYMBOL *sp)
{
 int params[MAXPARAMS+1], p, i;
 INTRINSIC *ff = infs;
 char *savetptr = tptr;
 frtn = 0;
 tptr = fnc;
 bypass();
 needtoken(LPAREN);
 for (p = 0; p < MAXPARAMS; ) { /* scan for parameters */
 if (iftoken(RPAREN))
 break;
 params[p++]=expression(sp); /* build params */
 if (!iftoken(COMMA)) { /* into parameter array */
 needtoken(RPAREN);
 break;
 }
 }
 params[p] = 0;
 while (ff->fname) { /* search the intrinsic table */
 if (strcmp(fnc,ff->fname) == 0) {
 frtn = (*ff->fn)(params); /* call intrinsic func */
 tptr = savetptr;
 return frtn;
 }
 ff++;
 }
 if ((tptr=findsymbol(globals,fnc,endglobals)->location)
 == NULL)
 error(NOTFUNC,fnc); /* function not declared */
 bypass();
 needtoken(LPAREN);
 sp = symtop;
 for (i = 0; i < p; i++) { /* params into local sym tbl */
 needtoken(IDENT);
 addsymbol(sp,tptr,NULL,NULL);
 (symtop-1)->ival = params[i];
 bypass();
 if (i < p-1)
 needtoken(COMMA);
 }
 needtoken(RPAREN);
 compound_statement(sp); /* execute the function */
 freesymbols(sp); /* release the local symbols */
 tptr = savetptr;
 breaking = returning = FALSE;
 return frtn; /* the function's return value */
}
/* ------- execute one statement or a {} block -------- */
static void statements(SYMBOL *sp)
{
 if (iftoken(LBRACE)) {
 --tptr;

 compound_statement(sp);
 }
 else
 statement(sp);
}
/* -------- execute a {} statement block ----------- */
static void compound_statement(SYMBOL *sp)
{
 char *svtptr = tptr;
 SYMBOL *spp = symtop;
 needtoken(LBRACE);
 while (iftoken(CHAR) iftoken(INT)) {
 while (TRUE) { /* local variables in block */
 if (iftoken(POINTER))
 ; /* bypass pointer token(s) */
 needtoken(IDENT);
 addsymbol(spp,tptr,NULL,NULL);
 bypass();
 if (iftoken(EQUAL)) /* handle assignments */
 (symtop-1)->ival=expression(sp);
 if (!iftoken(COMMA))
 break;
 }
 needtoken(SEMICOLON);
 }
 while (!iftoken(RBRACE) && !breaking && !returning)
 statements(sp);
 tptr = svtptr; /* point to the opening { brace */
 freesymbols(spp); /* free the local symbols */
 skippair(LBRACE, RBRACE); /* skip to end of block */
}
/* --------- execute a single statement ---------- */
static void statement(SYMBOL *sp)
{
 char *svtptr, *fortest, *forloop, *forblock;
 int rtn, tok = nexttoken();
 switch (tok) {
 case IF:
 needtoken(LPAREN);
 rtn = expression(sp); /* condition being tested */
 needtoken(RPAREN);
 if (rtn)
 statements(sp); /* condition is true */
 else
 skip_statements(sp); /* condition is false */
 while (iftoken(ELSE))
 if (rtn) /* do the reverse for else */
 skip_statements(sp);
 else
 statements(sp);
 break;
 case WHILE:
 rtn = TRUE;
 breaking = returning = FALSE;
 svtptr = tptr;
 while (rtn && !breaking && !returning) {
 tptr = svtptr;
 needtoken(LPAREN);
 rtn = expression(sp); /* the condition tested */

 needtoken(RPAREN);
 if (rtn)
 statements(sp); /* true */
 else
 skip_statements(sp); /* false */
 }
 breaking = returning = FALSE;
 break;
 case FOR:
 svtptr = tptr; /* svptr -> 1st ( after for */
 needtoken(LPAREN);
 if (!iftoken(SEMICOLON)) {
 expression(sp); /* initial expression */
 needtoken(SEMICOLON);
 }
 fortest = tptr; /* fortest -> terminating test */
 tptr = svtptr;
 skippair(LPAREN,RPAREN);
 forblock = tptr; /* forblock -> block to run */
 tptr = fortest;
 breaking = returning = FALSE;
 while (TRUE) {
 if (!iftoken(SEMICOLON)) {
 if (!expression(sp)) /* terminating test */
 break;
 needtoken(SEMICOLON);
 }
 forloop = tptr;
 tptr = forblock;
 statements(sp); /* the loop statement(s) */
 if (breaking returning)
 break;
 tptr = forloop;
 if (!iftoken(RPAREN)) {
 expression(sp); /* the end of loop expr */
 needtoken(RPAREN);
 }
 tptr = fortest;
 }
 tptr = forblock;
 skip_statements(sp); /* skip past the block */
 breaking = returning = FALSE;
 break;
 case RETURN:
 if (!iftoken(SEMICOLON)) {
 frtn = expression(sp);
 needtoken(SEMICOLON);
 }
 returning = !skipping;
 break;
 case BREAK:
 needtoken(SEMICOLON);
 breaking = !skipping;
 break;
 case IDENT:
 --tptr;
 expression(sp);
 needtoken(SEMICOLON);
 break;

 default:
 error(OUTOFPLACE, (char *) &tok);
 }
}
/* -------- bypass statement(s) ------------ */
static void skip_statements(SYMBOL *sp)
{
 skipping++; /* semaphore that suppresses assignments, */
 statements(sp); /* breaks,returns,++,--,function calls */
 --skipping; /* turn off semaphore */
}
/* -------- recursive descent expression analyzer -------- */
static int primary(SYMBOL *sp)
{
 int tok, rtn = 0;
 SYMBOL *spr;
 switch (tok = nexttoken()) {
 case LPAREN:
 rtn = expression(sp);
 needtoken(RPAREN);
 break;
 case NOT:
 rtn = !primary(sp);
 break;
 case CONSTANT:
 rtn = atoi(tptr);
 bypass();
 break;
 case POINTER:
 rtn = *(int *)primary(sp) & 255;
 break;
 case ADDRESS:
 case AUTOINC:
 case AUTODEC:
 needtoken(IDENT);
 case IDENT:
 if ((spr = ifsymbol(sp,tptr,symtop)) == NULL)
 spr = findsymbol(globals,tptr,endglobals);
 if (spr->location) {
 /* ---- this is a function call ---- */
 if (tok != IDENT)
 error(OUTOFPLACE, (char *) &tok);
 rtn = skipping ? 0 : function(tptr,sp);
 bypass();
 skippair(LPAREN,RPAREN);
 break;
 }
 bypass();
 if (spr->tblloc) {
 /* ---- this is a table ---- */
 rtn = (tok == ADDRESS ?
 (int) (&spr->tblloc) :
 (int) ( spr->tblloc) );
 break;
 }
 if (!skipping && tok == AUTOINC)
 ++(spr->ival);
 else if (!skipping && tok == AUTODEC)
 --(spr->ival);

 if (tok != ADDRESS && iftoken(EQUAL)) {
 rtn = expression(sp);
 spr->ival = skipping ? spr->ival : rtn;
 }
 rtn = tok == ADDRESS ? (int)&spr->ival : spr->ival;
 if (tok != ADDRESS)
 if (iftoken(AUTOINC) && !skipping)
 (spr->ival)++;
 else if (iftoken(AUTODEC) && !skipping)
 (spr->ival)--;
 break;
 case STRING:
 rtn = (int) tptr;
 bypass();
 break;
 default:
 error(OUTOFPLACE, (char *) &tok);
 }
 return rtn;
}
static int mult(SYMBOL *sp)
{
 int drtn, rtn = primary(sp);
 while (TRUE)
 if (iftoken(MULTIPLY))
 rtn = (rtn * primary(sp));
 else if (iftoken(DIVIDE)) {
 if ((drtn = primary(sp)) == 0)
 error(DIVIDEERR, "");
 rtn /= drtn;
 }
 else
 break;
 return rtn;
}
static int plus(SYMBOL *sp)
{
 int rtn = mult(sp);
 while (TRUE)
 if (iftoken(PLUS))
 rtn = (rtn + mult(sp));
 else if (iftoken(MINUS))
 rtn = (rtn - mult(sp));
 else
 break;
 return rtn;
}
static int le(SYMBOL *sp)
{
 int rtn = plus(sp);
 while (TRUE)
 if (iftoken(LE))
 rtn = (rtn <= plus(sp));
 else if (iftoken(GE))
 rtn = (rtn >= plus(sp));
 else if (iftoken(LESS))
 rtn = (rtn < plus(sp));
 else if (iftoken(GREATER))
 rtn = (rtn > plus(sp));

 else
 break;
 return rtn;
}
static int eq(SYMBOL *sp)
{
 int rtn = le(sp);
 while (TRUE)
 if (iftoken(EQUALTO))
 rtn = (rtn == le(sp));
 else if (iftoken(NOTEQUAL))
 rtn = (rtn != le(sp));
 else
 break;
 return rtn;
}
static int and(SYMBOL *sp)
{
 int rtn = eq(sp);
 while (iftoken(AND))
 rtn = (eq(sp) && rtn);
 return rtn;
}
static int expression(SYMBOL *sp)
{
 int rtn = and(sp);
 while (iftoken(OR))
 rtn = (and(sp) rtn);
 return rtn;
}
/* ----- skip the tokens between a matched pair ----- */
static void skippair(int ltok, int rtok)
{
 int pairct = 0, tok;
 if ((tok = nexttoken()) != ltok)
 error(ltok == LBRACE ? BRACERR : PARENERR, "");
 while (TRUE) {
 if (tok == ltok)
 pairct++;
 if (tok == rtok)
 if (--pairct == 0)
 break;
 if ((tok = nexttoken()) == EOF)
 error(ltok == LBRACE ? BRACERR : PARENERR, "");
 }
}
/* ----- a specified token is required next ----- */
static void needtoken(int tk)
{
 if (nexttoken() != tk)
 error(MISSING, (char *) &tk);
}
/* ----- test for a specifed token next in line ----- */
static int iftoken(int tk)
{
 if (nexttoken() == tk)
 return TRUE;
 --tptr;
 return FALSE;

}
/* ----- get the next token from the buffer ----- */
static int nexttoken(void)
{
 while (*tptr == LINENO)
 tptr += 4;
 return *tptr++;
}
/* ------ add a symbol to the symbol table ------------ */
static void
addsymbol(SYMBOL *s,char *sym,char *floc,char **tloc)
{
 if (ifsymbol(s,sym,symtop) != NULL)
 error(DUPL_DECLARE, sym);
 if (symtop == globals + MAXSYMBOLS)
 error(TABLEOVERFLOW, "");
 if ((symtop->symbol = malloc(strlen(sym) + 1)) == NULL)
 error(OMERR, "");
 strcpy(symtop->symbol, sym);
 symtop->location = floc;
 symtop->tblloc = tloc;
 symtop->ival = 0;
 symtop++;
}
/* --------- find a symbol on the symbol table --------- */
static SYMBOL *findsymbol(SYMBOL *s, char *sym, SYMBOL *ends)
{
 if ((s = ifsymbol(s, sym, ends)) == NULL)
 error(UNDECLARED, sym);
 return s;
}
/* -------- test for a symbol on the symbol table ------ */
static SYMBOL *ifsymbol(SYMBOL *s, char *sym, SYMBOL *sp)
{
 while (s < sp--)
 if (strcmp(sym, sp->symbol) == 0)
 return sp;
 return NULL;
}
/* ------- free the symbols from a symbol table ------- */
static void freesymbols(SYMBOL *s)
{
 while (s < symtop)
 free((--symtop)->symbol);
}
/* -------- post an error to the shell ------- */
static void error(enum errs erno, char *s)
{
 while (*tptr != LINENO && tptr > tokenbf)
 --tptr;
 sierror(erno, s, (*tptr == LINENO) ? atoi(tptr+1) : 1);
}






[LISTING THREE]


/* ---------- si.c -------------- */
#include <stdio.h>
#include <conio.h>
#include <stdlib.h>
#include "interp.h"
/* ----------- intrinsic interpreter functions ---------- */
static int iprntf(int *p) /* printf */
{
 printf((char*)p[0],p[1],p[2],p[3],p[4]);
 return 0;
}
static int igtch(int *p) /* getchar */
{
 return putch(getch());
}
static int iptch(int *c) /* putchar */
{
 return putchar(*c);
}
INTRINSIC ffs[] = { "printf", iprntf,
 "getchar", igtch,
 "putchar", iptch,
 NULL, NULL };
extern INTRINSIC *infs = ffs;
/* ---------- error messages ------------- */
char *erm[]={ "Unexpected end of file", "Unrecognized",
 "Duplicate ident", "Symbol table full",
 "Out of heap memory", "Undeclared ident",
 "Syntax Error", "Unmatched {}",
 "Unmatched ()", "Missing",
 "Not a function", "Misplaced break",
 "Out of place", "Too many strings",
 "Token buffer overflow", "Divide by zero" };
static FILE *fp;
void main(int argc, char *argv[])
{
 if (argc == 2)
 if ((fp = fopen(argv[1], "r")) != NULL) {
 loadsource();
 fclose(fp);
 interpret();
 }
}
void sierror(enum errs erno, char *s, int line)
{
 printf("\r\n%s %s on line %d\n",s,erm[erno],line);
 exit(1);
}
int getsource(void) { return getc(fp); }
void ungetsource(int c) { ungetc(c, fp); }











May, 1989
GRAPHICS PROGRAMMING


Fencing The Dog




Kent Porter


If you were awake, you noticed that viewports disappeared from the GRAFIX
library during March and April. Didn't matter much. We hadn't talked
specifically about viewports, so I decided to send them on vacation while
rethinking the whole subject. Now they're back, ready to go.
In a way, I'm sorry I brought them up at all in the first incarnation of the
GRAFIX library. As so often happens with any programming project, I had plans
for viewports that changed as the project progressed. Now we have to undo some
old damage before we can begin with the new.
Fortunately, that's not a Big Deal. All you have to do is delete a few lines
from GRAFIX.C. In general, all references to the VUPORT structure go. Here's
what to do:
About 20 lines down, remove the four lines beginning with the comment "/*
Viewport structure*/."
About a dozen lines below that, delete the two variables of type VUPORT.
At about line 85, strike the statement vuport = &def_vp;
At around old line 120, remove def_vp.height = 480;
Luckily, that's all there is to it: an annoyance rather than a hardship.
There's one more thing you have to do with GRAFIX.C. At the end of the
init_video( ) function, just before the return statement, add the following:
 if (mode = = VGA16 && vga) default_viewport (480);
Now save GRAFIX.C. Don't recompile it yet, though, because we have some other
fish to fry first.
In a strategic sense, what's the point of all this? As the project has
advanced, it's becoming rapidly apparent that we can't keep adding stuff to
GRAFIX.C. The source file would soon swell to enormous size if we did. This
would force us to search through zillions of functions to find the one we want
and to wait interminably during each recompile. That's the whole oomph behind
the concept of modular development: Keep related things together in their own
files.
So what we've done here is to remove the VUPORT references from GRAFIX.C,
where they don't belong, in preparation for constructing a new viewport
support file, where they do. The reference to default_viewport( ) merely calls
a viewport initialization function that lives among its relatives.
Now we can bring in the new. But first let's consider the question...


So What's a Viewport?


A viewport is to graphics what a window is to text: a subset of the display
that thinks it's an entire self-contained screen. After you define a viewport,
all operations are confined within its boundaries, and the coordinate system
is relative to the upper left corner.
For example, say you define a viewport with its upper left corner at X=320,
Y=175. If the device is an EGA, you have opened the viewport in the lower
right quadrant. This implies that, while the viewport is active, everything
above and to the left of the display area is inaccessible. All coordinate
references thus become relative to the viewport's origin. This leads to the
concept of coordinate remapping: A pixel written at {0, 0} is automatically
remapped to absolute location {320,175}. Don't worry right now about how it's
done. We're still at the conceptual level, with mechanics to follow later.
The second major concept of viewports is clipping. Say the viewport at
{320,175} has a width of 100 pixels and a height of 50. The right side of the
viewport is therefore at x=420 and the bottom is at y=225. These coordinates
form the boundaries of the clipping region. You can't write pixels outside of
them, just as you can't write pixels off-screen.
The idea is analogous to fencing the backyard. The dog can go anywhere it
wants within the yard, but that's all because the fence defines the limits of
its world. The dog can look through the fence and bark at the cat on the other
side, thus conceiving of a world beyond, but it can't get there. That's
clipping.
If a dog thought about such things, it might regard one corner of its world as
point {0, 0}, with the capability to move 100 paces along one axis and 50
along the other. The dog might decide to start at some arbitrary point and
walk 200 paces along a straight line. It could move for some distance, but
eventually its travel would be arrested -- that is, clipped -- upon
encountering the fence. The dog would thus have completed as much of the
actual journey as possible, with the rest existing only in its imagination.
Similarly, we can draw a line from a point within a viewport to a point
outside it, and the line stops when it hits the edge of the clipping region.
By extension, we can also draw a line between two points, both of which are
outside the clipping region, and we'll see only the portion that actually
falls within the viewport it crosses.
How? The obvious place to control remapping and clipping is at the
pixel-writing level. In DRAWPT.ASM (Listing One) the magic happens between
lines 24 and 43. This passage of code points ES:BX at the current viewport
structure, then compares the X and Y coordinates with the width and height,
respectively. If either is outside the viewport, the code jumps to the exit,
thus inhibiting a pixel write. When both coordinates are inside the viewport,
lines 39-43 remap the viewport coordinates to absolute screen coordinates
relative to the viewport's origin. HLINE.ASM (Listing Two) performs similar
clipping and remapping in lines 63-82.
These routines could be made more robust. For example, both assume that the
coordinates are in the positive domain, and so don't check for coordinates to
the left of or above the viewport origin (either of which would be negative
values). The intent here is to show how it works, so I've deliberately omitted
the extensive validity checks that rightfully characterize industrial-grade
software.
Note that I've added a short "hack" to each routine (lines 76 - 79 in
DRAWPT.ASM and lines 104 - 107 in HLINE.ASM). These enhancements program the
graphics chip to replace affected pixels. The 6845 can also perform bitwise
operations (AND, OR, XOR) on pixels. By setting bits 3 and 4 of the 6845's
Data Rotate/Function Select register to zeros, you tell the chip not to
perform any of these special color effects, but instead to change the affected
pixel to the new value without gamesmanship.
Make sure line 14 in each assembly-language routine reads as shown in the
listings, then reassemble them. You can implement the enhanced versions in
your copy of GRAFIX.LIB with the command
LIB grafix -+drawpt -+hline;
Now we're ready to take on viewports themselves.


Bring In The New!


The real work of coordinate remapping and clipping is done by the two
pixel-writing routines. A viewport descriptor is itself a rather simple
structure defining the origin coordinates, width, and height. However,
programs shouldn't have to concern themselves with viewport management,
instead dealing with strategic issues. Consequently the new VUPORT.C module
defines eight functions for operating on viewports, relieving applications of
the details and even hiding the descriptor structure itself.
The overall thrust of the VUPORT module is to treat viewports in a manner
analogous to files. Like a file, a given viewport has a handle (type VP_HAN)
assigned at its birth and remaining with it during its lifetime. The library
module provides functions to open and close viewports, to switch among
existing viewports, and to make inquiries.
Listing Three shows the additions to GRAFIX.H, the library header file, to
define the new viewport functions. After appending these entries to the end of
the header file, reccompile GRAFIX.C, then put it into the library with the
command
 LIB grafix -+grafix;
VUPORT.C in Listing Four provides the source for the new routines. Compile it,
then add it to the library with
 LIB grafix +vuport;
Now let's examine the VUPORT module to see what it does.
The VPNODE structure at the top of the file defines a viewport descriptor node
to be used in a dynamic list. Several of the functions manage this list, which
contains information about the viewports your program opens. In addition, the
module maintains the default viewport definition def_vp, which describes the
display as a whole, and the vuport variable, which always points to the
descriptor for the currently active viewport.
The default_viewport( ) function is called by init_video( ) from GRAFIX.C to
change the default viewport's height. The default structure is born with a
height of 350 pixels (EGA). The call from init_video( ) automatically adjusts
the height to 480 when the graphics mode is VGA 640 x 480. Though accessible
to your programs, there is little reason for them to call the
default_viewport( ) function.
The vp_open( ) routine creates a new viewport. The XY position of the upper
left corner is expressed in absolute coordinates and is not relative to the
currently active viewport. Upon completion of the routine, the newly opened
viewport is the active one and subsequent drawing occurs within it. Like the
familiar file open( ) function, vp_open( ) returns a handle identifying the
viewport. Handles are integers working upward from 1; the 0 handle refers to
the default full-screen viewport defined in def-vp.

vp-open( ) adds the new viewport to a doubly linked dynamic list, to which the
vplist variable acts as a head pointer. The list contains descriptor nodes for
all user-defined viewports that currently exist. A new viewport is appended at
the tail and assigned a handle numerically one greater than that of the old
tail. The routine makes the new viewport active by repointing the vuport
variable to it, and then returns the handle.
You can switch around among existing viewports by calling vp-use( ). The
argument is a handle identifying the desired viewport, or 0 for the
full-screen default. This routine searches the viewport list for a node whose
handle matches the non-zero argument, returning TRUE or FALSE to indicate the
outcome. When successful, vp_use( ) repoints the vuport variable to the
indicated viewport descriptor, thus activating it. Subsequent pixel operations
occur relative to the selected viewport.
A viewport ceases to exist when you pass its nonzero handle to vp-close( ).
The act of closing entails removing the descriptor node from the viewport
list. The routine does this by searching for the handle, then rearranging the
neighbors' pointers to bypass the closed node. This has the effect of pinching
the node out of the list. The memory space can then be freed. vp_close( ) also
checks the closed node against the global vplist and vuport pointers. If
you've closed the head node, its successor is promoted to the head of the
list, and if you've closed the active node, the default node becomes active.
Viewports are handy for confining drawing activity to a certain part of the
screen, and for dynamically placing visual objects according to conditions
encountered at runtime. Another use is analogous to text windows, in which a
region of the screen is set aside for some purpose such as displaying a graph.
In the latter case, programmers usually want to make the viewport visually
distinctive by outlining it. That's the purpose of the vp_outline( ) function.
vp_outline( ) draws a rectangle in the current foreground color. The border is
immediately outside the viewport itself, so that it doesn't impinge on the
drawing area or get clobbered by activity inside the viewport. The viewport to
be outlined doesn't have to be active at the time, but it must be open
inasmuch as the function requires a valid handle. After calculating the
border's location and dimensions, vp_outline( ) switches to full-screen mode
and draws the outline, then reinstates the former active viewport.
The library routines hide the structure of the viewport descriptor from
applications. Consequently some way must be available for programs to inquire
about important characteristics of the current viewport: its handle and
dimensions. The functions vp_active( ), vp_width( ), and vp_height( ) provide
those services. Using them, subprograms with no knowledge of the current
viewport can make intelligent decisions about, say, scaling output to fit
within the available drawing area. You'll see examples Real Soon Now.
Before we move on, don't forget to compile GRAFIX.C and VUPORT.C and then put
them into your GRAFIX.LIB.


Let's Put It To Work


A couple of application programs will prove the pudding we've cooked up here,
and probably give you some other ideas about how to use viewports.
Listing Five is VP.C, a simple program that draws some six-pointed stars in
viewports. The first star (magenta) is drawn in default full-screen mode. The
second (blue near the bottom of the screen) is inside a viewport, but you
can't tell that because there's no border. It appears to have been drawn with
different coordinates than the first, even though the positions of both are
the same in relation to their enclosing viewports. The third star near the
center of the screen is within a bordered viewport. The area is too small to
contain the star, so clipping occurs. The white star and blue border clearly
reveal that there is no conflict between the outline and objects within the
viewport, and also (lower right corner) that the portion of a line beginning
and ending outside the viewport shows up within the drawing area. The final
viewport is outlined in white, filled with green, and bears a red star. The
call to fill_rect( ) illustrates one application of inquiry functions.
The second program is LINEGRAF.C (Listing Six). This program illustrates how a
graphics subroutine can intelligently adjust its behavior according to the
dimensions of the active viewport. It plots the same series of data points in
three viewports of different sizes and shapes.
The smarts are in the graph( ) function. The first three expressions calculate
the factors required to scale the line graph. The number of data points is the
size of the data series array divided by the size of its type (assuming that
the array is always full). If there are n points, then the number of intervals
across the width of the plot area is n-1. Therefore the horizontal scaling
factor is the width divided by the number of intervals, which must be a
floating point value in order to avoid roundoff errors. This program knows
that all values in the data series are positive integers less than 100, so it
factors the vertical interval per unit on a scale of 100.
The graph itself is drawn by the loop, which pulls each line segment forward
from the previous point to the current point. Accordingly, before entering the
loop the program computes the location for element 0 as the previous point, so
that it has a valid place to draw from during the first iteration. Now let's
consider the sanity of the statement
 cury = vp_height( ) - (int)(data[p] * vint);
The second factor is fairly obvious: It finds the height of the point scaled
by the vertical factor, and because coordinates are integers, the cast
converts it appropriately. But why subtract the Y coordinate from the viewport
height? Because display coordinate systems are upside down. The rest of the
world thinks of Y as increasing upward, while on a display Y increases
downward. By subtracting the computed value from the viewport height we "flip"
the Y so that the graph comes out right side up.
So the old is out and the new is in, and that's how viewports work.


Availability


All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063; or call 800-356-2002 (from inside
Calif.) or 800-533-4372 (from outside Calif.). Please specify the issue number
and format (MS-DOS, Macintosh, Kaypro).


_Graphics Programming_
by Kent Porter




[LISTING ONE]

 1 ; DRAWPT.ASM: Writes pixel directly to 6845 Video Controller
 2 ; Microsoft MASM 5.1
 3 ; C prototype is
 4 ; void far draw_point (int x, int y);
 5 ; To be included in GRAFIX.LIB
 6 ; K. Porter, .MDUL/DDJ.MDNM/ Graphics Programming Column, February '89
 7
 8 .MODEL LARGE
 9 .CODE
 10 PUBLIC _draw_point
 11
 12 ; Externals in GRAFIX.LIB
 13 EXTRN _color1 : BYTE ; Pixel color reg value
 14 EXTRN _vuport : WORD ; far ptr to vuport structure
 15
 16 ; Arguments passed from C
 17 x EQU [bp+6] ; Arguments passed from C
 18 y EQU [bp+8]
 19
 20 _draw_point PROC FAR
 21 push bp ; Entry processing
 22 mov bp, sp
 23

 24 ; Point ES:[BX] to vuport structure
 25 mov ax, _vuport+2 ; get pointer segment
 26 mov es, ax
 27 mov bx, _vuport ; get offset
 28
 29 ; Clip if coordinates outside viewport
 30 mov cx, y ; get y
 31 cmp cx, es:[bx+6] ; is y within viewport?
 32 jl checkx ; ok if so
 33 jmp exit ; else quit
 34 checkx: mov ax, x ; get x
 35 cmp ax, es:[bx+4] ; is x within viewport?
 36 jl remap ; ok if so
 37 jmp exit ; else quit
 38
 39 ; Map pixel coordinates to current viewport
 40 remap: add ax, es:[bx] ; offset x by vuport.left
 41 mov x, ax ; save remapped X
 42 add cx, es:[bx+2] ; offset y by vuport.top
 43 mov y, cx ; save remapped Y
 44
 45 ; Point ES to video memory segment
 46 vmem: mov ax, 0A000h
 47 mov es, ax
 48
 49 ; Row offset = y * 80;
 50 mov bx, y ; Get y argument
 51 mov ax, 80
 52 mul bx ; Result in AX
 53 mov bx, ax ; Row offset in BX
 54
 55 ; Column offset = x SHR 3
 56 mov ax, x ; Get x
 57 mov cl, 3 ; Shift operand
 58 shr ax, cl ; Column offset
 59
 60 ; Complete address of pixel byte
 61 add bx, ax ; ES:BX = address
 62
 63 ; Build bit mask for pixel
 64 mov cx, x ; Get x again
 65 and cx, 7 ; Isolate low-order bits
 66 xor cl, 7 ; Number of bits to shift
 67 mov ah, 1 ; Start bit mask
 68 shl ah, cl ; Shift for pixel
 69 mov cl, ah ; Save it
 70
 71 ; Use write mode 2 (single-pixel update)
 72 mov dx, 03CEh ; 6845 command register
 73 mov al, 5 ; Specify mode register
 74 mov ah, 2 ; Read mode 0, write mode 2
 75 out dx, ax ; Send
 76 ; Following added May '89
 77 mov al, 3 ; Specify function select reg
 78 xor ah, ah ; Replace-pixel mode
 79 out dx, ax ; Send
 80
 81 ; Set 6845 bit mask register
 82 mov al, 8 ; Specify bit mask register

 83 mov ah, cl ; al = mask
 84 out dx, ax ; Send bit mask
 85
 86 ; Draw the pixel
 87 mov al, es:[bx] ; Load 6845 latch registers
 88 xor al, al ; Clear
 89 mov byte ptr es:[bx], al ; Zero the pixel for replace
 90 mov al, _color1 ; Get the pixel value
 91 mov es:[bx], al ; Write the pixel
 92
 93 ; Restore video controller to default state
 94 mov dx, 03CEh
 95 mov ax, 0005h ; write mode 0, read mode 0
 96 out dx, ax
 97 mov ax, 0FF08h ; default bit mask
 98 out dx, ax
 99 mov ax, 0003h ; default function select
 100 out dx, ax
 101 xor ax, ax ; zero Set/Reset
 102 out dx, ax
 103 mov ax, 0001h ; zero Enable Set/Reset
 104 out dx, ax
 105 mov dx, 03C4h ; 6845 address reg
 106 mov ax, 0F02h ; Data reg, enable all planes
 107 out dx, ax
 108
 109 exit:
 110 mov sp, bp
 111 pop bp
 112 retf
 113 _draw_point ENDP
 114 END






[LISTING TWO]

 1 ; HLINE.ASM: Fast horizontal line drawing routine
 2 ; Uses 6845 Write Mode 0 to update 8 pixels at a time on EGA/VGA
 3 ; C prototype is
 4 ; void far hline (int x, int y, int length_in_pixels);
 5 ; Writes in current color1 from GRAFIX.LIB
 6 ; Microsoft MASM 5.1
 7 ; K. Porter, .MDUL/DDJ.MDNM/ Graphics Programming Column, March 89
 8
 9 .MODEL LARGE
 10 .CODE
 11 PUBLIC _hline
 12 EXTRN _color1 : BYTE ; Current palette reg for pixel
 13 EXTRN _draw_point : PROC ; Pixel writing routine
 14 EXTRN _vuport : WORD ; far ptr to vuport structure
 15
 16 ; Declare arguments passed by caller
 17 x EQU [bp+6]
 18 y EQU [bp+8]
 19 len EQU [bp+10]

 20
 21 ; Declare auto variables
 22 last EQU [bp- 2] ; Last byte to write
 23 solbits EQU [bp- 4] ; Mask for start of line
 24 oddsol EQU [bp- 6] ; # odd bits at start of line
 25 eolbits EQU [bp- 8] ; Mask for end of line
 26 oddeol EQU [bp-10] ; # odd bits at end of line
 27 ; ----------------------------
 28
 29 _hline PROC FAR ; ENTRY POINT TO PROC
 30 push bp ; entry processing
 31 mov bp, sp
 32 sub sp, 10 ; make room for auto variables
 33 xor ax, ax ; initialize auto variables
 34 mov last, ax
 35 mov solbits, ax
 36 mov oddsol, ax
 37 mov eolbits, ax
 38 mov oddeol, ax
 39
 40 ; Do nothing if line length is zero
 41 mov bx, len ; get line length
 42 cmp bx, 0 ; length = 0?
 43 jnz chlen ; if not, go on
 44 jmp quit ; else nothing to draw
 45
 46 ; Call draw_point() with a loop if line length < 8
 47 chlen: cmp bx, 8
 48 jnb getvp ; go if len >= 8
 49 mov ax, y ; get args
 50 mov cx, x
 51 drpt: push bx ; push remaining length
 52 push ax ; push args to draw_point()
 53 push cx
 54 call _draw_point ; draw next pixel
 55 pop cx ; clear args from stack
 56 pop ax
 57 pop bx ; fetch remaining length
 58 inc cx ; next x
 59 dec bx ; count pixel drawn
 60 jnz drpt ; loop until thru
 61 jmp quit ; then exit
 62
 63 ; Point ES:[BX] to vuport structure
 64 getvp: mov ax, _vuport+2 ; get pointer segment
 65 mov es, ax
 66 mov bx, _vuport ; get offset
 67
 68 ; Clip if starting coordinates outside viewport
 69 mov cx, y ; get y
 70 cmp cx, es:[bx+6] ; is y within viewport?
 71 jl checkx ; ok if so
 72 jmp quit ; else quit
 73 checkx: mov ax, x ; get x
 74 cmp ax, es:[bx+4] ; is x within viewport?
 75 jl remap ; ok if so
 76 jmp quit ; else quit
 77
 78 ; Map starting coordinates to current viewport

 79 remap: add ax, es:[bx] ; offset x by vuport.left
 80 mov x, ax ; save remapped X
 81 add cx, es:[bx+2] ; offset y by vuport.top
 82 mov y, cx ; save remapped Y
 83
 84 ; Clip line length to viewport width
 85 mov ax, es:[bx+4] ; get vuport.width
 86 sub ax, x ; maxlength = width - starting x
 87 add ax, es:[bx] ; + vuport.left
 88 cmp ax, len ; if maxlength > length
 89 jg wm0 ; length is ok
 90 mov len, ax ; else length = maxlength
 91
 92 ; Set 6845 for write mode 0, all planes enabled, color selected
 93 wm0: mov dx, 03CEh
 94 mov ax, 0005h ; Set write mode
 95 out dx, ax
 96 mov ax, 0FF00h ; Set/Reset reg, enable all planes
 97 out dx, ax
 98 mov ax, 0FF01h ; Enable set/reset reg, all planes
 99 out dx, ax
 100 mov dx, 03C4h ; 6845 address reg
 101 mov al, 2 ; Data reg
 102 mov ah, _color1 ; Palette reg planes enabled
 103 out dx, ax ; Set color code
 104 ; Following added May '89
 105 mov ah, 3 ; Function select reg
 106 xor al, al ; Pixel-replace mode
 107 out dx, ax
 108
 109 ; Compute x coord for last byte to be written
 110 mov bx, x ; get start of line
 111 add bx, len ; end = start + length
 112 mov cx, bx
 113 and cx, 0FFF8h ; x coordinate where odd bits
 114 mov last, cx ; at end of line begin
 115
 116 ; Compute number of odd pixels at end of line
 117 sub bx, cx
 118 mov oddeol, bx ; save it
 119
 120 ; Construct pixel mask for last byte of line
 121 cmp bx, 0
 122 jz bsol ; go if no odd pixels
 123 xor ax, ax
 124 eolb: shr ax, 1 ; shift right and
 125 or ax, 80h ; set H/O bit
 126 dec bl ; until mask is built
 127 jnz eolb
 128 mov eolbits, ax ; then save mask
 129
 130 ; Compute number of odd pixels at start of line
 131 bsol: mov cx, x ; get starting X again
 132 and cx, 7 ; # of pixels from start of byte
 133 jz saddr ; go if none
 134 mov bx, 8
 135 sub bx, cx ; # of pixels to write
 136 mov oddsol, bx ; save
 137

 138 ; Construct pixel mask for first byte of line
 139 xor ax, ax
 140 solb: shl ax, 1 ; shift left and
 141 or ax, 1 ; set L/O bit
 142 dec bl ; until mask is built
 143 jnz solb
 144 mov solbits, ax ; then save mask
 145
 146 ; Translate last byte X into an address
 147 saddr: mov ax, 0A000h
 148 mov es, ax ; ES ==> video buffer
 149 mov bx, y ; get row
 150 mov ax, 80
 151 mul bx
 152 mov bx, ax ; BX = row offset = row * 80
 153 push bx ; save row offset
 154 mov ax, last ; get last byte X
 155 mov cl, 3
 156 shr ax, cl ; shift for col offset
 157 add bx, ax ; last offs = row offs + col offs
 158 mov last, bx
 159
 160 ; Compute address of first byte (ES:[BX])
 161 pop bx ; fetch row offset
 162 mov ax, x ; get col offset
 163 mov cl, 3
 164 shr ax, cl ; shift right 3 for col offset
 165 add bx, ax ; offset = row offs + col offs
 166 cmp bx, last ; is first byte also last?
 167 jz weol ; skip to end mask if so
 168
 169 ; Write start of line
 170 mov dx, 03CEh ; 6845 port
 171 mov ah, solbits ; start-of-line mask
 172 cmp ah, 0
 173 jz w8 ; go if empty mask
 174 mov al, 8 ; set bit mask reg
 175 out dx, ax
 176 mov cl, es:[bx] ; load 6845 latches
 177 mov ax, solbits
 178 neg al ; complement
 179 dec al ; for reversed bit mask
 180 and al, cl ; filter previously unset pixels
 181 mov es:[bx], al ; clear affected bits
 182 mov al, _color1
 183 mov es:[bx], al ; set affected bits
 184 inc bx ; next byte
 185 cmp bx, last ; ready for end of line yet?
 186 jae weol ; go if so
 187
 188 ; Write 8 pixels at a time until last byte in line
 189 w8: mov ax, 0FF08h ; update all pixels in byte
 190 out dx, ax ; set bit mask reg
 191 mov al, es:[bx] ; load 6845 latches
 192 xor al, al
 193 mov es:[bx], al ; clear all pixels
 194 mov al, _color1
 195 mov es:[bx], al ; set all bits
 196 inc bx ; next byte

 197 cmp bx, last ; thru?
 198 jnz w8 ; loop if not
 199
 200 ; Write end of line
 201 weol: mov dx, 03CEh ; 6845 port
 202 mov ah, eolbits ; end-of-line mask
 203 cmp ah, 0
 204 jz rvc ; go if empty mask
 205 mov al, 8 ; set bit mask reg
 206 out dx, ax
 207 mov cl, es:[bx] ; load 6845 latches
 208 mov ax, eolbits
 209 neg al ; complement
 210 dec al ; for reversed bit mask
 211 and al, cl ; filter previously unset pixels
 212 mov es:[bx], al ; clear affected bits
 213 mov al, _color1
 214 mov es:[bx], al ; set affected bits
 215
 216 ; Restore video controller to default state
 217 rvc: mov dx, 03CEh
 218 mov ax, 0005h ; write mode 0, read mode 0
 219 out dx, ax
 220 mov ax, 0FF08h ; default bit mask
 221 out dx, ax
 222 mov ax, 0003h ; default function select
 223 out dx, ax
 224 xor ax, ax ; zero Set/Reset
 225 out dx, ax
 226 mov ax, 0001h ; zero Enable Set/Reset
 227 out dx, ax
 228 mov dx, 03C4h ; 6845 address reg
 229 mov ax, 0F02h ; Data reg, enable all planes
 230 out dx, ax
 231
 232 ; End of routine
 233 quit: mov sp, bp
 234 pop bp
 235 retf
 236 _hline ENDP
 237 END
 238
 239






[LISTING THREE]


/* From May, '89 */
/* ------------- */
typedef int VP_HAN; /* viewport handle type */

void far default_viewport (int height); /* init default viewport */

VP_HAN far vp_open (int x, int y, int width, int height);

 /* open viewport, make it active */

int far vp_use (VP_HAN vp); /* make vp active */

void far vp_close (VP_HAN vp); /* close viewport */

void far vp_outline (VP_HAN vp); /* outline vp */

VP_HAN far vp_active (void); /* get handle of active vp */

int far vp_width (void) /* get active vp width */

int far vp_height (void) /* get active vp height */







[LISTING FOUR]


/* VUPORT.C: Library source for viewport support */
/* This is part of the GRAFIX library */
/* K. Porter, .MDUL/DDJ.MDNM/ Graphics Programming Column, May '89 */

#include <stdio.h>
#include <stdlib.h>
#include "grafix.h"

typedef struct vpnode { /* viewport descriptor node */
 int left, top, width, height,
 handle;
 struct vpnode far *next, far *prev;
} VPNODE;

#if !defined TRUE
#define FALSE 0
#define TRUE !FALSE
#endif

VPNODE far *vplist = NULL; /* pointer to list of vpnodes */
VPNODE def_vp = {0, 0, 640, 350, 0}; /* default viewport */
VPNODE far *vuport = &def_vp; /* ptr to current viewport */
/* -------------------------- */

void far default_viewport (int height)
{ /* initialize default viewport */
 vuport->height = height;
} /* ------------------------ */

VP_HAN far vp_open (int x, int y, int width, int height)
{ /* open a viewport */
VPNODE far *newvp, far *listnode;

 newvp = malloc (sizeof *newvp); /* make space */
 newvp->next = newvp->prev = NULL; /* init list ptrs */
 newvp->left = x; /* set descriptor */

 newvp->top = y;
 newvp->width = width;
 newvp->height = height;
 newvp->handle = 1;
 if (vplist == NULL)
 vplist = newvp; /* set first viewport */
 else {
 listnode = vplist; /* add viewport to list */
 while (listnode->next != NULL)
 listnode = listnode->next; /* find tail */
 listnode->next = newvp; /* update pointers */
 newvp->prev = listnode;
 newvp->handle = listnode->handle + 1; /* and handle */
 }
 vuport = newvp; /* make new viewport active */
 return newvp->handle;
} /* ------------------------------------------------------ */

int far vp_use (VP_HAN vp) /* make vp active */
{
VPNODE far *port;
int success = FALSE;

 if (vp == 0) { /* switch to default viewport */
 vuport = &def_vp;
 success = TRUE;
 } else { /* find desired viewport */
 port = vplist;
 do {
 if (port->handle == vp) {
 success = TRUE;
 break; /* jump out of loop when found */
 }
 port = port->next;
 } while (port != NULL);
 if (success)
 vuport = port; /* make active */
 }
 return success;
} /* ------------------------------------------------------ */

void far vp_close (VP_HAN vp) /* close a viewport */
{
VPNODE far *port;

 if (vp == 0) return; /* can't close default vp */
 port = vplist;
 while (port->next != NULL) { /* find desired viewport */
 if (port->handle == vp)
 break;
 port = port->next;
 }
 if (port != NULL) { /* viewport was found */
 if (port->next != NULL) /* so remove from list */
 port->next->prev = port->prev;
 if (port->prev != NULL)
 port->prev->next = port->next;
 if (port == vplist) /* if removing head */
 vplist = port->next;

 if (port == vuport) /* if removing active vp */
 vuport = &def_vp;
 free (port);
 }
} /* ------------------------------------------------------ */

void far vp_outline (VP_HAN vp) /* outline vp */
 /* outline is immediately outside viewport area */
{
VP_HAN active;
int x, y, w, h;

 active = vp_active(); /* save current viewport */
 if (vp_use (vp)) {
 x = vuport->left-1;
 y = vuport->top-1;
 w = vuport->width+1;
 h = vuport->height+1;

 /* draw outline */
 vp_use (0); /* full screen */
 draw_rect (x, y, w, h);
 vp_use (active); /* restore active viewport */
 }
} /* ------------------------------------------------------ */

VP_HAN far vp_active (void) /* get handle of active vp */
{
 return vuport->handle;
} /* ------------------------------------------------------ */

int far vp_width (void) /* get active vp width */
{
 return vuport->width;
} /* ------------------------------------------------------ */

int far vp_height (void) /* get active vp height */
{
 return vuport->height;
} /* ------------------------------------------------------ */






[LISTING FIVE]


/* VP.C: Draws several viewports */
/* Hex star inside each */
/* K. Porter, .MDUL/DDJ.MDNM/ Graphics Programming Col, 5/89 */

#include "grafix.h"
#include <conio.h>

int hex1[] = {25,55, 110,20, 110,90, 25,55};
int hex2[] = {55,20, 135,55, 55,90, 55,20};


void main()
{
VP_HAN vp1, vp2, vp3;

 if (init_video (EGA)) {

 /* Draw star in default viewport */
 set_color1 (5); /* magenta */
 polyline (3, hex1);
 polyline (3, hex2);
 set_color1 (9);

 /* Draw star in unbordered viewport */
 set_color1 (1); /* blue */
 vp1 = vp_open (60, 250, 200, 95);
 polyline (3, hex1);
 polyline (3, hex2);

 /* Draw star in small bordered viewport */
 vp2 = vp_open (200, 150, 100, 75);
 vp_outline (vp2);
 set_color1 (15); /* white */
 polyline (3, hex1);
 polyline (3, hex2);

 /* Draw star in filled bordered viewport */
 vp3 = vp_open (400, 60, 140, 100);
 vp_outline (vp3);
 set_color1 (2); /* green */
 fill_rect (0, 0, vp_width(), vp_height());
 set_color1 (4); /* red */
 polyline (3, hex1);
 polyline (3, hex2);

 /* Wait for keypress, then close vp's and quit */
 getch();
 vp_close (vp1);
 vp_close (vp2);
 vp_close (vp3);
 }
}






[LISTING SIX]

/* LINEGRAF.C: Self-scaling viewports showing a line graph */
/* K. Porter, .MDUL/DDJ.MDNM/ Graphics Programming Column, May '89 */

#include "grafix.h"
#include <conio.h>

int data [] = {30, 70, 10, 40, 30, 80, 60, 90, 40, 50, 40};

void main ()
{

void graph (void);
VP_HAN vp1, vp2, vp3;

 if (init_video (EGA)) {

 /* Open three viewports */
 vp1 = vp_open ( 1, 1, 360, 150);
 vp2 = vp_open ( 1, 170, 638, 170);
 vp3 = vp_open (500, 50, 80, 50);

 /* First version upper left quadrant */
 vp_use (vp1);
 set_color1 (14); /* yellow */
 vp_outline (vp1);
 graph();
 getch(); /* wait for keypress */
 vp_close (vp1);

 /* Second across bottom */
 vp_use (vp2);
 set_color1 (2); /* green */
 vp_outline (vp2);
 graph();
 getch(); /* wait for keypress */
 vp_close (vp2);

 /* Third at upper right */
 vp_use (vp3);
 set_color1 (3); /* cyan */
 vp_outline (vp3);
 graph();
 getch(); /* wait for keypress */
 vp_close (vp3);
 }
} /* ------------------------ */

void graph (void) /* draw self-scaling line graph */
{
int npts, p, prevx, prevy, curx, cury;
double vint, hint;

 npts = sizeof (data) / sizeof (int); /* # data points */

 /* Scaling factors for data points */
 hint = (double) vp_width() / (npts - 1); /* horizontal */
 vint = (double) vp_height() / 100.0; /* vertical */

 /* Draw the graph */
 prevx = 0; prevy = vp_height() - (int)(data[0] * vint);
 for (p = 1; p < npts; p++) {
 curx = (int)(p * hint);
 cury = vp_height() - (int)(data[p] * vint);
 draw_line (prevx, prevy, curx, cury);
 prevx = curx; prevy = cury;
 }
}





[GRAFIX.C, NOT A NUMBER LISTING IN DDJ 5/89]

/* Library source file GRAFIX.C */
/* EGA/VGA graphics subsystem */
/* Following library routines are external: */
/* DRAWPT.ASM Feb '89 */
/* HLINE.ASM Mar '89 */
/* EGAPALET.C Apr '89 */
/* PIXEL.ASM May '89 */
/* VUPORT.C May '89 */
/* K. Porter, DDJ Graphics Programming Column */
/* ------------------------------------------ */

#include <dos.h>
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
#include "grafix.h"

#if !defined TRUE
#define FALSE 0
#define TRUE !FALSE
#endif

/* Variables global to this library */
int color1, /* foreground color */
 oldmode = 0, /* pre-graphics mode */
 grafixmode = 0, /* default graphics mode */
 ega = FALSE, /* equipment Booleans */
 vga = FALSE,
 colormonitor = FALSE,
 curpos, /* text cursor position */
 textpage; /* active text page */
unsigned vidmem; /* video buffer segment */
char far *vidsave; /* video buffer save area */
/* -------------------------------------------------------- */

int far init_video (int mode)

/* Initializes video adapter and defaults for mode */
/* Sets up pc_textmode() to be called on pgm termination */
/* Returns TRUE or FALSE indicating success */
/* This function will be expanded in a later version */
{
union REGS r;
int result = FALSE;

 /* Determine attached adapter and monitor */
 r.h.ah = 0x1A; /* VGA inquiry function */
 r.h.al = 0;
 int86 (0x10, &r, &r); /* ROM BIOS call */
 if (r.h.al == 0x1A)
 switch (r.h.bl) {
 case 4 : ega = TRUE; /* EGA color */
 colormonitor = TRUE;
 break;
 case 5 : ega = TRUE; /* EGA mono */
 break;

 case 7 : ega = TRUE; /* VGA mono */
 vga = TRUE;
 break;
 case 8 : ega = TRUE; /* VGA color */
 vga = TRUE;
 colormonitor = TRUE;
 }
 else { /* No VGA, so check for EGA */
 r.h.ah = 0x12;
 r.x.bx = 0x10;
 int86 (0x10, &r, &r);
 if (r.x.bx != 0x10) { /* if EGA present... */
 ega = TRUE; /* set flag */
 r.h.ah = 0x12;
 r.h.bl = 0x10; /* find out which monitor */
 int86 (0x10, &r, &r);
 if (r.h.bl != 0)
 colormonitor = TRUE; /* EGA color */
 }
 }

 /* Proceed only if EGA or VGA present */
 if (ega vga) {
 set_color1 (15); /* default pixel color */
 r.h.ah = 0x0F; /* get current screen mode */
 int86 (0x10, &r, &r);
 oldmode = r.h.al; /* store it */
 textpage = r.h.bh; /* also active text page */

 if (colormonitor) /* point to video memory */
 vidmem = 0xB800;
 else
 vidmem = 0xB000;
 vidsave = malloc (4096); /* allocate save area */
 movedata /* save text screen contents */
 (vidmem, 0, FP_SEG (vidsave), FP_OFF (vidsave), 4096);

 r.h.ah = 3; /* get text cursor position */
 r.h.bh = textpage;
 int86 (0x10, &r, &r);
 curpos = r.x.dx; /* and save it */

 if ((mode == EGA) && ega) {
 r.h.ah = 0;
 r.h.al = mode; /* set EGA mode */
 int86 (0x10, &r, &r);
 grafixmode = mode; /* save mode */
 atexit (pc_textmode); /* register exit function */
 result = TRUE;
 } else
 if ((mode == VGA16) && vga) { /* typo fixed 5/89 */
 r.h.ah = 0;
 r.h.al = mode;
 int86 (0x10, &r, &r);
 grafixmode = mode;
 atexit (pc_textmode);
 result = TRUE;
 }
 }

 if (!result) { /* unable to switch to graphics */
 oldmode = 0; /* so cancel text screen save */
 free (vidsave);
 vidsave = 0;
 }
 if (mode == VGA16 && vga) /* viewport init added May '89 */
 default_viewport (480);
 return result;
} /* ------------------------------------------------------ */

void far pc_textmode (void)
/* SPECIFIC TO MS-DOS */
/* Restore text mode */
/* Automatically called on pgm termination */
{
union REGS r;

 if (oldmode) { /* if not in text mode now... */
 r.h.ah = 0;
 r.h.al = oldmode; /* restore text mode */
 int86 (0x10, &r, &r);
 movedata /* restore text screen */
 (FP_SEG (vidsave), FP_OFF (vidsave), vidmem, 0, 4096);
 free (vidsave); /* free allocated memory */
 vidsave = 0; /* zero pointer */
 oldmode = 0; /* reset */
 r.h.ah = 2; /* restore old cursor position */
 r.h.bh = textpage;
 r.x.dx = curpos;
 int86 (0x10, &r, &r);
 }
} /* ------------------------------------------------------ */

void far set_color1 (int palette_reg)
/* Select pixel color from palette register */
{
 color1 = palette_reg;
} /* ------------------------------------------------------ */

void far draw_line (int x1, int y1, int x2, int y2)
 /* Bresenham line drawing algorithm */
 /* x1, y1 and x2, y2 are end points */
{
int w, h, d, dxd, dyd, dxn, dyn, dinc, ndinc, p;
register x, y;

 /* Set up */
 x = x1; y = y1; /* start of line */
 w = x2 - x1; /* width domain of line */
 h = y2 - y1; /* height domain of line */

 /* Determine drawing direction */
 if (w < 0) { /* drawing right to left */
 w = -w; /* absolute width */
 dxd = -1; /* x increment is negative */
 } else /* drawing left to right */
 dxd = 1; /* so x incr is positive */
 if (h < 0) { /* drawing bottom to top */
 h = -h; /* so get absolute height */

 dyd = -1; /* y incr is negative */
 } else /* drawing top to bottom */
 dyd = 1; /* so y incr is positive */

 /* Determine major axis of motion */
 if (w < h) { /* major axis is Y */
 p = h, h = w, w = p; /* exchange height and width */
 dxn = 0;
 dyn = dyd;
 } else { /* major axis is X */
 dxn = dxd;
 dyn = 0;
 }

 /* Set control variables */
 ndinc = h * 2; /* Non-diagonal increment */
 d = ndinc - w; /* pixel selection variable */
 dinc = d - w; /* Diagonal increment */

 /* Loop to draw the line */
 for (p = 0; p <= w; p++) {
 draw_point (x, y);
 if (d < 0) { /* step non-diagonally */
 x += dxn;
 y += dyn;
 d += ndinc;
 } else { /* step diagonally */
 x += dxd;
 y += dyd;
 d += dinc;
 }
 }
} /* ------------------------------------------------------ */

void far draw_rect (int xleft, int ytop, int w, int h)
/* Draw outline rectangle in color1 from top left corner */
/* w and h are width and height */
/* xleft and ytop are top left corner */
{
 draw_line (xleft, ytop, xleft+w, ytop); /* top */
 draw_line (xleft+w, ytop, xleft+w, ytop+h); /* right */
 draw_line (xleft+w, ytop+h, xleft, ytop+h); /* bottom */
 draw_line (xleft, ytop+h, xleft, ytop); /* left */
} /* ------------------------------------------------------ */

void far polyline (int edges, int vertex[])
/* Draw multipoint line of n edges from n+1 vertices where: */
/* vertex [0] = x0 vertex [1] = y0 */
/* vertex [2] = x1 vertex [3] = y1 */
/* etc. */
{
int x1, y1, x2, y2, v;

 x1 = vertex[0];
 y1 = vertex[1];
 for (v = 2; v < (edges+1)*2; v+= 2) {
 x2 = vertex[v];
 y2 = vertex[v+1];
 draw_line (x1, y1, x2, y2);

 x1 = x2;
 y1 = y2;
 }
} /* ------------------------------------------------------ */

void far fill_rect (int xleft, int ytop, int w, int h)
/* Draw solid rectangle in color1 from top left corner */
{
register y;

 for (y = ytop; y < ytop+h; y++)
 hline (xleft, y, w);
} /* ------------------------------------------------------ */


[GRAPHIX.H, NOT A NUMBERED LISTING IN DDJ, 5/89]


/* Include file for GRAFIX.LIB */
/* EGA/VGA graphics subsystem */
/* K. Porter, DDJ Graphics Programming Column */
/* ------------------------------------------ */

/* Color constants from April, 89 */
#define Black 0 /* standard colors */
#define Blue 1
#define Green 2
#define Cyan 3
#define Red 4
#define Magenta 5
#define Brown 0x14
#define LtGray 7
#define DkGray 0x38
#define LtBlue 0x39
#define LtGreen 0x3A
#define LtCyan 0x3B
#define LtRed 0x3C
#define LtMagenta 0x3D
#define Yellow 0x3E
#define White 0x3F

#define RED0 0x00 /* basic hues for mixing */
#define RED1 0x20
#define RED2 0x04
#define RED3 0x24
#define GRN0 0x00
#define GRN1 0x10
#define GRN2 0x02
#define GRN3 0x12
#define BLU0 0x00
#define BLU1 0x08
#define BLU2 0x01
#define BLU3 0x09

#if !defined byte
#define byte unsigned char
#endif

/* Supported video modes */

#define EGA 0x10 /* EGA 640 x 350, 16/64 colors */
#define VGA16 0x11 /* VGA 640 x 480, 16/64 colors */

/* Function prototypes */
/* From February, '89 */
/* ------------------ */
int far init_video (int mode); /* init display in video mode */

void far pc_textmode (void); /* PC text mode */

void far draw_point (int x, int y); /* write pixel in color1 */

void far set_color1 (int palette_reg); /* set foreground color */

/* From March, '89 */
/* --------------- */
void far draw_line (int x1, int y1, int x2, int y2);
 /* Bresenham line drawing algorithm */

void far draw_rect (int left, int top, int width, int height);
 /* draw rectangle from top left corner */

void far polyline (int edges, int vertices[]); /* draw polyline */

void far hline (int x, int y, int len); /* horizontal line */

void far fill_rect (int left, int top, int width, int height);
 /* draw solid rectangle in color1 starting at top left corner */

/* From April, '89 */
/* --------------- */
byte far ega_palreg (int preg); /* color in EGA palette reg */

void far set_ega_palreg (int reg, int color); /* set palette reg */

byte far colorblend (byte r, byte g, byte b); /* blend hues */

void far get_ega_colormix (int preg, int *r, int *g, int *b);
 /* get mix of red, green, and blue in EGA pal register preg */

/* From May, '89 */
/* ------------- */
typedef int VP_HAN; /* viewport handle type */

void far default_viewport (int height); /* init default viewport */

VP_HAN far vp_open (int x, int y, int width, int height);
 /* open viewport, make it active */

int far vp_use (VP_HAN vp); /* make vp active */

void far vp_close (VP_HAN vp); /* close viewport */

VP_HAN far vp_active (void); /* get handle of active vp */

void far vp_outline (VP_HAN vp); /* outline vp */

int far vp_width (void); /* get active viewport width */


int far vp_height (void); /* and height */





























































May, 1989
RUN LENGTH ENCODING REVISITED


There's more than one way to skin RLE




Phil Daley


Phil was a technical editor of MICRO The 6502 Journal for two years and is now
a software engineer at Mentor Resources, 1 Tara Blvd., Nashua, NH 03062;
603-888-2580.


Many files stored on computer systems have a lot of wasted space: Often,
records have duplicated spaces or fixed length records that aren't fully used,
so blank areas of unused space exist that could be compressed out of the
records. (Notice how much smaller an .EXE file is after being processed by
EXEPACK, if it wasn't linked with /E.) In my work with user interfaces and
screen files, a good deal of each record is either blanks or zeros. Routines
to squeeze this space out of records can be very simple (saving a minimum
amount of space) or very complex (designed by the program for each record and
requiring significant amounts of processing time). What brought this all to
mind was the RLE presented in Robert Zigon's "Run Length Encoding" (DDJ,
February 1989). The approach presented in that article, however, doesn't do
well with runs of unique characters, doubling the length of each unique
character as it goes. Two changes to the encoding rules can result in a
dramatic increase in the compression factor without a large increase in
processing time: 1. Don't compress less than three duplicated characters; and
2. Mark areas of unduplicated characters the same as duplicated ones, using a
bit to indicate whether the next run is compressed or not.
Every byte in the file is marked as either "compressed" or "uncompressed." The
markers (or "compression bytes") are 1 byte in size. A compression byte has
the high bit set if it indicates a run of redundant bytes, and clear if it
marks a set of unique bytes. The other seven bits are used to indicate the
length (1 - 128) of the run of compressed or uncompressed bytes.
In the case of a compression byte with the high bit set (>=128), the next byte
is the character that is duplicated. When a compression byte indicates unique
characters (<=127), the next compression_byte + 1 characters are the normal
bytes in the file. The worst case (all unique characters) becomes 129 bytes
for 128 bytes, and the best case is 2 bytes for 128 duplicated bytes. This is
illustrated in Figure 1, which equals a total of 10 bytes, the same as
Zigon's. Notice that the CCD pattern only increased the character count by 1
byte. This would be true for any number of unique bytes in a row, up to 128.
Zigon's method increases each different character to 2 bytes each.
Figure 1: Compression byte with high list set

 Input buffer: A A A A B B B B B B C C D E E E E E

 Output buffer: 83 41 85 42 02 43 43 44 84 45

 Translation: 83 (4 compressed bytes value) 41
 85 (6 compressed bytes value) 42
 02 (3 uncompressed bytes value) 43 43 44
 84 (5 compressed bytes value) 45

In my application of saving screen images and other database-type records,
where a good deal of the record is either spaces or zeros, this routine saves
50 percent of the record space, on the average. The routine does require you
to save the record length along with the record, but most C implementations of
variable length indexed records require this information.
While I haven't coded this in assembly because the C implementation runs fast
enough for my practical purposes, it probably runs a little slower than
Zigon's for compression (See Listing One). The uncompress routine is probably
just as fast as his, because of its simplicity.


Notes


The disk program files were compiled using the large model, the only one for
which I keep libraries on my disk. The source would produce smaller, faster
.OBJ and .EXE files if it were recompiled with the small memory model.


Availability


All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063; or call 800-356-2002 (from inside
Calif.) or 800-533-4372 (from outside Calif.). Please specify the issue number
and format (MS-DOS, Macintosh, Kaypro).


_RLE Revisited_
by Phil Daley


[LISTING ONE]

/*****************************************************************************
* PROGRAM RLE.C *
* written by Phil Daley *
* February 3, 1989 *
*****************************************************************************/
int main(void);
int uncompress(unsigned char *,int,unsigned char *) ;

int compress(unsigned char *,int,unsigned char *) ;
int process_comp(unsigned char *,int,int) ;
int process_uncomp(unsigned char *,int,int) ;
#include <stdio.h>
#include <memory.h>
unsigned char screen[24][80] = {
"IMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM;",
": :",
": :",
": :",
": o This is a sample screen that would be typical of the type :",
": that would present information to a user for instructions :",
": or a help screen, etc. :",
": :",
": :",
": o While it contains a lot of unique characters in the text :",
": lines, it also contains a lot of white space in empty lines :",
": and margins. :",
": :",
": :",
": o It would be unusual for the compression algorithm used :",
": here to find any repeated characters other than spaces :",
": and the border. :",
": :",
": :",
": :",
": :",
": :",
": :",
"HMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM<"}
;

unsigned char new[2000] ;

/********************** main ***********************/
int main() /* this is a demo main */
{
 int orig_length = 1920 ;
 int compressed_length ;
 int i, j ;

 compressed_length = compress(screen[0],orig_length,new) ;
 printf("The original screen (1920) compressed to %d
bytes\n",compressed_length) ;
 memset(screen,0,1920) ; /* erase the orig */
 orig_length = uncompress(new,compressed_length,screen[0]) ;
 printf("And back to the original (%d) length\n",orig_length) ;
 for (i = 0; i < 24; i++)
 for (j = 0; j < 80; j++) /* show it */
 printf("%c",screen[i][j]) ;
 return(0) ;
}
/*********************** compress ****************************/
compress(in_array,in_size,out_array)
unsigned char *in_array ;
int in_size ;
unsigned char *out_array ;
{
 register int i = 0 ;
 register int j = 0 ;
 register int k ;

 register int l ;

 while (i < in_size) {
 if (in_array[i] == in_array[i + 1] && in_array[i + 1] == in_array[i + 2]) {
 k = process_comp(in_array,i,in_size) ;
 out_array[j++] = (unsigned char)k 0x80 ;
 out_array[j++] = in_array[i] ;
 i += k ;
 }
 else {
 k = process_uncomp(in_array,i,in_size) ;
 out_array[j++] = (unsigned char)k ;
 for (l = 0; l < k; l++)
 out_array[j++] = in_array[i++] ;
 }
 }
 return(j) ;
}
/*********************** process_comp ****************************/
process_comp(in_array,i,in_size)
unsigned char *in_array ;
int i ;
int in_size ;
{
 register int len = 0 ;

 while (in_array[i] == in_array[i + 1] && i < in_size) {
 len++ ;
 i++ ;
 if (len == 126)
 break ;
 }
 return(len + 1) ;
}
/*********************** process_uncomp ****************************/
process_uncomp(in_array,i,in_size)
unsigned char *in_array ;
int i ;
int in_size ;
{
 register int len = 0 ;

 while ((in_array[i] != in_array[i + 1] in_array[i] != in_array[i + 2]) && i <
in_size) {
 len++ ;
 i++ ;
 if (len == 127)
 break ;
 }
 return(len) ;
}


/********************** uncompress ***********************/
uncompress(in_array,in_size,out_array)
unsigned char *in_array ;
int in_size ;
unsigned char *out_array ;
{
 register int i ;

 register int j ;
 register int k=0 ;
 register int l ;

 for (i = 0; i < in_size;) {
 j = in_array[i++] ;
 if (j > 128) {
 for (j -= 128; j > 0; j--)
 out_array[k++] = in_array[i] ;
 i++ ;
 }
 else
 for (l = 0; l < j; l++)
 out_array[k++] = in_array[i++] ;
 }
 return(k) ;
}


-30-










































May, 1989
STRUCTURED PROGRAMMING


Pizza Terra




Jeff Duntemann, K16RA


One roasty Saturday last summer, Mr. Byte and I were steaming over The Hill in
the Magic Van to pick up some end mills in San Jose when the KWSS DJ caught my
ear:
"... and up next is the great new single by Pizza Terra!"
Pizza Terra? What a great name for a rock band! We haven't seen names like
that since the Strawberry Alarm Clock and the Peppermint Trolley Company, and
I was duly impressed --if also wondering why I hadn't ever heard of Pizza
Terra before.
Then he played the song. And moment by moment I found myself thinking, Boy,
that sure sounds like Peter Citera ...
Peter Citera ... Pete Citera ... Pizza Terra!
I'd been type cast!


Homonymously Yours


The last part of Pascal to resist my understanding was the subject of type
casting, and in Pizza Terra I've found a wonderful metaphor to make it go down
easier. What the DJ said was perfectly correct: Four spoken syllables that
sounded like "pee-tzuh-tear-uh." Given your typical DJ's level of enunciation,
those four syllables could as easily be taken to mean "Pizza Terra" as "Pete
Citera."
And that's the whole point.
Type casting is the process of taking a variable of one type and treating it
as a variable of another type. This sticks to the roofs of most mouths fed on
Pascal and Modula-2, both of which are pretty hardnosed about type checking.
Type casting jumps the type checking safety railing, allowing you to get into
two different kinds of trouble if you're not careful.
One kind of trouble is semantic nonsense. Both Turbo Pascal and Modula-2 allow
a variable of type Char to be cast onto a variable of type Boolean through
this notation:
 MyChar:= CHAR(MyBoolean);
This is completely legal. But what does it mean? Can a character be True? Is
False an aspect of the letter 'Q'? It sounds like nonsense to me --which is a
major reason we have type checking in civilized languages like Pascal and
Modula-2.
The Pascal/Modula-2 statement above implies that some sort of conversion
process is going on, and that processor cycles are being spent somehow
converting data from type Boolean to type Char before copying them to variable
MyChar. Not so --the data is simply copied from MyBoolean to MyCharwith nary a
thought for data typing. It's like picking up a cylindrical crucible of molten
aluminum and pouring the metal into a cubical mold. The shape of the aluminum
changes, but the metal itself is still aluminum. It doesn't change to copper
or zinc.
Underneath the level of Pascal or Modula-2, a Boolean variable is just a
single byte of memory with some bit pattern in it. Like the syllables of
speech, what a byte of memory means to us is pretty much what we decide it is
to mean, according to some preagreed scheme. If a byte of memory with the
value 01H is poured into a mold marked Boolean, we agree that it has the value
True, if we pour the same byte into a mold marked Char, it comes up on the
screen as a white smiley face.
Think of type casting as another form of metal casting: pouring raw materials
into a mold. The mold dictates the shape of the metal, but the underlying
nature of the raw material is unchanged in the process. Cast the four
syllables "pee-tzuh-tear-uh" into a mold marked "male vocalist" and you get
Pete Citera. Cast the same four syllables into a mold marked "imaginary
Sixties rock band" and you get Pizza Terra.


The Second Kind of Trouble


It's easy enough to imagine a second kind of trouble connected with breaking
the rules of type checking: What happens if you have more data in one type
than in another? If you cast 6 bytes of data into 10 bytes of data, what do
you have? Or worse, if you cast 10 bytes of data into 6 bytes of data, does
anything overflow? Do you overwrite unrelated variables that just happen to be
adjacent in memory?
Casting between any two types, even of different sizes, is possible in both
Turbo Pascal and Modula-2 by the same syntax shown earlier. If the source type
is smaller than the destination type, you end up with a partially filled
destination type. The generated code simply picks up bytes from the source
type, starting with the lowest memory address, and drops bytes into the
destination type until there are no more bytes to drop. Coping with the
resulting semantic nonsense is left to you, but there's no danger of
overwriting adjacent data.
When there's more source than destination, the code copies bytes from the
source until the destination is full, then stops. This eliminates any danger
of overwriting data adjacent in memory.


Sign Extension and Modula-2 Numeric Casts


One hiccup in the casting process occurs when data from one numeric type is
moved into another. The notion of simply moving bytes blindly from one
variable to another no longer applies.
Turbo Pascal and Modula-2 perform something called sign extension when data is
moved between a signed numeric type into a larger signed numeric type; for
example, from type Integer to type LongInt. The sign bit of the shorter type
is moved into the sign bit of the larger type, even though the bytes from the
shorter type do not entirely fill the larger type. The end result is that a
negative integer value cast onto a long integer will result in a negative long
integer value.
Most Turbo Pascal users take sign extension for granted, because numeric types
are nearly always assignment compatible, and explicit casting between numeric
types never needs to be done. However, even if you explicitly cast an integer
onto a long integer, sign extension will still happen.
Modula-2 is, again, much more stubborn than Pascal when moving numeric values
between types. Nearly all movement of values between numeric types must be
done through explicit casting. INTEGER and CARDINAL are assignment compatible,
as are SHORTCARD and SHORTINT. But that's about it. All other numeric value
transfers are done through an implied call to the VAL function, which includes
sign extension.
Numeric types demonstrate one Pascal safety railing that can't be jumped:
Turbo Pascal somewhat remarkably forbids a type cast of any real number type
onto any integer type, just as it forbids assignment of any real number type
to any integer type. Type Real cannot, in fact, be cast onto any other type.
To get at the bytes inside type Real, you have to use union labor, as I'll
explain shortly. Modula-2, by contrast, allows byte-by-byte casting of real
numbers onto any non-numeric type at all.


An All Union Cast


One problem with type casting in Turbo Pascal is that it's not really part of
the Pascal language, and other compilers in other environments may or may not
implement it. For most of us that doesn't matter; I consider the differences
in paradigms among environments (say, DOS, Mac, OS/2, and Unix) far more of a
barrier to portability than any deviation from an HLL's syntactic standard.
But the issue remains for those who care: There is really only one portable
way to perform typecasts in Pascal. This is the free union variant record (or
simply the free union), and it is about the most peculiar thing Niklaus Wirth
decided to include in his definition of Pascal.
The best way to approach free unions is to start with their less libertarian
relatives, discriminated unions. (We usually call discriminated unions variant
records.) The familiar Pascal variant record looks like this:

 ID =
 RECORD
 CASE Alien: Boolean OF
 True : (HomeStarID : StarID);
 False : (EarthID : CountryID);
 END;
There are three fields defined here (Alien, HomeStarID, and EarthID) but only
two exist at any one time. The Boolean field Alien is called the tag field and
is always present in any instance of the record type. Which of the other two
fields is considered present in the record depends on the value loaded into
Alien. If Alien contains True, then the other field in the record is
HomeStarID. If Alien contains False, then the other field is EarthID.
Beginners often ask: Well, who enforces that? Nobody; there's nothing to
enforce. People who think "enforcement" are thinking backwards. The tag field
exists to tell users of the record what information was written into the
record earlier so that those users can know how to interpret what they find.
The tag field is a flag, not a switch. Changing the value of Alien has no
effect on the data in the rest of the record. Given an instance of ID named
ScienceOfficer, the tag field is used this way:
 WITH ScienceOfficer DO
 IF Alien THEN
 SendHyperwaveTo
 (HomeStarID)
 ELSE
 SendTelegramTo(EarthID);
Changing the value in Alien in violation of the agreed upon scheme (that is,
setting it to False for that guy Spock) will cause confusion, but only
semantic confusion. The system won't crash.
If you're sharp it may occur to you that variant records are themselves a
means of type casting, and you're right. Once you write a HomeStarID into an
ID record, you can ignore the tag field and read the identical data right back
as an EarthID type, essentially casting a HomeStarID type onto an EarthID
type.
And if the only reason for the record is to cast one type onto another, who
needs the tag field? In fact, dropping the tag field is how we turn a
discriminated union into a free union:
 ScreenAtom =
 RECORD
 CASE Boolean OF
 True : (Ch : Char; Attr : Byte);
 False : (Atom : Word);
 END;
Like the Cheshire cat, the tag field has vanished, but its type remains. The
need for the tag field as flag is gone, but we need something to enumerate the
different variants, and the lonely type specifier Boolean does that quite well
by providing the True and False CASE labels. Any ordinal type (Char, Integer,
Word, Byte, or enumerations) would do as well, and you don't have to account
for every value in the type as long as you define at least two.
So what do we have here? A record of type ScreenAtom occupies 2 bytes of
storage, and that's all. The two variants may be considered two molds into
which those 2 bytes of storage may be cast. One mold provides two byte-sized
compartments called Ch and Attr. The other is a single 2-byte field called
Atom. We can write a word-sized value to the Atom field and then pick out its
two component bytes separately as Ch or Attr. The reverse, of course, is just
as true: We can build an atom in two operations, first by storing a character
in Ch, and then by storing its attribute in Attr. That done, we can read the
finished atom out as Atom:
 MyAtom.Ch := 'A';
 MyAtom.Attr := $07;
 MEMW[$B800 : 0]:= MyAtom.Atom;
If you haven't already recognized this record type, it's from the SCREENS.PAS
unit presented in last month's column. A character on the PC's text screen is
stored as 2 bytes side by side: The ASCII code for the character, and a byte
specifying the attribute of the character; that is, its color for color
screens, or things like underlining and reverse video for monochrome screens.
ScreenAtom lets us look at these two bytes as a unit, or else as the two
individual components, as we choose.
In the case of ScreenAtom, both of the two variants are of the same size, so
everything's tidy. That's not a requirement; if the several variants vary in
size, the record as a whole is as large as its largest variant. This prevents
any danger to adjacent data.


Registers on a Half Shell


The best example of a free union whose variants are not the same size is the
Registers type exported by the Turbo Pascal DOS unit, and the similar
Registerstype exported by TopSpeed Modula-2's SYSTEM module. In both cases,
the idea is to provide access to the 86-family CPU registers. Listing One
(UNIONS.SRC) shows the two free unions side by side for comparison purposes.
There are ten accessible registers altogether: AX, BX, CX, DX, BP, SI, DI, DS,
ES, and Flags. The first four, however, are commonly treated in two ways: As
16-bit units whose names end in X, or with each 16-bit unit seen as two 8-bit
components ending in H (for high) and L (for low.) For example, AX is composed
of AL and AH, BX of BL and BH, and so on.
Both free unions provide easy access to the four general-purpose registers AX,
BX, CX, and DX as both 16-bit units and pairs of 8-bit halves. Given an
instance of type Registers named Regs, when you want to access all of AX, you
specify Regs.AX. If you simply want to access the high half of AX without
disturbing the low half (say, for passing parameters to a DOS or BIOS service)
you specify Regs.AH.
A drawing of the internals of the Turbo Pascal Registers type is shown in
Figure 1, reprinted from my book, Complete Turbo Pascal, Third Edition. Each
variant maps a different structure on the same 20 bytes of memory. The 0
variant partitions those 20 bytes into ten 2-byte words. The 1 variant treats
the first eight bytes as single-byte quantities --and ignores the remaining 12
bytes.
The Modula-2 version provides the additional service of subdividing the Flags
register into the 16 separate bits of a Modula-2 BITSET type, allowing
individual flags to be tested by the very intuitive set operators.
In summary, what both Registerstypes provide are two different interpretations
of the same region of memory, safely and with zero overhead in processor
cycles. Like Pizza Terra and Pete Citera, it all depends on how you look at
(or listen to) things.


Boxes and Bars


Now, as the retired Tibetan nomad might have said after a big dinner, 'Nuff
yak. Code is It, as John Sculley didn't say --keep your soda pop salesmen
straight --but I will. And here's some more.
In last month's column I presented the core of a virtual screens module in
Turbo Pascal. The idea, to recap briefly, is to create a virtual screen on the
heap, the size of an 8 1/2-inch by 11-inch piece of paper, and use the full
display screen as a window onto that virtual screen.
The core unit, SCREENS.PAS, presented primitives for initializing and
disposing of virtual screens, clearing them and writing to them, and panning
the visible display up and down a virtual screen. In this issue I'm providing
the beginning of a second-level unit, VTOOLS.PAS, (Listing Two) to contain
higher-level display routines like forms and menus. In the future, when I say,
"Add this procedure to the VTOOLS unit," you'll know what I mean.
Building things on the screen demands easy access to the PC's line-drawing
characters, and this is what VTOOLS provides in this first iteration. The
characters come in two forms: single-line and double-line. I've defined each
character as a two-element array, indexed by the Boolean constants Single-Line
and DoubleLine, with the sense that DoubleLine (True) selects the double-line
characters. The character definitions are stored in a record constant whose
fields are of type LineChars. Picking a specific character from the record is
done this way:
 BoxChars.LLCorner[SingleLine]
This expression "cooks down" to character 192 --and makes a lot more sense
when you read the code. It's often useful to have vertical and horizontal bars
within easy reach for drawing forms and boxes. I've provided an array type
called BarStrings containing two 255-character long STRING types, again
indexed by those same two Boolean constants SingleLine and DoubleLine. For
example, when you need a long string of single-line vertical bars, you would
use this expression:
 VBars[SingleLine]
To generate a bar shorter than 255 characters, you would use the Copy
function:
Copy(HBars[DoubleLine], 1, BarLength)
Having a string full of vertical bar characters is pointless if there's no way
to write that string to the screen ... vertically. So, Listing Three,
(WRTDOWN. SRC) provides vertical string display to virtual screens. Add
WriteDownTo to the SCREENS.PAS source code file presented last issue! If you
don't, this month's code will not compile.
Prime user of WriteDownTo is MakeBox, which is quite straightforward and fast.
MakeBox draws boxes on a virtual screen, in either single-line or double-line
form as selected. To demonstrate MakeBox, try BOXTEST.PAS (Listing Four),
which I adapted from Complete Turbo Pascal to operate with virtual screens.
BoxTest just throws random-sized boxes all over the place, and allows you to
pan through the chaos without interrupting it --demonstrating that virtual
screens can be easily examined in media res without disrupting ongoing work.
Run BoxTest and pan with the up/down arrow keys --you'll be seeing boxes
before your eyes for some time to come.



A Never Ending Life Saver


There's a funny fantasy story by Bernard Wolfe called "The Never Ending Penny"
in which a man wishes that there would always be a penny in his pocket. He
gets his wish, and no matter how many times he pulls the penny from his
pocket, there is always another there. Unfortunately, it takes a lot of
pennies to buy anything useful, and when he complains that his arm is fit to
fall off from pulling pennies out of his pocket, his fairy godmother chides
him to be glad he didn't wish for a never ending Wintergreen Lifesaver -- the
chap who did soon weighed 300 pounds.
In reading through the documentation for Turbo Power Software's new Turbo
Professional 5.0, I felt that no matter how many routines I pulled out, there
would always be another one there. The range of this toolkit package is simply
astonishing. Just a few of its video-related abilities include mouse support,
a popup help system, pick lists and menus, and (gasp) a virtual screen system.
(I guess --fortunately for this column --Kim & Company don't share my
obsession with 66-line screens.) Also in the package are data entry screen
support, extended and expanded memory support, long ASCIIZ strings, critical
error handlers, in-memory sorts, and virtual data arrays up to 32 Mbytes in
size. The full $125 price of the package, however, may well be worth a single
section of the product devoted to interrupt service routines and TSRs.
Interrupts and TSRs are the LaBrea tarpits of our industry; no matter how many
times newcomer programmers are reminded how dangerous such criters are, they
march into the tar up to the neck and then squeak when they get stuck. In
Turbo Professional's scheme you write your program as a single, large
procedure and then drop the procedure into a very tidy TSR program shell. The
whole thing, when run, loads your procedure as a TSR and handles all the hairy
stuff like assigning hot-keys and keeping DOS from meeting itself in a dark
alley with bloody results.
I don't have time to say more, but I will say that if you intend to write TSRs
you must get this package. All the source is present, so if you choose to,
(and you should, for your own protection) you can read the source and
understand all the considerable magic going on beneath the surface.
To say "highly recommended" abuses the word "highly."


Cheaper Chips


While curled up with the Sunday paper this morning, I saw that Fry's is
selling 1MB 100ns DRAMs for $24.95, making a bank of nine cost only $225
--down by a third from when I last looked. Maybe it's time to see for myself
if OS/2 really is half an operating system, or truly the next generation.
Certainly it's time for a trip to Fry's, which (for those of you who haven't
heard) was a drugstore with a computer corner that eventually became a
computer store with a toothpaste-and-Kleenex aisle.
Besides, I think I'm out of Ultra Brite.


Availability


All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063, or call 800-356-2002 (inside Calif.) or
800-533-4372 (outside Calif.). Please specify the issue number and format
(MS-DOS, Macintosh, Kaypro).


Products Mentioned


Turbo Professional 5.0 Turbo Power Software P.O. Box 66747 Scotts Valley, CA
95066-0747 408-438-8608


_Structured Programming Column_
by Jeff Duntemann


[LISTING ONE]

{ Turbo Pascal 4.0/5.0 Registers type, from the DOS unit: }

Registers = RECORD
 CASE Integer OF
 0 : (AX,BX,CX,DX,BP,SI,DI,DS,ES,Flags : Word;
 1 : (AL,AH,BL,BH,CL,CH,DL,DH : Byte;
 END;

(* TopSpeed Modula 2's Registers type, from the SYSTEM module: *)

Registers = RECORD
 CASE : BOOLEAN OF
 TRUE : AX,BX,CX,DX,BP,SI,DI,DS,ES : CARDINAL;
 Flags : BITSET;
 FALSE : AL,AH,BL,BH,CL,CH,DL,DH : SHORTCARD;
 END;
 END;






[LISTING TWO]


{--------------------------------------------------------------}
{ VTOOLS }
{ }
{ Virtual screen I/O tools unit }
{ }
{ by Jeff Duntemann KI6RA }
{ Turbo Pascal 5.0 }
{ Last modified 1/17/89 }
{--------------------------------------------------------------}

UNIT VTools;

INTERFACE

USES DOS, { Standard Borland unit }
 TextInfo, { Given in DDJ 3/89 }
 Screens; { Given in DDJ 4/89 }

CONST
 SingleLine = False; { To specify single vs. double line }
 DoubleLine = True; { bars and boxes }

TYPE
 LineChars = ARRAY[SingleLine..DoubleLine] OF Char;
 BarStrings = ARRAY[SingleLine..DoubleLine] OF String;
 BoxRec = RECORD
 ULCorner, { Each field in this record }
 URCorner, { contains both the single }
 LLCorner, { line and double line }
 LRCorner, { form of the named line }
 HBar, { character, indexed by }
 VBar, { the Boolean constants }
 LineCross, { SingleLine and DoubleLine }
 TDown, { defined above. }
 TUp,
 TRight,
 TLeft : LineChars
 END;

CONST
 BoxChars : BoxRec =
 (ULCorner : (#218,#201); { Z I }
 URCorner : (#191,#187); { ? ; }
 LLCorner : (#192,#200); { @ H }
 LRcorner : (#217,#188); { Y < }
 HBar : (#196,#205); { D M }
 VBar : (#179,#186); { D : }
 LineCross : (#197,#206); { E N }
 TDown : (#194,#203); { B K }
 TUp : (#193,#202); { A J }
 TRight : (#195,#185); { C 9 }
 TLeft : (#180,#204)); { 4 L }


VAR
 HBars : BarStrings; { Horizontally oriented bars }
 VBars : BarStrings; { Vertically oriented bars }



PROCEDURE MakeBox(Target : ScreenPtr;
 X,Y,Width,Height : Integer;
 IsSingleLine : Boolean);


IMPLEMENTATION


PROCEDURE MakeBox(Target : ScreenPtr;
 X,Y,Width,Height : Integer;
 IsSingleLine : Boolean);

BEGIN
 GotoXY(Target,X,Y);
 WITH BoxChars DO
 BEGIN
 { Display the top line: }
 WriteTo(Target,ULCorner[IsSingleLine]+
 Copy(HBars[IsSingleLine],1,Width-2)+
 URCorner[IsSingleLine]);
 { Display the left side: }
 GotoXY(Target,X,Y+1);
 WriteDownTo(Target,Copy(VBars[IsSingleLine],1,Height-2));
 { Display the right side: }
 GotoXY(Target,X+Width-1,Y+1);
 WriteDownTo(Target,Copy(VBars[IsSingleLine],1,Height-2));
 { Display the bottom line: }
 GotoXY(Target,X,Y+Height-1);
 WriteTo(Target,LLCorner[IsSingleLine]+
 Copy(HBars[IsSingleLine],1,Width-2)+
 LRCorner[IsSingleLine]);
 END;
END;


{ VTOOLS Initialization Section }

BEGIN
{ This fills the predefined HBars/VBars variables with line characters: }
 FillChar(HBars[SingleLine],
 SizeOf(HBars[SingleLine]),
 BoxChars.HBar[SingleLine]);
 FillChar(HBars[DoubleLine],
 SizeOf(HBars[DoubleLine]),
 BoxChars.HBar[DoubleLine]);
 HBars[SingleLine,0] := Chr(255);
 HBars[DoubleLine,0] := Chr(255);

 FillChar(VBars[SingleLine],
 SizeOf(VBars[SingleLine]),
 BoxChars.VBar[SingleLine]);
 FillChar(VBars[DoubleLine],
 SizeOf(VBars[DoubleLine]),
 BoxChars.VBar[DoubleLine]);
 VBars[SingleLine,0] := Chr(255);
 VBars[DoubleLine,0] := Chr(255);
END.







[LISTING THREE]

{ V-Screen procedure that writes from X,Y downward: }

PROCEDURE WriteDownTo(Target : ScreenPtr; S : String);

VAR
 I,K : Integer;
 TY : Byte;
 ShiftedAttr : Word;

BEGIN
 { Put attribute in the high byte of a word: }
 ShiftedAttr := CurrentAttr SHL 8;
 WITH Target^ DO
 BEGIN
 TY := Y;
 K := 0;
 FOR I := 0 TO Length(S)-1 DO
 BEGIN
 IF Y+I > VHEIGHT THEN { If string goes past bottom of screen, }
 BEGIN { we wrap: }
 Inc(X); { Increment X value }
 Y := 1; TY := 1; { Reset Y and temp Y value to 1 }
 K := 0; { K is the line-offset counter }
 END;
 { Here we combine the character from the string and the }
 { current attribute via OR, and assign it to its location }
 { on the screen: }
 Word(ShowPtrs[Y+K]^[X]) := Word(S[I+1]) OR ShiftedAttr;
 Inc(TY); Inc(K);
 END;
 Y := TY; { Update Y value in descriptor record }
 END
END;






[LISTING FOUR]

{--------------------------------------------------------------}
{ BoxTest }
{ }
{ Character box draw demo program }
{ }
{ by Jeff Duntemann KI6RA }
{ Turbo Pascal V5.0 }
{ Last update 1/21/89 }
{--------------------------------------------------------------}

PROGRAM BoxTest;


USES Crt, { Standard Borland unit }
 Screens, { Given in DDJ; 4/89 }
 VTools; { Given in DDJ; 5/89 }


VAR
 WorkScreen : Screen;
 MyScreen : ScreenPtr;
 X,Y : Integer;
 Width,Height : Integer;
 Count : Integer;
 Ch : Char;
 Quit : Boolean;

BEGIN
 Randomize; { Seed the pseudorandom number generator }
 MyScreen := @WorkScreen; { Create a pointer to WorkScreen }
 InitScreen(MyScreen,True);
 ClrScreen(MyScreen,ClearAtom); { Clear the entire screen }
 Quit := False;

 REPEAT { Draw boxes until "Q" is pressed: }
 IF Keypressed THEN { If a keystroke is detected }
 BEGIN
 Ch := ReadKey; { Pick up the keystroke }
 IF Ord(Ch) = 0 THEN { See if it's an extended keystroke }
 BEGIN
 Ch := ReadKey; { If so, pick up scan code }
 CASE Ord(Ch) OF { and parse it }
 72 : Pan(MyScreen,Up,1); { Up arrow }
 80 : Pan(MyScreen,Down,1); { Down arrow }
 END { CASE }
 END
 ELSE { If it's an ordinary keystroke, test for quit: }
 IF Ch IN ['Q','q'] THEN Quit := True
 END;
 { Now we draw a random box. }
 { First get random X/Y position on the virtual screen: }
 REPEAT X := Random(VWIDTH-5) UNTIL X > 1;
 REPEAT Y := Random(VHEIGHT-5) UNTIL Y > 1;
 { Next get a random width and height to avoid wrapping: }
 REPEAT
 Width := Random(VWIDTH)
 UNTIL (Width > 1) AND ((X + Width) < VWIDTH);;
 REPEAT
 Height := Random(VHEIGHT)
 UNTIL (Height > 1) AND ((Y + Height) < VHEIGHT);;
 { Draw the box: }
 MakeBox(MyScreen,X,Y,Width,Height,DoubleLine); { and draw it! }
 UNTIL Quit
END.






































































May, 1989
OF INTEREST





The Lattice C++ compiler package now includes new documentation for the
standard Lattice C library. The separate volume describes the more than 300
Lattice C functions that are also available to C++.
Lattice C++ translates C++ source code into C programs that are then compiled
by the Lattice C compiler. The new volume of documentation gives users a
synopsis of each C function's use, a description of the function's purpose,
the return value provided by the function, and a cross reference. The
reference manual also provides application examples.
C++ programmers can call a Lattice C function for math, file I/O, memory
allocation, and system functions; write complete program modules in C when the
programs do not require C++ features; and build C++ objects utilizing C
functions.
Lattice's C++ package uses the AT&T translator kit. A driver, similar to the
LC driver in the Lattice AmigaDOS C compiler, is provided for translating,
compiling, and linking.
The price of Lattice C++ is $500. Registered users of Lattice C++ will receive
a copy of the new Lattice C Library manual at no charge. Lattice C++ requires
1.5-Mbyte memory and two floppy drives; a hard disk is recommended. Reader
Service No. 21.
Lattice Inc. 2500 S Highland Ave. Lombard, IL 60148 312-916-1600
Cortex Computing Corp.'s Enhance! adds several features to DOS 2.0 and
greater. Accessed from the command line and/or batch files, Enhance! coexists
with, not replaces, DOS and COMMAND.COM.
According to Cortex, Enhance! adds symbol/alias defining and processing, file
management (move, copy, remove, append, list, locate, and change files/
groups/attributes), retrieval and reissuing of previously entered commands,
the ability to issue several commands at one DOS prompt, command line editing,
and simplified DOS directory/ drive maneuvering. Enhance! is RAM resident and
can load and run in expanded memory.
Symbol (alias) processing allows the user to use symbols (words) at the DOS
command line for which Enhance! will substitute one or more other words.
Symbols may be defined to represent commands, parts of commands, several
commands, and other symbols. In some instances, symbol processing replaces
batch files with symbols and exact commands with abbreviations. For instance,
entering any of DIR through DIRECTORY could perform as though DIR were
entered.
With Enhance!'s file management, the user can perform the following from the
command line: list, copy, move, remove, protect, hide, and change the date and
time values of disk files. Another feature is the ability to specify and/or
exclude files with names and/ or extensions that begin with one or more sets
of letters, end with a certain letter or letters, or have a certain letter or
letters somewhere in between.
Retrieval and reissuing of previously entered commands allows the user to
recall and optionally modify and resend commands that have already been
entered. Several commands can be entered at once, and command line editing
supports lines longer than 80 characters.
This product sells for $79.95 and supports the IBM PC, XT, AT, PS/2, and
compatibles with at least 256K RAM and DOS 2.0 or greater. Reader Service No.
20.
Cortex Computing Corp. P.O. Box 116788 Carrollton, TX 75011 214-492-5124
Quadram has released Version 1.1 of its program development toolkit for the JT
Fax 9600 PC facsimile board. The toolkit contains instructions that enable
programmers to write to the JT Fax 9600's shared RAM interface, as well as
working programs for operations supported by the board.
The code necessary to transmit a facsimile resides on the board. In addition,
the board's facsimile capabilities can be accessed without Quadram code being
resident with the PC's main memory. The JT Fax 9600 can also be adapted to PC
bus systems running operating systems other than MS-DOS, such as Unix or
Xenix.
This product is a three-quarter size 9600 baud background board with an
on-board 80188 processor. It installs in an IBM PC, XT, AT, and most
compatibles, plus PS/2 Models 25 and 30. A unit retails for $795. The toolkit
is available free to registered JT Fax 9600 users by contacting Tom McElroy at
404-564-2353, ext. 2135.
Quadram produces three other PC facsimile products. JT Fax Internal, which
sells for $395, is a 4800 baud half card for the IBM PC, XT, AT, and PS/2
Models 25 and 30, as well as most compatibles. JT Fax Portable sells for $495
and is a 4800 baud external version, which plugs into a computer's serial
port. Selling for $595, JT Fax PS/Q is a 4800 baud full-length card that
installs inside the IBM PS/2 Models 50, 5OZ, 60, 70, 80, and most compatibles.
Reader Service No. 22.
Quadram One Quad Way Norcross, GA 30093-2919 404-923-6666
Formation, Version 1.1, by Aspen Scientific supports various application
interfaces, such as overlapping windows, different menu styles, and dialog
boxes. Formation in source code form is transported to several operating
systems. The source code is compatible with MS-DOS, OS/2, and Unix System V,
as well as Unix System look-a-likes, such as SCO Xenix System V/386.
Formation not only supports menu interfaces, such as drop-down menus, pop-up
menus, and Lotus-style menus, but it also provides dialog boxes for
user-to-application input. Formation dialog boxes support scrolling text
prompters, option buttons, check boxes, command buttons, and scrolling list
boxes. Additionally, Formation supports line borders, shadowing, and exploding
window refreshes.
Under MS-DOS and OS/2, this product supports VGA, EGA, CGA, and MDA displays
and C compilers. Formation under Unix supports terminals defined in the
terminfo database and exploits line-graphics capabilities where available.
Prices are $399 for the source code bundle supporting MS-DOS, OS/2, and Unix,
and $159 for the binary bundle supporting MS-DOS and OS/2. Formation requires
Aspen Scientific's Curses, Version 4.0, for MS-DOS and OS/2, and Unix Curses
(provided with Unix). Reader Service No. 23.
Aspen Scientific 10580 W 46th Ave. P.O. Box 72 Wheat Ridge, CO 80034-0072
303-423-8088
For writing memory-resident programs in Basic, MicroHelp has released Stay-Res
Plus, Version 3.0. One feature of Stay-Res Plus is the ability to specify up
to 21 "hot keys" that will pop up the program. Each hot key can be a regular
key press or a "shift only" hot key, such as Ctrl-Alt. The SysReq key can also
be used as a hot key.
The SRPOKER.COM program can be used to pop up the program from the DOS command
prompt or a batch file, or pop the program up after n ticks have occurred.
Other features include shelling other programs or batch files from resident
programs and removing the program from memory from within the program, or by
executing a .COM program from the DOS command line or a batch file.
The system's current environment variables can be accessed even if they have
been changed after the program has become resident. This includes PATH,
COMSPEC, and LIB. In addition, files can be left open between pop ups.
System requirements include an IBM PC, XT, AT, PS/2, 80386, or compatible;
Microsoft QuickBasic 2 - 4 or Bascom 6 with DOS 3.0 or later; QuickBasic 1 or
Bascom 1 - 5 with DOS 2.1 or later; an 80 column monitor; and a 360K floppy
disk.
Stay-Res Plus, Version 3.0, sells for $89; upgrades are $60. Reader Service
No. 24.
MicroHelp Inc. 4636 Huntridge Dr. Roswell, GA 30075 404-552-0565 800-922-3383
Human Intellect Systems has announced the upgrade of its first expert system
shell -- Instant-Expert -- for the Macintosh. Instant-Expert, which sells for
$69.95, is a software development tool used to created customized expert
systems.
User-entered logic or if/then rules drive the application to produce
information offered by the subject-area authority. Users can build a
stand-alone expert system based on their knowl dge of a particular field.
Release 2.0 features an inference engine that uses forward and backward
chaining, as well as a mixed strategy in which both forward and backward
reasoning are used. The product supports true, false, and unknowns, and allows
the user to ask "why?" and "how?" Instant-Expert also offers trace functions,
interactive deduction, optional search strategies (depth first or breadth
first), and uncertainty controls. The more advanced expert system shell
Instant-Expert Plus, offers advanced graphics and user-defined variables and
sells for $498.
Applications completed in Instant-Expert 2.0 are compatible with
Instant-Expert Plus. Registered users may also apply their purchase price
toward the purchase of Instant-Expert Plus. Registered Instant-Expert 1.5
owners can upgrade to Instant-Expert 2.0 for $25.
Both instant-Expert and Instant-Expert Plus run on the Apple Macintosh 512KE,
SE, and II computers. Instant-Expert Plus also runs on an IBM PC or compatible
with 640K. Reader Service No. 25.
Human Intellect Systems 1670 S Amphlett Blvd., Ste. 326 San Mateo, CA 94402
415-571-5939 800-522-5939
ParcPlace Systems will soon release the Smalltalk-80 object-oriented
development environment for DECstation 3100 computers. The DECstation 3100 is
a MIPS-based system using the Ultrix operating system. Smalltalk-80 features a
set of development and information access tools, an object-oriented
programming language, tested applications, and portability to a variety of
workstation and microcomputer environments.
With this product, users program by selecting code modules from a reusable
library and by using built-in applications and simulations. It also allows
programmers to move between writing code, debugging, and inspecting levels of
system operation. The environment can also link and access subroutines written
in other languages, such as C. Smalltalk-80 includes a graphical user
interface based on overlapping windows, pop-up menus, multiple fonts, and
mouse-oriented control.
Smalltalk-80 is compatible with the X-Window environment and is portable to
Unix, Mac OS, and MS-DOS operating systems using Motorola 680X0, Intel 80386,
Sun SPARC, or MIPS processors. Software support includes a customer hotline,
free upgrades to new releases, an electronic bulletin board, ParcPlace
newsletters, Smalltalk-80 books on advanced programming and user interface,
and in-house and on-site training covering introductory through advance level
programming.
This product sells for $3,995. Reader Service No. 26.
ParcPlace Systems 2400 Geng Rd. Palo Alto, CA 94303 415-859-1000


Errata


In the "Of Interest" column (April 1989) the price of Glockenspiel C++ 1.2 by
Imagesoft was incorrectly stated on page 146. The correct price of the product
is $495.








May, 1989
SWAINE'S FLAMES


Satanic Verses




Michael Swaine


Could Intel policy at last be forcing An innovation in second sourcing?
Spencer Katt's suggestion in January 16's PC Week that Chips & Technologies
may "second source" the Intel 386 by cloning it in RISC technology reminds me
of something Hal Hardenbergh wrote over seven years ago. In issue #4 of his
DTACK Grounded newsletter back in November 1981, Hardenbergh told a hilarious
story of Intel's search for a domestic second source (dss) for the 8086 to
satisfy its new customer, IBM. After negotiations with Jerry Sanders' AMD blew
up, Intel lined up Mostek as dss, but declined to give Mostek the mask set,
expecting Mostek engineers to reconstruct the chip from the logic design data.
This arrangement did not result in any 8086 chips being produced by Mostek,
and Mostek eventually defected to Motorola. At this point, under pressure from
its biggest customer to line up a dss, Intel looked around and found only one
candidate not pursuing its own device or working with Motorola, swallowed
hard, and approached that candidate. What Intel swallowed was apparently crow,
served at Jerry Sanders' table.
Hardenbergh opened, "Intel has never been philosophically attuned to the idea
of a second source."
Hardenbergh has generally been insightful, and all of three years ago pointed
out the overuse of the word "paradigm" in technical writing
Perils emerge in the new bus master: You're risking a crash if you drive much
faster.
Steve Gibson sometimes leaves his InfoWorld readers in the dust as he hot-rods
over the techie terrain. He took some abuse recently for complaining that
PostScript is too slow, which it is, but which is not interesting news to
people who believe that PostScript is the only game in town. Subsequently, he
pointed out a big problem with bus master cards for the original AT bus. Seems
that the cards, which take over control of the bus to speed SCSI hard disk
access, work OK with a 286 or a simple 386 system, but send the system south
fast when invoked while running in protected mode with memory mapping enabled.
He discussed it in the January 16 InfoWorld.
People and firms, it's my job to report, Are judged by the companies with
which they consort.
George Bush says he wants to be the education president. Well, American
education could use some attention, and he is the most intellectual president
we've had in eight years. The IEEE wants to help, and has offered George some
advice on education and scientific policy. The IEEE U.S. Technology Policy
Conference, Policy Imperatives for Commercialization of U.S. Technology,
presented the IEEE's agenda for engineering education, technical innovation,
international competitiveness, and effective utilization of science and
technology. Interestingly, from what I could tell (I wasn't there), the panel
on technological innovation appeared to be a pitch for federal support for
consortiums. Sure, I thought that was where innovation took place, didn't you?
The problem with which John Sculley must grapple Is, how many ways can you
slice up an apple?
How many different ways can a company be structured? Since Jobs left, Apple
has been organized and reorganized along -- at least -- product, business
function, and now geopolitical lines. My bet for the next organizational
dimension is product footprint. Seriously.
There is little of which M. Gasse is fonder Than comparing himself to the
chairman of Honda.
Jean-Louis Gasse wasn't at his charming best when he spoke at Macworld Expo in
January. His first mistake was in choosing Harry "Absent-Minded Professor"
Anderson to introduce him. Anderson's hilarious speech would have been tough
for anyone to follow. Then he downplayed the importance of voice recognition,
a technology crucial to Apple's public daydream, Knowledge Navigator. Finally,
he came off cold in responding to questioners who wanted to know why the
Education Computer Company couldn't sell a home-priced Mac that people could
buy for the kids. Gasse compared Apple to Honda, saying that Apple wasn't in
the low-priced computer business. That would go down easier if we didn't know
what would happen to a low-priced computer company that tried to fill the gap
with a Mac clone.





































June, 1989
June, 1989
EDITORIAL


You Make The Call




Jonathan Erickson


Every time we get close to wrapping up another month's worth of DDJ, I start
thinking of last February's issue with the multitasking juggler on the cover.
What with last minute changes in the number of pages we have available,
articles coming in long, articles coming in short, articles coming in late (or
not at all), and editors in and out of the office at all hours, it sometimes
amazes me that we get the magazine out at all. Add to that the desire for
variety and balance among the articles (we don't want to have all C or all DOS
articles) and we sometimes feel like we're juggling while walking on a
tightrope.
As a case in point, this month we planned on running the follow-up to our
February feature on the Rhealstone, our real-time benchmarking proposal.
However, we underestimated the volume of response (and thanks to all of you
who sent in comments) and we simply weren't able to evaluate and incorporate
the recommendations in time for this issue. Our follow-up will run later this
year. Likewise, in May we intended on running Martin Tracy's Zen Forth System,
but we flat ran out of room. It too will run later this year.
As for operating systems, we were recently asked by a leading application
developer what our cut -- or at least the cut of DDJ readers -- is on OS/2.
Their reason for asking, they said, was that they have a lot of OS/2 products
that (surprise, surprise) aren't selling too well; and a recent cross country
trip hadn't shored up their spirits as major customers said that more DOS --
not OS/2 -- applications were what they wanted.
Our experience is that we're beginning to see a rise in the OS/2 noise level,
although the major interest on the part of readers continues to be on DOS and,
to a lesser degree, Unix. The difference in our experience and that of the
company mentioned is that the noise we hear is coming from developers -- not
end users who seem to be reacting with silence. We get calls pretty much every
day from developers who are looking for OS/2 information. More significantly,
we're getting more and more article proposals from programmers who've been
working with OS/2. (One example of the latter is a nice little article by Nico
Mak, who wrote April's popular SWAP article. After becoming frustrated with
OS/2's cryptic error messages, Nico wrote a utility that provides a more
detailed error report. Unfortunately, we again ran into one of those
situations where we just didn't have room to run the utility this month, but
you can expect to see it before long.)
Another thing we're hearing from developers who've been working with OS/2 is
that once they've started programming under OS/2, they don't want to go back
to DOS. That's the opinion of Mansour Safai, the chief architect of Logitech's
MultiScope OS/2 debugger. (And if you get a chance to see this debugger, don't
pass it up). I suspect that Nico Mak concurs even though he hasn't explicitly
said so. What he has said is that he rarely, if ever, bothers to boot DOS
anymore, OS/2 is his primary working environment.
What this leads up to is a growing demand for OS/2 products, but not the
end-user applications people expected. Instead, the current market seems to be
built around sophisticated development tools (like new debuggers).
The question developers must grapple with is twofold: should they spin their
wheels working on OS/2 tools for a market that's still evolving, and how can
they get those tools into programmer's hands. Vaughn Vernon's idea sounds good
to me. Vaughn, who heads Aspen Scientific, thinks developers ought to give
OS/2 tools away by bundling them with existing DOS packages. His theory is
that developers can back into the OS/2 market with character-based tool kits,
buying time to develop "real" OS/2 programs that have Presentation Manager
support. Vaughn has been doing this for a while with his tools and I'm
interested in keeping track of whether or not it works. What do you think?









































June, 1989
LETTERS







C++Kernel Comments


Dear DDJ,
I'm new to C++ and to Zortech's implementation, so I really enjoyed Tom
Green's "A C++ Multitasking Kernel" in the February 1989 issue. There is a
simple error in Listing Two line 58. The new task, workspace constructor
method should end with a closing brace instead of the opening brace as shown
in the listing.
After entering all the listings, MASM was nowhere to be found. Since the
Zortech compiler (ZTC) calls the assembler program automatically (and assumes
MASM), I thought that I would probably not be able to produce an .EXE program.
By using Borland's Turbo Assembler and temporarily renaming it, I was able to
produce an .EXE program that is 14,685 bytes long (small model).
This was an interesting experience in piercing together a running program.
Keep up the good work.
James H. Wilbanks
Dacula, Georgia
Tom responds: I am glad you enjoyed my article. It was a lot of fun to write.
You are correct; there is an error in the code. When I got the first proofs
back from DDJ, there were a couple of lines of code missing. The last line of
the task class constructor was one of the missing lines. I guess we got
crossed up on what the missing line should be. As you also found out,
Borland's TASM works just fine with my code and with the Zortech C++ compiler,
if you rename it so that the ZTC shell can find it. In taskdemo.cpp, I say
that you must use MASM 5.xx. I use some of the advanced features introduced in
MASM 5.0 (such as .MODEL, .DATA, etc.). TASM also supports all of these
features. I use both TASM and MASM with my Zortech C++ compiler. Here are two
more errors in the code: In Listing Two, line 49 reads: task( ); //
destructor, but should read: ~task( ); // destructor. In Listing Two, line 78
reads: task::task(void), but should read: task::~task(void).


A Little More on Little and Mohr


Dear DDJ,
Your magazine is informative as usual. In reading through the letters to the
editor (February 1989), I couldn't help but notice where Frank Little of
Clydach, Swansea brought up the topic of paper transfer vs. computer transfer.
I also couldn't help but notice that he seems to lack a basic understanding of
the human. I'm not putting down, just amplifying the missed point. Frank, did
you ever see any student open only one book at a time when really digging in
to study?
Currently, one of the major differences between humans and computers is that
humans rely extensively on pattern recognition for even the most common of
day-to-day survival and computers (percentage wise) have yet to recognize any
patterns beyond simple binary "text" (instructions). Even on a CAD/CAM system,
the computer cannot point out the file cabinet on the floor plan. And you and
I each received and processed numerous patterns in order to navigate our way
to work today (and in "real time" no less).
To further belabor the point, even children that were raised by wolves and
such, who speak "no language," recognize weeds, rabbits, and rabbits in the
weeds. Try getting your computer to pick out your child or sweetheart in the
picture there. The "raised wild" child can.
More: I read Jay van Santen's request for help on large arrays, and I do not
understand Tonkin's reply. Jay made a straight request, and Tonkin simply went
off in his own world. I too now want to know: How large is the data segment?
More: Mr. Douglas Mohr is quite correct; Intel propagates ineptitude at an
alarming rate, and at the chip-level no less. Zilog has superior design and
implementation at most all levels of IC CPUs. I also agree that "ol' CP/M" is
still a better operating system. Data loss was far less than in my experiences
with MS-DOS, and data recovery was almost always possible, even in multiuser
environments. Data recovery under MS-DOS is almost nonexistent. The file
following (linked list) method of data storing is full of trouble. Always has
been, always will be. If you break the chain (lose the pointers), you get to
fish for your "anchor" (data) at the bottom of the bay with all the other
discarded anchors (released data blocks). Even under the best of conditions, I
find data recovery attempts impractical fewer than 50 percent of the time
under MS-DOS.
Who knows, Mr. Mohr, maybe someday MS-DOS will actually be documented so users
can use it. I'd even settle for consistent reactions to a given program. Don't
hold your breath though. I'm not.
Steven L. Turner
Sacramento, Calif.
My God, what a bunch of cry babies in your February letters page! Since there
were only three letters, may I respond to each of them?
To Mr. Frank Little, about your sentence "(Programmers') salaries are sky high
..." I've read this often, and my response is always "balderdash!" (Frequently
expressed in a pithier, shorter word.) I don't think I'm overpaid. My clients
don't think I'm overpaid. If they did, they would hire someone else. (On the
other hand, my wife thinks I'm underpaid.) I'll gladly charge less once
someone decides that my gasoline, housing, utilities, and taxes will be
comparably reduced.
To Mr. van Santen about QuickBasic and Bruce Tonkin in his reply: First, Mr.
van Santen obviously meant to say that QB uses the Compact memory model, not
the medium model. The 64K restriction is, as everyone knows, an artifact of
the Intel architecture. Every language implemented for the 80x86 family has
this problem, and most offer at least one way around it.
To Mr. Douglas Mohr (whose whinning prompted this letter): First, I'm sorry
you were "forced" to use a PC at work, and I hope you will soon be allowed to
work on a PDP 11 again. In the meantime, I'd like to douse your flames.
Flame Zero: Yes there is abysmal PC software, and maybe Aztec C is one
example. There are terrible restaurants, terrible cab drivers, and terrible
cities as well; very few of them come with free upgrades.
Flame One: In the Intel architecture, programmers must pick the memory model
for the same reason that they must pick the variable types, file structures,
and sorting algorithm. "One size fits all" might work for muu muus, but it
makes lousy code. I also prefer a linear address space, such as the 680x0
architecture provides.
Flame Two: Yes, you are being naive. In the case of the C Users Group, you are
getting unsupported source code for whatever machine and compiler someone
chooses to upload. If you use CP/M, and don't have BDS-C, then buy it! If you
were looking for executables, write to PC-SIG or FOG.
Flame Three: If you truly want a TECO-like editor for the CP/M, look at VEdit
(from CompuView) or PMate (from Phoenix, I think). Why not write your own
TECO, starting with source code from Edward Ream's public domain ED editor, or
David Conroy's micro-Emacs?
Flame Four: I also prefer Unix, but that's not where the market is. NeXT is
the next step up from CP/M? Funny, I thought it was a step up from Macintosh.
Mr. Mohr, you have the right to bitch as much as you want. DDJ editors, why
must you inflict this bitching on us readers?
Arthur Metz
Los Angeles, Calif.
I read with considerable interest Douglas Mohr's letter in the February 1989
issue of DDJ. I couldn't agree more! I am a programmer by profession; I write
and test software for my company's line of IBM compatible, industrial computer
systems. 8086 assembly code and MS-DOS bring home the bacon, but I have little
love for either. So I'll add a few more flames to the fire.
Continuing where Mohr left off. Flame Four: Why did Intel create, and
Microsoft propagate, a set of assembler pseudo-ops that are totally
unnecessary? MASM and its clones require high-level language constructs, such
as "PROCedures." What does that have to do with assembly language?
MASM also insists upon knowing the contents of segment registers. Why? Having
programmed just about every microprocessor from the 4004 forward, I have never
seen such a disastrous assembler. It is obvious that MASM is an assembler for
the C programmer, what with structures and records and such, not an assembler
for the experienced assembly language programmer. I use the version that runs
under CP/M-80, although an MS-DOS version is available. It isn't perfect, but
it allows programmers to write in assembly code, not in a pseudohigh-level
language.
Flame Five: Mohr is right in his statements regarding MS-DOS. What do you
expect from an illegitimate cross between CP/M and TRS-DOS? Except now it's a
three-way incestuous cross with Unix as well. The original MS-DOS combined the
worst features of CP/M and TRS-DOS, with the best features of neither. If you
want that, you're much better off the CP/M; it's a much friendlier development
environment, even for alien processors. Now they're thrown in some Unix
features. If you want that, you might as well go with pure Unix and get all
the benefits!
Flame Six: Now that we have cheap AT-class machines with ten times the memory,
we can write all our code in inefficient high-level languages, hog all kinds
of memory, and get about the same level of performance as we got ten years ago
from a 4MHz Z-80 running efficient machine code. The original 4.77MHz PX was a
joke. My 2-MHz IMSAI will run rings around it.
Flame Seven: This is something that really doesn't matter, but I've never seen
anyone say it before: MS-DOS machines in general, and the original PC/XT in
particular, have to be the ugliest computers in existence. Just before the PC
was introduced in 1981, we were starting to get some really nice, integrated
systems like the TRS-80 model III/IV, the NorthStar Advantage, the Superbrain
line, and so forth. Then along comes IBM with a system that instantly reminded
me of a bad cross between a model I TRS-80 and an Apple II+. Suddenly, we were
thrown back to 1977 with three boxes connected together by ten million miles
of cables. And the three boxes didn't even go together. At least the old model
I TRS-80's parts looked like they belonged together! I suppose I have to
reluctantly agree with the thought that this allows you to select your own
monitor and keyboard, which brings us to...
Flame Eight: Keyboards. There was a huge stink about keyboards when the PC was
introduced, and it has never really died down. With the millions of
replacement keyboards available, however, I have found only one that is
acceptable, the Jameco that sold for $30 a couple of years ago, a surplus
Televideo PC keyboard. IBM has totally ruined the world of keyboards. Almost
all the keyboards made prior to the PC were similar. Look at a TRS-80, any
older terminal, and especially an IBM Selectric typewriter. I'm not talking
about layout; I have typed on everything from old manual typewriters to
teletype machines, and can adapt to any layout. I'm talking about the size and
shape of the key caps, and the slope. I am a very fast touch typewise, and I
just flat cannot type on those flat IBM-style keyboards. I expect a discrete
step between the rows of keys and these keyboards do not have it. The key caps
are a different size, and many of the keys, such as Control, Shift, and Return
are "stepped" for whatever insane reason. My kingdom for a decent keyboard!
The above mentioned Televideo keyboard is pretty good; it is made in the old
style, and I can type on it. It's in the original PC layout, which, yes, is
dumb, but I can adapt to it if the keyboard is good. And the IBM keyboard
clicks...
End of Flames: Overall, the IBM world has brought little to us in a new
technology. So I'll just sit here in front of my IMSAI until a Mac II becomes
affordable, or even the NeXT. Absolutely a real first step up from CP/M.
Mark D. Pickerill
Salinas, Calif.
More Details.



Errata



Somewhere twixt the programmer and the printed page, our production equipment
developed the habit of eating tildes (~) when they appeared in program
listings over the last couple of months. We've solved the problem and you can
be assured that it won't recur. (Neither the source disk nor the listings on
CompuServe were involved.)
Two articles affected were May's "TAWK: A Simple AWK Interpreter in C++" by
Bruce Eckel and April's "C Programming Column" by Al Stevens. Please note the
corrections below. We apologize to readers and authors alike.

TAWK: A SIMPLE AWK INTERPRETER IN C++

 Listing 1, line 17: ~field( );
 Listing 2, line 16: field::~field( ) {
 Listing 3, line 17: ~csascii( ); // destructor
 Listing 4, line 38: csascii::~csascii( ) {
 Listing 6, line 32: ~token( );
 line 50: ~parse_array( );
 Listing 7, line 25: token::~token( ) { // delete heap if allocated:
 line 119: case'~': // the "else" part of an "if" statement
 line 135: "'(','<','?',':','~','.','p','m','c' or '@'");
 line 207: parse_array::~parse_array( ) {
 Listing 10, line 6: "@(0)","@(1)","@(2)","@(3)","@(4)@?4@: @~@.@(5)"
 Listing 11, line 10: @~@.@~@?4@:@(4)
 line 11: @~

C PROGRAMMING COLUMN

 Listing 4, line 75: writecomm (~bno); /* 1s complement */
 line 190: if ((nbn & 255) != (~blk & 255)) {



































June, 1989
INTERPROCESS COMMUNICATIONS IN OS/2


Ray Duncan


Ray Duncan is a software developer for Laboratory Microsystems. You can reach
him at 12555 W. Jefferson Blvd, Ste. 202, Los Angeles, CA 90066.


Regardless of whatever other nits you might pick with OS/2, you could never
complain that is array of interprocess communications (IPC) facilities is too
austere. In fact, OS/2's IPC support could aptly be termed an "embarrassment
of riches" (the title of a recent book, by the way, which has nothing
whatsoever to do with protected mode programming). Browsing through the
reference manual, one gets the distinct impression that the IBM/Microsoft IPC
Design Subcommittee couldn't agree on anything, so they threw in everything!
OS/2 offers all of the following classic IPC mechanisms: semaphores, pipes,
shared memory, queues, and signals. When the LAN Manager is running, OS/2 also
supports an IPC mechanism called mailslots (which will not be mentioned
further in this article).
Regrettably, while the IBM and Microsoft manuals are passably complete on the
"how to" for each individual IPC method, they are remarkably stingy with the
"which to," "why to," and "when to." In this article, I'll try to provide you
with a somewhat more cosmic overview of OS/2 IPC, including some ballpark
comparisons of capability, performance, and throughput.


Preliminary IPC Concepts


Most of the IPC mechanisms listed above rely on named, global objects or data
structures that are controlled and maintained by the operating system. The
names are said to be in the "file system name space" that is, they have the
general format of filenames with the same elements and delimiters, and are
subject to the same constraints on length and valid characters. The names of
IPC objects are distinguished from the names of true files by a reserved path
(such as \ SHAREMEM \, \ PIPE \, \ QUEUES \).
The resemblance between IPC objects and files does not end with their naming.
To gain access to most types of IPC objects, a program must first "open" or
"create" the object in a manner analogous to opening or creating a file. OS/2
then returns a token (a selector or an arbitrary "handle"), which the process
uses to manipulate the IPC object: reading, writing, querying the number of
waiting messages, and so on.
In order to understand OS/2 IPC, it's also crucial that you grasp two
essential OS/2 terms: processes and threads. In its simplest form, a process
is conceptually equivalent to a program loaded for execution under MS-DOS.
OS/2 creates a process by allocating memory to hold its code, data, and stack,
and by initializing the memory from the contents of a program (.EXE) file.
Once it is running, a process can obtain additional resources --such as memory
and access to files -- with appropriate system function calls.
The OS/2 module which oversees multitasking, however -- the scheduler -- cares
nothing for processes; it deals with entities called threads. A thread
consists of a set of register contents, a stack, an execution point, a
priority, and a state: executing, ready to execute, or waiting for some event
("blocking"). Each process starts life with a primary thread, whose execution
begins at the entry point designated in the .EXE file header, but that thread
can start other threads within the same process all of which execute
asynchronously and share ownership of the process's resources.
Here's why the distinction between processes and threads is important when
discussing IPC. When a process opens or creates a semaphore, pipe, queue, or
shared memory segment, OS/2 returns a handle that can be used by any thread
within that process. But when a thread issues an OS/2 function call that
blocks on (waits for) an IPC event, such as the clearing of a semaphore or the
availability of a queue message -- the other threads in the same process
continue to run unhindered.


Semaphores


Semaphores are simple IPC objects with two states. These two states can, in
turn, be viewed in two different ways, depending on how a semaphore is being
used. When a semaphore is being used for signalling between threads or
processes, it is said to be either "set" or "clear." Typically, one thread
sets the semaphore, and then clears it upon the occurrence of some event;
other threads, which wish to be notified of the same event, "block on" the
semaphore by issuing a "semaphore wait" function call that does not complete
until either the semaphore is cleared or a designated timeout interval has
elapsed.
When a semaphore is being used for mutual exclusion, it is said to be either
"owned" or "available." In this model, the semaphore symbolizes a resource
(such as a file or a data structure) that would be corrupted if it was
manipulated by more than one thread or process at a time. To prevent such
damage, threads or processes cooperate by refraining from accessing the
resource unless they have acquired ownership of the corresponding semaphore
with an OS/2 function call.
Aside from the two ways in which they may be used, OS/2 semaphores come in
three flavors: system semaphores, RAM semaphores, and Fast-Safe RAM
semaphores. System semaphores are named, global objects, which reside outside
every process's memory space and are completely under the control of the
operating system. They must be "opened" or "created" with a system call before
they can be used. System semaphores support "counting;" that is, a process can
make "nested" requests for ownership of the semaphore, and the semaphore will
not become available again until a corresponding number of "release" calls
have been issued. OS/2 also provides cleanup support for system semaphores; if
a process dies owning a semaphore that another process is waiting for, that
other process will be notified with a unique error code.
RAM semaphores, on the other hand, reside in memory controlled by a process.
They consist of any arbitrary, but properly initialized doubleword of memory
in the application's address space, and the "handle" for a RAM semaphore is
just its address (selector and offset) --no "open" or "create" operation is
required. The number of RAM semaphores that a process may use is limited only
by the amount of virtual memory it can allocate. RAM semaphores are used to
communicate between threads, but since memory segments can be shared, they can
also be used to communicate between processes. In the latter case, OS/2 does
not provide any assistance if a process dies owning a RAM semaphore and
another process is waiting for the same semaphore.
The so-called Fast-Safe RAM semaphores, which were added to OS/2 in Version
1.1, combine characteristics of both system semaphores and RAM semaphores.
They are implemented as 14-byte structures in a process's own memory space;
so, like plain vanilla RAM semaphores, the number of Fast-Safe RAM semaphores
that a process can use is huge. Like system semaphores, Fast-Safe RAM
semaphores support "counting" and are also endowed with a certain amount of
clean-up assistance by the operating system. Unfortunately, Fast-Safe RAM
semaphores must be manipulated with special-purpose function calls --the
general purpose set, request, wait, and clear functions employed for both
system and RAM semaphores cannot be used --and they support only the
"owned/available" model for mutual exclusion.


Pipes


Pipes, which were first popularized under Unix, are basically conduits for
byte streams. In OS/2, processes refer to pipes with handles that are
allocated out of the same sequence as file handles, and they read and write
pipes with the same function calls as are used for files. The transfer of
information through a pipe is much faster than it would be through an
intermediary file, however, because the ring buffer that implements a pipe is
always kept resident in memory.
OS/2, Version 1.1, supports two different species of pipes: anonymous pipes
and named pipes. When a process creates an anonymous pipe, no global name is
involved; the system merely returns read and write handles. These handles can
be inherited by child processes, which is what enables anonymous pipes to be
used for IPC. However, because a child has no way to predict what handle
should be used for what, a common practice is for a parent process to redirect
the child's standard input and standard output handles to pipe handles, so
that the child unknowingly communicates with the parent rather than with the
keyboard and display. The corollary handicap of anonymous pipes is that
processes which are not direct descendants of a pipe's creator cannot inherit
handles for the pipe and thus have no way to access it.
Named pipes, on the other hand, are global objects, and any process --related
or unrelated to the pipe's creator --can open the pipe by name to obtain
handles for reading and writing. Another important feature of named pipes is
that they can be used in either byte stream mode or message mode. In byte
stream mode, a named pipe behaves like an anonymous pipe --the exact number of
bytes requested is always read or written. In message mode, a named pipe acts
more like a first-in-first-out (FIFO) queue: the length of each message
written into the pipe is encoded in the pipe, and a read operation returns at
most one message at a time regardless of the number of bytes requested. Last
but not least, named pipes can be used to communicate between processes
running on two different nodes of a network, simply by prefixing the name of
the pipe with the name of the target machine.


Shared Memory


Shared memory segments are potentially the most efficient of all OS/2's IPC
mechanisms. If two or more processes have addressability for the same segment,
they can theoretically pass data back and forth at speeds limited only by the
CPU's ability to copy bytes from one place to another, with no need for
additional calls to the operating system. Of course, the threads and processes
using a shared segment are responsible for synchronizing any changes to the
segment's content, and this synchronization is often most convenient to
accomplish with semaphores (requiring system calls after all).
OS/2 supports two distinct methods by which processes can share memory:
creation of named segments and giving and getting of selectors for anonymous
segments. Each method offers different advantages for security and speed of
access. Named segments are restricted to a maximum size of 64K bytes; once a
named segment is created, any process which knows the name of the segment can
"open" it to obtain a selector with which it can read or write the segment.
The segment persists until all the processes, which have valid selectors for
the segment have either released the selector or have terminated.
Anonymous segments, on the other hand, can be any size at all (huge segments,
consisting of logically contiguous 64-bytes segments, can be as large as
available virtual memory), but sharing is more difficult to arrange. The
selectors for such shared segments must be explicitly made addressable for
each process that needs them, and passed between the processes by some other
means of IPC. One technique, called segment giving, requires the process that
created a segment to request an additional selector for use by a specific
other process then send the selector to that process.
The other technique, segment getting, requires the creating process to pass
its own selector for the segment to the other process by some IPC mechanism.
The other process then gains addressability to the shared segment by issuing a
function call that makes the selector valid. Segment getting allows far
pointers to be passed around freely, but it is correspondingly less secure
than the use of giveable selectors.


Queues


Queues are the most powerful IPC mechanism in OS/2, and inevitably are also
the most complex to use. Queues are named global objects, and any process
which knows a queue's name can "open" it and write records into it, although
only the process which created the queue can read messages from it or destroy
it.
In essence, an OS/2 queue is an ordered list of shared memory segments; the
operating system maintains and searches the list on behalf of the
communicating processes. Data in the queue is not copied from place to place,
instead, pointers are passed from the queue writer to the queue reader (the
operating system also provides the queue reader with supplementary information
such as the process ID of the queue writer). The items in a queue can be
ordered in several different ways: first-in-first-out (FIFO),
last-in-first-out (LIFO), or by a priority in the range 0 through 15.
Moreover, the queue reader has the freedom to inspect and remove queue
messages in any arbitrary order, if it needs to.
Writing a message into a queue is a relatively complicated process. First, the
queue writer must allocate a "giveable" memory segment and build the queue
message in it. Next, the writer must obtain a giveable selector for the
segment that is valid for the queue reader. Finally, the writer must request
the queue write, passing the giveaway selector, and release its own original
selector for the segment. Thus, a minimum of four system calls are typically
required at the queue writer's end for each queue transaction. At the queue
reader's end, luckily, only two system calls are usually required: one to read
the message (obtain a pointer to the message and its length), and one to
release the selector for the segment containing the message after it has been
processed.



Signals


Signals, which (like pipes) have their conceptual origin in Unix, are
analogous to a hardware interrupt. They are unique among OS/2's IPC mechanisms
in that the time of a signal's arrival is not completely under the control of
the receiving process. OS/2 supports two classes of signals. The first class,
which consists of signals generated by the operating system, includes the
following:
SIGINTR a Ctrl-C was detected
SIGBREAK a Ctrl-Break was detected
SIGTERM the process is being terminated
SIGBROKENPIPE a pipe read or write failed
Signals in the second class are explicitly sent by one process to another.
These are known as event flags, and three types are available (each of which
may have a distinct handler): Flags A, B, and C. Event flag signals may be
accompanied by an arbitrary word (16 bits) of data.
For each signal type, a process may either register its own handler, instruct
the system to ignore the signal, or allow the system's default handler to take
its usual action. If a particular signal occurs and the process has previously
indicated its desire to service that signal type, the primary thread of the
process is transferred forcibly to the routine designated as the signal
handler. When the handler completes its processing control is restored to the
point of interruption.
The system's default handling of the different signal types varies. SIGTERM
terminates the target process. SIGINTR and SIGBREAK are fielded by the
ancestor process which has registered an appropriate handler; if this ancestor
is CMD.EXE or the Presentation Manager shell, SIGBREAK and SIGINTR are
translated to SIGTERM. SIGBROKENPIPE and the Event Flag signals, on the other
hand, are by default discarded.


Assessing IPC Throughput


From the preceding discussion and the summary in Table 1, it is clear that the
characteristics of OS/2's various IPC facilities vary drastically. Yet, at
least several of them can be made to do essentially the same job. How does one
assess their relative performance and suitability for a specific application?
The OS/2 documentation gives little guidance here, except to note in passing
that RAM semaphores are faster than system semaphores, semaphores in general
are faster than everything else, and pipes are faster than queues.
Table 1: Summary of the various OS/2 interprocess communication facilities

 IPC Global Name Resident/ Maximum
 Mechanism Form Swappable Data Held

------------------------------------------------------------------------------

 RAM Semaphore not Swappable set/clear or
 applicable owned/available

 Fast-Safe not Swappable owned/available
 RAM Semaphore applicable

 System \SEM\name Resident set/clear or
 Semaphore owned/available

 Anonymous not Resident 64 Kbyte
 Pipe applicable

 Named Pipe \PIPE\name Resident 64 Kbyte

 Anonymous not Swappable limited only by
 Shared Memory applicable virtual memory

 Named Shared \SHAREMEM\name Swappable 64 Kbyte per
 Memory named segment

 Queue \QUEUES\name Swappable limited only by
 virtual memory

 Signal not not 16 bits passed
 (Event Flag) applicable applicable with signal

In order to try and get a feel for these issues, I carried out some simple
timings on the most commonly used IPC methods, which I will describe shortly.
The timings were obtained on a IBM PS/2 Model 80 at 16MHz with 4 Mbytes of
RAM, running under IBM's OS/2 Standard Edition, Version 1.1. The relevant
CONFIG.SYS parameters were:
 BUFFERS=30
 BREAK=OFF
 DISKCACHE=64
 IOPL=YES
 MAXWAIT=3
 MEMMAN=SWAP,MOVE
 PROTECTONLY=NO
 RMSIZE=640

 THREADS=128
The only significant processes that were running during the timings were the
Presentation Manager shell and two instances of LMI UR/FORTH in PM windows. I
judged the system to be lightly loaded, a conclusion supported by my
observation that no swapping occurred during the timings (as evidenced by the
fixed disk light) and by the fact that the DosMemAvail function returned the
size of the largest block of available physical memory as 1,367,520 bytes.
The programs used to obtain the timings were written in LMI UR/FORTH, my own
company's protected mode Forth interpreter/compiler for OS/2. Forth is an
ideal language for this sort of system probing because it is fast enough for
real-time work, yet it affords interactive, direct access to all operating
system functions.


Semaphore Performance


Let's look first at the semaphore family. To appraise the relative speeds of
system, RAM, and Fast-Safe RAM semaphores for both the "signalling" and
"mutual exclusion" models, I timed 100,000 request release cycles and
set/clear cycles for each semaphore type (Table 2). The tare time for the loop
was determined by substituting a dummy function for each system call that
simply returned a success status; this time was then subtracted from the total
before calculating the cycles per second.
Table 2: Performance comparisons of various OS/2 semaphore types. Timings are
based on 100,000 complete cycles (request then release, or set then clear) on
the same semaphore.

 Semaphore Type Request/Release Set/Clear
 Cycles per Second Cycles per Second

-------------------------------------------------------------------------------

 RAM Semaphore 16,507 17,156

 Fast-Safe 17,066 not applicable
 RAM Semaphore

 System Semaphore 7,464 7,532

As you can see from the Table, the difference between the performance of
system and RAM semaphores is not nearly as great as you might expect from
reading the OS/2 technical manuals. Your selection of system, RAM, or
Fast-Safe RAM semaphores should really be made on other grounds. I have
already mentioned some of the important differences (counting and cleanup),
but there are additional subtle differences that might prove important in a
real-life project.
First, the apparent performance advantage of RAM semaphores in a
lightly-loaded system cannot be generalized to a heavily-loaded system. System
semaphores are implemented in fixed, non-swappable memory owned by the
operating system; the access time to a system semaphore will always be
consistent. In contrast, RAM semaphores are located in memory owned by a
process --which is by default moveable and swappable. If the segment
containing a RAM semaphore has been swapped out to disk, a reference to the
semaphore could be delayed for an unpredictable length of time (but on the
order of tens or even hundreds of milliseconds) until the virtual memory
manager can roll the segment back into physical memory.
Another important aspect of system semaphores is that they are implemented in
memory below the 640K-byte boundary, so that they can be addressed in either
real mode or protected mode. This is vital if you wish to use semaphores to
communicate between a closely coupled process and device driver and the driver
might need to manipulate the semaphore during service of an hardware
interrupt, because the CPU mode at the time of an interrupt cannot be
predicted.
Finally, we should note that the location of system semaphores in physical
memory severely constrains the number that OS/2 can make available. The memory
below the 640K-byte boundary is dear, because it must be conserved for the
execution of real-mode programs in the DOS Compatibility Environment.
Consequently, the maximum number of system semaphores is 128 in OS/2, Version
1.0, and 256 in OS/2, Version 1.1, and many of these are used up by the
operating system itself. If you need large numbers of semaphores in your
application, you will have to use RAM or Fast-Safe RAM semaphores and simply
work around their other limitations.


Message-Passing Performance


As I thought about assessing the relative throughput of message passing using
shared memory, pipes, and queues, I realized that simplistic timings of system
calls would not be very helpful. The amount of tangential work that is
associated with the use of these IPC mechanisms can be fairly extensive
(allocating and deallocating memory segments, setting and clearing semaphores
to control access to shared segments, copying data to and from local buffers,
and so on).
Eventually, I settled upon a timing model which, I think, is at least
reasonably parallel to the IPC performed by real applications. I obtained each
set of timings by running two processes, a parent and a child. The parent's
only function was to launch the child, then serve as a message turnaround
point. As the parent received each message from its child via the IPC
mechanism under test, it would simply do whatever was necessary to ship the
message back to the child again (a more detailed sketch of the timing
procedure for each IPC method can be found in Figure 1, Figure 2, and Figure
3). A consistent message size of 512 bytes was used.
The results, which are reported in Table 3, are based on 100,000 message
round-trips --from child to parent and back again. The tare times were found
and subtracted using equivalent loops where the system calls had been replaced
with dummy functions that returned a success status or other reasonable
result.
Table 3: Comparison of IPC throughput using named shared memory, anonymous
pipes, and queues. The timings are based on 100,000 round-trips of a 512-byte
message between two processes.

 IPC Method Message Round-Trips
 Per Second
 ------------------------------------------------------------

 Share Memory 661

 Anonymous Pipe 346

 Queue 76

IPC performance via shared memory segments, even with the overhead of system,
calls to set and clear RAM semaphores that synchronize access to the segments,
is seen to be far faster than either pipes or queues. In fact, because
processes can easily simulate the behavior of a pipe by explicitly controlling
a ring buffer in a shared segment, the use of pipes for any reason other than
"transparent" communication with an oblivious child process is probably
ill-advised.
Communication by queues turns out, as expected, to be the slowest method. It
is an order of magnitude slower than IPC using shared memory, and two orders
of magnitude slower than signalling with system semaphores. It seems clear
that IPC with queues should be reserved for those occasions where message
prioritizing and selective message scanning and extraction are really needed.
The complexity of queue manipulation, the number of system calls involved, and
the relatively heavy demand for system resources, such as sharable selectors,
should deter you from casual use of queues.
As with the semaphores, these comparisons on a lightly-loaded system could
turn out quite differently on a heavily-loaded system, where applications have
over-committed virtual memory and the virtual memory manager and swapper are
constantly busy. Pipe performance should be relatively consistent, because the
system buffers used by pipes are not swappable. On the other hand, named
shared memory segments, and the giveable shared segments used in queue
messages, are swappable, so IPC performance via shared memory or a queue could
be quite erratic depending on swapper activity, thread priorities, and so on.


Some Final Thoughts


Although OS/2 has gotten off to a slow start, its eventual importance in the
desktop computer world can no longer be doubted. I feel strongly that the
appearance of the high-performance file system (HPFS) and 80386-specific
versions over the next year or so will make it the platform of choice for
software developers. Users will migrate more slowly (we have the history of
the Macintosh to guide us here), but the benefits of OS/2's multitasking,
virtual memory, and graphical user interface will eventually draw them in.
With such a complex system, though, the ad hoc design methods we all used in
the CP/M and MS-DOS days will no longer cut the mustard. We need detailed and
reliable metrics that can help us make tradeoffs between code size, code
complexity, and code performance at every level of an application --in short,
we need an understanding of the operating system's overall behavior that has
never before been necessary in the microcomputer world. The timings presented
in this article are crude and their scope is narrow, but perhaps (with luck)
they will inspire successor articles by wiser and more experienced DDJ
readers!






June, 1989
UNDOCUMENTED DOS


Uncover the mysteries behind undocumented DOS calls to break the 32-Mbyte
drive partition barrier




Rahner James


Rahner James is an independent consultant living near Sacramento, Calif. He
can be reached by calling 916-722-1939 or through CompuServe 71450, 757.


MS-DOS is a fairly well-documented operating system. A person only needs to
browse through the computer section of their local bookstore to get an idea
about how much has been written about MS-DOS. Powerful manuals for powerful
programmers developing powerful tools for powerful users.
After readers have shaken off the giddiness from all of that power going to
their heads, they begin to notice evidence of a mystery -- there are gaps in
the numbering sequence for system calls. At first glance, it's as if Rosemary
Woods went to work for Microsoft after her White House experience. At second
glance, an interested party can see that DOS does indeed respond to the
numbers that fill the gaps, but the purpose of those numbers may be unclear.
This article examines one of those undocumented system calls, DOS function
52h, which returns a pointer to an internal buffer that contains pointers to
various DOS structures. Once I've covered the system call, I'll show you how
it can be used to create drive partitions greater than 32 Mbyte. I have also
included a subroutine that can be added to any DOS block device driver to
allow for the larger drive partitions.


Using Undocumented Features


The first reaction to the idea of using any undocumented commands should
probably be one of skepticism. Why would anyone use an undocumented feature in
their program? If the feature is undocumented, then the manufacturer has no
reason to change it from version to version. Any program that uses such a call
would have to be changed every time a new version of DOS is released! In
theory, this is true -- but when marketing realities meet with theory, the
bigger word wins.
The fact is that most undocumented calls were written to allow upper layers of
the operating system to access the lower layers, while still maintaining a
standard interface. Because it is easier and more economical to continue using
old system call structures, there is a high probability of little or no change
to the major undocumented calls.
The undocumented system calls usually deal with low-level operating system
data structures. Normally, when a programmer wants low-level partition
information that cannot be found through the standard calls, he or she writes
a routine to read the partition boot record from the device, and then gleans
the required data from the record. Unfortunately, a DOS block device is not
required to have a partition boot record, and any assumption about the data
structures on the first logical sector are bound to be wrong in several cases.
Any data structure that is left up to the discretion of the developer should
be considered more troublesome than a data structure that has been defined and
used by Microsoft -- if the developers are given structural leeway by
Microsoft, it should be assumed that they will take it.


MS-DOS Function 52h


DOS function 52h is probably the most useful of the undocumented system calls
because it contains a great deal of information. The purpose of this function
call is to get the internal buffer pointers. The call returns a pointer to a
table of pointers and data. This table describes most of the DOS internal
structures that are associated with the storage subsystem. Figure 1 describes
the calling sequence used in calls to DOS function 52h.
The function returns a far pointer to an internal DOS buffer. The structure of
this buffer differs depending upon the major version of DOS. Although I have
not tested it, function 52h is probably not supported in Version 1.xx.
Table 1 shows the buffer structure for MS-DOS, Version 2.xx.
Table 1: The buffer structure for MS-DOS, Version 2

 Offset Size Description
 ------------------------------------------------------------------------

 -2 word Segment value of the first memory block available
 through the Allocate Memory call.

 0 dword Far pointer to the first MS-DOS Disk Parameter
 Block. This structure is defined after this table as
 the Disk Parameter Structure.

 4 dword Far pointer to a linked list of MS-DOS open file
 tables. This structure is defined after this table as
 the Open File Table List.

 8 dword Far pointer to the beginning of the CLOCK$ device driver.

 0Ch dword Far pointer to the beginning of the CON: device driver.

 10h byte Contains the number of logical drives presently
 supported by the system.

 11h word This contains the size of the largest logical sector size
 supported by the system.


 13h dword Far pointer to the first sector buffer structure used by
 the logical disks. The size of each sector buffer is
 equal to the largest logical sector size plus a sixteen
 byte header. This header is defined after this table as
 the Sector Buffer Structure. The number of these buffers
 is defined by the 'BUFFERS=xx' entry in the CONFIG.SYS
 file.

 17h This is the start of the first device driver (NUL).

The buffer structure for Version 3.xx is identical to 2.xx up to offset 10h.
The buffer structure for 3.xx, beginning with byte 10h, is shown in Table 2.
Table 2: The buffer structure for MS-DOS, Version 3, beginning with byte 10h

Offset Size Description
---------------------------------------------------------------

10h word Contains the size of the largest logical sector size
 supported by the system.

12h dword Far pointer to first sector buffer structure used by the
 logical disks. This is defined after this table as the
 Sector Buffer Structure.

16h dword Far pointer to the drive path and seek information
 table. This structure is defined after this table as the
 Drive Path Table.

1Ah dword Far pointer to a table of FCBs. This table is only valid
 if the "FCBS=xx" command is used in CONFIG.SYS.

1Eh word Size of FCB table.

20h byte Contains the number of logical drives presently supported
 by the system.

21h byte The value of "LASTDRIVE=xx" in CONFIG.SYS. The default
 value is 5.

22h This is the start of the first device driver (NUL).



Disk Parameter Structure


Each disk parameter block entry contains basic information about the logical
disk drives attached to the DOS system. The structure of the disk Parameter
Block entry is similar to the structure of the BIOS Parameter Block (BPB)
described in the DOS Technical Reference Manual. The use of the BIOS Parameter
Block structure to obtain information about the physical constraints of the
disk storage subsystem has several advantages.
In several of the disk-related applications that I have worked on over the
years, the programmer reads the Disk Parameter Block off logical partition
sector 0 in order to determine the physical characteristics of the disk
partition. The assumption was that all block devices followed the IBM
partition structure definition. Unfortunately, that definition is not a
standard, and turns out to be version- and implementation-dependent. In
contrast, the disk parameter structure has not changed since MS-DOS, Version
2.00 (although no testing has been done on Version 4.00), and contains all of
the necessary information (short of the physical disk drive characteristics).
Another advantage of using the disk parameter structure is that some of the
other DOS calls that are used to obtain logical drive information access the
drive for readiness. In some cases, the normal state for the drive is to be
not ready. If the device is not ready, repeated retries cause a delayed call
return, and extra programming is necessary in order to handle the
unintentional error that is returned.
In addition to providing informational advantages, the disk parameter
structure offers an additional feature that is not available through any other
system call -- the "media flag." The media flag indicates whether the device
has been read. If the programmer sets the flag to - 1, DOS forces a reread of
basic device information from the device. This approach is particularly useful
if the programmer modifies the FAT and wants DOS to update its internal
structures with the updated table.
The structure for the disk parameter block is shown in Table 3.
Table 3: The structure for the disk parameter block

Offset Size Description
----------------------------------------------------------------------

0 byte Disk unit number, basically this is the drive number zero
 based (0=A, 1=B, etc.). If this and the next byte are
 0FFh then this entry is the end of the linked list and

 invalid.

1 byte Disk unit number passed to the block device driver
 responsible for this logical drive.

2 word The logical sector size of this drive in bytes.

4 byte Number of sectors per cluster - 1. The number of sectors
 per cluster must be a power of two.

5 byte Allocation shift. It is the shift value used to calculate
 number of sectors from number of clusters without having
 to use division. Number of sectors = number of clusters
 << allocation shift.

6 word Reserved sectors. This is the number of reserved sectors
 at the beginning of the logical drive. The reserved
 sectors may include partition information.

8 byte Number of FATs. This is usually two.

9 word Number of root directory entries. The root is the only
 directory that has a structural limitation on the number
 of entries. This is usually 32 factored into some
 multiple of the logical sector size.

0Bh word First data sector. This is the first sector that contains
 file data. This is also defined as cluster two since
 cluster zero and one have special definitions.
 Conceivably, there could be a hidden area
 between the end of the root directory and the first data
 sector.

0Dh word Last cluster number. This is the number of clusters in
 the data area + 1. If this is less than 0FF6h then the
 FAT uses 12-bit entries; otherwise, it uses 16-bit
 entries.

0Fh byte FAT size. This the size of one FAT in logical sectors.

10h word First root sector. This is the sector number of the first
 root directory entry.

12h dword Far pointer to the block device driver.

16h byte Media descriptor. This is the media descriptor byte as
 described in the MS-DOS Technical Reference Manual.

17h byte Media flag. If this is 0, the drive has been
 accessed. If it is -1 (or is set to - 1), MS-DOS will
 build all its internal data structures concerning the
 drive when it is next accessed.
18h dword Far pointer to the next disk parameter block.



Open File Table List



DOS stores all of its information about open files in a linked list of tables.
Each node in the linked list is a table of file entries that contain
information about each open file. The first table always contains the five
predefined I/O handles. The linked list of tables is not particularly useful,
because the information that it contains can be gleaned from other, more
standard system calls. The list is included here for the sake of completeness
(though even that could be questioned since it is not completely defined). The
structure for the Open File Table List is shown in Table 4, and the structure
for an Open File Table entry is shown in Table 5.
Table 4: The structure for the Open File Table List

 Offset Size Description
 ------------------------------------------------------------------------

 0 dword Far pointer to the next table in the list. If the offset
 of this pointer is 0FFFFh, then the table is the final
 entry and invalid.

 4 word Number of table entries. Each table entry is 53 bytes
 long. There will be at least one entry in each table
 except the terminal entry.

 6 The open file table entries begin here.

Table 5: The structure for an Open File Table entry

 Offset Size Description
 ------------------------------------------------------------------------

 0 word The number of file handles that refer to this table entry.

 2 byte Not defined.
 [5]

 6 dword Far pointer to the device driver header if this is a
 character device. If it is a block device, this will be
 a far pointer to the Disk Parameter Block.

 0Ah byte Not defined.
 [21]

 20h byte This is the filename as it would appear on the diskspace
 [11] filled and with no period to separate the name from the
 extension.

 2Bh word This is the PSP segment number of the file's owner.

 2Dh dword Not defined.

 31h word Not defined.

 33h word Not defined.



Sector Buffer Structure


The number of sector buffers that DOS uses is predefined at boot time. Each
sector buffer consists of a 16-byte header followed by an area equal to the
size of the largest logical sector. The header contains information about the
origin of the data in the buffer, and a link to the next sector buffer in
line. The fact that this linked buffer list is not hardcoded, and that this
system call's pointer is the primary reference to this area, creates some
interesting possibilities (as you'll soon see). The structure for the header
of the sector buffer is shown in Table 6.
Table 6: The structure for the header of the sector buffer

 Offset Size Description
 ------------------------------------------------------------------------

 0 dword Far pointer to the next sector buffer. The buffers
 are filled in order of their appearance in this
 linked list. The last buffer has a 0FFFFFFFFh in

 this location and is a valid buffer.

 4 byte Drive number. This is the drive number that the
 data currently in this buffer was read/written
 from/to. If this buffer has not been used, this
 byte will be set of 0FFh.

 5 byte Data type flags. The bit fields show what area of
 the drive that the data is associated. The bits are
 defined as follows:

 Bit Description
 ------------------------------------------------------------------------

 1 File Allocation Table data

 2 Directory or subdirectory data

 3 File data


 6 word Logical sector number. This is the logical sector
 number that this structure has buffered.

 8 word Access number.

 0Ah dword Far pointer to the disk parameter block.

 0Eh word Unused.




Drive Path Table


The Drive Path Table contains the default path, drive head location, and
various flags and pointers. As with the Open File Table List, all of the
information in the Drive Path Table is also available through other documented
system calls. The number of table entries is equal to the number of valid
logical drives plus one. The last entry contains a zero in the flags variable
and does not have any useful data. The structure for a Drive Path Table entry
is shown in Table 7.
Table 7: The structure for a Drive Path Table entry

 Offset Size Description
 -------------------------------------------------------------------

 0 byte Contains the current default pathname in ASCIIZ
 [64] format. It contains the drive letter, colon delimiter
 and the initial '\'. For example, the string for the
 second table entry with default path of 'my_path' would
 be B:\MY_PATH.

 40h dword Reserved, set to 0.

 44h byte Flags variable. All valid entries contain a 40h; the
 last entry contains 0.

 45h; dword Far pointer to the Drive Parameter Block.

 49h word Current block or track/sector number for this drive.

 4Bh dword Far pointer. Unknown purpose.


 4Fh word Unknown storage.

As you can see, DOS function 52h holds a considerable amount of information.
Virtually everything that the programmer needs to know about the disk is
accessible through this call.


Creating Expanded Partition Sizes


Since the advent of the inexpensive hard disk, the DOS 32-Mbyte disk partition
limitation has been a real problem. Until the joint Microsoft/Compaq "BIGFOOT"
development on Version 3.31 and the later 4.00 release, Microsoft had not
offered any remedies to this deficiency. To fill the marketing requirement,
several companies have released hard disk partitioning software packages that
allow DOS to access partitions up to 1 gigabyte.
In theory, MS-DOS, Versions 2.xx and 3.xx, can access disk partitions up to 4
gigabytes simply by increasing the sector size from 512 bytes to some larger
power of two. In reality, a block device driver written to test this theory,
simply dies. The artificial limitation stems from the buffer initialization at
boot time, when somewhere in the recesses of DOS, an assumption is made that
the basic sector size is going to be 512 bytes. Because all of the buffers are
statically allocated according to that assumption, no sector size larger then
512 bytes are allowed. Obviously, since there are programs that allow larger
partitions, there must be a way to circumvent the assumption. In fact, there
are a couple of ways.
One of the easier routines that you can build allows a DOS block device driver
to access up to 2 gigabytes per partition and uses two of the items provided
by function 52h. The first item is the maximum sector size variable -- the
routine simply needs to change that variable to the maximum sector size
supported by the driver. The second item provided by function 52h is the
linked list of sector buffers. If a sector is read into a buffer that is too
small to hold that sector, catastrophic results and bit death usually occur.
To avoid the overwriting of useful data, the sector buffer linked list must be
redone.


Expanding the Partition Horizon


I have written a small routine in 8086 assembly language that can be included
in any MS-DOS block device driver that supports logical sector sizes above 512
bytes (see Listing One). These days DOS block device drivers are commonplace,
so a thorough explanation of how this type of device driver works is
unnecessary. But, a few relevant points should be reiterated before jumping
right into the routine.
Normal block device drivers have a basic data block size that is some power of
two. The basic block size, usually one 512-byte sector, is defined in the BIOS
Parameter Block provided by the driver as part of the initialization process.
Because the BPB storage variable for the number of blocks in the partition is
16 bits, a partition cannot be larger than 65,535 logical blocks. This
limitation means that in order to increase the storage capacity beyond 32
Mbytes, the block size must be increased beyond 512 bytes.
Most controllers do not support sector sizes other than 512 bytes. Rather than
expecting the hardware to be the answer to larger logical sectors, the easiest
way and the most generally applicable is to just read more 512-byte sectors.
For example, to get a block size of 1024 (partition size of 64 Mbytes), read
two sequential 512-byte sectors. To get a block size of 32,768 (partition size
of two gigabytes), read 64 sequential 512-byte sectors. A well designed block
device driver should easily accommodate this method.
After the block device driver initializes all the devices and their partition
structures, the BPB table should be complete. The largest block size can be
found by scanning the valid BPB array entries. This maximum block size is
required by the MODIFY_DOS routine in a word-size variable called
BIGGEST_SECTOR.
After getting the vector from DOS function 52h, a comparison is made between
the driver's largest block size and the largest block size supported by DOS at
the moment. If DOS already supports a block size equal to or larger than the
driver's block size, no changes are needed.
If the driver's block size is larger, the routine changes two parameters.
First, it alters DOS's largest block size variable with the driver's larger
value. Second, the routine works its way down DOS's block buffer list and
"revectors" them to a new storage location. Care should be taken that a buffer
is not allocated twice.


Tricks of the Trade


I should mention that the routine uses some of the features in Microsoft's
MASM, Version 5.10. I make use of the USES directive in the procedure
declaration. This directive PUSHes any registers listed after the word USES
and POPs them at the end of the routine. USES is a handy directive for the
mental defective (a group which includes myself) who are forever PUSHing and
POPing in the wrong order or the wrong registers.
I also make use of the local label (@@) for short jumps. The syntax for a
forward jump to a @@ label is @F. The syntax for a backward jump is @B. I
generally use these labels for tight loops and comparison skips to
differentiate them from labels referenced farther away in the program. This
type of label should either not be used in macros, or only used in macros
because subtle bugs could result. As an example, say a macro called DELAY has
been coded as shown below:
 DELAY macro
 jmp short @F
 @@:
 endm
This macro was used in the following code segment:
 (0) cmp bl, 8
 (1) jb @F
 (2) out 0a1h, al
 (3) delay
 (4) sub bl, 8
 (5) @@: mov INT_NUMBER, bl
In this case, the comparison skip goes to line 4 rather than line 5 as you
might expect.


Final Note


Several other undocumented MS-DOS system calls exist. Many of them provide
pointers to the data structures shown in this article. All of them need only a
debugger and a little patience to discover. In finding an application for the
information uncovered, the reader is only constrained by imagination and the
bouncer at the door.


Undocumented DOS
by Rahner James


[Listing One]

; *********************************************************************

; DATA STRUCTURES


; *********************************************************************


CALL52_2XX struc

dpb_ptr dd ? ; Far ptr to first disk parameter block
open_file_ptr dd ? ; Far ptr to Open File table
clock_ptr dd ? ; Far ptr to CLOCK$ device driver
con_ptr dd ? ; Far ptr to CON: device driver

num_drives_2xx db ? ; Number of logical drives
sector_sz_2xx dw ? ; Maximum sector size
buffer_ptr_2xx dd ? ; Far pointer to next sector buffer
NUL_driver_2xx db ? ; Beginning of the NUL device driver

CALL52_2XX ends


CALL52_3XX struc

 dd 4 dup(?) ; This are defined in previous structure

sector_sz_3xx dw ? ; Maximum sector size
buffer_ptr_3xx dd ? ; Far pointer to next sector buffer
path_ptr_3xx dd ? ; Far pointer to drive path table
FCB_ptr_3xx dd ? ; Far pointer to FCB table
FCB_sz_3xx dw ? ; Size of the FCB table
num_drives_3xx db ? ; Number of logical drives
lastdrive db ? ; LASTDRIVE number
NUL_driver_3xx db ? ; Beginning of the NUL device driver

CALL52_3XX ends


; *********************************************************************

; MODIFY_DOS
;
; Changes MS-DOS's internal maximum sector size and buffer linked list
; to allow MS-DOS to access partitions above 32-megabytes
;
; Given:
; BIGGEST_SECTOR set to largest logical sector of all partitions
; DOS_VERSION = major version of MS-DOS
; OLD_FILE_END -> initial end of driver
;
; Returns:
; TRASH_PTR set to new end of program
; MS-DOS's internal maximum sector setting changed
; MS-DOS's internal linked buffer list relocated

; *********************************************************************


modify_dos proc uses ax ds di si
local sect_para_size:word
local buffer_paragraph:word
 mov ax, offset old_file_end ; Get pointer to old end of file
 add ax, 15 ; Change it to a segment number so we

 mov cl, 4 ; only have to deal with a 16-bit number
 shr ax, cl
 mov bx, cs
 add ax, bx ; AX -> paragraph start
 mov buffer_paragraph, ax ; Save it for later

 mov ax, biggest_sector ; AX = size of buffer we may need to
 ; allocate
 add ax, 10h ; The buffer header too
 shr ax, cl ; Change bytes/sector to paras/sector
 mov sect_para_size, ax ; Save for later

 mov ah, 52h ; Our undocumented system call
 int 21h ; Set ES:BX -> DOS list of lists
 mov di, buffer_paragraph ; DI -> beg. of initialization trash

 mov si, sector_sz_2xx ; SI = offset to version 2.xx sector size
 cmp dos_version, 2 ; See if version 2.xx
 je @F ; Skip if it is
 mov si, sector_sz_3xx ; SI = offset to version 3.xx sector size

@@: mov cx, es:[bx+si] ; CX = the largest block size supported

 cmp cx, biggest_sector ; See if we need to change anything
 jnc done ; Jump if we don't need to

 mov cx, biggest_sector ; Update DOS with the new largest
 add bx, si ; BX -> sector size variable
 mov es:[bx], cx ; Update maximum sector size w/our sector
 add bx, 2 ; Buffer pointer is right after for
 ; both versions
 mov dx, cs
@@: mov si, bx ; DS:SI = ES:BX
 mov ax, es
 mov ds, ax

 les bx, dword ptr es:[bx] ; ES:BX -> next buffer in list
 cmp bx, -1 ; Are we at the end
 je done ; Quit if we are
 mov byte ptr es:[bx+4], -1 ; Set the nasty buffer flag

 mov ax, bx ; Calculate absolute memory placement
 mov cl, 4
 shr ax, cl
 mov cx, es
 add ax, cx
 cmp ax, dx ; See if we are above or below the line
 ja @B ; Loop again if already done this buffer

 les bx, dword ptr es:[bx]
 xchg si, bx ; xchg es:bx, ds:si
 push es
 mov ax, ds
 mov es, ax
 pop ds

 mov word ptr es:[bx], 0 ; Previous buffer -> this one
 mov es:[bx+2], di
 push es

 mov es, di
 mov es:[0], si ; This one -> what previous did
 mov es:[2], ds
 mov byte ptr es:[4], -1 ; Tell DOS this hasn't been used yet
 pop es
 add di, sect_para_size ; DI -> next buffer start

 cmp si, -1 ; See if we are at end of list
 je done ; Quit if we are
 mov bx, si ; ES:BX -> next buffer in line
 mov ax, ds
 mov es, ax
 jmp @B

done: mov cs:trash_pointer, di ; New end of program
 ret
modify_dos endp













































June, 1989
REAL-TIME DATA ACQUISITION


Collecting and storing large amounts of data, such as music, has special
requirements




Mike Bunnell and Mitch Bunnell


Mike and Mitch are engineers at Lynx Real-Time Systems Inc. and can be reached
at 550 Division St., Campbell, CA 95008.


The acquisition and storage of large amounts of data is vital in many
industrial, research, and aerospace/defense applications in which computer
systems typically must collect analog data at very high rates (1,000 to
1,000,000 samples/sec.) for relatively long periods (a few seconds to a few
days). Consequently, computers used in these environments must be able to
assemble huge volumes of data in real time and store them in an organized way
so that the information can be analyzed later. Until recently, systems that
could do this cost more than $100,000. With current hardware and software
technologies, however, equivalent PC-based systems are one-tenth that price.
A computer system -- large or small -- designed for real-time data acquisition
must meet four basic requirements: It needs to have an interface that converts
continuous real-world data into discrete, digitized samples at a rapid
sampling rate; there must be sufficient throughput to mass storage to save the
samples at the sampling rate; a data acquisition program must read the samples
from the interface and write them to mass storage; and the software must have
sufficient real-time response so data samples are never lost. In this article,
we'll describe the elements of a common PC-based data acquisition system and
show how it can be used in a typical application.


Principles of Real-Time Data Acquisition


Although most real-world data is analog in nature, digital computers, by
definition, can only process discrete digital data. To solve this dilemma, an
analog-to-digital (A/D) interface is needed. Because analog data is
continuous, the A/D interface must take a snapshot of the analog data at one
point in time. This process, called sampling, is usually done at a particular
sampling rate depending on how fast the analog data is expected to change. The
interface must digitize each sample by converting it to a numerical value. The
resolution, or number of bits, used to represent this value is dependent on
the analog interface. Unless the sampling rate is slow (less than 100
samples/sec.) the interface must provide some way of buffering the digitized
data until the CPU accesses it. This can be done either by providing a
first-in, first-out (FIFO) buffer as part of the analog interface or
direct-memory-access (DMA) capability so the interface can save the data
directly to computer memory without using the CPU.
Handling the analog interface is left to a device-dependent code -- the device
driver. The device driver sets the operating parameters of the interface,
maintains the data buffers, and services requests to read converted data or
set the sampling rate. In the same way as the analog-to-digital converter is
the interface between the computer and the real world, the device driver is
the interface between a data acquisition program and the converter hardware.
In a data acquisition system, sampled data is saved to mass storage (typically
a hard disk) for later processing. The reason for saving the data to disk and
not just computer memory is that there is usually more data that must be saved
than there is memory available. For convenience sake, it is preferable that
data be saved in named files on the disk so the data can be conveniently
accessed simply by reading it from the appropriate files.
Writing data to a file must be fast enough to keep up with the sampling rate.
If, for example, the sampling rate is 100,000 samples/sec. and each sample is
2 bytes, then the computer must be capable of writing 200,000 bytes/sec. to a
file on a sustained basis. Although disk-drive manufacturers may quote burst
transfer rate and seek rates, these do not reveal the sustained data rate. The
best way to determine this rate is to do real-life testing.
This testing process must include writing data to a file, a process that is
typically slower than writing directly to the disk. On most operating systems
(including Unix and DOS), file access is optimized assuming random
intermittent access and blocks are dynamically added to the file as it grows.
The data blocks that make up a file may be scattered all over the disk, making
access to some files faster than to others. To provide faster reads and writes
to files used for high-speed data acquisition, some operating systems offer
contiguous files that have data blocks that are preallocated on the media to
provide maximum speed for sequential access. The file cannot grow dynamically
beyond a preset maximum size, but it can be written to as fast as writing
directly to the disk.
To acquire data at high speed, there must be a program to read the data from
the analog input device and write it to the file on mass storage. This program
must use a FIFO buffering scheme to simultaneously read data from the analog
device and write it to disk. Buffering is important because it allows large
buffers to be transferred to disk without losing input samples during the disk
access. Although this can be done with a single task and asynchronous I/O, it
is much easier to use two tasks -- one to store sampled data into the buffer
and one to write data from the buffer to disk. The ability of a program to
write to a file on disk presupposes that the program is running under some
operating system. A multitasking operating system is needed to support the
data acquisition program so it can read and write simultaneously (the CPU
can't continually read from the analog input if it is in a busy wait loop
waiting for the disk).


Real-Time Response


For a real-time data acquisition system to work, much care must be taken to
ensure that samples are not lost because the input buffer of the analog device
driver has overflowed. Because the input buffer is filled at the sampling
rate, the buffer must be emptied within a certain period of time.
In a multitasking system, the problem of emptying the input buffer in time
becomes one of CPU throughput and task response. The computer must run fast
enough to be able to copy the data from the input buffer before any of it is
overwritten. This is the easy part; 32-bit microprocessors can typically copy
2 to 8 Mbytes/sec. and the setup time is minimal. The more difficult part is
task response. The input task waits for the input buffer to fill to a certain
point by giving up use of the CPU. This gives other tasks in the system the
chance to run (such as the task that writes the sampled data to disk). When
the buffer has filled to the certain point, the device driver schedules the
input task to run. The time to schedule a task, the amount of time for which
task switches are disabled, and the time it takes to do a task switch all
contribute to the worst-case task response.
Many operating system manufacturers quote only the best and typical task
responses because they have not taken care to limit the time for which task
switches are disabled. This is not useful because data samples can still be
lost if the input task cannot respond in time occasionally. For this reason,
programmers must construct their code with the worst-case task response times
in mind rather than best, typical, or average response times.
A real-time operating system guarantees fast worst-case task response, making
the operating system an important part of a high-speed data acquisition
system. These requirements and how they can be met are best explained with an
example of a data acquisition system.


A Data Acquisition System


For the purposes of this article, we'll describe a typical data acquisition
system that can collect 12-bit analog data on up to 16 channels and save it to
disk at rates from as low as once every two seconds up to 250,000/sec.
(aggregate). The hardware in our system consists of an 80386 AT-compatible
computer, an analog I/O board, and a high-capacity hard disk. The software
consists of a real-time operating system, an analog I/O device driver, and a
couple of utility programs.
The PC chosen for our high-speed data acquisition system is an 80386 AT
compatible manufactured by Mylex. The 20-MHz model comes with 4 Mbytes of main
memory and is rated at approximately 4 MIPS. The 80386 AT compatible was
chosen for several reasons. First, it has the ability to run the LynxOS,
Unix-compatible, real-time operating system. Second, many I/O boards are
available for the AT bus. Third, because they are produced in such large
volumes, 80386 AT compatibles have an excellent price/performance ratio.
Figure 1 shows the hardware configuration.
The analog I/O board used in this system is the DT2821 from Data Translation.
It features 16 input channels and 2 output channels, both with 12-bit (or
16-bit) resolution. The DT2821 was chosen because of its DMA support for both
input and output channels. Either a DMA interface or on-board FIFO is
necessary to allow high-speed transfers without hogging the CPU. The CPU needs
to be free to perform other tasks, not the least of which is saving the
transferred data to disk. Both the high-speed input and output capabilities of
the DT2821 are used in our example. The board has an input throughput of up to
250KHz and an output throughput of up to 130KHz per channel.
The last hardware component of the computer system is the mass storage device.
LynxOS, the operating system we use, comes with an optional high-speed SCSI
interface that breaks the normal DMA bottleneck on the AT by bursting 32 bytes
at a time over the bus using 16-bit transfers. This SCSI interface has a
maximum throughput of 3.5 Mbytes/sec. The average AT SCSI interface, which
does 8-bit DMA transfers, has a maximum throughput of 250K/sec. Our sample
system uses a CDC 270-Mbyte Wren IV drive. Its throughput in our system is
over 1 Mbyte/sec. We could have chosen the standard AT interface and hard
drive, which has a sustained throughput of 211K/sec., but we wanted higher
capacity.
The most important piece of software in our system is the operating system. We
use LynxOS, a real-time, multitasking, multiuser operating system especially
designed for closed-loop control and data acquisition. LynxOS is a 4.2BSD Unix
look-alike with features added from System V Unix.
Compatibility with Unix gives users a wide choice of programs with which to
develop software and process collected data, but the major feature of this
operating system is its real-time capability. It has guaranteed worst-case
interrupt response and task response delays, which means that the worst-case
response to external events can be calculated for every program running.
LynxOS also has user-controlled priority scheduling, which means that data
acquisition tasks can be set at high priority and will not be affected by
tasks running at lower priority. In addition, multiple streams of high-speed
data can be acquired concurrently. Program development can be done while
acquiring data, and data can be processed and displayed while other data is
being acquired.
Another important feature of LynxOS is the ability to create contiguous files
within the normal file system. Contiguous files are files whose data blocks
are sequential on the disk. Accesses to contiguous files do not go through the
disk cache. Instead, the data is transferred directly from user task memory
via DMA to and from the mass storage device. Not only is there practically no
CPU overhead transferring the data, but also larger amounts of data can be
transferred per request. Most SCSI disk drives can transfer data with 64K
requests more than twice as fast as 8K requests because of controller overhead
in SCSI command processing.
Contiguous files are just like regular files, with two restrictions: The
maximum size must be specified when creating them, and access requests to them
must be made in multiples of 512 bytes. The size of a contiguous file can vary
just as can a normal file. The creation size is simply the maximum size the
file can become.
The next piece of software is the device driver for the DT2821 I/O board. The
device driver is the link between the operating system and the DT2821 device.
The operating system and the device driver make the device look just like a
file on disk. The device can be opened, read from, or written to just as a
file can.
When the device is opened for reading, the device driver starts up a DMA
channel to read from the DT2821 and write into a 4K circular buffer. Figure 1
shows the circular DMA buffer in the DT 2821 driver.
The PC/AT DMA controller is programmed for auto-initialize mode so that when
it fills up the 4K buffer, it starts over again automatically. This mode is
important because at 250,000 samples/sec. there would only be 4 microseconds
to reload the DMA controller, which is not enough time.
When it receives a read request, the driver uses a timer to wait approximately
the time it takes to acquire 2K into the circular buffer. Then the driver
copies from the circular buffer to the buffer in the requesting task. Waiting
and copying is repeated until the requested number of bytes is copied, at
which time read( ) returns to the calling program. The driver handles writing
to the D/A ports of the DT2821 in a similar fashion.
The rest of the software is made up of two utility programs provided with the
operating system. The first of these is saio. Saio allows you to set the rate
of acquisition and the gain on each channel and to group channels for
simultaneous access. It does this by making requests through the ioctl system
call to the device driver.
The second utility program is dbuff. Dbuff reads data from standard input and
writes it to standard output continuously. Dbuff begins by forking itself into
two tasks. The input (producer) task puts its data into one of two shared
buffers; the output (consumer) task gets the data from each shared buffer when
full. Figure 3 shows the double-buffering scheme used by dbuff. This
double-buffering scheme allows large buffers to be written to the disk, a
necessity when high throughput is required. Dbuff uses many of the operating
system features, such as multitasking, shared memory, and semaphores. A
complete listing of dbuff is shown in Listing One.



Real-Time Considerations


Before we start acquiring data, we must make sure that we have the real-time
response necessary to make our system work. The task reading from the DT2821
device has to read the data out of the circular buffer before the DMA
controller overwrites the data. The task reading is awakened by the timer to
read out of the 4K circular buffer after the DMA has time to fill half the
buffer. This gives the task half of the time it takes to fill the buffer to
get the data. At our maximum DMA transfer speed of 500K/sec., 2,048/500,000
seconds (approximately 4 milliseconds), are available to transfer the data.
The operating system guarantees 500 microseconds (0.50 milliseconds)
worst-case response for the highest priority task on a 4-MIP 80386 computer.
Therefore, if the task reading from the DT2821 is the highest priority task,
we are assured of success. If there are other tasks at higher priority, such
as another data acquisition task, we would have to measure its longest
continuous CPU usage and add that to the 500 microseconds and make sure that
that value was less than 4 milliseconds.
We can estimate the longest continuous CPU usage for our data acquisition
tasks. The task accessing the DT2821 incurs a system call overhead of 25
microseconds when reading or writing. The longest stretch of time the DT2821
device driver uses the CPU is the 0.3 to 0.6 milliseconds it takes to copy 2K.
Thanks to contiguous files, the task writing to the disk uses even less CPU
time, only about 70 microseconds to set up the DMA controller and SCSI
controller to transfer the data to or from disk, including the system call
overhead. The continuous CPU usage for both data acquisition tasks is 0.02 +
0.60 + 0.07 = 0.69 milliseconds.
Thus the worst-case response of a task running at lower priority than both
tasks doing the data acquisition is the guaranteed worst-case response of 500
microseconds plus the CPU usage of the data acquisition tasks, or 0.69 + 0.50
= 1.19 milliseconds.


A Data Acquisition Session


As an application of a high-speed data acquisition system, we can record a
song on the hard disk at compact disc speeds and then play it back, first on
just one channel, then in stereo.
This particular application shows acquisition of analog data, which is the
kind of data usually acquired. The rate of acquisition is on the same order as
that of a typical application. The quantity of data acquired is also common to
many high-speed data acquisition applications. Finally, recording and playing
back music is a good test of the acquisition system because you can hear
whether data is acquired properly when it is played back.
Suppose we record the music from a home stereo receiver. Standard voltage
levels for these devices are in the range -1 to +1 volts. The DT2821 board's
bipolar range is -10 to 10 volts. So, we need to set the gain as close to 10
as we can for the input signal and use a voltage divider circuit made from two
resistors to get the output in the correct range. We tie input channel 0 on
the DT2821 to TAPE OUT LEFT on our stereo, and we tie output channel 0 to our
resistor circuit, then to AUX LEFT input on the back of the amplifier. For
stereo, we can tie input channel 1 and output channel 1 to TAPE OUT RIGHT and
AUX RIGHT. Figure 4 shows the stereo/computer combination.
Normally, you must use a Nyquist filter, which is a low-pass filter with a
cutoff of half the sampling frequency, on the input to the A/D converter.
Experimentation has shown, however, that the filtering through the stereo is
sufficient. It is also a good idea to put an anti-aliasing filter on the
output so that the output frequency can't be heard. An anti-aliasing filter is
not really necessary in this case because it is impossible to hear the
sampling frequency of 44KHz. The amplifier effects some filtration internally
as well.
Now let's do some data acquisition. Compact disc speed is 44,000 samples/sec.,
which means we will have to save 88K/sec. to disk for a single channel and
176K/sec. to disk for stereo.
First, we create a 30-Mbyte contiguous file to hold the song. With a file of
this size we can record about 6 minutes from a single channel or 3 minutes of
stereo. The LynxOS command to create a contiguous file is
mkcontig music.data 30m
Now we set input and output transfer rates and the input gain of the DT2821.
The device node for the DT2821 is called dtaio and is located in the
directory/dev. We'll just collect one channel of data first:
 saio - rate .000027 - gain 10 < /dev/dtaio

 saio - rate .000027 > /dev/dtaio
The driver will set the gain and rate values as close as it can to the
requested values. The actual values can be inspected as follows:
 saio < /dev/dtaio
Next we can set the stereo to our favorite FM station and record a song off
the air. We can use dbuff and redirect the input from /dev/dtaio and direct
the output to our file music.data:
 dbuff 21 20 < /dev/dtaio > music.data
When the song ends, we can press Ctrl-C to stop collecting. Note that dbuff
takes two numeric arguments. The first value is used for the priority of the
producer task, and the second value is used for the priority of the consumer
task. We have set the priority of the task reading from /dev/dtaio to 21 and
that of the task writing to the file music.data to 20. It is not really
necessary to set the task accessing /dev/dtaio to be of higher priority than
the consumer task in this case because the consumer is writing to a contiguous
file and will not use much CPU time.
Now let's play back the song. We set the stereo to AUX and type
 dbuff 20 21 < music.data > /dev/dtaio
To add echo or reverberation to the data, you can run the data through the
reverb program in Listing Two using:
 reverb < music.data dbuff 20 21 > /dev/dtaio
Reverb reads from standard input, so it is necessary to use redirection to
make it read from music.data.
To hear what a song sounds like backward, you can use the program in Listing
Three to reverse the samples. The program, hypothetically called reverse,
could be executed as follows:
 reverse music.data dbuff 20 21 > /dev/dtaio
The output from reverse is piped through dbuff, then sent to the DT2821. We
have told dbuff to set the priority of the task writing to /dev/dtaio higher
than that of the producer task to guarantee its response time. We can combine
these "filter" programs easily:
 reverse music.data reverb dbuff 20 21 > /dev/dtaio
To record and play in stereo, we use saio to group input channels 0 and 1 and
output channels 0 and 1:
 saio - group 0 1 < /dev/dtaio

 saio - group 0 1 > /dev/dtaio
The commands to record and play a song are the same as before.
We are now collecting data at 88,000 samples/sec., or 176K/sec. After
acquiring the data, it can be processed and analyzed on our system or sent to
another computer. (LynxOS comes with a powerful software development
environment and X Windows, which can be used to create programs to display,
edit, and process the acquired data. Also, it supports Ethernet and TCP/IP to
provide high-speed links to other computers.)


Summary


The computer system described in this article meets all requirements of a
high-speed data acquisition system. The DT2821 analog I/O board serves to
change the real-world data to discrete digital data. The SCSI disk system
provides the high-speed mass storage capability. LynxOS, dbuff, and the device
driver for the DT2821 are the software that performs the acquisition. The
operating system guarantees the real-time response and provides the
environment to analyze the data.


Availability


All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063, or call 800-356-2002 (from inside
Calif.) or 800-533-4372 (from outside Calif.). Please specify the issue number
and format (MS-DOS, Macintosh, Kaypro).


Real-Time Data Acquisition
by Mike and Mitch Bunnell



[LISTING ONE]

 /* dbuff.c Double buffering program for continuous
 reading from input and continuous writing to output

 */

 #include <stdio.h>
 #include <smem.h>
 #include <sem.h>

 extern char *malloc();
 extern int errno;

 #define BSIZE 65536 /* size of each buffer */

 struct xbuff {
 char buffer[BSIZE];
 int count;
 int psem;
 int csem;
 int done;
 struct xbuff *other;
 };

 /*
 Write function that is used by the output task
 */

 outputr(p, prio)
 register struct xbuff *p;
 int prio;
 {
 int count;

 setpriority(0, getpid(), prio);
 while () {
 sem_wait(p->csem); /* wait for buffer to fill */
 if (p->count <= 0) {
 sem_signal(p->psem); /* leave if finished or error */
 break;
 }
 count = write(1, p->buffer, p->count); /* write output */
 if (count <= 0) {
 /* exit if error on write */
 p->done = 1;
 sem_signal(p->psem);
 break;
 }

 /* tell producer buffer has been emptied */
 sem_signal(p->psem);
 p = p->other;
 }
 }

 /*


 Read function that is used by the input task
 */
 inputr(p, prio)
 register struct xbuff *p;
 int prio;
 {
 int count;

 setpriority(0, getpid(), prio);
 do {
 /* wait for consumer to empty buffer */
 sem_wait(p->psem);
 if (p->done) {
 break;
 }
 /* read from input and fill buffer */
 count = read(0, p->buffer, BSIZE);
 p->count = count;

 /* tell consumer task buffer is filled */
 sem_signal(p->csem);
 p = p->other;
 } while (count > 0); /* exit when no more data */
 }

 main(argc, argv)
 int argc;
 char **argv;
 {
 register struct xbuff *buffa, *buffb;
 int inprio, outprio;

 /* default to current priority */
 inprio = outprio = getpriority(0, 0);
 if (argc == 2) {
 /* Get input priority from command line if present */
 inprio = atoi(argv[1]);
 }
 if (argc == 3) {
 /* Get output priority from command line if present */
 inprio = atoi(argv[1]);
 outprio = atoi(argv[2]);
 }

 /* Allocate shared memory */
 buffa = (struct xbuff *) smem_get(
 "buffa",
 (long)sizeof(struct xbuff),
 SM_READ SM_WRITE);
 buffb = (struct xbuff *) smem_get(
 "buffb",
 (long)sizeof(struct xbuff),
 SM_READ SM_WRITE);

 /* delete old semaphores if they exist */
 sem_delete("buffac");
 sem_delete("buffap");
 sem_delete("buffbc");

 sem_delete("buffbp");

 buffa->csem = sem_get("buffac", 0); /* Create new semaphores to */
 buffa->psem = sem_get("buffap", 1); /* control access to shared */
 buffb->csem = sem_get("buffbc", 0); /* memory */
 buffb->psem = sem_get("buffbp", 1);
 buffa->done = buffb->done = 0;

 buffa->other = buffb;
 buffb->other = buffa;

 /*
 Create another task to write.
 This task will read.
 */

 if (fork() != 0) /* Create another task to */
 inputr(buffa, inprio); /* write. This task will */
 else /* read */
 outputr(buffa, outprio);
 }





[LISTING TWO]

 /* Reverb.c IIR filter program to add reverberation */

 #include <file.h>

 extern char *malloc();

 ewrite(s)
 char *s;
 {
 write(2, s, strlen(s));
 }

 /*
 Read the whole size read() under UNIX returns the amount it
 read. Last buffer is (biased) zero-filled.
 */
 fullread(fd, buff, size)
 int fd;
 char *buff;
 int size;
 {
 int i, j;

 i = 0;
 do {
 j = read(fd, &buff[i], size - i);
 if (j <= 0) {
 /* This must be the last buffer of the file */
 while (i < size)
 buff[i++] = 0x800;
 return -1;

 }
 i += j;
 } while (i < size);

 return size;
 }

 main(ac, av)
 int ac;
 char **av;
 {
 short *ibuff, *obuff;
 int delay;
 int i;
 int fd;
 int rundown;
 int rv;
 char *fn;
 register short *p, *q;

 if (ac > 2) {
 ewrite("usage: reverb [delay]\n (delay expressed in samples)\n");
 exit(1);
 }
 if (ac == 2)
 delay = atoi(av[1]);
 else
 delay = 10240;

 /* make sure delay is multiple of 512 bytes */
 delay -= delay & 511;

 /* make delay >= 512 andd <= 128K */
 if (delay < 512)
 delay = 512;
 if (delay > 128*1024)
 delay = 128*1024;

 fd = 0;

 ibuff = (short *) malloc(delay * sizeof(*ibuff));
 obuff = (short *) calloc(delay * sizeof(*obuff));

 do {
 /* Read a buffer, but don't check error status yet */
 rv = fullread(fd, ibuff, delay * sizeof(short));

 /*
 Add the fresh input samples to the old samples, after
 dividing the old samples by 2
 */
 for (p = ibuff, q = obuff, i = 0; i < delay; ++i, ++p, ++q)
 *q = ((*q - 0x800) >> 1) + *p;

 /*
 Write the output reverbed buffer
 */
 write(1, obuff, delay * sizeof(short));
 } while (rv != -1);


 /*
 Allow sound in output buffer to "die down"
 */
 for (rundown = 11; --rundown >= 0; ) {
 for (q = obuff, i = 0; i < delay; ++i)
 *q = (*q - 0x800) >> 1;

 write(1, obuff, delay * sizeof(short));
 }
 }







[LISTING THREE]

 /* reverse.c Write a file in reverse to standard output */

 #include <file.h>
 #include <types.h>
 #include <time.h>
 #include <stat.h>

 main(ac, av)

 int ac;
 char **av;
 {
 int fd;
 short buff[4096];
 int rc;
 int i, j, t;
 long pos;
 struct stat s;

 ++av;
 if ((fd = open(*av, O_RDONLY, 0)) == -1) {
 perror(*av); /* exit if can't open file */
 exit(1);
 }

 fstat(fd, &s); /* find the size of the file */
 pos = s.st_size & 1;

 while (pos > 0) {
 /* See how many bytes can be read now */
 if (pos < sizeof(buff))
 rc = pos;
 else
 rc = sizeof(buff);

 pos -= rc;
 /* Seek back a block and read */
 lseek(fd, pos, 0);
 read(fd, buff, rc);


 /* Reverse the samples in the block */
 for (i = 0, j = (rc / 2) - 1; i < j; ++i, --j) {
 t = buff[i];
 buff[i] = buff[j];
 buff[j] = t;
 }

 /* Write the reversed block */
 write(1, buff, rc);
 }

 close(fd);
 }
















































June, 1989
VARIABLE-LEVEL PROGRAMMING


This approach is a good compromise when performance considerations prevent the
complete usage of a high-level language




Ronald Fischer


Ronald Fischer is a software developer for Software-Entwicklung & Consulting
and can be reached at Straubinger Strasse 20, D8000 Munchen 21, W. Germany.


Looking back at approximately 30 years of steady development of programming
languages, there is no doubt that development strives towards more generality,
more abstraction, and --last but not least --more expressional power. The
first assembler programs of the early 1950s were a giant achievement over
machine code (binary) programming. Later, the first so called high-level
language was invented by software pioneer John Backus. This language was
called Fortran, and its descendant, Fortran 77, is still in use today.
The next development in programming languages was known as "non-procedural
languages" (sometimes called 5th-generation languages, a term that I do not
like because it has become rather worn over the years). The very same
expression is used today not only for pure database languages like SQL, Nomad,
and RAMIS but also for true general-purpose languages such as Prolog.
No one debates that you can code a given problem much faster using, say,
Prolog rather than assembler. We call this modern approach "high-level
programming," compared to the "low level" of assembly language. So why do
assemblers still exist? As every system programmer knows, there are situations
where you have to use assembler. Most operating system kernels are, for
instance, written in assembler. Even the famous Lilith machine, designed in
the early 1980s by a design team led by Pascal inventor Niklaus Wirth, had a
few critical routines written in machine language, although most of the
operating system was coded in Modula-2. The reasons for such decisions are
--in most cases --speed, but sometimes space considerations also play a
critical role.
On the other hand, given a sufficiently huge problem, like coding a multiuser
operating system, it becomes too complex for the human brain to manage at
assembly level. In most applications, a mix of assembler and high-level
language (HLL) is used. Examples are the PICK operating system
(Basic+assembler) and some of IBM's mainframe operating systems (PL/1 and
assembler). One could call this approach "two-level programming," because
languages at precisely two different levels are used.
When developing system software, two-level programming is often an
unsatisfactory solution: For much of the code, the HLL is "too high," but
assembly language is too "low." System programmers therefore have been looking
for languages that can better accommodate the range of needs. We call a
programming language that satisfies this property "variable level."
Many experiments were carried out in this area during the late 1960s and
throughout the 1970s: BCPL, BLISS, SPL3000 (Hewlett-Packard) and Siemens SPL
are just a few examples. Three languages, however, proved to be very
successful: C, which was used to write the Unix operating system, PL/M used
for system programming by Digital Research, and Charles Moore's Forth. But
only the latter really earns the tag "variable level," because neither C nor
PL/M, despite their success, provide a single framework for system
programming: Both Unix and Digital Research's Concurrent CP/M contain some
assembler code and the user does not have much influence on the degree of
abstraction or "level" of the statements used.
Forth is different. A Forth program consists of a collection of words (similar
to functions or procedures in other languages). Each word in turn is composed
of other words. The user therefore may assign to each conceptional level (or
abstraction layer) a family of Forth words that are defined in terms of
lower-level words only. The assembler is integrated into the language and
represents the bottom level. In practice, this means that a function for
searching through a text file uses high-level Forth words like EOF?,
GET-NEXT-LINE, or MATCH-PATTERN, while an interrupt handler manipulates bits
and bytes.


Variable-Level Approaches


Recently two new languages aimed at variable-level programming have gained
attention from system programmers: FUTURE86 and CDL2. The remainder of this
article covers some of the main features of these languages. For comparison
purposes, two sample programs are coded in FUTURE86 and CDL2. Because many
readers may be familiar with C, these sample programs are described first in
C. All programs were developed on a 80386 system running PC-DOS 3.3. The
compilers used were MIX Power C, Development Associates FUTURE86, and epsilon
CDL2Lab.
The first example is a subroutine, AUXOUT, which sends one character to the
serial device (called AUX under DOS). This is an example of a low-level system
dependent function, because it is unlikely that the very same call exists on
different operating systems. Although DOS, for example, has one primary serial
port --namely AUX -- other operating systems might always require an
identification of which serial interface to use.
Most C compilers support some generic interface to the host system. For
example, Power C has among its functions a routine called bdos, which accepts
three parameters and generates an INT 21H interrupt to call DOS. The
parameters are placed in the registers AH, DX, and AL, respectively, and
correspond therefore to function code, function argument, and function code
parameter. For this example, a quick look in IBM's DOS Technical Reference
Manual tells that we have to use 4 as the function code (auxiliary output) and
the character to be output as arguments. The code parameter is not used; we
deliberately set it to zero. Listing One shows the resulting function.
The second example is of much higher level. Suppose we have two character
strings, S and T, and one character transformation function XTRANS, which
accepts one character and delivers an arbitrarily transformed character. The
objective is to design a function SFILT, which appends S to T, but by doing so
applies XTRANS to every character of S. For this example, we assume that the
array holding T must be large enough to also accommodate S.
As an application, imagine that you want to send one record of a binary file
to the printer. Because the data may also contain nonprintable characters,
such as escape sequences, we cannot blindly output the record. Instead, each
nonprintable character should be replaced by a dash. This could be
accomplished with SFILT in the following way: Let S be the record to be
printed, let T be a sufficiently large buffer containing the string
"RECORD #12 (non-printables replaced by '-'): "
and finally let XTRANS be the function that maps nonprintable characters into
dashes and leaves others unchanged.
For the sake of brevity assume in the example a fixed function named XTRANS;
in practice, this function itself would be passed as a parameter for added
flexibility so that different calls of SFILT could use different
transformation functions. For the same reason, no concrete implementation of
XTRANS is given.
The resulting C program is shown in Listing Two. Efficiency is not the primary
goal, however, because FUTURE86 and CDL2 are biased towards recursive function
calls, we will also use recursion in the C example. This makes it easier to
compare the examples. In a real project, of course, you would use a loop.


The FUTURE86 Language


The history of FUTURE86 dates back to 1979. At this time Akira Katagiri, a
Japanese scientist, implemented a Forth compiler for a Z-80-based
microcomputer system. During this work, he found some deficiencies in this
language, most notably the lack of readability. He then devised a successor,
appropriately called FIFTH, and implemented it on different environments.
FIFTH is widely used in Japan.
FUTURE86 is an evolution of FIFTH created during the 1980s. The first compiler
was commercially available in 1987. Like FIFTH, it augmented the power of
Forth without being just another Forth dialect. Despite its short history,
FUTURE86 has already been used to implement numerous projects. For example,
software for sophisticated, automatic postal scales and a natural language
compiler has used FUTURE86. Like Forth, every FUTURE86 program implicitly
operates on two stacks, called the data stack (that is, "stack") and the
return stack. These names are a little bit misleading, because the return
stack may also contain temporary data in addition to return addresses. Most
predefined functions, however, operate on the data stack.
An expression is written in reverse polish notation (RPN). The sentence 15 27
101 + for instance pushes the numbers 15, 27, and 101 sequentially onto the
data stack. The operation "+" then consumes the two topmost elements, 27 and
101, and replaces them by their sum. The data stack now contains the numbers
15 and 128. For readers unfamiliar with Forth or the Hewlett-Packard pocket
calculators, this may look strange. In practice, you soon get acquainted with
this type of operation.
Not everything in FUTURE86 is RPN, however, because this paradigm only applies
to the evaluation of programs. Compile time expressions, on the other hand,
are written in the more traditional infix notation. The following statements,
for instance, set MAXLENGTH to 1024.
 BUFSIZE EQU 256
 NO-OF-BUFFERS EQU 4
 MAXLENGTH EQU BUFSIZE
 * NO-OF-BUFFERS
In fact, every sentence may contain compile time expressions in infix notation
by enclosing them in parentheses, like this:
 17 (34 * 45 + 8) VAR @ OVER -
Other differences from Forth concern the dependency between words: Because
FUTURE86 was designed as a compiled language, two-pass compilation allows
extensive forward referencing. This even applies to different compilation
units: It is common practice for words in different modules to mutually call
each other.
FUTURE86 also contains an assembler. Unlike Forth, assembler statements --
which may be freely intermixed with FUTURE86 words -- follow the syntax
generally available on the target machine. On a 80186 system, you could for
example, write: BOUND BX,[TESTLOC] inside any high-level definition. It is
therefore the responsibility of the programmer to select the desired
abstraction level at every single line of code!
Other minor differences from Forth concern the usage of some reserved words.
The FUTURE86 version of LOOP functions, for example, is slightly different
than its Forth counterpart.
Now let's look at how the sample programs are written in FUTURE86. Listing
Three demonstrates one possible solution of the AUXOUT problem. The definition
of AUXOUT, as every definition of a new word, starts with a colon, followed by
the name of the function.
We assume that the character to be output is located on the stack. That is, to
send the escape character to the serial interface you write: 27 AUXOUT. The
predefined word !DL pops the top value from the stack and stores it in the
pseudo CPU DL register. Of course, words like !DL are hardware dependent. The
next word, AUXOUT-FUN, pushes this constant onto the stack, which now contains
the number 4 only; the 27 is still in pseudoregister DL. Then, the word
INT_21H is called. This function removes the top value of the stack (4), uses
it as DOS function code (that is, pushes it into AH) and performs an interrupt
(21 hex). The semicolon closes the definition.
At first, words like !DL seem strange for programmers getting used to
traditional languages such as Fortran or Pascal. In reality, however, even
this enhances readability. Because FUTURE86 --such as Forth or Lisp --is not
committed to the traditional "Letter+Digit" identifiers, one can devise more
meaningful names, like: IS-THERE-STILL-ROOM-IN-THE-BUFFER? which, by the way,
is a perfectly legal FUTURE86 word. In the case of !DL, one needs to
understand that the exclamation mark is generally used in the context "store
from stack into ...," while the character "@" does the inverse.
The SFILT program in Listing Four is more complicated. The function XTRANS was
--for testing purposes --defined as nonoperation, which is the shortest
possible function definition. To comprehend SFILT, you have to understand
FUTURE86's unique string concept.

A string always consists of two parts: A string buffer somewhere in memory,
holding left justified the string characters, and a control block called SINFO
(String INFOrmation), consisting of a pointer to the buffer and a length
field. For string operations the SINFO is stored in the data stack. The
simplest way to create a string is to place it within a quote, like this:
 : HELLO "Hello world!" SPRINT;
The execution of HELLO would allocate a place in memory, large enough to hold
all the characters, place the appropriate SINFO on the data stack and call the
word SPRINT (String PRINT), which causes the string to be printed.
Our definition of SFILT assumes that the SINFO's of the destination string T
and the source string S (in that order) are placed on the stack. On return,
these SINFOs should be replaced by the updated SINFO of the destination
string. This behavior of SFILT is depicted in the comment line labeled "stack
flow."
Following the C example SFILT starts by determining the source strings length,
which is the second part of its SINFO and therefore on top of the stack.
Because IF consumes its argument, this data item has to be duplicated by the
word SLEN. Many FUTURE86 functions remove their arguments from the stack, so
this is quite a common operation.
The next steps (CGET and XTRANS) remove the first character from the source
string and transform it. The following three lines temporarily save the
character to the return stack, interchange the stack position of the two
SINFOs, then restore the character to the stack.
The word C+ appends the character to the destination string. Then, the SINFOs
stack position is interchanged again SFILT is recursively called.
The ELSE part simply discards the now empty SINFO of the source string from
the stack to fulfill the functional description of SFILT. The word THEN
matches the IF and is similar to Fortran's ENDIF statement.
The lines below demonstrate the usage of SFILT. The destination buffer is
called DEST and may hold up to 80 bytes. This buffer is initialized with the
function SETUP.MAIN calls SETUP, then SFILT, and finally prints the resulting
string using SPRINT. This also shows how words work together: As mentioned,
SFILT leaves the destination string's SINFO on the stack which, in turn, is
consumed by SPRINT.


The CDL2 Language


CDL2 is a radically different language. It's precursor was CDL (compiler
description language), a generator used to produce compilers from a given
affix grammar. The first implementation of CDL took place at the Mathematical
Center in Amsterdam (The Netherlands), and soon it became popular at a number
of European universities.
In 1974 the language was extended by adding modularity and the ability to
serve as a general-purpose programming language --CDL2 was born. Since 1980 it
has become available outside the university community. It has been used for
numerous commercial applications: A Cobol compiler from the German company
MBP, the Prolog compiler MProlog, Mephisto (a chess computer), and EUMEL are
but a few examples. EUMEL is a German operating system featuring the
programming language Elan, which is targeted to education institutions (high
schools, universities, and so on).
There are four basic elements of a CDL2 program: actions, tests, predicates,
and functions. The most traditional form is the action: A piece of code that
performs some operations, possibly changing its environment. The function is a
similar construct, but preserving the environment like a mathematical
function. A test is a special function that delivers a Boolean result.
Depending on the outcome, succeeding statements in a clause are skipped or
executed. Finally, a predicate is an action that may either fail or succeed.
Failure causes backtracking so the behavior is similar to Prolog clauses.
Every building block (action, test, and so on) may itself call other building
blocks, in the same way FUTURE86 words may call other words. There are,
however, no primitives in CDL2! The programmer therefore has to define the
bottom layer himself. This is accomplished by defining an Action as a Macro,
that is, consisting of assembly language instructions only. The same is true,
of course, for tests, predicates, and functions. For simplicity, I refer in
the sequel to "actions" only when I mean all four kinds of building blocks.
Unlike FUTURE86, assembly statements and high-level CDL2 statements cannot be
intermixed when defining an action.
To put different actions together is, however, not sufficient to produce a
CDL2 program. The language was definitely designed for
programming-in-the-large, for projects requiring a large staff, and delivering
tens of thousands of code lines. Therefore, the design of the project is an
integral part of the language.
All actions belonging to the same topic are grouped in a SECTION. A set of
sections forms a LAYER (see Listing Five). Layers can be thought of as stacked
one on top of another. Each layer must explicitly state which actions are
exported to or imported from other layers. The top layer, for instance, could
represent the specification of the problem. The next layer then could be a
prototype, and by step-wise refinement, we reach the bottom layer, consisting
of the applications low-level words. Therefore, most assembly language will be
found in this bottom layer but not necessarily all: This is completely up to
the designer. So, variable-level programming may not only be accomplished by
thinking in words or statements but in a more general abstract entity, the
layer.
Different layers together form a MODULE. A module is vaguely related to
separate compilation units in other programming languages. In practice, each
programmer on a CDL2 project will work on one and only one module at a time.
Finally, different modules together form a PROGRAM. It should be clear by now
that CDL2 was designed to be only used for large applications. We therefore
only sketch our example programs by defining a few actions, leaving out the
lowest part and overall program organization.
Listing Six shows the layout of the AUXOUT action. As in the previous
examples, the DOS function code is defined by a compile time constant. Note
that for improved readability CDL2 identifiers may contain embedded spaces as
that constant has been called "AUXOUT FUN."
The action AUXOUT expects one parameter C, the character to be output. Every
CDL2 statement consists of a procedure evocation (where a procedure could be
action, test, predicate or function) and actual parameters are separated by
plus signs (+). Statements are separated by commas (sequential execution) or
semicolons (parallel alternatives).
The action header follows the same format but states the formal parameters. In
addition, the type of the parameter (input, output, or inout) is specified by
placing the '>' character to the appropriate side (left, right, or both) of
the parameter name. In this example C is therefore an input parameter. A colon
at the end of the header line specifies the ACTION as containing CDL2
statements. An equal sign would specify it as assembly language action.
The subsequent three lines load the registers and perform the interrupt. In
this particular example I also enclosed the definitions necessary to get the
assembly instructions. The CDL2 words used in this small example are defined
in the last three lines of Listing Six.
SFILT is only loosely sketched, because a complete CDL2 implementation would
be beyond the scope of this article. As can be seen from Listing Seven, the
destination string TARGET is defined as an out parameter because it will be
modified during evaluation of the action. Note also the different usage of
semicolon and comma to separate the statements.


Conclusion


Today there is no generally accepted opinion on how a programming language
should look. This begins with the schism between functional programming (no
assignment statements, no state change, no loops, no history), object-oriented
programming (every object has its own history) and traditional imperative
programming ("let Cobol and Fortran live forever!") and ends with debates if
Basic should be allowed/ forbidden in high schools. Industry programmers and
scientists agree only on the fact that we have some kind of software crisis
(we would like to do much more programming but we do not know how to manage
it).
Variable-level programming is certainly a good compromise for those projects
where performance considerations prevent the complete usage of a high-level
language. Very likely this will still be true for at least the next 20 years.
Due to its roots in grammar analysis CDL2 may attract even more attention in
the near future. As parallel processing becomes more feasible, rule-based
languages, like CDL2 or Prolog, could benefit immediately by new technologies
because they are easily expandable to cope with simultaneous exploration of
different, but parallel branches or rules.
In the next century we will likely have hardware that runs Smalltalk, Prolog,
or Miranda at the speed of a today's 8086 object code. Maybe then we will no
longer require FUTURE86, CDL2, or their future successors. My opinion,
however, is that the requirements of projects will increase --at more or less
-- the same rate of the performance of new hardware. The problems may shift
slightly, but they won't disappear.


Notes


The examples have been developed using the POWER C compiler by MIX Software.
This is one of the least expensive C compiler available but is complemented
with the debugger CTRACE, a very powerful development tool and probably the C
compiler with the best price/performance ratio available. For further
information, contact Mix Software, 1132 Commerce Dr., Richardson, TX 75081.
At present, only one implementation of FUTURE86 is available in the U.S.,
although in Japan a similar version is sold by RIGY Corp. The U.S. version is
available from Development Associates. It runs under PC/MS-DOS and contains
code generators for both the 8086 and 80186. Other instruction sets, like the
protected mode instructions of the 80286 processor, can be user defined with a
CODEMACRO facility.
The compiler comes complete with debugger (FDT86) and libraries of CGA
graphics, serial communications, Hayes and XModem protocols, and floating
point. Libraries are delivered in source code; thus, the graphics library may
be extended to accommodate EGA or VGA formats. An optional debugger (REM86) is
available for target use and functions via the serial communication line.
Although the debugger commands are line oriented and remind me of the now
obsolete Microsoft SYMDEB, it has a powerful operation mode called "F-mode."
In this mode, FUTURE86 commands may be entered interactively, and it is even
possible to compile new words. In this respect FDT86E acts similarly to an
interactive Forth environment and has proven useful for debugging the example
programs. For more information, contact Development Associates, 1520 South
Lyon, Santa Ana, CA 92705.
The implementation used was the PC-DOS version of CDL2LAB from the German
company epsilon. Several other versions are available from epsilon, including
versions for OS/2 (IBM PS/2), VMS (VAX), NOS(Cyber 180), BSD Unix (VAX), and
Unix V (Sun 3/50, PCS Cadmus, Nixdorf Targon). Code generators for cross
compilation are also available for TOS (Atari) and even the good old CP/M.
CDL2LAB extends the CDL2 compiler by providing an integrated environment,
grouped around a central database. Documentation is available in English. For
further information on CDL2, contact epsilon, Kurfurstendamm 188/189, D1
Berlin 15, W. Germany.


Availability


All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063; or call 800-356-2002 (from inside
Calif.) or 800-533-4372 (from outside Calif.). Please specify the issue number
and format (MS-DOS, Macintosh, Kaypro).


Variable-Level Programming
by Ronald Fisher



[LISTING ONE]

#include <dos.h>
#define AUXOUT 4 /* DOS function id for aux.output */

void auxout(char c)
/* Function: Sends character c to auxiliary output */

{ bdos(AUXOUT,c,0); /* generate INT 21 */ }







[LISTING TWO]

#include <string.h>
extern char xtrans(char c);

char *sfilt(char *source, char *target)
/* Appends source to target and applies function xtrans to */
/* every character appended. */
{
 if(*source) /* check if length of source > 0 */
 {
 /* find terminating 0 of target string */
 char *end_of_target = target+strlen(target);

 /* replace terminating 0 by transformed first char of source */
 *(end_of_target++) = xtrans(*source);

 /* add new terminating 0 to target - hope we have enough room */
 *end_of_target = '\0';

 /* drop leading character from source and call sfilt again */
 sfilt(strcpy(source,source+1),target);
 }

 /* we are done! */
 return target;
}







[LISTING THREE]

\ Synopsis: c AUXOUT
\ Stack diagram: c ---
\ Function: Sends c to serial output device AUX

AUXOUT-FUN EQU 4 \ Function code for serial output


: AUXOUT
 !DL \ load c into DL
 AUXOUT-FUN \ load function code
 INT_21H ; \ call DOS







[LISTING FOUR]

\ String transformation
\ Synopsis: destination, source SFILT
\ Stack flow: sinfo1 sinfo2 --- sinfo1'
\ Function: Appends string referred to by sinfo2 to string sinfo1
\ and applies transformation function XTRANS to every character
\ of the source string. Returns updated sinfo1.

DEST DSB 80 \reserve 80 char string buffer

TEMP DSB 80 \provide temporary buffer space for 2nd string

: XTRANS ; \empty definition of arbitrary transformation function

\input is sinfo1 sinfo2; output is sinfo1+sinfo2(transformed)
: SFILT

 SLEN \sinfo1 sinfo2 --- sinfo1 sinfo2 stringlength2
 IF \compare length of source string to zero
 CGET \ not zero: --- sinfo1 sinfo2' char
 XTRANS \ apply transformation: --- sinfo1 sinfo2' char'
 \stack has sinfo1, sinfo2, trsanslatedchar
 >R \save chr
 SSWAP \sinfo2 sinfo1
 R> \get chr
 C+ \append chr to sinfo1
 SSWAP \sinfo1 sinfo2
\next instruction is assembler...if we use high level call linkage
\there is danger of return stack overflow...this way is stable
 JMPS SFILT \recursively call til entire string scanned
 ELSE
 SDROP \ string exhausted...discard sinfo
 THEN ; \complete


: DEST-INIT \ --- sinfo1
 "You know that " ;

: SOURCE \ --- sinfo2

 "Some different alpha characters are commonly used in Germany." ;

: SETUP \ --- sinfo
 DEST-INIT DEST SCOPY ; \copy string to string buffer

\main procedure
: MAIN

 SETUP \initialize destination area
 SOURCE SFILT \translate and append source
 SPRINT ; \output result to console

\ Define MAIN as startup routine:
END MAIN







[LISTING FIVE]

MODULE string handling.
 LAYER main layer.
 SECTION string transformers.
 ....
 ENDSEC string transformers.

 SECTION char transformers.
 ....
 ENDSEC char transformers.
 ENDLAY main layer.
ENDMOD string handling.








[LISTING SIX]

CONST auxout fun = 4. # DOS function code 'serial output' #

# Synopsis: auxout +c #
# Function: Sends character c to device AUX #
# Note: AUX defaults to COM1. This assignment may be #
# changed by the MODE command of DOS #

ACTION auxout +>c :
 loadax +auxout fun,
 loaddl +c,
 int21.

# The syntax of the following terminal productions depends #
# on the code generator used. #

ACTION loadax +>value = "MOV AX," value.
ACTION loaddl +>value = "MOV DL," value.
ACTION int21 = "INT 21H".







[LISTING SEVEN]

# Synopsis: sfilt +source string +target string #
# Function: Appends source to target string by applying #
# transformation 'xtrans' to every character of #
# source string #
VAR first char . # temporary variable to hold one char #

ACTION sfilt +>source +>target> :
 nullstring +source; # finish when source=NULL #
 (but first +source +first char,# else drop first char #
 xtrans +first char, # translate it #
 append char +first char+target,# append to target #
 sfilt +source +target). # repeat the process #

# Refinement. Note that the following base words have to be #
# expanded by appropriate assembler statements. #
TEST nullstring +>string = ........ # test if string=NULL #
ACTION but first +>string> +char> = .........
 # move head of string to char #
ACTION xtrans +>a char> = ......... # translate 'a char' #
ACTION append char +>char +>str>=.. # append char to str #







































June, 1989
OPTIMIZATION TECHNOLOGY


Code optimizers are one way to take the drudgery out of programming




Keith Rowe


Keith is a programmer with HCR, and he can be reached at HCR Corporation, 130
Bloor Street West, Toronto, Ontario, Canada M5S 1N5.


If you ask any programmer what are the three chief design goals of his next
project he'll probably answer, "speed, speed, and more speed." Writing faster
code is something we all want to do, but it usually means a lot of extra,
tedious work. Writing faster code also requires the hard-won knowledge of the
underlying architecture and in the modern heterogeneous computing environment,
it means learning the internals of a number of different machines.
Optimizers, however, let you create faster code without learning the innermost
secrets of a particular configuration. This article explains how optimizers
work and highlights some of the benefits you can expect from them.


What is an Optimizer?


An optimizer is a program that takes as input a code sequence and generates a
second code sequence which is identical in effect but is more efficient in its
use of memory space or in its execution time. In today's machines, memory
space is a plentiful resource, so most optimizers concentrate on execution
time. The term "optimizer," however, can be a misnomer, because there is no
guarantee that the resulting version of the code is the fastest version
possible.
In general, the optimizer examines the input code sequence for patterns of
instructions that can be replaced with more efficient patterns. These
replacements are called transformations. Valid transformations must have three
properties: 1. Transformations must be correct. The resulting code sequence
must have the exact same semantics as the original program. 2. Transformations
must be more efficient. The average execution time of the transformed program
must be faster than the original for most cases. (Note that in some cases some
transformations may slow down the code, rather than speed it up. The designer
of the optimizer must carefully examine such risky transformations to see if
they are justified.) 3. Transformations must be worth finding. Some
transformations take a great amount of computing resources to uncover. If the
speedups they provide are minimal, they may not be worth the effort --
especially if the code being optimized will be run infrequently or is still
being tested and changed.
A simple example of a valid transformation is constant folding. Expressions
which can be seen to evaluate to a constant value, are calculated during
optimization and replaced with the result. For instance, a=3+6*5 would be
replaced with a=33. This clearly has the same effect --a still has a value of
33 --and the assignment of the value directly saves an addition and a
multiplication operation (along with some register operations), so it is more
efficient. Because transformations like this one are easy to spot and they
will always speed up the executing code, they are certainly worth finding.
Optimizers are closely associated with compilers. Optimization can be thought
of as an additional pass (or in some cases passes) the compiler makes to
improve the code. This pass may occur between parsing and code generation
passes or after code generation, depending on the type of optimization being
performed.
The most obvious reason to use an optimizer is to create faster code. The
optimized executable performs the same task in fewer cycles and thus increases
the throughput of the machine. In a way, this is like buying a cheap form of
"upgrade" for your system --more gets done on the same machine without
spending a lot of money on new hardware. Beyond the obvious speed-up of
executable code, there are other advantages to using an optimizer that make
themselves felt at all points in the software development cycle.
With an optimizer, much of the often tedious work of code tweaking is done
automatically and with greater consistency. Note that automatic optimization
is unlikely to be more efficient than exquisitely hand-tuned code, but it
requires far less work from the programmer to produce a similar result. Also,
no optimizer will automatically replace a bubble sort with a quick sort -- the
responsibility of selecting an appropriate algorithm still rests firmly with
the programmer.
Optimizers also allow code to be more readable and more portable. Most
programmers have seen code that has had to do such unusual things to run
faster that the functionality of the code is almost impossible to fathom. An
optimizer can do many of these rewrites during compilation of the code, which
means the programmer can concentrate on making the algorithm clear and the
logic clean, which improves code maintainability. Similarly, many
machine-specific "tricks" to improve code can be kept in the optimizer so that
the original source code can be more easily ported.
Finally, optimizers allow hardware designers to produce more efficient
architectures. RISC machines like the MIPS and the IBM RT rely on the fact
that they have optimizing compilers to ensure that the machine code that they
execute takes full advantage of their unique architectural features and avoids
their areas of weakness as much as possible.


Global and Local Optimization


Optimizers can be designed to look for transformations at two main levels:
global and local. Global optimization seeks to modify the source code (or the
compiler's intermediate representation of it) to find transformations that are
apparent only when a function is examined as a whole, whereas local
optimization usually operates on small pieces of assembly code to find
improvements of a more immediate nature.
A great number of profitable transformations can be made by looking only at
small pieces of assembly code without needing to understand the program as a
whole. Programs that find these localized, low-level transformations are
called peephole optimizers or sometimes just peepholers.
Figure 1 shows two typical code sequences (in IBM RT assembly code) and their
replacements. In the first case, an unconditional branch is made to the
immediately following line. This commonly occurs when a compiler is creating
jump tables and can be eliminated. The removal of the branch instruction is
correct, more efficient and trivial to find, and making it an ideal
transformation.
Figure 1: Two code sequences and their respective optimized replacements

 Original Code Replacement
 -------------------------------
 b L.16 L.16: ... L.16: ...
 -------------------------------
 st 0, 16(14) st 0, 16(14)
 l 1, 16(14) lr 1, 0

In the second case, register 0 is spilled out to memory and this memory
location is immediately reloaded into register 1. Like most machines, memory
accesses are slower than register transfers on the RT. It is therefore
appropriate to replace the regular load with a register load.
Notice that in both cases finding the transformations did not require anything
more than looking at the immediately surrounding code. Peepholers do not need
to do a lot of analysis and so they can be quite quick. In general, a
peepholer makes the compiler's job much easier because the compiler doesn't
need to keep excessive state information.
There is a much larger class of transformations that can be found only when an
entire function is examined at one time. Global optimizers can find these
high-level transformations by analyzing the semantic content of a given
function. Constant folding is an example of a global optimization.
Global optimizers can be designed to work directly with the source code, some
intermediate representation produced by the compiler, the assembly code, or
even the assembly output. If you choose to work with the assembly code, a
great deal of the high-level semantic content is obscured, but working
directly with the source code requires a parsing and analysis phase similar to
standard compilation. The <ENTRY>optimizer I work with (HCR's Portable Code
Optimizer, PCO) takes the intermediate code from the first pass of the
compiler and generates an optimized intermediate code representation, which
can be sent on to the second pass for code generation. Because this
intermediate code is semantically similar to source code, but is in a more
easily analyzable format, PCO has the advantage of being able to work at a
source code level without the overhead usually associated with it.
There are a number of differences in the use of global and peephole
optimizers. Peepholers execute quickly but global optimizers, which need to do
a more complete analysis of the code, tend to take much longer. This makes
global optimization something best left undone during the iterative test and
debug cycle of program development.


A Closer Look at Global Optimization


There are quite a few transformations that can be made by a global optimizer
and, depending on the architecture of the machine, some may be done and others
may not. As an example of the kind of transformations that a global optimizer
can perform, let's look at two functions from a typical piece of C code shown
in Example 1. This code sets the first five values of array a to a value. One
interesting feature of the calculation of the value is that the division is
kept as a separate function that checks the divisor and issues an error
message if it is zero. Although, this example will show the results of
intermediate steps in C code, you should remember that all the transformations
occur in the optimizer, in its own internal representation.

Example 1: Sample code before optimization

 set_array (j)
 int j;
 {
 int x,n,i;
 x = 5; n = 2;
 for (i=0;i<5;i++) {
 a[i] = testDiv (x,n) + j;
 }
 }
 testDiv (a,b)
 int a;
 int b;
 {
 if (b != 0 ) {
 return ( a/b);
 } else {
 fprintf(stderr, "division by zero\n");
 return( 0 );
 }
 }

The first transformation performed is inlining. If a function is very short
and is called in only a few places it may be worthwhile to write out the whole
function wherever it is called, and thus save the overhead of the function
call. (C++ provides inlining as a standard part of the language, but C does
not, so an optimizer is needed to make this change.) Example 2 shows a version
of the code with testDiv inlined.
Example 2: Same code as in Example 1 but with function testDiv inlined

 set_array (j)
 int j;
 {
 int x,n,i;
 x = 5; n = 2;
 for (i=0;i<5;i++) {
 int temp;
 if ( n != 0 ) {
 temp = x/n;
 } else {
 fprintf(stderr, "division by zero\n");
 temp = 0;
 }
 a[i] = temp + j;
 }
 }

The decision to perform inlining involves many factors. Because the function
is short it will not expand the size of the executable to a great degree. The
saving of one function call would not be that important but in this case it is
within a loop. Any saving inside a loop is magnified by the number of times
the loop repeats. In this case, inlining is a profitable transformation but
the breakpoint, where it becomes unprofitable, depends to a great degree on
the underlying architecture. Finding this breakpoint is part of the tuning
that takes place when an optimizer is designed.
The second transformation involves constant propagation and constant folding.
The use of immediate values is more efficient than accessing memory, and any
expression that can be evaluated at compile time will save cycles at runtime.
Thus, it is important to find all the places in the code where values can be
predicted. A flow analysis of the variables in a piece of code can show where
they are set and reset. In our example, x and n are set at the beginning and
never changed --they act as constants. Thus, the optimizer can propagate the
constant value of these variables to every place they are used. The value of i
cannot be predicted, so it must remain.
Notice that when x and n are replaced with their values we get the expression
temp=5/2. Here is another opportunity for constant folding. Constant
propagation and constant folding often have this kind of interdependence
between them. Each transformation may provide opportunities for other
transformations that did not exist before.
For example, the replacement of n in the if statement (Example 3) makes the
conditional a constant, which allows us to make another valuable
transformation --dead code elimination. The code in the else branch of the
statement is now unnecessary and so it and the if statement can be removed.
Dead code elimination wins on two counts: it reduces the size of the
executable and it saves performing an expensive test and branch operation.
Example 3: The replacement of n in the if statement makes the conditional a
constant to make dead-code elimination optimization possible

 set_array(j)
 int j;
 {
 int i;
 for (i=0;i<5;i++) {
 int temp;
 if ( 2 != 0) {
 temp = 2;
 } else {
 fprintf(stderr, "division by zero\n");

 temp = 0;
 }
 a[i] = temp + j;
 }
 }

After eliminating the dead code the value of temp won't change inside the loop
and so its value can be propagated into the next statement leaving us with the
code in Example 4.
Example 4: The code in Example 3 after dead code has been eliminated

 set_array(j)
 int j;
 {
 int j;
 for (i=0;i<5;i++) {
 a[i] = 2 + j;
 }
 }

Now we have an expression, 2+j, which can't be folded because the value of j
cannot be predicted, but still it is wasteful to repeatedly evaluate it every
time the loop executes because its value will not change. An optimizer can
move blocks of code like this outside the loop using a transformation called
loop invariant code motion. In this instance, the profit gained by the
transformation may be marginal and the replacement may not have taken place at
all if the optimizer had been tuned differently, but in general, code motion
is an important source of increased code efficiency.
At last we have the code in Example 5. Although both Example 1 and 5 have the
same net result, the number of assembler instructions the code executes on an
IBM RT drops from 156 to 122, an improvement of 21 percent. Note that
instruction counts do not give an exact representation of the improvement
because the timing of the instructions must also be considered, but on the RT
most instructions are identical in execution time. This code is representative
of the kind of improvement that can be expected with global optimization.
Example 5: Both Example 2 and 5 have the same net result, the number of
instructions the code executes improves by 21 percent

 set_array(j)
 int j;
 {
 int temp2,i;
 temp2 = 2 + j;
 for (i=0;i<5;i++) {
 a[i] = temp2;
 }
 }



Implementing a Global Optimizer


Global optimizers are large and complicated pieces of code. (The PCO, for
instance, is half again as large as the rest of the compiler with which it
works.) This is reasonable because the understanding of the code required for
optimization is much greater than for compilation. Rather than trying to
understand all of this code, let's focus on a few of the principal data
structures and show how they are used to find optimizing transformations.
The first step in analyzing the code is to break it into basic blocks. A basic
block is a sequence of instructions that execute without any transfer of
control occurring within the block. Most of the work of the optimizer is
performed on these blocks or on structures built up from them. Basic blocks
are used to create a flow graph, which adds the flow of control information to
the set of basic blocks. Each block is a node in the graph with each directed
arc representing a possible transfer of control from one block to another.
Figure 2 shows the translation of a typical piece of code into a flow graph.
Notice how the use of loops and if statements both effect the flow of control.
Flow graphs are a much easier representation to work with because the powerful
techniques of graph theory can be used to analyze the relationships between
the various nodes (or in this case, basic blocks).
For example, the problem of finding a loop in the code is translated to
finding a strongly connected subgraph (a subset of nodes in the graph such
that each node in the set can reach all other nodes in the subset) with a
unique entry point. Although this doesn't sound easier, it is well-defined
mathematically, and graph theory provides algorithms to find such a subgraph.
An important extension of the flow graph is the data flow graph. A data flow
graph is created by taking a flow graph and adding information about the
variable usage to each node. The information added varies, depending on the
use the data flow graph is put to, but the information is usually one or more
sets of variable names added to each of the blocks.
Dead code elimination uses a type of data flow analysis called live variable
analysis. If a variable is defined (appears on the left hand side of an
assignment) and is not used subsequently, the defining statement can be
eliminated. Because all the statements in a basic block occur in sequential
order, it is fairly easy to detect these cases within a single block. However,
what if the variable is used in a later piece of code? Detecting this case
requires a complete analysis of the use and definition patterns of the code
and data flow analysis becomes essential.
In live-variable analysis, each of the blocks in the flow graph has two sets
of variable names: in and out. A variable is considered to be live in a block
if it is a member of the out set of the block. The out set of a block is the
union of all the in sets of the block's successors. The in set of a block
contains all the variable names used in an expression in the block, as well as
all the names in the out set that are not defined in the block. The last
definition of a variable in a block can be eliminated as dead code if the
variable name does not appear in the out set of the block.
Figure 3 shows a data flow graph created for live variable analysis. If you
imagine that the variables c and d are used in a later piece of code, the in
and out sets will be as shown. The bottom block has the same in set as its
successor's out set but adds an a to the in set to reflect its use in that
block. The middle two blocks both remove the c from their in sets, because the
c is defined in those blocks. The top block is of particular interest, because
the definition of b occurs and there is no entry for it in the out set. Thus,
the line b++; is dead code and can be eliminated. Other forms of data flow
analysis are used to perform loop invariant code detection, copy propagation,
and some types of common subexpression removal.
The structures discussed so far are used to analyze large pieces of the code.
Many transformations can be found within individual basic blocks and another
data structure: The directed acyclic graph is used in this case. A directed
acyclic graph (a dag) is a data structure where each node represents either a
variable or an operator. Each operator node will have pointers to other nodes
to represent the operands. The dag is constructed in such a way that no
circular references will ever appear. A dag does not model control flow, so it
is perfect to form a more easily analyzed representation of the basic block.
A dag starts with a collection of leaf nodes, one for each variable used in
the block. The statements in the block are examined one at a time and for each
operator, a node is created. The children of this node are the leaf nodes of
the variables that are the operands. If the result of the operation is stored
in a variable, the node is labelled with the variable's name. The next time
this variable is used, the new node points at this operator node rather than
the original leaf node. If a node to be added has identical children and an
identical operator, it is not added to the dag, but the variable it is
assigned to is added to the variable list of the existing node. Figure 4 shows
a dag created for a typical instruction sequence. The second use of d uses the
same leaf node, but the second use of b does not because it has been used to
label the '+' operator node.
One use of the dag is to assist in dead code elimination. If a node has no
parents, the variable it is labelled with has the value of the tree under it
when the block completes. When we know this variable is dead (using the
data-flow analysis techniques shown above) then this node can be eliminated.
This in turn may leave other nodes without parents and expose them to similar
elimination. Looking again at Figure 4, if c is dead in the following block,
the '/' node can be removed. This leaves the '+' node without a parent, and if
b is also dead it can now be removed, thus cascading the dead code
elimination. Dags are also used for most other transformations that occur
within a block.


Performance


To demonstrate the kind of speedups that the average user can expect when
using a good optimizer, I have done a few brief experiments. Like all
benchmarking, these tests should be taken with a grain of salt, but they do
give an idea of the range of performance improvement available. As I said
before, optimization is machine specific. Some machines provide more latitude
for improvements than others. As the car makers say "your mileage may vary."
The benchmarks were done on an IBM RT, model 115, with an AFP card installed.
This RT was running Version 2.2.1 of AIX and used the standard cc compiler.
(This compiler includes a "perform optimization" option, which invokes PCO as
a global optimizer and IBM's copt as a peepholer.)
The first test was to compile and execute the standard Dhrystone benchmark
with and without optimization. Because many of the statements in the benchmark
have no effect, the global optimizer can perform a great deal of dead code
elimination and thus the Dhrystone/second rating increases by about 75 percent
(see Table 1).
Table 1: Performance results with and without optimization

 Program No. Opt. Optimized Speed-up
 -------------------------------------------------

 Dhrystone (dhry/s) 4838 8474 75%
 grep (s) 18.74 15.88 18%
 sort (s) 38.56 33.6 14%
 yacc (s) 14.95 12.19 22%

To get a better feel for what happens to real code, I took source for the
common Unix utilities grep, sort, and yacc, compiled them with and without
optimization and set them to large tasks. Although none of these utilities are
perfect, they do represent better than average attempts at writing fast code.
grep was asked to search a 2.5-Mbyte on-line dictionary for a particular word
pattern. The optimized version ran 18 percent faster over 10 tries. sort was
asked to put a smaller on-line dictionary into reverse alphabetic order. Here
the speedup was 14 percent. yacc was given a C grammar and asked to generate a
parser for it. In this case the speedup was 22 percent.


Conclusion


Optimizers have proven to be a valuable tool for the software developer. They
help remove the drudgery of writing efficient code, allow the code to be more
readable and portable, and even in tight code, they can find significant
improvements. Still, they can only do so much, and the most intellectually
challenging part of creating fast code, the design of good algorithms, will
remain with the programmer.


Further Reading


For a more complete explanation of optimization techniques and their
implementation look for Compilers, Principles, Techniques, and Tools, Aho,
Sethi, and Ullman (Addison-Wesley, 1986) or Crafting a Compiler, Fischer,
LeBlanc (Benjamin/Cummings, 1988).


Optimization Technology
by Keith Rowe



Example 1:


set_array(j)
int j;
{
 int x,n,i;
 x = 5; n = 2;
 for (i=0;i<5;i++) {
 a[i] = testDiv(x,n) + j;
 }
}

testDiv(a,b)
int a;
int b;
{
 if (b != 0 ) {
 return ( a/b);
 } else {
 fprintf(stderr,"division by zero\n");
 return( 0 );
 }
}

Example 2:

set_array(j)
int j;
{
 int x,n,i;
 x = 5; n = 2;
 for (i=0;i<5;i++) {

 int temp;

 if ( n != 0 ) {
 temp = x/n;
 } else {
 fprintf(stderr,"division by zero");
 temp = 0;
 }
 a[i] = temp + j;
 }
}



Example 3:

set_array(j)
int j;
{
 int i;
 for (i=0;i<5;i++) {
 int temp;
 if ( 2 != 0 ) {
 temp = 2;
 } else {
 fprintf(stderr,"division by zero");
 temp = 0;
 }
 a[i] = temp + j;
 }
}

Example 4:

set_array(j)
int j;
{
 int j;
 for (i=0;i<5;i++) {
 a[i] = 2 + j;
 }
}

Example 5:

set_array(j)
int j;
{
 int temp2,i;
 temp2 = 2 + j;
 for (i=0;i<5;i++) {
 a[i] = temp2;
 }
}




































































June, 1989
WRITING AWK-LIKE EXTENSIONS TO C


When neither AWK nor C can do the job, try a combination of the two




Jim Mischel


Jim Mischel is a former financial systems programmer and database consultant.
He can be reached at 20 Stewart St., Durango, CO 81301 or on CompuServe:
73717,1355.


The process of string searching is an important part of many computer
applications. Text editors, database managers, spreadsheet programs, and
countless utility programs use string searching to one degree or another.
Unfortunately, other than the common case-insensitive search, most programs
require that the search string be specified exactly. Very few programs allow a
search on strings that are described by such information as "all strings that
begin with a capital letter, contain at least one digit, and end with a
period." This restriction isn't very surprising, considering the inadequacy of
the string search functions in most popular languages.
The AWK programming language is unique with respect to string search
functions. Along with many other interesting features, AWK contains the
facilities to search a string for substrings that match a regular expression
(see the accompanying sidebar on regular expressions). AWK is a fantastic tool
for in-house use, but there are drawbacks to using it for commercial software
development. First, AWK is interpreted, which makes its programs execute more
slowly than compiled code. In addition, the interpreter is an extra-cost item
that must be included in order for the program to run. Second, AWK provides
little access to either the operating system or the underlying hardware. In
the following discussion, some familiarity with the AWK programming language
will be helpful, but is not required.


AWK's Functions


AWK is designed around an implicit processing loop that automatically reads
each input record, splits the record into fields, and then performs
programmer-specified actions upon the record. AWK also provides global
variables that contain several bits of information about program status: the
number of records read, fields in the current record, and so on. This
functionality can be nearly duplicated in C by means of a simple controlling
loop and the AWKLIB getline( ) function, as shown in the QSTRING.C program in
Listing One. (getline( ) and the other AWKLIB functions are declared in the
header file AWKLIB.H, which is shown in Listing Two. AWKLIB.H must be included
in any program that uses AWKLIB.)
The awk_init( ) function, which must be called before any of the other
routines are used, initializes the AWKLIB global variables. Failure to call
this function may produce some interesting (and probably erroneous) results.
getline( ) has the same calling sequence as fgets( ) but returns EOF on
end-of-file or error, rather than returning NULL. (This change in return
values was a purely selfish design decision on my part.) getline( ) reads the
next input record (line) into memory, splits that record into fields on the
global field separator FS (discussed later), stores the fields in the FIELDS[]
array, and updates the field counter (NF). Unlike AWK routines, these C
routines do not allow the record separator to be changed, and they assume that
the record separator is a newline \ n.
The match( ) function searches the supplied string for substrings that match
the supplied regular expression. This function returns a pointer to the
beginning of the matched substring, or else returns NULL if no match is found.
match( ) also updates the RSTART and RLENGTH variables. The following example
returns a pointer to the s in she:
 char *c;
 c = match ("We followed wherever she went", "s?he");
In this example, RSTART is 22 and RLENGTH is 3.
The functions sub( ) and gsub( ) provide string substitution capabilities.
sub( ) substitutes the replacement string for the first occurrence of a
specified substring, and gsub( ) performs the substitution for every
occurrence of the substring. Both of these functions return the number of
substitutions that were made. For example, the following returns 1, and s
reads "William took Bill's book."
 char s[80] = "Will took Bill's book";,
 sub ("[WB]ill", "William", s);
The next line then returns 2, and s reads "William took William's book:"
 gsub ("[WB]ill", "William", s);
Notice that gsub( ) makes only one pass through the string, and it doesn't try
to substitute strings that have already been substituted (which is a good
thing, because such a step would send the above example an infinite loop).
Both sub( ) and gsub( ) allow you to use the "&" operator to reference the
matched substring in the replacement string. The following example appends
"ed" to occurences of the word "want" in a sentence by modifying s to read "I
wanted to go home!"
 char s[80] = "I want to go home!";
 gsub ("[Ww]ant", "&ed", s);
In order to include the "&" character literally in a string, preceed that
character with the escape character. The following code modifies s to read "I
&ed to go home!"
 gsub ("[Ww]ant", "\ &ed", s);
Neither sub( ) nor gsub( ) affect any of the global variables.
The split( ) function allows you to split a record into fields on a specified
field separator, which can be any regular expression. (The global field
separator, FS, defaults to "[ \ t]+," which means "one or more spaces or
tabs." FS may be changed by using the setfs( ) function.) The function
getline( ) uses split( ) and the global field separator in order to split
input lines into the FIELDS[] array. split( ) can also be called by the
application program, which may specify its own fields array or else use the
global FIELDS. split( ) updates NF and then returns NF. The following example
sets NF to 4 and returns 4:
char *a[128]; /* enough room for 127 fields */
split ("Will::took:Bill's:book.", a, ":+");
In this example, the a[] array is:
 a[0] - "Will::took:Bill's:book."
 a[1] - "Will"
 a[2] - "took"
 a[3] - "Bill's"
 a[4] - "book."
Note that the first element of the array always contains the unmodified
string. When calling split( ), make sure that the fields array contains enough
spaces for all of the fields in the record. Failure to include enough spaces
will crash the program because split( ) handles allocation of memory for each
field but cannot allocate memory for more fields.


The Functions at Work


The AWKLIB routines (Listing Three) are divided into three groups:
1. the compiler (makepat( ) and related functions) 2. the interpreter (match(
) and related functions) 3. the user interface (match( ), split( ), sub( ),
gsub( ), and their re_...alternates).

The compiler is based upon a simple recursive-descent design. (Because this
article is not about compilers, I won't cover the compiler specifics.) For
more information on how compilers are built, consult the references listed at
the end of this article. Input to the compiler consists of a human-readable
regular expression. The compiler outputs the regular expression in prefix
form, which is used more easily by the matching functions. The compiled form
is very much like the form presented in Kernighan and Plauger's Software Tools
in Pascal (which, by the way, is an excellent book).
The simplest form of regular expression, which is a literal character such as
"a," is compiled into an expression such as "cae," which means "Match literal
character 'a' and then end." A string literal is compiled into a series of
literal characters. For example, "hi" becomes "chcie." The way that the
interpreter handles this process will be discussed shortly.
The process of compiling character classes is also fairly simple. A compiled
character class is represented simply by a "[" (or "]" if the character class
is negated), followed by a count of characters, followed by the characters
themselves. For example, the character class "[0-9A-Fa-f]," which specifies a
hexadecimal digit, is represented by "['22'0123456789ABCDEFabcdefe." Note that
the 22 is a single byte value, chr(22).
(This encoding scheme restricts the lengths of character classes and closure
expressions -- neither may be longer than 255 characters. This restriction
presents only a minor problem in the case of character classes because there
are only 256 character codes. Any character class that contains N characters
can be expressed as a negated character class of 256-N characters, or a
maximum of 128.)
This process of handling alternation is a little more complex. The regular
expression "a b" compiles to " caecbee." In this compiled expression, the 'e's
tell the matching routines where to stop scanning. Alternation was the last
item that I included in my compiler, and it created some problems that I
hadn't anticipated. This operator forced the inclusion of the end-of-term
('e') characters in the compiled strings.
The compiled form of a closure is much like the compiled form of a character
class. The compiled representation of a closure consists of the closure
operator (?, *, or +), followed by the length of the expression, followed by
"e." For example, "(a b)*" is encoded as "*'7' caecbee." Note again that the 7
actually is chr(7). With the above discussion in mind, let's take a look at
how the matching functions work.
In a real sense, the matching functions, starting with match_term( ), make up
an interpreter --these functions run a program (the compiled regular
expression) against input data (the string to be searched) in order to produce
output (the matched substring). The interpreter is based upon a fairly simple
design (thanks to the pre-processing done by makepat( )) and holds very few
tricks. In fact, the only pieces that gave me any real trouble were the
alternation processing (" " operator) and the closure matching.
The heart of the interpreter is the match_term( ) function, which determines
what is to be matched and then, depending upon the operator, either attempts
to match the next character or else dispatches to the proper routine for
further processing. "Further processing" is either a character class,
alternation, or a closure operation.
Character classes are easy to match by using match_ccl( ), which looks for a
match by scanning the list of characters. match_ccl( ) then returns the proper
value, depending upon if a match was found and whether that match is a negated
character class.
More Details.
Alternation matching was the most difficult part for me to implement
correctly. There is probably a better way to handle this step, but my solution
works well and is fast enough for my purposes. The match_or( ) function breaks
the remaining pattern into two patterns (one pattern for each branch),
searches both branches, and then returns the longest substring that matched,
if any matches occurred. For example, the match_or( ) breaks the regular
expression "(b c)d," which reads " cbeccecde" when compiled into the two
patterns "cbcde" and "cccde," and then uses each pattern to search the string.
The step of splitting the patterns is performed by the skip_term( ) function.
Closure is handled by generating the longest possible substring that matches
the closure operator, then backtracking to discover where the closure should
stop in order to correctly match the rest of the pattern. Because of the
amount of backtracking that's involved, closure can become very inefficient.
This is especially true when nested closure operators, such as (a*b)*, are
involved. Since closure, by definition, can match a null string, making this
process work correctly for all cases was a little difficult.
I had originally designed the match_closure( ) function to return a Boolean
value, but ran into trouble during testing when I (inadvertently) tried a
pattern that contained a double closure operator ((a*)*). According to the
automata theory books, "(a*)*" is the same as "a*" --but my program didn't
know that. Consequently, match_closure( ) kept looping between the failure of
the outer closure and the success of the inner closure. After spending several
fruitless days attempting to have the compiler optimize the expressions (which
is quite a bit more difficult than it looks), I finally hit upon the simple
solution. Why not add another return to match_closure( ) to indicate that the
closure matches a null? Sometimes the simplest solutions are the most
difficult to come up with.


Using the Functions


The QSTRING.C program (in Listing One) locates and outputs all quoted strings
into a C source file. This is the first version of a utility I've been working
on; although QSTRING.C is primitive, it illustrates the use of the AWKLIB
routines.
The most important point here is that awk_init( ) should always be called at
the beginning of any program that uses the AWKLIB functions. Failure to do so
will cause the functions to act erratically.
The makepat( ) function has been provided so that the program can compile a
regular expression and then save the compiled form of the expression for later
processing. The routines re_match( ), re_sub( ), re_gsub( ), and re_split( )
use a compiled pattern in lieu of a regular expression in order to speed
processing. The two program fragments shown in Example 1 produce the same
output, but Program A executes much faster because it only compiles the
regular expression once. In contrast, Program B compiles the regular
expression each time that match( ) is called. Typically, a program will
attempt to match multiple strings against one or two regular expressions. In
this case, the program executes much faster if the regular expressions are
compiled first with makepat( ), and then re_match( ) (rather than match( )) is
used inside the processing loop. This increase in program speed also holds
true with respect to use of the other major functions: split( ) (use re_split(
)), sub( ) (use re_sub( )), and gsub( ) (use re_gsub( )).
Example 1: Program A executes faster because it only compiles the regular
expression once, while Program B compiles the regular expression each time
that match( ) is called.

Program A Program B

... ...
while (getline (line, 80, f) != EOF) { makepat ("a+bc", pat);
 if (match (line, "a+bc") != NULL) while (getline (line,
 puts (line); 80, f) != EOF) {
} if (re_match (line, pat)
 != NULL) puts (line);
... }
 ...
Notice in the example program that gets( ), rather than getline( ), is used to
read the file. Because getline( ) must split the input record into fields,
it's much slower than gets( ). Use getline( ) only when the record must be
split into fields on every (or most) input records.
The RLENGTH global variable extracts the matched substring from the rest of
the string and updates the string pointer for the next match.


Conclusion


This entire project has been quite an education in compilers, interpreters,
and automata theory. Construction of the compiler was made much easier by a
tutorial series on compiler design that I found in the Computer Language Forum
on CompuServe. While attempting to make the compiler optimize regular
expressions, I had to delve into automata theory -- and I intend to pursue
that interesting subject further.


Bibliography


Aho, A.; Kernighan, B.; Weinberger, J. The AWK Programming Language. Reading,
Mass.: Addison-Wesley, 1988. Aho, A.; Ullman, J. Principles of Compiler
Design. Reading, Mass.: Addison-Wesley, 1977.
Crenshaw, Jack W. "Let's Build a Compiler," Parts I-VII, unpublished papers,
(available for electronic download from CompuServe, Computer Language Forum)
Reading, Mass.
Hopcroft, J.; Ullman, J. Introduction to Automata Theory, Languages, and
Computation. Reading, Mass.: Addison-Wesley, 1979.
Kernighan, B.; Plauger, P. Software Tools in Pascal. Reading, Mass.:
Addison-Wesley, 1981.


Availability


All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063, or call 800-356-2002 (from inside
Calif.) or 800-533-4372 (from outside Calif.). Please specify the issue number
and format (MS-DOS, Macintosh, Kaypro).


Regular Expressions




The regular expressions used by AWK and similar programs are derived from the
notation used in automata theory to describe formal languages and finite state
machines. Regular expressions consist of normal characters (operands), such as
"a," "0", and "," plus meta-characters (operators), such as "+," "," and "[."
A regular expression, which is similar to familiar arithmetic expressions, can
be either a basic expression or a complex expression that is formed by
applying operators to several basic expressions.
The regular expression meta-characters and their uses are:
 \
The escape character is used in escape sequences to specify characters which
would otherwise have no representation. These escape sequences are similar to
those used in the C language:

 \b Backspace
 \t Horizontal tab
 \n Newline or linefeed
 \f Newpage or formfeed
 \r Carriage return
 \ddd Octal value "ddd." ddd can
 be one to three digits long.
 \c "c" can be any character
 that is taken literally

^ The carat matches the beginning of a string. For example, "^a" matches only
those strings that have "a" as the first character. When "^" is the first
character in a character class, the character denotes a negated character
class.
$ The dollar sign matches the end of a string. For example, "z$" matches only
those strings that have "z" as the last character.
. The period matches any character. Be careful with this one ".*" matches
everything.
[ The open bracket denotes the beginning of a character class.
] The closed bracket denotes the end of a character class.
 This character is the alternation operator. "a b" matches either "a" or "b".
( ) Parenthesis are used to group expressions in much the same manner in that
they are used with arithmetic expressions.
The closure operator matches 0 or more of the specified expression. "a*"
matches a string of 0 or more "a"s.
The plus sign indicates positive closure and matches 1 or more of the
specified expression. "z+" matches a string of 1 or more "z"s.
? The question mark matches 0 or 1 of the specified expression. "9?" matches
either the null string or "9".
Character classes are "shorthand" for matching one of several characters. For
example, [AaBb] is the same as (A a B b) and matches "A," "a," "B," or "b."
There are also character ranges, such as [A - Z], which match any uppercase
alpha character. Negated character classes act in the opposite way, and they
specify characters that should not be matched. For example, [^A-Z] matches
everything except uppercase alpha characters.
With the above in mind, let's examine some regular expressions.
"^[a-z]$" matches any line that consists of only lowercase letters.
"[A - Za-z_][A-Za - z_0-9]*" matches a C variable anywhere on a line.
"(soft loud) (classical rock country) music" matches six strings from "soft
classical music" to "loud country music."
"^[0-9].*[^0-9]$" matches any line that begins with a digit and doesn't end in
a digit.
"80[123]?8[86] 680[01234]0" matches any of the more popular microprocessors,
plus a few that never made it.



Writing AWK-like Extensions to C
by Jim Mischel


[LISTING ONE]
/*
 * qstring.c - finds and outputs all quoted strings in a C source file.
 *
 * Copyright 1988, Jim Mischel
 */
#include <stdio.h>
#include <string.h>
#include "awklib.h"

void main (void) {
 char pat[MAXPAT];
 char buff[MAXSTR];
 char s[MAXSTR];
 char *c;

 awk_init ();

 if (makepat("\"[^\"]*\"", pat) == NULL) {
 fprintf (stderr, "Error compiling pattern string.\n");
 return;
 }

 while (gets (buff) != NULL) {
 c = buff;
 while ((c = re_match (c, pat)) != NULL) {
 strncpy (s, c, RLENGTH);
 s[RLENGTH] = '\0';
 puts (s);
 c += RLENGTH;
 }
 }
}






[LISTING TWO]

/*
 * awklib.h - defines, global variables, and function prototypes for
 * AWKLIB routines.
 *
 * Copyright 1988, Jim Mischel. All rights reserved.
 *
 */
extern int RSTART;
extern int RLENGTH;
extern int NF;
extern char *FS;
extern char *FS_PAT;
extern char *FIELDS[];

#define MAXSTR 1024
#define MAXPAT 2*MAXSTR

void pascal awk_init (void);
char * pascal setfs (char *fs);
char * pascal match (char *s, char *re);
char * pascal re_match (char *s, char *pat);
char * pascal makepat (char *re, char *pat);
int pascal split (char *s, char **a, char *fs);
int pascal re_split (char *s, char **a, char *pat);
int pascal getline (char *s, int nchar, FILE *infile);
int pascal sub (char *re, char *replace, char *str);
int pascal re_sub (char *pat, char *replace, char *str);
int pascal gsub (char *re, char *replace, char *str);
int pascal re_gsub (char *pat, char *replace, char *str);
char * pascal strins (char *s, char *i, int pos);
char * pascal strcins (char *s, int ch, int pos);
char * pascal strdel (char *s, int pos, int n);
char * pascal strcdel (char *s, int pos);
char * pascal strccat (char *s, int c);







[LISTING THREE]
/*
 * awklib.c - C callable routines that provide field splitting and regular
 * expression matching functions much like those found in AWK.
 *
 * Copyright 1988, Jim Mischel. All rights reserved.
 */
#include <stdio.h>
#include <string.h>
#include <alloc.h>
#include <process.h>

#define TRUE -1
#define FALSE 0
#define ALMOST 1 /* returned when a closure matches a NULL */

#define ENDSTR '\0'
#define EOL '$'
#define BOL '^'
#define NEGATE '^'
#define CCL '['
#define NCCL ']'
#define CCLEND ']'
#define ANY '.'
#define DASH '-'
#define OR ''
#define ESCAPE '\\'
#define LPAREN '('
#define RPAREN ')'
#define CLOSURE '*'
#define POS_CLO '+'
#define ZERO_ONE '?'
#define LITCHAR 'c'
#define END_TERM 'e'
#define FS_DEFAULT "[ \t]+"

#define MAXSTR 1024
#define MAXPAT 2*MAXSTR
#define MAXFIELD 128

/*
 * AWKLIB global variables. These variables are defined in AWKLIB.H and may
 * be accessed by the application.
 */
int RSTART; /* start of matched substring */
int RLENGTH; /* length of matched substring */
int NF; /* number of fields from most current split */
char * FS; /* global field separator */
char * FS_PAT; /* compiled field separator */
char * FIELDS[MAXFIELD]; /* contents of fields from most current split */

/*
 * Internal function prototypes.
 */
char * pascal re_match (char *s, char *pat);

char * pascal makepat (char *re, char *pat);
int pascal re_split (char *s, char **a, char *pat);
char * pascal do_sub (char *str, int inx, int len, char *replace);
char parse_escape (void);

/*
 * These string routines, while designed specifically for this application,
 * may be useful to other programs. Their prototypes are included in the
 * AWKLIB.H file.
 */
char * pascal strins (char *s, char *i, int pos);
char * pascal strcins (char *s, int ch, int pos);
char * pascal strdel (char *s, int pos, int n);
char * pascal strcdel (char *s, int pos);
char * pascal strccat (char *s, int c);

/*
 * Initialize AWKLIB global variables. This routine MUST be called before
 * using the AWKLIB routines. Failure to do so may produce some strange
 * results.
 */
void pascal awk_init (void) {
 int x;
 char * pascal setfs (char *fs);

 FS = FS_PAT = NULL;
 setfs (FS_DEFAULT);
 RSTART = RLENGTH = NF = 0;
 for (x = 0; x < MAXFIELD; x++)
 FIELDS[x] = NULL;
} /* awk_init */

/*
 * Sets the field separator to the regular expression fs. The regular
 * expression is compiled into FS_PAT. FS_PAT is returned. NULL is returned
 * on error and neither FS or FS_PAT is modified.
 */
char * pascal setfs (char *fs) {
 char pat[MAXPAT];

 pat[0] = ENDSTR;
 if (makepat (fs_DEFAULT, pat) == NULL)
 return (NULL);
 if (FS != NULL)
 free (FS);
 if (FS_PAT != NULL)
 free (FS_PAT);
 FS = strdup (fs);
 FS_PAT = strdup (pat);
 return (FS_PAT);
} /* setfs */

/*
 * makepat() - "compile" the regular expression re into pat and return a
 * a pointer to the compiled string, or NULL if the compile fails.
 *
 * Performs a recursive descent parse of the expression.
 */


char *_re_ptr; /* global for pattern building */

char * pascal makepat (char *re, char *pat) {
 char *t;
 char * parse_expression (void);

 _re_ptr = re;
 if ((t = parse_expression ()) == NULL)
 return (NULL);
 else if (*_re_ptr != ENDSTR) {
 free (t);
 return (NULL);
 }
 else {
 strcpy (pat, t);
 free (t);
 return (pat);
 }
} /* makepat */

/*
 * parse_expression() - Parse and translate an expression. Returns a pointer
 * to the compiled expression, or NULL on error.
 */
char * parse_expression (void) {
 char pat[MAXPAT];
 char *arg1;

 char * parse_term (void);

 pat[0] = ENDSTR;
 if ((arg1 = parse_term ()) == NULL) /* get the first term */
 return (NULL);

 while (*_re_ptr == OR) { /* parse all subsequent terms */
 strccat (pat, OR);
 strcat (pat, arg1);
 strccat (pat, END_TERM);
 free (arg1);
 _re_ptr++;
 if ((arg1 = parse_term ()) == NULL)
 return (NULL);
 }
 strcat (pat, arg1);
 strccat (pat, END_TERM);
 free (arg1);
 return (strdup (pat));
} /* parse_expression */

/*
 * parse_term() - parse and translate a term. Returns a pointer to the
 * compiled term or NULL on error.
 */
char * parse_term (void) {
 char *t;
 char pat[MAXPAT];

 int isfactor (char c);
 char * parse_factor (void);


 pat[0] = ENDSTR;
 if (*_re_ptr == BOL)
 strccat (pat, *_re_ptr++);
 do {
 if ((t = parse_factor ()) == NULL)
 return (NULL);
 else {
 strcat (pat, t);
 free (t);
 }
 } while (isfactor (*_re_ptr)); /* parse all factors of this term */
 return (strdup (pat));
} /* parse_term */

/*
 * isfactor() - returns TRUE if c is a valid factor character
 */
int isfactor (char c) {
 static char nfac_chars[] = "^)]+?*";
 return (strchr (nfac_chars, c) == NULL) ? TRUE : FALSE;
} /* isfactor */

/*
 * parse_factor() - parse and translate a factor. Returns a pointer to the
 * compiled factor or NULL on error.
 */
char * parse_factor (void) {
 char pat[MAXPAT];
 char *t;

 char * parse_expression (void);
 int parse_closure (char *pat, char c);
 char * parse_ccl (void);

 pat[0] = ENDSTR;
 switch (*_re_ptr) {
 case LPAREN : /* parenthesised expression */
 _re_ptr++;
 t = parse_expression ();
 strcat (pat, t);
 free (t);
 if (*_re_ptr++ != RPAREN)
 return (NULL);
 break;
 case CCL : /* character class */
 _re_ptr++;
 t = parse_ccl ();
 strcat (pat, t);
 free (t);
 if (*_re_ptr++ != CCLEND)
 return (NULL);
 break;
 case ANY : /* '.' or '$' operators */
 case EOL :
 strccat (pat, *_re_ptr++);
 break;
 case ESCAPE : /* ESCAPE character */
 _re_ptr++;

 strccat (pat, LITCHAR);
 strccat (pat, parse_escape ());
 break;
 case CLOSURE :
 case POS_CLO :
 case ZERO_ONE :
 case NEGATE :
 case CCLEND :
 case RPAREN :
 case OR : /* not valid characters */
 return (NULL);
 default : /* literal character */
 strccat (pat, LITCHAR);
 strccat (pat, *_re_ptr++);
 break;
 }
 /*
 * check for closure
 */
 if (*_re_ptr == CLOSURE *_re_ptr == ZERO_ONE *_re_ptr == POS_CLO)
 if (parse_closure (pat, *_re_ptr++) == FALSE)
 return (NULL);
 return (strdup (pat));
} /* parse_factor */

/*
 * parse_escape () - returns ASCII value of character(s) following ESCAPE
 */
char parse_escape (void) {
 unsigned char ch;
 switch (*_re_ptr) {
 case 'b' : _re_ptr++; return ('\b'); /* backspace */
 case 't' : _re_ptr++; return ('\t'); /* tab */
 case 'f' : _re_ptr++; return ('\f'); /* formfeed */
 case 'n' : _re_ptr++; return ('\n'); /* linefeed */
 case 'r' : _re_ptr++; return ('\r'); /* carriage return */
 case '0' : /* 0-7 is octal constant */
 case '1' :
 case '2' :
 case '3' :
 case '4' :
 case '5' :
 case '6' :
 case '7' :
 ch = *_re_ptr++ - '0';
 if (*_re_ptr >= '0' && *_re_ptr < '8') {
 ch <<= 3;
 ch += *_re_ptr++ - '0';
 }
 if (*_re_ptr >= '0' && *_re_ptr < '8') {
 ch <<= 3;
 ch += *_re_ptr++ - '0';
 }
 return (ch);
 default : /* otherwise, just that char */
 return (*_re_ptr++);
 }
} /* parse_escape */


/*
 * parse_closure() - place closure character and size before the factor
 * in the compiled string.
 */
int parse_closure (char *pat, char c) {
 int len;

 memmove (pat+2, pat, strlen (pat) + 1);
 pat[0] = c;
 len = strlen (pat + 2);
 if (len > 255)
 return (FALSE); /* closure expression too large */
 else {
 pat[1] = len;
 return (TRUE);
 }
} /* parse_closure */

/*
 * parse_ccl() - parse and translate a character class. Return pointer to the
 * compiled class or NULL on error.
 */
char * parse_ccl (void) {
 char pat[MAXPAT];
 int first = TRUE;
 int len;

 char * parse_dash (char *pat, char ch);

 strcpy (pat, "[ ");
 if (*_re_ptr == NEGATE) { /* if first character is NEGATE */
 pat[0] = NCCL; /* then we have a negated */
 _re_ptr++; /* character class */
 }

 /*
 * parse all characters up to the closing bracket or end of string marker
 */
 while (*_re_ptr != CCLEND && *_re_ptr != ENDSTR) {
 if (*_re_ptr == DASH && first == FALSE) { /* DASH, check for range */
 if (*++_re_ptr == NCCL)
 strccat (pat, DASH); /* not range, literal DASH */
 else
 parse_dash (pat, *_re_ptr++);
 }
 else {
 if (*_re_ptr == ESCAPE) {
 _re_ptr++;
 strccat (pat, parse_escape ());
 }
 else
 strccat (pat, *_re_ptr++);
 }
 first = FALSE;
 }
 len = strlen (pat+2);
 if (len > 255)
 return (NULL); /* character class too large */
 else {

 pat[1] = len; /* store CCL length at pat[1] */
 return (strdup (pat));
 }
} /* parse_ccl */

/*
 * parse_dash() - fill in range characters.
 */
char * parse_dash (char *pat, char ch) {
 int ch1;

 for (ch1 = pat[strlen (pat) - 1] + 1; ch1 <= ch; ch1++)
 strccat (pat, ch1);
 return (pat);
} /* parse_dash */

/*
 * match() - Return a pointer to the first character of the left-most longest
 * substring of s that matches re or NULL if no match is found. Sets
 * RSTART and RLENGTH. This routine compiles the regular expression re and
 * then calls re_match to perform the actual matching.
 */
char * pascal match (char *s, char *re) {
 char pat[MAXPAT];

 pat[0] = ENDSTR;
 if (makepat (re, pat) == NULL)
 return (NULL);
 return (re_match (s, pat));
} /* match */

/*
 * re_match() - Return a pointer to the first character of the left-most
 * longest substring of s that matches pat, or NULL if no match is found.
 * Sets RSTART and RLENGTH. The != FALSE test below must NOT be changed
 * to == TRUE. match_term() can return TRUE, FALSE, or ALMOST. Both TRUE
 * and ALMOST are considered TRUE by this routine.
 */
char *_s_end; /* global points to last character matched */

char * pascal re_match (char *s, char *pat) {
 char *c = s;
 int pascal match_term (int inx, char *s, char *pat);

 _s_end = NULL;
 while (*c != ENDSTR) {
 if (match_term (c-s, c, pat) != FALSE) {
 RSTART = c-s;
 RLENGTH = _s_end - c;
 return (c);
 }
 c++;
 }
 RSTART = RLENGTH = 0;
 return (NULL);
} /* re_match */

/*
 * Match a compiled term. Returns TRUE, FALSE, or ALMOST.

 */
int pascal match_term (int inx, char *s, char *pat) {
 int pascal match_or (int inx, char *s, char *pat);
 int pascal match_ccl (char c, char *pat);
 int pascal match_closure (int inx, char *s, char *pat, char *clopat);
 int pascal match_0_1 (int inx, char *s, char *pat);

 _s_end = s;
 if (*pat == ENDSTR)
 return (FALSE);
 do {
 switch (*pat) {
 case BOL : /* match beginning of line */
 if (inx != 0)
 return (FALSE);
 pat++;
 break;
 case LITCHAR : /* match literal character */
 if (*s++ != *++pat)
 return (FALSE);
 pat++;
 break;
 case END_TERM : pat++; break; /* skip end-of-term character */
 case ANY : /* match any character ... */
 if (*s++ == ENDSTR) /* ... except end of string */
 return (FALSE);
 pat++;
 break;
 case OR : return (match_or (inx, s, pat));
 case CCL : /* character class requires */
 case NCCL : /* special processing */
 if (*s == ENDSTR)
 return (FALSE);
 if (!match_ccl (*s++, pat++))
 return (FALSE);
 pat += *pat + 1;
 break;
 case EOL : /* match end of string */
 if (*s != ENDSTR)
 return (FALSE);
 pat++;
 break;
 case ZERO_ONE : return (match_0_1 (inx, s, pat));
 case CLOSURE :
 case POS_CLO : {
 char clopat[MAXPAT];
 strncpy (clopat, pat+2, *(pat+1));
 clopat[*(pat+1)] = ENDSTR;
 return (match_closure (inx, s, pat, clopat));
 break;
 }
 default :
 /*
 * If we get to this point, then something has gone very wrong.
 * Most likely, someone has tried to match with an invalid
 * compiled pattern. Whatever the case, the only thing to do
 * is abort the program.
 */
 fputs ("In match_term: can't happen", stderr);

 exit (1);
 break;
 } /* switch */
 _s_end = s;
 } while (*pat != ENDSTR);
 return (TRUE);
} /* match_term */

/*
 * match_or() - Handles selection processing.
 */
int pascal match_or (int inx, char *s, char *pat) {
 char workpat[MAXPAT];
 char *t1, *t2, *junk;

 int pascal match_term (int inx, char *s, char *pat);
 char * pascal skip_term (char *pat);

 /*
 * The first case is build into workpat. Second case is already there.
 * Both patterns are searched to determine the longest matched substring.
 */
 workpat[0] = ENDSTR;
 pat++;
 junk = skip_term (pat);
 strncat (workpat, pat, junk-pat);
 strcat (workpat, skip_term (junk));
 t1 = (match_term (inx, s, workpat) != FALSE) ? _s_end : NULL;
 /*
 * The second pattern need not be searched if the first pattern results
 * in a match through to the end of the string, since the longest possible
 * match has already been found.
 */
 if (t1 == NULL *_s_end != ENDSTR) {
 t2 = (match_term (inx, s, junk) != FALSE) ? _s_end : NULL;
 /*
 * determine which matched the longest substring
 */
 if (t1 != NULL && (t2 == NULL t1 > t2))
 _s_end = t1;
 }
 return (t1 == NULL && t2 == NULL) ? FALSE : TRUE;
} /* match_or */

/*
 * Skip over the current term and return a pointer to the next term in
 * the pattern.
 */
char * pascal skip_term (char *pat) {
 register int nterm = 1;

 while (nterm > 0) {
 switch (*pat) {
 case OR : nterm++; break;
 case CCL :
 case NCCL :
 case CLOSURE:
 case ZERO_ONE:
 case POS_CLO:

 pat++;
 pat += *pat;
 break;
 case END_TERM: nterm--; break;
 case LITCHAR: pat++; break;
 }
 pat++;
 }
 return (pat);
} /* skip_term */

/*
 * Match the ZERO_ONE operator. First, this routine attempts to match the
 * entire pattern with the input string. If that fails, it skips over
 * the closure pattern and attempts to match the rest of the pattern.
 */
int pascal match_0_1 (int inx, char *s, char *pat) {
 char *save_s = s;

 if (match_term (inx, s, pat+2) == TRUE)
 return (TRUE);
 else if (match_term (inx, save_s, pat+2+*(pat+1)) == FALSE)
 return (FALSE);
 else
 return (ALMOST);
} /* match_0_1 */

/*
 * Match CLOSURE and POS_CLO.
 * Match as many of the closure patterns as possible, then attempt to match
 * the remaining pattern with what's left of the input string. Backtrack
 * until we've either matched the remaing pattern or we arrive back at where
 * we started.
 */
int pascal match_closure (int inx, char *s, char *pat, char *clopat) {
 char *save_s = s;

 if (match_term (inx, s, clopat) == TRUE) {
 save_s = _s_end;
 if (match_closure (inx, save_s, pat, clopat) == TRUE)
 return (TRUE);
 else
 return (match_term (inx, save_s, pat+2+*(pat+1)));
 }
 else if (*pat != CLOSURE)
 return (FALSE); /* POS_CLO requires at least one match */
 else if (match_term (inx, save_s, pat+2+*(pat+1)) == TRUE)
 return (ALMOST);
 else
 return (FALSE);
} /* match_closure */

/*
 * Match a character class or negated character class
 */
int pascal match_ccl (char c, char *pat) {
 register int x;
 char ccl = *pat++;


 for (x = *pat; x > 0; x--)
 if (c == pat[x])
 return (ccl == CCL);
 return (ccl != CCL);
} /* match_ccl */

/*
 * Substitue 'replace' for the leftmost longest substring of str matched by
 * the regular expression re.
 * Return number of substitutions made (which in this case will be 0 or 1).
 */
int pascal sub (char *re, char *replace, char *str) {

 if (match (str, re) != NULL) {
 free (do_sub (str, RSTART, RLENGTH, replace));
 return (1);
 }
 else
 return (0);
} /* sub */

/*
 * Substitue 'replace' for the leftmost longest substring of str matched by
 * the compiled regular expression pat.
 * Return number of substitutions made (which in this case will be 0 or 1).
 */
int pascal re_sub (char *pat, char *replace, char *str) {
 int pascal re_sub (char *pat, char *replace, char *str);

 if (re_match (str, pat) != NULL) {
 free (do_sub (str, RSTART, RLENGTH, replace));
 return (1);
 }
 else
 return (0);
} /* re_sub */

/*
 * Substitute 'replace' globally for all substrings in str matched by the
 * regular expression re.
 * Return number of substitutions made.
 *
 * This routine uses makepat() to compile the regular expression, then calls
 * re_gsub() to do the actual replacement.
 *
 * NOTE: gsub() makes only 1 pass through the string. Replaced strings
 * cannot themselves be replaced.
 */
int pascal gsub (char *re, char *replace, char *str) {
 int pascal re_gsub (char *pat, char *replace, char *str);

 char pat[MAXPAT];

 pat[0] = ENDSTR;
 if (makepat (re, pat) == NULL)
 return (0);
 return (re_gsub (pat, replace, str));
} /* gsub */


/*
 * Substitute 'replace' globally for all substrings in str matched by the
 * compiled regular expression pat.
 * Return number of substitutions made.
 *
 * NOTE: gsub() makes only 1 pass through the string. Replaced strings
 * cannot themselves be replaced.
 */
int pascal re_gsub (char *pat, char *replace, char *str) {
 char *m = str;
 int nsub = 0;
 char *p;

 while ((m = re_match (m, pat)) != NULL) {
 p = do_sub (m, 0, RLENGTH, replace);
 nsub++;
 m += strlen (p);
 free (p);
 }
 return (nsub);
} /* re_gsub */

/*
 * remove 'len' characters from 'str' starting at position 'inx'. Then insert
 * the replacement string at position 'inx'.
 */
char * pascal do_sub (char *str, int inx, int len, char *replace) {
 char *p;
 char * pascal makesub (char *replace, char *found, int len);

 p = makesub (replace, &str[inx], len);
 strdel (str, inx, len);
 strins (str, p, inx);
 return (p);
} /* do_sub */

/*
 * Make a substitution string.
 */
char * pascal makesub (char *replace, char *found, int len) {
 char news[MAXSTR];
 char *c = replace;
 int x;

 news[0] = ENDSTR;
 while (*c != ENDSTR) {
 if (*c == '&')
 for (x = 0; x < len; x++)
 strccat (news, found[x]);
 else if (*c == '\\') {
 _re_ptr = c+1;
 strccat (news, parse_escape ());
 c = _re_ptr - 1;
 }
 else
 strccat (news, *c);
 c++;
 }
 return (strdup (news));

} /* makesub */

/*
 * split - split the string s into fields in the array a on field separator
fs.
 * fs is a regular expression. Returns number of fields. Also sets the global
 * variable NF. This routine compiles fs into a pattern and then calls
 * re_split() to do the work.
 */
int pascal split (char *s, char **a, char *fs) {
 char pat[MAXPAT];

 pat[0] = ENDSTR;
 makepat (fs, pat);
 return re_split (s, a, pat);
} /* split */

/*
 * re_split() - split the string s into fields in the array on field seperator
 * pat. pat is a compiled regular expression (built by makepat()). Returns
 * number of fields. Also sets the global variable NF.
 */
int pascal re_split (char *s, char **a, char *pat) {
 int rstart = RSTART; /* save RSTART and RLENGTH */
 int rlength = RLENGTH;
 char *c = s;
 char *oldc = s;

 NF = 0;
 if (a[0] != NULL)
 free (a[0]);
 a[0] = strdup (s);

 while (*oldc != ENDSTR) {
 while ((c = re_match (oldc, pat)) == oldc)
 oldc += RLENGTH;
 if (*oldc != ENDSTR) {
 if (c == NULL)
 c = &oldc[strlen (oldc)];
 if (a[++NF] != NULL)
 a[NF] = realloc (a[NF], c-oldc+1);
 else
 a[NF] = malloc (c-oldc+1);
 memcpy (a[NF], oldc, c-oldc);
 a[NF][c-oldc] = ENDSTR;
 oldc = c;
 }
 }
 RSTART = rstart; /* restore globals */
 RLENGTH = rlength;
 return (NF);
} /* re_split */

/*
 * Reads a line from infile and splits it into FIELDS. Returns EOF on
 * end-of-file or error.
 */
int pascal getline (char *s, int nchar, FILE *infile) {
 char *c;


 if (fgets (s, nchar, infile) == NULL)
 return (EOF);
 if ((c = strchr (s, '\n')) != NULL)
 *c = ENDSTR; /* look for and replace newline */
 re_split (s, FIELDS, FS_PAT);
 return (0);
} /* getline */

/*
 * add a character to the end of a string
 */
char * pascal strccat (char *s, int ch) {
 register int len = strlen (s);
 s[len++] = ch;
 s[len] = ENDSTR;
 return (s);
}

/*
 * removes the character at pos from the string.
 */
char * pascal strcdel (char *s, int pos) {
 memcpy (s+pos, s+pos+1, strlen (s) - pos);
 return (s);
} /* strcdel */

/*
 * inserts the character ch into the string at position pos. Assumes there
 * is room enough in the string for the character.
 */
char * pascal strcins (char *s, int ch, int pos) {
 memmove (s+pos+1, s+pos, strlen (s) - pos + 1);
 s[pos] = ch;
 return (s);
}

/*
 * removes n characters from s starting at pos
 */
char * pascal strdel (char *s, int pos, int n) {
 memcpy (s+pos, s+pos+n, strlen(s)-pos-n+1);
 return (s);
}

/*
 * inserts the string i into the string s at position pos. Assumes there
 * is sufficient memory in s to hold i.
 */
char * pascal strins (char *s, char *i, int pos) {
 char *p = s+pos;
 int ilen = strlen (i);

 memmove (p+ilen, p, strlen (s) - pos + 1);
 memcpy (p, i, ilen);
 return (s);
}




Example 1. The Program A executes faster because it only
compiles the regular expression once, while Program B compiles it
each time match() is called.

Program A Program B

 ... ...
while (getline(line,80,f) != EOF) { makepat("a+bc",pat);
 if (match(line,"a+bc") != NULL) while (getline(line,80,f) != EOF) {
 puts(line); if (re_match(line,pat) != NULL)
} puts(line);
 ... }
 ...

















































June, 1989
 CREATING TSRS PROGRAMS WITH TURBO PASCAL: PART II


Putting common TSR tasks to work




Ken L. Pottebaum


Ken Pottebaum is a professional mechanical engineer for the Small Disk
Division of Imprimis, a subsidiary of Control Data Corp. He can be reached at
321 Redbud, Yukon, OK 70399.


This is the second part of a two-part article about creating TSR
(Terminate-and-Stay-Resident) programs with Turbo Pascal 5.0 and TSRUnit. The
first part dealt with the inside workings of TSRUnit. This part focuses on how
to use TSRUnit and provides an example program. Although the example program
is simple, it illustrates methods for accessing files, printing, inserting
characters into the keyboard input stream, and other useful tasks. If you
missed the first part of the series, you can still create your own TSR
programs because the INTERFACE section of TSRUnit (Listing One) provides
information about how to use the unit.


Using TSRUnit


In order to use TSRUnit, your program must provide a function for TSRUnit to
call when the unit pops up and also perform a call to TSRInstall.
The function that TSRUnit calls when it pops up may have any valid function
name. For simplicity, I will refer to this function as DemoTasks. (DemoTasks
is also the name for this function in the example program TSRDemo, which is
discussed later in this article.) DemoTasks must be compiled as a far function
(without any parameters) that returns a WORD value. The WORD value is the
number of characters to be inserted into the keyboard input stream. (The
character insertion option is a special feature provided by TSRUnit and is
explained later, along with the other features illustrated in the example
program.) Generally speaking, the far function can contain any valid Turbo
Pascal statement, including calls to other procedures and functions. Common
sense is important, however, when any of the routines from the standard DOS
UNIT are used. For example, it would not make sense to use the Keep procedure
because Keep has already been used. Also, if you use any of the procedures
that change interrupt vectors, be sure to restore those procedures before
exiting DemoTasks.
As shown in Listing One, the call to TSRInstall requires four parameters. The
first parameter is a character string that contains the name of the TSR; this
name is displayed when the TSR is installed. The second parameter is a
procedure-type parameter that specifies the function that TSRUnit calls when
the TSR is popped up. The last two parameters specify the default hot key
combination that is pressed in order to pop up the TSR. The first of the hot
key parameters contains a byte code that represents the shift keys that must
be pressed while the key for the character specified in the last parameter is
pressed. The shift key code can be any combination of the following constants:
AltKey, CtrlKey, LeftKey, and RightKey. The constants stand for the Alt, Ctrl,
Left Shift, and Right Shift keys, respectively. Combinations of two or more
shift keys may be formed by using either the OR or + operators to combine the
constants. The valid characters for the last parameter are 0-9 and A-Z.


Example TSR


The example program, TSRDEMO.PAS, is shown in Listing Two. Although Turbo
Pascal normally defaults the stack size to 16K, the compiler memory directive
may be used to set the stack and heap sizes. TSRDemo uses the compiler
directive to reduce its memory requirements by specifying a 2K stack and no
heap. To find the minimum stack size with which your program will work, simply
run the program with different stack sizes. In order for the stack size check
to be valid, test all of the program's features (unless you already know which
feature of the program uses the most stack space). If you know the worst case,
limit the testing to the worst case operations. Set the stack size for your
program slightly higher than the minimum size for which the program worked, in
order to allow for variations in stack usage that occur due to differences in
versions of DOS and BIOS used and due to the presence of other TSR programs.
Although you can reduce memory usage by specifying a stack that is only as
large as necessary, do not confuse this step with avoiding using the stack
altogether -- the use of the stack for local variables and for passing
information is an efficient allocation of memory.
When TSRDemo is popped up, its routine, DemoTasks, is called from TSRUnit.
DemoTasks creates a window on the screen, displays some instructions, and
displays miscellaneous information about the computer system. It then echoes
the keyboard input to the screen until a key that activates a demonstration of
one of DemoTasks' special features is pressed. DemoTasks provides four special
features: file accessing, printing, accessing the saved screen image, and
inserting characters.


File Accessing


When the F1 key is pressed, the standard file-handling routines are used to
write a short message to the TSRDEMO.DAT file. It is advisable to disable the
compiler I/O error-checking directive, and to use the IOResult function to
check for errors when files are accessed.


Printing


When F2 is pressed, DemoTasks performs the operations that are required in
order to print a short message. Programs normally use the standard PRINTER
unit to automatically assign, open, and close the printer device. The TSR
installation process causes a printer device that was opened in this fashion
to be closed. Therefore, the TSR program must treat the printer as a normal
file device, and use the Assign, Rewrite, and Close procedures to open and
close it.
Printing from a TSR poses a special hazard. If the TSR writes to the printer
when the printer is not ready, an error occurs. When the standard IOResult
function is used, the error is reported, but damage that may be caused by the
error is not prevented. The consequence of this "device not ready" error is
that the computer may crash when the TSR is popped back down. An easy way to
prevent this problem is to always check the printer status before using the
printer. TSRUnit provides two auxiliary functions for this purpose:
PrinterOkay and PrinterStatus. PrinterOkay returns a TRUE value if the printer
is selected, the printer has paper, and no I/O or time-out error has occurred.
PrinterStatus returns the actual status byte for the printer.


Accessing Saved Screen Text


When F3 is pressed, the program prompts you for the row number of the saved
screen text to be displayed. TSRUnit provides the pointer TSRScrPtr, which
points to the saved screen image. In addition, TSRUnit provides two auxiliary
routines that return the contents of one row of a text image. The function
ScreenLineStr returns all of the characters contained on the line in a
character string. The procedure ScreenLine returns an array of records, where
each record contains a character and the character's display attribute.


Inserting Characters


DemoTasks' fourth special feature is the insertion of characters into the
keyboard input stream. When F8 is pressed, you are prompted to specify the
characters to be inserted. After you type the characters, the TSR pops back
down and inserts them into the keyboard input stream. A useful application of
this feature is to insert a result obtained from the TSR into a spreadsheet or
word processor program.
In order for TSRUnit to insert the characters, the pointer TSRChrPtr must be
set to point to the first character to be inserted, and the return value for
DemoTasks must be set to the number of characters. The only restriction on the
number of characters to be inserted is that all of the characters must reside
in the same 64K segment of memory. Because the example program restricts the
number of characters to 255, it stores the characters in string InsStr and
puts the address of InsStr[1] in TSRChrPtr.



Overriding Default Hot Keys


To allow the users of TSRs to avoid hot key conflicts with other programs,
TSRUnit permits the default hot key combination to be overridden when the TSR
is installed. To replace the hot key combination, include some command-line
parameters along with the TSR's program name. The format for the parameters is
[/A][/C][/R][/L][/"[K["]]]
The square brackets surround optional items. (Do not include the brackets.)
Spaces or characters between the parameters are ignored and the parameters may
be listed in any order. Although the shift key parameters (/A, /C,/R, and /L)
are cumulative, only the last character key specified is used. The command to
install TSRDemo with hot key combination Alt, Left Shift, T is
TSRDEMO A/L/"T


Caution: The Effects of Run-Time Errors


Although it is always good programming technique to take steps to prevent
run-time errors from occurring, it is especially important to do so in a TSR
program. Depending upon the error-handling methods used, the occurrence of an
error can cause the execution of the TSR to be aborted. In some cases,
execution will resume at the DOS prompt level. In more severe cases, the
computer will "hang up" either immediately, or later when the TSR is popped
back down.
Several approaches can be used to deal with run-time error situations. The
example program TSRDemo illustrates two approaches. One of these methods is to
replace Turbo Pascal's standard I/O operation error-checking code with code
that simply displays an error message. (The default error-checking code
terminates the program after displaying an error message.) The standard I/O
error checking is disabled with the compiler directive {$I-}. As an example of
a second type of error checking, TSRDemo uses TSRUnit's PrinterOkay function
to verify that the printer is ready before writing to it. This approach can
also be used to prevent numerical errors, such as divide-by-zero errors. A
third method, which is too complicated to cover in this article, is to
intercept the error handling routines.
One important category of errors that should be prevented from occurring is
that of "device not ready" errors -- this type of error causes the computer to
"hang up" either when the error occurs or when the TSR pops back down. The
most likely cause for this type of error is an attempt to use either a
diskette drive or a printer when it is not ready. To prevent either of these
events from occurring, the example program takes the precaution of querying
the user and checking the printer status.


Availability


All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063, or call 800-356-2002 (from inside
Calif.) or 800-533-4372 (from outside Calif.). Please specify the issue number
and format (MS-DOS, Macintosh, Kaypro).


Creating TSRs with Turbo Pascal, Part II
by Ken Pottebaum



[Listing One]


UNIT TSRUnit; {Create TSR programs with Turbo Pascal 5.0 & TSRUnit}
INTERFACE {=======================================================}
{
The author and any distributor of this software assume no responsi-
bility for damages resulting from this software or its use due to
errors, omissions, incompatibility with other software or with
hardware, or misuse; and specifically disclaim any implied warranty
of fitness for any particular purpose or application.
}
USES DOS, CRT;
CONST
{*** Shift key combination codes. }
 AltKey = 8; CtrlKey = 4; LeftKey = 2; RightKey = 1;

 TSRVersion : WORD = $0203; {Low byte.High byte = 2.03 }

TYPE
 String80 = STRING[80];
 ChrWords = RECORD CASE INTEGER OF
 1: ( W: WORD );
 2: ( C: CHAR; A: BYTE );
 END;
 LineWords = ARRAY[1..80] OF ChrWords;
 WordFuncs = FUNCTION : WORD;

VAR
 TSRScrPtr : POINTER; {Pointer to saved screen image. }

 TSRChrPtr : POINTER; {Pointer to first character to insert. }
 TSRMode : BYTE; {Video mode --------- before TSR popped up.}
 TSRWidth : BYTE; {Number of screen columns-- " " " " .}
 TSRPage : BYTE; {Active video page number-- " " " " .}
 TSRColumn : BYTE; {Cursor column number ----- " " " " .}
 TSRRow : BYTE; {Cursor row number -------- " " " " .}
{
** Procedure for installing the TSR program. }
PROCEDURE TSRInstall( TSRName : STRING; {Name or title for TSR. }
 TSRFunc : WordFuncs;{Ptr to FUNCTION to call}
 ShiftComb: BYTE; {Hot key--shift key comb}
 KeyChr : CHAR ); {Hot Key--character key.}
{
 ShiftComb and KeyChr specify the default hot keys for the TSR.
 ShiftComb may be created by adding or ORing the constants AltKey,
 CtrlKey, LeftKey, and RightKey together. KeyChr may be
 characters 0-9 and A-Z.

 The default hot keys may be overridden when the TSR is installed
 by specifying optional parameters on the command line. The
 parameter format is:
 [/A] [/C] [/R] [/L] [/"[K["]]]
 The square brackets surround optional items--do not include them.
 Any characters between parameters are ignored. The order of the
 characters does not matter; however, the shift keys specified are
 cummulative and the last character key "K" specified is the used.
}
{
** Functions for checking status of printer LPT1. }
FUNCTION PrinterOkay: BOOLEAN; {Returns TRUE if printer is okay.}
FUNCTION PrinterStatus: BYTE; {Returns status of printer.
 Definition of status byte bits (1 & 2 are not used), if set then:
 Bit: -- 7 --- ---- 6 ---- -- 5 --- -- 4 --- -- 3 -- --- 0 ---
 Not busy Acknowledge No paper Selected I/O Err. Timed-out
}
{
** Routines for obtaining one row of screen characters. }
FUNCTION ScreenLineStr( Row: BYTE ): String80; {Returns char. str.}
PROCEDURE ScreenLine( Row: BYTE; VAR Line: LineWords; {Returns }
 VAR Words: BYTE ); {chr & color}






[Listing Two]

PROGRAM TSRDemo; {An example TSR program created using TSRUnit. }

{$M $0800,0,0} {Set stack and heap size for demo program. }

USES CRT, DOS, TSRUNIT; {Specify the TSRUNIT in the USES statement.}
 {Do not use the PRINTER unit, instead treat}
 {the printer like a file; i.e. use the }
 {Assign, Rewrite, and Close procedures. }

CONST DemoPgmName : STRING[16] = 'TSR Demo Program';


VAR
 Lst : TEXT; {Define variable name for the printer. }
 TextFile : TEXT; { " " " " a data file. }
 InsStr : STRING; {Storage for characters to be inserted into}
 {keyboard input stream--must be a gobal or }
 {heap variable. }

FUNCTION IOError: BOOLEAN; {Provides a message when an I/O error}
VAR i : WORD; {occurs. }
BEGIN
 i := IOResult;
 IOError := FALSE;
 IF i <> 0 THEN BEGIN
 Writeln('I/O Error No. ',i);
 IOError := TRUE;
 END;
END; {OurIOResult.}
{
***** Demo routine to be called when TSRDemo is popped up.
 be compiled as a FAR FUNCTION that returns a WORD containing
 the number of characters to insert into the keyboard input
 stream.
}
{$F+} FUNCTION DemoTasks: WORD; {$F-}
CONST
 FileName : STRING[13] = ' :TSRDemo.Dat';
 EndPos = 40;
 Wx1 = 15; Wy1 = 2; Wx2 = 65; Wy2 = 23;
VAR
 Key, Drv : CHAR;
 Done, IOErr : BOOLEAN;
 InputPos, RowNumb : INTEGER;
 DosVer : WORD;
 InputString : STRING;

 PROCEDURE ClearLine; {Clears current line and resets line pointer}
 BEGIN
 InputString := ''; InputPos := 1;
 GotoXY( 1, WhereY ); ClrEol;
 END;

BEGIN
 DemoTasks := 0; {Default to 0 characters to insert.}
 Window( Wx1, Wy1, Wx2, Wy2 ); {Set up the screen display. }
 TextColor( Black );
 TextBackground( LightGray );
 LowVideo;
 ClrScr; {Display initial messages. }
 Writeln;
 Writeln(' Example Terminate & Stay-Resident (TSR) program');
 Writeln(' --written with Turbo Pascal 5.0 and uses TSRUnit.');
 Window( Wx1+1, Wy1+4, Wx2-1, Wy1+12);
 TextColor( LightGray );
 TextBackground( Black );
 ClrScr; {Display function key definitions. }
 Writeln;
 Writeln(' Function key definitions:');
 Writeln(' [F1] Write message to TSRDEMO.DAT');
 Writeln(' [F2] " " to printer.');

 Writeln(' [F3] Read from saved screen.');
 Writeln(' [F8] Exit and insert text.');
 Writeln(' [F10] Exit TSR and keep it.');
 Write( ' or simply echo your input.');

 {Create active display window. }
 Window( Wx1+1, Wy1+14, Wx2-1, Wy2-1 );
 ClrScr;
 {Display system information. }
 Writeln('TSRUnit Version: ', Hi(TSRVersion):8, '.',
 Lo(TSRVersion):2 );
 Writeln('Video Mode, Page:', TSRMode:4, TSRPage:4 );
 Writeln('Cursor Row, Col.:', TSRRow:4, TSRColumn:4 );

 DosVer := DosVersion;
 Writeln('DOS Version: ', Lo(DosVer):8, '.', Hi(DosVer):2 );

 InputString := ''; {Initialize variables. }
 InputPos := 1;
 Done := False;

 REPEAT {Loop for processing keystrokes. }
 GotoXY( InputPos, WhereY ); {Move cursor to input position. }
 Key := ReadKey; {Wait for a key to be pressed. }
 IF Key = #0 THEN BEGIN {Check for a special key. }
 Key := ReadKey; {If a special key, get auxiliary}
 CASE Key OF {byte to identify key pressed. }

{Cursor Keys and simple editor.}
{Home} #71: InputPos := 1;
{Right} #75: IF InputPos > 1 THEN Dec( InputPos );
{Left} #77: IF (InputPos < Length( InputString ))
 OR ((InputPos = Length( InputString ))
 AND (InputPos < EndPos )) THEN Inc( InputPos );
{End} #79: BEGIN
 InputPos := Succ( Length( InputString ) );
 IF InputPos > EndPos THEN InputPos := EndPos;
 END;
{Del} #83: BEGIN
 Delete( InputString, InputPos, 1 );
 Write( Copy( InputString, InputPos, EndPos ), ' ');
 END;

{Function Keys--TSRDemo's special features.}
{F1} #59: BEGIN {Write short message to a file. }
 ClearLine;
 REPEAT
 Write('Enter disk drive: ',FileName[1] );
 Drv := UpCase( ReadKey ); Writeln;
 IF Drv <> #13 THEN FileName[1] := Drv;
 Writeln('Specifying an invalid drive will cause your');
 Write('system to crash. Use drive ',
 FileName[1], ': ? [y/N] ');
 Key := UpCase( ReadKey ); Writeln( Key );
 UNTIL Key = 'Y';
 Writeln('Writing to ',FileName );
 {$I-} {Disable I/O checking.}
 Assign( TextFile, 'TSRDemo.Dat' );
 IF NOT IOError THEN BEGIN {Check for error. }

 Rewrite( TextFile );
 IF NOT IOError THEN BEGIN
 Writeln(TextFile,'File was written by TSRDemo.');
 IOErr := IOError;
 Close( TextFile );
 IOErr := IOError;
 END;
 END;
 {$I+} {Enable standard I/O checking.}
 Writeln('Completed file operation.');
 END; {F1}

{F2} #60: BEGIN {Print a message, use TSRUnit's auxiliary }
 {function PrinterOkay to check printer status. }
 ClearLine;
 Writeln('Check printer status, then print if okay.');
 IF PrinterOkay THEN BEGIN {Check if printer is okay}
 Assign( Lst, 'LPT1' ); {Define printer device. }
 Rewrite( Lst ); {Open printer. }
 Writeln( Lst, 'Printing performed from TSRDemo');
 Close( Lst ); {Close printer. }
 END
 ELSE Writeln('Printer is not ready.');
 Writeln( 'Completed print operation.' );
 END; {F2}

{F3} #61: BEGIN {Display a line from the saved screen image--not}
 {valid if the TSR was popped up while the }
 {display was in a graphics mode. }
 ClearLine;
 CASE TSRMode OF {Check video mode of saved image.}
 0..3,
 7: BEGIN
 {$I-}
 REPEAT
 Writeln('Enter row number [1-25] from ');
 Write('which to copy characters: ');
 Readln( RowNumb );
 UNTIL NOT IOError;
 {$I+}
 IF RowNumb <= 0 THEN RowNumb := 1;
 IF RowNumb > 25 THEN RowNumb := 25;
 Writeln( ScreenLineStr( RowNumb ) );
 END;
 ELSE Writeln('Not valid for graphics modes.');
 END; {CASE TSRMode}
 END; {F3}
{F8} #66: BEGIN {Exit and insert string into keyboard buffer.}
 ClearLine;
 Writeln('Enter characters to insert;');
 Writeln('Up to 255 character may be inserted.');
 Writeln('Terminate input string by pressing [F8].');
 InsStr := '';
 REPEAT {Insert characters into a}
 Key := ReadKey; {until [F8] is pressed. }
 IF Key = #0 THEN BEGIN {Check for special key.}
 Key := ReadKey; {Check if key is [F8]. }
 IF Key = #66 THEN Done := TRUE; {[F8] so done. }
 END

 ELSE BEGIN {Not special key, add it to the string.}
 IF Length(InsStr) < Pred(SizeOf(InsStr)) THEN
 BEGIN
 IF Key = #13 THEN Writeln
 ELSE Write( Key );
 InsStr := InsStr + Key;
 END
 ELSE Done := TRUE; {Exceeded character limit. }
 END;
 UNTIL Done;
 DemoTasks := Length( InsStr ); {Return no. of chr. }
 TSRChrPtr := @InsStr[1]; {Set ptr to 1st chr.}
 END; {F8}

{F10} #68: Done := TRUE; {Exit and Stay-Resident. }

 END; {CASE Key}
 END {IF Key = #0}
 ELSE BEGIN {Key pressed was not a special key--just echo it. }
 CASE Key OF
{BS} #08: BEGIN {Backspace}
 IF InputPos > 1 THEN BEGIN
 Dec( InputPos );
 Delete( InputString, InputPos, 1 );
 GotoXY( InputPos, WhereY );
 Write( Copy( InputString, InputPos, EndPos ), ' ');
 END;
 END; {BS}
{CR} #13: BEGIN {Enter}
 Writeln;
 InputString := '';
 InputPos := 1;
 END; {CR}
{Esc} #27: ClearLine;
 ELSE
 IF Length( InputString ) >= EndPos THEN
 Delete( InputString, EndPos, 1 );
 Insert( Key, InputString, InputPos );
 Write( Copy( InputString, InputPos, EndPos ) );
 IF InputPos < EndPos THEN
 Inc( InputPos );
 END; {CASE...}
 END; {ELSE BEGIN--Key <> #0}
 UNTIL Done;
END; {DemoTasks.}

BEGIN
 TSRInstall( DemoPgmName, DemoTasks, AltKey, 'E' );
END. {TSRDemo.}






[Listing One from May, 1989]

UNIT TSRUnit; {Create TSR programs with Turbo Pascal 5.0 & TSRUnit}
(*

The author and any distributor of this software assume no responsi-
bility for damages resulting from this software or its use due to
errors, omissions, incompatibility with other software or with
hardware, or misuse; and specifically disclaim any implied warranty
of fitness for any particular purpose or application.
*)

{$B-,F-,I+,R-,S+} {Set compiler directives to normal values.}

INTERFACE {=======================================================}
USES DOS, CRT;
CONST
{*** Shift key combination codes. }
 AltKey = 8; CtrlKey = 4; LeftKey = 2; RightKey = 1;

 TSRVersion : WORD = $0202; {Low byte.High byte = 2.02 }

TYPE
 String80 = STRING[80];
 ChrWords = RECORD CASE INTEGER OF
 1: ( W: WORD );
 2: ( C: CHAR; A: BYTE );
 END;
 LineWords = ARRAY[1..80] OF ChrWords;

VAR
 TSRScrPtr : POINTER; {Pointer to saved screen image. }
 TSRChrPtr : POINTER; {Pointer to first character to insert. }
 TSRMode : BYTE; {Video mode --------- before TSR popped up.}
 TSRWidth : BYTE; {Number of screen columns-- " " " " .}
 TSRPage : BYTE; {Active video page number-- " " " " .}
 TSRColumn : BYTE; {Cursor column number ----- " " " " .}
 TSRRow : BYTE; {Cursor row number -------- " " " " .}
{
** Procedure for installing the TSR program. }
PROCEDURE TSRInstall( PgmName, {Ptr to a char. string. }
 PgmPtr : POINTER; {Ptr to FUNCTION to call}
 ShiftComb: BYTE; {Hot key--shift key comb}
 KeyChr : CHAR ); {Hot Key--character key.}
{
 ShiftComb and KeyChr specify the default hot keys for the TSR.
 ShiftComb may be created by adding or ORing the constants AltKey,
 CtrlKey, LeftKey, and RightKey together. KeyChr may be
 characters 0-9 and A-Z.

 The default hot keys may be overridden when the TSR is installed
 by specifying optional parameters on the command line. The
 parameter format is:
 [/A] [/C] [/R] [/L] [/"[K["]]]
 The square brackets surround optional items--do not include them.
 Any characters between parameters are ignored. The order of the
 characters does not matter; however, the shift keys specified are
 cummulative and the last character key "K" specified is the used.
}
{
** Functions for checking status of printer LPT1. }
FUNCTION PrinterStatus: BYTE; {Returns status of printer. }
FUNCTION PrinterOkay: BOOLEAN; {Returns TRUE if printer is okay.}


{
** Routines for obtaining one row of screen characters. }
FUNCTION ScreenLineStr( Row: BYTE ): String80; {Returns char. str.}
PROCEDURE ScreenLine( Row: BYTE; VAR Line: LineWords; {Returns }
 VAR Words: BYTE ); {chr & color}

IMPLEMENTATION {==================================================}
VAR
 BuffSize, InitCMode : WORD;
 NpxFlag : BOOLEAN;
 Buffer : ARRAY[0..8191] OF WORD;
 NpxState : ARRAY[0..93] OF BYTE;
 RetrnVal, InitVideo : BYTE;

CONST {Offsets to items contained in PROCEDURE Asm. }
 UnSafe = 0; Flg = 1; Key = 2; Shft = 3;
 StkOfs = 4; StkSs = 6; DosSp = 8; DosSs = 10;
 Prev = 12; Flg9 = 13; InsNumb = 14;
 Dos21 = $10; Dos25 = Dos21+4; Dos26 = Dos25+4;
 Bios9 = Dos26+4; Bios16 = Bios9+4; DosTab = Bios16+4;
 Our21 = DosTab+99; Our25 = Our21+51; Our26 = Our25+24;
 Our09 = Our26+24; Our16 = Our09+127+8; InsChr = Our16+180-8;
 PopUp = InsChr+4; Pgm = PopUp+4;

PROCEDURE Asm; {Inline code--data storage and intercept routines. }
INTERRUPT;
BEGIN
INLINE(
{*** Storage for interrupt vectors. }
 {Dos21: } >0/>0/ {DOS func. intr vector. }
 {Dos25: } >0/>0/ {DOS abs. disk read intr. vector. }
 {Dos26: } >0/>0/ {DOS abs. sector write intr.vector. }
 {Bios9: } >0/>0/ {BIOS key stroke intr. vector. }
 {Bios16: } >0/>0/ {BIOS buffered keybd. input intr.vect.}

 {DosTab: ARRAY[0..98] OF BYTE = {Non-reetrant DOS functions.}
 0/0/0/0/0/0/0/0/ 0/0/0/0/0/1/1/1/ 1/1/1/1/1/1/1/1/
 1/1/1/1/1/1/1/1/ 1/1/1/1/1/1/0/1/ 1/1/1/1/1/1/1/0/
 1/0/0/0/0/0/1/1/ 1/1/1/1/1/1/1/1/ 1/1/1/1/1/1/1/1/
 0/0/0/0/0/0/1/1/ 0/0/0/0/1/0/1/1/ 0/1/1/1/1/0/0/0/ 0/0/0/

{*** OurIntr21 ******* Intercept routine for DOS Function Intr.***}
{ 0} $9C/ { PUSHF ;Save flags. }
{ 1} $FB/ { STI ;Enable interrupts. }
{ 2} $80/$FC/$63/ { CMP AH,63H ;Assume unsafe if new }
{ 5} $73/<22-7/ { JNB IncF ;function--skip table.}
{ 7} $50/ { PUSH AX ;Save registers. }
{ 8} $53/ { PUSH BX ;Load offset to table.}
{ 9} $BB/>DosTab/ { MOV BX,[DosTab] }
{ 12} $8A/$C4/ { MOV AL,AH ;Load table entry }
{ 14} $2E/ { CS: ;index. }
{ 15} $D7/ { XLAT ;Get value from table.}
{ 16} $3C/$00/ { CMP AL,0 ;If TRUE then set flag}
{ 18} $5B/ { POP BX ;Restore registers. }
{ 19} $58/ { POP AX ; }
{ 20} $74/$17/ { JZ JmpDos21 ;Jump to orig. intr. }
{ 22} $2E/ {IncF: CS: ; }
{ 23} $FE/$06/>UnSafe/ { INC [UnSafe] ;Set UnSafe flag. }
{ 27} $9D/ { POPF ;Restore flags. }

{ 28} $9C/ { PUSHF ; }
{ 29} $2E/ { CS: ; }
{ 30} $FF/$1E/>Dos21/ { CALL FAR [Dos21] ;Call orig. intr. }
{ 34} $FB/ { STI ;Enable interrupts. }
{ 35} $9C/ { PUSHF ;Save flags. }
{ 36} $2E/ { CS: ; }
{ 37} $FE/$0E/>UnSafe/ { DEC [UnSafe] ;Clear UnSafe flag. }
{ 41} $9D/ { POPF ;Restore flags. }
{ 42} $CA/$02/$00/ { RETF 2 ;Return & remove flag.}

{ 45} $9D/ {JmpDos21: POPF ;Restore flags. }
{ 46} $2E/ { CS: ; }
{ 47} $FF/$2E/>Dos21/ { JMP FAR [Dos21] ;Jump to orig. intr. }
{ 51}
{*** OurIntr25 ********** Intercept routine for DOS Abs. Read *** }
{ 0} $9C/ { PUSHF ;Save flags. }
{ 1} $2E/ { CS: ; }
{ 2} $FE/$06/>UnSafe/ { INC [UnSafe] ;Set UnSafe flag. }
{ 6} $9D/ { POPF ;Restore flags. }
{ 7} $9C/ { PUSHF ; }
{ 8} $2E/ { CS: ; }
{ 9} $FF/$1E/>Dos25/ { CALL FAR [Dos25] ;Call DOS abs. read. }
{ 13} $83/$C4/$02/ { ADD SP,2 ;Clean up stack. }
{ 16} $9C/ { PUSHF ;Save flags. }
{ 17} $2E/ { CS: ; }
{ 18} $FE/$0E/>UnSafe/ { DEC [UnSafe] ;Clear UnSafe flag. }
{ 22} $9D/ { POPF ;Restore flags. Leave}
{ 23} $CB/ { RETF ;old flags on the stk.}
{ 24}
{*** OurIntr26 ********** Intercept routine for DOS Abs. Write ***}
{ 0} $9C/ { PUSHF ;Save flags. }
{ 1} $2E/ { CS: ; }
{ 2} $FE/$06/>UnSafe/ { INC [UnSafe] ;Set UnSafe flag. }
{ 6} $9D/ { POPF ;Restore flags. }
{ 7} $9C/ { PUSHF ; }
{ 8} $2E/ { CS: ; }
{ 9} $FF/$1E/>Dos26/ { CALL FAR [Dos26] ;Call DOS abs. write. }
{ 13} $83/$C4/$02/ { ADD SP,2 ;Clean up stack. }
{ 16} $9C/ { PUSHF ;Save flags. }
{ 17} $2E/ { CS: ; }
{ 18} $FE/$0E/>UnSafe/ { DEC [UnSafe] ;Clear UnSafe flag. }
{ 22} $9D/ { POPF ;Restore flags. Leave}
{ 23} $CB/ { RETF ;old flags on the stk.}
{ 24}

{*** OurIntr9 ********** Intercept for BIOS Hardware Keyboard Intr}
{ 0} $9C/ { PUSHF ;Entry point. }
{ 1} $FB/ { STI ;Enable interrupts. }
{ 2} $1E/ { PUSH DS ; }
{ 3} $0E/ { PUSH CS ;DS := CS; }
{ 4} $1F/ { POP DS ; }
{ 5} $50/ { PUSH AX ;Preserve AX on stack.}
{ 6} $31/$C0/ { XOR AX,AX ;Set AH to 0. }
{ 8} $E4/$60/ { IN AL,60h ;Read byte from keybd }
{ 10} $3C/$E0/ { CMP AL,0E0h ;If multi-byte codes, }
{ 12} $74/<75-14/ { JE Sfx ;then jump and set }
{ 14} $3C/$F0/ { CMP AL,0F0h ;multi-byte flag, Flg9}
{ 16} $74/<75-18/ { JE Sfx ; }
{ 18} $80/$3E/>Flg9/$00/ { CMP [Flg9],0 ;Exit if part of }

{ 23} $75/<77-25/ { JNZ Cfx ;multi-byte code. }
{ 25} $3A/$06/>Key/ { CMP AL,[Key] ;Exit if key pressed }
{ 29} $75/<88-31/ { JNE PreExit ;is not hot key. }

{ 31} $50/ { PUSH AX ;Hot key was pressed, }
{ 32} $06/ { PUSH ES ;check shift key }
{ 33} $B8/$40/$00/ { MOV AX,0040h ;status byte. First }
{ 36} $8E/$C0/ { MOV ES,AX ;load BIOS segment. }
{ 38} $26/ { ES: ; }
{ 39} $A0/>$0017/ { MOV AL,[0017h] ;AL:= Shift key status}
{ 42} $07/ { POP ES ;Restore ES register. }
{ 43} $24/$0F/ { AND AL,0Fh ;Clear unwanted bits. }
{ 45} $3A/$06/>Shft/ { CMP AL,[Shft] ;Exit if not hot key }
{ 49} $58/ { POP AX ;shift key combination}
{ 50} $75/<88-52/ { JNE PreExit ;(Restore AX first). }

 { ;Hot Keys encountered.}
{ 52} $3A/$06/>Prev/ { CMP AL,[Prev] ;Discard repeated hot }
{ 56} $74/<107-58/ { JE Discard ;key codes. }
{ 58} $A2/>Prev/ { MOV [Prev],AL ;Update Prev. }
{ 61} $F6/$06/>Flg/3/ { TEST [Flg],3 ;If Flg set, keep key }
{ 66} $75/<99-68/ { JNZ JmpBios9 ;& exit to orig. BIOS }
{ 68} $80/$0E/>Flg/1/ { OR [Flg],1 ;9. Else set flag and}
{ 73} $EB/<107-75/ { JMP SHORT Discard;discard key stroke. }

{ 75} $B4/$01/ {Sfx: MOV AH,1 ;Load AH with set flag}
{ 77} $88/$26/>Flg9/ {Cfx: MOV [Flg9],AH ;Save multi-byte flag.}
{ 81} $C6/$06/>Prev/$FF/ { MOV [Prev],0FFh ;Change prev key byte.}
{ 86} $EB/<99-88/ { JMP SHORT JmpBios9 }

{ 88} $3C/$FF/ {PreExit: CMP AL,0FFh ;Update previous key }
{ 90} $74/<99-92/ { JE JmpBios9 ;unless key is buffer-}
{ 92} $3C/$00/ { CMP AL,0 ;full code--a 00h }
{ 94} $74/<99-96/ { JZ JmpBios9 ;0FFh }
{ 96} $A2/>Prev/ { MOV [Prev],AL ;Update previous key. }

{ 99} $58/ {JmpBios9: POP AX ;Restore registers and}
{100} $1F/ { POP DS ;flags. }
{101} $9D/ { POPF ; }
{102} $2E/ { CS: ; }
{103} $FF/$2E/>Bios9/ { JMP [Bios9] ;Exit to orig. intr 9.}

{107} $E4/$61/ {Discard: IN AL,61h ;Clear key from buffer}
{109} $8A/$E0/ { MOV AH,AL ;by resetting keyboard}
{111} $0C/$80/ { OR AL,80h ;port and sending EOI }
{113} $E6/$61/ { OUT 61h,AL ;to intr. handler }
{115} $86/$E0/ { XCHG AH,AL ;telling it that the }
{117} $E6/$61/ { OUT 61h,AL ;key has been }
{119} $B0/$20/ { MOV AL,20h ;processed. }
{121} $E6/$20/ { OUT 20h,AL ; }
{123} $58/ { POP AX ;Restore registers and}
{124} $1F/ { POP DS ;flags. }
{125} $9D/ { POPF ; }
{126} $CF/ { IRET ;Return from interrupt}
{127}

{*** OurIntr16 ***** Intercept routine for Buffered Keyboard Input}
{ 0} $58/ {JmpBios16: POP AX ;Restore AX, DS, and }
{ 1} $1F/ { POP DS ;FLAGS registers then }

{ 2} $9D/ { POPF ;exit to orig. BIOS }
{ 3} $2E/ { CS: ;intr. 16h routine. }
{ 4} $FF/$2E/>Bios16/ { JMP [Bios16] ; }

{ 8} $9C/ {OurIntr16: PUSHF ;Preserve FLAGS. }
{ 9} $FB/ { STI ;Enable interrupts. }
{ 10} $1E/ { PUSH DS ;Preserve DS and AX }
{ 11} $50/ { PUSH AX ;registers. }
{ 12} $0E/ { PUSH CS ;DS := CS; }
{ 13} $1F/ { POP DS ; }
{ 14} $F6/$C4/$EF/ { TEST AH,EFh ;Jmp if not read char.}
{ 17} $75/<48-19/ { JNZ C3 ;request. }

 {*** Intercept loop for Read Key service.}
{ 19} $F6/$06/>Flg/1/ {C1: TEST [Flg],1 ;If pop up Flg bit is }
{ 24} $74/<29-26/ { JZ C2 ;set then call INLINE }
{ 26} $E8/>122-29/ { CALL ToPopUp ;pop up routine. }
{ 29} $F6/$06/>Flg/16/{C2: TEST [Flg],10h ;Jmp if insert flg set}
{ 34} $75/<48-36/ { JNZ C3 ; }
{ 36} $FE/$C4/ { INC AH ;Use orig. BIOS }
{ 38} $9C/ { PUSHF ;service to check for }
{ 39} $FA/ { CLI ;character ready. }
{ 40} $FF/$1E/>Bios16/ { CALL FAR [Bios16];Disable interrupts. }
{ 44} $58/ { POP AX ;Restore AX and save }
{ 45} $50/ { PUSH AX ;it again. }
{ 46} $74/<19-48/ { JZ C1 ;Loop until chr. ready}

{ 48} $F6/$06/>Flg/17/{C3: TEST [Flg],11h ;Exit if neither bit }
{ 53} $74/<-55/ { JZ JmpBios16 ;of Flg is set. }
{ 55} $F6/$06/>Flg/$01/ { TEST [Flg],1 ;If pop up Flg bit is }
{ 60} $74/<65-62/ { JZ C4 ;set then call INLINE }
{ 62} $E8/>122-65/ { CALL ToPopUp ;pop up routine. }
{ 65} $F6/$06/>Flg/$10/{C4:TEST [Flg],10h ;Exit unless have }
{ 70} $74/<-72/ { JZ JmpBios16 ;characters to insert.}
{ 72} $F6/$C4/$EE/ { TEST AH,0EEh ;If request is not a }
{ 75} $75/<-77/ { JNZ JmpBios16 ;chr. request, exit. }

 {*** Insert a character. }
{ 77} $58/ { POP AX ;AX := BIOS service no}
{ 78} $53/ { PUSH BX ;Save BX and ES. }
{ 79} $06/ { PUSH ES ; }
{ 80} $C4/$1E/>InsChr/ { LES BX,[InsChr] ;PTR(ES,BX) := InsChr;}
{ 84} $26/ { ES: ;AL := InsChr^; }
{ 85} $8A/$07/ { MOV AL,[BX] ; }
{ 87} $07/ { POP ES ;Restore ES and BX. }
{ 88} $5B/ { POP BX ; }
{ 89} $F6/$C4/$01/ { TEST AH,01h ;IF AH IN [$01,$11] }
{ 92} $B4/$00/ { MOV AH,00h ; THEN ReportOnly; }
{ 94} $75/<114-96/ { JNZ ReportOnly ;Set Scan code to 0. }
{ 96} $FE/$06/>InsChr/ { INC [InsChr] ;Inc( InsChr ); }
{100} $FF/$0E/>InsNumb/ { DEC [InsNumb] ;Dec( InsNumb ); }
{104} $75/<111-106/ { JNZ SkipReset ;IF InsNumb = 0 THEN }
{106} $80/$26/>Flg/$EF/ { AND [Flg],0EFh ; Clear insert chr flg}
{111} $1F/ {SkipReset: POP DS ;Restore BX, DS, and }
{112} $9D/ { POPF ;FLAGS, then return }
{113} $CF/ { IRET ;from interrupt. }

{114} $1F/ {ReportOnly: POP DS ;Report char. ready. }
{115} $9D/ { POPF ;Restore DS and FLAGS.}

{116} $50/ { PUSH AX ;Clear zero flag bit }
{117} $40/ { INC AX ;to indicate a }
{118} $58/ { POP AX ;character ready. }
{119} $CA/>0002/ { RETF 2 ;Exit & discard FLAGS }

 {*** Interface to PopUpCode Routine. }
{122} $50/ {ToPopUp: PUSH AX ;Save AX. }
{123} $FA/ { CLI ;Disable interrupts. }
{124} $F6/$06/>UnSafe/$FF/{TEST [UnSafe],0FFh ;IF UnSafe <> 0 }
{129} $75/<177-131/ { JNZ PP2 ; THEN Return. }
{131} $A0/>Flg/ { MOV AL,[Flg] ;Set in-use bit; clear}
{134} $24/$FE/ { AND AL,0FEh ;pop up bit of Flg. }
{136} $0C/$02/ { OR AL,2 ;Flg := (Flg AND $FE) }
{138} $A2/>Flg/ { MOV [Flg],AL ; OR 2; }
 { ;**Switch to our stack}
{141} $A1/>StkOfs/ { MOV AX,[StkOfs] ;Load top of our stack}
{144} $87/$C4/ { XCHG AX,SP ;Exchange it with }
{146} $A3/>DosSp/ { MOV [DosSp],AX ;stk.ptr, save old SP.}
{149} $8C/$16/>DosSs/ { MOV [DosSs],SS ;Save old SS. }
{153} $8E/$16/>StkSs/ { MOV SS,[StkSs] ;Replace SS with our }
{157} $FB/ { STI ;SS. Enable interrupts}

{158} $9C/ { PUSHF ;Interrupt call to pop}
{159} $FF/$1E/>PopUp/ { CALL FAR [PopUp] ;up TSR routine. }

{163} $FA/ { CLI ;Disable interrupts. }
{164} $8B/$26/>DosSp/ { MOV SP,[DosSp] ;Restore stack ptr }
{168} $8E/$16/>DosSs/ { MOV SS,[DosSs] ;SS:SP. Clear in-use }
{172} $80/$26/>Flg/$FD/ { AND [Flg],0FDh ;bit of Flg. }

{177} $FB/ {PP2: STI ;Enable interrupts. }
{178} $58/ { POP AX ;Restore AX. }
{179} $C3 ); { RET ;Return. }
{180}
END; {Asm.} {END corresponds to 12 bytes of code--used for storage}

PROCEDURE PopUpCode; {Interface between the BIOS intercept }
INTERRUPT; {routines and your TSR function. }
CONST BSeg = $0040; VBiosOfs = $49;
TYPE
 VideoRecs = RECORD
 VideoMode : BYTE;
 NumbCol, ScreenSize, MemoryOfs : WORD;
 CursorArea : ARRAY[0..7] OF WORD;
 CursorMode : WORD;
 CurrentPage : BYTE;
 VideoBoardAddr : WORD;
 CurrentMode, CurrentColor : BYTE;
 END;
VAR
 Regs : Registers;
 VideoRec : VideoRecs;
 KeyLock : BYTE;
 ScrnSeg : WORD;
BEGIN
 SwapVectors; {Set T.P. intr. vectors.}
 Move( Ptr(BSeg,VBiosOfs)^, VideoRec, {Get Video BIOS info. }
 SizeOf(VideoRec) );
 WITH VideoRec, Regs DO BEGIN

 IF (VideoMode > 7) OR {Abort pop up if unable}
 (ScreenSize > BuffSize) THEN BEGIN {to save screen image. }
 SwapVectors; {Restore intr. vectors.}
 Exit;
 END;
 KeyLock := Mem[BSeg:$0017]; {Save lock key states. }
 IF VideoMode = 7 THEN ScrnSeg := $B000 {Save screen--supports }
 ELSE ScrnSeg := $B800; {text, MGA & CGA modes.}
 Move( PTR( ScrnSeg, MemoryOfs )^, Buffer, ScreenSize );
 AX := InitVideo; {If in graphics mode, }
 IF (VideoMode >=4) {switch to text mode. }
 AND (VideoMode <= 6) THEN Intr( $10, Regs );
 AX := $0500; {Select display page 0.}
 Intr( $10, Regs );
 CX := InitCMode; {Set cursor size. }
 AH := 1;
 Intr( $10, Regs );

 TSRMode := VideoMode; {Fill global variables }
 TSRWidth := NumbCol; {with current information}
 TSRPage := CurrentPage;
 TSRColumn := Succ( Lo( CursorArea[CurrentPage] ) );
 TSRRow := Succ( Hi( CursorArea[CurrentPage] ) );

 IF NpxFlag THEN {Save co-processor state.}
 INLINE( $98/ $DD/$36/>NpxState ); {WAIT FSAVE [NpxState] }
{
*** Call user's program and save return code--no. char. to insert.
}
 INLINE( $2E/$FF/$1E/>Pgm/ { CALL FAR CS:[Pgm] }
 $2E/$A3/>InsNumb ); { MOV CS:[InsNumb],AX }

 IF Mem[CSeg:InsNumb] > 0 THEN BEGIN {Have char. to insert. }
 MemL[CSeg:InsChr] := LONGINT( TSRChrPtr );
 Mem[CSeg:Flg] := Mem[CSeg:Flg] OR $10;
 END;
{
*** Pop TSR back down--Restore computer to previous state.
}
 IF NpxFlag THEN {Restore co-prcssr state.}
 INLINE( $98/ $DD/$36/>NpxState ); {WAIT FSAVE [NpxState] }

 Mem[BSeg:$17] := {Restore key lock status.}
 (Mem[BSeg:$17] AND $0F) OR (KeyLock AND $F0);

 IF Mem[BSeg:VBiosOfs] <> VideoMode THEN BEGIN
 AX := VideoMode; {Restore video mode. }
 Intr( $10, Regs );
 END;
 AH := 1; CX := CursorMode; {Restore cursor size. }
 Intr( $10, Regs );
 AH := 5; AL := CurrentPage; {Restore active page. }
 Intr( $10, Regs );
 AH := 2; BH := CurrentPage; {Restore cursor positon. }
 DX := CursorArea[CurrentPage];
 Intr( $10, Regs ); {Restore screen image. }
 Move( Buffer, PTR( ScrnSeg, MemoryOfs )^, ScreenSize );

 SwapVectors; {Restore non-T.P. vectors.}

 END;
END; {PopUp.}
{
***** Printer Functions:
}
FUNCTION PrinterStatus: BYTE; {Returns status of LPT1.}
{ Definition of status byte bits (1 & 2 are not used), if set then:
 Bit: -- 7 --- ---- 6 ---- -- 5 --- -- 4 --- -- 3 -- --- 0 ---
 Not busy Acknowledge No paper Selected I/O Err. Timed-out
}
VAR Regs : Registers;
BEGIN
 WITH Regs DO BEGIN
 AH := 2; DX := 0; {Load BIOS function and printer number. }
 Intr( $17, Regs ); {Call BIOS printer services. }
 PrinterStatus := AH; {Return with printer status byte. }
 END;
END; {PrinterStatus.}

FUNCTION PrinterOkay: BOOLEAN; {Returns TRUE if printer is okay. }
VAR S : BYTE;
BEGIN
 S := PrinterStatus;
 IF ((S AND $10) <> 0) AND ((S AND $29) = 0) THEN
 PrinterOkay := TRUE
 ELSE PrinterOkay := FALSE;
END; {PrinterOkay.}
{
***** Procedures to obtain contents of saved screen image.
}
PROCEDURE ScreenLine( Row: BYTE; VAR Line: LineWords;
 VAR Words: BYTE );
BEGIN
 Words := 40; {Determine screen line size.}
 IF TSRMode > 1 THEN Words := Words*2; {Get line's }
 Move( Buffer[Pred(Row)*Words], Line, Words*2 ); {characters and }
END; {ScreenLine.} {colors. }

FUNCTION ScreenLineStr( Row: BYTE ): String80; {Returns just chars}
VAR
 Words, i : BYTE;
 LineWord : LineWords;
 Line : String80;
BEGIN
 ScreenLine( Row, LineWord, Words ); {Get chars & attributes. }
 Line := ''; {Move characters to string}
 FOR i := 1 TO Words DO Insert( LineWord[i].C, Line, i );
 ScreenLineStr := Line;
END; {ScreenString.}
{
***** TSR Installation procedure.
}
PROCEDURE TSRInstall( PgmName, PgmPtr: POINTER;
 ShiftComb: BYTE; KeyChr: CHAR );
CONST
 ScanChr = '+1234567890++++QWERTYUIOP++++ASDFGHJKL+++++ZXCVBNM';
 CombChr = 'RLCA"';
VAR
 PgmNamePtr, PlistPtr: ^STRING;

 i, j, k : WORD;
 Regs : Registers;
 Comb, ScanCode : BYTE;
BEGIN
 IF Ofs( Asm ) <> 0 THEN EXIT; {Offset of Asm must be 0}
 MemW[CSeg:StkSs] := SSeg; {Save pointer to top of }
 MemW[CSeg:StkOfs] := Sptr + 308; {TSR's stack. }
 MemL[CSeg:PopUp] := LONGINT(@PopUpCode); {Save PopUpCode addr. }
 MemL[CSeg:Pgm] := LONGINT(PgmPtr); {Save PgmPtr. }
 PgmNamePtr := PgmName; {Convert ptr to string ptr.}

 Writeln('Installing Stay-Resident program: ',PgmNamePtr^);
{
***** Save intercepted interrupt vectors: $09, $16, $21, $25, $26.
}
 GetIntVec( $09, POINTER( MemL[CSeg:Bios9] ) );
 GetIntVec( $16, POINTER( MemL[CSeg:Bios16] ) );
 GetIntVec( $21, POINTER( MemL[CSeg:Dos21] ) );
 GetIntVec( $25, POINTER( MemL[CSeg:Dos25] ) );
 GetIntVec( $26, POINTER( MemL[CSeg:Dos26] ) );
{
***** Get equipment list and video mode.
}
 WITH Regs DO BEGIN
 Intr( $11, Regs ); {Check equipment list for }
 NpxFlag := (AL AND 2) = 2; {math co-processor. }
 AH := 15; {Get current video mode }
 Intr( $10, Regs ); {and save it for when TSR }
 InitVideo := AL; {is activated. }
 AH := 3; BH := 0; {Get current cursor size }
 Intr( $10, Regs ); {and save it for when TSR }
 InitCMode := CX; {is activated. }
 END; {WITH Regs}
{
***** Get info. on buffer for saving screen image.
}
 BuffSize := SizeOf( Buffer );
 TSRScrPtr := @Buffer;
{
*** Determine activation key combination.
}
 Comb := 0; i := 1; {Create ptr to }
 PlistPtr := Ptr( PrefixSeg, $80 ); {parameter list. }
 WHILE i < Length( PlistPtr^ ) DO BEGIN {Check for parameters.}
 IF PlistPtr^[i] = '/' THEN BEGIN {Process parameter. }
 Inc( i );
 j := Pos( UpCase( PlistPtr^[i] ), CombChr );
 IF (j > 0) AND (j < 5) THEN Comb := Comb OR (1 SHL Pred(j))
 ELSE IF j <> 0 THEN BEGIN {New activation char. }
 Inc( i ); k := Succ( i );
 IF i > Length(PlistPtr^) THEN KeyChr := #0
 ELSE BEGIN
 IF ((k <= Length(PlistPtr^)) AND (PlistPtr^[k] = '"'))
 OR (PlistPtr^[i] <> '"') THEN KeyChr := PlistPtr^[i]
 ELSE KeyChr := #0;
 END; {ELSE BEGIN}
 END; {ELSE IF ... BEGIN}
 END; {IF PlistPtr^[i] = '/'}
 Inc( i );

 END; {WHILE ...}
 IF Comb = 0 THEN Comb := ShiftComb; {Use default combination. }
 IF Comb = 0 THEN Comb := AltKey; {No default, use [Alt] key.}
 ScanCode := Pos( UpCase( KeyChr ), ScanChr ); {Convert char. to}
 IF ScanCode < 2 THEN BEGIN {scan code. }
 ScanCode := 2; KeyChr := '1';
 END;
 Mem[CSeg:Shft] := Comb; {Store shift key combination}
 Mem[CSeg:Key] := ScanCode; {and scan code. }
{
*** Output an installation message: Memory used & activation code.
}
 Writeln( 'Memory used is approximately ',
 ( ($1000 + Seg(FreePtr^) - PrefixSeg)/64.0):7:1,' K (K=1024).');
 Writeln(
'Activate program by pressing the following keys simultaneously:');
 IF (Comb AND 1) <> 0 THEN Write(' [Right Shift]');
 IF (Comb AND 2) <> 0 THEN Write(' [Left Shift]');
 IF (Comb AND 4) <> 0 THEN Write(' [Ctrl]');
 IF (Comb AND 8) <> 0 THEN Write(' [Alt]');
 Writeln(' and "', KeyChr, '".');
{
*** Intercept orig. interrupt vectors; Then exit and stay-resident.
}
 SetIntVec( $21, Ptr( CSeg, Our21 ) );
 SetIntVec( $25, Ptr( CSeg, Our25 ) );
 SetIntVec( $26, Ptr( CSeg, Our26 ) );
 SetIntVec( $16, Ptr( CSeg, Our16 ) );
 SetIntVec( $09, Ptr( CSeg, Our09 ) );
 SwapVectors; {Save turbo intr.vectors.}
 MemW[CSeg:UnSafe] := 0; {Allow TSR to pop up. }
 Keep( 0 ); {Exit and stay-resident. }
END; {TSRInstall.}
END. {TSRUnit.}




























June, 1989
MAINTAINING SYSTEM SECURITY


Dale Moir


Dale handles the Unix system security auditing for Lachman Associates Inc. He
can be reached at 1901 Naper Blvd., Naperville IL 60540-1031.


Maintaining computer security, whether it be "open" systems like the Unix
system or "closed" systems like those carrying government approved security
ratings, involves a wide range of complex issues. Among the more pressing
security issues are vulnerabilities in the user file system and threats to
network security. Although, these areas are not necessarily related, similar
approaches can be applied to each.


Vulnerabilities to User File Systems


Computer system administrators are responsible for maintaining security,
sanity, and day-to-day functionality. To some extent, users are capable of
interfering with most or all of these goals and there are several possible
security risks that can be caused by naive or inattentive users. Because
threats to computer security can come from the outside in the form of hackers
or computer intruders or from the inside in the form of malicious or
disgruntled employees, there exists a need to protect users from each other
and from themselves. With this in mind, I'll look first at the vulnerabilities
that can occur in the user file system or the area where users store their
programs and data, the "home directory."
File permissions determine whether a file is readable, writable, or executable
and are an important component of information access control in any system.
Consider the case where a malicious user wants to dupe a naive user into
executing some command. Perhaps the simplest way to achieve this is by placing
that command in an executable file owned by the naive user. This can be
accomplished if the naive user has an executable file that is writable by
users other than himself. The malicious user need only place the malicious
commands in the writable executable file and wait for the user to run the
contaminated program.
Consider the following "malicious command:"
 cd$HOME rm-rf*
The first command, derived from Unix, changes the current directory to be the
user's home directory. The second command recursively removes all files and
directories located beneath the current directory. Thus, if a malicious user
were able to cause an unsuspecting user to execute these commands, then the
unsuspecting user would unknowingly remove all his files and directories. Of
course, there are more subtle methods of attack available to the malicious
user but this one is quite effective.
The vulnerability in this case stems from the fact that one user is "giving
away" his execution privilege to another in the form of a writable executable
file. The point is that all executable files imply an execution privilege --
some user will be invoking that process at some time or another. As such, all
executable files should be write protected, just like the system executable
files. Assuming that you have access to a list of a user's home directories,
it is a good idea to look for vulnerable user files; a recursive algorithm
like that in Example 1 shows a way of keeping tabs on vulnerable files.
Example 1: A recursive algorithm like this provides one means of keeping tabs
on vulnerable files.

 FOR (every users' home directory) DO
 checkdir (directory)
 DONE

 PROCEDURE checkdir (directory)
 FOR (every file in the directory) DO
 IF ((file is executable) AND (file is writable)) THEN flag this file
 ELSE IF (file is a directory) THEN checkdir (file)
 DONE
 ENDPROCEDURE

Many people seem to believe that a write-protected file is always safe, even
if it resides in a writable directory. A writable directory, however, allows
other users to move (or rename) write-protected files. Once done, a file of
the same name can be created in the same writable directory. As an example,
consider the case where a user's start-up file, such as profile in Unix, is
write protected, but the user's home directory is writable. A malicious user
would be able to rename the start-up file and create a new file called
"profile." This new start-up file could contain the malicious shell commands
shown earlier. When the naive user logs in, the malicious start-up file will
be executed by the log-in shell. Even a write-protected file is vulnerable if
the directory it resides in is not write protected, too.
Files and directories should always be write protected, especially executable
files and directories containing executable files. A file that is not
executable but is writable might also present a problem. On systems where
users are billed for disk utilization, for example, it would be cost effective
for malicious users to append their data files to other users' writable files
or place them in other users' directories. To address the more general issue
of write protection, you might expand the program in Example 1 to that in
Example 2.
Example 2: To handle the more general issue of write protection you might
expand the code in Example 1 to this.

 FOR (every user's home directory) DO
 checkdir(directory)
 DONE

 PROCEDURE checkdir (directory)
 IF (directory is writable) THEN flag this directory
 FOR (every file in the directory) DO
 IF ((file is executable) AND (file is writable)) THEN flag this file
 IF (file is writable) THEN maybe flag this file
 ELSE IF (file is directory) THEN checkdir(file)
 DONE
 ENDPROCEDURE

The methods of attack described so far are often termed "Trojan horse"
attacks. There are general search techniques that might be used to search for
them. The code example in Example 3 seeks to eliminate the risk of a Trojan
horse attack. You might also include in your examination of users' home
directories a search for possible existing Trojan horses. To do so, you need
to know what a Trojan horse might look like.
Example 3: This code presents one way to eliminate the risk of Trojan horse
attack.

 FOR (every file in the file system) DO
 IF (filename is in likely-list) THEN flag this file
 DONE


First, you know that to be useful, a Trojan horse program must be executed.
Because users' commands are usually limited to the set of commands provided by
the system, a successful Trojan horse might seek to masquerade as a system
command. The simplest method of achieving this is to create an executable file
with the same name as a commonly used system command. This type of Trojan
horse can be placed in any writable directory anywhere in the file system.
An effective method of searching for existing Trojan horses consists of two
steps. First, define a list of "likely" Trojan horse names derived from
commonly used system commands. Then, search the entire file system for files
having these names. The program might look like that in Example 3.
The actual implementation might be improved by including a mechanism for
defining a list of files "authorized" to have a given name and ignoring those
files in the search. By searching the entire file system, user and system
areas are checked for the existence of potential security risks.


Network Security


In the previous section, I described some tools that help to protect users
from themselves and from other users. In this section, I'll focus more on
outside threats to network security.
A network connection should be treated as a doorway to the world. The security
considerations center on two basic issues: 1. What should be permitted to
enter the system via the network? 2. What should be permitted to leave the
system via the network?
One way to impose security on a networked system is by controlling the
capabilities of the network software. This feature, if available, is usually
provided through some sort of configuration mechanism. By imposing control
over the network configuration you can impose some level of control over
network security.
A brief description of some common network threats will help determine what
type of network configuration is appropriate. Information entering your system
may pose a variety of threats. If a remote user or site can arrange for an
executable file to be placed on your system, then the threat of a Trojan horse
or a virus attack exists. If a remote user or site can directly execute
commands, then the threat of a direct attack on users or resources exists.
Similarly, information leaving your system may divulge information useful to
an intruder for planning other types of attacks. In general, your network
configuration should be as limited as possible while still allowing a
reasonable level of communication.
Having defined an acceptable network configuration, the next step is to
automate the process of verifying the configuration. By way of example I'll
describe the capabilities of a common network utility and show how such a
verification program might be written. The example I'll use is derived from
Unix where the most common network connection is called uucp (Unix to Unix
copy).
The uucp facility is controlled by a series of configuration files that
determine the capabilities of each remote system. The available capabilities
include remote file copy in, local file copy out, and remote command
execution. The configuration files determine what parts of the local life
system can be accessed by a remote machine and what commands are available for
remote execution. Thus, for the uucp facility the configuration files
determine the network capabilities. The verification tool will examine the
configuration files relative to the predefined level of network security
desired.
Let's assume that some remote clients are not completely trusted and a
reasonably controlled network environment is called for. In this case, you
might elect to limit the access of all remote machines to one single
directory. Thus, any information entering or leaving your system must be
copied into or out of a single location. Similarly, you might elect to limit
the remotely executable commands to those that are designed specifically for
remote execution. In reality, this configuration is quite reasonable -- users
can send and receive files, and commands designed for remote execution are
made accessible.
The next step is to create a software tool that checks the network
configuration files, and ensures that the network policy decisions are
reflected in the actual configuration. Using the policy described earlier you
can construct an example of such a program. Let's further assume that a
directory named public has been reserved for the remote machine access point,
and that the commands available for remote execution are called rmail and lpr.
A simple checking program would look like that in Example 4.
Example 4: One approach to network checking.

 FOR (each remote host that is defined) DO
 IF (directories other than "public" accessible) THEN flag this host
 FOR (each command name in command list) DO
 IF ((command !="rmail") AND (command!="lpr")) THEN flag this command
 DONE
 DONE

For those of you familiar with uucp, this example may seem trivial. And in
truth, a robust uucp checking program would perform many more checks than
those shown here. Nevertheless, this program is effective and the explanatory
text for uucp is neatly avoided. The reason the approach in Example 4 is
effective is because it automates the task of maintaining security. Any time
you can incorporate your security policy into an administrative tool, you have
succeeded in achieving several goals. First of all, you acquire a means by
which your administrator can audit the current state of the system against the
criteria suggested by your security policy. Given such a tool, the
administrative overhead of maintaining security is greatly reduced. Second,
your security policy is no longer just a virtual set of guidelines. By
incorporating the policy into your software tools, the policy itself has a
physical representation. Given this, it becomes much more likely that the
policy will be adhered to. By externalizing the security policy, the effort
required to train a new administrator is also reduced.


Conclusion


Whether your concern is based on user vulnerability or attacks from remotely
connected network sites the solution is the same. A good security policy is
made much more effective, simpler, and easier to enforce when the tasks are
automated. After all, policy without practice is no policy at all.


Maintaining System Security
by Dale Moir


Example 1. A recursive algorithm like this provides one means of
keeping tabs on vulnerable files.


 FOR (every users' home directory) DO
 checkdir(directory)
 DONE

 PROCEDURE checkdir (directory)
 FOR (every file in the directory) DO
 IF ((file is executable) AND (file is writable)) THEN flag this file
 ELSE IF (file is a directory) THEN checkdir(file)
 DONE
 ENDPROCEDURE



Example 2. To handle the more general issue of write-protection,
you might expand the program in Example 1 to this.



 FOR (every user's home directory) DO
 checkdir(directory)
 DONE

 PROCEDURE checkdir (directory)
 IF (directory is writable) THEN flag this directory
 FOR (every file in the directory) DO
 IF ((file is executable) AND (file is writable)) THEN flag this file
 IF (file is writable) THEN maybe flag this file
 ELSE IF (file is directory) THEN checkdir(file)
 DONE
 ENDPROCEDURE



Example 3. This code presents one way to eliminate the risk of
trojan horse attack.


 FOR (every file in the file system) DO
 IF (filename is in likely-list) THEN flag this file
 DONE



Example 4. A sample network checking program

 FOR (each remote host that is defined) DO
 IF (directories other than "public" accessible) THEN flag this host
 FOR (each command name in command list) DO
 IF ((command !="rmail") AND (command!="lpr")) THEN flag this command
 DONE
 DONE




























June, 1989
GENERATING PARSERS WITH PCYACC


C ++, SQL, Smalltalk, and PostScript are just a few of the grammars provided
with this parser generator




Alex Lane


Alex Lane is a knowledge engineer for Technology Applications Inc. of
Jacksonville, Fla. He can be reached as a lane on BIX and as ALANE on MCI
mail.


YACC is an acronym for Yet Another Compiler Compiler. Its name reflects both
the preoccupation with parser generators that engaged computer scientists back
in the early 70s, as well as Unix's sometimes whimsical naming practices. For
many years, YACC (or more properly, yacc) has been a powerful basic tool in
the toolchests of compiler writers working with Unix-based systems. Recently,
however, several companies have ported yacc to the MS-DOS environment. Among
them is Abraxas Software, which publishes PCYACC. I reviewed Version 2.0 of
the product on an ARC 386i equipped with 1 Mbyte of RAM and a hard disk.
A bare-bones definition of a compiler is "a program that reads an input file
of instructions written in one language and translates them into an output
file of instructions in another, lower-level language." Although the most
common form of output file that the casual compiler user deals with is a
linkable object code file, the output file may contain statements in any
language, including a high-level language.
Indeed, the output from PCYACC is a C source program. This C program can, in
turn, be compiled and linked to form an executable file. This executable file
is capable, in its turn, of parsing input from a file written in whatever
language was specified in the input files for PCYACC. It's perhaps a mite
confusing in words, so maybe a glance at Figure 1 will help clarify the
relationships among all these files.
The input, or source language, for PCYACC is what the folks at Abraxas call a
grammar description language (GDL). Statements in this language resemble
Backus-Naur Form (BNF) notation, which is commonly used to describe
context-free grammars.
A PCYACC input program, called a grammar description program (GDP), is made up
of three sections, arranged in the following order: a declaration section, a
grammar rule section, and a program section. The three sections are separated
from one another by the delimiter %%.
In the declaration section, anything enclosed with %{ and %} is passed
directly to the output without change. This is a convenient place for C
declarations and itemization of #include files. PCYACC tokens are identified
in this section by the keyword %token, and operator associativity is declared
using the %left, %right, and %nonassoc keywords. Precedence of operators is
implied by the order in which associativity declarations appear. Data types of
grammar symbols are declared with the keyword %type, and the keyword %start
can be used to declare the start symbol of the grammar.
The second section of the GDP contains all the grammar description language
statements for the target compiler. Statements in GDL have the following form:
 LHS: RHS {C code segment};
The LHS (left-hand side) of the rule consists of a nonterminal symbol. The RHS
(right-hand side) is a sequence of zero or more grammar symbols. Following the
RHS is a segment of C code that describes an associated action to be performed
when the RHS is encountered. The LHS and RHS are separated by a colon, and the
rule itself is terminated with a semicolon. Multiple rules having a common LHS
are often expressed using a vertical bar to denote an OR choice, like this:
 LHS : RHS1 {C code for RHS1}; RHS2 {C code for RHS2}; ...
Everything in the program section of the PCYACC input file is copied to the
output without change. The manual states that the program section must contain
definitions for at least three functions: the familiar main( ), a
lexical-analyzer function called yylex( ), and an error-reporting function
yyerror( ). I found that although definitions for these three functions must
appear somewhere in the C source code for the target parsing program, the
auxiliary files supplied with PCYACC show that the definitions do not, as
stated in the book, have to be in the GDP.
The output of the PCYACC program is a C source file that contains any code
passed in from the declaration section and all the code from the program
section. In addition, the output file contains the code for a function called
yyparse( ), which was generated using the grammar definition rules.
The yyparse( ) function performs look-ahead left and right (LALR) parsing for
the language using the rules from the PCYACC input file. The LALR method is an
efficient, bottom-up syntax-analysis technique that is used to parse virtually
any programming language constructs expressible in context-free grammar terms.
The yyparse( ) function is fed by a user-written function called yylex( ),
which performs lexical analysis. Basically this means that the input is broken
into meaningful chunks called "tokens," which are in turn passed to the
parser.


What You Get


The PCYACC package consists of a 126-page spiral bound, 8 1/2 X 11-inch manual
entitled Compiler Construction on Personal Computers (with PCYACC) and three
360K diskettes. The main PCYACC diskette contains the basic PCYACC. EXE
program, a couple of explanatory text files, and several subdirectories of
fairly simple examples. A second diskette provides grammar description files
for several languages, including K&R and ANSI C, Pascal, dBase III and IV,
SQL, Smalltalk, Prolog, PostScript, and HyperTalk. The third diskette supplies
a fairly complete set of grammar definition and C source files for a C++
preprocessor.
The PCYACC manual assumes that users are familiar with C, which is not an
unreasonable assumption under the circumstances. It is well organized, makes
good use of fonts and white space, and for the most part is clear in its
explanations. Following a brief overview that includes an explanation of
typographical conventions in the manual, the book uses two fairly simple
examples as a basis for presenting PCYACC concepts. Much of the rest of the
book then concerns itself with the theory and practice behind PCYACC,
including techniques for debugging. There will be tough going here for anyone
just becoming familiar with parser generators and yacc, but between the clear
explanations and the numerous examples that come with PCYACC, it's an
achievable goal.
From a hardware perspective, PCYACC works on most PC-style microcomputers,
including PS/2 and 386-based machines. PCYACC can be run using either two
floppy drives or a hard disk and one floppy drive. Software required to do
compiler development with PCYACC includes a text editor (such as Brief) and a
C language compiler. For those who use the Brief product, Abraxas supplies two
macro files (in both compiled and source format) for use with the editor.
Installation is straightforward; I merely copied the files (and
subdirectories) into a subdirectory on my hard disk.


Using PCYACC


Developing software using PCYACC is a six-step procedure. First, as with any
project, you identify the problem. Because, typically, parsing input is only
part of the problem, PCYACC does not play a central role at this stage of the
development process.
The second step is to define the source language. This definition is most
commonly done using a context-free grammar notation such as BNF, although the
specification for the grammar can be written directly in PCYACC's GDL. The
third step is to incorporate the rules into a PCYACC grammar description file.
Next, the grammar rules should have associated action code written for them.
The required support routines (main( ), yylex( ), and yyerror( )) should be
coded to allow debugging of the grammar description program. This is an ideal
place to use the -S command-line option, which allows the grammar description
to be checked without going to the added trouble of generating a C source
file. You can really appreciate this feature when working with large grammar
definitions. You use PCYACC's other command-line options mostly to override
default file-name conventions or to indicate actions you want the program to
perform in addition to what it does automatically. Using the help option (-h),
you can get a brief synopsis of all available options.
Perhaps the second most valuable service provided by a compiler is the
reporting of errors in input files, where accuracy and detail count for a lot
of time saved tracking down subtle errors in logic. Here, PCYACC takes
advantage of the LALR parser's capability to detect syntactic errors as soon
as possible while performing a left-to-right scan of the input. This results
in good error trapping while generating the parser. In addition, there are a
couple of features of PCYACC you can use to enhance the debugging process.
You can, for example, use the -t command-line option mentioned previously to
build a parser that outputs text file parse trees to the default file YY.AST.
This option is useful for debugging grammar files and studying source code
written in a particular language and can help determine whether there's a
problem in a new compiler or in a source file.
Alternately, you can use the -v (verbose) command-line option to produce
verbose output from PCYACC in the default file YY.LRT. This file notes all the
applicable rules and actions to use for all possible states of the parser. In
addition, a summary at the end of the file shows a number of vital statistics,
including several terminal symbols used, the number of grammar rules in the
program, the number of states, and the number of errors (both of the shift/
reduce and reduce/reduce variety) found in the GDP.
Once the grammar description program has been debugged to the developer's
satisfaction, PCYACC is used to compile the program into source code for
yyparse( ). Other C code should be developed at this point as well. The last
step is to build an executable file that correctly solves the problem you
identified in the first place.
To give you an idea of how PCYACC works, let's see what it does with the
specification for a crude desk calculator shown in Listing One. The code for
this example draws heavily on the yacc specification for a simple desk
calculator presented in the book Compilers: Principles, Techniques, and Tools,
by Aho et al. (Addison-Wesley, 1986). The specification has been modified
slightly for use with PCYACC.
The declaration section first passes the names of two include files to the
output file and identifies a token called DIGIT. Next, both the '+' and '*'
operators are declared to be left-associative, with '+' having a higher
precedence than '*'. The grammar rule section contains the rules for the
"language" to be used by the calculator. The symbols $$, $1, and so on have
special meanings in the semantic actions. For instance, consider the following
rule:
 expr: expr '+' term {$$ = $1 + $3;}
The symbol $$ refers to the value associated with the nonterminal on the LHS
of the rule. The symbols $1 and $3 refer to the expr and term grammar symbols
on the RHS of the rule. (If the symbol $2 were used, it would refer to the '+'
operator in the RHS of the rule.)
The program part of the GDP simply provides a main( ) function (which calls
the yyparse( ) generated by PCYACC); a yyerror( ) function that can accept an
error string when called; and a yylex( ) function, which performs crude
lexical analysis (only single-digit numbers are accepted!) and produces pairs
consisting of a token and its associated value. In the case of our calculator,
there is only one token (DIGIT), and its value is communicated to the parser
through the yacc variable yylval.
The PCYACC specification is quite short when compared to the resulting C
source program, shown in Listing Two. This listing was created by invoking
PCYACC with no command-line options, as follows:
 PCYACC TEST.Y
The default name for the C source file was TEST.C. I compiled and linked this
file using the Microsoft C compiler (Version 5.0) and obtained an executable
file 11K in size.



Other Files


Earlier I mentioned that PCYACC comes with several grammar definition files
for languages such as Pascal and Smalltalk. Before anyone gets carried away
thinking that Abraxas is giving away the store and is handing out compilers
for all those languages, hold on! The grammar files that come with PCYACC
basically have the ability to check and see whether a file contains
syntactically correct "sentences" in a particular language. So, if you go to
the trouble of compiling and linking the Pascal grammar file, you'll end up
with a program that can parse Pascal code and tell you whether it's succeeded
or not. Although some may shrug their shoulders and stifle yawns, these files
nevertheless represent some of the best example programs I've seen bundled
with a professional software tool.
The one exception to the limited syntax-analysis capabilities of the grammar
rule files is the collection of source code files and a PCYACC specification
for a C++ preprocessor. Working with these files taught me two things about
PCYACC. First, the product is capable of industrial-size projects. The PCYACC
output file for the C++ preprocessor is more than 100K in size, and the
overall program requires compiling and linking nearly a dozen object files.
Second, for large specifications, PCYACC can take a long time to do its work.
In fact, the first time I attempted to compile the C++ grammar specification,
I ended up rebooting my machine because I thought it had hung. The same thing
happened the second time I tried it, and only a chance look at the C++
README.DOC file (which simply notes that "this grammar takes a long time to
compile") convinced me to wait patiently for PCYACC to do its job. Despite
running on an 80386-based machine, PCYACC took more than five minutes to
compile and output the parsing code for the C++ preprocessor.


Computer Boot Camp


Today, writing a compiler is one of those things that a typical computer
scientist or engineer does in the course of pursuing an undergraduate degree.
For most, the experience is sort of a computer boot camp, requiring a lot of
discipline and application over the short haul but something that's eventually
left behind to pursue other goals. In the old days, much of the effort of
writing a compiler was devoted to the issue of parsing the input. Now this
phase is one of the easiest to implement, and thanks to packages such as
PCYACC, it can be easily implemented on the PC architecture.


Product Information


PYACC: Abraxas Software Inc., 7033 SW Macadam Ave., Portland, OR 97219. IBM
PC, XT, AT or compatible (Macintosh and OS/2 versions also available).
Requires 512K RAM and any C compiler. Price: $395.


Availability


All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063, or call 800-356-2002 (from inside
Calif.) or 800-533-4372 (from outside Calif.). Please specify the issue number
and format (MS-DOS, Macintosh, Kaypro).


Generating Parsers with PCYACC
by Alex Lane



[Listing One]


/* TEST.Y

This specification is based largely on the yacc
specification for a simple desk calculator from "Compilers:
Principles, Techniques and Tools," by Aho, et al. (p. 259,
Addison-Wesley, 1986).

 2/2/89 a.lane
*/

%{
#include <stdio.h>
#include <ctype.h>
%}

%token DIGIT
%left '+'
%left '*'

%%

line : /* nothing */
 expr '\n' { printf("%d\n", $1); }
 ;
expr : expr '+' term { $$ = $1 + $3; }

 term
 ;
term : term '*' factor { $$ = $1 * $3; }
 factor
 ;
factor: '(' expr ')' { $$ = $2; }
 DIGIT
 ;
%%
main() {
yyparse();
}

yylex() {
 int c;
 if ( isdigit( ( c = getchar() ) ) ) {
 yylval = c - '0';
 return DIGIT;
 }
 return c;
}

yyerror(s)
char *s;
{
 fprintf(stderr, "PYERR: %s\n", s);
}






[Listing Two]


# line 11 "test.y"
#include <stdio.h>
#include <ctype.h>
#define DIGIT 257
#ifndef YYSTYPE
#define YYSTYPE int
#endif
YYSTYPE yylval, yyval;
#define YYERRCODE 256

# line 33 "test.y"

main() {
yyparse();
}

yylex() {
 int c;
 if ( isdigit( ( c = getchar() ) ) ) {
 yylval = c - '0';
 return DIGIT;
 }
 return c;

}

yyerror(s)
char *s;
{
 fprintf(stderr, "PYERR: %s\n", s);
}

FILE *yytfilep;
char *yytfilen;
int yytflag = 0;
int svdprd[2];
char svdnams[2][2];

int yyexca[] = {
 -1, 1,
 0, -1,
 -2, 0,
 0,
};

#define YYNPROD 9
#define YYLAST 218

int yyact[] = {
 5, 7, 13, 4, 8, 9, 3, 1,
 2, 0, 0, 0, 0, 12, 10, 11,
 4, 0, 3, 2, 0, 0, 0, 0,
 0, 0, 0, 0, 0, 0, 0, 0,
 0, 0, 8, 0, 0, 0, 0, 0,
 0, 0, 0, 0, 0, 0, 0, 0,
 0, 0, 0, 0, 0, 0, 0, 0,
 0, 0, 0, 0, 0, 0, 0, 0,
 0, 0, 0, 0, 0, 0, 0, 0,
 0, 0, 0, 0, 0, 0, 0, 0,
 0, 0, 0, 0, 0, 0, 0, 0,
 0, 0, 0, 0, 0, 0, 0, 0,
 0, 0, 0, 0, 0, 0, 0, 0,
 0, 0, 0, 0, 0, 0, 0, 0,
 0, 0, 0, 0, 0, 0, 0, 0,
 0, 0, 0, 0, 0, 0, 0, 0,
 0, 0, 0, 0, 0, 0, 0, 0,
 0, 0, 0, 0, 0, 0, 0, 0,
 0, 0, 0, 0, 0, 0, 0, 0,
 0, 0, 0, 0, 0, 0, 0, 0,
 0, 0, 0, 0, 0, 0, 0, 0,
 0, 0, 0, 0, 0, 0, 0, 0,
 0, 0, 0, 0, 0, 0, 0, 0,
 0, 0, 0, 0, 0, 0, 0, 0,
 0, 0, 0, 0, 0, 0, 0, 0,
 0, 0, 0, 0, 0, 0, 0, 0,
 0, 0, 0, 0, 0, 0, 0, 0,
 0, 6,
};

int yypact[] = {
 -40, -1000, -9, -37, -1000, -40, -1000, -1000,
 -40, -40, -39, -37, -1000, -1000,
};


int yypgo[] = {
 0, 7, 8, 6, 3,
};

int yyr1[] = {
 0, 1, 1, 2, 2, 3, 3, 4,
 4,
};

int yyr2[] = {
 2, 0, 2, 3, 1, 3, 1, 3,
 1,
};

int yychk[] = {
 -1000, -1, -2, -3, -4, 40, 257, 10,
 43, 42, -2, -3, -4, 41,
};

int yydef[] = {
 1, -2, 0, 4, 6, 0, 8, 2,
 0, 0, 0, 3, 5, 7,
};

int *yyxi;


/*****************************************************************/
/* PCYACC LALR parser driver routine -- a table driven procedure */
/* for recognizing sentences of a language defined by the */
/* grammar that PCYACC analyzes. An LALR parsing table is then */
/* constructed for the grammar and the skeletal parser uses the */
/* table when performing syntactical analysis on input source */
/* programs. The actions associated with grammar rules are */
/* inserted into a switch statement for execution. */
/*****************************************************************/


#ifndef YYMAXDEPTH
#define YYMAXDEPTH 200
#endif
#ifndef YYREDMAX
#define YYREDMAX 1000
#endif
#define PCYYFLAG -1000
#define WAS0ERR 0
#define WAS1ERR 1
#define WAS2ERR 2
#define WAS3ERR 3
#define yyclearin pcyytoken = -1
#define yyerrok pcyyerrfl = 0
YYSTYPE yyv[YYMAXDEPTH]; /* value stack */
int pcyyerrct = 0; /* error count */
int pcyyerrfl = 0; /* error flag */
int redseq[YYREDMAX];
int redcnt = 0;
int pcyytoken = -1; /* input token */



yyparse()
{
 int statestack[YYMAXDEPTH]; /* state stack */
 int j, m; /* working index */
 YYSTYPE *yypvt;
 int tmpstate, tmptoken, *yyps, n;
 YYSTYPE *yypv;


 tmpstate = 0;
 pcyytoken = -1;
#ifdef YYDEBUG
 tmptoken = -1;
#endif
 pcyyerrct = 0;
 pcyyerrfl = 0;
 yyps = &statestack[-1];
 yypv = &yyv[-1];


 enstack: /* push stack */
#ifdef YYDEBUG
 printf("at state %d, next token %d\n", tmpstate, tmptoken);
#endif
 if (++yyps - &statestack[YYMAXDEPTH] > 0) {
 yyerror("pcyacc internal stack overflow");
 return(1);
 }
 *yyps = tmpstate;
 ++yypv;
 *yypv = yyval;

 newstate:
 n = yypact[tmpstate];
 if (n <= PCYYFLAG) goto defaultact; /* a simple state */


 if (pcyytoken < 0) if ((pcyytoken=yylex()) < 0) pcyytoken = 0;
 if ((n += pcyytoken) < 0 n >= YYLAST) goto defaultact;


 if (yychk[n=yyact[n]] == pcyytoken) { /* a shift */
#ifdef YYDEBUG
 tmptoken = pcyytoken;
#endif
 pcyytoken = -1;
 yyval = yylval;
 tmpstate = n;
 if (pcyyerrfl > 0) --pcyyerrfl;
 goto enstack;
 }

 defaultact:

 if ((n=yydef[tmpstate]) == -2) {
 if (pcyytoken < 0) if ((pcyytoken=yylex())<0) pcyytoken = 0;
 for (yyxi=yyexca; (*yyxi!= (-1)) (yyxi[1]!=tmpstate); yyxi += 2);
 while (*(yyxi+=2) >= 0) if (*yyxi == pcyytoken) break;

 if ((n=yyxi[1]) < 0) { /* an accept action */
 if (yytflag) {
 int ti; int tj;
 yytfilep = fopen(yytfilen, "w");
 if (yytfilep == NULL) {
 fprintf(stderr, "Can't open t file: %s\n", yytfilen);
 return(0); }
 for (ti=redcnt-1; ti>=0; ti--) {
 tj = svdprd[redseq[ti]];
 while (strcmp(svdnams[tj], "$EOP"))
 fprintf(yytfilep, "%s ", svdnams[tj++]);
 fprintf(yytfilep, "\n");
 }
 fclose(yytfilep);
 }
 return (0);
 }
 }


 if (n == 0) { /* error situation */
 switch (pcyyerrfl) {
 case WAS0ERR: /* an error just occurred */
 yyerror("syntax error");
 yyerrlab:
 ++pcyyerrct;
 case WAS1ERR:
 case WAS2ERR: /* try again */
 pcyyerrfl = 3;
 /* find a state for a legal shift action */
 while (yyps >= statestack) {
 n = yypact[*yyps] + YYERRCODE;
 if (n >= 0 && n < YYLAST && yychk[yyact[n]] == YYERRCODE) {
 tmpstate = yyact[n]; /* simulate a shift of "error" */
 goto enstack;
 }
 n = yypact[*yyps];


 /* the current yyps has no shift on "error", pop stack */
#ifdef YYDEBUG
 printf("error: pop state %d, recover state %d\n", *yyps, yyps[-
1]);
#endif
 --yyps;
 --yypv;
 }


 yyabort:
 if (yytflag) {
 int ti; int tj;
 yytfilep = fopen(yytfilen, "w");
 if (yytfilep == NULL) {
 fprintf(stderr, "Can't open t file: %s\n", yytfilen);
 return(1); }
 for (ti=1; ti<redcnt; ti++) {
 tj = svdprd[redseq[ti]];
 while (strcmp(svdnams[tj], "$EOP"))

 fprintf(yytfilep, "%s ", svdnams[tj++]);
 fprintf(yytfilep, "\n");
 }
 fclose(yytfilep);
 }
 return(1);


 case WAS3ERR: /* clobber input char */
#ifdef YYDEBUG
 printf("error: discard token %d\n", pcyytoken);
#endif
 if (pcyytoken == 0) goto yyabort; /* quit */
 pcyytoken = -1;
 goto newstate; } /* switch */
 } /* if */


 /* reduction, given a production n */
#ifdef YYDEBUG
 printf("reduce with rule %d\n", n);
#endif
 if (yytflag && redcnt<YYREDMAX) redseq[redcnt++] = n;
 yyps -= yyr2[n];
 yypvt = yypv;
 yypv -= yyr2[n];
 yyval = yypv[1];
 m = n;
 /* find next state from goto table */
 n = yyr1[n];
 j = yypgo[n] + *yyps + 1;
 if (j>=YYLAST yychk[ tmpstate = yyact[j] ] != -n) tmpstate =
yyact[yypgo[n]];
 switch (m) { /* actions associated with grammar rules */

 case 2:
# line 22 "test.y"
 { printf("%d\n", yypvt[-1]); } break;
 case 3:
# line 24 "test.y"
 { yyval = yypvt[-2] + yypvt[-0]; } break;
 case 5:
# line 27 "test.y"
 { yyval = yypvt[-2] * yypvt[-0]; } break;
 case 7:
# line 30 "test.y"
 { yyval = yypvt[-1]; } break; }
 goto enstack;
}













June, 1989
PROGRAMMING PARADIGMS


Babbit's Guide to OOP




Michael Swaine


"He was earnest about these objects. They were of eternal importance, like
baseball or the Republican Party."
--Babbit, Sinclair Lewis
We appear to be witnessing the mainstreaming of object-oriented programming.
Regardless of the actual number of lines of code written this year or the next
in object-oriented languages as compared with other languages, it looks as
though object-oriented programming is on the verge of becoming the dominant
programming paradigm in personal computer software development. An area that
had recently been a frontier has now nearly been cleared for farming, 4-H, and
the Rotary Club.
Although DDJ has been investigating OOP techniques and the implications of the
paradigm for some time now, you'll be seeing more detail in the near future as
we cover this phenomenon. In April, Mike Floyd's article set the stage on
getting started in object-oriented programming. In the coming months, Kent and
Jeff will be analyzing significant new developments in OOPs. This month's
"Programming Paradigms" is a sort of bridge between Mike's piece and greater
OOP coverage coming in DDJ in that it defines some basic terms and points out
some of the chief issues in OOP without attempting to draw any deep
conclusions.


Tell Me Again, What Is Object-Oriented Programming?


Object-oriented programming (or software design) is "the construction of
software systems as structured collections of abstract data-type
implementations," according to Bertrand Meyer. It is a programming paradigm in
which the emphasis is on the thing to be manipulated rather than on the
functions to be performed. The things to be manipulated are represented as
abstract data-type representations, and these representations are called
classes, that's why Mike Floyd said in April that OOP could be called
Class-Oriented Programming.
Defining "object" to mean "an instance of an abstract data-type
implementation" is a little dry; especially because the idea behind the
concept of an object is much more intuitive than this. Object-oriented
programming began with the language Simula, which, while capable of much more,
was conceived as a language for designing simulations. If you are writing a
simulation in an object-oriented language, the objects of your simulation will
be exactly the software simulations of the physical objects in the real-world
system you are simulating. A valve, a gear, and an engine are all good
candidates for objects in some systems. Most programs are not direct
simulations of real-world systems, but the simulation view of an object is
still useful to keep in mind when evaluating candidates for objecthood. A
screen, a grafPort, and a window are some less real objects.
Brad Cox defines objects thus: "An object is some private data and a set of
operations that can access that data. An object is requested to perform one of
its operations by sending it a message telling the object what to do." This
sounds like applying a function to some data, hardly a revolutionary concept
in programming. The difference is all in how the operation gets matched to the
data on which it operates. Because an object is not a data structure or a set
of procedures but a combination of the two, a powerful new approach to
matching the operation to the data is possible. Messages are the key.
A message is a function call with a specified receiver. Sending the message
triggers a selection mechanism that uses the receiver name to branch to the
chunk of code appropriate to that receiver. The technique is simple, but the
effect is dramatic in terms of the division of labor in software development.
When a programmer writes code that causes a message to be sent to an object,
the responsibility for ensuring that the operation is appropriate to the data
rests not with the programmer but with the person who wrote the definition of
the class to which the object belongs.
The result is reusable software components, at least in theory. Everyone who
writes about object-oriented programming seems to agree that the motivation
for using the technique is something like "increased productivity,
maintainability, and reliability through reusable software components" that
can be plugged into new systems. These reusable components in OOP are classes.
Classes in a true object-oriented system are not independent data types, but
are part of one large structure with the structuring principle being
inheritance.
When a new class is created, it is defined to be a subclass of some existing
class and inherits all the functionality of the parent class. What inheritance
buys you is the ability to define key elements of the functionality of a
system once, to use these elements as building blocks, and to do it all
smoothly and naturally. The smoothness of the process is the real advance that
OOP represents; as Zack Urlocker has said, "Many programmers are pleasantly
surprised to find that object-oriented languages encourage them to use
techniques that they have been faking for years in other languages."


Issues in Object-Oriented Programming


The preceding topics of objects, classes, messages, and inheritance need to be
discussed at the beginning of any overview of OOP because they are fundamental
to an understanding of more advanced topics. The order of presentation of
these other topics, however, may depend on one's particular viewpoint on OOP.
I have deliberately avoided presenting them in any of the logical sequences in
which they could be placed and have instead treated each issue as an item unto
itself, a sort of reusable editorial component. The order is alphabetical, and
the presentation is encyclopedic in structure (though not in scope).
Abstract data types -- To specify an abstract data type is to define a set of
data structures strictly in terms of the features of the structures and the
operations defined on them, and not in terms of the actual physical
implementation of the data structure. Classes in object-oriented programming
are abstract data types.
Client relationship -- In addition to the inheritance relationship between
classes, one class may be a client of another. This simply means that the
implementation of the first (client) class relies on the second (supplier)
class, for example, by using objects belonging to the supplier class. Deciding
which relationship should apply is sometimes tricky but more often obvious.
The class HOUSE would naturally inherit from the class BUILDING, but could be
a client of classes ROOF and WALLS.
Concurrency -- One potential benefit of object-oriented programming is
concurrency. Although current object-oriented systems do not provide for
concurrency, the decision to build a system from independent objects that
communicate via messages naturally suggests an implementation that uses
concurrent processors.
Dynamic binding -- Binding or linking is the process of putting together
elements of functionality from different sources into one executable image.
Dynamic (or delayed, or late) binding is binding after compile time, probably
during execution. Dynamic binding in object-oriented systems assigns to the
supplier, rather than to the client, the responsibility for ensuring that an
operation is appropriate to the data on which it operates.
Efficiency -- The clearest benefits of object-oriented programming are in the
design of large and complex systems. When it comes to the small details of
implementation that strongly affect system performance, it seems to be widely
acknowledged that calling a function is usually faster than passing a message.
Current personal computer hardware could, with some justification, be referred
to as C machines; hardware ideally suited to object-oriented programming has
yet to be brought successfully to market and may have to wait for parallel
architectures. Language and operating system support for object-oriented
programming is on the increase and may be more important than hardware
support, however, as operating systems impose themselves more and more between
the programmer and the hardware.
Encapsulation -- To the user of an object the data is invisible, the
procedures for accessing and operating on that data is visible. This is
encapsulation, and Meyer has shown how it can be implemented in even a
non-object-oriented language such as Fortran.
Hybrid paradigms -- Smalltalk is the best-known example of a pure
object-oriented programming system in which all actions are implemented as
messages sent to objects. Objective C and C++ are well-known hybrids that add
object-oriented features to C but retain some non-OOP features. The arguments
usually advanced for the pure approach are just the arguments for
object-oriented programming: reliability, maintainability, and ease of
development. The usual arguments for the hybrids are familiarity and speed.
Information hiding -- Information hiding is a desideratum of modular
programming: All information is a module should be private to that module
unless made public through a definition known as the interface. Encapsulation
is one answer to what should remain private and what should go into the
interface.
Modularity -- Meyer states five criteria of modularity: decomposability,
composability, understandability, continuity, and protection. Modules should
aid in the top-down analysis of a problem into subproblems, permit building
large systems for smaller building blocks, be understandable on their own, and
be sufficiently separate from one another that a design change or run-time
abnormality affecting one module will not significantly affect the whole
system. The desire for modularity is a primary motivation behind the
development and use of object-oriented programming.
Multiple inheritance -- Eiffel has it; most other OOPs don't. Multiple
inheritance is the capability for a class to inherit features from two or more
ancestor classes. With only single inheritance, the inheritance structure is a
tree; with multiple inheritance, it's a forest. Multiple inheritance raises
some hotly-debated questions, such as how to resolve clashes between features
of the parent classes. Some cases of multiple inheritance are what Meyer calls
marriages of convenience in which the two ancestors complement each other
nicely. On the other hand, it may be convenient to create a class that
inherits from two parent classes that are not entirely complementary: Each
parent may contribute a version of the same method, for example.
Polymorphism -- This is the ability to send the same message to different
objects and have them respond differently.
Renaming -- This is a mechanism that permits references to do the same thing
under different names, depending on the class involved. It is used in Eiffel
to resolve clashes in multiple inheritance.
Redefinition -- This is the mechanism that allows the client programmer to use
the same name to refer to different things, depending on the class to which it
is applied. It makes polymorphism possible.
Repeated inheritance -- A special case of multiple inheritance is repeated
inheritance in which one class P is an ancestor of another class C in more
than one way, possibly a direct ancestor. It presents difficulties beyond
those of ordinary multiple inheritance.
Reusability -- The issue was stated well by Meyer: "Why isn't software more
like hardware? Why must every new development start from scratch? There should
be catalogs of software modules, as there are catalogs of VLSI devices: When
we build a new system, we should be ordering components from these catalogs
and combining them, rather than reinventing the wheel every time. We would
write less software and perhaps do a better job at that which we do develop.
Then wouldn't the problems everyone laments the high costs, the overruns, the
lack of reliability just go away? Why isn't it so?"
Selective inheritance -- While multiple inheritance is an issue in OOP,
selective inheritance, the ability to exclude features in inheriting from a
class is not. No object-oriented language implementation seems to support it,
probably because it is a Pandora's box. But selective inheritance would allow
object-oriented systems to more closely match the world-model that we carry in
our heads. We have no trouble thinking of an ostrich as a flightless bird, but
in the object-oriented aviary this is a hard thought to think. What we can do
is use redefinition to make flight mean something different for this
particular bird. Depending on how you define the problem (or whether you think
there is a problem) this may or may not be a solution.


References


Brad Cox. Object-Oriented Programming: An Evolutionary Approach.
Addison-Wesley, 1986-7.
Bertrand Meyer. Object-Oriented Software Construction. Prentice Hall, 1988.
Bertrand Meyer. "Reusability: The Case for Object-Oriented Design." IEEE
Software, March 1987.
"Whitewater's Actor: An Introduction to Object-Oriented Programming Concepts,"
Zack Urlocker, Microsoft Systems Journal, March 1989.

"A Class Act," Michael Floyd, DDJ, April 1989.





























































June, 1989
C PROGRAMMING


Scripts for SMALLCOM




Al Stevens


Last month we built a generic interpreter of a C-like language that can be
used to support communications scripts or editor macro processing. The
language is called S, and the interpreter is called SI. We included an example
shell program that you can now discard. The shell was to show how you surround
the interpreter engine with a user interface and intrinsic functions to give
purpose to the interpreted code. The SI interpreter engine has no permanent
dedicated shell because the different custom shell programs that you might
develop give each use of SI its purpose. Therefore, an SI implementation will
always include the development of a custom shell program.
This month we will build a shell that uses SI to implement a communications
script processor into the SMALLCOM program. The shell and the interpreter
program link with the rest of SMALLCOM through a hook in smallcom.c. We will
need all the code used to build SMALLCOM from months past, the files named
interp.h and interp.c from May 1989, and the script.c file included this month
as Listing One. The complete source code package dates back to September,
1988. Each succeeding month is a complete toolset, usable in many other
programs, and yet the code from each month adds to the program we are
building.
A reader asked if I have the communications program already completed and am
parcelling it out in little chunks. The answer is no. This program is
developing as you watch it. I am about four months ahead of you, though; these
words are being written in February for the June issue. This incremental
approach has its pros and cons. On the up side, the code is new and known to
work with the latest versions of the Microsoft and Turbo C compilers, and we
build new layers of software drawing on our experiences with the old. On the
down side, I sometimes find that I have coded myself into a corner. In the
process of adding functions to the existing program I sometimes wish I could
make little changes to the earlier code. In the real world, you would do just
that --modify the program to fit the purpose of the hour. This, however, is
not the real world. In the continuing saga of this column and its project, I
am trying to preserve the code already published. That is often frustrating.
As I learn new and better ways to do things, I want to go into that old code
and fix it up. Maybe a future column can be dedicated to revisiting old code
and giving it a polish. I have always believed that all software would be
better if we were allowed to write it twice --once to learn how it should work
and once to do the things we wish we had done the first time out. The
three-piece suits tell me that my approach is not practical when viewed from
the perch of the bottom line.
Every now and then I have no choice but to fix some old code from columns
past. For example, this month I learned that I needed the file pointer for the
SMALLCOM log file to be external, rather than static, so that I could use it
from within the script processor. Listing Two is junehook.c, the code that has
the changes we need to add the script interpreter to SMALLCOM. We need to make
two changes to smallcom.c from the March '89 column. The first makes the log
file pointer external. The second adds the script processor's hook to the
program. There are no apologies for the second one; that is the modus operandi
described for this project in April.
The SMALLCOM program was built with the Compact memory model in March. The
only advantage to that is that long documents are permitted in the integrated
editor. The script interpreter uses integer offsets as pointers. The address
and pointer operators are significantly easier to implement in the small
model, and so we will change the SMALLCOM model to the small one. The .MAK
file for Microsoft C users and the environment setup for Turbo C users must be
changed to the small model. In smallcom.c change the MAXLINES assignment from
800 to 200. This assignment is in the comeditor function on about line 474.
Documents that you edit from within SMALLCOM are now limited to 200 lines.
This is no loss because on-line messages should be kept short anyway.


The Script Language


Our script language is S, in the syntax and grammar described last month. You
will recognize it as a subset of C. To be a communications script language, S
must be extended by the addition of intrinsic functions built into its shell.
The script.c shell program has several such functions as described next. When
you write a script, you build a text file that contains what looks like a C
program, give the file a name with the extension .SCR, and put its name into
the SMALLCOM phone directory. The directory was described in April, and it has
a file maintenance screen that lets you add script names to an entry
associated with a phone number. The file name goes in the entry associated
with the service that uses the script, and you do not put the .SCR extension
in the directory record. When you call a service that has a script associated
with it, the script interpreter shell is executed.
The S interpreter provides the management of generic S data types and control
flow of the script program. The intrinsic functions in the custom shell
integrate the script and the communications program. Be aware of how the
intrinsic functions are called. SI recognizes that a program is calling an
intrinsic function by finding the name in the global array of INTRINSIC
structures named ffs. The structure contains the ASCII name of the function
and the address of the script.h function to be executed. The intrinsic
function mechanism uses a generic parameter-passing technique where the
address of an array of integers is passed to the intrinsic function. SI has
built that array from the parameters that the interpreted call to the function
is passing. The integers in the array represent integer values or pointers.
If the name of an array of character pointers is passed, the integer is taken
to mean a pointer to a pointer. The straight character or integer pointers are
really integer offsets to the locations in the SI token buffer. The pointers
to pointers are integer offsets from the interpreter's data segment to an
array of pointers that is built when the function call is interpreted.
The script.c program is written to be compiled with one of the small data
models, so the pointer parameter integer offsets can be cast as pointers and
the shell works fine. Each intrinsic function knows what kind and how many
parameters it expects. Here's the rub. If you should ever want to use the SI
interpreter in a large data model program, you would need to coerce those
integer offsets into far pointers by using the segment value assigned to the
token buffer. You would do this in the intrinsic functions in your shell. You
would also need to change the way that interp.c deals with the address-of and
pointer operators since these are managed by brute-force casts of integers to
pointers. It just seemed easier at the time.
In the paragraphs that follow I describe the communications script intrinsic
functions as if they had ANSI-style prototypes. They do not. In practice, you
simply call the functions without declaring them. This convention is closer to
the style permitted by the original K&R definition of the C language.
Remember, though, that SI functions are implicitly defined as returning
whatever they actually return at run time. It is up to your SI program to know
what is coming back when it calls a function.
void logon(char*filename);. This function turns on logging of the data
received from the remote computer across the serial line. The parameter
specifies a file name. If logging is already on, the current log file is
closed, and this new file is created if it does not exist or is opened for
appending if it does exist. You would use this function in a script where you
wanted to capture the incoming data in a named file, perhaps a file of
messages for a particular online forum.
void logoff(void);. This function turns logging off.
void upload(char*filename, charprotocol);. Call this function to start
uploading a file to the remote computer. The first parameter is the file name
and the second is 'a' for an ASCII upload and 'x' for an XModem upload. In a
script you might use the ASCII mode of this function when it is time to send a
message to a forum. You would assume the user has created a message by using
the SMALLCOM editor or another, and that the file name is the name you include
in the script.
void download(char*filename, char protocol);. Call this function to start
down-loading a fi e from the remote computer. The first parameter is the file
name to be created locally and the second is 'a' for an ASCII upload and 'x'
for an XModem upload.
void hangup(void);. This function breaks the connection between the local and
remote computers.
void quit(int code);. This function tells the program to terminate SMALLCOM.
If the code parameter is zero, the termination returns to the SMALLCOM
program. If the code is non-zero, the quit function stuffs characters into the
BIOS keyboard read-ahead buffer to cause SMALLCOM to return to DOS.
void sendstring(char *string);. This function sends a string of characters to
the remote computer. You can code the string as a literal within the call or
use a pointer.
void sendchar(char ch);. This function sends a single character to the remote
computer. The character can be a literal or a char or int variable.
int waitforstrings(char *str[]);. This function reads the input stream and
watches for the appearance of one of the strings pointed to by the array. The
function returns an integer relative to zero as the subscript to the string
that was seen. If 60 seconds pass without one of the strings coming in, the
function returns oxffff(or -1; a good reason to have the unary operator
supported by SI, which we do not). You would use this function in a script
where your logic might be altered depending on what comes in. The strings "You
have mail waiting" and "No mail today" can imply different paths in the
script. Here is an example of how that sequence would appear:
 char *str[] = { "You have mail waiting", "No mail today" }; int answer;
answer = waitforstrings(str); if (answer = = 1) ...
int waitfor(char *str);. This function waits for the appearance of the
specified string. It returns 0 if the string is found and -1 if the 60-second
timeout occurs. You would use this form when there are no choices, such as
when you know the remote service always asks for the password. An example of
the use of this function follows.
 if (waitfor("PASSWORD:") = = 0xffff) ....
void system(char *command);. This function is the equivalent of the Turbo C
and Microsoft C system functions. The string is a DOS command that is to be
executed. You might use this to delete or rename a message file after it has
been transmitted. For example:
 system("del ddjforum.xmt");
void message(char *msg);. This command writes a message at the bottom of the
screen in the status line. It is a way for the script to keep the user advised
of its progress. The earlier status line is saved and restored when the script
is done. If you use this function, you should use it throughout the script to
maintain consistency in your program. message("Receiving Messages");.


Building the Program


Modify the smallcom.prj or smallcom .mak files from when we originally built
SMALLCOM. Add the script.c and interp.c code modules. The example that follows
is a script that talks to a ProComm host-mode computer.


An Example of a Script


Listing Three is procomm.scr, an example script. A casual browser of the
magazine will think they are looking at a non-ANSI C program. Instead, this is
a script that lets a SMALLCOM program call another computer that is running
ProComm, the shareware communications program. ProComm has a host mode that
turns it into a miniature bulletin board system. When ProComm is in that mode
(activated by Alt-Q) and a call comes in, the caller is asked to enter a name
and password. Then ProComm displays a menu of file-related commands that the
caller can use. These include upload and download commands. This environment
is handy for testing a script, and so our procomm.scr script exercises this
feature.
To run the script, you must put its name, procomm, in the directory entry of
the remote system that is running ProComm. If you do not have two computers,
two modems, and two phone lines, maybe you can arrange for a friend to help
you out. Call the ProComm computer and watch it go. The password programmed
into the script must match the one programmed for the ProComm host mode. Let's
step through this script and see what it does.
The first item of interest is the array of character pointers named strs. This
array illustrates how you express the list of strings that the intrinsic
waitforstrings function expects. In this case, we will wait for one of the two
strings after we send the password. If the password is correct, the script
will key on the "choice?" string. The "denied" string tells us that ProComm
does not like our password.
Next are some character pointers initialized to point to strings that ProComm
sends and that we will wait for unconditionally. These strings could be coded
into the calls to waitfor as string literals, but because several of them are
used more than once, I coded them individually to save token space. For
consistency, I put the strings that get sent by sendstring into external
pointers, too. This is a good coding practice. It gets all the data values
that might be changed for another script up where they can be found without a
search through the script's code. If you call a lot of RBBS or FIDO bulletin
boards and want to automate the signon, you'll be building a lot of similar
scripts.

SI has no #define preprocessor command, so instead we use initialized int
variables with identifiers of upper case. The main function of procomm.scr
posts a message about signing on and calls the signon function. If that
function returns a true value, the sign on procedure was a success. The signon
function waits for the "name" prompt and sends the caller's name. It waits for
the "password" prompt and sends the password. Then it waits for either the
"choice" or "denied" messages and determines from this to return a true or
false value. If main gets the false return it posts a message about being
denied access and is done. Otherwise it calls downloadops to download a file
from the ProComm system, uploadops to upload a file, posts a sign off message
and sends a 'g' to the ProComm system to tell it "goodbye."
The downloadops and uploadops functions post messages, wait for the strings
from ProComm that tell them what to do, send the answers, and call the
intrinsic download and upload functions to transfer files. The downloadops
function calls the intrinsic system function to rename the file it received.


Software Development '89


I attended the Software Development '89 conference in Burlingame, Calif.,
south of San Francisco. SD89 was Miller Freeman's second annual conference for
software developers. The dominant development platform is still C, but
object-oriented programming is rapidly gaining momentum.
The new C product of note is TopSpeed C from Jensen & Partners International
(Mountain View, Calif.) TopSpeed C is the "Turbo C that was to be." Among its
developers are founders of Borland, and while they were building what was to
be the Turbo C compiler, Borland's management jerked the rug from under them
by buying and adapting Wizard C into Turbo C. The original team then pulled
out and took their compiler code with them to form JPI. From the unfailing
perspective of perfect hindsight we see that Borland's change of direction was
a good move. Nearly two years after the announcement of Turbo C, JPI's
TopSpeed C is not yet ready to ship. But what they are showing is impressive.
Watch this one.
Bill Gates, Microsoft's youthful CEO, gave the welcome address. SD89 presented
the perfect audience for Gates because he did what most of us dream of --wrote
programs and made a billion dollars. Of course, there's more to it than that,
but when a billionaire speaks, folks tend to listen. The point of his address
was that OS/2 is the wave of the future, but that there will always be MS-DOS.
Surprise.
Gates maintains that there is no future in developing text-based applications.
According to him, the MS-DOS Windows and OS/2 Presentation Manager graphical
interfaces are where the opportunities lie. It's easy to figure out the
message. When a billionaire programmer tells programmers (many of them
presidents of multi-hundred dollar software companies) what code to write to
make a lot more money, the masses pay attention. But let's not forget that
Gates has an axe to grind; Microsoft has an investment in the realization of
his advice. Before you jump on the bandwagon though, be advised -- the
learning and cost ramps for those pretty pictures are mighty steep.
His only real news was that Microsoft's full compiler products (MSC to us)
will soon have integrated environments after the fashion of the Quick language
products.
Gates pitched incremental compiles/ link, a feature that QuickC 2.0 already
has to some extent. This is where the environment compiles only the function
just changed. He said that hypertext-like editors would be big stuff some day,
too. You will be able to put the cursor on a function name, press a key, and
jump to that function in its source file. You will then be able to change the
function's name and have the integrated environment fix all references to it
in all related code modules and automatically recompile and link. I heard a
strange sound. Terry Colligan, president of Rational Systems Inc., was
snickering into his fist. Instant-C has had those features for years.
Gates talked about Microsoft's presence in the international arena. He made a
point of their relationship with the USSR. It seems Microsoft sells a lot of
software over there. (I don't feel so guilty now about writing this column on
a Toshiba laptop.) But then he told us about how one of their overseas sales
organizations unloaded their last 500 copies of QuickC 1.0 on the Soviets. For
the first time since the Rosenbergs were executed I felt sorry for a
communist.
Wednesday's bright spot was Liz Oakley, vivacious publisher of Programmer's
Journal, sitting on the floor in the corridor outside of the Microsoft
hospitality suite. Liz was holding court for a seated circle of enchanted
followers (me included). The denizens of the more dignified inner suite closed
the door to block out the spectacle.
On Thursday Microsoft bowed out of the Codeview/Turbo Debugger shootout. David
Intersimone of Borland graciously consented to objectively represent both
views. He gave a brief rundown of the features shared by both debuggers and
then treated us to a dazzling demonstration of the Turbo Debugger. Code-who?
Lest we forget, CodeView was a trailblazer, setting the standard in source
debuggers for others to shoot at. Since then it has been left in the dust, and
we tend to forget how much we loved it in its early days. No doubt a
next-generation CodeView is in the wings, if only to silence the muffled
titters heard whenever the two debuggers are compared. In the meantime Turbo
Debugger is a must-have for serious debugging.
Borland hosted a grand party on Thursday night. There was food, drink,
T-shirts, and a competent jazz quintet of Borland employees -- guitar, flute,
keyboards, bass, and drums. In the second set, Philippe Kahn, the flamboyant
leader of Borland, sat in and played the tenor saxophone. Don't give up the
day gig, Philippe.


Availability


All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063; or call 800-356-2002 (from inside
Calif.) or 800-533-4372 (from outside Calif.). Please specify the issue number
and format (MS-DOS, Macintosh, Kaypro).


C Programming (column)
by Al Stevens



[Listing One]

/* -------- junehook.c ------------ */

/*
 * make these changes to smallcom.c to install the script
 * processor program
 */

/* ------- making logfp external ---------- */
FILE *logfp;
static FILE *uploadfp, *downloadfp, *cfg;


/* ------- the hook to script processors ---------- */
extern void script(void);
void (*script_processor)(void) = script;







[Listing Two]


/* ---------- script.c -------------- */


/*
 * The SI shell to implement interpreted scripts in SMALLCOM
 */

#include <stdio.h>
#include <conio.h>
#include <stdlib.h>
#include <ctype.h>
#include <dos.h>
#include <setjmp.h>
#include "window.h"
#include "serial.h"
#include "interp.h"
#include "modem.h"

#if COMPILER == MSOFT
#define MK_FP(s,o) ((void far *) \
 (((unsigned long)(s) << 16) (unsigned)(o)))
#endif

void upload_ASCII(FILE *);
void download_ASCII(FILE *);
void upload_xmodem(FILE *);
void download_xmodem(FILE *);
int waitforstring(char **, int, int);
char *prompt_line(char *, int, char *);
void reset_prompt(char *, int);

/* ----------- intrinsic interpreter functions ---------- */
static int si_logon(int *);
static int si_logoff(int *);
static int si_upload(int *);
static int si_download(int *);
static int si_hangup(int *);
static int si_quit(int *);
static int si_sendstring(int *);
static int si_sendchar(int *);
static int si_waitforstrings(int *);
static int si_waitfor(int *);
static int si_system(int *);
static int si_message(int *);

INTRINSIC ffs[] = {
 "logon", si_logon,
 "logoff", si_logoff,
 "upload", si_upload,
 "download", si_download,
 "hangup", si_hangup,
 "quit", si_quit,
 "sendstring", si_sendstring,
 "sendchar", si_sendchar,
 "waitforstrings", si_waitforstrings,
 "waitfor", si_waitfor,
 "system", si_system,
 "message", si_message,
 NULL, NULL
};

extern INTRINSIC *infs = ffs;

extern FILE *logfp;

/* ------------------ error messages ----------------------- */
char *erm[]={ "Unexpected end of file", "Unrecognized",
 "Duplicate ident", "Symbol table full",
 "Out of heap memory", "Undeclared ident",
 "Syntax Error", "Unmatched {}",
 "Unmatched ()", "Missing",
 "Not a function", "Misplaced break",
 "Out of place", "Too many strings",
 "Token buffer overflow", "Divide by zero" };

static FILE *fp;
static char *prompt = NULL;
extern char scriptfile[];
extern char *tokenbf;

jmp_buf errorjmp;

/* ---------- process the named script file --------- */
void script()
{
 if ((fp = fopen(scriptfile, "r")) != NULL) {
 if (setjmp(errorjmp) == 0) {
 loadsource();
 interpret();
 }
 fclose(fp);
 if (prompt != NULL)
 reset_prompt(prompt, 25);
 if (tokenbf != NULL)
 free(tokenbf);
 }
}

/* ----------- syntax error in script language --------- */
void sierror(enum errs erno, char *s, int line)
{
 char msg[80];
 sprintf(msg, "SI Error: %s %s line %d\n",s,erm[erno],line);
 error_message(msg);
 longjmp(errorjmp, 1);
}

/* ---------- get a character of script source code -------- */
int getsource(void)
{
 return getc(fp);
}

/* -------- unget a character of script source code ------- */
void ungetsource(int c)
{
 ungetc(c, fp);
}

/* ------------ intrinsic functions -------------- */

/* ---------- turn logging on ----------------- */

static int si_logon(int *ptr)
{
 si_logoff(ptr);
 logfp = fopen((char *) *ptr, "ab");
 return 0;
}

/* ---------- turn logging off ----------------- */
static int si_logoff(int *ptr)
{
 if (logfp)
 fclose(logfp);
 logfp = NULL;
 return 0;
}

/* ----------- upload a file ----------------- */
static int si_upload(int *ptr)
{
 FILE *up;
 int x = wherex();
 int y = wherey();

 if ((up = fopen((char *) ptr[0], "rb")) != NULL) {
 if (toupper(ptr[1]) == 'A')
 upload_ASCII(up);
 else if (toupper(ptr[1]) == 'X')
 upload_xmodem(up);
 fclose(up);
 }
 gotoxy(x,y);
 return 0;
}

/* --------- download a file ------------- */
static int si_download(int *ptr)
{
 FILE *dn;
 int x = wherex();
 int y = wherey();

 if ((dn = fopen((char *) ptr[0], "wb")) != NULL) {
 if (toupper(ptr[1]) == 'A')
 download_ASCII(dn);
 else if (toupper(ptr[1]) == 'X')
 download_xmodem(dn);
 fclose(dn);
 }
 gotoxy(x,y);
 return 0;
}

/* ---------- hangup the call ------------ */
static int si_hangup(int *ptr)
{
 disconnect();
 return 0;
}


/* ----------- terminate the program ---------- */
static int si_quit(int *ptr)
{
 int far *bp = MK_FP(0x40, 0x1a); /* BIOS read-ahead buff */

 if (*ptr) {
 *bp++ = 0x1e; /* next off pointer */
 *bp++ = 0x22; /* next on pointer */
 *bp++ = 27; /* Esc key */
 *bp = 'y'; /* 'y' for Yes */
 }
 longjmp(errorjmp, 1);
}

/* ---------- send a string to the callee --------- */
static int si_sendstring(int *ptr)
{
 char *cp = (char *) *ptr;

 while (*cp)
 writecomm(*cp++);
 return 0;
}

/* ---------- send a character to the callee --------- */
static int si_sendchar(int *ptr)
{
 writecomm(*ptr);
 return 0;
}

/* --- wait for one of a set of strings from the callee --- */
static int si_waitforstrings(int *ptr)
{
 return waitforstring((char **) ptr[0], 60, 0);
}

/* ------- wait for a string from the callee ------- */
static int si_waitfor(int *ptr)
{
 static char *ws[] = {NULL, NULL};

 ws[0] = (char *) *ptr;
 return waitforstring(ws, 60, 0);
}

/* ---------- execute a system (DOS) command ----------- */
static int si_system(int *ptr)
{
 char cmd[80];
 sprintf(cmd, "%s >nul", (char *) *ptr);
 system(cmd);
 return 0;
}

/* --------- display a message to the user ------------ */
static int si_message(int *ptr)
{
 int x = wherex();

 int y = wherey();
 prompt = prompt_line((char *) *ptr, 25, prompt);
 gotoxy(x,y);
 return 0;
}






[Listing Three]

/*
 * PROCOMM.SCR : A SMALLCOM script that calls a ProComm system,
 * downloads a file, and uploads another file
 */

/* ----- key strings that ProComm sends ----- */
char *strs[] = {
 "choice?",
 "Denied"
};
char *Name = "Name:";
char *Password = "Password:";
char *choice = "choice?";
char *procedure = "procedure.";
char *spec = "spec?";

/* -------- strings that SMALLCOM sends -------------- */
char *name = "Al Stevens\r";
char *password = "PASSWORD\r";
char *filename1 = "test1.fil";
char *filename2 = "test2.fil";

/* -------- poor man's #define or enum ---------- */
int CHOICE = 0;
int DENIED = 1;
int XMODEM = 'X';
int UPLOAD = 'u';
int DOWNLOAD = 'd';
int GOODBYE = 'g';

/* -------- main entrance to the script --------- */
main()
{
 message(" --> Signing on to ProComm <--");
 if (signon()) {
 downloadops();
 uploadops();
 waitfor(choice);
 message(" Signing Off");
 sendchar(GOODBYE);
 }
 else
 message(" Not allowed access! ");
 hangup();
}


/* ---------- sign on and send the password ----------- */
signon()
{
 waitfor(Name);
 sendstring(name);
 waitfor(Password);
 sendstring(password);
 if (waitforstrings(strs) == CHOICE)
 return 1;
 return 0;
}

/* --------- download a file ---------- */
downloadops()
{
 message(" Downloading");
 sendchar(DOWNLOAD);
 waitfor(choice);
 sendchar(XMODEM);
 waitfor(spec);
 sendstring(filename1);
 sendchar('\r');
 waitfor(procedure);
 download(filename1, XMODEM);
 system("del download.fil");
 system("ren test1.fil download.fil");
}

/* --------- upload a file ---------- */
uploadops()
{
 waitfor(choice);
 message(" Uploading");
 sendchar(UPLOAD);
 waitfor(choice);
 sendchar(XMODEM);
 waitfor(spec);
 sendstring(filename2);
 sendchar('\r');
 waitfor(procedure);
 upload(filename2, XMODEM);
}




















June, 1989
GRAPHICS PROGRAMMING


The Software White Cane


 This article contains the following executables: GRAFIX.H


Kent Porter


Where I went to college, there was a professor who often strode purposefully
about the campus, head high, eyes fixed resolutely on the horizon. He never
looked down. Approaching a curb or stairs, he negotiated them confidently. At
a door, he'd seize the knob without a flick of the eye and go inside.
Sometimes, if late, he'd jog. This doesn't seem particularly remarkable until
you realize that the man was totally blind.
One day I asked him how he did it. "I just count steps," he said offhandedly.
"It's no big deal. I know every inch of this campus."
Then one day I ran into him in a nearby town. There, away from his well-known
world, he was like other blind men, tapping his way along with a white cane.
Scarcely less vigorously, I might add, but he didn't know where the obstacles
were. The cane helped him find them.
Now it might seem like a far jump of the imagination to relate this blind
professor to a graphics program, but there are parallels. Both move around in
their worlds knowing there are things in the way, but unable to see them. To
avoid hazards, both need a replacement for sight.
The person at the thinking end of a white cane finds walls, curbs, and other
obstacles by detecting when the cane hits something unexpected. A program
finds them by detecting a change in the pixel value. Therefore, the software
white cane is a routine that senses pixels.
There are in fact several ways to get pixel information back from the display.
The choice of method depends on what we're trying to accomplish. The ability
to "feel" pixels and avoid obstacles gives graphics programs the freedom to do
many things rather easily that would otherwise be difficult. That's exactly
what a white cane does for a blind person.
This month we'll examine ways to detect individual pixels and an important
application: filling closed figures.


This Is Not The Way To Do It


The PC ROM BIOS Video Services furnish a built-in function to read a graphics
pixel value. You place ODh into register AH and the X and Y coordinates into
registers CX and DX respectively, then issue Int 10h. On return, the pixel
value is in register AL. Couldn't be simpler.
Couldn't be much slower, either. I've complained about it before and I'll do
it again: the ROM BIOS video routines are inexcusably slow. On my 8MHz AT
clone, it takes 139.56 seconds to read all 224,000 EGA pixels, for a rate of
1605/sec.
And if you're one of those programmers who's fallen into the trap of believing
faster chips make up for inefficient code, think again. Exactly the same
program running on my 16MHz '386 machine takes a hair under 300 seconds,
yielding a rate --if you want to call it that --of 748 pixel reads per second:
less than half the throughput despite twice the cycles.
Certainly some part of this discrepancy has to do with differences in the ROM
BIOS (they're both Phoenix). That's not the point. This is: Don't use the ROM
BIOS pixel read. It's just too slow.


The Improved White Cane


You can achieve much better pixel reading rates by going directly to the video
controller with EGAPIXEL.ASM (Listing One). This routine achieves a rate of
16,500 pixels/sec. on the AT and 32,350 on my '386. Still far short of the
pixel-blasting rate for the bline( ) routine, but 10.3 to 43.2 times better
than the ROM BIOS Video Services.
The writing and reading of pixels have a lot in common: both have to remap the
coordinates into the viewport, compute the video buffer address, and set the
bit mask. Thus the first 70 or so lines of EGAPIXEL.ASM and DRAWPT.ASM (from
February) are quite similar. Then things change dramatically.
Probably because pixel reading is less often done than writing, the Video
Controller is less efficient at it. To write, you jam a bit pattern into a
hypothetical memory location and the controller takes care of splitting it
apart and updating the affected bit planes. In reading, the program has to
fetch a byte from each bit plane separately and assemble the value itself, bit
by bit.
The PIXVAL macro reads each bit plane for the pixel position and extracts its
relevant bit. The pixel value is accumulated in register BH through rotation,
reading from bit plane 3 (most significant) downward. The bytes at ES:[SI] are
actually taken from the video controller's latch registers thanks to
background meddling by the 6845, but the program doesn't know that. It just
thinks it's reading the same memory location over and over. The pixel value
accumulated in BH gets passed back to the caller (unless the location is
outside the viewport, in which case -1 is returned).
Why use a macro when a loop would do? Because conditional jumps in assembly
language are time-stealers. My friend Michael Abrash is doing a wonderful
series of books on squeezing the utmost performance out of assembly language.
Jeff Duntemann mentions it this month over in the "Structured Programming"
column, and I've had a peek at the manuscript for Volume 1. Michael abhors
conditional jumps, especially inside loops. I tested this routine first with a
loop, then with the macro, and got a 10 percent improvement simply by
eliminating the conditional jump. Now I abhor them, too.
A white cane isn't as good as eyesight, and pixel reading isn't as good as
pixel writing. At best, it's a high-overhead operation. So why introduce any
more overhead than absolutely necessary? In this case, a few extra bytes of
code deliver more performance.
After assembling EGAPIXEL.ASM, add it to your copy of GRAFIX.LIB with the DOS
command
 LIB grafix +egapixel;
Okay, so now we can read a pixel. What do we do with it?
RICOCHET.C in Listing Two might not be a very serious program, but it gives a
feel for the possibilities. It's a variant on the idea of the rat in the maze.
The screen has a border and several interior obstacles, all in white. The
"rat" is a moving red pixel subject to the rule that it must always move on
the diagonal. As it travels, it leaves "footprints" or, in other words, draws
a line. At each step, it feels ahead for an obstacle: a white pixel. Upon
encountering one, it changes direction appropriately.
The path the line takes is erratic, ricocheting around among the barriers in a
frenetic pattern that gradually fills every nook and cranny of the maze with
red. The fill never becomes completely solid, though, because eventually the
racing rat begins retracing its steps. When that happens, the program appears
to freeze; it's actually redrawing the same complex path all over again. You
can stop the program any time by pressing a key.


Speaking of Filling...


RICOCHET is obviously neither efficient nor satisfactory as an algorithm for
filling a closed figure. However, it illustrates the important point that any
filling algorithm has to know where to stop. And in order to do that, it must
be able to detect the border color.
The exception to this general observation is drawing solid rectangles, which
we covered back in March. But fill_ rect( ) just draws a series of horizontal
lines bounded by the dimensions of the object. One could also draw other
regular solid polygons in this manner, such as diamonds and circles. At any
point along the left edge, you can calculate the distance to the corresponding
point on the right edge and draw a line. Although the visual effect indicates
otherwise, that's not really filling. Instead, it's a particular application
of line-drawing, and it works only for regular polygons, unless you want to do
an enormous amount of computation.
A more general-purpose filling algorithm will fill any closed object, regular
or otherwise. It does this by spreading "paint" outward from a known interior
point --usually called the seed --until it encounters the border outlining the
object. It will also leave holes in the interior, provided the closed inside
figures have the same border pixel value as their container and the seed is
not within one of them. The usual name for this kind of algorithm is flood
fill.
The need for flood filling is obvious. As it draws the lines forming various
objects, a program can't possibly remember the location of every single pixel
it writes. It's like my blind professor writing on the blackboard: Once the
act was done, he couldn't recall where he'd written, and sometimes he wrote
right over other things. There was no white cane to help him in this case, but
there is for the program. It can keep tapping ahead looking for the border
pixels that comprise the brick walls for the filling algorithm.
Computer science research has produced a number of flood fill methods. Perhaps
the simplest is the recursive fill, written for the GRAFIX environment as
follows:
 void fill (int x, int y, byte border){ if (egapixel (x, y) != border){
draw_point (x, y); fill (x+1, y, border); fill (x-1, y, border); fill
(x, y+1, border); fill (x, y-1, border); } }

The fill restlessly spreads outward from the seed in all directions, painting
pixel positions until it can no longer find any within the border.
The recursive fill works, but it has a couple of drawbacks. One is that it's
not very efficient. The multiple recursive calls spend, on average, something
like 75 percent of the time revisiting adjacent positions that have already
been filled. Even more serious, though, is its hunger for stack space. Stack
demands rise logarithmically with the size of the region being filled.
Programs that use recursive fill are always in danger of crashing due to stack
overflow.
A better flood fill algorithm is the line adjacency method. It too is
recursive, but much less so. The amount of stack space it consumes depends
more on the complexity of the polygon -- holes to work around, strange angles
-- than on the size of the filled region. Nevertheless, if you're using
Microsoft C or QuickC, the 2K default stack probably won't be enough.
The line adjacency algorithm thinks vertically and draws horizontally. Row by
row it strives upward from the seed. Whenever it finds an unfilled row, it
searches for the right and left borders and draws a line between them. It
returns to the seed row when no unfilled positions remain upward, and begins
striving downward in the same fashion. Because it's recursive, the algorithm
remembers where it hasn't yet checked its neighboring row, and eventually it
always goes back to take care of unfinished business. The procedure is done
when there are no unfilled pixel positions within the closed region.
This is a somewhat simplified explanation of how the line adjacency algorithm
works. For a more thorough understanding, watch it in action or, better yet,
trace its execution with a debugger.
This month's contribution to the growing GRAFIX library is EGAFILL.C ( Listing
Three). The module contains floodfill( ), an implementation of the line
adjacency algorithm, along with some supporting functions. If you're typing in
the code, you get off easy this month: All the listings are short. After
compiling this module, add it to the library with
LIB grafix +egafill;
Listing Four shows the additions to your copy of GRAFIX.H. The libraries on
CompuServe and the DDJ diskette both contain a complete, up-to-date version of
the header file.
And finally there's Listing Five a program called FILLS.C that puts it all to
work. FILLS produces three filled objects of increasing complexity, showing
how the line adjacency algorithm does its thing. The first is a simple
triangle. This is followed by a mis-shapen star that resembles a fighter
plane. It's not actually meant to look like anything, but instead to show how
the fill algorithm handles a complex shape. And finally we have a house, which
is made interesting by having six unfilled islands -- the windows -- within
the filled area. Note how the fill routine taps its way around them, and how
it returns at the end to paint the spaces between them.
This program works fine with the default stack size in Turbo C, but with
Microsoft and QuickC you have to override the 2K default with a 5K stack. You
can specify this with the link option /ST:5120.
So that's how filling works with the aid of a software white cane. There are
more efficient ways of doing the same thing, and sooner or later we'll get
around to them. For now we have a working tool that's not difficult to
understand, and which opens up new dimensions of computer graphics. Too bad
that professor can't see them.


Pulling My FAT from the Fire


Horror story: Last week, while working on a program unrelated to this column,
I clobbered the FAT on my hard disk. A 44-meg hard disk, I might add, that was
nearly full. I'm still not sure how it happened. There were some problems with
a wild indexing scheme, so the program probably corrupted some code that, when
it got control later, did an absolute write to several FAT sectors. Naturally,
I hadn't done a backup in some time ("Tomorrow I'll do it").
When you lose pieces of the FAT, you're in a heap of trouble. Just for
starters I lost the entire directory for this column. Fortunately I had a
backup of programs already published or in the mill, but the future stuff I
was working on was gone. So were a lot of other things.
Heroics were in order, and I went to work in a panic. First I got everything
off the disk that I could. Using another computer, I wrote some hasty software
that allowed me to browse sectors and reconstruct broken chains manually. Talk
about a blind man tapping his way around.
But what really pulled my FAT from the fire was the new Norton Utilities
Advanced Edition. Between the Norton Disk Doctor (NDD) and the multifaceted NU
program, I was able to find the problems and twiddle the bits to fix them. I
didn't get everything back, but I got probably 90 percent of the things I
cared about.
I can't say this too strongly. Norton's Advanced Edition is a superlative
product. Get it. It brought me back from a catastrophe, and it might do the
same for you.


Availability


All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063, or call 800-356-2002 (from inside
Calif.) or 800-533-4372 (from outside Calif.). Please specify the issue number
and format (MS-DOS, Macintosh, Kaypro).



Graphics Programming (column)
by Kent Porter


[Listing One]

; EGAPIXEL.ASM: Reads EGA pixel value directly from video memory
; Returns pixel value at x, y, or -1 if outside viewport
; Microsoft MASM 5.1
; C prototype is
; int far egapixel (int x, int y);
; To be included in GRAFIX.LIB
; K. Porter, .MDUL/DDJ.MDNM/ ``Graphics Programming'' column, June '89

.MODEL LARGE
 PUBLIC _egapixel
 EXTRN _vuport : WORD ; far ptr to vuport structure

; Arguments passed from C
x EQU [bp+6] ; Arguments passed from C
y EQU [bp+8]

; Macro to build pixel value in BL
pixval MACRO
 out dx, ax ; set 6845 for bit plane
 mov bh, BYTE PTR es:[si] ; get byte from current bit plane
 and bh, ch ; mask bit
 neg bh ; and flip it
 rol bx, 1 ; move to bit 0 in accum (BH)

 dec ah ; next bit plane
 ENDM

.CODE
_egapixel PROC FAR
 push bp ; Entry processing
 mov bp, sp
 push si ; Save SI (used here)

; Point ES:[BX] to vuport structure
 mov ax, _vuport+2 ; get pointer segment
 mov es, ax
 mov bx, _vuport ; get offset

; Quit if coordinates outside viewport
 mov al, -1 ; return value if outside
 mov cx, y ; get y
 cmp cx, WORD PTR es:[bx+6] ; is y within viewport?
 jge exit ; quit if not
 mov dx, x ; get x
 cmp dx, WORD PTR es:[bx+4] ; is x within viewport?
 jge exit ; quit if not

; Map pixel coordinates to current viewport
 add dx, WORD PTR es:[bx] ; offset x by vuport.left
 push dx ; save remapped X (used later)
 add cx, WORD PTR es:[bx+2] ; offset y by vuport.top

; Point ES to video memory segment
 mov bx, 0A000h
 mov es, bx

; Row offset = y * 80;
 push dx ; (modified by MUL)
 mov ax, cx ; get y
 mov bx, 80
 mul bx ; y * 80
 mov bx, ax ; into BX

; Column offset = x SHR 3
 pop ax ; get x back
 mov cl, 3 ; shift operand
 shr ax, cl ; column offset

; Complete address of pixel byte
 add bx, ax ; BX = row offset + col offset
 mov si, bx ; ES:SI = address

; Build bit mask for pixel
 pop cx ; get x back
 and cl, 7 ; isolate low-order bits
 xor cl, 7 ; number of bits to shift
 mov ch, 1 ; start bit mask
 shl ch, cl ; shift for pixel
 xor bl, bl ; accumulator for pixel value

; Set graphics controller Read Map Select Register
 mov dx, 03CEh ; 6845 command register
 mov ax, 0304h ; first bit plane = 3


; Read bit planes 3-0, accumulating bits in BL
 pixval
 pixval
 pixval
 pixval

; AX = return value for function
 mov al, bl

; Send pixel value back in AL
exit: xor ah, ah
 pop si
 mov sp, bp
 pop bp
 retf
_egapixel ENDP
 END






[Listing Two]


/* RICOCHET.C: A line moving diagonally "feels" its way among obstacles */
/* Illustrates pixel-reading with egapixel() */
/* K. Porter, .MDUL/DDJ.MDNM/ ``Graphics Programming'' column, June '89 */

#include "grafix.h"
#include <conio.h>

void main ()
{
int x = 638; /* current X position of line */
int y = 348; /* and its Y */
int xdir = -1; /* current X direction (-1 = left, 1 = right) */
int ydir = -1; /* and Y direction (-1 = up, 1 = down */

 if (init_video (EGA)) {

 /* Draw a maze and border */
 set_color1 (15);
 draw_rect (0, 0, 639, 349);
 draw_line (141, 40, 141, 119);
 hline (141, 119, 120);
 draw_line (421, 0, 421, 100);
 hline (540, 199, 80);
 hline (360, 160, 120);
 hline (300, 261, 120);
 draw_line (200, 259, 200, 330);
 draw_line (99, 240, 99, 349);

 /* Send the line bouncing around */
 set_color1 (4);
 while (!kbhit()) {
 if (egapixel (x+xdir, y) == 15) /* feel ahead horizontally */

 xdir = -xdir; /* reverse if obstacle */
 if (egapixel (x, y+ydir) == 15) /* same vertically */
 ydir = -ydir;
 x += xdir; /* advance position */
 y += ydir;
 draw_point (x, y);
 }
 getch(); /* clear keyboard buffer */
 }
}







[Listing Three]

/* EGAFILL.C: Line adjacency flood fill algorithm for EGA */
/* For inclusion in GRAFIX.LIB */
/* K. Porter, DDJ Graphics Programming Column, June '89 */
/* ---------------------------------------------------- */

#include "grafix.h"

extern int color1; /* from GRAFIX library */
int border = 15; /* border for floodfill */
int dir = -1; /* fill direction */

void far setfillborder (int color)
{
 border = color;
} /* ----------------------------------------------------------------- */

int far floodfill (int sx, int sy) /* fill bounded region */
{
int x; /* current column */
int left, rite; /* ends of current line */
byte pixel; /* detected pixel value */
int near leftborder (int, int, byte), /* local functions */
 near rightborder (int, int, byte);

 /* find ends of seed row */
 left = leftborder (sx, sy, border);
 rite = rightborder (sx, sy, border);

 /* fill seed row */
 draw_line (left+1, sy, rite-1, sy);

 /* fill adjacent rows in same direction */
 for (x = left+1; x < rite; x++) {
 pixel = egapixel (x, sy+dir); /* inspect adjacent row */
 if ((pixel != color1) && (pixel != border))
 x = floodfill (x, sy+dir);
 }

 /* fill adjacent rows in opposite direction */
 for (x = left+1; x < rite; x++) {

 pixel = egapixel (x, sy-dir);
 if ((pixel != color1) && (pixel != border))
 x = floodfill (x, sy-dir);
 dir = -dir;
 }

 for (x = left+1; x < rite; x++) {
 pixel = egapixel (x, sy-dir);
 if ((pixel != color1) && (pixel != border))
 x = floodfill (x, sy-dir);
 dir = -dir;
 }
 return rite;
} /* -------------------------------------------------------------- */
/* Following are local routines serving floodfill() */

int near leftborder (int x, int y, byte border)
{
byte pixel;

 do {
 --x;
 pixel = egapixel (x, y);
 if ((pixel == border) (pixel == color1))
 break;
 } while (x > 0);
 return x;
} /* -------------------------------------------------------------- */

int near rightborder (int x, int y, byte border)
{
byte pixel;

 do {
 ++x;
 pixel = egapixel (x, y);
 if ((pixel == border) (pixel == color1))
 break;
 } while (x < vp_width());
 return x;
} /* -------------------------------------------------------------- */






[Listing Four]

/* From June, '89 */
/* -------------- */
int far egapixel (int x, int y); /* get pixel value at x, y */

void far setfillborder (int color); /* border for floodfill */

int far floodfill (int sx, int sy); /* fill bounded region */








[Listing Five]

/* FILLS.C: Various filled figures */
/* K. Porter, DDJ Graphics Programming Column, June '89 */

#include "grafix.h"
#include <conio.h>

int triangle[] = {
 20,300, 120,300, 70,200, 20,300
};
int star[] = {
 60, 10, 120,80, 20,60, 100,140, 140,110,
 260,190, 220,70, 370,30, 180, 40, 60, 10
};
int house[] = {
 400,200, 400,340, 600,340, 600,200,
 500,100, 400,200
};

void main() {
int x, y;

 if (init_video (EGA)) {
 set_color1 (15);
 polyline (3, triangle);
 set_color1 (4);
 setfillborder (15);
 floodfill (70, 250);

 set_color1 (14);
 polyline (9, star);
 set_color1 (6);
 setfillborder (14);
 floodfill (140, 80);

 set_color1 (15);
 polyline (5, house);
 for (x = 430; x < 560; x += 60)
 for (y = 220; y < 300; y += 60)
 draw_rect (x, y, 20, 35);
 set_color1 (7);
 setfillborder (15);
 floodfill (500, 200);
 getch();
 }
}



[GRAFIX.LIB]

/* Include file for GRAFIX.LIB */
/* EGA/VGA graphics subsystem */
/* K. Porter, DDJ Graphics Programming Column */

/* ------------------------------------------ */

/* Color constants from April, 89 */
#define Black 0 /* standard colors */
#define Blue 1
#define Green 2
#define Cyan 3
#define Red 4
#define Magenta 5
#define Brown 0x14
#define LtGray 7
#define DkGray 0x38
#define LtBlue 0x39
#define LtGreen 0x3A
#define LtCyan 0x3B
#define LtRed 0x3C
#define LtMagenta 0x3D
#define Yellow 0x3E
#define White 0x3F

#define RED0 0x00 /* basic hues for mixing */
#define RED1 0x20
#define RED2 0x04
#define RED3 0x24
#define GRN0 0x00
#define GRN1 0x10
#define GRN2 0x02
#define GRN3 0x12
#define BLU0 0x00
#define BLU1 0x08
#define BLU2 0x01
#define BLU3 0x09

#if !defined byte
#define byte unsigned char
#endif

/* Supported video modes */
#define EGA 0x10 /* EGA 640 x 350, 16/64 colors */
#define VGA16 0x11 /* VGA 640 x 480, 16/64 colors */

/* Function prototypes */
/* From February, '89 */
/* ------------------ */
int far init_video (int mode); /* init display in video mode */

void far pc_textmode (void); /* PC text mode */

void far draw_point (int x, int y); /* write pixel in color1 */

void far set_color1 (int palette_reg); /* set foreground color */

/* From March, '89 */
/* --------------- */
void far draw_line (int x1, int y1, int x2, int y2);
 /* Bresenham line drawing algorithm */

void far draw_rect (int left, int top, int width, int height);
 /* draw rectangle from top left corner */


void far polyline (int edges, int vertices[]); /* draw polyline */

void far hline (int x, int y, int len); /* horizontal line */

void far fill_rect (int left, int top, int width, int height);
 /* draw solid rectangle in color1 starting at top left corner */

/* From April, '89 */
/* --------------- */
byte far ega_palreg (int preg); /* color in EGA palette reg */

void far set_ega_palreg (int reg, int color); /* set palette reg */

byte far colorblend (byte r, byte g, byte b); /* blend hues */

void far get_ega_colormix (int preg, int *r, int *g, int *b);
 /* get mix of red, green, and blue in EGA pal register preg */

/* From May, '89 */
/* ------------- */
typedef int VP_HAN; /* viewport handle type */

void far default_viewport (int height); /* init default viewport */

VP_HAN far vp_open (int x, int y, int width, int height);
 /* open viewport, make it active */

int far vp_use (VP_HAN vp); /* make vp active */

void far vp_close (VP_HAN vp); /* close viewport */

VP_HAN far vp_active (void); /* get handle of active vp */

void far vp_outline (VP_HAN vp); /* outline vp */

int far vp_width (void); /* get active viewport width */

int far vp_height (void); /* and height */

/* From June, '89 */
/* -------------- */
int far egapixel (int x, int y); /* get pixel value at x, y */

int far floodfill (int sx, int sy, /* seed coords */
 int border, /* border color */
 int dir, /* pass as 0 */
 int pleft, int prite); /* make same as sx */














June, 1989
STRUCTURED PROGRAMMING


Is This Angst Really Necessary?




Jeff Duntemann, K16RA


I was in an art supplies store in Rochester, New York the afternoon Rochester
Gas and Electric's Ginna nuclear power plant burped and spat a little wisp of
radioactive steam into the wind. I'd long since learned to stop worrying and
love alpha particles; between the radon from my basement and the cosmic rays
one can't help but meet at 30,000 feet, I figured Ginna wasn't adding any
statistically significant risk to my life. The owner of the art store, on the
other hand, was obviously petrified. She had a little portable TV propped up
on a pile of Strathmore pads, and was hanging on the news announcer's every
word -- her hands shaking as she kept lighting one cigarette off the end of
the last, going through a full pack in 20 minutes.
"The man who invented radioactivity ought to be shot," she complained as she
rang up my drafting pencils. By that time you could barely see in there, and I
hastened out the door while I still had my lungs.
My bitter sense of irony told me that she had the perfect response to nuclear
angst: Chain smoke long enough and low-level radioactivity ceases to be any
kind of threat. There is a similar (and just as pointless) angst prevalent
among naive programmers: The fear that high-level languages invariably produce
glacially slow code. In terror of a threat that they can't even understand,
much less begin to measure, they react by the HLL equivalent of chain-smoking,
which is to replace one routine after another in their programs with
badly-conceived and barely-operable assembly language. Eventually the programs
implode under a cancerous profusion of interrelated bugs that freeze their
machines solid -- at which point slow code performance or any other kind of
code performance ceases to be an issue.


Don't Worry -- Be Happy


Is this angst really necessary? Cripes almighty, no! Five years of crawling
through native code HLL programs with a debugger have convinced me of this:
Modern high-level languages write better assembly language than 99 percent of
all the programmers who have ever lived or will ever live. Not only are your
chances of writing faster code than the compiler not good, the chances are
excellent that your own assembly language will be slower.
This is truer now than ever before. The battle of the titans between Bill
Gates and Philippe Kahn has produced some killer code generators. (This
includes TopSpeed Modula-2, which began as Turbo Modula-2 when Nils Jensen was
still a Borlander.) Out of the woodwork are coming new compilers like Watcom
C, which beats both Turbo and Quick C in the time trials and are raising the
ante in the programmers' equivalent of the nuclear arms race.
The bottom line is this: Unless you're as good an assembly language programmer
as Anders Hejlsberg, the author of Turbo Pascal, don't worry about HLL code
performance. If you must do something, think hard about your algorithms,
remembering that a bubble sort in assembler is almost certainly slower than a
shell sort in interpreted Basic. Do what you do best -- program in your chosen
HLL -- and let Anders and his rare breed do what they do best: Write programs
that write assembly language.


The 20 Percent Rule


I've identified two broad categories of situations where your chances of
beating the compiler rise to about 50/50: The first category includes
situations when time resolution is a factor. The second category includes any
movement of screen data that represents more than 20 percent of the area of
your screen.
The first category is a thin one indeed. I'm speaking of things like pulling
data points in from some kind of analog/digital converter. The simplest
example is the PC joystick. You can poll the joystick from Pascal or Modula-2,
but you get better resolution of the stick when you poll it from assembler.
The resolution of the stick is directly proportional to the number of times
per second that your code can grab a value from a PC I/O port. Assembler
doesn't necessarily win, but it can win. Keep in mind that you have to be a
good enough assembly language programmer to code the tightest possible loop,
taking advantage of assumptions and special information that the compiler's
code generator doesn't necessarily have.
The second category, on the other hand, is an area of critical concern,
especially when the screen in question is a graphics screen. My 20 percent
rule applies to text screens only; in graphics modes scale that back to 5
percent, or even 0 percent -- maybe I'm picky, but graphics screens are never
fast enough to suit me. Screen refreshes that are slow enough to watch are too
slow. Lines, windows, and other screen objects should appear and vanish
instantly. Accept nothing less.


Blasting Lines


I've managed to build the virtual screens library through three columns
without resorting to assembly language for anything but a simple INLINE macro
for clearing lines in SCREENS.PAS. (DDJ, April 1989.) That macro (ClearLine)
was nothing more than setup and minimal safety check for a single, powerful
op-code: REP STOSW. The STOSW opcode takes whatever's in AX and blasts it out
into memory at the fastest speed of which the '86 iscapable. It's a
machine-code loop that executes entirely within the CPU, behind the mask of a
single instruction. Compilers are not generally smart enough to identify
situations in which REP STOSW is useful, so we have to code them up ourselves.
Turbo Pascal does have the FillChar procedure, which uses the REP STOSB, which
is close, but not quite what we need to clear a screen line with both a chosen
clear character and a chosen attribute character. Also, there is considerable
fooling around that the more general FillChar routine must go through to set
itself up to perform the REP STOSB. We don't need that in this case, but the
compiler can't help generating it. If I needed a maximum-speed buffer filler,
I might in fact be tempted to code it up because there's not that much to it.
The guidance here is to keep it simple, and know your assembly language. I
can't teach you assembly language in this column, lordy, but if you're going
to try it the least I can do is counsel you to read up on it and try to
understand what it's about. Tom Swan has an excellent book: Mastering Turbo
Assembler, on the stands right now, and my own introduction to assembly
language, Assembling From Square One (which speaks of both TASM and MASM) will
be out this fall. Also well into production is Michael Abrash's two-volume
opus The Zen of Assembler, which is definitely the category-killer book on
advanced 86-family assembly language. When both volumes and all 1,600 pages of
Zen are in print, I feel confident that the work will stand as the definitive
reference to performance programming in assembler well into the Twenty-First
Century. Master Zen and you'll be qualified to outthink the sharpest compiler.
On the other hand, having come away from the technical edit of Volume 1:
Knowledge with smoking eyeballs, I will advise you that the learning process
could take awhile.


Cloning a Calendar


So -- to show you a situation where assembly language is both necessary and
effective, let's clone the Sidekick calendar for virtual screens.
The CALTEST.PAS demo program (Listing One) puts a calendar in the middle of
your visible screen window. The right arrow takes you "up-time" and bumps the
month by one into the future; the left arrow takes you "down-time" and bumps
the month by one into the past. Pound on the arrow keys and try to catch any
"flow" as the calendar comes in. You won't, not even on a 4.77MHz-8088
machine.
There are three important elements comprising the calendar. The CALENDAR.PAS
unit (Listing Two) handles the display of the calendar. The calendar
calculations are done in CALCALC.PAS (Listing Three). CALCALC.PAS comes from
Michael Covington at the University of Georgia, and I use it here with his
gracious permission. The key routine for this discussion, however, lies in
file BLKBLAST.ASM (Listing Four). BlkBlast is the assembly language external
to Turbo Pascal that actually moves the calendar patterns to the virtual
screen. BlkBlast was placed in CALENDAR.PAS for convenience's sake; as a
virtual screens primitive it rightfully belongs in SCREENS.PAS and I recommend
you put it there.
The patterns themselves can be placed as arrays of STRING in typed constants,
but I've chosen to gather them into a separate assembly language file,
CALBLKS.ASM ( Listing Five) so that you can see how it's done. Note that the
patterns stored in CALBLKS.ASM are called procedures, but (pretty obviously)
are not meant to be executed. They need to be packaged as procedures to force
the Turbo Pascal linker into loading and linking them to your program. They
are, in fact, data, and are accessed by deriving pointers to them and passing
them as pointer parameters to BlkBlast.


Blasting Blocks


Called from a Turbo Pascal program, BlkBlast copies a linear array of bytes
from one location of memory to a rectangular region in a virtual screen. The
pointer to the array of bytes where the pattern is stored is passed in
StoreEnd, whereas the pointer to the virtual screen is passed in ScreenEnd.
ScreenX and ScreenY are passed the current X and Y size of the visible screen
(that is, the display adapter's buffer), respectively. My TextInfo unit (DDJ,
March 1989) exports two preinitialized variables, VisibleX and VisibleY, that
I use for that purpose.
ULX and ULY are passed the X,Y position of the upper left corner of the
rectangular region within the virtual screen where the pattern is to be
placed. Width and Height are passed the width and height of the pattern stored
at the address passed in StoreEnd. Attribute is passed the attribute byte to
be used for the display of the pattern.
DeadLines and TopStop are special parameters that serve the calendar
application. A calendar is displayed by first blasting the CalFrame pattern
(stored in CALBLKS.ASM) up to the virtual screen. The CalCalc unit is used to
calculate an offset into the CalData pattern. This offset will vary depending
on what day of the week the first of any given month falls on. The sooner in
the week the first falls, the farther into CalData you must move for a
starting point. Starting at this offset position, CalData is blasted atop
CalFrame. Bingo! You have a calendar.
There is some trickiness involving the blast of the data onto the calendar
frame. If you'll notice from CALBLKS.ASM, the numbers for the calendar are
stored in a single-spaced array, when in fact they must be overlaid upon the
calendar frame as double-spaced, in order to leave undisturbed the frame's
lines that separate the rows of day blocks. DeadLines specifies some number of
virtual screen lines that will be skipped between lines of the pattern blasted
to the screen. A line of CalData is blasted to the screen; then a screen line
is skipped; the next line of CalData is blasted to the screen, and so on. It
is not a matter of inserting empty lines, but a matter of "spreading out" the
lines of the pattern on the screen.

TopStop addresses another calendar gotcha: The months have different numbers
of days. To avoid having Februaries with 31 days, BlkBlast must stop blasting
CalData's data onto a February calendar frame before it reaches the characters
for the 29th (except in leap years) 30th, and 31st days. If only a specific
number of bytes of the pattern are to be blasted to the screen, that number is
passed in TopStop. If the entire pattern is to be sent to the screen (as for
the calendar frame) then TopStop is passed a 0.


Modelling a Stack Frame


I won't do an instruction-by-instruction explanation of how BlkBlast works.
There simply isn't room here. What I do want to do is point to some general
techniques that may help you create your own assembly language external
modules.
First of all, model the stack frame in an assembly language structure. This is
the item named ONSTACK in BlkBlast. The structure allows you to access fields
by "dotting," in a fashion similar to what we do with Pascal records. Make
sure you understand what Turbo Pascal pushes on the stack for your parameters,
particularly in areas, like pointers, where it may not be immediately clear
whether the segment or offset portion of the pointer is pushed first. (The
segment is pushed first.)
The identifier ONSTACK is not actually referenced in the routine; it's only
there because all structures must have names. Instead, the structure is
referenced as the referent of SS:[BP], and individual fields within the
structure resolve to offsets from BP. In other words, a structure reference
like MOV AX,[BP]. Attr is equivalent to MOV AX,[BP+10]. The idea is to avoid
having to calculate offsets from BP, especially when you find yourself
inserting new fields into the structure or rearranging the ones that are
there.
Notice the field named ENDMRK at the end of the structure. ENDMRK is not
actually the name of anything pushed onto the stack. Instead, it allows us to
use an assembly language expression to calculate the number of bytes to be
removed from the stack when the external procedure returns control to Turbo
Pascal. The expression comes at the very end of BlkBlast:
 RET ENDMRK-RETADDR-4
The operand to the RET instruction is what "cleans up the stack." When it
executes, RET <n> adds <n> bytes to the stack pointer (remember, this is a
push-down stack), at one stroke wiping the stack frame clean from the stack.
<n> can be calculated by subtracting the start of the stack frame from its
end, minus 4 bytes for the return address. It's important to exclude the size
of the return address from the calculation, because by the time the RET
instruction gets around to adjusting the stack pointer, it has already removed
the return address from the stack.
Also, remember that BlkBlast is a FAR procedure. Had it been written as a NEAR
procedure, the return address would have been a 16-bit offset, rather than a
full 32-bit address, and the expression would have been:
RET ENDMRK-RETADDR-2


Other Assembly Language Pointers


Be sure to declare the name of the external routine as PUBLIC, or Turbo
Pascal's linker will not be able to find the name in the .OBJ file's symbol
table. Any identifier that is to be known outside the .ASM file must be
explicitly declared as PUBLIC. This is in addition to adding the PUBLIC
directive to the declaration of the external's code segment.
The key to fast assembly language work is to identify the tight loops that are
executed dozens or hundreds of times (usually in moving data from one place to
another) and to keep those loops as simple and empty as possible. If you can,
use the 8086 string instructions MOVSB, STOSB, SCASB, and CMPS with the REP
prefix. As I mentioned earlier, this allows you to (in effect) code a loop
that runs entirely within the CPU, fetching no instructions and losing no time
to unnecessary memory accesses. The string instructions can't be used to solve
every problem, but when they can, the results will scorch your eyeballs.
Notice BlkBlast's innermost loop:
 DoChar: LODSB
 STOSW
 LOOP DoChar
Because the attribute must be "mixed" with the pattern on the way from storage
to the virtual screen, this is one case where the REP prefix can't be used.
However, the loop has been pared to its barest essentials: You load a byte
from the pattern into AL (the attribute is already loaded into AH) and then
store AX to the virtual screen as a character/attribute pair. The LODSB and
STOSB instructions increment the SI and DI registers after each pass through
the loop, relieving your own code from that burden.


From The Book CASE


If, like me, you have a fondness for taking your machine by the throat and
shaking it, do not fail to obtain Ray Duncan's new quick reference guide from
Microsoft Press, IBM ROM BIOS. I've used various sources of information on ROM
BIOS over the years, starting with the original 1981 IBM PC Technical
Reference, and most recently Peter Norton's still excellent Programmer's Guide
to the IBM PC. Ray's new book is lean, lucid, and eminently thumbable. It
summarizes all BIOS services and is current through the PS/2. Furthermore, it
includes summaries of interrupt and I/O port usage on all machines through
PS/2. My only gripe is that it doesn't summarize ROM BIOS memory usage, but
that's a small matter. For $5.95, it's a must-have.
One of the multitude of little stupidities that has kept Microsoft from being
richer than it is is their obsession with keeping Windows and PM development
an affair for C programmers. I don't use an environment I can't program, and
the C-flavored Windows API is about as organized and easy to grasp as a gauze
curtain in a Chicago gale. Microsoft's insistence that Windows development is
"not for beginners" insults me. The Windows API wastes my time, and I have
very little patience with things like that. Until Whitewater Group's Actor
language came along, Windows simply gathered dust on my disk.
If for whatever reason you must confront Windows development under C
(shudder), you could do worse than pick up Introduction to Windows Programming
by Guy Quedens and Pamela Beason. It's not the entire story by any means, but
it's as good an entry point into the chaos of the Windows API as you're likely
to find. You'd better know C before opening it, but if you do, the rest will
come naturally. The treatment of the Windows GDI is particularly good, with
plenty of screen shots and figures to make the technical prose gel. And even
if you're using Actor, the background on the Windows environment is well worth
reading.


OOPS!


One of my crustier correspondents sent me a letter asking (among other things)
"Who needs OOPS?" I've been soaking my tail in object-oriented programming
miscellany for the past couple of months, and my response is this: If you've
ever said it, you need it.
We'll begin taking a look at your OOPS options in my next column. The best OOP
tools have nothing to do with C. Nice thought, isn't it?


Availability


All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063; or call 800-356-2002 (from inside
Calif.) or 800-533-4372 (from outside Calif.). Please specify the issue number
and format (MS-DOS, Macintosh, Kaypro).


Products Mentioned


Mastering Turbo Assembler by Tom Swan Howard Sams & Sons, 1989 ISBN
0672-48435-8 $24.95
Programmer's Quick Reference Series: IBM ROM BIOS by Ray Duncan Microsoft
Press, 1988 ISBN 1-55615-135-7 $5.95
Introduction to Windows Programming by Guy Quedens and Pamela Beason Scott,
Foresman & Company, 1988 ISBN 0-673-38058-0 $21.95 Listings disks (2)
available from Guy Quedens for $19.95

Structured Programming" (column)
by Jeff Duntemann



[Listing One]

{ Calendar unit demo program }
{ Jeff Duntemann -- 2/3/89 }


PROGRAM CalTest;


USES DOS,Crt, { Standard Borland units }
 Screens, { Given in DDJ 4/89 }
 Calendar; { Given in DDJ 6/89 }

CONST
 YellowOnBlue = $1E; { Text attribute; yellow chars on blue background }
 CalX = 25;
 CalY = 5;


VAR
 MyScreen : ScreenPtr; { Type exported by Screens unit }
 WorkScreen : Screen; { Type exported by Screens unit }
 Ch : Char;
 Quit : Boolean;
 ShowFor : DateTime; { Type exported by DOS unit }
 I : Word; { Dummy; picks up dayofweek field in GetDate }


BEGIN
 MyScreen := @WorkScreen; { Create a pointer to WorkScreen }
 InitScreen(MyScreen,True);
 ClrScreen(MyScreen,ClearAtom); { Clear the entire screen }
 Quit := False;

 WITH ShowFor DO { Start with clock date }
 GetDate(Year,Month,Day,I);

 ShowCalendar(MyScreen,ShowFor,CalX,CalY,YellowOnBlue);

 REPEAT { Until Enter is pressed: }
 IF Keypressed THEN { If a keystroke is detected }
 BEGIN
 Ch := ReadKey; { Pick up the keystroke }
 IF Ord(Ch) = 0 THEN { See if it's an extended keystroke }
 BEGIN
 Ch := ReadKey; { If so, pick up scan code }
 CASE Ord(Ch) OF { and parse it }
 72 : Pan(MyScreen,Up,1); { Up arrow }
 80 : Pan(MyScreen,Down,1); { Down arrow }
 75 : BEGIN { Left arrow; "down time" }
 WITH ShowFor DO
 IF Month = 1 THEN
 BEGIN
 Month := 12;
 Dec(Year)
 END
 ELSE Dec(Month);
 ShowCalendar(MyScreen,ShowFor,CalX,CalY,YellowOnBlue);

 END;
 77 : BEGIN { Right arrow; "up time" }
 WITH ShowFor DO
 IF Month = 12 THEN
 BEGIN
 Month := 1;
 Inc(Year)
 END
 ELSE Inc(Month);
 ShowCalendar(MyScreen,ShowFor,CalX,CalY,YellowOnBlue);
 END;
 END { CASE }
 END
 ELSE { If it's an ordinary keystroke, test for quit: }
 IF Ch = Chr(13) THEN Quit := True
 END;
 UNTIL Quit;
 ClrScreen(MyScreen,ClearAtom) { All this stuff's exported by Screens }
END.



[LISTING Two]

{--------------------------------------------------------------}
{ CALENDAR }
{ }
{ Text calendar for virtual screen platform }
{ }
{ by Jeff Duntemann KI6RA }
{ Turbo Pascal 5.0 }
{ Last modified 2/3/89 }
{--------------------------------------------------------------}

UNIT Calendar;

INTERFACE

USES DOS, { Standard Borland unit }
 TextInfo, { Given in DDJ 3/89 }
 Screens, { Given in DDJ 4/89 }
 CalCalc; { Given in DDJ 6/89 courtesy Michael Covington }

TYPE
 DaysOfWeek = (Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday);
 Months = (January,February,March,April,May,June,July,
 August,September,October,November,December);


PROCEDURE ShowCalendar(Target : ScreenPtr;
 ShowFor : DateTime;
 CalX,CalY : Integer;
 Attribute : Byte);


IMPLEMENTATION

TYPE
 String10 = STRING[10];


CONST
 MonthNames : ARRAY[January..December] OF String10 =
 ('January','February', 'March','April','May','June','July',
 'August', 'September','October','November','December');
 Days : ARRAY[January..December] OF Integer =
 (31,28,31,30,31,30,31,31,30,31,30,31);

{$L CALBLKS}
{$F+} PROCEDURE CalFrame; EXTERNAL;
 PROCEDURE Caldata; EXTERNAL;
{$F-}

{$L BLKBLAST}
{$F+}
PROCEDURE BlkBlast(ScreenEnd,StoreEnd : Pointer;
 ScreenX,ScreenY : Integer;
 ULX,ULY : Integer;
 Width,Height : Integer;
 Attribute : Byte;
 DeadLines : Integer;
 TopStop : Integer);
 EXTERNAL;
{$F-}



FUNCTION IsLeapYear(Year : Integer) : Boolean;

{ Works from 1901 - 2199 }

BEGIN
 IsLeapYear := False;
 IF (Year MOD 4) = 0 THEN IsLeapYear := True
END;




PROCEDURE FrameCalendar(Target : ScreenPtr;
 CalX,CalY : Integer;
 Attribute : Byte;
 StartDay : DaysOfWeek;
 DayCount : Integer);

TYPE
 PointerMath = RECORD
 CASE BOOLEAN OF
 True : (APointer : Pointer);
 False : (OfsWord : Word;
 SegWord : Word)
 END;

VAR
 DataPtr : Pointer;
 FudgeIt : PointerMath;
 DayInset : Word;
 DayTopStop : Word;


BEGIN
 { DayInset allows is to specify which day of the week the first of the }
 { month falls. It's an offset into the block containing day figures }
 DayInset := (7-Ord(StartDay))*4;
 { DayTopStop allows us to specify how many days to show in the month. }
 DayTopStop := 28+(DayCount*4)-DayInset;
 BlkBlast(Target,@CalFrame, { Display the calendar frame }
 VisibleX,VisibleY, { Genned screen size from TextInfo unit }
 CalX,CalY, { Show at specified coordinates }
 29,17, { Size of calendar frame block }
 Attribute, { Attribute to use for calendar frame }
 0, { No interspersed empty lines }
 0); { No topstop; show the whole thing. }

 WITH FudgeIt DO { FudgeIt is a free union allowing pointer arithmetic }
 BEGIN
 APointer := @CalData; { Create the pointer to the days block }
 OfsWord := OfsWord+DayInset; { Offset into block for start day }

 BlkBlast(Target,APointer, { Blast the day block over the }
 VisibleX,VisibleY, { calendar frame }
 CalX+1,CalY+5, { Pos. of days relative to frame }
 28,6, { Size of day block }
 Attribute, { Show days in same color as frame }
 1, { Insert 1 line between block lines }
 DayTopStop) { Set limit on number of chars to }
 END { be copied from block to control }
END; { how many days shown for a month }




PROCEDURE ShowCalendar(Target : ScreenPtr;
 ShowFor : DateTime;
 CalX,CalY : Integer;
 Attribute : Byte);

CONST
 NameOffset : ARRAY[January..December] OF Integer =
 (8,8,10,10,11,10,10,9,7,8,8,8);

VAR
 StartDay : DaysOfWeek;
 TargetMonth : Months;
 TargetDay : Real;
 DaysInMonth : Integer;

BEGIN
 { First figure day number since 1980: }
 WITH ShowFor DO TargetDay := DayNumber(Year,Month,1);
 { Then use the day number to calculate day-of-the-week: }
 StartDay := DaysOfWeek(WeekDay(TargetDay)-1);
 TargetMonth := Months(ShowFor.Month-1);
 DaysInMonth := Days[TargetMonth];
 { Test and/or adjust for leap year: }
 IF TargetMonth = February THEN
 IF IsLeapYear(ShowFor.Year) THEN DaysInMonth := 29;
 { Now draw the frame on the virtual screen! }
 FrameCalendar(Target,

 CalX,CalY,
 Attribute,
 StartDay,
 DaysInMonth);
 { Add the month name and year atop the frame: }
 GotoXY(Target,CalX+NameOffset[TargetMonth],CalY+1);
 WriteTo(Target,MonthNames[TargetMonth]+' '+IntStr(ShowFor.Year,4));
END;



END.



[LISTING Three]

UNIT CalCalc;

{ --- Calendrics --- }

{ Long-range calendrical package in standard Pascal }
{ Copyright 1985 Michael A. Covington }

INTERFACE

function daynumber(year,month,day:integer):real;

procedure caldate(date:real; var year,month,day:integer);

function weekday(date:real):integer;

function julian(date:real):real;

IMPLEMENTATION


function floor(x:real) : real;
 { Largest whole number not greater than x. }
 { Uses real data type to accommodate large numbers. }
begin
 if (x < 0) and (frac(x) <> 0) then
 floor := int(x) - 1.0
 else
 floor := int(x)
end;



function daynumber(year,month,day:integer):real;
 { Number of days elapsed since 1980 January 0 (1979 December 31). }
 { Note that the year should be given as (e.g.) 1985, not just 85. }
 { Switches from Julian to Gregorian calendar on Oct. 15, 1582. }
var
 y,m: integer;
 a,b,d: real;
begin
 if year < 0 then y := year + 1
 else y := year;

 m := month;
 if month < 3 then
 begin
 m := m + 12;
 y := y - 1
 end;
 d := floor(365.25*y) + int(30.6001*(m+1)) + day - 723244.0;
 if d < -145068.0 then
 { Julian calendar }
 daynumber := d
 else
 { Gregorian calendar }
 begin
 a := floor(y/100.0);
 b := 2 - a + floor(a/4.0);
 daynumber := d + b
 end
end;

procedure caldate(date:real; var year,month,day:integer);
 { Inverse of DAYNUMBER; given date, finds year, month, and day. }
 { Uses real arithmetic because numbers are too big for integers. }
var
 a,aa,b,c,d,e,z: real;
 y: integer;
begin
 z := int(date + 2444239.0);
 if date < -145078.0 then
 { Julian calendar }
 a := z
 else
 { Gregorian calendar }
 begin
 aa := floor((z-1867216.25)/36524.25);
 a := z + 1 + aa - floor(aa/4.0)
 end;
 b := a + 1524.0;
 c := int((b-122.1)/365.25);
 d := int(365.25*c);
 e := int((b-d)/30.6001);
 day := trunc(b - d - int(30.6001*e));
 if e > 13.5 then month := trunc(e - 13.0)
 else month := trunc(e - 1.0);
 if month > 2 then y := trunc(c - 4716.0)
 else y := trunc(c - 4715.0);
 if y < 1 then year := y - 1
 else year := y
end;

function weekday(date:real):integer;
 { Given day number as used in the above routines, }
 { finds day of week (1 = Sunday, 2 = Monday, etc.). }
var
 dd: real;
begin
 dd := date;
 while dd > 28000.0 do dd:=dd-28000.0;
 while dd < 0 do dd:=dd+28000.0;
 weekday := ((trunc(dd) + 1) mod 7) + 1

end;

function julian(date:real):real;
 { Converts result of DAYNUMBER into a Julian date. }
begin
 julian := date + 2444238.5
end;

END. { CalCalc }





[LISTING Four]

;===========================================================================
;
; B L K B L A S T - Blast 2D character pattern and attributes into memory
;
;===========================================================================
;
; by Jeff Duntemann 3 February 1989
;
; BLKBLAST is written to be called from Turbo Pascal 5.0 using the EXTERNAL
; machine-code procedure convention.
;
; This version is written to be used with the SCREENS.PAS virtual screens
; unit for Turbo Pascal 5.0. See DDJ for 4/89.
;
; Declare the procedure itself as external using this declaration:
;
; PROCEDURE BlkBlast(ScreenEnd,StoreEnd : Pointer;
; ScreenX,ScreenY : Integer;
; ULX,ULY : Integer;
; Width,Height : Integer;
; Attribute : Byte;
; DeadLines : Integer;
; TopStop : Integer);
; EXTERNAL;
;
; The idea is to store a video pattern as an assembly-language external or
; as a typed constant, and then blast it into memory so that it isn't seen
; to "flow" down from top to bottom, even on 8088 machines.
;
; During the blast itself, the attribute byte passed in the Attribute
; parameter is written to the screen along with the character information
; pointed to by the source pointer. In effect, this means we do a byte-sized
; read from the source character data, but a word-sized write to the screen.
;
; The DeadLines parm specifies how many screen lines to skip between lines of
; the pattern. The skipped lines are not disturbed. TopStop provides a byte
; count that is the maximum number of bytes to blast in from the pattern.
; If a 0 is passed in TopStop, the value is ignored.
;
; To reassemble BLKBLAST:
;
; Assemble this file with MASM or TASM: "C>MASM BLKBLAST;"
; (The semicolon is unnecessary with TASM.)

;
; No need to relink; Turbo Pascal uses the .OBJ only.
;
;========================
;
; STACK PROTOCOL
;
; This creature puts lots of things on the stack. Study closely:
;

ONSTACK STRUC
OldBP DW ? ;Caller's BP value saved on the stack
RetAddr DD ? ;Full 32-bit return address. (This is a FAR proc!)
TopStop DW ? ;Maximum number of chars to be copied from block pattern
DeadLns DW ? ;Number of lines of dead space to insert between blasted lines
Attr DW ? ;Attribute to be added to blasted pattern
BHeight DW ? ;Height of block to be blasted to the screen
BWidth DW ? ;Width of block to be blasted to the screen
ULY DW ? ;Y coordinate of upper left corner of the block
ULX DW ? ;X coordinate of the upper left corner of the block
YSize DW ? ;Genned max Y dimension of current visible screen
XSize DW ? ;Genned max X dimension of current visible screen
Block DD ? ;32-bit pointer to block pattern somewhere in memory
Screen DD ? ;32-bit pointer to an array of pointers to screen lines
ENDMRK DB ? ;Dummy field for stack struct size calculation
ONSTACK ENDS


CODE SEGMENT PUBLIC
 ASSUME CS:CODE
 PUBLIC BlkBlast

BlkBlast PROC FAR
 PUSH BP ;Save Turbo Pascal's BP value
 MOV BP,SP ;SP becomes new value in BP
 PUSH DS ;Save Turbo Pascal's DS value

;-------------------------------------------------------------------------
; If a zero is passed in TopStop, then we fill the TopStop field in the
; struct with the full size of the block, calculated by multiplying
; BWidth times BHeight. This makes it unnecessary for the caller to
; pass the full size of the block in the TopStop parameter if topstopping
; is not required.
;-------------------------------------------------------------------------
 CMP [BP].TopStop,0 ; See if zero was passed in TopStop
 JNZ GetPtrs ; If not, skip this operation
 MOV AX,[BP].BWidth ; Load block width into AX
 MUL [BP].BHeight ; Multiply by block height, to AX
 MOV [BP].TopStop,AX ; Put the product back into TopStop

;-------------------------------------------------------------------------
; The first important task is to get the first pointer in the ShowPtrs
; array into ES:DI. This involved two LES operations: The first to get
; the pointer to ShowPtrs (field Screen in the stack struct) into ES:DI,
; the second to use ES:DI to get the first ShowPtrs pointer into ES:DI.
; Remembering that ShowPtrs is an *array* of pointers, the next task is
; to index DI into the array by multiplying the top line number (ULY)
; less one (because we're one-based) by 4 using SHL and then adding that
; index to DI:

;-------------------------------------------------------------------------
GetPtrs: LES DI,[BP].Screen ; Address of ShowPtrs array in ES:DI
 MOV CX,[BP].ULY ; Load line address of block dest. to CX
 DEC CX ; Subtract 1 'cause we're one-based
 SHL CX,1 ; Multiply CX by 4 by shifting it left...
 SHL CX,1 ; ...twice.
 ADD DI,CX ; Add the resulting index to DI.

 MOV BX,DI ; Copy offset of ShowPtrs into BX
 MOV DX,ES ; Copy segment of ShowPtrs into DX
 LES DI,ES:[DI] ; Load first line pointer into ES:DI

;-------------------------------------------------------------------------
; The inset from the left margin of the block's destination is given in
; struct field ULX. It's one-based, so it has to be decremented by one,
; then multiplied by two using SHL since each character atom is two bytes
; in size. The value in the stack frame is adjusted (it's not a VAR parm,
; so that's safe) and then read from the frame at the start of each line
; blast and added to the line offset in DI.
;-------------------------------------------------------------------------
 DEC [BP].ULX ; Subtract 1 'cause we're one-based
 SHL [BP].ULX,1 ; Multiply by 2 to cover word moves
 ADD DI,[BP].ULX ; And add the adjustment to DI

;-------------------------------------------------------------------------
; One additional adjustment must be made before we start: The Deadspace
; parm puts 1 or more lines of empty space between each line of the block
; that we're blasting onto the screen. This value is passed in the
; DEADLNS field in the struct. It's passed as the number of lines to skip,
; but we have to multiply it by 4 so that it becomes an index into the
; ShowPtrs array, each element of which is four bytes in size. Like ULX,
; the value is adjusted in the stack frame and added to the stored offset
; value we keep in DX each time we set up the pointer in ES:DI to blast the
; next line.
;-------------------------------------------------------------------------
 SHL [BP].DEADLNS,1 ; Shift dead space line count by 1...
 SHL [BP].DEADLNS,1 ; ...and again to multiply by 4

 LDS SI,[BP].Block ; Load pointer to block into DS:SI

;-------------------------------------------------------------------------
; This is the loop that does the actual block-blasting. Two counters are
; kept, and share CX by being separate values in CH and CL. After
; each line blast, both pointers are adjusted and the counters swapped,
; the LOOP counter decremented and tested, and then the counters swapped
; again.
;-------------------------------------------------------------------------
MovEm: MOV CX,[BP].BWidth ; Load atom counter into CH
 MOV AH,BYTE PTR [BP].Attr ; Load attribute into AH
DoChar: LODSB ; Load char from block storage into AL
 STOSW ; Store AX into ES:DI; increment DI by 2
 LOOP DoChar ; Go back for next char if CX > 0

;-------------------------------------------------------------------------
; Immediately after a line is blasted from block to screen, we adjust.
; First we move the pointer in ES:DI to the next pointer in the
; Turbo Pascal ShowPtrs array. Note that the source pointer does NOT
; need adjusting. After blasting through one line of the source block,
; SI is left pointing at the first character of the next line of the

; source block. Also note the addition of the deadspace adjustment to
; BX *before* BX is copied into DI, so that the adjustment will be
; retained through all the rest of the lines moved. Finally, we subtract
; the number of characters in a line from TopStop, and see if there are
; fewer counts left in TopStop than there are characters in a block line.
; If so, we force BWidth to the number of remaining characters, and
; BHeight to one, so that we will blast only one remaining (short) line.
;-------------------------------------------------------------------------
 MOV ES,DX ; Copy ShowPtrs segment from DX into ES
 ADD BX,4 ; Bounce BX to next pointer offset
 ADD BX,[BP].DeadLns ; Add deadspace adjustment to BX
 LES DI,ES:[BX] ; Load next pointer into ES:DI
 ADD DI,[BP].ULX ; Add adjustment for X offset into screen

 MOV AX,[BP].TopStop ; Load current TopStop value into AX
 SUB AX,[BP].BWidth ; Subtract BWidth from TopSTop value
 JBE GoHome ; If TopStop is <= zero, we're done.
 MOV [BP].TopStop,AX ; Put TopStop value back in stack struct
 CMP AX,[BP].BWidth ; Compare what remains in TopStop to BWidth
 JAE MovEm ; If at least one BWidth remains, loop again
 MOV [BP].BWidth,AX ; Otherwise, replace BWidth with remainder
 JMP MovEm ; and jump to last go-thru

;-------------------------------------------------------------------------
; When the outer loop is finished, the work is done. Restore registers
; and return to Turbo Pascal.
;-------------------------------------------------------------------------

GoHome: POP DS ; Restore Turbo Pascal's
 MOV SP,BP ; Restore Turbo Pascal's stack pointer...
 POP BP ; ...and BP
 RET ENDMRK-RETADDR-4 ; Clean up stack and return as FAR proc!
 ; (would be ENDMRK-RETADDR-2 for NEAR...)

BlkBlast ENDP
CODE ENDS
 END





[LISTING Five]


 TITLE CalBlks -- External calendar pattern blocks

; By Jeff Duntemann -- TASM 1.0 -- Last modified 3/1/89
;
; For use with CALENDAR.PAS and BLKBLAST.ASM as described in DDJ 6/89

CODE SEGMENT WORD
 ASSUME CS:CODE


CalFrame PROC FAR
 PUBLIC CalFrame
 DB '͸'
 DB ' '

 DB 'Ĵ'
 DB 'SunMonTueWedThuFriSat'
 DB 'Ĵ'
 DB '       '
 DB 'Ĵ'
 DB '       '
 DB 'Ĵ'
 DB '       '
 DB 'Ĵ'
 DB '       '
 DB 'Ĵ'
 DB '       '
 DB 'Ĵ'
 DB '       '
 DB ';'
Calframe ENDP

CalData PROC FAR
 PUBLIC CalData
 DB '       '
 DB ' 1 2 3 4 5 6 7'
 DB ' 8 9 10 11 12 13 14'
 DB ' 15 16 17 18 19 20 21'
 DB ' 22 23 24 25 26 27 28'
 DB ' 29 30 31    '
 DB '       '
 DB '       '

CalData ENDP


CODE ENDS

 END




























June, 1989
OF INTEREST





The availability of an ANSI Fortran 77 standard compiler for FlexOS 386 -- a
real-time, protected-mode operating system--has been announced by Digital
Research. Offered by TransWare Enterprises of San Jose, Calif., the Lahey F77L
compiler's capabilities include real-time control in a multiuser and
multitasking environment, and Fortran and DOS support.
The Lahey F77L for FlexOS 386 supports the functionality of FlexOS, including
32-bit protected-mode code generation for Intel 80386 microprocessor-based
systems and the mutiltasking/multiuser capabilities of FlexOS 386.
Additionally, F77L-386 provides adherence to the IEEE standard for floating
point arithmetic. Suggested real price is $1,277. Reader Service No. 20.
Digital Research, Inc. Box DRI Monterey, CA 93942 408-649-3896
Abraxas Software has announced a new software development tool called
CodeCheck, which analyzes code for portability, maintainability, and style.
CodeCheck is designed to target code for compatibility among PC DOS, OS/2,
Unix, VMS, and Macintosh environments. According to Abraxas, CodeCheck also
identifies complex code as it is written.
Selling for $295, CodeCheck supports C compilers from major vendors and
requires 512K memory. Reader Service No. 21.
Abraxas Software, Inc. P.O. Box 19586 Portland, OR 97219 503-244-5253
Guidelines Software Inc., creator of Guidelines C++ for MS-DOS, has released
Guidelines C++ for Unix V/386. C++ is a superset of the C language and is ANSI
C compatible.
C++ supports object-oriented programming with such features as classes,
inheritance, member functions, constructors and destructors, data hiding, and
data abstractions. C++ is implemented as a translator and requires a back-end
C compiler to produce executable code.
Guidelines C++ for MS-DOS sells for $295 and requires 640K of RAM, a hard
disk, and Microsoft C 3.0 or later. Guidelines C++ for Unix V/386 sells for
$495 and works with standard Unix C compilers; it also supports cross
translation with MS-DOS as the target system. Reader Service No. 22.
Guidelines Software, Inc. P.O. Box 749 Orinda, CA 94563 415-254-9183
StarPath Systems has released Vmos/3, a virtual multitasking operating system
for the 80386. Vmos/3 supports multitasking of DOS software on 80386 PCs,
using demand and virtual paging of memory.
Vmos/3 supports multitasking on standard 1-Mbyte DOS systems; it also manages
and pages memory in 4K portions, rather than as fixed partitions or virtual
machines. Vmos/3 makes use of available RAM for an automatic disk cache. This
cache memory is automatically released for task use, as required.
Video adapter support includes all modes of MDA, CGA, EGA, and VGA in
background and foreground. Graphics tasks running in background automatically
have their screens displayed when brought into foreground; there is no need to
"repaint" the screen to see its current contents. An operator mode provides
real-time status displays of resource usage. A running system log may be
displayed and examined, and is stored on disk.
Several Vmos/3 supplemental products are in development at StarPath, such as a
multiuser that supports serial terminals under DOS, and a 32-bit API, which is
an inegrated LAN, DOS, and 386 development environment.
Vmos/3 sells for $99; the multiuser option is $99. Reader Service No. 23.
StarPath Systems 4700 S Hagadorn Rd. East Lansing, MI 48823 800-456-8667
An 80386 native mode software debugger, which will be available by year end,
has been introduced by Intel. The debugger features a windowed, source-level
user-interface technology with drop-down menus, functions keys, and mouse
support.
With this product, users can scroll across source files, "point and go" to a
line at the cursor, and use drop-down menus to view the sequence of procedure
calls, parameters, and local variable values.
The 80386 software development package will include the software debugger
feature in the new user interface, compiler, assembler, linker, locator, and
other software utilities. The single unit price of the package will be $4,500.
Reader Service No. 24.
Intel Corp. Literature Dept. BP10 3065 Bowers Ave. Santa Clara, CA 95051
800-548-4725 800-874-6835
Pascal-2 is now available for Xenix/386 machines, Oregon Software has
announced. In addition to the DEC hosts, Pascal-2 is available for Sun-3,
Motorola 680x0, and Apollo workstations, as well as the 80286 running MS-DOS.
According to Oregon Software, it has adhered to level 1 of the Pascal language
standard as defined by the International Standards Organization; thus, users
of Pascal-2 can port their Pascal-2 code from one machine to another.
The 80386 environment can address two segments: data and text. Each segment
supports up to 1 gigabyte of memory. The text section supports 32-bit
addressing. The Pascal-2 Xenix/386 compiler is accompanied by a debugger,
assembly language interface, source code formatter, procedure
cross-referencer, and identifier cross-referencer.
Oregon Software also provides C++ and Modula-2 compilers on the Sun-3. Oregon
Modula-2 is available on VAX/VMS and Xenix/386. Oregon C++ will be available
on Xenix/386 in June.
Pascal-2 Xenix/386 lists for $995 for a single user license, and $250 per year
for support services. Network licenses are also available. Reader Service No.
26.
Oregon Software 6915 SW Macadam Ave. Portland, OR 97219-2397 503-245-2202
Parallel Logic Programming (PLP) has announced the launch of MacParlog and
PC-Parlog, implementations of Parlog for the Macintosh and IBM PC families of
microcomputers.
Parlog (PARallel LOGic) is a logic programming language designed at Imperial
College in London. It supports a declarative style of programming, combined
with an execution mechanism for parallel architectures. On sequential
architectures, Parlog programs are executed by timesharing the single
processor between processes.
MacParlog is compatible with the Macintosh Plus, SE, and II; PC-Parlog works
on IBM PCs and compatibles. These products are aimed at researchers and
product developers who need a testbed environment for experiment and prototype
construction in AI and parallel computing, as well as those wishing to learn
about concurrent logic programming and fifth generation computing.
MacParlog and PC-Parlog feature an incremental compiler for Parlog, a
concurrent debugger with dynamic window tracing, a base of primitives, and a
run-time system that uses a "bounded depth-first" scheduling mechanism.
MacParlog is integrated into the Macintosh WIMP environment, and MacParlog
programs can create and maintain windows, menus, and dialogues.
In an arrangement with Logic Programming Associates, PLP has developed toolkit
versions of these two products that coreside with LPS's products MacProlog and
Prolog Professional. These versions are targeted to third-party product
developers seeking to develop mixed-language stand-alone applications.
The single-machine license for MacParlog or PC-Parlog sells for $250. Reader
Service No. 25.
Parallel Logic Programming Ltd. P.O. Box 49 Twickenham England TW2 5PH UK
PURART has released Trapper, a debugger board that assists Borland's Turbo
Debugger by watching for memory and I/O activity from the debugger's
breakpoint menu. Trapper also supports an optional coprocessor pod for
developers who want to trap program instruction accesses to the math
coprocessor chip.
With Trapper, developers can locate "memory bashing" bugs by providing
breakpoints whenever a program accesses a memory address or range of address;
code in ROM, such as BIOS routines; and an I/O port or range of ports.
Additionally, Trapper provides a breakout button, allowing developers to get
back into the debugger even if a program has disabled interrupts. Trapper
sells for $195.95. Reader Service No. 27.
PURART, Inc. P.O. Box 189 Hampton Falls, NH 03844 603-772-9907
A ROM-resident operating system, ROM-DOS has been launched by Datalight. It
will boot up on a standard PC, operate from within ROM, and run MS-DOS
executable (.EXE and .COM) files.
ROM-DOS is compatible with MS-DOS and runs programs such as Microsoft Word,
Turbo C, and Lotus 123. Developers can write code using Basic, C, Pascal, and
so on; compile it with their compiler for MS-DOS operation on a PC; and then
load it into ROM along with ROM-DOS to run in the target system with no
further modification. ROM-DOS takes about 29K of ROM and employs 5K of RAM
when running.
In addition, Datalight has developed a mini-BIOS for use in embedded systems.
Developers may choose to use a full standard BIOS or the Datalight mini-BIOS
in the final system. The mini-BIOS provides support for a remote console (via
a serial port), hardware timer, and serial ports. The mini-BIOS requires less
than 3K bytes of ROM and uses the standard for BIOS RAM area.
ROM-DOS supports memory management, a standard file system, time functions,
and installable device drivers. The 32K ROM of an Intel Wildcard can hold
ROM-DOS and mini-BIOS.
ROM-DOS sells for $6 per licensed copy in quantity of 5,000. A ROM-DOS
developer's kit sells for $495, and a license to the source code costs $5,000.
Reader Service No. 28.
Datalight 17505 68th Ave. NE, Ste. 304 Bothell, WA 98011 206-486-8086
Compuquest has announced its family of data communication products that
correct data errors without retransmission. Included is a 4800-baud cellular
data modern that, according to Compuquest, allows users to transmit error-free
data via cellular telephone at high speeds. The modern sells for $1,695.
Compuquest has also released a 5-lb. laptop terminal with a built-in 1200 baud
cellular data modem and VT-220 emulation. The terminal is priced at $1,798.
Other members of the new product line include a V.33 leased line modem with
error-free transmission rates up to 28000 baud (selling for $3,490), and a
9600 baud V.32 modem (selling for $1,595).
All of the products use a proprietary protocol called Compuquest
Communications Protocol (CCP) for data compression, forward error correction,
and dynamic data management. Reader Service No. 29.
Compuquest, Inc. 801 Morse Ave. Schaumburg, IL 60193 312-529-2552






June, 1989
SWAINE'S FLAMES


It's Not Me, Either




Michael Swaine


Coming out of a screaming six-G dive, Pilot Spiff unleashes the terrible
firepower of his P51 Mustang's six .50-caliber machine guns. Blam! Blooey! Two
critical struts weakened, the Eiffel Tower totters for a horrifying second and
then crashes to the Paris streets in a heap of twisted steel.
Yep. Donald Hill's P51 Mustang Flight Simulator is a fine cathartic. Now on to
the mailbag.
First, a big thank you to all of you who have written in to say that you
agreed with me about writers' responsibility for the quality and accuracy of
their work. I'm especially grateful for all those examples of typos,
misspellings, factual errors, and general evidence of senility that you found
in my recent columns and were kind enough to send along for my edification,
often with witty annotations of your own. You can stop now.
Here are the patches: In the February "Flames," replace "Kerouak" with
"Kerouac," "National" with "Notional" (the National Science Foundation does
not in fact belong to author Nick Herbert), "Sometimes" with "Some time," and
move the comma after "ecliptic" outside the parentheses. In the March
"Programming Paradigms," replace "PC Magazin" with "Design Elektronik"
globally. Jurgen Fey was miffed that I couldn't remember what magazine he
worked for.
"Typos Hapen" is a bumpersticker I haven't seen but would gladly buy. In fact,
I'd buy two and give one to an editor friend of mine who begged me not to
mention his name in my column this month. No, its not Jurgen. But it wasn't
typos that I had in mind when I announced my intention a few months ago to
point out egregious uses of words and figures in the computer press, but bits
like these from a recent Microsoft Word 4.0 ad:
"...startling, cutting-edge technology." (And it holds the road like a dream.)
"...features as dynamic and diverse as its users." (OK, maybe Microsoft
researched this one.)
"And, software fans...." (Oh, you crazy word processor groupies.)
"...practically the only thing it can't do for you is think. Yet." (Or write.
Evidently.)
Microsoft wasn't so funny back when Tyler Sperry was writing copy for them.
Poor Tyler. Last I heard he was sleeping in the streets or editing Embedded
Systems Programming, something like that.
Speaking of embedded things, which I do with some trepidation, considering the
keen linguistic sensitivity of some readers of this column, reminds me of the
subject still languishing at the bottom of the mailbag: fighting software
piracy by embedding machine serial numbers in ROM. Several readers wrote in
about the subject, all cogently and politely trashing the scheme.
Peter Aitkin of Durham, North Carolina, stated the basic objection: "Each
software package that I purchase is installed on all of my computers; the
manuals are schlepped back and forth as needed. No one but me ever uses any of
these machines. Surely such an arrangement is within the spirit, if not the
letter, of software licensing agreements." After all, it is the person, not
the computer, who pays for the software. Shouldn't it be the person, rather
than the computer, to whom it is licensed?
Aitkin also pointed out that tying the software to a serial number in ROM
makes it impossible for the user to upgrade the hardware while keeping the old
software.
Diehl Martin of Huntsville, Alabama, sent a thoughtful dissertation on the two
issues of copyright protection and copy protection. Among other points, he
opines that software developers are no better than anyone else when it comes
to copying software. "Since imitation is the sincerest form of flattery,"
Diehl says, "what wonderful things we must be saying about each others' work!"
Tim Deardeuff of Provo, Utah, nicely demonstrated that the ROM scheme affords
no greater protection than any software copy protection scheme, since all the
user has to do to defeat it is to copy the disk before installing the software
on the target machine. The developer could always make the disk difficult to
copy, but this reduces to the old discredited scheme of software copy
protection, and the serial number in ROM buys the developer nothing extra
while inconveniencing the user.
Did the critical clause of the non-competition contract between Steve Jobs and
Apple Computer expire? The machine that would be sold Only to Higher Education
can now be ordered at Businesslands everywhere. What a surprise.
And just as the courts informed Microsoft that its license from Apple doesn't
go beyond Windows 1.0, the company that managed the Beatles informed Apple
that its agreement on the use of the name Apple explicitly excludes use in any
music-related activity. I dunno; maybe litigation is funnier than publishing.
































July, 1989
July, 1989
EDITORIAL


Credit Where Credit's Due




Jonathan Erickson


In May 1988, we published what turned out to be a popular article on TIFF, the
tagged image file format that's become an industry standard for storing
bit-mapped images. In classic DDJ spirit, Tony Meadows and his co-authors said
that DDJ readers could receive free copies of the TIFF Library Package from
one of two sources: A PC version from Dest Corporation and a Mac version from
Tony's company, Bear River Associates. Thanks to Los Angeles reader Don Black
(who wanted me to plug "Eager," his new 3-D rendering and animation software
-- but I told him I couldn't do that), we found out that this policy has
changed. But first, a little background....
About two years or so ago, I was at the news conference where Microsoft,
Aldus, and Hewlett-Packard introduced the TIFF standard and, like most in
attendance, I was impressed with the proposal and the possibilities that it
suggested. In retrospect, one company that wasn't mentioned or, at least to my
mind wasn't given justifiable credit, was Dest, a small Silicon Valley company
known primarily for its scanners and imaging software. As it turns out, it was
the R&D group at Dest who actually did a lot of the groundwork for the TIFF
standard and it was the same group who provided the PC-version of the TIFF
Library free to DDJ readers.
After the article was published, Dest was deluged with requests for the
software. Some months, in fact, Dest spent more than $20,000 distributing the
material until the company was eventually forced into changing its policy and
began charging $25 per package. Alas, Dest has since run upon some rocky
economic shoals, most recently going into reorganization so it comes as no
surprise that they've discontinued the distribution program altogether. I
don't know how or if the company will be successful in its reorganization, but
I do think DDJ, its readers, and the industry-at-large owe Dest a special
thanks for getting some really useful software into the hands of programmers
-- and free at that.
If you need the TIFF Library, all isn't lost. You can now get both the PC and
Mac versions from Image Software Associates. For $25 ($35 international) you
get documentation (the TIFF standard and a programmer's reference guide),
source code, object files, and demonstration programs. Make checks or money
orders payable to Image Software Associates, Research & Development, P.O. Box
1634, Danville, CA 94526. Don't forget to specify PC or Mac.
If you're a telecommunications kind of person, you'll be glad to know that
we'll be making DDJ source code listings available via an additional source.
Thanks to contributing editor David Betz and his partner in programming Bill
Garrison, DDJ listings will be available free of charge on an on-line system
out of New Hampshire. For the time being, you will have to take care of the
long distance charges yourself, but that and other changes may be in the
offing.
The system supports 300/1200/2400 baud, uses 8-data bits, no parity, and
1-stop bit. To access the system, dial 603-882-1599. When the system answers,
the Unix login prompt will appear and you should type listings (be sure to use
lowercase) and press Return. You'll then be led to the listings. This month's
listings will be on-line as well as past months, at least through the
beginning of this year.
Finally, new applications of computer software continue to make the news here
in Northern California. Most recently, an "escort service" ring was rung up by
the local gendarmes. It seems that some ladies-of-the-night were allowing
their clients to pay up using credit cards and, instead of listing the charge
as "services rendered," they were filling out the credit slips as charges for
computer software. Hmmm ... that certainly adds a new dimension to the concept
of user friendly. The next step will be probably sex-object-oriented
programming (that's right, SOOPs). Or maybe real genetic programming. All
right, if you can do any better let me know but keep it clean, okay?









































July, 1989
LETTERS







Crotchet Contemplation


Dear DDJ,
These notes were prompted by "Crotchet No. 5: The Great Debate" in the October
1988 DDJ ("C Programming" by Al Stevens) and are offered in the spirit of
thoughtful contemplation as to what the hell we're supposed to be doing and
how the hell we should best achieve it. Others may have different views!
Systems analysis is the distillation of a user requirement into a programmable
specification. In my experience, the whole process seems to reduce to the
following two fundamental precepts: 1. Derive and understand the concepts and
rules by which data objects may attributed, operated on, and related to other
objects to produce a meaningful end result. 2. Maintain and know how to use a
set of tools, libraries, and techniques to enable the crafting of the
masterpiece.
I think most people would agree that the use of these precepts is regarded as
a skill. These precepts, however, are more general than they seem. With
suitable generalizations they may be applied to any engineering discipline and
indeed to any traditional art form.
My dictionary describes "art" as "skill applied to imitation and design."
Thus, a program does not need aesthetic quality to be art. Assuming the above
precepts are valid and that a program is the medium of an analysis much as a
painting is the medium of an expression, then programming is an art.
The definition given for "aesthetic" is "appreciated as having beauty." I
regard (nearly) all my programs as things of beauty! More importantly, I
generally regard other programs as things of beauty if well-crafted. Thus,
programs have aesthetic appeal; what comprises that appeal in a program could
be (and has been) argued over at length.
Furthermore, the definition given for "science" is "the pursuit of systematic
and formulated knowledge." Programming, being the application of defined
knowledge, is thus not a science. The development of the rules of programming,
however, may be regarded as such by this definition. Hence the apt term
"computer science" describing the formulation and pursuit of programming
knowledge.
Thus, I would argue that computer science is a true science that involves the
formulation and pursuit of programming knowledge using the art (application)
of programming (software engineering) to help develop and impart that
knowledge.
Tomorrow I will, as Douglas Adams said (roughly), "go on to prove that black
is white and get run over at the next zebra crossing!" This argument can be
generally applied to other disciplines, as in chemistry as a science and
chemical engineering as its application. And yes, programming is a discipline
even though its practitioners may not be!
Now we come to the process by which the analysis may be turned into a useful
program. To this end, programmers will have at their disposal an array of
tools and libraries that implement general concepts to allow construction of
specific code. In C this will typically be achieved by making a set of
structures (objects) with sets of attributes (fields) and writing/using a set
of routines to operate on and relate the objects.
Object-oriented programming tools are designed to partially automate and so
make more reliable the process by which the analysis is turned into operating
code. C++ is one such tool. It attempts to allow the programmer to think and
work in the context of objects, attributes, and operators rather than having
to translate them mentally into structures, fields, and routines.
Additionally, maintenance is simplified because object-oriented programming
imposes greater control over the scope of the code and the type of the
objects. Interestingly, this is exactly what Pascal attempts to do; however,
its scope and type rules are bound into the language thus, severely
restricting its usefulness. A language, such as C++, allows programmers to
define a set of type and scope rules according to their applications. The
compiler then enforces these self-imposed rules. This is exactly what we want!
For this, programmers have to think harder and more clearly. The extra effort
pays off in reduced debug and maintenance time. The concept of increasing
abstraction is vital to code generation and maintenance cost-effectiveness. In
days gone by the simple goto was used to implement all sorts of wonderful
conceptual constructs. Structured programming languages evolved to formalize
and automate some of the more common of these constructs to provide concepts
such as repeat, while, and so on, using machine code gotos for their
implementation. Similar arguments apply to every keyword that any language
employs. Ratfor was a good early example: Pascal and C have persisted longer
and further.
More Details.
In conclusion, this is a limited discussion of programming issues, but I think
they are the fundamental points from which further arguments develop. C++ is
more subtle than might appear from this brief summary. I have just bought
Zortech's compiler (three cheers for a well-thought-out system at a reasonable
price) and have yet to really make it sweat. My arguments are made from the
"wants" list, rather than the "fully implemented" list.
Guy McIlroy
Norseman Systems
Australia


Down and Dirty, That's Us


Dear DDJ,
I've wanted to write this letter for a long time, but haven't had time to put
printhead to paper. It seems that Dr. Dobb's is getting further and further
away from the kind of programming that really matters, and more and more into
obscure research topics in computer science.
Where I work, milliseconds are important. What matters is how many messages
you can process in ten milliseconds. And it's never enough, so you go back and
sweat over the code. Believe me, all those object-oriented paradigms aren't
worth a plugged nickel in the real world. All that structured-programming data
abstraction is the wrong way to go, too. To get speed, you don't distance
yourself from your data -- you've got to get down and dirty with your data.
The guys who are fighting the war are not the generals in the hotel rooms with
their sparkling white maps and military icons, sipping tea from Wedgewood
china. No, they're the guys in the trenches, with their buffers and flags and
registers.
Now what audience are you going to write for? Do you want to be the Vogue of
programming? The National Enquirer? Cosmopolitan? Gee, you used to be more
like Popular Mechanics. Just remember, there are a lot more privates than
there are generals. And Popular Mechanics had Mimi, too.
Rick Rodman
Manassas, Virginia
DDJ: Gosh Rick, we've always been more comfortable in the trenches than the
hotel rooms and our approach continues to be more Popular Mechanics than
Scientific American. We write for the serious programmer -- as evident by the
hundreds of lines of source code we publish in each issue -- and, by sharing
faster, better, smaller programming techniques that solve today's problems
today, DDJ will always be a hotrodding magazine for programmers. But we also
have a responsibility to lead the way as technology evolves. Five years from
now, programming won't be the same as today, just as today's environments are
different from those five years in the past.


Shakespeare Said Something Similar


Dear DDJ,
I just received my second copy of DDJ and my comment on memory limitation is
that about the time the technical limitations were overcome, they were
replaced by the corporate money-grubbers who, with the wholehearted
cooperation of Congress, replaced the technical barriers with a cost barrier.
I am convinced that about half of the impediments to progress could be cured
by shooting all corporate accountants. Shooting all corporate lawyers wouldn't
hinder solving the other half of the problems, either.
I am not (at least as yet) a professional programmer. I'm just a hard-core
electronics nut just getting into his second half-century of obsessive
interest in the field. I was a professionally trained repair technician back
in the 40s and 50s, but I found other work when transistors and PC boards came
on. What I know about computers and programming comes from self-education and
I'm still learning. Diving into Dr. Dobb's Journal was a little like wandering
onto a nude beach by accident -- a little shocking like any transition to a
new environment, but stimulating and educational.
Billy R. Pogue
Lake Havasu City, Arizona


Superlinearity Without Mirrors


Dear DDJ,

I would like to describe a case of superlinearity in a computer with parallel
architecture, that will not look like magic to Michael Swaine.
We need to search a disordered space of 10 million strings to see whether it
contains a particular string. With a single processor, we examine string 1,
then string 2, and so on, until we find a match or until all strings have been
examined. But with ten parallel processors, we conduct ten searches
simultaneously, with the i - th processor beginning at string (i - 1)*1e6.
This parallel search will be complete in one million steps.
Now let j be a multiple of one million, and let x be any number less than one
million. If the given string is to be found at location j + x, then the
sequential search would take j + x steps, and linearity would require (j+x)/10
steps. The parallel search would take x steps. Clearly, superlinearity would
raise its mystic head whenever (j + x)/ 10 > x (about half of the time).
Phil Geffe
(Consulting Engineer)
Cincinnati, Ohio


It's A Dirty Job But...


Dear DDJ,
My new issue of DDJ arrived today and it's the first really good programming
related thing to happen in a week. As usual I read the "Programming Paradigms"
column first. Well, Michael Swaine, you have become the apple of my eye. I
have been screwing around with a major database crash at the site I contract
with. The reason it has been a problem for a whole week is that most of the
folks involved are victims of the current educational system's method of
teaching programming. They are, unlike myself, company employees who, if they
can't find some techno jargon to hide their screw-ups, will face spending the
rest of their careers in a somewhat lesser department without rapid
advancement, but not termination by any means. All of them are folks who
cannot get beyond "what de book say," and down to the real problem: Trouble
with paradigms.
Memo writing is a much more coveted skill than that. I will go one step beyond
praising your idea regarding the proper method of educating programmers, and
say that anyone who thinks that the real job of programming is to "write a
little code" should be shot. The second course the people will need (CS102 I
guess), is healthy skepticism. (Odd that you should have mentioned that in
your column also ... everything seems so clear at 3 a.m., ya know?)
There is an absolute need for more programmers, but not for more folks who are
seeking "good clean work with no heavy lifting." What is needed is a crop of
grunts who really love this crap.
Robert L. Hume
CompuServe: 71220, 1066


You Asked For It, You Got It


Dear DDJ,
I own an Amiga and was wondering if you could cover this computer in your
magazine. I still enjoy reading your magazine (even though it seems to be for
IBM only). If you started covering the Amiga, I would subscribe to your
magazine in a second!
Matt Childress
Holland, Michigan
DDJ: We hope the article by Chuck McManis fills the bill, Matt.


Just Say No!


Dear DDJ,
I enjoyed and appreciated your editorial in the March '89 issue of Dr. Dobb's.
I agree, professionals should be held responsible for the precise sculpting of
their product. However, I don't agree that education is not the solution to
the pirating problem.
Mr. Gillette's idea of embedding a serial number in the bios of each machine
is a good one and one I've wondered about myself. Why don't manufacturers do
that? The problem is that even if a serial number was available, any software
protection scheme would add another layer of complexity to the development and
support of a software product. For example, how do you handle hardware
upgrades where the serial number would have changed?
This complexity "buys" the developer very little protection against even a
moderately experienced computer user. All that needs to be done to subvert the
protection is copy the original disks before installing on the target machine.
As Nancy Reagan discovered in her work against drug abuse, jailing the
distributor isn't completely effective. As with drugs, the way to stop
software pirating is to make it unpopular to do. Maybe we can license their
slogan, "Software pirating.... Just say no!"
Tim Deardeuff
Provo, Utah


The Serial # Scheme


Dear DDJ,
I want to pass along a couple of thoughts I had regarding your March
[Paradigms] column. Writers should indeed be held to high standards. All too
much writing in the computer field is turgid, hackneyed, rambling -- you name
it! I too am a writer, and always strive (with varying degrees of success) to
produce work that is clear, concise, and literate. Knowing how difficult the
task can be, I am doubly appreciative when I find those qualities in the work
of others.
Copyright protection for software does remain a problem, but the serial number
technique that you mention has at least two serious flaws. For one, it makes
no provision for people or businesses that upgrade their computers while
keeping old software. Nor would it be acceptable for people like myself, who
use different computers in different locations. Each software package that I
purchase is installed on all of my computers; the manuals are schlepped back
and forth as needed. No one but me ever uses any of these machines. Surely
such an arrangement is within the spirit, if not the letter, of software
licensing agreements. The serial number scheme, by tying a program to a single
computer, would prevent me from using my software in this fashion.
Software copyrights need to be protected, but serial number keying is not the
way.
Peter G. Aitken
Durham, North Carolina


More on RLE


Dear DDJ,
Let's not overstate the usefulness of run length encoding. Robert Zigon is
correct in pointing out ("Run Length Encoding," February 1989) that RLE is
simple, fast, and elegant, and I have no doubt it works magic on Mr. Zigon's
frame-grabber data, a type of data for which it is ideally suited. But RLE
will not work on English text, and its usefulness in compacting object files
is next to nil.
Don't get me wrong; the article itself was good. The author did give the
impression, however, that RLE will compress a wide variety of file types by 20
percent or more. In fact, it will make most files grow, and any file (such as
an .EXE file) that contains all, or nearly all, 256 ASCII byte values will
need switchout codes and exception-handling to account for switchout codes,
over and above Zigon's simple routine.
Kas Thomas

Stamford, Conn.
Dear DDJ,
The article by Robert Zigon, (February 1989, page 126), caught my eye, but
unfortunately I cannot use it. The problem lies in the fact that the English
language has very few "double" letters in the words. I went through some
200,000 bytes of English prose to find out how many characters are the same
right after each other. The average is about 2 percent. Some other languages
like Dutch or Finnish have a much higher value, e.g. Dutch about 5 percent,
and Finnish also about 5 percent. In any event, even for those languages the
output file would about double, rather than be compressed.
Paul A. Elias
Fountain Hills, Arizona
Eds: RLE is applicable chiefly to the compression of data that has a high
degree of redundancy. RLE works great, for instance, with graphics data where
there are many consecutive bytes, all having the same value.


Errata


Please note the following required changes to the TSRUnit in "Creating TSRs
Using Turbo Pascal" by Ken L. Pottebaum (May and June, 1989).

 Line 87 Our26 = Our25+27;
 88 Our09 = Our26+27;

The INLINE code changes are to byte number 13 in OurIntr25 and OurIntr26,
which are interrupt intercept routines contained in PROCEDURE Asm. Change
OurIntr25 to:

 { 13} $68/>Our25+19/ { PUSH Our25+19 ;Clean up stack with- }
 $C2/$02/$00/ { RET 2 ;out changing flags. }

Make a similar change in OurIntr26 so that it becomes:

 { 13} $68/>Our26+19 { PUSH Our26+19 ;Clean up stack with- }
 $C2/$02/$00/ { RET 2 ;out changing flags. }

Also in June, Jim Mischel reported one change to his article "Writing AWK-like
Extensions to C." In Listing Three, page 94, the line that reads:

if (makepat (fs_DEFAULT, pat) = = NULL)

should read:

if (makepat (fs, pat) = = NULL)





























July, 1989
LINE-OF-BEST-FIT


Putting mouse and graphics primitives to work in the Presentation Manager's
protected-mode environment




William H. Murray and Chris H. Pappas


William H. Murray and Chris H. Pappas are professors of Computer Studies at
Broome Community College. Together they have written several books, including
Presentation Manager Graphics: An Introduction (Osborne/McGraw-Hill, 1989)
from which portions of this article were adapted. Bill and Chris can be
reached at Broome Community College, Binghamton, NY 13902.


With consistent device-independent programming environments such as Microsoft
Windows and OS/2 Presentation Manager, applications can be executed and
results ported to any installed hardware driver accordant with Microsoft's
specifications. This consistent environment frees you from the necessity of
also developing software drivers for zillions (or at least hundreds) of
printers, plotters, and graphics display) devices. As a matter of fact, the
latest version of Windows (Version 3.0) brings the Windows' environment in
much closer alignment with the OS/2 Presentation Manager (PM), making the task
of porting programs between real and protected modes easier.
The price you pay for entering into this consistent user interface is a rather
steep learning curve. In this article, we will concentrate on a PM program
that plots points on the screen and draws a line between them with a linear
least-squares-fit. A least-squares-fit works on the principle that data
points, collected on a chart, are often related mathematically. Because of
this relationship, the group of points takes on the appearance of a line or
curve. In the case of a linear mathematical relationship between point values,
a single line (called the line-of-best-fit) can be drawn between all points.
The data points for this program will be plotted by moving the mouse to a
specific region on the screen and clicking the left button. When all points
are entered, a click of the right mouse button will end data entry and draw a
line-of-best-fit. The equation for that line will be plotted at the bottom of
the screen in slope-intercept form. Among other things, this example
illustrates how the mouse and other graphics primitives can be utilized under
the protected-mode environment.
If you are unfamiliar with Windows or the PM philosophy, we strongly recommend
you read Herb Schildt's article, "A Presentation Manager Application
Template," which appeared in the March 1989 issued of DDJ. Herb describes one
of three Presentation Manager "foundations" -- the Cached-Presentation Space.
(The other two are the Micro-Presentation Space and the Normal-Presentation
Space.) Each type offers an increasingly greater array of available functions
and unfortunately larger code sizes. For this reason, it will be easiest for
you to learn simple PM programming with the Cached-Presentation Space, which
is what we implemented in this article. From there you can move to the other
two presentation spaces as projects demand.
To enter and execute the code you will need the PM (Version 1.1 or later)
operating on a 80386/80286 computer. Additionally, you'll need the Microsoft C
Compiler (Version 5.1 or later), the Microsoft OS/2 Software Developer's Kit,
and a Microsoft mouse. The program for this article was developed on an IBM
Model 80 with VGA display and 6 Mbytes of RAM.


The Sum of the Parts Equals the Whole


The program itself is called MFIT (for mouse fit). MFIT does not consist of a
single program file, but a group of files that are eventually linked together.
As a matter of fact, there are ten files that will be found on your disk after
a successful compilation: MFIT, MFIT.DEF, MFIT.EXE, MFIT.H, MFIT.ICO,
MFIT.OBJ, MFIT.PTR, MFIT.RC, MFIT.RES, and MFIT.C.
MFIT, shown in Listing One is used by the MAKE utility and is responsible for
the compilation of the C code by the C compiler, the assembling of the
resource code by the Resource Compiler, and the eventual linking of this code
with the icon and pointer information.
MFIT.DEF (Listing Two) is the definition file for the Presentation Manager.
Definition files are used by both Windows and the Presentation Manager. The
DEF file is responsible for establishing the heap and stack sizes, identifying
the operating mode (real or protected), and naming exports. For our example,
exports are the individual procedures used in the C program.
MFIT.H is a header file for unique identification numbers that will be used by
the resource file and the C program. Listing Three shows a list of #define
statements. Usually the #define instruction is followed by an ID name,
followed by a unique integer constant. The names and the constants are your
choice, but careful planning will sometimes eliminate later problems, such as
the insertion of additional ID numbers.
MFIT.ICO is a file containing the shape of the minimum icon. The minimum icon
is displayed at the bottom of the screen if that sizing option is chosen from
the main menu of the PM. The icon is drawn with the use of the Icon Editor,
which is part of the Microsoft Toolkit. (Because MFIT.ICO is not a text file
that can be listed, it is only available on an optional diskette or on line.)
MFIT.PTR is a file containing the shape of the screen pointer. The screen
pointer, by default, is an arrow. The user has the option of using the Icon
Editor to draw a new pointer. This pointer will be present in the application
window and can be used to make selections. In this case, a miniature line
chart is used as the pointer. When the left button is pushed (in the example
program) a small cross symbol will be plotted on the screen at the spot
pointed to by the pointer. Again, the file returned by the Icon Editor is not
a text file.
MFIT.RC is the resource file needed for describing the Menu, About Box, and
Dialog Box and is shown in Listing Four. The resource file is a can of worms
in itself. First, the About and Dialog Boxes are created with the Dialog Box
Editor provided with the Microsoft Toolkit. Once created, the information is
saved in a file with the extension .RES, which is not a text file. A text file
version can be selected. We used the .DLG extension for this option. Our
experience is that the text file is important for touching up the Dialog Box
information. The catch is that this .DLG file cannot be read by the Dialog Box
Editor. You will not have that problem with this example, because the
placement of the Dialog Box and all controls have already been worked out. You
will simply have to type the file MFIT.RC. The MAKE utility will compile that
file into a .RES file and link it to the final executable program. The About
Box is shown in Figure 1 and the Dialog Box in Figure 2.
MFIT.C is the C program proper. Listing Five contains all the necessary PM
overhead required to establish and communicate with an application program.
Specifically, two procedures or functions are required to process the Dialog
Box messages for the About and LineDiaProc Dialog Boxes. A final procedure,
GraphicProc, handles all other messages for the mouse position, buttons, and
graphics routines.


Power in the Loop


PM and Windows use the concept of a message loop or queue for receiving and
sending information throughout an application program. (Establishing this
queue is the job of the main( ) function as described in Schildt's article.)
Once the message queue is established, it is your responsibility to tap into
the queue with your requests. Our program will use several procedures or
functions. One procedure will help establish an About Box. In PM and Windows,
the About Box is used to transmit information regarding the program. This
usually consists of one or more of the following: the program name, a short
description of the program, the developer's name(s), and the copyright date.
The About Box is usually selected (as an item) from a program menu. Another
procedure will establish a Dialog Box. Dialog Boxes, also selected as items
from a user defined menu, establish a sophisticated means of data input. They
go beyond the simple buttons and check boxes of menus and allow the user to
enter string or numeric data from the keyboard. The Dialog Box in our program
will allow the entry of a title, x axis and y axis labels, and maximum integer
values for both the x and y axes. Finally, a third procedure will establish
the "core" of our current application. It is within this procedure that mouse
information is collected on coordinate positions and button status. It should
be pointed out that only integers can be passed through the message queue.
This does not mean that your program cannot use real numbers, it just means
that you cannot pass them through the queue.
There is a similar structure for intercepting messages in the LineDiaProc.
This Dialog Box is created if case IDM_INPUT is true during the processing of
the WM_COMMAND function in the procedure. This is the Dialog Box that queries
the user for labels and plotting ranges. In this case, information will only
be returned from the Dialog Box when the "Okay" button is selected. If the
"Cancel" button is selected, the Dialog Box will be destroyed, but no new
information will be intercepted. The functions WinQueryDlgItemText and
WinQueryDlgItemShort return the string information for the labels and the
integer information for the plotting ranges, respectively. In both cases, the
final outcome of selecting either push button is to process the message,
destroy the Dialog Box, and return to the application program.


Processing the GraphicProc Information


There are seven switch-case statements in this procedure for processing
message information: WM_CREATE, WM_BUTTON1DOWN, WM_MOUSEMOVE, WM_BUTTON2DOWN,
WM_SIZE, WM_COMMAND, and WM_PAINT. It is within these cases that data will be
collected and processed.
You have already seen that WM_COMMAND is responsible for establishing one of
the two Dialog Boxes when requested. WM_CREATE will replace the default arrow
pointer with the pointer created in the Icon Editor by calling Win
LoadPointer. This pointer tracks the current mouse position with WM_MOUSEMOVE.
In WM_MOUSEMOVE, the WinSetPointer function places the pointer in the window.
WM_SIZE processes the request for the mouse menu. The mouse menu allows the
selection of the two Dialog Box options mentioned earlier.
WM_BUTTON1DOWN and WM_BUTTON2DOWN report on the status of the two mouse
buttons. If the left button is pushed, a marker is placed on the screen at the
current mouse location. Additionally, if the left button is pushed, that
coordinate information is also placed in two global arrays: ptxorg[] and
ptyorg[]. This coordinate information will be used when plotting the
line-of-best-fit between the data points. Note that during WM_BUTTON1DOWN,
GpiSetMarker is used to draw a system marker symbol. GpiSetColor sets the
symbol drawing color to blue. Mouse message data is returned through the parm1
parameter, which is of type MPARAM. MPARAM is a pointer to this additional
message information. Up to 100 points can be collected in this manner. When
all data points have been entered, the user can push the right mouse button to
generate a line of best fit and the corresponding slope-intercept equation.
WM_BUTTON2DOWN uses the information returned to the global arrays, ptxorg[]
and ptyorg[], to draw the line. First, however, this information is processed
and scaled. When WM_BUTTON2DOWN is called, the slope and intercept of the line
are used to establish a beginning and ending point on the graph. When the
starting point is found, the GpiMove function places the cursor at that
position. The GpiLine function then draws a red line between that point and
the final point on the screen. Finally, the equation is drawn to the screen
with a series of GpiBox, GpiColor, GpiMove, and GpiCharString function calls.
Numerous lines-of-best-fit can be plotted on the same graph. Simply plot a
group of points and request a line, then plot another group and request
another line. The screen can be cleared by selecting the mouse data Dialog Box
and selecting either push button option.
WM_PAINT is responsible for setting the background color, drawing the
coordinate axes, plotting tic marks, setting the character mode for drawing
text to the screen, and plotting axis labels. Lines are drawn by moving the
cursor to the starting point of the line with the GpiMove function. The
coordinates (ptl.x and ptl.y) are of PM type POINTL. GpiLine specifies the
ending point. The axes and tic marks are drawn with just these two functions.
The GpiCbarStringAt function is used to draw the various labels.
GpiCharStringAt requires that a handle, coordinate position, length of string
parameter, and the actual label be provided for the function. If you examine
the listing, you will see how this is done. It should be noted that for the
default drawing mode, used by PM, the screen coordinates start at the
lower-left portion of the screen (0,0) and extend to the upper-right portion
of the screen. For the VGA monitor the upper-right coordinates are (639,479).


Calculating the Line-of-Best-Fit


When WM_BUTTON1DOWN has completed the process of filling the ptxorg[] and
ptyorg[] arrays, it will be possible to calculate the line-of-best-fit. Most
of the calculations take place in the GraphicProc procedure, immediately after
the data declarations. Two groups of arrays are used: ptxorg[] and ptyorg[]
along with ptxscaled[] and ptyscaled[]. The arrays ptxorg[] and ptyorg[] are
used to determine the slope-intercept data for the line that will be drawn on
the screen. These contain the screen coordinate values returned by the mouse.
However, because this information is biased by the starting, ending, and
scaling of the screen, it must be adjusted in order to provide "real world"
values for the line-of-best-fit equation that will also be printed to the
screen. Thus, ptxscaled[] and ptyscaled[] are arrays that hold the unbiased
values. The screen window for gathering points varies from (100,100) to
(500,400).
Two standard equations (shown in Figure 3) are used for the linear curve
fitting program. They are derived from the calculus and give the slope and
intercept of the final line -- the line-of-best-fit. More information on
general curve fitting can be found in Numerical Recipes in C (Press, Flannery,
Teukolsky, and Vetterling, Cambridge University Press, 1988.)
In these equations, n represents the number of data points with each x[i] and
y[i] representing the individual x and y values for each data point. If you
study the code under the comment "/* Calculate values for screen location */"
you will notice that two numerators, num1 and num2, are calculated along with
one denominator, deno2. If num1 is divided by deno2 the intercept will be
calculated. If num2 is divided by deno2, the slope of the line is calculated.
This is done under the call to WM_BUTTON2DOWN. A similar technique is used for
the array of scaled data points.



Running the Program


If you have entered the code and compiled the program give it a try. Make sure
you have a Microsoft mouse or equivalent connected and installed. Experiment
with several lines. Figure 4 shows one line-fitting example.
The program suffers from two weaknesses. First, it only allows you to plot
points in the first quadrant. Second, it will fail if a vertical line (an
infinite slope) is requested. Both problems are relatively easy to solve.
Plotting in all four quadrants can be achieved by remapping the axes and
adjusting the scaling factors. The vertical line problem is even easier --
simply use a case statement to capture the request and then use the GpiLine
function to draw a vertical line at the current x position.


Availability


All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063; or call 800-356-2002 (from inside
Calif.) or 800-533-4372 (from outside Calif.). Please specify the issue number
and format (MS-DOS, Macintosh, Kaypro).

_LINE-OF-BEST-FIT_
by William Murray and Chris Pappas



[LISTING ONE]


mfit17.obj : mfit17.c mfit17.h
CL /c /Lp /Zp /Od /G2sw /W2 mfit17.c

mfit17.res : mfit17.rc mfit17.ico mfit17.h mfit17.ptr
RC -r mfit17

mfit17.exe : mfit17.obj mfit17.def mfit17.res
link mfit17, /align:16, NUL, OS2, mfit17
rc mfit17.res



[LISTING TWO]

;MFIT17.DEF for C Compiling

NAME mfit17
EXPORTS About
EXPORTS LineDiaProc
EXPORTS GraphicProc
PROTMODE
HEAPSIZE 2048
STACKSIZE 9216




[LISTING THREE]

#define ID_RESOURCE 10
#define ID_ABOUT 15
#define ID_OK 20
#define ID_CANCEL 25
#define ID_INPUT 30

#define IDM_ABOUT 40
#define IDM_INPUT 45

#define IDM_LINEINPUT 55

#define DM_XAXIS 470
#define DM_YAXIS 475
#define DM_TITLE 480
#define DM_X1 485
#define DM_Y1 490




[LISTING FOUR]

#include <os2.h>
#include "mfit17.h"

POINTER ID_RESOURCE mfit17.ico
POINTER IDP_POINTER mfit17.ptr

MENU ID_RESOURCE
BEGIN
 SUBMENU "Mouse_Data",IDM_LINEINPUT
 BEGIN
 MENUITEM "About...", IDM_ABOUT
 MENUITEM "Program Data...", IDM_INPUT
 END
END

DLGTEMPLATE ID_ABOUT
BEGIN
 DIALOG "",ID_ABOUT,50,300,180,80,FS_DLGBORDER
 BEGIN
 CTEXT "Mouse Data Input Program",-1,2,60,176,10
 CTEXT "by William H. Murray & Chris H. Pappas",-1,2,45,
 176,10
 CONTROL "OK",ID_OK,75,10,32,14,WC_BUTTON,BS_PUSHBUTTON
 BS_DEFAULTWS_GROUPWS_TABSTOPWS_VISIBLE
 END
END

DLGTEMPLATE ID_INPUT LOADONCALL MOVEABLE DISCARDABLE
BEGIN
 DIALOG "Mouse Program Information",ID_INPUT,76,247,148,193,
 FS_NOBYTEALIGNFS_DLGBORDERWS_VISIBLE
 WS_CLIPSIBLINGSWS_SAVEBITS,FCF_TITLEBAR
 BEGIN
 CONTROL "Mouse Program Labels",257,2,92,145,72,WC_STATIC,
 SS_GROUPBOXWS_GROUPWS_VISIBLE
 CONTROL "X & Y Maximums",259,2,42,145,42,WC_STATIC,
 SS_GROUPBOXWS_GROUPWS_VISIBLE
 CONTROL "Enter Title:",260,5,133,51,8,WC_STATIC,SS_TEXT
 DT_LEFTDT_TOPWS_GROUPWS_VISIBLE
 CONTROL " x-axis:",261,5,118,54,8,WC_STATIC,SS_TEXT
 DT_LEFTDT_TOPWS_GROUPWS_VISIBLE
 CONTROL " y-axis:",262,5,102,50,8,WC_STATIC,SS_TEXT
 DT_LEFTDT_TOPWS_GROUPWS_VISIBLE
 CONTROL "Mouse Data Points",DM_TITLE,54,132,89,13,
 WC_ENTRYFIELD,ES_LEFTES_AUTOSCROLLES_MARGIN
 WS_TABSTOPWS_VISIBLE

 CONTROL "x - axis label",DM_XAXIS,54,117,89,13,
 WC_ENTRYFIELD,ES_LEFTES_AUTOSCROLLES_MARGIN
 WS_TABSTOPWS_VISIBLE
 CONTROL "y - axis label",DM_YAXIS,54,101,89,13,
 WC_ENTRYFIELD,ES_LEFTES_AUTOSCROLLES_MARGIN
 WS_TABSTOPWS_VISIBLE
 CONTROL " x:",273,25,57,20,8,WC_STATIC,SS_TEXTDT_LEFT
 DT_TOPWS_GROUPWS_VISIBLE
 CONTROL " y:",283,75,57,20,8,WC_STATIC,SS_TEXTDT_LEFT
 DT_TOPWS_GROUPWS_VISIBLE
 CONTROL "500",DM_X1,50,57,20,15,WC_ENTRYFIELD,ES_LEFT
 ES_AUTOSCROLLES_MARGINWS_TABSTOPWS_VISIBLE
 CONTROL "400",DM_Y1,100,57,20,15,WC_ENTRYFIELD,ES_LEFT
 ES_AUTOSCROLLES_MARGINWS_TABSTOPWS_VISIBLE
 CONTROL "OK",ID_OK,17,13,24,14,WC_BUTTON,BS_PUSHBUTTON
 WS_TABSTOPWS_VISIBLE
 CONTROL "Cancel",ID_CANCEL,101,13,34,14,WC_BUTTON,
 BS_PUSHBUTTONWS_TABSTOPWS_VISIBLE
 END
END




[LISTING FIVE]


/* Cached-PS Window Platform for PM Graphics */
/* (c) William H. Murray and Chris H. Pappas, 1989 */

#define INCL_PM

#include <os2.h>
#include <stddef.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include "mfit17.h"

MRESULT EXPENTRY About(HWND,USHORT,MPARAM,MPARAM);
MRESULT EXPENTRY LineDiaProc(HWND,USHORT,MPARAM,MPARAM);
MRESULT EXPENTRY GraphicProc(HWND,USHORT,MPARAM,MPARAM);

#define maxpts 100

HPS hps;
HWND hfrm,hcnt;
QMSG qmsg;
long ptxorg[maxpts];
long ptyorg[maxpts];
long xaxismax=500;
long yaxismax=400;
char title[80]="Mouse Data Points";
char xstring[80]="x - axis label";
char ystring[80]="y - axis label";
POINTL mousepos;

main ()
 {

 static CHAR pszClassName[]="FirstClass";
 HAB hab;
 HMQ hmq;
 HDC hdc;
 SIZEL page;
 ULONG flFrameFlags=FCF_SYSMENUFCF_TITLEBAR
 FCF_SIZEBORDERFCF_MINMAX
 FCF_ICONFCF_MENU;
 ULONG flFrameStyle=WS_VISIBLE;

 hab=WinInitialize(0);
 hmq=WinCreateMsgQueue(hab,0);
 WinRegisterClass(hab,pszClassName,GraphicProc,
 CS_SIZEREDRAW,0);
 hfrm=WinCreateStdWindow(HWND_DESKTOP,flFrameStyle,
 &flFrameFlags,pszClassName,
 "Cached-PS PM Graphics",
 0L,NULL,ID_RESOURCE,&hcnt);
 hdc=WinOpenWindowDC(hcnt);
 hps=GpiCreatePS(hab,hdc,&page,PU_PELSGPIA_ASSOC);
 WinSetWindowPos(hfrm,NULL,10,10,610,470,SWP_SIZE
 SWP_MOVESWP_SHOW);
 while (WinGetMsg(hab,&qmsg,NULL,0,0))
 WinDispatchMsg(hab,&qmsg);
 WinDestroyWindow(hfrm);
 WinDestroyMsgQueue(hmq);
 WinTerminate(hab);
 return 0;
 }

MRESULT EXPENTRY About(hwnd,messg,parm1,parm2)
 HWND hwnd;
 USHORT messg;
 MPARAM parm1,parm2;
 {
 switch (messg)
 {
 case WM_COMMAND:
 switch (LOUSHORT(parm1))
 {
 case ID_OK:
 WinDismissDlg(hwnd,TRUE);
 return 0;
 }
 break;
 }
 return (WinDefDlgProc(hwnd,messg,parm1,parm2));
 }

MRESULT EXPENTRY LineDiaProc(hwnd,messg,parm1,parm2)
 HWND hwnd;
 USHORT messg;
 MPARAM parm1;
 MPARAM parm2;
 {
 short X1;
 short Y1;
 switch (messg)
 {

 case WM_COMMAND:
 switch (LOUSHORT (parm1))
 {
 case ID_OK:
 WinQueryDlgItemText(hwnd,
 DM_TITLE,
 80,
 title);
 WinQueryDlgItemText(hwnd,
 DM_XAXIS,
 80,
 xstring);
 WinQueryDlgItemText(hwnd,
 DM_YAXIS,
 80,
 ystring);
 WinQueryDlgItemShort(hwnd,
 DM_X1,
 &X1,
 1);
 WinQueryDlgItemShort(hwnd,
 DM_Y1,
 &Y1,
 1);
 xaxismax=(long) X1;
 yaxismax=(long) Y1;
 WinDismissDlg(hwnd,TRUE);
 return 0;

 case ID_CANCEL:
 WinDismissDlg(hwnd,TRUE);
 return 0;
 }
 break;
 }
 return WinDefDlgProc (hwnd,messg,parm1,parm2);
 }

MRESULT EXPENTRY GraphicProc(hwnd,messg,parm1,parm2)
 HWND hwnd;
 USHORT messg;
 MPARAM parm1,parm2;
 {
 static LONG ColorDataInfo[]={CLR_NEUTRAL,
 RGB_BLACK,
 CLR_BACKGROUND,
 RGB_WHITE};
 static HPOINTER linePtr;
 static HWND hmenu;
 HPS hgpi;
 HDC hdc;
 POINTL ptl;
 GRADIENTL gradl;
 RECTL rcl;
 int i,alength,blength;
 int lenxstring,lenystring,lentitle,
 lenmaxxlabel,lenmaxylabel;
 long ptxmax,ptymax,sumx1,sumy1,sumxy1,
 sumxsqr1,deno1,numa1,numb1,

 sumx2,sumy2,sumxy2,sumxsqr2,
 deno2,numa2,numb2;
 long ptxscaled[maxpts],ptyscaled[maxpts],maxx;
 char maxxlabel[4],maxylabel[4],
 astring[10],bstring[10];
 double a1,b1,a2,b2,y;
 static int j=0;
 static int npts=0;

 /* Length of various strings */
 lentitle=strlen(title);
 lenxstring=strlen(xstring);
 lenystring=strlen(ystring);

 /* Convert maximum x value to a string */
 itoa((int)xaxismax,maxxlabel,10);
 lenmaxxlabel=strlen(maxxlabel);

 /* Convert maximum y value to a string */
 itoa((int)yaxismax,maxylabel,10);
 lenmaxylabel=strlen(maxylabel);

 /* Scale all x values in original array. */
 for (i=0;i<npts;i++)
 ptxscaled[i]=((ptxorg[i]-100)*xaxismax/400);

 /* Scale all y values in original array. */
 for (i=0;i<npts;i++)
 ptyscaled[i]=((ptyorg[i]-100)*yaxismax/300);

 /* Calculate values for screen equation */
 sumxsqr1=0;
 sumx1=0;
 sumy1=0;
 sumxy1=0;
 for (i=0;i<npts;i++)
 {
 sumxsqr1+=ptxscaled[i]*ptxscaled[i];
 sumx1+=ptxscaled[i];
 sumy1+=ptyscaled[i];
 sumxy1+=ptxscaled[i]*ptyscaled[i];
 }
 deno1=(npts*sumxsqr1)-(sumx1*sumx1);
 numa1=(sumxsqr1*sumy1)-(sumx1*sumxy1);
 numb1=(npts*sumxy1)-(sumx1*sumy1);

 /* Calculate values for drawing line on screen */
 sumxsqr2=0;
 sumx2=0;
 sumy2=0;
 sumxy2=0;
 for (i=0;i<npts;i++)
 {
 sumxsqr2+=ptxorg[i]*ptxorg[i];
 sumx2+=ptxorg[i];
 sumy2+=ptyorg[i];
 sumxy2+=ptxorg[i]*ptyorg[i];
 }
 deno2=(npts*sumxsqr2)-(sumx2*sumx2);

 numa2=(sumxsqr2*sumy2)-(sumx2*sumxy2);
 numb2=(npts*sumxy2)-(sumx2*sumy2);

 switch (messg)
 {

 case WM_CREATE:
 linePtr=WinLoadPointer(HWND_DESKTOP,
 NULL,IDP_POINTER);
 return 0;

 case WM_BUTTON1DOWN:
 GpiSetMarker(hps,MARKSYM_PLUS);
 GpiSetColor(hps,CLR_BLUE);
 mousepos.x=(LONG) LOUSHORT(parm1);
 mousepos.y=(LONG) HIUSHORT(parm1);
 if(mousepos.x>99 & mousepos.x<500 &
 mousepos.y>99 & mousepos.y<400)
 {
 ptxorg[j]=mousepos.x;
 ptyorg[j]=mousepos.y;
 j++;
 npts=j;
 GpiMarker(hps,&mousepos);
 }
 return TRUE;

 case WM_MOUSEMOVE:
 WinSetPointer(HWND_DESKTOP,linePtr);
 return 0;

 case WM_BUTTON2DOWN:
 GpiSetColor(hps,CLR_RED);
 j=0; /*reset pointer */
 maxx=0;
 /* Slope and intercept for org. and scaled */
 a1=(double) numa1/deno1;
 b1=(double) numb1/deno1;
 a2=(double) numa2/deno2;
 b2=(double) numb2/deno2;

 /* Starting point for line of best fit */
 for (i=99;i<510;i++)
 {
 y=a2+(b2*i);
 if(y>99.0 && y<410.0)
 {
 ptl.x=i;
 ptl.y=(long) y;
 break;
 }
 }
 GpiMove(hps,&ptl);

 /* Ending point for line of best fit */
 for (i=510;i>99;i--)
 {
 y=a2+(b2*i);
 if(y>99.0 && y<410.0)

 {
 ptl.x=i;
 ptl.y=(long) y;
 break;
 }
 }
 GpiLine(hps,&ptl);

 /* Draw the equation to the screen. */
 GpiSetColor(hps,CLR_WHITE);
 ptl.x=100;
 ptl.y=80;
 GpiMove(hps,&ptl);
 ptl.x=500;
 ptl.y=55;
 GpiBox(hps,DRO_FILL,&ptl,0L,0L);
 GpiSetColor(hps,CLR_RED);

 gcvt(a1,7,astring);
 alength=strlen(astring);
 gcvt(b1,7,bstring);
 blength=strlen(bstring);
 ptl.x=300-(LONG)(alength+blength+8)*4;
 ptl.y=62;
 GpiMove(hps,&ptl);
 GpiCharString(hps,4L,"y = ");
 GpiCharString(hps,(long) alength,astring);
 GpiCharString(hps,3L," + ");
 GpiCharString(hps,(long) blength,bstring);
 GpiCharString(hps,3L,"(x)");
 return TRUE;

 case WM_SIZE:
 if (hmenu==NULL)
 hmenu=WinWindowFromID(WinQueryWindow(hwnd,
 QW_PARENT,FALSE),FID_MENU);
 return 0;

 case WM_COMMAND:
 {
 switch (COMMANDMSG(&messg)->cmd)
 {
 case IDM_ABOUT:
 if (WinDlgBox(HWND_DESKTOP,hwnd,About,
 NULL,ID_ABOUT,NULL))
 WinInvalidateRect(hwnd,NULL,FALSE);
 return 0;

 case IDM_INPUT:
 if (WinDlgBox(HWND_DESKTOP,hwnd,LineDiaProc,
 NULL,ID_INPUT,NULL))
 WinInvalidateRect(hwnd,NULL,FALSE);
 return 0;

 return 0;
 }
 }
 break;


 case WM_PAINT:
 hgpi=WinBeginPaint(hwnd,NULL,NULL);
 GpiCreateLogColorTable(hgpi,LCOL_RESET,
 LCOLF_INDRGB,0L,4L,ColorDataInfo);
 GpiErase(hgpi);

/*--------- your routines below ----------*/

 GpiSetColor(hgpi,CLR_BLACK);

 /* Draw X and Y coordinate axis */
 ptl.x=99;
 ptl.y=410;
 GpiMove(hgpi,&ptl);
 ptl.y=99;
 GpiLine(hgpi,&ptl);
 ptl.x=510;
 GpiLine(hgpi,&ptl);

 /* Draw Y axis tic marks */
 ptl.y=130;
 for (i=0;i<10;i++)
 {
 ptl.x=95;
 GpiMove(hgpi,&ptl);
 ptl.x=99;
 GpiLine(hgpi,&ptl);
 ptl.y+=30;
 }

 /* Draw X axis tic marks */
 ptl.x=140;
 for (i=0;i<10;i++)
 {
 ptl.y=99;
 GpiMove(hgpi,&ptl);
 ptl.y=95;
 GpiLine(hgpi,&ptl);
 ptl.x+=40;
 }

 GpiSetCharMode(hgpi,CM_MODE3);

 /* Center and print line chart title */
 ptl.y=410;
 ptl.x=300-((LONG) (lentitle/2)*6);
 GpiCharStringAt(hgpi,&ptl,(LONG) lentitle,title);

 /* Center and print horizontal axis label */
 ptl.y=40;
 ptl.x=300-((LONG) (lenxstring/2)*6);
 GpiCharStringAt(hgpi,&ptl,(LONG) lenxstring,
 xstring);

 /* Print horizontal axis maximum value */
 ptl.y=82;
 ptl.x=490;
 GpiCharStringAt(hgpi,&ptl,(long) lenmaxxlabel,
 maxxlabel);


 /* Print vertical axis maximum value */
 ptl.y=400;
 ptl.x=70;
 GpiCharStringAt(hgpi,&ptl,(long) lenmaxylabel,
 maxylabel);

 /* Center and print vertical axis label */
 ptl.y=240-((LONG) (lenystring/2)*6);
 ptl.x=70;
 gradl.x=0;
 gradl.y=90;
 GpiSetCharAngle(hgpi,&gradl);
 GpiCharStringAt(hgpi,&ptl,(LONG) lenystring,
 ystring);

 /*--------- your routines above ----------*/

 WinEndPaint(hgpi);
 break;

 default:
 return WinDefWindowProc(hwnd,messg,parm1,parm2);
 }
 return 0;
 }

/LT


































July, 1989
AN ICON EDITOR


Roll your own icons using a mouse and this program




Keith Weiskamp and Loren Heiny


Keith and Loren are the authors of Power Graphics Using Turbo C published by
John Wiley & Sons. They can be reached at 3120 E. Paradise Ln., Suite 12,
Phoenix, AZ 85032 or through CompuServe at 72561, 1536.


George Orwell, the great author and futurist, was born forty years too early
to be a hacker, but he had a lot to say about languages. In an essay he
published in 1946 entitled "Politics and the English Language," he introduced
his famous principle of optimization: "Never use a long word where a short one
will do." Unfortunately, George's crystal ball wasn't fine-tuned enough to
predict the Macintosh, Windows, and Presentation Manager explosion. If it
were, he could have taken credit for the rule that all serious user interface
designers have posted by their machines: "Never use a word where a picture
will do."
Indeed, these little pictures, commonly called icons, have invaded our lives.
They're on the streets, in the courts, and now they're on our screens. And if
you've ever used a visually oriented application, such as a painting program,
you probably have already discovered that clicking on a picture of a line in
order to draw a line is much easier than having to type a command that you
can't remember.
In this article, we'll show how to develop an icon editor in Turbo C called
"ICONED." With this editor you can use a mouse to create, save, and edit
icons. The icons that you design can be read from files and then displayed in
your applications to enhance your user interfaces. Our application has two
parts: The main program, which contains the code for the icon editor (see
Listing One), and a set of tools that are required to support the mouse (see
Listings Two and Three).


Supporting Graphics Devices


In the past, the process of writing a graphics-based program for the PC was
about as frustrating as cleaning a swimming pool in a dust storm. As soon as
you got your eye-catching program up and running, another graphics standard
would be introduced. Fortunately, compiler vendors such as Microsoft and
Borland now provide device-independent graphics tools with their language
products. This means that you can write a program that works with all of the
major graphics adapters including the CGA, EGA, and VGA.
For this article, we decided to use Turbo C because of its flexible BGI
(Borland Graphics Interface). (After making some minor modifications to the
code, you could also compile it with Microsoft's QuickC.) The BGI tools
provided with Turbo C (and with all of the other Turbo languages) is a useful
graphics package because it lets you use a variety of PC graphics hardware
devices, from CGA to VGA, to perform high-and low-level graphics. More than 70
graphics routines are provided for performing tasks that range from detecting
graphics adapters to drawing and filling polygons.


Getting to Work


The BGI provides all of the high-level graphics support tools that are needed
in order to create the icon editor. To support the icon editor, a graphics
adapter supported by the BGI plus a Microsoft-compatible mouse are required.
Mouse support is handled by some of the functions that we present in this
article. These functions initialize the mouse, read the mouse position, and
hide and display the mouse cursor. (We won't spend a lot of time discussing
how the mouse works. For more information about the mouse, see the references
at the end of this article.)
A sample of the type of icons generated by the icon program is shown in Figure
1. The functions used in the main program are listed and briefly described in
Table 1. Essentially, these functions fall into two categories: "graphics
screen processing" and "file processing." The graphics screen processing
functions, such as init_bigbit( ) and init_graphics( ), control how icon
images are displayed. The file processing functions, such as read_icon( ) and
save_icon( ), are used to read and save icon images. The mouse support
functions are listed in Table 2.
Table 1: Functions used in the icon editor

 Function Name Description
 ------------------------------------------------------------------------

 draw_enlarged_icon() Draws an enlarged view of an icon
 init_bigbit() Initializes a single big icon bit
 init_graphics() Initializes the graphics hardware
 read_icon() Reads an icon file
 save_icon() Saves an icon image to a file
 show_icon() Displays the icon pattern stored in the icon array
 toggle_bigbit() Turns a big bit on or off
 toggle_icons_bit() Turns a standard size icon bit on or off

Table 2: Mouse support functions

 Function Name Description
 -----------------------------------------------------------------

 mouse() The mouse interface function
 initmouse() Initializes the mouse
 showmouse() Displays the mouse cursor
 hidemouse() Hides the mouse cursor
 waitforinput() Reads a mouse or keyboard input
 testbutton() Tests the state of one of the mouse buttons
 getmousecoords() Obtains the coordinates of the mouse cursor




The Icon Editor in Action


The icon editor is shown in Figure 2. Notice that two primary regions are
displayed. The left side of the screen shows an enlarged representation of the
icon that is being edited. The editing process is performed within this
window. The right side of the screen displays the icon pattern in the
pattern's actual size while the pattern is being edited. The current state of
the icon is updated after every mouse action.
To edit an icon, simply click the mouse on any of the big icon bits in the
enlarged icon window. Each time you click the mouse on a big pixel, the
pixel's state is toggled either ON or OFF. To exit the editing process and
save or discard your work, select the Esc key. If the edited icon is saved,
then it can be read in later and edited again.


Representing Icons


Icons can be represented in many different ways. Our goal is to develop icons
that look good when displayed with the different graphics modes supported by
the BGI. An icon drawn in one mode may appear quite different in another mode
because of that mode's different screen resolution and aspect ratio.
We've standardized the size of our icons at 16-by-16 pixels. This size works
well in most cases (at least in the standard graphics modes supported by the
CGA, EGA, and VGA adapters). Internally, an icon pattern is represented in the
editor as a two-dimensional array.
Each location in the icon array corresponds to a pixel setting in the icon
pattern. Because our icons are displayed in a single color, they can be easily
represented by storing the values 1 or 0 in each location of the icon array.
If the stored value is 1, then the corresponding icon pixel is displayed. If
the stored value is 0, then the pixel is set to the current background color.
(If you wanted to store color attributes, you could modify the icon array data
structure.)


Saving Icons


As each icon is created, it is saved in its own file. This file consists of a
header and a body. The header is a single line that specifies the icon's width
and height. Because the size of each newly created icon is 16-by-16 pixels,
each icon's header will always start with those two numbers. (This feature
lets you easily modify the program to work with icons of different sizes.) The
remainder of the file contains the icon pattern, which is organized in a row
and column format. A 16-by-16 icon has 16 numbers per line where each pixel of
the icon image is represented by a 0 or a 1. Figure 3 shows a sample icon and
Figure 4 shows the file that stores the icon shown in Figure 3. Compare the
two figures and note the one-to-one correspondence between each pixel set in
the icon and each value of 1 in the file.
Figure 4: The contents of the file that make up the icon shown in Figure 3.

 16 16
 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0
 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0
 0 0 0 0 0 0 0 1 0 0 0 0 1 0 0 0
 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0
 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0
 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0
 0 0 0 1 1 1 0 0 0 1 0 0 0 0 0 0
 0 0 1 0 0 0 1 0 1 0 0 0 0 0 0 0
 0 0 1 0 1 0 1 1 0 0 0 0 0 0 0 0
 0 0 1 1 0 0 1 0 0 0 1 1 0 0 0 0
 0 0 1 1 1 1 0 0 0 1 0 0 1 0 0 0
 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0
 0 0 0 0 1 0 0 1 0 0 0 0 0 0 1 0
 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

The function used to save an icon image is save_icon(). To simplify user
input, save_icon() is designed to be invoked while the program is in text
mode. The process of saving an icon involves several steps. First, save_icon()
prompts you to enter the name of the file where the icon is to be stored.
Next, fopen() is called to open and initialize the file. Finally, a nested for
loop writes the icon pattern stored in the array icon to the file and calls
fclose() to close the file.


Reading an Icon File


The process of reading an icon file into the icon editor is similar to the
process of writing a file. An icon file is read into the icon editor by the
function read_icon(), which is designed to run in text mode in order to
simplify text input.
read_icon() first initializes the icon pattern so that the pattern contains
all zeros. (This step ensures that the icon pattern starts as a "blank"
pattern.) read_icon() then asks if you wish to edit an existing icon file. If
your response is affirmative (you type any character except the letter N),
read_icon() requests the name of the icon file. Remember that during this
process, the data entry operations are performed in text mode.
Once the filename is specified, read_icon() attempts to open the corresponding
file. If that operation succeeds, then the first line of the file, which
contains the header, is read. If the first line consists of two values that
are equivalent to ICONWIDTH and ICONHEIGHT, read_icon() continues. If any
other numbers are found, then the file is invalid and read_icon() terminates
by returning 0.
The icon pattern is read from the file into the array icon by two for loops
(like those used in save_icon(), which write the icon data to a file. In this
case, the forloops use fscanf() to read the icon pattern from the file. Once
the icon pattern has been read into the array icon, the pattern can be
displayed.


The Core of the Icon Editor


Now that we've discussed the process of representing icons and saving them in
external files, let's jump in and explore the main program. The first ten
statements in ICONED (Listing One) initialize the various components of the
icon editor. The process of initialization consists of reading an icon file,
initializing the graphics mode, initializing the mouse, generating the
enlarged icon pattern, and displaying the initial state of the icon pattern.
The functions called are:
 read_icon();

 init_graphics();
 initmouse();
 draw_enlarged_icon();
 show_icon();
These statements must be called in the order shown above. In particular,
read_icon() should be kept in text mode (before graphics initialization
occurs) in order to simplify user interaction. In addition, the mouse
initialization process must always occur after the graphics initialization
process is completed.
The next five statements print a series of banners to the screen by calling
the outtextxy() function. Notice that a pair of hidemouse() and showmouse()
function calls surround the screen output statements. These calls ensure that
the mouse cursor is turned off while the screen is being updated with the
displayed messages.
After the screen is initialized, the icon editor is ready for business. Two
types of input are accepted while an icon is being edited:
1. A left mouse button, which toggles the icon pixel that is being pointed to
by the mouse cursor;
2. The Esc key, which terminates the program. The while loop in the function
main() reads and processes the user inputs as shown below:
 while ((c=waitforinput(LEFT_BUTTON))!= ESC) {

 if (c < 0) {
 getmousecoords(&x, &y);
 toggle_bigbit(x, y);
 }
 } while (!done);
The loop uses the general purpose mouse input function waitforinput(), (see
Listing Two). This function returns a negative value if the specified button
(in this case, the left mouse button) is pressed. If a key is pressed, then
waitforprint() returns the value of that key. The program will continue until
the Esc key is pressed. If the left mouse button is pressed, the body of the
while loop is entered. At this point, getmousecoords() retrieves the current
location of the mouse cursor and passes it to the function toggle-bigbit(),
which changes the setting of the icon pixel that the mouse is pointing to.


Creating an Enlarged Icon


The function draw_enlarged_icon(), which is called from main(), displays an
editing grid on the left side of the screen. The grid consists of 17
horizontal and vertical dotted lines that designate a 16-by-16 grid for the
icon pattern. Before anything is written to the screen, the mouse cursor is
turned off by a call to hidemouse(). Later, the mouse cursor is restored by a
call to showmouse(). This technique eliminates the problem of overwriting the
mouse cursor.
Examine the code in draw_enlarged_icon() and note that this function relies
upon numerous macro constants. A list of these macro constants, along with a
description of their meanings, is shown in Table 3. You may want to refer to
this table as you work your way through draw_enlarged_icon().
Table 3: Macro constants used in drawing the enlarged icon pattern

 Macro Constant Description
 ---------------------------------------------------------------------

 BIGICONLEFT The column where the big icon pattern begins
 BIGICONTOP The top row of the big icon pattern
 BIGBITSIZE Size of the big pixels in the enlarged icon pattern
 ICONWIDTH Width of an icon
 ICONHEIGHT Height of an icon
 DOTTED_LINE From graphics.h; specifies the line type
 NORM_WIDTH From graphics.h; pixel width of lines

The last line in draw_enlarged_icon() is a call to init_bigbit(), which
creates an image of an enlarged icon pixel. To toggle the icon pixels in the
enlarged icon pattern, we'll exclusive OR the image created by init_bigbit()
to the rectangular regions within the enlarged icon pattern.
A nested for loop is used to create the image of one of the enlarged bits at
the beginning of init_bigbit(), as shown below:
 for (j=bby+1; j<=bby+BIGBITSIZE; j++)
 for (i=bbx+1; i<=bbx+2*BIG BITSIZE; i++))
 putpixel(i,j,getmaxcolor();
These two loops paint a block of pixels in the top-left corner of the enlarged
icon pattern. This block is the size of BIGBITSIZE. Notice that the width of
the enlarged icon is drawn to be twice the width of BIGBITSIZE. This step
adjusts the icon image to the aspect ratio of the graphics screen.
After the enlarged icon pixel is created, it is copied into the array bigbit.
(Once the pixel pattern has been stored in bigbit, an image of the pattern can
be exclusive-ORed later during the editing process in order to toggle one of
the enlarged icon pixels.)
Before the enlarged icon pixels' image can be copied into bigbit, space for
the image must be allocated by a call to malloc(). After the space is
allocated, getimage() is called to copy the image of the big pixel. Next,
putimage() is invoked with the XOR_PUT replacement in order to remove the big
pixel from the top-left corner of the enlarged icon pixel:
 putimage(bbx+1, bby+1, bigbit, XOR_PUT);
Finally, the mouse cursor is restored by a call to showmouse(), and then
init_bigbit() terminates.


Displaying the Original Icon


The next step in the screen initialization process is to display the icon
pattern that was read in at the beginning of the program. (Remember, if an
icon file is not read in, then read_icon() initializes the icon array to
zeros.) The function show_icon() displays the current state of the icon
pattern. This function consists of two nested for loops that sequence through
the array icon. For each byte location that stores the value 1, a
corresponding big bit is toggled in the enlarged icon, and the small icon
pattern is updated. The process of setting one of the big icon pixels is a
matter of exclusive-ORing the image of the big icon pixel (as discussed
earlier) at the appropriate locations. The putimage() function, which is
located within the inner for loop, performs this step.
The call to toggle_icons_bit() is required in order to turn on the appropriate
pixel in the small icon pattern. This function accepts the index of a pixel in
the icon array as an argument, and checks the corresponding pixel by testing
whether that pixel is equal to the background color. Depending upon the
pixel's current value, the small icon's pixels are toggled. The small icon is
displayed at the column indicated by ICONLEFT. The top row of the small icon
coincides with BIGICONTOP. Notice that although the icon is represented as a
16-by-16 pattern, the small icon is displayed in a 32-by-16 format. In other
words, each column of pixels is displayed twice --and that's why the
multiplication factor of 2 in the x coordinate calculation is required.


Toggling an Icon Pixel



The process of toggling a pixel in an icon that is being edited involves three
steps:
1. The pixel's value in the icon array must be changed;
2. The big pixel image in the enlarged icon pattern must be toggled; and
3. The icon's pixel in the small icon pattern must be updated.
Each of these actions is set in motion by invoking toggle_bigbit(). This
function takes two arguments that correspond to the screen coordinates of the
enlarged icon bit that is to be changed. These screen coordinates are
determined from the location of the mouse cursor at the time of the button
press. The mouse coordinates are determined by our mouse routine,
getmousecoords().
The bulk of toggle_bigbit() is involved in deciding which icon bit (if any)
should be toggled. This decision is made by the two for loops that sequence
through the locations of the big icon pattern and test whether the coordinates
passed to toggle_bigbit() fall within any of the rows or columns in the big
icon pattern. If the passed coordinates fall within the big icon pattern,
putimage() is used to exclusive-OR an image of the bigbit image that was made
earlier over the current location in the icon pattern. This step toggles the
icon pixel in the large icon pattern. Because the corresponding bit in the
small icon pattern must be changed, the routine is designed so that the line
number and the column number determined by the for loops will correspond to
the indices that can access the same bit in the icon array. These values,
which are stored in the variables i and j, are passed to another function,
toggle_icons_bit(). This step changes the icon array value and updates the
small icon on the screen.


Exiting the Icon Editor


The while loop (described earlier) that controls user interaction continues to
loop until the Esc key is pressed. Once the Esc key is pressed, the mouse
cursor is disabled, the screen is returned to text mode, and the user is
prompted to save the icon currently in the icon editor. If the user responds
with anything other than the letter N, the program enters the save_icon()
function and prompts the user for the name of the file in which the icon will
be stored.


Putting it Together


To create an executable version of the icon editor, compile the files iconed.c
and mouse.c (provided in Listings One and Two) and then link them with Turbo
C's graphics library file, graphics.lib. To use the icons that you create with
the icon editor in your application programs, include the read_icon() and
show_icon() functions. You'll also need to write a routine that allows the
user to select icons by clicking on them with the mouse.


Enhancements


There are a number of enhancements that you can make to the icon editor. For
instance, we have designed the icon editor to work with fixed-size icons.
Because of the flexibility of the program, you can change the default icon
size or include an option that lets you design and store icons of different
sizes.
We've already pointed out some of the mouse functions used in the icon editor,
such as hidemouse(), showmouse(), getmousecoords(), and waitforinput(). All of
these functions directly or indirectly invoke the main mouse interface
function, mouse(). This function communicates with the mouse software drivers
by invoking interrupt 33h with Turbo C's int86() function. If you don't have a
mouse installed, the main program will terminate after calling initmouse().
Alternatively, you could modify the mouse support routines so that they
support the keyboard if a mouse is not available.
These suggestions represent the tip of the iceberg. The icon editor can serve
as the first of a series of tools that will help you design more flexible user
interfaces. Because icons are created and stored independent of the
application program, you can redesign an interface without the need to make
major changes to the program.


References


Kent Porter, "Mouse Mysteries, Part I: Text," TURBO TECHNIX 1:4, (May/June
1988).
Kent Porter, "Mouse Mysteries, Part II: Graphics," TURBO TECHNIX 1:5,
(July/August 1988).


_AN ICON EDITOR_
by Keith Weiskamp and Loren Heiny


[LISTING ONE]

/* iconed.c -- a special purpose icon editor.
 *
 * This program allows you to interactively edit icons that
 * can be used in a graphics program. You can create an icon,
 * edit an existing one, or save an icon pattern to a file. The
 * program requires a mouse. The icon files produced are of the
 * form:
 * ICONWIDTH ICONHEIGHT
 * one row of icon pattern
 * next row of icon pattern
 * . . .
 * last row of icon pattern
 *
 */

#include <stdio.h>
#include <stdlib.h>
#include <conio.h>

#include <graphics.h>
#include <stdarg.h>
#include <alloc.h>
#include "mouse.h" /* mouse and keyboard routines */

#define BIGICONLEFT 20 /* left side of the big icon pattern */
#define BIGICONTOP 50 /* top side of the big icon pattern */
#define BIGBITSIZE 8 /* big bits are 8 pixels in size */
#define ICONWIDTH 16 /* an icon is a 16x16 pattern */
#define ICONHEIGHT 16
#define ICONLEFT 400 /* small icon pattern located here */
#define ESC 27 /* the value of the ESC key */

/* Here are the functions used in iconed.c: */

void draw_enlarged_icon(void);
void toggle_bigbit(int x, int y);
void toggle_icons_bit(int x, int y);
void init_bigbit(void);
void toggle_cursor(int x, int y);
void save_icon(void);
int read_icon(void);
void init_graphics(void);
void show_icon(void);

/* The global variables */

void *bigbit; /* points to image of a big bit */
 /* holds icon pattern */
unsigned char icon[ICONHEIGHT][ICONWIDTH];

void main()
{
 int x, y;
 int c;

 read_icon(); /* read in an icon file */
 init_graphics();
 /* initialize mouse */
 if (!initmouse()) { /* must have mouse */
 restorecrtmode();
 printf("\nMouse not installed");
 printf("\nQuitting the Icon Editor");
 exit(1);
 }
 draw_enlarged_icon(); /* an empty big icon pattern */
 show_icon(); /* Draw the icon */

 hidemouse(); /* mouse cursor must be turned off */
 /* before writing to screen */
 outtextxy(BIGICONLEFT, 10, "Press ESC when finished ...");
 outtextxy(BIGICONLEFT, BIGICONTOP-20, "Enlarged Icon");
 outtextxy(ICONLEFT, BIGICONTOP-20, "Actual Icon");
 showmouse(); /* redisplay mouse cursor */

 /* get input from mouse/keyboard */
 while ((c=waitforinput(LEFT_BUTTON)) != ESC) {
 if (c < 0) { /* if true mouse button pressed */
 getmousecoords(&x, &y); /* get current position */

 toggle_bigbit(x, y); /* toggle big bit */
 }
 }

 hidemouse(); /* turn mouse off and then get out */
 closegraph(); /* of graphics mode */
 printf("Do you want to save this icon to a file? (y) ");
 if (getch() != 'n')
 save_icon();
}


void draw_enlarged_icon(void)
{
/*
 * This function draws an enlarged view of the icon pattern.
 * You can click a big bit in this pattern to toggle the
 * corresponding icons on and off in the actual icon. The
 * icon is drawn at BIGICONLEFT, BIGICONTOP, to right, bottom.
 */
 int i;
 int right,bottom;

 setlinestyle(DOTTED_LINE, 0, NORM_WIDTH);
 right = 2 * (BIGICONLEFT + (ICONWIDTH-1) *
 (BIGBITSIZE + NORM_WIDTH));
 bottom = BIGICONTOP + ICONHEIGHT * (BIGBITSIZE + NORM_WIDTH);

 hidemouse(); /* draw vertical and horizonatal dashed */
 for (i=0; i<=ICONHEIGHT; i++)
 line(BIGICONLEFT, BIGICONTOP+i*(BIGBITSIZE+NORM_WIDTH),
 right, BIGICONTOP+i*(BIGBITSIZE+NORM_WIDTH));
 for (i=0; i<=ICONWIDTH; i++)
 line(BIGICONLEFT+2*(i*(BIGBITSIZE+NORM_WIDTH)), BIGICONTOP,
 BIGICONLEFT+2*(i*(BIGBITSIZE+NORM_WIDTH)), bottom);
 showmouse();
 init_bigbit(); /* create the big bit image */
}


void init_bigbit(void)
{
/* This function creates the image of a single big bit. This
 * image is used to toggle the big bits whenever the user
 * clicks on the big icon pattern.
 */
 int bbx, bby;
 int i,j;

 bbx = BIGICONLEFT;
 bby = BIGICONTOP; /* corner of the big icon */

 hidemouse(); /* hide the mouse before drawing */
 for (j=bby+1; j<=bby+BIGBITSIZE; j++) {
 for (i=bbx+1; i<=bbx+2*BIGBITSIZE; i++) {
 putpixel(i,j,getmaxcolor());
 }
 }


 /* Set aside memory for the big bit image and then use
 getimage() to capture its image. */

 bigbit = malloc(imagesize(bbx,bby,bbx+2*BIGBITSIZE,
 bby+BIGBITSIZE));
 getimage(bbx+1,bby+1,bbx+2*BIGBITSIZE,bby+BIGBITSIZE,
 bigbit);

 /* Erase the big bit by exclusive ORing it with itself */

 putimage(bbx+1, bby+1, bigbit, XOR_PUT);
 showmouse(); /* turn the mouse back on */
}


void toggle_bigbit(int x, int y)
{
/*
 * This function toggles a big bit and the corresponding pixel
 * in the icon pattern. The x and y coordinates specify the
 * mouse position.
 */
 int i, j;
 int line1, line2, col1, col2;

 for (j=0; j<ICONHEIGHT; j++) {
 line1 = BIGICONTOP+j*(BIGBITSIZE+NORM_WIDTH);
 line2 = BIGICONTOP+(j+1)*(BIGBITSIZE+NORM_WIDTH);
 if (line1 <= y && y < line2) {
 for (i=0; i<ICONWIDTH; i++) {
 col1 = BIGICONLEFT+2*(i*(BIGBITSIZE+NORM_WIDTH));
 col2 = BIGICONLEFT+2*((i+1)*
 (BIGBITSIZE+NORM_WIDTH));
 if (col1 <= x && x < col2) {
 hidemouse();
 putimage(col1+1,line1+1,bigbit,XOR_PUT);
 showmouse();
 toggle_icons_bit(i, j);
 return;
 }
 }
 }
 }
}


void toggle_icons_bit(int x, int y)
{
/*
 * This function toggles a single pixel in the icon pattern.
 * The pixel's color and value is changed in the icon array.
 * Arguments x and y are between 0 and ICONWIDTH or ICONHEIGHT.
 * The array icon saves the icon pattern. If a location is set
 * to 1, the corresponding pixel in the icon is displayed.
 */
 hidemouse();
 /* if pixel is not black, make it black */
 if (getpixel(2*x+ICONLEFT,BIGICONTOP+y) != BLACK) {
 putpixel(2*x+ICONLEFT,BIGICONTOP+y,BLACK);

 putpixel(2*x+1+ICONLEFT,BIGICONTOP+y,BLACK);
 icon[y][x] = 0;
 }
 else { /* draw all pixels on with the max color */
 putpixel(2*x+ICONLEFT,BIGICONTOP+y,getmaxcolor());
 putpixel(2*x+1+ICONLEFT,BIGICONTOP+y,getmaxcolor());
 icon[y][x] = 1;
 }
 showmouse();
}


void save_icon(void)
{
/* This function writes the icon pattern to a file. The user
 * is prompted for the filename. The format of the file is
 * presented at the top of the iconed.c file.
 */
 char filename[80];
 FILE *iconfile;
 int i, j;

 printf("\nEnter the file name to store the icon in: ");
 scanf("%s",filename);
 if ((iconfile = fopen(filename,"w")) == NULL) {
 printf("Could not open file.\n");
 return;
 }
 /* Write the header to the file */
 fprintf(iconfile, "%d %d\n", ICONWIDTH, ICONHEIGHT);

 for (j=0; j<ICONHEIGHT; j++) { /* Write the icon */
 for (i=0; i<ICONWIDTH; i++) /* pattern to a file */
 fprintf(iconfile, "%x ", icon[j][i]);
 fprintf(iconfile, "\n");
 }
 fclose(iconfile);
}


int read_icon(void)
{
/* This function reads an icon file into the icon array and
 * calls show_icon() to turn the appropriate pixels on. If the
 * file header doesn't match ICONWIDTH and ICONHEIGHT, the
 * the file is invalid and the icon is not read. The function
 * returns a 0 if a file is not read; otherwise 1 is returned.
 */
 char filename[80];
 FILE *iconfile;
 int i, j;
 int width, height;

 for (j=0; j<ICONHEIGHT; j++) { /* Initialize icon array */
 for (i=0; i<ICONWIDTH; i++) /* to all blanks */
 icon[j][i] = 0;
 }

 printf("\n\n----------- ICON EDITOR -------------\n\n");

 printf("Do you want to edit an existing icon? (y) ");

 if (getch() == 'n')
 return(0);

 printf("\nEnter name of the file to read the icon from: ");
 scanf("%s",filename);
 if ((iconfile = fopen(filename,"r")) == NULL) {
 printf("Cannot open file.\n");
 return(0); /* return a failure flag */
 }
/* Read first line of the icon file. It should contain two
 numbers that are equal to ICONWIDTH and ICONHEIGHT.
 */
 fscanf(iconfile,"%d %d", &width, &height);
 if (width != ICONWIDTH height != ICONHEIGHT) {
 printf("Incompatible icon file.\n");
 return(0); /* return a failure flag */
 }

 for (j=0; j<ICONHEIGHT; j++) {
 for (i=0; i<ICONWIDTH; i++)
 fscanf(iconfile, "%x", &icon[j][i]);
 }
 fclose(iconfile);
 return(1); /* return a success flag */
}


void init_graphics(void)
{
/* This function initializes the graphics hardware.
*/
 int gdriver = CGA;
 int gmode, gerror;

 gmode =4;
 initgraph(&gdriver,&gmode,"");
 if ((gerror = graphresult()) < 0) {
 printf("Failed graphics initialization: gerror=%d\n",
 gerror);
 exit(1);
 }
}

void show_icon(void)
{
/* This function displays the icon pattern stored in the
 * icon array.
 */

 int x, y;

 for (y=0; y<ICONHEIGHT; y++)
 for (x=0; x<ICONWIDTH; x++) {
 if (icon[y][x] == 1) {
 putimage(BIGICONLEFT+2*(x*(BIGBITSIZE+NORM_WIDTH))+1,
 BIGICONTOP+y*(BIGBITSIZE+NORM_WIDTH)+1, bigbit,
 XOR_PUT);

 toggle_icons_bit(x, y);
 }
 }
}







[LISTING TWO]

/* mouse.c -- routines to support a Microsoft compatible mouse.
 * This package assumes that you are running under
 * graphics mode.
 */

#include <dos.h>
#include <conio.h>
#include "mouse.h"
#include "graphics.h"

#define TRUE 1
#define FALSE 0
int mouseexists; /* internal variable set true if a */
 /* mouse driver is detected */


void mouse(int *m1, int *m2, int *m3, int *m4)
{
/*
 * This function provides the interface between the mouse
 * driver and an application program. Several predefined mouse
 * functions supported by the Microsoft mouse are available.
 * Parameters are passed and returned with the ax, bx, cx and
 * dx registers.
 */
 union REGS inregs, outregs;

 inregs.x.ax = *m1;
 inregs.x.bx = *m2;
 inregs.x.cx = *m3;
 inregs.x.dx = *m4;
 int86(0x33, &inregs, &outregs);
 *m1 = outregs.x.ax; /* return parameters */
 *m2 = outregs.x.bx;
 *m3 = outregs.x.cx;
 *m4 = outregs.x.dx;
}


int initmouse(void)
{
/*
 * This function initializes the mouse and displays
 * the mouse cursor at the top, left of the screen, if one
 * is present.
 */

 int m1, m2, m3, m4, gmode;
 char far *memory = (char far *)0x004000049L;

 mouseexists = TRUE;
 m1 = RESET_MOUSE;
 mouse(&m1, &m2, &m3, &m4);
 if (m1) { /* if mouse reset okay, assume */
 gmode = getgraphmode(); /* mouse exists */
 if (gmode == HERCMONOHI) { /* Test for Hercules */
 *memory = 0x06;
 }
 m1 = SET_MOUSE_COORD;
 mouse(&m1, &m2, &m3, &m4); /* mouse exists and draw the */
 showmouse(); /* cursor on the screen at 0,0 */
 return(1); /* return a success flag */
 }
 else { /* no mouse installed */
 mouseexists = FALSE;
 return(0); /* return a no-mouse found flag */
 }
}


void hidemouse(void)
{
/* This function removes the mouse cursor from the screen. It
 * should be called before displaying data on the screen.
 * Use showmouse() to redisplay the mouse cursor. The mouse
 * cursor still moves even though it is not visible. Don't
 * call hidemouse() if the mouse is not already visible.
 */
 int m1, m2, m3, m4;

 if (mouseexists) { /* check for mouse */
 m1 = HIDE_MOUSE; /* hide the mouse cursor */
 mouse(&m1, &m2, &m3, &m4);
 }
}


void showmouse(void)
{
/* This function displays the mouse cursor. You should not call
 * this function if the mouse is already visible.
*/
 int m1, m2, m3, m4;

 if (mouseexists) { /* make sure mouse exists */
 m1 = SHOW_MOUSE;
 mouse(&m1, &m2, &m3, &m4); /* display mouse cursor */
 }
}


void getmousecoords(int *x, int *y)
{
/*
 * This function returns the position of the mouse cursor.
 */

 int m1, m2;

 if (mouseexists) {
 m1 = GET_MOUSE_STATUS;
 mouse(&m1, &m2, x, y);
 /* adjust for virtual coordinates */
 if (getmaxx() == 319) (*x) /= 2;
 }
}


int testbutton(int testtype, int whichbutton)
{
/*
 * This function tests a mouse button state. It returns TRUE
 * if the specified mouse button (whichbutton) meets the
 * specified action (as indicated by testtype); otherwise the
 * function returns FALSE.
 */
 int m1, m2, m3, m4;

 m1 = testtype;
 if (whichbutton == LEFT_BUTTON whichbutton ==
 EITHER_BUTTON) {
 m2 = LEFT_BUTTON;
 mouse(&m1, &m2, &m3, &m4);
 if (m2) return(TRUE); /* return TRUE if action ok*/
 }
 if (whichbutton == RIGHT_BUTTON whichbutton ==
 EITHER_BUTTON) {
 m1 = testtype;
 m2 = RIGHT_BUTTON;
 mouse(&m1, &m2, &m3, &m4);
 if (m2) return(TRUE); /* return TRUE if action ok */
 }
 return(FALSE); /* return FALSE as a catch all */
}


int waitforinput(int whichbutton)
{
/* This function returns a character if a key has been pressed,
 * -1 if a mouse button has been pressed; otherwise a zero. If
 a mouse exists, this routine favors any keyboard action.
 */
 int c = 0;

 while (!c) {
 if (kbhit()) /* check if a key has been pressed */
 c =getch(); /* return the character */
 else {
 if (testbutton(CHECK_BUTTON_PRESS, whichbutton)) {
 while (!testbutton(CHECK_BUTTON_RELEASE,
 whichbutton));
 c = -1;
 }
 else if (testbutton(CHECK_BUTTON_RELEASE,
 whichbutton)) {
 c = -1;

 }
 }
 }
 return(c);
}






[LISTING THREE]


/* mouse.h -- this file includes function prototypes and macro
 * constants for the functions in mouse.c.
 */

/* The following is a list of constants that correspond to the
 * mouse functions supported in mouse.c.
 */

#define RESET_MOUSE 0
#define SHOW_MOUSE 1
#define HIDE_MOUSE 2
#define GET_MOUSE_STATUS 3
#define SET_MOUSE_COORD 4
#define CHECK_BUTTON_PRESS 5
#define CHECK_BUTTON_RELEASE 6
#define SET_MOUSE_HORIZ_RANGE 7 /* not used */
#define SET_MOUSE_VERT_RANGE 8 /* not used */
#define SET_GRAPHICS_CURSOR 9 /* not used */
#define GET_MOUSE_MOVEMENT 11
#define SET_MICKEY_RATIO 15 /* not used */

#define LEFT_BUTTON 0 /* use left button */
#define RIGHT_BUTTON 1 /* use right button */
#define EITHER_BUTTON 2 /* use either button */


/* The function prototypes for the functions in mouse.c
 */

void mouse(int *m1, int *m2, int *m3, int *m4);
int initmouse(void);
void getmousecoords(int *x, int *y);
void hidemouse(void);
void showmouse(void);
int testbutton(int testtype, int whichbutton);
int waitforinput(int whichbutton);












July, 1989
MULTITASKING OS AND GRAPHICS COPROCESSORS


More is better when it comes to graphics




Chuck McManis


Chuck McManis has been working with microcomputers since 1977 when he
purchased a Digital Group Z-80 system. His personal computer collection
includes everything from S-100 machines to the Amiga. In the past he's worked
on graphics coprocessors for the Intel Corporation, but now Chuck is working
at Sun Microsystems on Unix networking software. He can be reached at Sun
Microsystems, 2550 Garcia Ave., Mountain View, CA 94043 or on BIX as cmcmanis.


Graphics coprocessors have traditionally been used only on specialized
high-performance graphics display systems running sophisticated and complex
graphics applications. Today, however, the trend towards window-based,
graphical user-interfaces on standard PCs -- with the resulting CPU burdens
and performance degradation -- is forcing hardware developers to begin
thinking of coprocessors as standard equipment for desktop systems as well.
But graphics coprocessors are only half the story: The development of
multitasking operating systems (OSs) has also played a part in increasing the
overall CPU use and has helped make more powerful graphics applications more
of a reality instead of a dream.
In this article, I'll examine the costs and benefits of using graphics
coprocessors in a multitasking OS environment. In particular, I'll focus on
the hardware and software architecture of Commodore's 68000-based Amiga
personal computer, which combines a high-performance multitasking operating
system with autonomous graphics coprocessors. Why the Amiga? Because it is the
only mainstream PC that comes with a multitasking OS as standard equipment.
This, combined with the innovative use of dedicated LSI logic to offload CPU
tasks, has resulted in a graphics platform that is unique among PCs in its
price range. The Amiga's flexible architecture and modular OS allows
programmers legal access to coprocessors so they can be used to their fullest
potential.


Multitasking, or Who Really Has the Coprocessor?


When a program running a single-tasking OS (like MS-DOS) asks a coprocessor to
draw a line on the screen, the CPU waits for that line to be drawn while
(usually) looping on some sort of status bit or through an interrupt chain.
When a program requests a line draw with a multitasking OS, the graphics
coprocessor begins the drawing process and the task requesting the line draw
is dismissed; another task that does not need the coprocessor is then allowed
to use the CPU. This is a fundamental difference in behavior between
single-tasking and multitasking OSs. Because the CPU doesn't sit idle until
the coprocessor finishes its task, there is an increase in CPU utilization.
For programmers, one drawback of this is that you must always assume that some
other program is using the coprocessor or the display and you can no longer
simply blast away at the hardware and expect reliable results. Instead,
arbitration of access to resources falls under the domain of the OS, which in
turn must provide the mechanisms to request resources. You must write programs
that are able to consider if a resource is available. When the OS interface is
well-designed, this resource arbitration is handled invisibly and you need
only use the interfaces provided to guarantee that the program does not
interfere with other tasks.


The Amiga Hardware


The Amiga system architecture includes a generalized coprocessor, the Copper,
and a dedicated graphics engine, the Blitter. These peripherals -- as well as
four D/A channels, four A/D converters, the system control functions, and a
25-channel DMA controller -- are implemented in a set of three custom chips
named Agnus, Denise, and Paula. These chips share a dedicated memory bus that
allows them to access memory without interfering with the CPU. The custom
chips can only access memory on this bus, which is why this memory is referred
to as Chip Memory. As Figure 1 shows, the custom chips are situated on the
chip memory bus. Although the architecture sets aside 2 Mbytes of the address
space for Chip Memory, the original Amiga chipset only supported 512K, and a
recently introduced upgraded chipset supports only 1 Mbyte. The primary
benefit of this extra memory is that the CPU can execute code in memory
outside of this range without any performance degradation resulting from
contention with the custom chips. The control registers for the custom chips
are visible to the CPU as memory addresses in the range of $DFFOOO through
$DFFFFF.
The heart of the coprocessor group is the System Controller. It is here that
the CPU can start and stop the DMA channels that feed data to these chips and
enable or disable interrupts from the chips to the CPU. The DMA channels feed
data to various peripherals such as the D/A converters. The four D/A channels
on the Amiga are some of the simpler peripheral interfaces. By adding the DMA
channel, the D/A converter is elevated to the status of a simple coprocessor.
By having their waveforms automatically feed into them from the DMA chips, the
coprocessor can output a digitized waveform without any intervention from the
CPU.
The most sophisticated peripheral is the Copper, so named because it is the
traffic cop of the Amiga. The Copper -- with three instructions (MOVE, WAIT,
and SKIP) that allow it to store data and provide conditional testing, and
programmatic branching -- is, in some ways, a RISC processor in its own right.
The Copper is able to write to any of the custom chip's control registers,
even its own. This allows the Copper to be the CPU's lieutenant by taking over
mundane tasks such as setting up the CRT controller (implemented in Denise) to
build on the display every frame. A program, called the Copper List, is
started at every vertical blank from the start address, which is stored in the
COP1LC register. The Copper is synchronized with the video beam so that it
knows where the beam on the screen is at all times and can wait for it to
arrive at a particular position. This feature comes in handy when playing with
graphical objects called sprites. The Copper code in Example 1 could be used
to notify the CPU that video frame has ended. The Copper is built to use
alternate clock cycles to access memory. This allows it to minimize contention
between itself and other coprocessors for the Chip Memory bus. This
coprocessor is half of the equation that makes the Amiga such an effective
graphics platform.
Example 1: Sample Copper code

 WAIT 0,261 ; Wait for line 261
 MOVE COPR, INTREQ ; Interrupt the CPU
 WAIT 0,$FFFE ; Wait until VBlank

The other half of this equation is the Blitter. The Blitter is a graphics
computation unit that transfers 16-bit words around in memory while performing
bitwise logical operations on those words. The Blitter is capable of reading
data from up to three addresses, performing any logical operation on that
data, and then optionally storing the result at a fourth address. The
addresses are automatically updated after each operation by an amount you
specify. This allows the Blitter to work within a programmed area of a larger
bitmap. Two additional functions built into the Blitter are the ability to
draw lines between two arbitrary points and the ability to fill in an area
with an arbitrary 16-bit binary pattern.
The Blitter's contribution to the Amiga is the ability to draw lines and
render filled polygons at a rate that is equivalent to the speed that the
68000 CPU could if it were rendering them. The key, of course, is that the
68000 can now be off doing something else. Window-based, user-interface
operations often map a sequence of commands directly to the Blitter. Because
of this, operations like moving windows, rendering menus, and scrolling text
are completed much faster on the Amiga than on other 68000-based machines.
The Amiga also contains a sprite engine that is capable of overlaying its
contents on the screen at an internally specified X and Y coordinate. The
sprite can be thought of as a self-contained mini-bitmap that is independently
positioned on the screen and only relies on the CPU to tell it where and when
to move. This activity costs very little in terms of CPU cycles. The Amiga
uses one of the eight available sprites as the window system cursor.


The Amiga Software


The Amiga's OS has three major parts: The multitasking kernel, the graphics
interface, and the disk operating system. I'll discuss the first two here. For
further information on the DOS, see the The AmigaDOS Reference Manual (Bantam
Books, 1987).
The software foundation of the OS is the multitasking kernel, Exec. This
kernel provides the fundamental controls that the other pieces are layered
upon. Exec is composed of several component parts. The basic grouping consists
of the list management routines, resident library support, message management,
interrupt handling, task management, resource management, and devices.
Completely describing Exec is beyond the scope of this article, so I will only
describe those aspects that support the coprocessors and graphics.
To prevent processes from conflicting with each other when they attempt to use
the coprocessors, Exec must keep those processes synchronized to the
availability of the resource; Exec uses messages and semaphores to accomplish
this.
A message port is a rendezvous point where many competing process requests can
be effectively managed. Exec queues messages to a port using a first-in,
first-out policy. The overhead for this synchronization is fairly high and is
generally restricted to those activities that are fairly time-consuming
(device I/O, for instance). The task that owns a resource creates a message
port upon which it receives requests to use its resource. As requests arrive,
the task removes messages one at a time, serializing specific requests to that
resource. A significant advantage to this system is that the interface between
an application and the task that owns a resource is defined by a message
protocol. As long as the task follows this protocol, it may be replaced by a
new or upgraded equivalent task at any time without effecting users of the
resource. Some resources (such as the Blitter) cannot be used simultaneously,
yet the whole reason for having a Blitter is to minimize the work that is
required for the CPU to render graphics. A semaphore is used to provide a
simpler and faster method to ensure exclusive access to the Blitter without a
lot of overhead.
A semaphore can be thought of as a restroom key that is hanging on a hook by
the door. People wanting to use the restroom must have the key to open the
door. However, because access is only allowed if you have the key, the key
will be unavailable when the restroom is in use (exclusivity is assured).
Because the key is available right next to the door that requires it, you are
not needlessly delayed if the restroom is free.
Every task that uses the screen will require the use of the Blitter. A
semaphore is maintained by Exec, and only the task that is currently drawing
owns it. A library function, OwnBlitter(), is provided in the graphics library
to acquire this semaphore. This function requests the Blitter semaphore from
Exec. If it is available, it is immediately granted, and if it is not
available the task is placed in a queue for Blitter use and dismissed. When
the Blitter is available, Exec will give the semaphore to the next task in the
queue and resume it. From the programmer's perspective, the call to
OwnBlitter() will simply return. The programmer is now free to use the Blitter
for his own purposes because no one else will be granted access until he gives
back the semaphore to indicate that the Blitter is no longer needed. This is
accomplished with DisownBlitter().
The key concept is one of ownership. Because a program may not be the only one
running, it cannot assume it owns the machine; rather, it must synchronize its
behavior so that it does not interfere with another process that is using
non-shareable resources. Even though you might suspect otherwise, this
additional overhead of synchronization with other tasks does not lead to
slower or to unacceptable performance.
Another one of the kernel's jobs is to manage resident libraries that are
different from the libraries you would link in a compiled program. The
difference is that while there may be several processes using routines within
a resident library, there is only one copy of that resident library in memory.
The graphics library mentioned earlier is one such library. The savings in
memory is tremendous and is one of the reasons that the Amiga is fully
multitasking in 512K bytes of RAM, whereas you would probably need up to four
to eight times that much memory to achieve the same capabilities on other PCs.
It is also another way of providing transparent layering of capabilities to
the programmer.
When using graphics on the Amiga, three libraries make up the layers between
an application and the hardware (from highest to lowest): Intuition (the
windowing system); Layers (which control clipping); and Graphics (the
lowermost layer that provides the primitives). These three layers provide
progressively closer access to the hardware. Intuition provides the
highest-level functions, most of which are designed to set up the window
system environment and to facilitate communication with the user. Like the
application program, Intuition is a user of the Layers and Graphics libraries.
Where Intuition provides the window environment, Layers provides the drawing
environment. Rendering commands are clipped to a specified region in this
environment. A layer appears to a program as a contiguous drawing surface. The
actual surface (the screen) may actually be divided into several distinct
regions. The coprocessors and the OS come together at the Graphics layer. It
is here that a call to Draw( ) is converted into a Blitter operation.



The Graphics Architecture


The primary data structure of Amiga graphics is the Rastport, which contains
such information as where the memory for the bitmaps resides, the current text
font, the current pen colors, the coordinates of the last point drawn, and so
on. Graphics also contains a LayerInfo structure that, when present, is used
by the layers library to determine how to clip graphics and text rendered on
the screen.
The Copper and the Blitter are the main tools of the graphics library. The
Copper is used to set up the physical characteristics of the display, and the
Blitter is used to render graphic objects into that display as quickly as
possible. Both functions have a data structure that provides the needed
information for them to accomplish what is required of them. For the Copper,
that data structure is a View.
A View is a particular display mode with a particular set of colors in a
particular orientation on the screen. Typically, this sort of thing is
controlled by the CRT controller in the PC. On the Amiga the CRT controller is
implemented in Denise. As CRT controllers go, Denise is very straightforward.
It fetches data from the bitplanes in chip memory, feeds this to a set of 32
color registers, each containing a 12-bit color triple: four bits for red,
four for green, and four for blue. This triple is then input into three D/A
converters that convert the value into a voltage level that goes out onto the
video connector. The registers, however, that control certain processes (where
the data for display is stored, what is in the color registers, and so on) are
accessible by the Copper, which is synchronized to the video beam. Thus, a
specialized Copper List could be constructed to control Denise on a
line-by-line basis, and in fact this is exactly what is done by the graphics
library. The required display is described by filling in various fields in a
View and one or more ViewPort structures. These structures are passed along to
the graphics library that constructs a specialized Copper List to provide that
View. If the View is fairly simple, such as a 320 x 200 pixel screen that
isn't interlaced, the Copper List would be a simple set of initialization
instructions that would be loaded into the CRT controller on each vertical
blank. The power of this combination becomes evident when the displays become
more complicated.
Because the Amiga is capable of displaying 32 colors when in the 320
pixel/line mode, it is often common to use this mode for games, paint
programs, and other applications that use lots of colors. With only 320 pixels
on a line, however, a program is limited to 40 characters if it is using the
standard system font. As a status line, this can be very limiting. Using the
Copper, it's possible to design a display that has a 320 x 180 pixel screen on
top, and a 640 x 20 pixel screen on the bottom of the display. These two areas
make up one complete 200 line display. The Copper is required to switch the
display mode after the video beam has drawn 180 lines on the monitor in low
resolution. Thus the last 20 lines, which are enough for two lines of 80
character text in the standard font, can be used for a status display.
Needless to say, this is not an easy thing to do with a dedicated CRT
controller, but with the Amiga this is fairly simple. Listing One shows an
example of setting up a View like the one described earlier. In the example,
the Copper changes not only the resolution, but also the values of the colors
in the color table. Listing Two shows the Copper List that is created by the
MakeVPort( ) and MrgCop( ) routines; Figure 2 illustrates the display
generated by this program.
While the View is the structure that the Copper uses, the corresponding
structure for the Blitter is the RastPort. This structure contains information
such as where the bitmap data can be found, what logical operation the Blitter
will perform, what pattern to use for patterned lines, and what font to use.
The graphics library takes this information (along with the requested
operation) and determines the values needed to execute the operation. The
library also acquires the Blitter's semaphore and performs the function. Once
the Blitter has been started the library returns immediately. Control can
return to the current program or any other program that needs the CPU while
the Blitter is drawing.
To demonstrate the accelerating effect of letting the Blitter draw the lines
instead of the CPU, I wrote a simple benchmark called blit.c (Listing Three).
When compiled without WAIT_BL IT defined, the graphics library will start a
line draw and then return to allow the CPU to compute the next endpoint while
the Blitter is drawing. With WAIT_BL IT defined, the program is forced to wait
for the Blitter to complete each line before beginning the calculations for a
new line. The results? About a 12 percent difference in speed. (Your mileage
may vary, of course.)


Conclusions


The Amiga integrates the use of graphics coprocessors at all levels of its
operation. The results are a noticeably faster response time to user requests,
especially in window operations. By using those coprocessors efficiently, the
Amiga can do such things as display animations while playing a soundtrack. In
fact, some of the applications that were pioneered on the Amiga (desktop video
production, for example) have only recently showed up on some of the newer
32-bit machines (the Macintosh II and 80386-based MS-DOS PCs) and are not
available at all with the current set of 8- or 16-bit processor-based
machines. The conclusion I draw from this is that for high-performance
graphics applications, the implementation of coprocessors and a multitasking
OS puts the Amiga into the performance range of the 32-bit machines, yet at
the price of a 16-bit machine.


Availability


All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063; or call 800-356-2002 (from inside
Calif.) or 800-533-4372 (from outside Calif.). Please specify the issue
number.


_MULTITASKING OS AND GRAPHICS COPROCESSORS_
by Chuck McManis


[LISTING ONE]

/* dualvp.c - Dual Viewports on the amiga
 * Written 4/4/89 by C. McManis using Lattice C 5.02
 */

#include <exec/types.h>
#include <exec/memory.h>
#include <graphics/gfx.h>
#include <graphics/view.h>
#include <graphics/gfxbase.h>
#include <graphics/rastport.h>
extern struct GfxBase *GfxBase;
char *TextString = "Amiga Graphics Example";

 /* Viewport 0 colors */
UWORD colors0[4] = {0xccc, 0x000, 0x0f0, 0x00f},
 /* Viewport 1 colors */
 colors1[4] = {0x0f0, 0xc0c, 0xf00, 0xfff};
void
_main()
{
 struct View MyView, *OldView;
 struct ViewPort Vp0, Vp1;
 struct BitMap Bits;
 struct RasInfo MyRaster;
 struct RastPort rp;
 int i;

 /* Open the resident graphics library */
 GfxBase = (struct GfxBase *)OpenLibrary("graphics.library",0L);
 if (!GfxBase)

 exit(1);
 OldView = GfxBase->ActiView; /* Save this away */

 /* Initialize the View structures */
 InitView(&MyView);
 InitVPort(&Vp0);
 InitVPort(&Vp1);
 Vp1.Next = NULL;
 Vp0.Next = &Vp1; /* create a linked list of viewports */
 MyView.ViewPort = &Vp0; /* With the first one being Vp0 */

 /* Set up some display memory */
 InitBitMap(&Bits, 2, 640, 200);
 Bits.Planes[0] = (PLANEPTR)
 AllocMem(2*RASSIZE(640, 200),MEMF_CHIP+MEMF_CLEAR);
 Bits.Planes[1] = Bits.Planes[0] + RASSIZE(640, 200);
 if (!Bits.Planes[0])
 goto cleanup;
 MyRaster.BitMap = &Bits;
 MyRaster.RxOffset = 0;
 MyRaster.RyOffset = 0;
 MyRaster.Next = NULL;

 /* Both viewports are looking at the same display memory but have
 * different sets of colors
 */
 Vp0.RasInfo = &MyRaster;
 Vp0.DWidth = 320;
 Vp0.DHeight = 175;
 Vp0.ColorMap = (struct ColorMap *)GetColorMap(4);
 LoadRGB4(&Vp0, colors0, 4);
 Vp1.RasInfo = &MyRaster;
 Vp1.DWidth = 640;
 Vp1.DHeight = 20;
 Vp1.DyOffset = 179;
 Vp1.Modes = HIRES;
 Vp1.ColorMap = (struct ColorMap *)GetColorMap(4);
 LoadRGB4(&Vp1, colors1, 4);

 /* Initialize a RastPort so that we can draw into that memory. */
 InitRastPort(&rp);
 rp.BitMap = &Bits;
 SetAPen(&rp, 1); /* Foreground color */
 SetBPen(&rp, 0); /* Background color */
 Move(&rp, 3, 12); /* Move the graphics cursor to (3, 12) */
 /* Write something */
 Text(&rp, TextString, strlen(TextString));
 MakeVPort(&MyView, &Vp0); /* Build the copper list for Viewport 0 */
 MakeVPort(&MyView, &Vp1); /* Build the copper list for Viewport 1 */
 MrgCop(&MyView); /* Merge it into the final list */
 LoadView(&MyView); /* Show it off */

 /* SPIN FOR A WHILE */
 for (i=0; i<1000000; i++)
 ;
 LoadView(OldView); /* Return to the old view */
cleanup:

 /* Now give back the memory other tasks may need it */

 if (!Vp0.ColorMap)
 FreeColorMap(Vp0.ColorMap);
 if (!Vp1.ColorMap)
 FreeColorMap(Vp1.ColorMap);
 FreeVPortCopLists(&Vp0);
 FreeVPortCopLists(&Vp1);
 FreeCprList(MyView.LOFCprList);
 if (!Bits.Planes[0])
 FreeMem(Bits.Planes[0], 2*RASSIZE(640, 200));
 if (!GfxBase)
 CloseLibrary(GfxBase);
 exit(0);
}







[LISTING TWO]

000208a8 2b01:fffe WAIT (00,2b) ; (x = 0, y = 43)
--- load color table ---
000208bc 008e:0581 MOVE 0581,diwstrt ; (left = 129, top = 5)
000208c0 0100:0200 MOVE 0200,bplcon0
000208c4 0104:0024 MOVE 0024,bplcon2
000208c8 0090:40c1 MOVE 40c1,diwstop ; (right = 449, bottom = 320)
000208cc 0092:0038 MOVE 0038,ddfstrt ; pixel val = 112
000208d0 0094:00d0 MOVE 00d0,ddfstop ; pixel val = 416
000208d4 0102:0000 MOVE 0000,bplcon1
000208d8 0108:0028 MOVE 0028,bpl1mod
000208dc 010a:0028 MOVE 0028,bpl2mod
000208e0 00e0:0002 MOVE 0002,bpl1pth
000208e4 00e2:86e8 MOVE 86e8,bpl1ptl
000208e8 00e4:0002 MOVE 0002,bpl2pth
000208ec 00e6:c568 MOVE c568,bpl2ptl
000208f0 2c01:fffe WAIT (00,2c) ; (x = 0, y = 44)
000208f4 0100:2200 MOVE 2200,bplcon0
000208f8 db01:fffe WAIT (00,db) ; (x = 0, y = 219)
000208fc 0100:0200 MOVE 0200,bplcon0
00020900 de01:fffe WAIT (00,de) ; (x = 0, y = 222)
--- load color table ---
00020914 008e:0581 MOVE 0581,diwstrt ; (left = 129, top = 5)
00020918 0100:0200 MOVE 0200,bplcon0
0002091c 0104:0024 MOVE 0024,bplcon2
00020920 0090:40c1 MOVE 40c1,diwstop ; (right = 449, bottom = 320)
00020924 0092:003c MOVE 003c,ddfstrt ; pixel val = 120
00020928 0094:00d0 MOVE 00d0,ddfstop ; pixel val = 416
0002092c 0102:0000 MOVE 0000,bplcon1
00020930 0108:0000 MOVE 0000,bpl1mod
00020934 010a:0000 MOVE 0000,bpl2mod
00020938 00e0:0002 MOVE 0002,bpl1pth
0002093c 00e2:86e8 MOVE 86e8,bpl1ptl
00020940 00e4:0002 MOVE 0002,bpl2pth
00020944 00e6:c568 MOVE c568,bpl2ptl
00020948 df01:fffe WAIT (00,df) ; (x = 0, y = 223)
0002094c 0100:a200 MOVE a200,bplcon0
00020950 f301:fffe WAIT (00,f3) ; (x = 0, y = 243)

00020954 0100:0200 MOVE 0200,bplcon0
00020958 ffff:fffe WAIT (7f,ff) ; (x = 127, y = 255)







[LISTING THREE]

/* blit.c - Demonstrates the benefit of the blitter.
 * Written 4/9/89 by C. McManis using Lattice C 5.02
 * The difference on my machine between waiting for the blitter
 * to complete before calculating the next set of draw parameters
 * is 1.6 vs 1.4 seconds, about a 12.5% increase in speed.
 */

#include <exec/types.h>
#include <exec/memory.h>
#include <intuition/intuition.h>
#include <graphics/gfx.h>
extern struct IntuitionBase *IntuitionBase;
extern struct GfxBase *GfxBase;
struct NewScreen NS = {
 0, 0, /* Position on the display */
 320, 200, 4, /* Attributes (Width, Height, Depth) */
 1,0, /* Detail and Block pens */
 0, /* ViewModes nothing special */
 CUSTOMSCREEN, /* It is our own screen we want */
 0, /* Using the Default font */
 "Blitter Test", /* With a simple title. */
 0, /* No special gadgets */
 0 /* And no special bitmap */
 };
struct Screen *MyScreen;
struct RastPort *RPort;
void
cleanup(n)
 int n;
{
 if (GfxBase)
 CloseLibrary(GfxBase);
 if (MyScreen)
 CloseScreen(MyScreen);
 if (IntuitionBase)
 CloseLibrary(IntuitionBase);
 exit(n);
}
void
main()
{
 int i, /* Loop counter */
 x, y, c, /* some random draw parameters */
 t0[2], /* Start Time */
 t1[2]; /* End time */
 IntuitionBase = (struct IntuitionBase *)
 OpenLibrary("intuition.library",0L);
 if (! IntuitionBase)

 cleanup(1);
 GfxBase = (struct GfxBase *)
 OpenLibrary("graphics.library", 0L);
 if (! GfxBase)
 cleanup(1);

 /* This does all of the view construction for us */
 MyScreen = (struct Screen *) OpenScreen(&NS);
 if (!MyScreen)
 cleanup(1);
 timer(t0); /* Start the clock running */

 /* Get the RastPort of this screen */
 RPort = &(MyScreen->RastPort);
 SetAPen(RPort, 1); /* Foreground pen = 1 */
 SetBPen(RPort, 0); /* Background pen = 0 */
 srand(42); /* set the seed */
 Move(RPort, 160, 100); /* Move to the moiddle of the screen */

 /*
 * Note we generate psuedo random numbers (eg the same set of
 * random numbers every time.
 */
 for (i=0; i<1000; i++) {
 x = (rand() % 300) + 10;
 y = (rand() % 180) + 10;
 c = rand() % 16;
 SetAPen(RPort, c);
 Draw(RPort, x, y);
#ifdef WAIT_BLIT
 WaitBlit(); /* Simulate non-coprocessor */
#endif
 }
 timer(t1); /* stop the clock */
#ifdef WAIT_BLIT
 printf("With waiting for the blitter, we took %d microseconds.\n",
 (t1[0] - t0[0]) * 1000000 + (t1[1] - t0[1]));
#else
 printf("Without waiting for the blitter, we took %d microseconds.\n",
 (t1[0] - t0[0]) * 1000000 + (t1[1] - t0[1]));
#endif
 cleanup(0);
}



Example 1. Sample Copper code

 WAIT 0,261 ; Wait for line 261
 MOVE COPR, INTREQ ; Interrupt the CPU
 WAIT 0,$FFFE ; Wait until VBlank











July, 1989
IMAGE MATHEMATICS


Image enhancement by numbers




Victor Duvanenko


Victor is a graduate student in electrical and computer eng., with a minor in
computer graphics at North Carolina State University. At the time of this
writing he was a consultant at Silicon Engineering, in Santa Cruz, Calif.
Before that be worked as an IC designer at Intel. He can be reached at 1001
Japonica Ct., Knightdale, NC 27545.


Computer scientists and users have been jealous of photographers, movie
makers, and television producers for some years because computer monochrome
images, or even ones with 256 colors, just didn't compare with
photographic-quality (true color) images. Consequently, computer users have
been forced to use cartoon-like animation and imaging. This may have been
sufficient for adventure games, spreadsheets, and low-end word-processing
programs, but it just wasn't good enough for image processing and photography.
In short, without true color capabilities, the dream of advanced applications,
such as on-line encyclopedias that provide images and movie clips, will never
materialize.
In this article I'll discuss some basic functions of image processing and
demonstrate one possible implementation of arithmetic functions through
logical operations. Then I will use these arithmetic functions to combine and
enhance images.


Imaging and Photography


When we use a camera to take a picture, the light reflected from an object
passes through a reducing lens and exposes the film. We take this film to a
photo lab which, using a series of chemical processing steps, develops the
film. Next, photographic paper is exposed by shining light through the
developed film and an enlarging lens. Chemical processing is then used again
to develop the photograph.
In the electronic version of photography, we can use an electronic "camera"
connected to image-capture (frame grabber) hardware to take a picture, and we
can use a color printer to get a paper copy of the picture. In this process,
the electronic camera serves as a reducing lens and organizes the light
intensity and color information, reflected from an object, into a serial
stream. The image capture hardware converts this stream into numbers, which
are then stored in the computer's memory. A monitor or a printer serves as an
enlarging lens, and some form of magnetic memory medium, such as disk or tape,
is used to store the captured image for later recall.
But a computer is much more than a mere storage medium. It can also be used as
a processing lab because, once an image has been captured and stored as
numbers, that image can be enhanced or changed using mathematical
manipulations to achieve some of the same tricks of the trade photographers
have used for years. These techniques include the use of filters to get rid of
glare, create special effects, or enrich certain colors. Other techniques
include the use of black and white to create a mood, the use of zoom lenses to
capture only the interesting sections of images, and the use of double
exposure to superimpose several images. What's significant is that each of
these techniques can be accomplished mathematically, rather than chemically.


Logical Operations


Graphics and image manipulations have traditionally been limited to logical
(Boolean) operations -- XOR, NOT, OR, AND, and so on -- mostly because of
inexperience, simplicity, and the dominance of monochrome hardware. Because
our goal is to modify an image or combine two images, we must understand fully
how picture elements (pixels) can be manipulated or combined. The simplest
case is monochrome graphics, 1 bit per pixel, where 1 bit in memory
corresponds to 1 dot on the screen.
A single memory bit can hold only two possible values -- 0 or 1. These two
values constitute a binary (Boolean) set. Two bits can be combined in only 4
possible ways to produce a single bit (see Table 1). The process of combining
2 bits to generate a single bit is called a logical operation. There are 16
possible logical operations (see Table 2). Logical operations always produce a
result that is of the same length as the original elements. In other words, if
two 1-bit elements are combined, the result is always 1 bit long.
Table 1: Four possible combinations of two bits

 X Y Output
 ------------------

 0 0 output [0]
 0 1 output [1]
 1 0 output [2]
 1 1 output [3]

Table 2: 16 possible logical operations

 --------------------------------------------------------
 X Y 0 1 2 3 4 5 6 7 8 9 A B C D E F 
 ------------------------------------------------------
 
 0 0 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 
 0 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0 1 1 
 1 0 0 0 0 0 1 1 1 1 0 0 0 0 1 1 1 1 
 1 1 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 
 -------------------------------------------------------
 G N N N X N A X (Y) (X) O P
 R O O O O A N N R O
 O R T T R N D O W
 U (X) (Y) D R E
 N R

 D

Many graphics software packages allow programmers to set a "logical operation
attribute" and then draw in that mode. This means that in XOR mode, as a white
line is drawn over an image, all the bits along the path of that line are
reversed. This is a useful technique for cursors and is used in many CAD
packages.
In the monochrome world, we can emulate arithmetic operations with logic
operations. If you look at the AND operation you can quickly recognize a
multiply, and the OR is an add with saturate (explained later). At 1 bit per
pixel, logical operations do the job perfectly. For larger pixel depths
(multiple bits per pixel), however, logical operations can no longer serve as
substitutes for actual arithmetic operations.


Arithmetic Operations and Saturation


Arithmetic operations on two operands have a nasty habit of producing a result
of length unequal to the length of either operand. Maybe that is the reason
why we make a distinction between logical and arithmetic operations. In the
case of addition, if the first operand is N bits long and the second is M bits
long, the result can be up to (max(N, M) + 1) bits. For example, when we add
the binary numbers 1 and 1 the result is 10. In other words, for addition we
may require one more bit than the longest of the two operands to store the
result. Some textbooks call the generation of this extra bit a "carryout" or
an overflow condition. Multiplication is even worse -- we require up to (N +
M) bits to store the result. For example, multiplying binary 11 (decimal 3) by
11 (3) results in 1001 (9), which requires 4 bits of storage.
By this time you are probably wondering why we are going through this
nonsense. Well, the reason is, if we add two pixels that are each 8 bits long
and get a 9-bit result, how do we display this result? If we have only 8-bit
numbers, how do we represent a 9-bit number?
The fact that the result uses more bits than either of the operands is
irritating even in microprocessors. It is not pleasant to get a 33-bit result
when adding two 32-bit numbers. How do you put the result back into a 32-bit
register? Most microprocessors don't have a "crowbar" instruction. And how do
you deal with the result of multiplication?
The microprocessor world has come up with a simple solution -- let the
software deal with it! So most microprocessors just inform the software that
an overflow condition has occurred by raising a flag. The software must then
look at the flag and do something about the overflow. In the case of
multiplication, the microprocessor simply stores the result in two registers
and, once again, leaves it to the software to figure out how to fit the
result, which now occupies twice as many bits, into a single register.
Most programmers don't concern themselves with overflow conditions. If they
have even a small inkling that a data type might encounter an overflow
condition, they simply go to the next larger data type. For example, if there
was a possibility that an array would grow beyond 64K elements, a long
(32-bit) data type would simply be used for the array index. The only
reasonable aid that most high-level-language compilers give us is that if two
operands of different size are being combined, the result is of the larger
size. (I hope this is sufficient warning to computer scientists to make sure
that results don't overflow.)
But what are graphics programmers to do when they've got only a single data
type that we'll call pixel. They have no other data types to fall back on. The
result must fit into the pixel data type, and the resulting display must make
sense (the greens are not allowed to spill over into the reds and so on).
The solution is actually quite simple. We represent all pixels that take more
bits to represent than the pixel data type can hold by the largest number that
a pixel data type will hold. For example, suppose we have a 24-bit-per-pixel
system in which three primary colors (red, green, and blue) are used to make
up each pixel, and we use 8 bits for red, 8 bits for green, and 8 bits for
blue. Any intensities of blue that take more than 8 bits to represent are
represented by the largest value that does fit into 8 bits (255 decimal). In
other words, when adding two pixels, if the blues are added together and the
result is greater than 255, then the result is forced to be equal to 255. Some
books call this technique saturation. The key point is that the resultant
color is always brighter (or of equal brightness) than either of the operand
pixels. If saturation isn't used, it is possible for the resultant pixel to be
dimmer than either of the operand pixels -- and this makes no visual sense.
For subtraction, the result is never allowed to go negative. If subtraction of
two pixels results in a negative number, the resultant pixel is forced equal
to zero. This is called starvation. Thus, the resultant color for subtraction
is always dimmer (or of equal brightness) than the brightest operand pixel.
Multiplication is handled in the same way as addition, and division doesn't
cause problems (except for generating a non-integer result).
The code for this technique is simple and is shown in Listing One. Note that
8-bit color values are first promoted to the 16-bit integer data type and then
added together to ensure no overflow. The resultant pixel value is then
saturated, as discussed earlier.
A more mathematically philosophical explanation might be that unsigned integer
arithmetic must be used and the following rules obeyed:
1. When adding two numbers, the result must always be greater than or equal to
either operand, but less than or equal to the maximum possible value.
2. When subtracting two numbers, the result must always be less than or equal
to the first operand, but greater than or equal to zero.
3. When multiplying by other than zero, the result must always be greater than
or equal to either operand, but less than or equal to the maximum possible
value.


Bit Block Transfer


A Bit Block Transfer, referred to as Bit_blt, is probably the most useful
graphical operation. It is used to move a rectangular block of pixels from one
position within an image to another position but is really nothing more than a
fancy DMA (direct memory access). Bit_blt is a bit trickier than a DMA on two
counts, however:
1. DMA treats memory as linear, one-dimensional space and moves contiguous
memory regions. Bit_blt treats memory as a two-dimensional array and moves an
arbitrary-size rectangle from one spot to another.
2. DMA always treats memory as a linear array of bytes, words, or other
byte-divisible elements. Bit_blt treats memory as a two-dimensional array of
pixels, usually of 1, 2, 4, 8, 16, 24, 32, or even 48 bits in size. This
implies that Bit_blt handles boundary conditions very carefully, as it may
need to modify a part of a byte or a word because several pixels may fit into
a byte or a word.
Bit_blt is used extensively for scrolling and windowing because it is
efficient at handling rectangular regions of pixels.
After a while, merely moving pixels from one place in the image to another
becomes dreadfully boring. So, graphics programmers decided to make Bit_blt
perform logical operations as it moves pixels of a rectangular area. For
example, an XOR Bit_blt would move a rectangular region of pixels from one
location, called the source, to another, called the destination. As it moves
each pixel, Bit_blt would read both the source and the destination pixel and
combine every bit of the two pixels using an XOR logical operation. The result
would be placed in the destination pixel. This would be done on all pixels of
the source and destination rectangle. Obviously, the source and the
destination have to be of the same size for this to work properly. In any
case, with an XOR logical operation, the result is more exhilarating than a
mere movement of a rectangular region.
Because a Bit_blt operation is a popular way of manipulating bit-map graphics,
many graphics processors and coprocessors handle it with great efficiency,
moving pixels at rates of millions per second. But, because most graphics
programmers are satisfied with logical operations, most devices limit their
repertoire to logical operations only.
Being able to manipulate images with only logical operations is fine for
windowing and scrolling but is not enough for image processing and
photography. Somehow, we must get from logical operations to arithmetic
operations.


From Logic to Arithmetic


The techniques for combining logic operations to make other useful functions
are well-established in the field of logic design. It is, for instance,
possible to combine logic operations/gates to make arithmetic functions. A
typical adder logic circuit is shown in Figure 1. This circuit, called a full
adder, adds two bits and a carryout from the addition of the lower bits.
Several single-bit adders can be configured together to make a multiple-bit
adder; Figure 2 shows an 8-bit adder configuration. Other useful circuits,
such as a full subtracter and a half adder, are shown in Figures 3 and Figure
4. Several full subtracters can be cascaded together to form an 8-, 16-, or
even 32-bit subtracter circuit.
A modified version of the full adder logic circuit is shown in Figure 5. Note
the subtle difference of using the sum to generate the carryout. The advantage
of this algorithm/circuit is that it is tuned toward serial implementation,
which means that it can be implemented in software more readily. The algorithm
uses fewer intermediate results (in fact, it uses only one), as this is very
costly in graphics. For example, a single 640 x 480 image, at 24 bits/pixel
occupies approximately 1 Mbyte of memory. Every intermediate result of an
operation also takes 1 Mbyte of memory, so it is costly to have intermediate
results. And, if your hardware has only 3 Mbytes total, you are limited to one
temporary variable when combining two 640 x 480 images. You could break the
image up into smaller pieces, thereby reducing the amount of needed temporary
storage, but this complicates things and makes them less efficient.
In any case, the main message is that it is possible to combine logic
operations in some fashion to come up with arithmetic functions. It is not
necessary to use standard algorithms. You can modify these algorithms to tune
them for a specific application, to squeeze more performance out of them.


Putting the Functions to Work


Now that we have these functions, how can we use them to perform simple
image-processing and photography tasks? Simple! By adding two images together,
with saturation, a double exposure effect is created. Another technique might
be to make color adjustments by setting the second image to a constant color
and then adding or subtracting it to/from the first image. You can create the
effect of underexposure by subtracting some constant value from all reds,
greens, and blues, which will make an image appear darker or underexposed. For
overexposure, just add a constant value.
Subtraction of two images can be useful for data compression. For example,
when compressing a movie, frame-to-frame differences are quite small most of
the time. Thus, if you compress only the differences between the frames, you
will avoid compressing the data that has not changed. Of course, you can do
just about anything by directly manipulating the image in the graphics memory
(bit map).


Benchmarks


By now you are probably wondering why we are going through so much pain. The
reason is performance! Because Bit_blt with logical operations is available,
why not use it? All we have to do is combine several Bit_blts with different
logic operations to give us an arithmetic function. Of course, we could also
write our own Bit_blt routine that would perform arithmetic. But, you shall
see that this is not as efficient, even when using a 20-MHz 80386.
I used a 20-MHz 80386 IDR PC AT for software development and benchmarking. I
also used Digihurst image-capture and display-board, along with a Sony
Multiscan monitor. The Digihurst board is a true color (24 bits/pixel)
graphics board capable of image capture from an NTSC source (TV). You can hook
it up to a VCR, watch a movie on the monitor, and capture any frame. This
board uses three Intel 82786 graphics coprocessors, one for each primary
color. The board also has 3 Mbytes of graphics memory -- 1 Mbyte for each
color -- and is capable of storing three full images. After the images have
been captured, they can be stored on disk.
The implementation of the modified ADDer algorithm is shown in Listing Two. As
you can see, it is nothing more than a series of logical Bit_blts
(cmd_copy_image procedure calls), in the order specified in Figure 5. The
cmd_copy_image and cmd_wm_copy_image procedures are actually direct Intel
82786 Bit_blt commands (given to all three 82786s simultaneously).
Performance benchmarks were done on two preloaded 640 x 480 true color (24
bits/pixel) images using the code shown in Listing One. The results are shown
in Table 3. The 80386 program was written in C (see Listing One) and used
pointers and registers wherever possible. The Digihurst board ran the 82786s
at 12.5MHz, which is below the 20-MHz top-speed specification. Running the
82786s at 2OMHz will double the performance, which will result in close to a
5:1 performance ratio.
Table 3: Performance results

 80386 82786 Ratio
 ------------------------------------------------

 Addition 10.7 sec. 4.4 sec. 2.4:1
 Subtraction 10.7 sec. 4.4 sec. 2.4:1
 Addition 12.7 sec. 5.5 sec. 2.3:1
 (with saturation)
 Subtraction 12.7 sec. 5.5 sec. 2.3:1
 (with starvation)



Conclusions


The benchmarks, as usual, don't tell the whole story. The 80386 performed
poorly mainly because of graphics memory bandwidth limitations. In a PC AT,
the graphics board connects to the 80386 through the PC AT bus, which runs at
1OMHz at best and is only 16 bits wide. Thus, the PC AT bus presents a
bottleneck to a 20-MHz 80386. The 82786 accesses graphics memory directly,
without bottlenecks, plus DRAM tricks such as "page mode" accesses are used to
improve graphics memory bandwidth even further. The 82786 graphics memory
bandwidth peaks at 40 Mbytes/sec.
Using a graphics coprocessor can have many other benefits. Because the
graphics coprocessor possesses smarts, it can share the processing load. The
key here is parallelism. While the graphics coprocessor is performing a task,
the CPU can be doing something else in parallel. This is not possible with
pure display devices such as VGA, EGA, CGA, and so on, where the CPU must do
all the work. This leads me to the conclusion that even if the algorithm runs
slower on the coprocessor, it may still be advantageous to use the coprocessor
for that task because of parallelism.
In any case, the algorithm discussed here used logical operations to perform
simple arithmetic functions that can be used to add and subtract images with
and without saturation. The performance of these functions executed by
graphics coprocessors was dramatically superior to that of the CPU.


Availability


All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063; or call 800-356-2002 (from inside
Calif.) or 800-533-4372 (from outside Calif.). Please specify the issue number
and format (MS-DOS, Macintosh, Kaypro).


Bibliography


Intel Corp., Intel 82786 Graphics Coprocessor User's Manual. 1987.
Kohavi, Zvi. Switching and Finite Automata Theory. New York: McGraw-Hill,
1978: 138 - 144.
McCluskey, Edward J. Logic Design Principles. Englewood Cliffs, N.J.:
Prentice-Hall, 1986.
How Things Work, Volume I. New York: Simon & Schuster: 202 - 203.


_IMAGE MATHEMATICS_
by Victor Duvanenko



[LISTING ONE]

/* Created by Victor J. Duvanenko 10-17-88
 Image addition using the CPU/80386. The program reads a buffer full of
 pixels from two images, adds their pixels together and stores the result
 back into the second image.
*/

#include "cmdtypes.h"
#include <stdio.h>
#include <setjmp.h>
#include "const.h"
#include <fcntl.h>

#define DEBUG 0
#define ADD 1
#define DRAW 0

extern int _fmode; /* file mode */


/* CMD configuration parameters: */
unsigned memseg=0xD000; /* memory segment */
int pbase=0x208; /* port base */

/* entire bitmap expressed as a RECT: */
RECT screen={0,0,XRES-1,YRES-1};

POINT vertices[3];

extern jmp_buf errjmp;
int frame_grab=0; /* frame/field grab */
int hiquality=0; /* use high quality image reduction routines? */

/* current video parameters */
int hue=101;
int sat=111;
int con=123;
int bri=40;

/* Red, green and blue buffers */
unsigned char red_1[ 640 * 13 ],
 red_2[ 640 * 13 ],
 green_1[ 640 * 13 ],
 green_2[ 640 * 13 ],
 blue_1[ 640 * 13 ],
 blue_2[ 640 * 13 ];

/*-------------------------------------------------------------------
 Invocation: add_blt
---------------------------------------------------------------------*/
main(argc,argv)
int argc;
char *argv[];
{
register i, tmp, line;
RECT image1;
int saturate;
unsigned char *r1_p, *r2_p, *g1_p, *g2_p, *b1_p, *b2_p;

 cmd_init( 0xD000, 0x208 );
 cmd_display( 2 );

#if DRAW
 cmd_clear( 1 );
 cmd_clear( 2 );
 cmd_clear( 3 );
 cmd_text_fgcolor( 127, 127, 127 );
 cmd_text_bgcolor( 0, 0, 0 );
 cmd_line_attrib( 0, 0xffff, 127, 127, 127 );
 cmd_write_string( 1, 10, 1, "First bitmap." );
 cmd_line( 1, 639,0, 0,479 );
 cmd_line( 1, 0,0, 639,479 );
 cmd_write_string( 2, 20, 1, "Second bitmap." );
 cmd_line( 2, 639,0, 0,479 );
 cmd_line( 2, 0,0, 639,479 );
#endif

 saturate = TRUE;


 for( line = 0; line < 480; line += 13 )
 {
#if ADD
 r1_p = red_1;
 r2_p = red_2;
 g1_p = green_1;
 g2_p = green_2;
 b1_p = blue_1;
 b2_p = blue_2;

#endif
 /* Perform the same operation using the CPU - 80386 in our case. */
 image1.llx = 0;
 image1.lly = line;
 image1.urx = 639;
 image1.ury = line + 12;
 cmd_read_area( 1, &image1, red_1, green_1, blue_1 );
 cmd_read_area( 2, &image1, red_2, green_2, blue_2 );

#if ADD
 /* Add all of the pixels together - use pointers for speed. */
 if ( !saturate )
 for( i = 0; i < 640 * 13; i++ )
 {
 *r2_p++ += *r1_p++;
 *g2_p++ += *g1_p++;
 *b2_p++ += *b1_p++;
 }
 else
 for( i = 0; i < 640 * 13; i++, r1_p++, g1_p++, b1_p++ )
 {
 tmp = (int)( *r2_p ) + (int)( *r1_p );
 if ( tmp > 255 ) tmp = 255;
 *r2_p++ = (unsigned char)( tmp );

 tmp = (int)( *g2_p ) + (int)( *g1_p );
 if ( tmp > 255 ) tmp = 255;
 *g2_p++ = (unsigned char)( tmp );

 tmp = (int)( *b2_p ) + (int)( *b1_p );
 if ( tmp > 255 ) tmp = 255;
 *b2_p++ = (unsigned char)( tmp );
 }
#endif

 /* Place them back into bitmap 2 */
 cmd_write_area( 2, &image1, red_2, green_2, blue_2, 0 );
 }

 return( 0 );

} /* end main */







[LISTING TWO]

/*-------------------------------------------------------------------
 Procedure that is an extension to 'cmd_copy_image'. It performs
 the image copy with an ADD operation.
 This procedure figures out which of the 3 bitmaps is not being used
 for the operation and uses it for scratch space. The effected area
 is at the same location as the destination. The user has to be aware
 of this.
---------------------------------------------------------------------*/
static void add_blt( from_bm, image, to_bm, to_x, to_y, saturate )
int from_bm;
RECT *image;
int to_bm,
 to_x,
 to_y,
 saturate; /* 1 - saturate, 0 - don't saturate */
{
 register i;
 int scratch_bm, one_used, two_used, three_used;
 unsigned int mask;
 RECT to_image,
 to_image1; /* 1 bpp coordinates */

 /* Figure out which bitmap is not used in the operation and use it */
 /* for scratch space. */
 one_used = two_used = three_used = 0;
 if ( from_bm == 1 to_bm == 1 )
 one_used = 1;
 if ( from_bm == 2 to_bm == 2 )
 two_used = 1;
 if ( from_bm == 3 to_bm == 3 )
 three_used = 1;
 if ( !one_used )
 scratch_bm = 1;
 else if ( !two_used )
 scratch_bm = 2;
 else
 scratch_bm = 3;

 to_image.llx = to_x;
 to_image.lly = to_y;
 to_image.urx = to_x + ( image->urx - image->llx );
 to_image.ury = to_y + ( image->ury - image->lly );

 /* Adjust X only for 8 bpp to 1 bpp conversion */
 to_image1.llx = to_x << 3;
 to_image1.lly = to_y;
 to_image1.urx = (( to_image.urx + 1 ) << 3 ) - 1;
 to_image1.ury = to_image.ury;

 /* Place a copy of the destination image in scratch space */
 cmd_copy_image( to_bm, &to_image, scratch_bm, to_x, to_y, 0 );

 /* Form the XOR/SUM result, in the destination bitmap */
 cmd_copy_image( from_bm, image, to_bm, to_x, to_y, 1 );

 /* Form the AND/CARRY result, in the scratch bitmap */
 cmd_copy_image( from_bm, image, scratch_bm, to_x, to_y, 2 );


 /* Now bit 0 of all pixels is done */
 /* The sum 0 is already in destination bitmap and carry 0 is in AND */

 /* We now need to form the sum 1, which is the XOR of bit 1 and carry 0 */
 /* This requires bit 1 of destination bitmap to be XOR'ed with bit 0 of */
 /* the AND/CARRY bitmap. The result is placed in the destination. */

 /* This can not be accomplished at 8 bpp, as all operations happen on */
 /* pixel boundaries - 8 bits. Therefore, from this point forward we */
 /* must treat our bitmap as 1 bpp, since this allows single bit */
 /* manipulation. */

 for( i = 1, mask = 0x0202; i < 8; i++, mask <<= 1 )
 {
 /* Form the sum of bit i. */;
 cmd_wm_copy_image( scratch_bm, &to_image1, to_bm, to_image1.llx - 1,
to_image1.lly, 1, mask, 1 );

 if (( mask == 0x8080 ) && ( !saturate ))
 break; /* don't form the cary - done! */

 /* Form the carry bit i by first using the carry in and AND'ing */
 /* it with !SUM */
 cmd_wm_copy_image( to_bm, &to_image1, scratch_bm, to_image1.llx + 1,
to_image1.lly, 6, mask >> 1, 1 );

 /* Form the OR part of the carry bit i. */
 cmd_wm_copy_image( scratch_bm, &to_image1, scratch_bm, to_image1.llx - 1,
to_image1.lly, 3, mask, 1 );
 }
#if 0
 /* the old method - took 8 blits */
 /* Saturate the color based on the carry(7). */
 if ( saturate )
 for ( i = 7, mask = 0x0101; i >= 0; i--, mask <<= 1 )
 cmd_wm_copy_image( scratch_bm, &to_image1, to_bm, to_image1.llx + i,
to_image1.lly, 3, mask, 1 );
#endif
 /* new and faster algorithm - takes only 4 blits */
 if ( saturate )
 {
 /* replicate carry[7] throughout the whole pixel of scratch space */
 cmd_wm_copy_image( scratch_bm, &to_image1, scratch_bm, to_image1.llx + 1,
to_image1.lly, 0, 0x4040, 1 );
 cmd_wm_copy_image( scratch_bm, &to_image1, scratch_bm, to_image1.llx + 2,
to_image1.lly, 0, 0x3030, 1 );
 cmd_wm_copy_image( scratch_bm, &to_image1, scratch_bm, to_image1.llx + 4,
to_image1.lly, 0, 0x0f0f, 1 );

 /* saturate the image all bits at once */
 cmd_wm_copy_image( scratch_bm, &to_image1, to_bm, to_image1.llx,
to_image1.lly, 3, 0xffff, 1 );
 }

 /* Restore modified bitmaps to 8 bpp */
 set_gpbitmap_bpp( to_bm, 8 );
 set_gpbitmap_bpp( scratch_bm, 8 );
}
/*-------------------------------------------------------------------
 Procedure that is an extension to 'cmd_copy_image'. It performs
 the image copy with an ADD operation.
 This procedure figures out which of the 3 bitmaps is not being used
 for the operation and uses it for scratch space. The effected area
 is at the same location as the destination. The user has to be aware
 of this.
---------------------------------------------------------------------*/

static void sub_blt( from_bm, image, to_bm, to_x, to_y, saturate )
int from_bm;
RECT *image;
int to_bm,
 to_x,
 to_y,
 saturate; /* 1 - saturate, 0 - don't saturate */
{
 register i;
 int scratch_bm, one_used, two_used, three_used;
 unsigned int mask;
 RECT to_image,
 to_image1; /* 1 bpp coordinates */

 /* Figure out which bitmap is not used in the operation and use it */
 /* for scratch space. */
 one_used = two_used = three_used = 0;
 if ( from_bm == 1 to_bm == 1 )
 one_used = 1;
 if ( from_bm == 2 to_bm == 2 )
 two_used = 1;
 if ( from_bm == 3 to_bm == 3 )
 three_used = 1;
 if ( !one_used )
 scratch_bm = 1;
 else if ( !two_used )
 scratch_bm = 2;
 else
 scratch_bm = 3;

 to_image.llx = to_x;
 to_image.lly = to_y;
 to_image.urx = to_x + ( image->urx - image->llx );
 to_image.ury = to_y + ( image->ury - image->lly );

 /* Adjust X only for 8 bpp to 1 bpp conversion */
 to_image1.llx = to_x << 3;
 to_image1.lly = to_y;
 to_image1.urx = (( to_image.urx + 1 ) << 3 ) - 1;
 to_image1.ury = to_image.ury;

 /* Place a copy of the destination image in scratch space */
 cmd_copy_image( to_bm, &to_image, scratch_bm, to_x, to_y, 0 );

 /* Form the XOR/SUM result, in the destination bitmap */
 cmd_copy_image( from_bm, image, to_bm, to_x, to_y, 1 );

 /* Form the AND/CARRY result, in the scratch bitmap */
 cmd_copy_image( from_bm, image, scratch_bm, to_x, to_y, 6 );

 /* Now bit 0 of all pixels is done */
 /* The sum 0 is already in destination bitmap and carry 0 is in AND */

 /* We now need to form the sum 1, which is the XOR of bit 1 and carry 0 */
 /* This requires bit 1 of destination bitmap to be XOR'ed with bit 0 of */
 /* the AND/CARRY bitmap. The result is placed in the destination. */

 /* This can not be accomplished at 8 bpp, as all operations happen on */
 /* pixel boundaries - 8 bits. Therefore, from this point forward we */

 /* must treat our bitmap as 1 bpp, since this allows single bit */
 /* manipulation. */

 for( i = 1, mask = 0x0202; i < 8; i++, mask <<= 1 )
 {
 /* Form the sum of bit i. */;
 cmd_wm_copy_image( scratch_bm, &to_image1, to_bm, to_image1.llx - 1,
to_image1.lly, 1, mask, 1 );

 if (( mask == 0x8080 ) && ( !saturate ))
 break; /* don't form the cary - done! */

 /* Form the carry bit i by first using the carry in and AND'ing */
 /* it with !SUM */
 cmd_wm_copy_image( to_bm, &to_image1, scratch_bm, to_image1.llx + 1,
to_image1.lly, 2, mask >> 1, 1 );

 /* Form the OR part of the carry bit i. */
 cmd_wm_copy_image( scratch_bm, &to_image1, scratch_bm, to_image1.llx - 1,
to_image1.lly, 3, mask, 1 );
 }
#if 0
 /* Saturate the color based on the borrow(7). */
 if ( saturate )
 for ( i = 7, mask = 0x0101; i >= 0; i--, mask <<= 1 )
 cmd_wm_copy_image( scratch_bm, &to_image1, to_bm, to_image1.llx + i,
to_image1.lly, 6, mask, 1 );
#endif
 /* new and faster algorithm - takes only 4 blits */
 if ( saturate )
 {
 /* replicate carry[7] throughout the whole pixel of scratch space */
 cmd_wm_copy_image( scratch_bm, &to_image1, scratch_bm, to_image1.llx + 1,
to_image1.lly, 0, 0x4040, 1 );
 cmd_wm_copy_image( scratch_bm, &to_image1, scratch_bm, to_image1.llx + 2,
to_image1.lly, 0, 0x3030, 1 );
 cmd_wm_copy_image( scratch_bm, &to_image1, scratch_bm, to_image1.llx + 4,
to_image1.lly, 0, 0x0f0f, 1 );

 /* saturate the image all bits at once */
 cmd_wm_copy_image( scratch_bm, &to_image1, to_bm, to_image1.llx,
to_image1.lly, 6, 0xffff, 1 );
 }

 /* Restore modified bitmaps to 8 bpp */
 set_gpbitmap_bpp( to_bm, 8 );
 set_gpbitmap_bpp( scratch_bm, 8 );
}






















July, 1989
TURBO PASCAL WITH OBJECTS


Combining the OOP approach with structured concepts seems only natural




Michael Floyd


Mike is a technical editor for DDJ and can be contacted at 501 Galveston
Drive, Redwood City, CA 94063. On CompuServe 76703,4057, or on MCI Mail as
MFLOYD.


If artificial intelligence was the buzzword for the mid-1980s, then its
equivalent for the late 1980s (and early 1990s) must be object-oriented
programming (OOP). But instead of the distant promises offered by AI, OOP
represents a superior method for the design and construction of today's
software.
Because OOP's approach encourages good programming practices that complement
structured programming, combining Pascal and OOP seems a natural evolutionary
step, one that Borland has taken with its most recent release of Turbo Pascal.
In this article, I'll explore the major features that have been added to Turbo
Pascal (TP) 5.5, including object-oriented extensions to the language and to
the debugger. We'll also examine TP's overlay manager and take a look at the
new capabilities of the Smart Linker. Finally, we'll provide a full-blown
example that demonstrates most of TP 5.5's object-oriented features.


Language Overview


Turbo Pascal has added only four new keywords to the language set: object,
virtual, constructor, and destructor. With these four keywords, you can
declare objects statically within your program or create them dynamically at
run time. These objects can inherit code and data from a parent (or ancestor)
object type, and you can override any inherited method. Constructors and
destructors provide automated initialization and cleanup of objects. In
addition, constructors and destructors can be used in conjunction with the
expanded New and Dispose procedures to allocate and deallocate heap storage
for dynamic objects.
TP 5.5 supports the notion of virtual methods (as does C++). Virtual methods
allow you to implement polymorphism. Both TP's Integrated Debugger and Turbo
Debugger 1.5 provide object support. Additionally, as previously mentioned,
enhancements have been made to TP's overlay manager and the Smart Linker and
improvements have been made to speed up the compiler.


Syntax


In Turbo Pascal, objects can be thought of as records that have the ability to
inherit code and data from ancestor object types. The generalized form for an
object definition is shown in Example 1. Defining an object is similar to
defining a record. The object keyword replaces the record keyword, and object
takes an optional argument -- the ancestor that this new object will inherit
from.
Example 1: The general form for an object definition

 type
 ObjectName = object (Ancestor)
 variable definitions;
 method definitions; {virtual}
 end;

Methods in TP are simply procedure and function headers placed directly in the
object definition. The virtual keyword (which is optional) is used to define a
virtual method. Methods not using the virtual keyword are static methods.
As an example, consider a windowing environment that contains a menu bar,
pull-down menus and a text window that, in turn, contains other windows such
as scroll bars and resize boxes. Window systems are a natural for an
object-oriented programming approach.
First, let's create the basic window object type like that coded in Example 2.
From this basic window, we can create an object hierarchy of more specialized
windows like the MenuBar and PullDown object types shown in Example 3 .
Example 2: Code for a basic window object

 type
 Window = object
 WindowNo: Integer;
 X1, Y1, X2, Y2: Integer;
 ...
 constructor Init (XA, YA, XB, YB: Integer);
 destructor Done; virtual;
 procedure Show; virtual;
 procedure Hide; virtual;
 ...
 end;

Example 3: Code-fragment to create specialized windows

 type

 MenuList = string [80];

 MenuBar = object (Window)
 Menus: MenuList;
 ...
 constructor Init (M: MenuList);
 procedure Show; virtual;
 procedure Hide; virtual;
 procedure Highlight (Item: Integer);
 procedure Select (Item: Integer);
 ...
 end;

 PullDown = object (Window)
 ...
 end;

Notice that MenuBar inherits three virtual methods (Done, Show, and Hide) and
then adds two static methods (Highlight and Select). Also notice that the
constructors for each of these object types are different as well. Each method
is like a forward declared procedure that must be defined in the same unit or
program module. The syntax for defining a method is ObjectType.Method and is
illustrated in Example 4.
Example 4: The Draw Window method for the Window object

 ...

 procedure Window.Show;
 begin
 { Draw the window }
 end;

 var
 AWin : Window;

 begin
 AWin.Show;
 ...
 end.

You invoke an object's method in the same manner that you would reference the
fields of a record using the familiar dot syntax. Refer to Example 4 for an
example of how to call the Window object's Show method.
TP objects are just like any other variable -- they can be declared statically
in a VAR declaration or dynamically allocated on the heap and referenced via
pointers.
I've left out a lot of the details in this example so that you can easily see
the overall structure and calling sequence. I've also provided an example
program that highlights the creation and use of objects in TP 5.5. FDEMO.PAS
(see Listing One) implements a simple forms editor.
The fact that objects can be used across units is important, particularly as
you begin developing your own libraries. Therefore, the example consists of
two units and a main. FDEMO, as previously mentioned, contains the main
program. FORMS.PAS (see Listing Two) is a unit that implements a Form object
and its associated Fields, and SLIDERS.PAS (see Listing Three) adds a graphic
counter feature.
Because of space considerations, I won't go into detail on the example. You
should, however, find the forms demo instructive as you read through the next
section.


Virtual Methods


There are two required features for any object-oriented programming language
and TP 5.5 has both. The first, which you've already seen, is inheritance. The
other is support for late binding of virtual methods and polymorphism.
From an implementation standpoint, the major difference between static and
virtual methods is simple when they are bound: A static method call is just a
special procedure call that can be resolved at compile-time for maximum
efficiency (early binding).
A virtual method call, on the other hand, requires a table lookup based on the
RUN-TIME identity of the object. This table lookup gives an object the ability
to respond appropriately to a requested action (late binding).
To illustrate this important idea, consider an example where a linked list of
windows can contain either menu windows or text windows. This allows a display
routine to traverse the linked list and send a "display message" to each
window:
 NextWin := Head;
 while NextWin <> nil do
 begin
 NextWin^.Show;
 NextWin := NextWin^.Next;
 end;
To resolve a virtual method call at runtime, TP creates one Virtual Method
Table (VMT) in the data segment for each object type. The VMT contains the
size of an object type, and a pointer to the code of each of the methods for
that object. An instance of an object is linked to the VMT through a
constructor call.
Defining a method as virtual is a simple matter of appending the virtual
keyword to the method's header. However, there are a few rules to keep in
mind.
If you declare a method as virtual in an ancestor type, you must also declare
any overriding methods as virtual. In addition, the parameters for overriding
methods must be exactly the same, including the number of parameters and the
parameter types. This is different than the case for static methods, which can
vary, both the number of parameters and the parameter type.
Also, as mentioned earlier, each instance of an object must be linked to its
VMT so every newly instantiated object must be initialized by a constructor
call. Failure to make the constructor call can result in a system crash. You
can detect this error by setting the $R+ compiler directive. This compiler
directive provides range checking for all virtual methods called in your
program, and issues a run-time error if the object's VMT pointer has not been
initialized.

Objects are structured types that generally require some initialization before
they are used and some cleanup before they are disposed of. A typical sequence
of instructions would be
1. Allocate an object on the heap
2. Call its "initialization" method and pass in relevant parameters for
storage or computation inside the object
3. Use the object (repeat until done with object)
4. Call its "cleanup" method to close files, release any dynamic memory used
by the object, and so on
5. Deallocate the object
The standard procedures New and Dispose already perform the first and last
operations described above. TP's new constructor and destructor are intended
to perform steps 2 and 4 respectively. Because the ordering of these steps is
important, New has been extended to take a constructor method call as an
optional second parameter. Similarly, Dispose accepts a destructor method call
as a second parameter.


Debugger Support


Turbo Pascal provides support for debugging objects as well. The integrated
debugger, for instance, allows you to either step over or trace through method
calls. Because the debugger is actually executing the code in your program,
there's no difference between tracing a static or virtual method.
The Integrated Debugger allows you to view objects using the Evaluate Window.
You can view just the object's data fields or you can view the address of the
method's code. In addition, an object can be added to the Watch window. In
both the Watch and the Evaluate windows, all expressions that are valid for
records are also valid for objects.
Finally, the Integrated Debugger allows you to enter expressions into the Find
Procedure command (available from the Debug menu). A legal expression must
evaluate to an address in the code segment.
The stand-alone debugger, Turbo Debugger (TD) 1.5, has everything in the
Integrated Debugger plus an object inspector (for both object types and
instances) and a hierarchy browser. Again, there's no difference between
tracing a static or virtual method because the debugger is actually executing
the code in your program. This brings up another important point. Methods can
be executed from within the Turbo Debugger environment.
The best way to get a feel for the objects in your program and explore the
features of TD 1.5 at the same time is to load your program up into TD and
crank up the hierarchy browser. The hierarchy browser brings up a two-paned
window that displays an alphabetical list of objects used in the left pane,
and the ancestor/descendent relationships between objects in the right pane.
The hierarchy browser is shown in Figure 1.
The hierarchy browser allows you to scroll through a graphical representation
of the hierarchy and highlight object types. TD also provides an incremental
search feature that allows you to quickly locate object types in a complex
hierarchy. Once you've highlighted an object type, you can select it by
pressing Enter, which brings up the Object Type Inspector.
The Object Type Inspector (see Figure2) is also a two-paned window that
displays the data fields of an object type in the upper pane and its
associated methods in the lower pane. Highlighting and selecting a data field
brings up another Object Type Inspector for browsing complex or nested object
structures. Highlighting and selecting a method, on the other hand, brings up
a Method Inspector.
This Method Inspector displays the method's code address, the names and types
of the method's parameters, and whether it is a procedure or a function.
Pressing Enter anywhere in the Method Inspector takes you to the source code
for that method.
TD also provides an Object Instance Inspector that allows you to examine the
data of object instances. The Object Instance Inspector, which is similar to
the record inspector, adds a new feature that displays an instance's methods
along with their associated code addresses. Of course, these addresses take
polymorphic objects and their VMT into account.


Overlay Manager


One of the more interesting parts of TP is the overlay manager. Until now, the
internals of the overlay manager have largely been undocumented. TP's
optimized Least Recently Used (LRU) algorithm provides a major step in
optimizing the way overlays are loaded into memory. Before describing this new
optimization, however, let's take a look at how TP 5.0's overlay manager
works.
When an overlay is called, it is loaded into a section of memory between the
stack segment and the heap called the overlay buffer. This buffer can be
thought of as a "ring" buffer that has two pointers -- one pointing to the
beginning of the buffer (head pointer) and the other pointing to the end (tail
pointer). Figure 3 illustrates the behavior of the ring buffer and should be
referred to in the following discussion.
Initially, the head and tail pointers point to the same address. When an
overlay is loaded into the overlay buffer, the head pointer advances into the
free memory area and marks the beginning of the overlay. As more overlays are
loaded the free area is eventually taken up. Typically, some free space
remains, but not enough to accommodate the size of the overlay.
At this point, when another overlay is called, the head pointer wraps around
to the bottom of the overlay buffer keeping the free area between the head and
tail pointers. The new overlay is then loaded at the head, which slides
everything up in the buffer and bumps the first overlay off the ring. This is
referred to as the least recently loaded method.
One possible problem with the least recently loaded method is that the ring
buffer doesn't take frequency of use into account. So, it is possible for a
less frequently used overlay to replace one that is used more often simply
because it was loaded last. TP 5.5 provides an option to optimize the loading
of overlays by intelligently selecting which overlay gets bumped off the ring.
TP accomplishes this selection by placing on "probation" overlays nearing the
tail. While on probation, the overlay manager monitors and traps any calls
made to the overlay. If a call is made, the overlay is placed at the head of
the overlay buffer and is given a free ride around the ring.
The overlay manager adds two new routines to get and set the size of the
probation area. The size of the probation area will depend largely on your
application. Therefore, the OvrLoadCount and OvrTrapCount variables are
provided to monitor how often an overlay has been loaded or trapped. By
placing these variables in the Watch window of the debugger, for instance, you
can monitor the effect of different probation sizes on your program.
The overlay manager provides a couple of other goodies, such as OvrFileMode,
to get and set an overlay's file mode. This is particularly useful in a
network environment. Another variable, OvrReadBuf, allows you to intercept
overlay load operations for error handling, or to check for a removable disk.
Finally, TP 5.5 allows you to append overlays to the end of your.EXE files.


Smart Linker


TP 5.0 included a built-in linker that removes code and data not actually
referenced in the program. In building an .EXE file, the smart linker removed
code on a per procedure basis, and removed data on a per declaration basis.
TP 5.5 extends this capability to objects. In particular, the smart linker
removes code for static methods on a per method basis, meaning that if your
program never calls a particular static method, the code for that method will
not be included in the .EXE file. Virtual methods of a given object type,
however, are treated as a single group by the linker. If your program ever
instantiates an object of a type that contains any virtual methods, all of
them will be linked into the .EXE.
The benefits of the smart linker with objects will become particularly
apparent as you build your own libraries of standard objects. You'll be able
to link these object libraries (as units) with your programs knowing that the
smart linker will strip out any unused objects.


Conclusion


One of the difficulties with AI was the tremendous learning curve that
programmers had to go through. First you had to learn a language like Lisp or
Prolog. Next, you had to become familiar with concepts like expert system
design, neural networks, natural language processing, and the like.
TP 5.5, on the other hand, provides an excellent migration path to OOP. Old
code runs fine under the new compiler, and the OOP approach works well with
structured concepts. You can, therefore, add objects as you need them and
alleviate the need to learn everything at once. Ultimately, you'll gain the
benefits of objects including flexibility, reusability, and extensibility.

Example 1. An object definition

type
 ObjectName = object(Ancestor)
 variable definitions;
 method definitions; {virtual}
 end;




Example 2. Code for a basic window

type

 Window = object
 WindowNo : Integer;

 procedure DrawWindow(RowX, ColY, WHeight, WLen : Integer);
 procedure RemoveWindow(WindoNo);
 end;



Example 3. Code to create specialized windows.

 MenuList = string;
 MenuBar = object(Window)

 procedureDrawWindow(RowX,ColY,WHeight,WLen:Integer;
 List:MenuList);
 procedure Highlight(Item : Integer);
 procedure MenuSelect(Item : Integer);
 { etc... }
 end;

 Pulldown = object(Window)
 { pulldown methods }
 end;



_TURBO PASCAL WITH OBJECTS_
by Michael Floyd



[LISTING ONE]


program FDemo;

uses Crt, Forms, Sliders;

type

 Person = record
 Firstname: string[30];
 Lastname: string[30];
 Address: string[32];
 City: string[16];
 State: string[2];
 Zipcode: Longint;
 Counter: array[1..3] of Longint;
 Slider: array[1..2] of Integer;
 end;

const


 Frank: Person = (
 Firstname: 'Frank';
 Lastname: 'Borland';
 Address: '1800 Green Hills Road';
 City: 'Scotts Valley';
 State: 'CA';
 Zipcode: 95066;
 Counter: (10, 1000, 65536);
 Slider: (85, 25));

var
 F: Form;
 P: Person;

begin
 Color(BackColor);
 ClrScr;
 Color(ForeColor);
 GotoXY(1, 1); ClrEol;
 Write(' Turbo Pascal 5.5 Object Oriented Forms Editor');
 GotoXY(1, 25); ClrEol;
 Write(' F2-Save Esc-Quit');
 F.Init(10, 5, 54, 16);
 F.Add(New(FStrPtr, Init(3, 2, ' Firstname ', 30)));
 F.Add(New(FStrPtr, Init(3, 3, ' Lastname ', 30)));
 F.Add(New(FStrPtr, Init(3, 5, ' Address ', 32)));
 F.Add(New(FStrPtr, Init(3, 6, ' City ', 16)));
 F.Add(New(FStrPtr, Init(25, 6, ' State ', 2)));
 F.Add(New(FZipPtr, Init(34, 6, ' Zip ')));
 F.Add(New(FIntPtr, Init(3, 8, ' Counter 1 ', 0, 99999999)));
 F.Add(New(FIntPtr, Init(22, 8, ' 2 ', 0, 99999999)));
 F.Add(New(FIntPtr, Init(33, 8, ' 3 ', 0, 99999999)));
 F.Add(New(FSliderPtr, Init(3, 10, ' Slider One ', 0, 100, 5)));
 F.Add(New(FSliderPtr, Init(3, 11, ' Slider Two ', 0, 100, 5)));
 P := Frank;
 F.Put(P);
 F.Show;
 if F.Edit = CSave then F.Get(P);
 F.Done;
 NormVideo;
 ClrScr;
 WriteLn('Resulting Person record:');
 WriteLn;
 with P do
 begin
 WriteLn('Firstname: ', Firstname);
 WriteLn(' Lastname: ', Lastname);
 WriteLn(' Address: ', Address);
 WriteLn(' City: ', City);
 WriteLn(' State: ', State);
 WriteLn(' Zipcode: ', Zipcode);
 WriteLn(' Counters: ', Counter[1], ' ', Counter[2], ' ', Counter[3]);
 WriteLn(' Sliders: ', Slider[1], ' ', Slider[2]);
 end;
end.







[LISTING TWO]

unit Forms;

{$S-}

interface

uses Crt;

const

 CSkip = ^@;
 CHome = ^A;
 CRight = ^D;
 CPrev = ^E;
 CEnd = ^F;
 CDel = ^G;
 CBack = ^H;
 CSave = ^J;
 CUndo = ^R;
 CLeft = ^S;
 CClear = ^Y;
 CNext = ^X;
 CQuit = ^[;

type

 FStringPtr = ^FString;
 FString = string[79];

 FieldPtr = ^Field;
 Field = object
 Next: FieldPtr;
 X, Y, Size: Integer;
 Title: FStringPtr;
 Value: Pointer;
 constructor Init(PX, PY, PSize: Integer; PTitle: FString);
 destructor Done; virtual;
 procedure Beep; virtual;
 function Edit: Char; virtual;
 function ReadChar: Char; virtual;
 procedure Show; virtual;
 function Prev: FieldPtr;
 end;

 FTextPtr = ^FText;
 FText = object(Field)
 Len: Integer;
 constructor Init(PX, PY, PSize: Integer; PTitle: FString;
 PLen: Integer);
 function Edit: Char; virtual;
 procedure GetStr(var S: FString); virtual;
 function PutStr(var S: FString): Boolean; virtual;
 procedure Show; virtual;
 procedure Display(var S: FString);
 end;


 FStrPtr = ^FStr;
 FStr = object(FText)
 constructor Init(PX, PY: Integer; PTitle: FString; PLen: Integer);
 procedure GetStr(var S: FString); virtual;
 function PutStr(var S: FString): Boolean; virtual;
 end;

 FIntPtr = ^FInt;
 FInt = object(FText)
 Min, Max: Longint;
 constructor Init(PX, PY: Integer; PTitle: FString;
 PMin, PMax: Longint);
 procedure GetStr(var S: FString); virtual;
 function PutStr(var S: FString): Boolean; virtual;
 end;

 FZipPtr = ^FZip;
 FZip = object(FInt)
 constructor Init(PX, PY: Integer; PTitle: FString);
 procedure GetStr(var S: FString); virtual;
 function PutStr(var S: FString): Boolean; virtual;
 end;

 FormPtr = ^Form;
 Form = object
 X1, Y1, X2, Y2: Integer;
 Last: FieldPtr;
 constructor Init(PX1, PY1, PX2, PY2: Integer);
 destructor Done; virtual;
 function Edit: Char; virtual;
 procedure Show; virtual;
 procedure Add(P: FieldPtr);
 function First: FieldPtr;
 procedure Get(var FormBuf);
 procedure Put(var FormBuf);
 end;

 ColorIndex = (BackColor, ForeColor, TitleColor, ValueColor);

procedure Color(C: ColorIndex);

implementation

type

 Bytes = array[0..32767] of Byte;

procedure Abstract(Method: String);
begin
 WriteLn('Error: Call to abstract method ', Method);
 Halt(1);
end;

{ Field }

constructor Field.Init(PX, PY, PSize: Integer; PTitle: FString);
begin
 X := PX;

 Y := PY;
 Size := PSize;
 GetMem(Title, Length(PTitle) + 1);
 Title^ := PTitle;
 GetMem(Value, Size);
 FillChar(Value^, Size, 0);
end;

destructor Field.Done;
begin
 FreeMem(Value, Size);
 FreeMem(Title, Length(Title^) + 1);
end;

procedure Field.Beep;
begin
 Sound(500); Delay(25); NoSound;
end;

function Field.Edit: Char;
begin
 Abstract('Field.Edit');
end;

function Field.ReadChar: Char;
var
 Ch: Char;
begin
 Ch := ReadKey;
 case Ch of
 #0:
 case ReadKey of
 #15, #72: Ch := CPrev; { Shift-Tab, Up }
 #60: Ch := CSave; { F2 }
 #71: Ch := CHome; { Home }
 #75: Ch := CLeft; { Left }
 #77: Ch := CRight; { Right }
 #79: Ch := CEnd; { End }
 #80: Ch := CNext; { Down }
 #83: Ch := CDel; { Del }
 else
 Ch := CSkip;
 end;
 #9, #13: Ch := CNext; { Tab, Enter }
 end;
 ReadChar := Ch;
end;

procedure Field.Show;
begin
 Abstract('Field.Show');
end;

function Field.Prev: FieldPtr;
var
 P: FieldPtr;
begin
 P := @Self;
 while P^.Next <> @Self do P := P^.Next;

 Prev := P;
end;

{ FText }

constructor FText.Init(PX, PY, PSize: Integer; PTitle: FString;
 PLen: Integer);
begin
 Field.Init(PX, PY, PSize, PTitle);
 Len := PLen;
end;

function FText.Edit: Char;
var
 P: Integer;
 Ch: Char;
 Start, Stop: Boolean;
 S: FString;
begin
 P := 0;
 Start := True;
 Stop := False;
 GetStr(S);
 repeat
 Display(S);
 GotoXY(X + Length(Title^) + P, Y);
 Ch := ReadChar;
 case Ch of
 #32..#255:
 begin
 if Start then S := '';
 if Length(S) < Len then
 begin
 Inc(P);
 Insert(Ch, S, P);
 end;
 end;
 CLeft: if P > 0 then Dec(P);
 CRight: if P < Length(S) then Inc(P) else;
 CHome: P := 0;
 CEnd: P := Length(S);
 CDel: Delete(S, P + 1, 1);
 CBack:
 if P > 0 then
 begin
 Delete(S, P, 1);
 Dec(P);
 end;
 CClear:
 begin
 S := '';
 P := 0;
 end;
 CUndo:
 begin
 GetStr(S);
 P := 0;
 end;
 CSave, CNext, CPrev:

 if PutStr(S) then
 begin
 Show;
 Stop := True;
 end else
 begin
 Beep;
 P := 0;
 end;
 CQuit: Stop := True;
 else
 Beep;
 end;
 Start := False;
 until Stop;
 Edit := Ch;
end;

procedure FText.GetStr(var S: FString);
begin
 Abstract('FText.GetStr');
end;

function FText.PutStr(var S: FString): Boolean;
begin
 Abstract('FText.PutStr');
end;

procedure FText.Show;
var
 S: FString;
begin
 GetStr(S);
 Display(S);
end;

procedure FText.Display(var S: FString);
begin
 GotoXY(X, Y);
 Color(TitleColor);
 Write(Title^);
 Color(ValueColor);
 Write(S, '': Len - Length(S));
end;

{ FStr }

constructor FStr.Init(PX, PY: Integer; PTitle: FString; PLen: Integer);
begin
 FText.Init(PX, PY, PLen + 1, PTitle, PLen);
end;

procedure FStr.GetStr(var S: FString);
begin
 S := FString(Value^);
end;

function FStr.PutStr(var S: FString): Boolean;
begin

 FString(Value^) := S;
 PutStr := True;
end;

{ FInt }

constructor FInt.Init(PX, PY: Integer; PTitle: FString;
 PMin, PMax: Longint);
var
 L: Integer;
 S: string[15];
begin
 Str(PMin, S); L := Length(S);
 Str(PMax, S); if L < Length(S) then L := Length(S);
 FText.Init(PX, PY, 4, PTitle, L);
 Min := PMin;
 Max := PMax;
end;

procedure FInt.GetStr(var S: FString);
begin
 Str(Longint(Value^), S);
end;

function FInt.PutStr(var S: FString): Boolean;
var
 N: Longint;
 E: Integer;
begin
 Val(S, N, E);
 if (E = 0) and (N >= Min) and (N <= Max) then
 begin
 Longint(Value^) := N;
 PutStr := True;
 end else PutStr := False;
end;

{ FZip }

constructor FZip.Init(PX, PY: Integer; PTitle: FString);
begin
 FInt.Init(PX, PY, PTitle, 0, 99999);
end;

procedure FZip.GetStr(var S: FString);
begin
 FInt.GetStr(S);
 Insert(Copy('0000', 1, 5 - Length(S)), S, 1);
end;

function FZip.PutStr(var S: FString): Boolean;
begin
 PutStr := (Length(S) = 5) and FInt.PutStr(S);
end;

{ Form }

constructor Form.Init(PX1, PY1, PX2, PY2: Integer);
begin

 X1 := PX1;
 Y1 := PY1;
 X2 := PX2;
 Y2 := PY2;
 Last := nil;
end;

destructor Form.Done;
var
 P: FieldPtr;
begin
 while Last <> nil do
 begin
 P := Last^.Next;
 if Last = P then Last := nil else Last^.Next := P^.Next;
 Dispose(P, Done);
 end;
end;

function Form.Edit: Char;
var
 P: FieldPtr;
 Ch: Char;
begin
 Window(X1, Y1, X2, Y2);
 P := First;
 repeat
 Ch := P^.Edit;
 case Ch of
 CNext: P := P^.Next;
 CPrev: P := P^.Prev;
 end;
 until (Ch = CSave) or (Ch = CQuit);
 Edit := Ch;
 Window(1, 1, 80, 25);
end;

procedure Form.Show;
var
 P: FieldPtr;
begin
 Window(X1, Y1, X2, Y2);
 Color(ForeColor);
 ClrScr;
 P := First;
 repeat
 P^.Show;
 P := P^.Next;
 until P = First;
 Window(1, 1, 80, 25);
end;

procedure Form.Add(P: FieldPtr);
begin
 if Last = nil then Last := P else P^.Next := Last^.Next;
 Last^.Next := P;
 Last := P;
end;


function Form.First: FieldPtr;
begin
 First := Last^.Next;
end;

procedure Form.Get(var FormBuf);
var
 I: Integer;
 P: FieldPtr;
begin
 I := 0;
 P := First;
 repeat
 Move(P^.Value^, Bytes(FormBuf)[I], P^.Size);
 Inc(I, P^.Size);
 P := P^.Next;
 until P = First;
end;

procedure Form.Put(var FormBuf);
var
 I: Integer;
 P: FieldPtr;
begin
 I := 0;
 P := First;
 repeat
 Move(Bytes(FormBuf)[I], P^.Value^, P^.Size);
 Inc(I, P^.Size);
 P := P^.Next;
 until P = First;
end;

procedure Color(C: ColorIndex);
type
 Palette = array[ColorIndex] of Byte;
const
 CP: Palette = ($17, $70, $30, $5E);
 MP: Palette = ($07, $70, $70, $07);
begin
 if LastMode = CO80 then TextAttr := CP[C] else TextAttr := MP[C];
end;

end.






[LISTING THREE]

unit Sliders;

{$S-}

interface

uses Crt, Forms;


type

 FSliderPtr = ^FSlider;
 FSlider = object(Field)
 Min, Max, Delta: Integer;
 constructor Init(PX, PY: Integer; PTitle: FString;
 PMin, PMax, PDelta: Integer);
 function Edit: Char; virtual;
 procedure Show; virtual;
 procedure Display(I: Integer);
 end;

implementation

constructor FSlider.Init(PX, PY: Integer; PTitle: FString;
 PMin, PMax, PDelta: Integer);
begin
 Field.Init(PX, PY, 2, PTitle);
 Min := PMin;
 Max := PMax;
 Delta := PDelta;
end;

function FSlider.Edit: Char;
var
 I: Integer;
 Ch: Char;
 Stop: Boolean;
begin
 I := Integer(Value^);
 Stop := False;
 repeat
 Display(I);
 GotoXY(X + Length(Title^) + 1, Y);
 Ch := ReadChar;
 case Ch of
 CLeft: if I > Min then Dec(I, Delta);
 CRight: if I < Max then Inc(I, Delta);
 CHome: I := Min;
 CEnd: I := Max;
 CUndo: I := Integer(Value^);
 CSave, CQuit, CNext, CPrev: Stop := True;
 else
 Beep;
 end;
 until Stop;
 if Ch <> CQuit then Integer(Value^) := I;
 Edit := Ch;
end;

procedure FSlider.Show;
begin
 Display(Integer(Value^));
end;

procedure FSlider.Display(I: Integer);
var
 Steps: Integer;

 S: FString;
begin
 Steps := (Max - Min) div Delta + 1;
 S[0] := Chr(Steps);
 FillChar(S[1], Steps, #176);
 S[(I - Min) div Delta + 1] := #219;
 GotoXY(X, Y);
 Color(TitleColor);
 Write(Title^);
 Color(ValueColor);
 Write(' ', Min, ' ', S, ' ', Max, ' ');
end;

end.
















































July, 1989
GETTING THE BUGS OUT WITH TURBO DEBUGGER


We all make mistakes; Borland's Turbo Debugger can make finding programming
mistakes fast and almost fun




Bill Catchings and Mark L. Van Name


Bill Catchings and Mark L. Van Name are free-lance writers and independent
computer consultants based in Raleigh, N.C.


The Turbo Debugger (TD) installation program is easy to use, asks few
questions, and is almost foolproof. The first thing you notice about TD is its
attractiveness, especially on a VGA monitor. The 50-line VGA display contains
a lot of information, while still remaining readable.


What It Is and What It Isn't


A major part of the screen displays the source code of your assembler, C, or
Pascal main routine. You can debug the source code, but to compile or edit it
you must leave TD. You can call the editor of your choice from within TD as
long as the editor of your choice fits into the memory limitations of running
TD. We called Epsilon while debugging a 3OK-executable file with 6K of source.
Larger editors might not fit in the remaining memory space.
When you return from editing a source file, TD marks the file as modified. To
cause the object code to reflect your changes, you must exit TD and recompile
the source. You can't push to DOS and recompile, because TD doesn't leave
enough memory to hold any of Borland's compilers.


Entering Commands


TD's interface is similar to that of the current Turbo languages. Its basic
display has a menu bar across the top of the screen and a line of command key
labels across the bottom.
You can get to the menu bar by pressing F10. You can also go directly to any
menu by pressing the Alt key in combination with the first letter of the
chosen menu. Once you've entered a menu, you can choose an option by pointing
to it or by entering its first letter.
The line of command key labels initially displays TD's global command key
assignments. Some of these key combinations are shortcut equivalents to menu
choices. If you hold down the Alt key for a couple of seconds the label line
will switch to a second line of global command keys that can be entered by
pressing Alt plus a function key. These two sets of command keys are available
in any TD window.
TD also has local key assignments that vary between windows. To see the local
keys that are currently available, hold down Ctrl for a couple of seconds and
the bottom label line will change to display them. You can trigger these local
commands by pressing Ctrl plus a single mnemonic character; Ctrl-w, for
example, is the watch command.
Many of these commands are context-sensitive and follow an object/verb
metaphor. Move the cursor to an object on the screen and then enter a
command. TD will perform that command on the selected object. This
"point-and-pick" approach cries out for a mouse, but TD, somewhat
surprisingly, will not work with a mouse.
Some of TD's command-key sequences will be familiar if you've used either
Turbo C or Turbo Pascal. Alt-F5, for example, will switch from TD's display to
your program's output screen; Alt-X will exit TD.


Windows and Panes


Between the menu bar and the key labels, TD displays 13 different types of
windows, although it initially displays only two -- the Module and Watch
windows.
Unlike CodeView's limited, tiled windows, TD's windows are the "real thing."
They can be overlapped, moved, and resized, although each window has a fixed
minimum size. The active window has a double-line border and a highlighted
title with every window displaying a number in its upper border.
TD pushes the constraints of the window concept even further by dividing some
windows into smaller units it calls "panes." Panes display different aspects
of a single window, and sometimes even use different local command keys. Panes
cannot be rearranged or explicitly resized, but TD will automatically adjust
them when the window size changes.
TD can also automatically track your cursor location. All of the functions in
TD are context-sensitive. In particular, pressing the F1 key will call up a
help message relevant to the cursor's current location.


The Module Window


Typically, the largest window is TD's initial Module window, which will
display as much of your source code as it can hold. You can move around in the
code with the standard arrow and PageUp/PageDown keys. A triangle on the left
border will mark the line on which execution is currently paused.
Because the Module window is, in many ways, TD's main window, it gives you
access to quite a few of TD's most powerful features. For example, you can
execute your program in several different ways via command keys.
If you want to test a complete run through of your program, press F9 (Run) and
the program will execute until it either completes successfully or bombs. A
more cautious approach can be taken with any of three different single-step
commands.
The "trace into" command (F7) will run your program one source line at a time.
If a line is a call to another routine, this command follows that branch into
the new routine and lets you single-step through the code there. You can also
single-step through a routine without going into any routines it calls with
the step over command (F8). This command treats routine calls as single
instructions.
Another command, animate (Alt-F4), causes TD to single-step continuously,
although you can stop execution at any time by pressing any key. (One negative
note, however, is that TD lacks an undo feature like that of the debugger in
Microsoft QuickC 2.0. TD's undo command (Alt-F6) only allows you to restore a
window you've deleted, not undo the effects of a line of code.) After it has
read each line of code, TD updates all of its windows, and you can sit back
and watch the program run.
You can also instruct TD to run until it executes a return with the until
return command (Alt-F8). This command is useful when you're stuck in a
subroutine and want to get out, as can happen when your code enters a library
routine for which you don't have source code.
You can stop your program at any time by entering Ctrl-Break. This approach is
fine as long as your program has easily identifiable stopping points, such as
requests for input.
Sometimes, though, the finer level of control found with breakpoints is
needed. Although the terminology differs, TD offers all of the breakpoint
facilities of CodeView.
CodeView gives you three different breakpoint options. A breakpoint stops
execution at a given code location. A watchpoint stops the program when an
expression becomes true. Finally, a tracepoint stops the program when it
modifies a specific memory location.
TD unites these three mechanisms into one common breakpoint facility. A TD
breakpoint has four possible parts. The main component is a location. The
location can be either a single line of code (a CodeView breakpoint) or an
indicator that it is global.

The next component is a condition that must be true for execution to stop. For
breakpoints with single locations, this condition can only be "always." For
global breakpoints, you can enter a specific condition.
You can use this condition to stop execution when an expression becomes true
(CodeView's watchpoint). You can write the expression in your choice of C,
Pascal, or assembler; TD supports full expressions in any of these languages.
However, caution must be taken in writing these expressions, because serious
side effects can be caused by the use of such items as i++.
A condition can be used to tell TD to stop when a memory location changes
(CodeView's tracepoint).
A final condition feature lets you stop program execution when a hardware
interrupt occurs. This is a nice capability if you want to use TD in
conjunction with a hardware debugger such as Atron's 386 Source Probe.
The third component of a TD breakpoint is a count, which defaults to 1. This
count tells TD how many times it should let the breakpoint occur before it
takes over.
You can control the action that TD takes at that point with the final
breakpoint component. You can stop TD execution, log the value of an
expression, or even execute another piece of code.
All of the breakpoints that occur in your program and the types of each can be
seen in the Breakpoint's window. This window is summoned via the View menu in
the menu bar, which has an option for each of TD's windows.
The current state of your variables can also be seen by using one of three TD
windows. The first of these is the Watch window, another initial TD window.
Place the cursor on a variable and press Ctrl-w (watch), and the variable will
appear in this window. You can display an unlimited number of variables here.
Any time the value of a variable in this window changes, the new value will
appear.
The Inspector window also keeps track of variables, but a separate window must
be used to display each variable. You can, however, have many Inspector
windows active at once. To bring up an Inspector window, place the cursor on a
variable and press Ctrl-i (inspect). Once this has been done, a single command
(F3) will close all of the active Inspector windows.
The main difference between the variable displays in these two windows is
their appearance. The single Watch window has a necessarily crammed display,
while each Inspector window has more room for its one variable. The Watch
window is handy for simple variables, while an Inspector window's additional
space is better for complex structures.
Both window types understand C structures and Pascal records, as well as other
variable types. This is a definite improvement over CodeView, which lacks this
capability. Consider, for example, the pointers in a linked list of
structures. Inspect the first element. To see the next element, move the
cursor to the next structure pointer and press the Ctrl-i again. You can
easily wander through a linked list.
The third way to examine variables is through the two panes of the Variable's
window. The left pane displays the global variables, while the right shows
your current routine's local variables.
In any of these three windows you can change the value of a variable. Just
position the cursor on the variable, enter Ctrl-c (change), and type a new
value.


Other Key Windows


While breakpoints and variables are important, it's TD's other windows that
provide the necessary information about your program's execution.
The Stack window, for example, shows the routines in the stack form. Its local
commands let you examine any routine in the stack. In a similar manner, you
can look at the local variables for any stack frame.
The User Screen window displays the screen output of your program. If you have
a color monitor that supports multiple pages, TD will use that feature to
speed switching between its display and your program's. You can examine any
ASCII file via TD's File window.


Getting Down to Nuts and Bolts


There are times when a high-level language isn't enough. To get to the
assembly code, you first need to bring up the CPU window. This complex window
has five panes.
The first is the code pane. It can display code in any of three ways: by
showing only assembler and no source code; a display of each source line above
the assembler lines that it generated; or each routine in the language in
which it was written.
One of the CPU window's local commands, Ctrl-c (caller), takes you to the line
of code that called the current routine. TD gets this information from the
stack. If the right stack frame isn't located, TD may become "confused" and
work incorrectly.
Another local command, Ctrl-n (new) CS:IP, lets you start execution at any
line of code, which is a handy way to skip unwanted code.
The second pane is the data pane, which looks a lot like the old DOS DEBUG D
command. It displays lines that show a hex address, a hex data value, and the
ASCII characters for that value. TD goes a step further than DEBUG, providing
you with options that display values as eight hex bytes, four hex words, two
hex long words, or one float. These options can save you the byte-swapping
hassles that plague DEBUG.
TD also solves another of DEBUG's problems. Because DEBUG is the currently
executing program, the screen addresses and interrupt vectors displayed always
show information about DEBUG itself, not about your program. TD shows your
program's screen values and interrupt vectors in its data pane. It has to lie
to do this, of course, because it is the currently executing program, but in
this case fiction is much more useful than truth.
Another nifty feature of the data pane is its local command, Ctrl-F (follow).
This command lets you "walk" down a chain of pointers.
The CPU window's third pane, the Stack pane, displays the top few 32-bit
quantities on the stack. As in the data pane, you can follow pointers on the
stack.
The final two panes are the Registers and Flags panes. The Registers pane
shows all of the registers in either 16- or 32-bit format. The Flags pane
displays all of the flags.
You can change any of the items, including code, in any of these panes. TD
contains a full assembler to process code that you enter although, as we noted
earlier, TD does not allow you to save code changes.
Two more of TD's windows, the Registers and Dump windows, are basically
equivalent to the Registers and Data panes of the CPU window.


Still More Windows


TD also contains a Log window, which provides a handy way to retrace your
steps. The log contains error messages and other information, such as the
output of any breakpoint actions. The log can be kept in this window or stored
in a log file. CodeView lacks such a log function.
TD's final window, the Numeric Coprocessor window, displays the full state of
your PC's math coprocessor chip or emulator. TD will automatically detect
whether you're using an emulator or a chip. If you're using a chip, it will
tell you which kind. This window is great for debugging floating point
assembler code, which is another feature lacking in CodeView.


Wait, There's More!


In addition to all of its windows and commands, TD offers many other useful
features, several of which make it easier for you to perform repetitive tasks.
For example, every TD menu, unlike those found in Turbo C, will remember your
last choice. TD also compiles a history list for every area in which you can
type data. TD will remember the last ten responses for most prompt boxes. You
can move to the one you want, edit it if necessary, and then press Enter to
execute it again.
TD will also let you define keystroke macros, which can contain as many
keystrokes as you want.
If you don't want to lose your carefully chosen window layouts and macros,
this information, as well as other settings, can be saved.


Advanced Debugging


In addition to all of its other features, TD also offers several interesting
options for more advanced debugging environments. Two displays can be hooked
to your PC, allowing you to view TD in one and display your program's output
in the other. This capability is great for debugging highly interactive
applications.

Because TD.EXE is a 17OK-executable program, TD also offers many ways to
conserve memory.
If your computer has EMS memory, you can start by moving TD's symbols into
EMS. TD saves and restores the EMS driver's state in the process, so that
programs that use the driver can still take advantage of this memory-saving
feature. This technique won't recover much memory, but sometimes every little
bit helps.
A more radical approach is to use TD's remote debugging capabilities. To
accomplish this you must run your program on one machine and TD on another. By
connecting the serial ports of the two PCs, TD can "talk" to your program over
that connection at speeds of up to 115K-bits per sec. With this approach, the
only memory you will lose on the PC with your program is the 15K that TD's
communication program, TDREMOTE.EXE, requires. You can also use a TD program
called TDRF.EXE on the debugging PC to transfer files between the two systems.
If you've got a 386, TD can use the 386's virtual mode. TD's 386 version,
TD386.EXE, is a standard part of the package. This program requires 700K of
extended memory and loads above the 1M line. The program you're debugging
loads exactly where it would during normal execution. In addition to saving
memory, this approach is great for finding bugs that are dependent on the
program's position in memory.
TD386.EXE uses a device driver, TDH386.SYS, that works only on Real mode
programs. TD itself offers 386 instructions beyond those restricted to
Protected mode. It can also use the 386's hardware debugging registers for
greater speed.
If your program is close to fitting into memory, but can't quite make it, the
TDPACK.EXE utility may do the trick. It shrinks the debugging information in
an .EXE. When you've finished debugging, that information can be removed with
the TDSTRIP.EXE utility, which allows you to avoid recompiling your program
without the debug switch.


Working with the Competition


Two other useful utilities help TD to work with code from other compilers.
TDMAP.EXE combines a program's .EXE and .MAP (output from Microsoft LINK)
files so that you can do symbolic debugging. In contrast, CodeView can't use
.MAP files, and forces you to compile with full debugging on (-v for C) for
source-level debugging.
TD can work on any file that CodeView can handle, courtesy of the TDCONVRT.EXE
utility that converts files to TD's format. This utility will work on any
executable program, including one produced by Microsoft's Fortran or Basic
compilers. This is one instance where TD falls short of CodeView, as CodeView
can accept input in either language's syntax, while TD is limited to
assembler, C, and Pascal syntax.


A Great Product at a Great Price


We're sold on TD as the current ideal debugging environment on a flat-screen
VGA monitor in 50-line mode, with a 25-MHz 386, with 4 Mbytes of extended
memory running the whole show. If you do much programming, TD may represent
the best $149.95 you can spend.


Product Information


Turbo Debugger 1.0. Requirements: 100 percent IBM-compatible and IBM
PS/2-compatible systems; PC-DOS or MS-DOS 2.0 or later, 384K RAM minimum; hard
disk recommended but not required; will work with any monitor. Price: $149.95
(includes Turbo Assembler). Available options: Comes as part of the Turbo C
Professional and Turbo Pascal Professional packages, each of which costs
$249.95. Current Turbo C and Turbo Pascal users can upgrade to the
Professional packages for $99.95 each. Company information: Borland
International, 1800 Green Hills Road, P.0. Box 660001, Scotts Valley, CA
95066-0001, 408-438-8400.





































July, 1989
FASTER STRING SEARCHES


Boyer-Moore may be the algorithm you need




Costas Menico


Costas Menico is a senior software developer and part owner of The Software
Bottling Company. He can be reached at 6600 Long Island Expressway, Maspeth,
NY 11378.


While programming several of my company's products, I have many times come
across the need to perform a string search to accomplish a certain task. A
string search is the function used to find a string A (the pattern) in a
string B (the text) and return the position of A in B.
After realizing that the built-in string search functions of many language
compilers used simple brute-force algorithms, I set out to find and implement
an equivalent string-search function that would perform at a fraction of the
speed of the built-in function.


The Brute-Force Method


As the name implies, the brute-force method of performing a string search
involves an exhaustive search from the start of the pattern until there is a
match or the end of the string is reached.
Each character in the pattern is compared left to right with each character in
the string. The scanning is continued until all of the characters in the
pattern match an equivalent length in the string. If no match occurs, the
pattern is moved one character to the right and a new search is begun.
Assume we want to search for the pattern HEAD in the string
"MAXIMOODHEADROOM." The search is shown in Figure 1. The method shown is
time-consuming, because every character in the string must be compared with
every other character until a final match is found. Nine steps were taken to
discover the sample pattern. Using a program that must perform thousands of
searches is a time-consuming way of looking up a pattern.
Figure 1: Brute-force search

 Pattern= HEAD
 String= M A X I M O O D H E A D R O O M
 1) H H Does not match, move right.
 2) H H Does not match, move right.
 3) H H Does not match, move right.
 4) H H Does not match, move right.
 5) H H Does not match, move right.
 6) H H Does not match, move right.
 7) H H Does not match, move right.
 8) H H Does not match, move right.
 9) H E A D HEAD finally matches.



The Boyer-Moore Method


After looking through some impressive (and complex) algorithms, I came across
an elegant and easily implemented algorithm called the Boyer-Moore method,
named for the pair of scientists who invented it. This is a hybrid algorithm
that uses pattern analysis combined with brute force.
To implement the algorithm in assembler, I used MASM and linked it into Turbo
Pascal 4.0. The algorithm is also easily converted to work with C and Basic.
For the assembly language program (called POSBM), see Listing One; for the
benchmark test program see Listing Two.
The basic idea behind the Boyer-Moore technique is the movement of the pattern
more than one character at a time during the search for a matching character.
This saves search steps, thereby increasing search speed. You may wonder how a
whole pattern can be moved to the right without a search through any of the
characters in between. The answer is simple. All that is required is a
256-character array that defines the pattern. This array (called a SKIPARRAY)
is built every time the search function is called. Each position corresponds
to the value of each character in the ASCII character set.
The algorithm is simple. A SKIPARRAY of 256 bytes is filled with the length of
the pattern. Starting from the right of the pattern, each character is read
and its offset placed into the SKIPARRAY, using the character's ASCII value as
the index (see Figure 2).
Figure 2: SKIPARRAY values with the HEAD pattern

 A . . D E . . H . . .. Z
 SKIPARRAY position... 65 66 67 68 69 70 71 72 73 ... 91
 Original values 4 4 4 4 4 4 4 4 4 .... 4
 Values with "HEAD" 1 4 4 0 2 4 4 3 4 .... 4

The pattern is then placed against the string for the search. At the rightmost
position of the pattern the corresponding character in the string is read and
its ASCII value used as an index into the SKIPARRAY. The value in the
SKIPARRAY will determine how far to slide the pattern to the right, with one
exception. A value of 0 means that the rightmost character in the pattern is
lined up with an exact character in the string. Because this is a possible
match, a reverse compare is performed on the remaining characters in the
string (that is, a reverse brute-force compare). If the pattern matches, the
offset is noted in the string. Otherwise, the pattern slides over by one
position and the process is repeated.
To better explain this process I will use as an example the pattern HEAD. The
SKIPARRAY is first filled with the length of the pattern. Once this is
completed the SKIPARRAY will contain 4s in all 256 positions.
Next, we put the skip value for each character in the pattern into the
SKIPARRAY. To do this, the offset of each character is taken from the end of
HEAD and placed at the ASCII offset position in the SKIPARRAY. For the pattern
HEAD, the SKIPARRAY is transformed by putting in the corresponding values of
3, 2, 1, and 0 at the 72 (H), 69 (E), 65 (A) and 68 (D) positions (see Figure
3).

Figure 3: Using Boyer-Moore, we find a match in four steps

 Pattern= H E A D
 String= M A X I M O O D H E A D R O O M
 1) H E A D
 2) H E A D
 3) H E A D
 4) H E A D

The pattern is then scanned right to left to determine how far to move the
pattern for each character in the string. To find the pattern HEAD in
MAXIMOODHEADROOM, HEAD is positioned at the start of the string. This places
the character D of the pattern HEAD under the character I in the string (see
step 1 in Figure 3).
Because there is no match, that is, D is not equal to I, the SKIPARRAY is
indexed at the position of the value for I, which is found to be 4. As I is
not in the pattern, HEAD is positioned four characters to the right of the
mismatched position.
The D in the pattern (see step 2 in Figure 3) is now positioned under the D in
the string. This indicates that the last character of the pattern matches the
corresponding character in the string. We therefore proceed to match the
remaining characters by using a reverse brute-force method. In this case, the
A in the pattern will not match the O in the string. Because this was a
brute-force compare, we can only move the pattern one character to the right.
The D (see step 3 in Figure 3) is now positioned under the H in the string. As
D is not equal to H, we index in to the SKIPARRAY and find that the skip value
for H is 3. HEAD now slides to the right three characters.
The D in the pattern (see step 4 in Figure 3) is again lined up under the D in
the string. By comparing the remaining characters of HEAD backwards from the D
position, we find that the pattern matches successfully and record the
starting position in the string.
The program used to perform the Boyer-Moore search method is shown in Listing
One. It is assembled using MASM (or TASM) and the OBJ file is linked in to the
Turbo Pascal program. It is called by using the name POSBM the same way you
would call the POS() function in Turbo Pascal. The way this is set up will
only work on parameters of type STRING (see Listing Two), which limit the
search to 255 characters. You can easily modify the program to pass arrays the
same way they are passed in C.
To call POSBM from C you must modify the structure CSTK to conform to the C
calling sequence. You must change the initial code to copy the addresses of
the PATADDR and STRADDR from the calling stack, to PATTXT and STRTXT. The
length of the strings must also be computed (by searching for a 0 using a
SCASB instruction) and placed in the variables PATLEN and STRLEN. Last, the
way in which you return from the function should be changed to leave the
calling values on the stack. Note that the function POSBM is declared FAR.


Limitations to the Boyer-Moore Method


Some patterns may not be suitable for this method. If your pattern has
repeating characters matching against a text of repeating characters, the
Boyer-Moore method will actually be slower than brute force. For example, the
bit-map pattern 01111111 in a bit-map string of all 1111111111..... would
require a lengthy search because the 0 in the pattern would not be compared
until all the 1s have been matched from right to left. The pattern would then
be moved over by one position and the search tried again, thus defeating the
algorithm's purpose.
There have been some clever ways to avoid this problem by analyzing the
pattern but, for all intents and purposes, the simple algorithm works very
well for text searches.


Benchmarks


To illustrate the speed difference between brute force and Boyer-Moore, I have
put together a small test program that compares the speed of the built-in
function POS( ) of Turbo Pascal 4.0 (which uses the brute-force method) and
the assembly language function POSBM( ) presented in this article.
To run the program, you must first assemble the function POSBM.ASM in Listing
Two with MASM or TASM. This will create an .OBJ module. Next, compile the
Turbo Pascal program in Listing One. The function will be linked in when the
{$L POSBM} directive is encountered. Because the function is declared as FAR,
we must include the directive {$F+} in the program to notify Pascal that the
function should be called with FAR instructions.
The program starts out by generating a random string 255 characters long and
uses the last five characters of the string as the pattern. The pattern is
then searched in string using POS( ) and POSBM( ) by putting the searches in a
long loop to emphasize the time difference. You may want to increase the value
of longloop if you have a fast machine to see the significantly higher start
and end times. The POSBM( ) is usually 400 percent faster than the POS( )
function.


Conclusion


I have used this algorithm in a number of programs with impressive results. If
you are looking for a way to speed up your program, check to see if the
bottleneck is your string search functions. The Boyer-Moore algorithm will
give immediate results.


Availability


All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063, or call 800-356-2002 (from inside
Calif.) or 800-533-4372 (from outside Calif.). Please specify the issue number
and format (MS-DOS, Macintosh, Kaypro).



_FASTER STRING SEARCHES_
by Costas Menico


[LISTING ONE]


;====================================================================
; Faster string search routine to substitute the POS()
; function in Turbo Pascal 4 or 5. Based on the Boyer-Moore
; algorithm.
; Program author: Costas Menico.
; Declare as follows:

; {$F+}
; {$L search.obj}
; function posbm(pat,str:string): byte; external;
; Call as follows from Turbo 4 or Turbo 5
; location := posbm(pat, str);
;====================================================================

skiparrlength equ 256 ; Length of the skip array

; Function work stack
dstk struc
patlen dw ? ; pattern length (also BP base work area)
strlen dw ? ; string length
skiparr db skiparrlength dup(?) ; skip array
pattxt dd 0 ; pattern address
strtxt dd 0 ; string text address
dstk ends

; Total stack (Callers plus work stack)
cstk struc
ourdata db size dstk dup(?); work stack size
bpsave dw 0 ; save bp here
retaddr dd 0 ; points to return address
straddr dd 0 ; points to string address
pataddr dd 0 ; points to pattern address
cstk ends

paramsize equ size pataddr + size straddr ; Size of parameter list.

public posbm ; Function name declaration

code segment para public 'code'
 assume cs:code
;
; ---- Entry point to POSBM function.
;
posbm proc far
 push bp ; Save BP
 sub sp, size dstk ; Create work area
 mov bp, sp ; Adjust our base pointer

 push ds ; Save callers data segment

 xor ah, ah ; Clear register and
 cld ; Set direction flag

; Get and save the length and address of the pattern
 lds si, [bp.pataddr];
 mov word ptr [bp.pattxt][2], ds
 lodsb ; Get length of pattern (1 byte)
 or al, al ; If pattern length is null then exit
 jne notnullp
 jmp nomatch

notnullp:
 mov cx, ax ; Save length to check if 1 later.

 mov [bp.patlen], ax ; Save length of pattern
 mov word ptr [bp.pattxt], si; Save address


; Get and save the length and address of the string text
 lds si, [bp.straddr]
 mov word ptr [bp.strtxt][2], ds
 lodsb ; Get length of string
 or al, al ; If string text is null then exit
 jne notnulls
 jmp nomatch

notnulls:
 mov [bp.strlen], ax ; Save length of string
 mov word ptr [bp.strtxt], si; Save address

 cmp cx, 1 ; Is length of pattern 1 char?
 jne do_boyer_moore ; No - Do Boyer-Moore.
 lds si, [bp.pattxt] ; Yes - Do a straight search.
 lodsb ; Get the single character pattern.
 les di, [bp.strtxt] ; Get the address of the string.
 mov cx, [bp.strlen] ; Get length of string.
 repne scasb ; Search.
 jz match1 ; Found - Adjust last DI pos.
 jmp nomatch ; Not Found - Exit.
match1:
 mov si, di ; Transfer DI pos to SI.
 sub si, 2 ; Adjust SI position.
 jmp exactmatch ; Determin offset

do_boyer_moore:

; Fill the ascii character skiparray with the
; length of the pattern
 lea di, [bp.skiparr] ; Get skip array address
 mov dx, ss
 mov es, dx
 mov al,byte ptr [bp.patlen] ; Get size of pattern
 mov ah,al ; Put in to AH as well
 mov cx, skiparrlength / 2 ; Get size of array
 rep stosw ; Fill with length of pat

; Replace in the ascii skiparray the corresponding
; character offset from the end of the pattern minus 1
 lds si, [bp.pattxt] ; Get pattern address
 lea bx, [bp.skiparr]; Get skip array address
 mov cx, [bp.patlen] ; Get length minus one.
 dec cx

 mov bx, bp ; save BP
 lea bp, [bp.skiparr]; Get skip array address
 xor ah, ah
fill_skiparray:
 lodsb ; Get character from pattern
 mov di, ax ; Use it as an index
 mov [bp+di], cl ; Store its offset in to skip array
 loop fill_skiparray

 lodsb
 mov di, ax
 mov [bp+di], cl ; Store the last skip value
 mov bp, bx ; Recover BP


; Now initialize our pattern and string text pointers to
; start searching
 lds si, [bp.strtxt] ; Get the string address.
 lea di, [bp.skiparr]; Get the skip array address.
 mov dx, [bp.strlen] ; Get the string length.
 dec dx ; minus 1 for eos check.
 mov ax, [bp.patlen] ; Get the pattern length.
 dec ax ; Starting skip value.
 xor bh, bh ; Zero high of BX.
 std ; Set to reverse compare.

; Get character from text. Use the character as an index
; in to the skip array, looking for a skip value of 0 .
; If found, execute a brute force search on the pattern.
searchlast:
 sub dx, ax ; Check if string exhausted.
 jc nomatch ; Yes - no match.
 add si, ax ; No - slide pattern with skip value.
 mov bl, [si] ; Get character, use as an index
 mov al, ss:[di+bx] ; and get the new skip value.
 or al, al ; If 0, then possible match
 jne searchlast ; try again by sliding to right.

; We have a possible match, therefore
; do the reverse Brute-force compare
 mov bx, si ; Save string address.
 mov cx, [bp.patlen] ; Get pattern length.
 les di, [bp.pattxt] ; Get pattern address
 dec di ; adjust
 add di, cx ; and add to point to eos.
 repe cmpsb ; Do reverse compare.
 je exactmatch ; If equal we found a match
 mov ax, 1 ; else set skip value to 1.
 lea di, [bp.skiparr]; Get address of skip array.
 mov si, bx ; Get address of string.
 xor bh, bh ; No - Zero high of BX.
 jmp short searchlast ; Try again.

exactmatch:
 mov ax, si ; Save current position in string.
 lds si, [bp.strtxt] ; Get start of strtxt.
 sub ax, si ; Subtract and add 2 to get position
 add ax, 2 ; in strtxt where pattern is found.
 jmp short endsearch ; Exit function

nomatch:
 xor ax, ax ; No match, return a 0

endsearch:
 cld
 pop ds ; Recover DS for Turbo Pascal

 mov sp, bp ; Recover last stack position.
 add sp, size dstk ; Clear up work area.
 pop bp ; Recover BP
 ret paramsize ; Return with ax the POSBM value.
posbm endp


code ends
end






[LISTING TWO]

{ -- Benchmark program to demonstrate the speed difference
 -- between the POS() in Turbo Pascal 4 or 5 brute-force
 -- and the Boyer-Moore Method function POSBM()
 -- Program Author: Costas Menico
}
program search;
uses dos,crt;

{ -- Link in the POSBM Boyer-Moore function -- }
{$F+}
{$L POSBM}
function posbm(pat,str:string):byte; external;

{ Prints bencmark timing information }
procedure showtime(s:string; t: registers);
begin
 writeln(s,' Hrs:',t.ch,' Min:',t.cl,' Sec:',t.dh,' Milsec:',t.dl);
end;

var
 pat,str: string;
 i:integer;
 j:integer;
 start,finish: registers;

const
 longloop = 2000;

begin

 clrscr;

 { -- Create a random string of length 255 -- }
 randomize;
 for i:=1 to 255 do str[i]:=chr(random(255)+1);
 str[0]:=chr(255);

 { -- Initialize a pattern string with the last five characters
 -- in the random string as the pattern to search for. -- }
 pat:=copy(str,251,5);

 { -- First do a search with the regular POS function -- }
 writeln('Search using Brute-Force Method');
 start.ah := $2c;
 msdos(start); { -- Get start time -- }

 for j:=1 to longloop do
 i:=pos(pat,str); { -- Do search a few times Brute-Force -- }


 finish.ah := $2c; { -- Get finish time -- }
 msdos(finish);

 showtime('Start ',start); { -- Show start time -- }
 showtime('Finish',finish); { -- Show finish time -- }
 writeln('Pattern found at =',i); { -- Print string position -- }
 writeln;

 { -- Now do search with the POSBM() (Boyer-Moore) function -- }
 writeln('Search using Boyer-Moore Method');
 start.ah := $2c;
 msdos(start); { -- Get start time -- }

 for j:=1 to longloop do
 i:=posbm(pat,str);{ -- Do search a few times Boyer-Moore -- }

 finish.ah := $2c; { -- Get finish time -- }
 msdos(finish);

 showtime('Start ',start); { -- Show start time -- }
 showtime('Finish',finish); { -- Show finish time -- }
 writeln('Pattern found at =',i); { -- Print string position -- }

 writeln;
 writeln('DONE.. PRESS ENTER');
 readln;
end.



































July, 1989
PROGRAMMING PARADIGMS


How a No-Nonsense Hardware Engineer came to Embrace the Parapsychology of
Artificial Intelligence




Michael Swaine


"Paradigm is a rather unlovely word, which is commonly used in technical
writings when the author wants to obscure the fact that there are no facts in
his writings. Psychologists and psychiatrists and M&T Publishing's writers are
especially fond of the word."
-- Hal Hardenbergh
Hal Hardenbergh's writing is generally strong on factual content or at least
on empirical content. He is wont to speak in what philosophers of science call
"highly falsifiable assertions." He'll make an outrageously bold claim and
phrase it in such specific terms that it is empirically testable on several
grounds. Hence falsifiable, though not necessarily false.
Rarely false, in fact. When you press him on one of his points -- or sometimes
even without pressure -- he trots out facts and figures, names and dates to
support his position. When Hardenbergh fingers his culprit, he usually has a
Zapruder film in his pocket.
He's also wont to pounce on errors or excesses in the use of the English
language, as I observed with chagrin the last time I used the word "wont" in
print. Gremlins had slipped an apostrophe into the word as it went to press,
and it was Hal Hardenbergh who called the error to my attention. And he has
suggested that I use the word "paradigm" too much. I think he's had his eye on
me since I misspelled his name some years back.
His own publication, DTACK GROUNDED, was always good reading and embodied
Hardenbergh's philosophy of no-nonsense, pedal-to-the-metal,
get-the-HLL-out-of-my-machine computing. He wrote and produced that newsletter
continuously from July 1981 to September 1985 (and occasionally thereafter
with the byline "The Junk Mail Flyer"). It went out mainly to customers of his
company, Digital Acoustics, and carried twenty-some pages a month of incisive
industry analysis and juicy gossip, tightly-reasoned technical discussions,
and code. The magazine's bias was speed; it was a computer hardware hacker's
hot-rodding magazine, and Hardenbergh's pet peeve was application programs
written in high-level languages. DTACK GROUNDED was a far cry from this
column, and while writing DTACK GROUNDED, Hardenbergh would no doubt have
scoffed at the inefficiency of Smalltalk and Prolog and at many of the other
topics discussed in this space. Digital Acoustics and DTACK GROUNDED, alas,
are history, but Hardenbergh has been gainfully employed for the past year at
Vicom Systems, an image-processing company in San Jose, Calif.
When I heard that Hardenbergh had got into neural networks I was surprised.
Neural networks represent a branch of artificial intelligence work that some
might consider antithetical to the hard-nosed and the hands-on. The whole
point of neural nets was to remove from human hands a great deal of the
control over what the machine was doing. And demonstrable, practical results
of neural network research were hard to find. Furthermore, existing neural net
implementations are slow. Agonizingly slow. It seemed anything but an area in
which you'd expect to find an inveterate bit-twiddler. I decided to go see
Hardenbergh and find out what the attraction was.
The plan, as usual in this column, was to explore a new paradigm by examining
the thinking that led one computer professional to embrace that paradigm. Be
forewarned that you will probably not agree with everything Hal Hardenbergh
says. But there is usually much to be learned from watching a sharp mind slice
through an interesting problem. Hardenbergh's view of the history and present
value of neural net paradigms is worth examining.
When I arrived at Vicom, Hardenbergh led me to a conference room and offered
me the choice of an interview or a dissertation. I told him to start rolling
and that I'd break in when I got lost. In translating the resulting discussion
to the pages of DDJ I have found it desirable to break in a little more often
than was actually the case during the interview. Or dissertation. But to the
best of my transcriptive ability, Hardenbergh's words are Hardenbergh's words.


Cutting Through the Crap


Swaine: What's the attraction of neural networks for a hardware engineer?
Hardenbergh: I can't say that what I'm doing here at Vicom is dull. Realtime
video processing is hardly boring. But neural nets let me feel like I'm
pushing the envelope a little.
Swaine: But how did you settle on neural nets, rather than some other
envelope-pushing paradigm?
Hardenbergh: When it comes to AI and machine learning, you have four
paradigms. One is the symbolic approach using, typically, Lisp, that Minsky
and Papert championed out of the MIT AI Lab. A lot of money has gone down that
rat hole, and now people have stopped pouring because they noticed that it
wasn't coming back up. The second is expert systems. If you want to invest
some money in AI and have a reasonable expectation of getting something back,
that's where you invest it. The third paradigm is both very old and very new,
and that's neural networks. And the fourth is fuzzy logic. To the best of my
knowledge, these are the four; if you're going into AI, you'll have to tackle
one of them. What the three (that are not neural networks) have in common is
that they require an enormous amount of programming to do anything. The
potential advantage of neural networks is that they program themselves.
Swaine: How did you first get into neural nets?
Hardenbergh: [Vicom co-worker] Tom Waite was looking into neural nets and one
day I asked Tom to teach me about them. He put some equations with integrals
in front of me. For an engineer I'm a pretty decent mathematician, but I told
Tom, "I know how to add and subtract and multiply and divide with a computer,
but I don't know what to do with this." But eventually I got the equations
into pseudo-Basic so I could understand them.
Swaine: You've been at it less than a year, then. But you've done more than
code some integrals.
Hardenbergh: I've been taking classes, reading books, and Tom and I have
submitted an article on neural networks to one of your competitors. (At the
time of the interview, the Hardenbergh and Waite article was scheduled for the
June issue of Programmer's Journal, a magazine that Hardenbergh often writes
for.)
Swaine: Neural networks is an exploding area of research and development.
There's a lot of information to wade through: I have several rather thick
books on neural nets, and there are different models -- the relationships
among which I frankly don't understand. You've apparently found a path through
it all to the information you want.
Hardenbergh: I recommend an article by Lippmann in the April 1987 ASSP
Magazine -- that's the IEEE acoustics and signal processing publication --
it's a tutorial on neural nets that by-passes all the associative memory crap.
Swaine: "Associative memory crap"?
Hardenbergh: Some of what people talk about when they talk about neural
networks is of interest from a historical viewpoint, but not from the
viewpoint of artificial intelligence as I understand it. One of these things
is associative memory. Associative memory maps ones and zeros into ones and
zeros, and it doesn't even do that reliably. If you think ones and zeros have
a lot of intelligence, you'll love associative memory. Then there's adaptive
bidirectional associative memory, or adaptive resonance theory (ART), by
Grossberg, who has a patent in this area. There's a story about why he's
working on ART, rather than something useful like a multilevel perceptron.
Swaine: I gather that you've concluded that, for your purposes at least, the
multilevel perceptron is the only approach worth pursuing.
Hardenbergh: Multilevel perceptrons are my idea of a real-world neural
network.


Multilevel Perceptrons


Swaine: Tell me about how you narrowed your own search down to perceptrons.
Hardenbergh: Lippmann does a taxonomy. He talks about Hopfield nets and
Hamming nets and ART, which, like the other two, is of historical interest
only. And he describes the single-level perceptron.
Swaine: That would be Rosenblatt's perceptron, from back in the late 1950s.
Hardenbergh: The perceptron was the start of all the neural network work. In
1958 Rosenblatt was doing research on natural neural nets, the wet stuff, and
he developed a model of a simplified neuron. There were certain things that it
could do. There are still certain things that it can do. And it generated a
lot of interest in AI in 1958. The people in the MIT AI Lab became disturbed
about funding moving over to perceptrons, and Minsky and Papert decided to do
something about it. What they did was to start writing papers, culminating in
a book called Perceptrons, with a copyright date of 1969. The book
demonstrated that there were certain things that a single perceptron cannot
do. One of the things that a single perceptron cannot do is the exclusive-OR
problem.
Swaine: That's not a trivial limitation. Back when I was doing research in
cognitive psychology, studying the process of concept formation, we found that
modeling the human ability to form concepts of the nature of "A or B but not
both" was very difficult, but we felt that we didn't have a model of concept
formation without that exclusive OR.
Hardenbergh: Oh, the difficulties were real ones. In the meantime Rosenblatt
had suggested that one solution would be to use many perceptrons, perhaps
arranged in layers, but it was only a suggestion, because in 1969 a
mathematical method of adjusting the weights did not exist.
Swaine: Explain about adjusting the weights.
Hardenbergh: You have the desired output, call it the target. You compare the
actual output to the target and measure the error. Then you propagate the
error up through the net and use it to adjust the weights [on the
connections].
Swaine: So certain connections get stronger over time, and the network
responds more and more appropriately as this training proceeds.
Hardenbergh: But you couldn't train the damn thing, so nobody built one, or if
they did, it didn't work, so they didn't write about it. Minsky and Papert's
book Perceptrons was then, and is today, highly regarded, except for the last
chapter. Because they had proven that perceptrons could not solve certain
real-world problems, they concluded that nothing along this line would ever be
useful. The book crushingly discredited neural nets and funding dried up
completely.


The Politics of Discovery



Swaine: What happened next?
Hardenbergh: The next events generally known occurred in the 1980s, but in
1974 an event occurred that was known only to two people. As part of his Ph.D.
research a Harvard graduate student, Paul Werbos, developed the mathematical
technique required to train multilevel perceptrons. His adviser was Steven
Grossberg.
Swaine: This must be the Grossberg story.
Hardenbergh: Right. Grossberg was well aware of Minsky and Papert's work on
perceptrons, and he told this student that his work was of no value. And
indeed it proved of no value, because it was pigeonholed and that was it.
Swaine: That was the technique of back propagation?
Hardenbergh: Yes, I guess it would be hard to do work on multilevel
perceptrons after derailing the discovery of the technique that makes them
feasible. But back propagation is known now, and people are doing work on
multilevel perceptrons.
Swaine: What happened?
Hardenbergh: In the 1980s, about a dozen years later, things began to happen.
One of the things that happened was Hopfield nets. Another was that Rumelhart
and others formed the PDP group (at the Institute for Cognitive Science at the
University of California at San Diego). The PDP group attracted some
interesting people to work with them, including [DNA co-discoverer] Francis
Crick. But in 1982, another event took place that nobody knew about. A
22-year-old Stanford student independently invented the mathematical theory of
back propagation.
Swaine: That would be David Parker. I interviewed him last year, but I'm
planning to talk with him again soon.
Hardenbergh: Parker discovered the theory and went to people who were funding
AI activities and asked for funding. They asked, "Is this an expert system?"
He didn't get the funding and eventually went off on his own.
Swaine: And then?
Hardenbergh: In 1984 there was the Hopfield net, and in 1985 there was the
first public report of a neural net that worked -- barely. That was the
Boltzmann machine and its author was G.E. Hinton, and it was slow even for a
neural net, and neural nets have a deserved reputation for being slow. Then in
1986 there was the publication by Rumelhart, et al. of "Learning Internal
Representations by Error Propagation," the third invention of back
propagation. This one led to the current explosion of interest in neural nets.
Since then there's been a tremendous amount of activity.
Swaine: So back propagation was independently discovered three different
times? What made the difference the third time around?
Hardenbergh: Rumelhart was well known; Parker wasn't.


Smoke Without Fire


Swaine: There certainly has been, as you put it, an explosion of interest in
neural nets, but to date it looks like a lot of smoke and very little fire.
Expert systems really are making money for people and solving real-world
problems. That particular AI technology, while it may not deserve all the hype
it's received, does have unarguable success stories to tell. Why haven't we
seen any breakthrough practical applications of neural net technology?
Hardenbergh: Unfortunately, neural nets are slow. You can't do neural nets on
a PC. And nobody's doing the $4,000 parts-cost solution. So you need to get
Uncle Sugar to give you the latest Cray full-time for a month.
Swaine: Your mention of a $4,000 parts-cost solution sounds like Vicom has a
neural net board in the works.
Hardenbergh: There's interest, but no commitment to a product yet. This is
something Tom and I are pursuing on our own. But management doesn't object to
my meeting with an editor in the conference room on company time to discuss
neural nets. They're supportive.
Swaine: What are the hardware issues?
Hardenbergh: Floating-point chips these days are so good the real problem is
the memory system. You can't use static RAM. You need to interleave DRAM, use
multiported memory. All the hardware cards for neural nets are from software
companies trying to do neural net work. None of them are very good designs.
Swaine: What do you think of the Transputer?
Hardenbergh: It's fundamentally flawed as a concept. It has no
register-to-register add. It has only three registers, arranged in a stack.
Floating Point Systems and the British government lost a lot of money on the
Transputer; Thorne EMI went through about $300,000,000.
Swaine: And the Occam programming language developed for concurrent
programming of networks of Transputers?
Hardenbergh: Occam is a failure.
Swaine: I'll be talking to an Occam programmer in a few weeks, so I'll let him
defend the language then. But is there no good work going on in neural nets?
Are there no success stories?
Hardenbergh: All the good stuff is classified. Chevron owns the seismic
research. Parker may be doing something interesting, but he is not publishing.
Swaine: But you think there are things that you could do with neural nets,
given the right hardware?
Hardenbergh: In combination with image-processing methods. You pre-process
with DSP or whatever, and don't overload the network. Don't make it do what we
already have good algorithms for. The company is very interested in the
possibilities. I'm enjoying myself.


Joke or Not, I Kept the Money


Swaine: The Lippmann article says that three levels of perceptrons not only
solve the exclusive-OR problem but are sufficient for any arbitrary
classification problem. But you said there were certain things that single
perceptrons could do.
Hardenbergh: One spinoff of Rosenblatt's work was the adaptive linear filter.
That was a success, and all of the high-speed modems use it. The reason the
Telebit modem is so successful is ALFs. ALFs are used in phone lines to cancel
noise. An ALF is just a single perceptron. Wait here. [Hardenbergh got up and
left the room. A moment later he returned and put twenty cents into my hand.]
It's pair of dimes. I've been wanting to do that for a long time.
Toward the end of the interview, Tom Waite came into the room. Waite and
Hardenbergh told me how they thought neural nets could supplement existing
image-processing techniques, and Waite shared some ideas about neural net
algorithms. It was Tom Waite who gave me the characterization of neural nets
as the parapsychology of artificial intelligence, a characterization that he
does not agree with. I will look at some of the algorithms next month. I'll
also be talking with neural net algorist David Parker again soon, and within
the next two months I hope to report on that discussion, as well as on a
follow-up interview with Jurgen Fey regarding the transputer board he has
designed specifically to support neural networks.


References


Lippmann, Richard P. "An Introduction to Computing with Neural Nets," IEEE
ASSP Magazine, April, 1987.
Minsky, M. and Papert, S. Perceptrons: An Introduction to Computational
Geometry, MIT Press, 1969.
Parker, D.B. "A Comparison of Algorithms for Neuron-like Cells" in J.S. Denker
(ed) AIP Conference Proceedings 151, Neural Networks for Computing, Snowbird,
Utah, AIP, 1986.
Rosenblatt, R. Principles of Neuro-dynamics. Spartan Books, New York, 1959.
Rumelhart, D.E., Hinton, G.E., and Williams, R.J. "Learning Internal
Representations by Error Propagation" in D.E. Rumelhart and J.L. McClelland
(eds), Parallel Distributed Processing: Explorations in the Microstructure of
Cognition. Vol. 1: Foundations. MIT Press, 1986.









July, 1989
C PROGRAMMING


There's One Born Every Minute




Al Stevens


It is now time to put the SMALLCOM project into the cask and let it age. Some
of you are using it and sending in comments, suggestions, and problems. The
project has spanned ten months, and newer readers are finding it necessary to
get back issues to catch up. I might update the project in a future column and
add more functions to it. For now, though, vacation is upon us, and time has
come to reveal our hidden hedonism with a trip to the islands and the tables
of chance.
Rodney Dangerfield says that he went to Las Vegas in a $25,000 automobile and
came home in a $200,000 bus. I went to Puerto Rico to build an income tax
system and came home with a Blackjack simulation program.
San Juan, Puerto Rico has gambling casinos. My pal Fast Eddie Dwyer lives
there, deals with the Puerto Rican version of the IRS by day, and pokes around
the casinos by night. Armed with a book on how to win at Blackjack, he has
planned an early retirement. With luck, however, he will merely have a lot of
fun spending (losing) not a lot of money. Notice the operative word -- luck.
I'm no gambler, but I'm drawn to a challenging problem. I decided to test Fast
Eddie's Blackjack systems without risking any of my own dough. What better way
than with computer simulation in a C program? The program in Listing One is
bj.c, and it models the game and some of the strategies being taught by the
self-proclaimed masters. My conclusions about the viability of these
strategies can be seen in the results delivered by the program. In the long
haul, you cannot win. Not everyone will agree.
There was a time when you could find a Blackjack game at some casinos that
used one deck. If you were able to mentally master all the strategies, you
could win consistently. The casinos learned this and changed some of the
rules, one of them being the number of decks in the play. Now they use six
decks, and the strategies, even if you can execute them flawlessly, which no
one can, do not work. Fast Eddie still has a day job.
The casinos all have signs that forbid the use of personal computers and other
electronic aids. The signs are there to convince you that "systems" can work,
to make you want to try, to get you to play, to get your money. You could take
all the electronic help you wanted. You'd still lose.
There are three fundamental strategies touted to make you a Blackjack winner.
The first strategy involves observing your hand and the dealer's up card.
Based on a chart (provided in Fast Eddie's how-to book) you decide whether to
hit or stand. If your first two cards are the same denomination, another chart
tells you whether you should split the hand. When you get the second card, a
third chart tells you if you should double-down the hand. The trick is
memorizing all three charts and being able to implement them instantly during
the fast pace of the deal. To complicate matters, the charts are different
depending on whether you are playing in Nevada, Atlantic City, or the
Caribbean.
The second strategy involves counting cards. You keep a running count of two
groups of cards. The first group is the good group, and it consists of the
face cards, tens, and aces. The second group is bad. It is made up of all the
other cards, deuces through nines. You have to watch all the cards that were
dealt to you, the dealer, and all the other players. You have to mentally
compute a running balance of the two groups to know what ratio of good to bad
cards are left in the undealt portion of the deck. The more good cards, the
better your chances of winning. The more bad cards, the more likely you are to
lose. When the ratio gets to a certain level in your favor, you increase your
bet. When the ratio goes far enough against you, you stop playing. The ratio
you apply is a function of the total number of cards in play. As play proceeds
you add one to a running total each time you see a good card. You subtract one
each time you see a bad card. You apply this running total to the total number
of cards to figure the ratio of good to bad cards left in the deck. That is
the essence of card counting. I no longer have the specifics because Fast
Eddie kept the book when I left San Juan. He's still trying.
The third strategy is called money management. You change your bets based on
your recent win/loss ratio. I never understood this one, and could not
simulate it properly. Fast Eddie and I have a pal who believes in it
fervently. The last time he was in San Juan, he borrowed money to get home.
I simulated both of the first two strategies for a while. Only the first is
included in bj.c, however, because I neglected to bring the card-counting
algorithm back into the States. No matter, card-counting had no measurable
impact on the model's outcome unless the deal was a one-deck game, something
you cannot find any more in the big gambling houses. If you have a
card-counting algorithm that you think might work, you can add it to the
program. But if you let all your simulated players count cards, they all drop
out when the ratio gets too unfavorable, and the game grinds to a halt.
Bear in mind that the techniques practiced by the gamblers and simulated by
bj.c require perfect concentration and a near-photographic memory. Those who
hawk the methods insist that with practice the necessary skills can be
mastered, and that might be true. But when a student fails to make money, the
teachers usually blame the student's inability to correctly administer the
procedures rather than the methods. The truth seems to be that the methods do
not work. The computer does not make any of those mistakes (unless, of course,
the mistakes are in the code), and even though it plays flawlessly, the
computer cannot win either.


The BJ.C Program


The bj.c program is written mostly in generic ANSI C. I compiled it with Turbo
C 2.0. The program uses two macros that assume the ANSI.SYS driver is
installed in a PC. These two macros are clr_scrn and cursor. If you are using
a different system you will need to change these macros to clear the screen
and position the cursor with the protocols of your system.
There are two global variables you can change to modify the game. One is
PLAYERS, which defines how many players are in the game. The other is DECKS,
which defines how many decks the game uses. I used a much more complicated
version of the program in San Juan, and you can add the features it had if you
wish. It let me select the deck size and player count at run time. I could
choose a chair and play along, too.
The program uses the IBM PC graphics character set to display the cards on the
screen. If your terminal cannot support these characters, remove the #define
IBMPC statement, and the program will use ASCII characters to represent the
cards.
Between deals in a Blackjack game, the cards are divided between the "shoe," a
box that holds the undealt cards and the "discards," a stack of the cards that
have already been used. When the shoe runs out of cards, the dealer
re-shuffles the cards and puts them back in the shoe. You will see both terms
throughout the program.
The program uses a typedef for the CARD, which contains a value (ace, deuce,
trey ... king) and a suit. The suit is not relevant to the simulation but I
included it to make the program more realistic and more interesting. There is
an array of CARDs called shoe and one called discards.
The PLAYER typedef contains everything about a player including an array of
the CARDs in the current hand, the amount of money in the player's bank, the
amount the player is betting, a pointer to the player's hit/stand strategy
function, and some other operational variables that tell the program what the
player is doing at a given place in the deal. There is an array of PLAYERs
named players. The array has two PLAYER entries for each actual player. The
second entry is set aside for that time when a player decides to split the
hand. The last PLAYER entry is the dealer.
When you run the program, it displays each hand being dealt and the outcome.
Each player's bank is displayed. You can run the program so that it stops
after each hand to let you look at the results, or you can have it run
continuously without intervention. You can run the program to bypass the
displays, showing the banks only. This runs the simulation a lot faster and
gets the same results.
Each player and the dealer start with a bank value of zero. When a player
wins, the player's bank is increased by the amount of the bet and the dealer's
bank is decreased by the same amount. When the player loses, the reverse
occurs. A negative bank means the player is losing. A positive bank means the
player is winning.
A player can "double-down" a hand after the second card. This means the player
doubles the bet but can get no more than one additional card. A player can
split a hand if the first two cards are the same denomination. For example, if
you get two fives, you can play them as two independent hands with the same
bet on each. You can double-down on either or both of the split hands. You can
win one of the hands, both of them, or neither.
If a player's hand ties with the dealer, the player wins, but not the full
amount. In theory, you win half of what you bet. But because the chips do not
come in denominations evenly divisible by two, a tie is called a "push." Your
bet stays up, and to keep it, you have to win the next hand, too.
There is something in Blackjack called "insurance" that I never understood. It
is not in the bj.c program. Detractors of my conclusions can point to this
deficiency if it helps their argument.
The shuffle is done by computing a random subscript into the discards, moving
the card at the offset into the shoe, and moving all the following discards
down one position. This procedure continues until all the cards are in the
shoe. When the program begins, all the cards are in the discards ready for the
first shuffle. Each time the deck is shuffled, the first card is "buried,"
that is, moved to the discard pile.
The dealer's hit/stand strategy is a simple one. The rules say the dealer must
hit if the hand value is 16 or below and stand if the hand value is 17 or
above. Dealers cannot split a hand. The dstrategy function implements the
dealer's strategy.
Players do not have the restrictions imposed on dealers, and the essence of
the simulation is in the two functions named split and pstrategy. These
functions decide what a player is going to do with the hand and are designed
to use the strategies of the three charts given in Fast Eddie's book for
Caribbean casinos.
The split decision is based on what your identical cards are and what the
dealer's up card is. The switch statement in the split function makes the
decision.
The pstrategy function decides whether to double-down or not and whether to
take a hit or not. It looks at the value of the player's hand and the dealer's
up card. If either of the first two cards is an ace, the third card strategy
is different than otherwise. These switch statements are based on the charts.
You can run the bj.c program and draw your own conclusions. Often a player
will have a long run of luck. Often you will see the dealer losing for an
extended time. But if you let the program run long enough, the dealer wins big
and the players lose big.
This month's column was fun but it might raise some controversy. It has
concluded with computer simulation that you cannot consistently win at
Blackjack. Some people do not want to believe that. There is a dominant trait
of character common in most gamblers. They do not want to believe that winning
is a matter of luck, losing is a function of odds, and the systems do not
work. Anyone who doubts these arguments is welcome to find the flaws in my
model and correct them. But if, on the other hand, they are that sure that I
am wrong, they are too busy winning money to mess with a trifling computer
program.
Those who write or send a CompuServe message to disagree with these
conclusions are requested to restrict their criticisms to the accuracy of the
code and the degree to which they think the simulation reflects the real
world. I am not interested in tales of success at the tables or the
specifications for Blackjack systems that are guaranteed to work. Books and
seminars on how to beat the Blackjack tables abound. They are almost as
plentiful as the books, home study cassette courses, and seminars on how to
get rich in real estate. It makes you wonder where the real money is. If the
systems work, one might ask, how come their promoters are in the seminar and
book business?
The code in bj.c is tossed out for those of you who would care to try it and
maybe tweak it. I doubt that you'll ever get it to win. If you do, I doubt
that you'll ever be able to take its skill to the tables yourself. The program
convinced me to stay away from the Blackjack tables. I watched it consistently
lose simulated money for its simulated players, and I stood in casinos and
watched real people lose real money. If the program can convince you the same
way it convinced me, then it is a public service and has served its purpose.
Spend your money instead going to see Wayne Newton.


Availability


All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063, or call 800-356-2002 (from inside
Calif.) or 800-533-4372 (from outside Calif.). Please specify the issue number
and format (MS-DOS, Macintosh, Kaypro).


_C PROGRAMMING COLUMN_

by Al Stevens



[LISTING ONE]


/* ------------- bj.c ---------------- */
/*
 * A Blackjack Simulation
 */
#include <conio.h>
#include <stdio.h>
#include <stdlib.h>
#include <ctype.h>
#include <time.h>

#define TRUE 1
#define FALSE 0
#define IBMPC

/* ------- ANSI.SYS screen controls --------- */
#define clr_scrn() puts("\033[2J")
#define cursor(x,y) printf("\033[%02d;%02dH",y+1,x+1)

/* --------- for display purposes ------------- */
#define CARDWIDTH 5
#define CARDHEIGHT 4
#define PLAYERS 7 /* number of players (not incl dealer) */
#define DECKS 6 /* number of decks to play */
#define CARDS (52*DECKS)

/* -------- the card display characters -------- */
#ifdef IBMPC
#define UL 218 /* IBM Graphics upper left box corner */
#define HZ 196 /* box horizontal line */
#define UR 191 /* upper right box corner */
#define VT 179 /* box vertical line */
#define LL 192 /* lower left box corner */
#define LR 217 /* lower right box corner */
#define HEARTS 3
#define DIAMONDS 4
#define CLUBS 5
#define SPADES 6
#else
#define UL '.' /* ASCII upper left box corner */
#define HZ '-' /* box horizontal line */
#define UR '.' /* upper right box corner */
#define VT '' /* box vertical line */
#define LL '\'' /* lower left box corner */
#define LR '\'' /* lower right box corner */
#define HEARTS 'h'
#define DIAMONDS 'd'
#define CLUBS 'c'
#define SPADES 's'
#endif

/* --------- the card display values ---------- */
char topline[] = {UL, HZ, HZ, HZ, UR, 0};

char midline[] = {VT, ' ', ' ', ' ', VT, 0};
char botline[] = {LL, HZ, HZ, HZ, LR, 0};
int suits[] = {HEARTS,DIAMONDS,CLUBS,SPADES};
char *vals[] = {"A","2","3","4","5","6","7","8","9","10",
 "J","Q","K"};
/* --------- a card ----------- */
typedef struct crd {
 int value;
 int suit;
} CARD;
/* ---------- a player -------- */
typedef struct ply {
 CARD phand[10]; /* the player's hand */
 int playing; /* true if the player is playing */
 int cards; /* the card count in the hand */
 int bank; /* player's bank account */
 int bet; /* player's bet amount */
 int mode; /* D = double-down */
 int pushing; /* true = player tied last hand */
 int (*strat)(int); /* pointer to hit/pass strategy */
} PLAYER;

/* ----- 1 per player, 1 per split, 1 dealer ------ */
#define NBRPLAYERS (PLAYERS*2+1)
int hands;
#define dealer (NBRPLAYERS-1)
PLAYER players [NBRPLAYERS+1]; /* the players */
CARD shoe[CARDS]; /* the shoe */
CARD discards[CARDS]; /* the discards */
int nextshoe; /* shoe subscript */
int inshoe; /* # cards in the shoe */
int nextdisc; /* discard subscript */
int display=TRUE; /* true if hands are being displayed */
int stopping=TRUE; /* true if display stops at every hand */
int stopper; /* the key that restarts the display */

/* ---------- prototypes ----------- */
void shuffle(void);
void bury(void);
void hand(CARD *lst, int x, int y);
void card(int sut, int val, int x, int y);
void cardframe(int x, int y);
void play(void);
void playahand(void);
int split(int p);
void discs(void);
int pstrategy(int p);
void doubledown(int p);
int bj_bust(int p);
int dstrategy(int p);
void winlose(int p);
void won(int p);
void lost(int p);
void push(int p);
void post(int p, char *s);
void nextcard(CARD *cd);
void shoes(void);
void showhand(int pl);
void cleargame(void);

void nohand(CARD *lst, int x, int y);
void nocardframe(int x, int y);
int handvalue(int pl);
void stat(int p, char *s);

void main(void)
{
 int i, s, v;
 clr_scrn();
 /* --- build the decks of cards in the discard pile --- */
 for (i = 0; i < DECKS; i++)
 for (s = 0; s < 4; s++)
 for (v = 0; v < 13; v++) {
 discards[nextdisc].value = v;
 discards[nextdisc++].suit = suits[s];
 }
 /* ----- initialize the players ----- */
 for (i = 0; i < NBRPLAYERS-1; i++) {
 players[i].bet = 1;
 players[i].strat = pstrategy;
 /* --- every other player is a split --- */
 players[i].playing = !(i & 1);
 }
 players[dealer].playing = TRUE;
 play();
 clr_scrn();
}
/* --------- begin play of the game --------- */
void play(void)
{
 int c;
 while (TRUE) {
 if (stopping kbhit()) {
 if (!stopping NBRPLAYERS == 1)
 c = getch();
 else {
 c = stopper;
 stopper = FALSE;
 }
 if (c == 27)
 break;
 if (tolower(c) == 'd')
 display ^= TRUE;
 if (c == 's')
 stopping ^= TRUE;
 }
 playahand();
 }
}
/* --------- play one hand of blackjack ------------- */
void playahand(void)
{
 int p, i, bnk;
 CARD *cd;
 /* -------- deal everyone one card ----------- */
 for (p = 0; p < NBRPLAYERS; p++) {
 if (players[p].playing == TRUE) {
 nextcard(players[p].phand);
 players[p].cards++;

 showhand(p);
 }
 }
 /* ----- deal each of the players a second card ----- */
 for (p = 0; p < dealer; p++) {
 if (players[p].playing == TRUE) {
 nextcard(players[p].phand+1);
 players[p].cards++;
 /* -- test to see if this player should split -- */
 if (split(p)) {
 /* ----- split the hand ------- */
 players[p].cards = 1;
 players[p].phand[1].suit =
 players[p].phand[1].value = 0;
 bnk = players[p+1].bank;
 players[p+1] = players[p];
 players[p+1].bank = bnk;
 stat(p, "SPLT");
 }
 showhand(p);
 }
 }
 /* deal the rest of the hand for each player in turn */
 for (p = 0; p < NBRPLAYERS-1; p++) {
 if (players[p].playing == TRUE) {
 while (!bj_bust(p) && (*players[p].strat)(p))
 nextcard(players[p].phand+(players[p].cards++));
 showhand(p);
 }
 }
 /* ------ see if all the players went bust ------- */
 for (p = 0; p < NBRPLAYERS-1; p++)
 if (players[p].playing == TRUE && handvalue(p) <= 21)
 break;
 /* ----- if so, the dealer doesn't have to play ---- */
 if (p < NBRPLAYERS-1) {
 /* ------ deal the rest of the dealer's hand ------- */
 while (!bj_bust(dealer) && dstrategy(dealer))
 nextcard(players[dealer].phand+
 (players[dealer].cards++));
 showhand(dealer);
 }
 /* ------ post players' wins and losses --------- */
 for (p = 0; p < NBRPLAYERS-1; p++) {
 if (players[p].playing == TRUE)
 winlose(p);
 players[p].mode = FALSE;
 }
 post(dealer, "DEAL");
 cursor(0, 1);
 for (i = 0; i < NBRPLAYERS+1; i += 2)
 printf("%5d ", players[i].bank+players[i+1].bank);
 /* -- gather the players' cards into the discard pile -- */
 for (p = 0; p < NBRPLAYERS; p++) {
 cd = players[p].phand;
 for (i = 0; i < players[p].cards; i++)
 discards[nextdisc++] = *cd++;
 }
 /* ---- if display stops on every hand, read a key ---- */

 if (stopping)
 stopper = getch();
 cleargame();
 discs();
 ++hands;
 if (display) {
 cursor(40,0);
 printf("HANDS: %d ", hands);
 }
}
/* ---- test to see if a player should split the hand ---- */
int split(int p)
{
 int a, b;
 a = players[p].phand[0].value + 1;
 b = players[dealer].phand[0].value + 1;
 if (b > 10)
 b = 10;
 if (a == players[p].phand[1].value + 1) {
 switch (a) {
 case 1: return TRUE;
 case 2:
 case 3: return (b > 1 && b < 8);
 case 4: return (b == 5 b == 6);
 case 5: return FALSE;
 case 6: return (b > 1 && b < 7);
 case 7: return (b > 1 && b < 8);
 case 8: return TRUE;
 case 9: return !(b == 7 b == 10 b == 1);
 case 10: return FALSE;
 }
 }
 return FALSE;
}
/* -------- display the discards pile count -------- */
void discs(void)
{
 if (display) {
 cursor(20,0);
 printf("DISCARDS: %d ", nextdisc);
 }
}
/* ---- test if a player has blackjack or went bust ---- */
int bj_bust(int p)
{
 int rtn;
 if ((rtn = (players[p].cards == 2 && handvalue(p) == 21))
 == TRUE)
 stat(p, "*BJ*"); /* player has blackjack */
 else if ((rtn = (handvalue(p) > 21)) == TRUE)
 stat(p, "BUST"); /* player went bust */
 return rtn;
}
/* ---- player strategy (true = hit, false = stand) ---- */
int pstrategy(int p)
{
 int b, h;
 /* ---- smart player watches dealers up card ---- */
 b = players[dealer].phand[0].value+1;

 h = handvalue(p);
 if (players[p].mode == 'D')
 return 0;
 if (players[p].cards == 2 &&
 players[p].phand[0].value+1 == 1 
 players[p].phand[1].value+1 == 1) {
 switch (h) {
 case 3:
 case 4: if (b == 5 b == 6)
 doubledown(p);
 return TRUE;
 case 5:
 case 6: if (b > 3 && b < 7)
 doubledown(p);
 return TRUE;
 case 7: if (b > 2 && b < 7)
 doubledown(p);
 return TRUE;
 case 8: if (b > 3 && b < 7)
 doubledown(p);
 if (b == 2 b == 7 b == 8)
 return FALSE;
 return TRUE;
 default: break;
 }
 }
 switch (h) {
 case 4:
 case 5:
 case 6:
 case 7:
 case 8: return TRUE;
 case 9: if (b > 2 && b < 7)
 doubledown(p);
 return TRUE;
 case 10: if (b > 1 && b < 10)
 doubledown(p);
 return TRUE;
 case 11: if (b != 1)
 doubledown(p);
 return TRUE;
 case 12: return !(b > 3 && b < 7);
 case 13:
 case 14:
 case 15:
 case 16: return !(b > 1 && b < 7);
 case 17:
 case 18:
 case 19:
 case 20:
 case 21: return FALSE;
 }
 return FALSE;
}
/* ----------- double down the hand ---------- */
void doubledown(int p)
{
 players[p].mode = 'D';
 stat(p, "DBDN");

}
/* --------- the dealer strategy ---------- */
int dstrategy(int p)
{
 /* - dealer hits on 16 or below, stands on 17 or above - */
 return (handvalue(p) < 17);
}
/* ------- test if a hand wins or loses -------- */
void winlose(int p)
{
 /* --- doubled-down hand bets twice as much --- */
 if (players[p].mode == 'D')
 players[p].bet *= 2;
 /* ---- value > 21 is a bust ---- */
 if (handvalue(p) > 21)
 lost(p);
 /* - blackjack wins if dealer does not have blackjack - */
 else if (handvalue(p) == 21 && players[p].cards == 2 &&
 !(handvalue(dealer) == 21 &&
 players[dealer].cards == 2))
 won(p);
 /* ----- value greater than dealer wins ----- */
 else if (handvalue(p) > handvalue(dealer))
 won(p);
 /* ----- if dealer busts, player wins ----- */
 else if (handvalue(dealer) > 21)
 won(p);
 /* -- if dealer's hand > player's hand, player loses -- */
 else if (handvalue(p) < handvalue(dealer))
 lost(p);
 /* ---- tied hand (push) if none of the above --- */
 else
 push(p);
 /* ------ reset bet for doubled-down hand ------- */
 if (players[p].mode == 'D')
 players[p].bet /= 2;
}
/* -------- compute the value of a hand ------ */
int handvalue(int pl)
{
 CARD *hd;
 int vl = 0, cd, aces = 0;

 hd = players[pl].phand; /* point to 1st card in hand */
 while (hd->suit) {
 cd = hd->value+1; /* value of the card */
 if (cd > 10) /* jack, queen, king = 10 */
 cd = 10;
 if (cd == 1) { /* ace = 11 */
 cd = 11;
 aces++; /* count aces in the hand */
 }
 vl += cd; /* accumulate hand value */
 hd++; /* point to next card */
 }
 while (vl > 21 && aces--) /* adjust for aces if > 21 */
 vl -= 10; /* ace = 1 */
 return vl;
}

/* ------ the player won the hand :-) -------- */
void won(int p)
{
 players[p].bank += players[p].bet + players[p].pushing;
 players[p].pushing = 0;
 players[dealer].bank -= players[p].bet;
 post(p, "WIN ");
}
/* -------- the player lost the hand :-( --------- */
void lost(int p)
{
 players[p].bank -= players[p].bet + players[p].pushing;
 players[p].pushing = 0;
 players[dealer].bank += players[p].bet;
 post(p, "LOSS");
}
/* ------- the player tied :- -------- */
void push(int p)
{
 players[p].pushing = players[p].bet;
 post(p, "PUSH");
}
/* ------- post the WIN/LOSS/PUSH ------ */
void post(int p, char *s)
{
 if (display) {
 cursor(1+p*5, 24);
 printf("%s", s);
 }
}
/* -------- get the next card from the shoe ------ */
void nextcard(CARD *cd)
{
 if (nextshoe == inshoe) {
 shuffle(); /* time to reshuffle */
 bury(); /* bury one */
 }
 *cd = shoe[nextshoe++];
 (cd+1)->suit = FALSE;
 shoes();
}
/* --------- shuffle the discards into the shoe -------- */
void shuffle(void)
{
 int cdp, nd;

 if (display) {
 cursor(0,0);
 printf("SHUFFLE");
 }
 randomize();
 nd = nextdisc;
 for (nextshoe = 0; nextshoe < nd; nextshoe++) {
 cdp = random(nextdisc);
 shoe[nextshoe] = discards[cdp];
 while (cdp < nextdisc) {
 discards[cdp] = discards[cdp+1];
 cdp++;
 }

 --nextdisc;
 }
 discs();
 inshoe = nextshoe;
 nextshoe = 0;
 if (display) {
 cursor(0,0);
 printf(" ");
 }
}
/* ---------- bury the first card ----------- */
void bury(void)
{
 CARD cd[2];

 nextcard(cd);
 discards[nextdisc++] = *cd;
 if (display) {
 card(cd[0].suit, cd[0].value, 1, 16);
 cursor(1, 15);
 printf("BURIED");
 if (stopping)
 stopper = getch();
 nocardframe(1, 16);
 cursor(1, 15);
 printf(" ");
 }
}
/* ----- display the number of cards left in the shoe ----- */
void shoes(void)
{
 if (display) {
 cursor(10, 0);
 printf("SHOE: %d ", inshoe-nextshoe);
 }
}
/* ------- display the hand and the player's money ------ */
void showhand(int pl)
{
 if (display) {
 cursor(1+pl*5, 3);
 printf("%d", handvalue(pl));
 hand(players[pl].phand, pl*CARDWIDTH, 4);
 }
}
/* --------- display a hand -------- */
void hand(CARD *lst, int x, int y)
{
 while (lst->suit) {
 card(lst->suit, lst->value, x++,y);
 lst++;
 y += 2;
 }
}
/* ---------- display a card ---------- */
void card(int sut, int val, int x, int y)
{
 cardframe(x, y);
 cursor(x+1, y+1);

 printf(" \b\b\b");
 printf("%s%c", vals[val], sut);
}
/* ---------- display the card frame -------- */
void cardframe(int x, int y)
{
 int y1;

 cursor(x, y);
 printf(topline);
 for (y1 = y+1; y1 < y+CARDHEIGHT-1; y1++) {
 cursor(x,y1);
 printf(midline);
 }
 cursor(x,y1);
 printf(botline);
}
/* --------- clear the game display ---------- */
void cleargame(void)
{
 int i;

 for (i = 0; i < NBRPLAYERS; i++) {
 if (players[i].playing == TRUE) {
 if (display) {
 cursor(1+i*5, 3);
 printf(" ");
 nohand(players[i].phand, i*CARDWIDTH, 4);
 }
 players[i].cards = 0;
 players[i].playing = !(i & 1);
 stat(i, " ");
 post(i, " ");
 }
 }
}
/* ----- display a null hand to erase the hand ----- */
void nohand(CARD *lst, int x, int y)
{
 while (lst->suit) {
 nocardframe(x++,y);
 y += 2;
 lst++;
 }
}
/* ----- null card frame ------------ */
void nocardframe(int x, int y)
{
 int y1;

 for (y1 = y; y1 < y+CARDHEIGHT; y1++) {
 cursor(x,y1);
 printf(" ");
 }
}
/* ---- print the status *BJ*, BUST, DBLD, SPLT, etc. ---- */
void stat(int p, char *s)
{
 if (display) {

 cursor(p*CARDWIDTH, 2);
 printf(s);
 }
}


























































July, 1989
GRAPHICS PROGRAMMING


Do-It-Yourself Coordinates




Kent Porter


This marks what my wife would call the "six-month-aversary" of the "Graphics
Programming" column. There's something about years that are multiples of five
and months occurring on even boundaries of six that summon up the tendency to
assess The Situation. A f rther spur is the increasing amount of mail and
CompuServe traffic we've received at DDJ concerning this column. Most like it.
A few hate it. One fellow even canceled his subscription because of it. I'm
sorry he did. On the other hand, several thousand new members have married
into the DDJ family during the past six months. Certainly, I don't take all
the credit for that, but I'd like to think this column had something to do
with it. Still, I feel compelled to answer a few criticisms and let you in on
the grand strategy. And then we'll get down to this month's advancement of
computer graphics, which is pretty significant.
Okay, critics, here's the Q&A:
Q: Why do I concentrate on the EGA/VGA, excluding all others?
A: Because that's the configuration I have, and most readers have as well.
Admittedly there's also the Atari, Amiga, Mac II, TMS 34010, DGIS, Nth, Sun,
Apollo. . . . You name it, and all wonderfully capable of graphics. And if I'd
tried to serve every one of them, it would have taken a year to get past
writing a single pixel, by which time 99 percent of the readership would have
died of boredom. We have to make some progress here, and we do that by
rallying around the common denominator. According to DDJ reader surveys,
that's the EGA/VGA on a PC. All graphics rests, ultimately, on pixel-writing.
If you have developed other machine solutions to the GRAFIX library, you're
more than welcome to post them on CompuServe.
Q: Why C and assembly language?
A: Why not? Once again, we must have some common basis on which to proceed. C
is used by 75 percent of DDJ readers, and it's the pre-eminent system's
programming language. Most C programmers recognize that assembler is the
alternative for writing high-performance subroutines. Certainly other
languages are graphics capable as well. Prime examples are Pascal and
Modula-2, for which I have a distinct and public fondness. A reader has
offered to translate the GRAFIX library into Turbo Pascal and I'm all for it.
But again, we must have a common point of reference. It's not hard to
translate C code into other languages.
Q: Why don't I worry more about performance? For example, why is the Bresenham
line-drawing algorithm written in C instead of assembly language?
A: Because the purpose of this column is to see how it's done. It's easier to
understand C than assembler. Optimization --particularly in assembly language
--obscures the underlying algorithm. Performance issues are, however,
addressed. That's why hline() exists and why some other low-level routines are
coded in assembly language. We're not developing a commercial graphics package
here, but instead exploring the hows and whys. All of us --myself included and
especially --are learning as we go along. You read a magazine that clearly
states on its cover "for the professional programmer." That suggests a certain
savoir faire, and I assume you're capable of rising to the occasion if you
want more performance.
So much for the critics, whose wounds I have hopefully salved. Where are we
going? Right now we're concentrating on the EGA, which is easier to understand
than the VGA and produces acceptable graphics for most purposes. You'll note
that some functions include "ega" in their names, as in egapixel(),
set_ega_palreg(), and so on. Later we'll add functions with "vga" as part of
their identifiers, and a top that we'll pile a set of bindings that select the
appropriate function based on the current video mode.
In the long run we're headed for the VGA. It's the only commonly available and
affordable PC video adapter capable of producing subtle shading and other
visual effects that characterize high-level graphics.
Meanwhile, there's a lot of ground to cover. It would be wise to put a math
coprocessor into your hardware budget, because by the end of the year we're
going to be heavily into the floating point matrix operations that are
inevitable in 3-D graphics.
In fact, if you don't have an '87 now, you'll begin this month to experience
some of the performance degradation that floating point emulation introduces.
For July is the month in which we enter the mystical realm of ...


Virtual Coordinates


All display devices have some sort of fixed coordinate system. In EGA graphics
it's 640 x 350; for VGA it's 640 x 480. Others of less interest here might
have a resolution of 1024 x 768 or 320 x 200 or whatever. This fixed system is
generally referred to as the device coordinate space.
There are some problems with device coordinates. For one thing, the origin is
(on the PC, at least) always in the upper left corner, which means that the Y
value increases downward. That's counterintuitive, the reverse of graphing
techniques taught from grade school onward. For another thing, the origin is
not relocatable. What if you want the zero point in the middle of the screen
with positive and negative coordinates? Yet another problem is distortion;
pixels are not square in EGA graphics, yielding vertically elongated circles
and other visual oddities. And finally, data values seldom map neatly into the
device coordinate space. For example, how do you fit 30 data points evenly
across 640 pixels?
The LINEGRAF.C program presented back in May dealt with some of these issues,
but not in a very satisfying way. That's because it necessarily included a
number of extraneous calculations required to convert the program's internal
data representation into device coordinates.
Certainly it's necessary to perform these calculations, but there ought to be
a better way: A way in which the programmer can define the screen coordinate
space in terms of the data to be displayed, and then concentrate on the
problem without worrying about the mechanics of mapping values to a specific
display device.
And indeed there is. That's what virtual coordinates are all about.
The word virtual is so overworked in our industry that I hesitate to use it
yet again. However, it's appropriate in this context. According to good old
Webster, virtual means "denoting in essence or effect, without being so in
fact." If you declare an effective display width of, say, 100 units when its
actual width is 640, you've defined a virtual space. The same is of course
true in the vertical (Y) space, which you might describe as something besides
the EGA's 0 - 349 working downward.
The ability to redefine the screen dimensions in any terms you like greatly
simplifies graphics programming. Your programs can then work with intuitive
units as their internal data representation: dollars, percentages, months,
data points, whatever. You can scale each axis independently, and also use
different data types in the X and Y directions. For example, in plotting
trigonometric functions you might assign the integral angles -360 to +360 as
units along the X axis and floating-point increments from +1.0 at the top to
-1.0 at the bottom along the Y. We'll do something like this later.
Virtual coordinates solve a number of graphics problems. The example just
given "flips" the Y axis to correspond with the normal mathematical practice
of increasing the Y value upward. Note also that it relocates the origin to
the center of the physical screen.
Another vexing problem overcome by virtual coordinates is the "un-squareness"
of pixels, a phenomenon often referred to in graphics literature as aspect
ratio or orthogonal distortion. The normal display area has a height that is
75 percent of its width. If you divide height by width, the result is 0.75 if
the pixels are square, or in other words if the vertical and horizontal units
of measurement are equal. That works out to be true for the VGA in 640 x 480
graphics, but not for the EGA's 640 x 350 mode. The result is about 0.547,
meaning that the physical height of a pixel is greater than its width. Some
correction is therefore necessary on the EGA to produce round circles and
square squares and other figures of consistent dimensions along both axes.
Virtual coordinates let you make these corrections.
VCOORDS.C in Listing One performs the necessary magic to implement virtual
coordinates. Your program defines the virtual space it wants by calling
setcoords( ). The arguments are the left, top, right, and bottom of the
display area, expressed in virtual units. For example, to set the
trigonometric plot area mentioned earlier, write
 setcoords (-360, 1.0, 360, -1.0);
If you want a conventional (Y downward) plot area with virtually square
pixels, write
 setcoords (0, 0, 639, 479);
With this virtual space, the EGA emulates the VGA's coordinate space, and it
will produce images of dimensions identical to those drawn in native VGA mode.
The arguments are of type double. Because of the function prototype given in
this month's additions to GRAFIX.H (Listing Two), the compiler automatically
casts integral arguments to doubles, thus allowing you to pass any numeric
data type to setcoords( ) with the expectation of reliability.
The secret of virtual coordinates lies in the four variables at the top of the
VCOORDS compile unit: xf, yf, ox, and oy. The first two are conversion
factors, which are multiplied by virtual coordinates to derive the
corresponding device coordinates. By default these values are 1.0, yielding a
1-for-1 mapping of virtual to device units. If you set a virtual width of 320,
however, xf becomes 2.0, meaning there are two device X units per virtual X.
The other two variables, ox and oy, represent the virtual origin in terms of
device coordinates. They give the offsets to be added to converted virtual
coordinates in order to map a virtual point to its specific device location.
These values are normally zero, which initializes the virtual coordinate
system to an origin in the upper-left corner. If you set up a virtual space
with a call such as
 setcoords (-160, -120, 159, 119);
then after setcoords( ) executes, ox and oy will contain values close to the
physical center of the screen.
The discussion so far assumes that you're operating in full-screen mode.
However, the virtual coordinate system is relative to the current viewport. If
the viewport's proportions differ from those of the full screen, the virtual
space assumes a mapping that distorts accordingly. That is, when the current
viewport has a physical height twice its width, a 4-to-3 ratio in the virtual
space will yield "square" virtual pixels that are twice as high as wide. This
characteristic produces infinite possibilities for scaling graphics output to
suit your needs. We'll see an example shortly.
But first it's necessary to incorporate the virtual coordinate package into
your copy of GRAFIX.LIB. Append the contents of Listing Two to GRAFIX.H, then
compile VCOORDS.C. Add the object module to the library with the command
 LIB grafix + vcoords;
Having defined a virtual space with setcoords( ), your program can work
internally with intuitive virtual units. An example is graphing budgeted
amounts by quarter. The X axis can represent quarters 1 - 4, while the Y axis
represents millions of dollars. At output time, the program calls upon the
VCOORDS routines to map the virtual values to device coordinates by calling
dx( ) and dy( ). Similarly, if the program wants to translate a virtual
distance into device units, as in calling hline( ), draw_rect( ), and so on,
it can call dxunits( ) and dyunits( ).
Let's see how it works to use virtual coordinates. The first example is
TRIGPLOT.C in Listing Three. This program plots the sine, cosine, and their
sum through two full circles (-360 to +360 degrees). The X axis is thus
subdivided into 720 virtual units corresponding to integral degrees. The sine
and cosine never exceed an absolute magnitude of 1.0, and so the Y axis uses
floating point units. Addition of the curves, which have different periods,
results in a fluctuation of about plus and minus 1.4, so the Y axis is scaled
from +1.5 on top to -1.5 below. The virtual origin is at the center of the
screen.
The program begins by scaling the axes with a call to setcoords( ), and then
it draws the registration lines. The major axes appear in green, and reference
lines in blue at virtual distances of +1.0 and -1.0 vertically. Three
successive loops, stepping conveniently by degrees, control the drawing of the
curves. The plot( ) routine does the real work, finding the virtual elevation
of the point by calling the parametric function (cos( ), sin( ), or sum( ))
and then advancing the curve to that point from the elevation of the previous
angle.
The next example is LENDIST.C, Listing Four. This is a very different
application of virtual coordinates. The program displays a histogram -- that
is, a horizontal bar chart --showing the distribution of line lengths in a
text file. Lines are grouped by multiples of five characters: 0 - 4, 5 - 9,
etc. The longest possible line is assumed to be 80 characters. If yours are
longer, modify MAXLEN at the top of the listing accordingly.
The program passes through the file named on the command line, counting the
number of text lines within each grouping. It then finds the group with the
highest count and normalizes all lines to that value, using 100 as the
reference. That is, if maxcount is 86, the normalized value for the group
containing 86 lines is 100. If another group contains 43 lines, its normalized
value is 50, or half as many as the largest group.
Normalization is useful in this case because the graph must accommodate a wide
range of potential values while setting aside a fixed amount of space for the
bar legends. By normalizing the largest value to 100, we know how much space
to leave for text: 22 virtual X units. And because there are 25 text rows on
the screen, we can also establish the virtual Y axis as 25 units, with a range
of -2 through 23 to leave room at the top for a label. Hence the values passed
to setcoords( ) and the newlines in the printf( ) statements.
The rest is fairly obvious from the listing. The graph area setup entails
drawing a rectangle around the whole works and marking registration lines at
normalized intervals of 10 across the region where the bars will occur.
Because of the virtual Y spacing, whose origin is two text lines down from the
top, each bar lines up with its legend. Its height is 75 percent of a single Y
unit to provide cosmetic spacing.

Hey, we're starting to produce useful graphics applications!
The final example considers the effect of the viewport shape on the scaling of
the virtual coordinate space. A "square" space with a 4-to-3 ratio turns out
not to be square if the containing viewport isn't of the same ratio as the
device. Instead, the vertical and horizontal virtual spacing automatically
adjust to fit the viewport dimensions.
The case in point is RESIZE.C, Listing Five. This program displays a
four-pointed star in each of three viewports. The same routine -- drawstar( )
-- draws and fills the figure each time using identical virtual coordinates,
which assume a square space with its origin at the center. However, because
set_up( ) establishes different viewport dimensions, the lengths of the X and
Y units change. The first viewport is square, with a 4-to-3 ratio. The other
two, however, are tall and skinny, and short and fat. The star in each
maintains its spatial relationship to the viewport, but the shape is
dramatically different.
The ability to redefine the display's coordinate system opens the door to
great possibilities for data representation in graphics. Now that we know how
to do it, we'll use virtual coordinates often as we move onward.
Happy six-month-aversary.


Availability


All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063; or call 800-356-2002 (from inside
Calif.) or 800-533-4372 (from outside Calif.). Please specify the issue number
and format (MS-DOS, Macintosh, Kaypro).


_GRAPHICS PROGRAMMING COLUMN_
by Kent Porter



[LISTING ONE]


/* VCOORDS.C: Implements virtual coordinates in GRAFIX library */
/* K. Porter, DDJ Graphics Programming Column, July '89 */

#include <math.h>
#include "grafix.h"

/* Variables for this compile unit */
double xf = 1.0, yf = 1.0; /* x, y conversion factors */
double ox = 0.0, oy = 0.0; /* virtual x, y origin */
/* ------------------------------------------------------------------ */
/* set virtual coordinate space */
void far setcoords (double left, double top, double right, double bottom)
{
 xf = (double) vp_width() / (right - left); /* conversion factors */
 yf = (double) vp_height() / (bottom - top);
 ox = (double) vp_width() - (right * xf); /* origin */
 oy = (double) vp_height() - (bottom * yf);
} /* ---------------------------------------------------------------- */

int far dx (double vx) /* convert virtual x to device x */
{
 return (xf * vx) + ox;
} /* ---------------------------------------------------------------- */

int far dy (double vy) /* convert virtual y to device y */
{
 return (yf * vy) + oy;
} /* ---------------------------------------------------------------- */

int far dxunits (double vx) /* device x units for vx */
{
 return (int)(fabs)(xf * vx);
} /* ---------------------------------------------------------------- */

int far dyunits (double vy) /* device y units for vy */
{
 return (int)(fabs)(yf * vy);
} /* ---------------------------------------------------------------- */







[LISTING TWO]

Caption: Add this to your copy of GRAFIX.H

/* From July, '89 */
/* -------------- */
void far setcoords /* set virtual coordinate space */
 (double left, double top, double right, double bottom);

int far dx (double vx); /* convert virtual x to device x */

int far dy (double vy); /* convert virtual y to device y */

int far dxunits (double vx); /* device x units for vx */

int far dyunits (double vy); /* device y units for vy */







[LISTING THREE]

/* TRIGPLOT.C: Plot sine, cosine, and their sum */
/* X axis is integral, Y axis is floating-point */

#include <math.h>
#include <conio.h>
#include "grafix.h"
#define DEG2RAD 3.1415927 / 180.0 /* degrees to radians */

int px = 999, py; /* previous point */

void main ()
{
void plot (int angle, double (*fcn)(double theta));
double sum (double);
int angle;

 if (init_video (EGA)) {

 /* Scale the axes */
 setcoords (-360, 1.5, 360, -1.5);

 /* Draw the registration lines */
 set_color1 (2); /* green */
 hline (dx(-360), dy(0), dxunits (720)); /* x axis */
 draw_line (dx(0), dy(1.3), dx(0), dy(-1.3)); /* y axis */
 set_color1 (1); /* blue */
 hline (dx(-360), dy(1), dxunits (720)); /* +1.0 */
 hline (dx(-360), dy(-1),dxunits (720)); /* -1.0 */


 /* Plot the sine curve */
 set_color1 (3); /* cyan */
 for (angle = -360; angle <= 360; angle++)
 plot (angle, sin);

 /* Plot the cosine curve */
 set_color1 (5); /* magenta */
 px = 999; /* new curve */
 for (angle = -360; angle <= 360; angle++)
 plot (angle, cos);

 /* Plot the sum of sine and cosine */
 set_color1 (15); /* white */
 px = 999; /* new */
 for (angle = -360; angle <= 360; angle++)
 plot (angle, sum);

 /* Wait for keypress and quit */
 getch();
 }
} /* ------------------------ */

double sum (double theta)
{
 return (sin (theta) + cos (theta));
} /* ------------------------ */

void plot (int angle, double (*fcn)(double theta))
{
double theta, y;

 theta = (double) angle * DEG2RAD; /* convert to radians */
 y = (*fcn) (theta); /* derive value */
 if (px != 999) /* if not first point */
 draw_line (px, py, dx (angle), dy (y));
 px = dx (angle); /* save current point */
 py = dy (y);
} /* ------------------------ */







[LISTING FOUR]


/* LENDIST.C: Produces a histogram of line lengths in a text file */
/* Illustrates integral virtual coords, self-scaling bar chart */

#include <stdio.h>
#include <string.h>
#include <conio.h>
#include <stdlib.h>
#include "grafix.h"
#define MAXLEN 80 /* max length of a line */
#define MAXCAT MAXLEN / 5 /* number of categories */


void main (int argc, char *argv[])
{
double group [MAXCAT]; /* results array */
int category; /* grouping by length (0-4, 5-9, etc) */
double maxcount = 0; /* highest count for any category */
int color = 1; /* color of histogram bar */
char buf [MAXLEN]; /* receiving buffer */
FILE *fp; /* text file pointer */

 /* check command line for a filename */
 if (argc < 2) {
 puts ("Usage: LENDIST <filename.ext>");
 exit (EXIT_FAILURE);
 }

 /* clear the length array */
 for (category = 0; category < MAXCAT; category++)
 group [category] = 0.0;

 /* pass thru the file getting lengths of lines */
 if ((fp = fopen (argv[1], "r")) == NULL) {
 printf ("Unable to open %s\n", argv[1]);
 exit (EXIT_FAILURE);
 } else {
 while (fgets (buf, MAXLEN, fp)) {
 category = strlen (buf) / 5; /* group 0-4, 5-9, etc. */
 ++group [category]; /* count */
 }
 fclose (fp);
 }

 /* find highest count of any group */
 for (category = 0; category < MAXCAT; category++)
 if (group [category] > maxcount)
 maxcount = group [category];

 /* Normalize groupings to 100 */
 for (category = 0; category < MAXCAT; category++)
 group [category] /= (maxcount / 100);

 /* enter graphics mode */
 if (init_video (EGA)) {

 /* scale the histogram width */
 /* leave space on top and left for text */
 setcoords (-22, -2, 100, 23);

 /* label the histogram */
 printf ("Distribution of line lengths in %s\n\n", argv[1]);
 for (category = 0; category < MAXCAT; category++)
 printf (" Length %2d-%2d:\n", category*5, (category*5)+4);

 /* set up the graph area */
 draw_rect (dx(-22), dy (-1), dxunits (122),
 dyunits(MAXCAT + 2));
 set_color1 (1); /* blue */
 for (category = 0; category < 100; category+=10)
 draw_line (dx(category), dy(-0.5), dx(category), dy(MAXCAT));


 /* draw the histogram bars */
 for (category = 0; category < MAXCAT; category++) {
 set_color1 (color);
 if (++color == 16) color = 1;
 fill_rect (dx (0), dy (category), dxunits (group [category]),
 dyunits (0.75));
 }

 /* hold for keypress, then quit */
 getch();
 pc_textmode();
 } else
 puts ("Unable to enter EGA graphics");
}







[LISTING FIVE]

/* RESIZE.C: Effects of resizing virtual coords to fit viewports */

#include "grafix.h"
#include <conio.h>
#include <stdio.h>
#define BORDER 15

void main ()
{
void set_vp (int, int, int, int);

 if (init_video (EGA)) {

 /* Set colors for shading */
 set_ega_palreg (1, ega_blend (RED3, GRN1, BLU0));
 set_ega_palreg (2, ega_blend (RED2, GRN1, BLU0));
 set_ega_palreg (3, ega_blend (RED2, GRN0, BLU0));

 /* Draw 4-pointed star in variously-sized viewports */
 set_vp (1, 1, 160, 120); /* square */
 set_vp (200, 1, 120, 347); /* tall and skinny */
 set_vp (360, 270, 238, 78); /* short and fat */

 /* Wait for keypress and quit */
 getch();
 pc_textmode();
 } else
 puts ("Unable to enter EGA graphics");
} /* ---------------------------------- */

void set_vp (int left, int top, int width, int height)
{
VP_HAN vp;
void drawstar (void);


 vp = vp_open (left, top, width, height);
 set_color1 (BORDER);
 vp_outline (vp);
 setcoords (-120, 120, 120, -120); /* constants scale to vp size */
 drawstar();
 vp_close (vp);
} /* ---------------------------------- */

void drawstar (void)
{
 set_color1 (BORDER); /* border color */
 setfillborder (BORDER);
 draw_rect (dx(-30), dy(-30), dxunits(60), dyunits(60));
 set_color1 (3); /* center of star */
 floodfill (dx(0), dy(0));

 set_color1 (BORDER);
 draw_line (dx(-30), dy(30), dx(0), dy(100)); /* top ray */
 draw_line (dx(0), dy(100), dx(30), dy(30));
 draw_line (dx(0), dy(100), dx(0), dy(30));

 draw_line (dx(-30), dy(30), dx(-100), dy(0)); /* left ray */
 draw_line (dx(-100), dy(0), dx(-30), dy(-30));
 draw_line (dx(-100), dy(0), dx(-30), dy(0));

 draw_line (dx(-30), dy(-30), dx(0), dy(-100)); /* bottom ray */
 draw_line (dx(0), dy(-100), dx(30), dy(-30));
 draw_line (dx(0), dy(-100), dx(0), dy(-30));

 draw_line (dx(30), dy(-30), dx(100), dy(0)); /* right ray */
 draw_line (dx(100), dy(0), dx(30), dy(30));
 draw_line (dx(100), dy(0), dx(30), dy(0));

 set_color1 (1);
 floodfill (dx(10), dy(40));
 floodfill (dx(-40), dy(20));
 floodfill (dx(10), dy(-40));
 floodfill (dx(40), dy(20));

 set_color1 (2);
 floodfill (dx(-10), dy(40));
 floodfill (dx(-40), dy(-20));
 floodfill (dx(-10), dy(-40));
 floodfill (dx(40), dy(-20));
}



[GRAFIX.LIB]

/* Include file for GRAFIX.LIB */
/* EGA/VGA graphics subsystem */
/* K. Porter, DDJ Graphics Programming Column */
/* ------------------------------------------ */

/* Color constants from April, 89 */
#define Black 0 /* standard colors */
#define Blue 1
#define Green 2

#define Cyan 3
#define Red 4
#define Magenta 5
#define Brown 0x14
#define LtGray 7
#define DkGray 0x38
#define LtBlue 0x39
#define LtGreen 0x3A
#define LtCyan 0x3B
#define LtRed 0x3C
#define LtMagenta 0x3D
#define Yellow 0x3E
#define White 0x3F

#define RED0 0x00 /* basic hues for mixing */
#define RED1 0x20
#define RED2 0x04
#define RED3 0x24
#define GRN0 0x00
#define GRN1 0x10
#define GRN2 0x02
#define GRN3 0x12
#define BLU0 0x00
#define BLU1 0x08
#define BLU2 0x01
#define BLU3 0x09

#if !defined byte
#define byte unsigned char
#endif

/* Supported video modes */
#define EGA 0x10 /* EGA 640 x 350, 16/64 colors */
#define VGA16 0x11 /* VGA 640 x 480, 16/64 colors */

/* Function prototypes */
/* From February, '89 */
/* ------------------ */
int far init_video (int mode); /* init display in video mode */

void far pc_textmode (void); /* PC text mode */

void far draw_point (int x, int y); /* write pixel in color1 */

void far set_color1 (int palette_reg); /* set foreground color */

/* From March, '89 */
/* --------------- */
void far draw_line (int x1, int y1, int x2, int y2);
 /* Bresenham line drawing algorithm */

void far draw_rect (int left, int top, int width, int height);
 /* draw rectangle from top left corner */

void far polyline (int edges, int vertices[]); /* draw polyline */

void far hline (int x, int y, int len); /* horizontal line */

void far fill_rect (int left, int top, int width, int height);

 /* draw solid rectangle in color1 starting at top left corner */

/* From April, '89 */
/* --------------- */
byte far ega_palreg (int preg); /* color in EGA palette reg */

void far set_ega_palreg (int reg, int color); /* set palette reg */

byte far colorblend (byte r, byte g, byte b); /* blend hues */

void far get_ega_colormix (int preg, int *r, int *g, int *b);
 /* get mix of red, green, and blue in EGA pal register preg */

/* From May, '89 */
/* ------------- */
typedef int VP_HAN; /* viewport handle type */

void far default_viewport (int height); /* init default viewport */

VP_HAN far vp_open (int x, int y, int width, int height);
 /* open viewport, make it active */

int far vp_use (VP_HAN vp); /* make vp active */

void far vp_close (VP_HAN vp); /* close viewport */

VP_HAN far vp_active (void); /* get handle of active vp */

void far vp_outline (VP_HAN vp); /* outline vp */

int far vp_width (void); /* get active viewport width */

int far vp_height (void); /* and height */

/* From June, '89 */
/* -------------- */
int far egapixel (int x, int y); /* get pixel value at x, y */

int far floodfill (int sx, int sy, /* seed coords */
 int border, /* border color */
 int dir, /* pass as 0 */
 int pleft, int prite); /* make same as sx */




















July, 1989
STRUCTURED PROGRAMMING


Dodging Steamships




Jeff Duntemann, K16RA


When it's time to build steamships, everybody builds steamships -- or so the
chestnut goes. It must be steamship time, then, because lately I've been
dodging them left and right.
The steamships in question are Object-Oriented Programming systems (OOPs).
That acronym may be the most unfortunate or perhaps the most deliciously
ironic one ever used in our industry. Laugh if you must (I do, regularly) but
OOPs are coming of age, and you owe it to yourself to spend a little time in
understanding them.
In a handful of words, object oriented programming is the next generation of
structured methods -- code has married data, and the offspring are Something
Else Entirely. In keeping with the mission of "Structured Programming," my
next several columns will be devoted to object-oriented languages, with an
emphasis on how you can get involved.


The Steam Whistle Bloweth


I first heard the steamship whistles last year at Software Development '88, a
show devoted utterly to (hallelujah!) programmers and their craft. There
wasn't much in the line of products on the show floor, but the OOPs sessions
were SRO. I knew what OOPs were, having worked at Xerox through most of the
time that PARC's seminal OOPs research was going on. I programmed in Smalltalk
on an Alto workstation in 1979, an experience that led me to write OOPs off as
costing more in machine performance than it was worth. I was dead wrong there,
having shot the marathon runner for riding his little brother's tricycle.
At Software Development '89, I saw real OOPs code steaming along at flank
speed. The code was written in a variant of (arrgh) C, but the point was not
lost on me: Taken together, object-oriented methods are a way of thinking
about programming that are tied to no particular language. With minimal
syntactic extension, any language at all can be object-oriented ...
... as we're beginning to find out.


Structure of Structures


Object orientation is extremely difficult to define. Most people don't try to
define it; they simply start by saying, "Object-oriented programming allows
you to create easily reusable software components ..." or "Object-oriented
programming allows you to extend existing modules without requiring source
code ...," or something else. This isn't explaining what object-oriented
programming is, this only tells you what it does -- and only part of what it
does, at that.
Let me be glib and hand you my own definition, which (like most very
high-level definitions) needs some expounding: Object-oriented programming is
structured structured programming. It's the second derivative of software
development, the Grand Unifying Theory of program structure. Choose your
metaphor, I got a million of 'em. What you must understand is that object
orientation is no single technique, no single language, and no single
approach. It is many techniques bound together in synergistic fashion to
produce numerous benefits, under the guidance of a very high-level mindset
that Mike Swaine would call a paradigm shift.
For this reason, it's a little scary, and as with all scary things like sex,
death, or steam locomotives there's a tendency to make it all mystical and
legendary. Resist -- and relax. Rather than a fount of mystical wisdom,
object-oriented techniques are more like modern sanitation: A way of living
that combines numerous small elements -- indoor plumbing, running water,
regular bathing, washing of food, refrigeration, childhood inoculation, and so
forth -- into an easy regimen that has nearly doubled the human lifespan in
the last hundred years. That's not immortality, but it's a helluva step in the
right direction.
Which is how one should feel about OOP: Not instantaneous bug-free
programming, but still a helluva step in the right direction. Over the next
several columns I'll be gradually defining object-oriented programming, and
showing the tools you can use to learn it and benefit from it. I'll have to
beg your patience -- the subject is such that you won't catch the essence in
200 words or less. There is no single good place to begin, so let's catch the
first steamship that happens by.


Catching the NeXT Steamship


I throw regular parties down here, and I exhort the invitees to bring wives,
husbands, or Significant Others. Bruce Webster took me at my word about a
month ago. He has a perfectly wonderful wife, but instead he brought his
current Significant Other: A NeXT workstation.
So significant was NeXT that the party goers forgot all about the pool, the
hot tub, and even my copy of Tetris. What might have been an academic
curiosity or (my own view) a dazzling way to waste several hundred million
dollars has been turned into a killer product by the simple decision to market
the unit to ordinary people, through the Businessland retail chain. $10,000
may seem like a lot of money, but most new cars cost more than that, and if
you're a front-line knowledge worker, your machine may mean far more to your
bottom line than your car.
But I'll leave that decision to you. NeXT interests me because its operating
environment is object-oriented, from top to bottom. Some say jobs was fishing
for publicity and hooking into a fad when he made NeXT an object-oriented
machine. No way -- Jobs used OOP techniques because he simply had no choice.
It was that or drown in complexity.
Which brings me to the #1 raison d'etre for OOPs: The management of
complexity. Some of you may well have tried developing for the Mac. Fewer of
you (I suspect) have taken a stab at Windows development, and almost no one in
the civilized galaxy has bothered to crack the OS/2 Presentation Manager
documentation.
The problem in all cases is the complexity of the API. DOS presents us with a
few dozen function calls, most of them independent and taking at best three or
four parameters. Full-blown operating environments like the Mac, Windows, or
PM contain hundreds of function calls, many of them needing a dozen or more
parameters that interlock with other function calls and other parameters in
ways that make your head spin. It is the nature of the human mind to focus on
one or two things at once. (Close your eyes and try to clearly imagine a field
of more than four abstract items and see how easy it is.) Developing for one
of the operating environments requires that you float that whole interlocking
mess in front of your mind from the word go.
A couple of years ago, Apple produced an object-oriented wrapper for the Mac
API called MacApp. It imposed a structure on the Mac API by identifying the
elements of the interactive environment as quasi-independent components.
Windows, menus, scroll bars, buttons, all of these were set out in a hierarchy
looking a lot like a taxonomy chart. Each type of element was called a class,
and to create and use a window, you simply created an instance of the window
class. This window instance was an object, and it contained an interface to
only that code and data needed to create and use a window. You might think of
objects as halfway between code and data, or as a larger concept that embraces
both code and data. But never forget that code and data must be considered
together in OOPs, and nevermore put asunder.


Each Object in its Place


I've drawn up a simple object hierarchy chart in Figure 1. It's not intended
to represent MacApp specifically, but could apply to nearly any windowing
environment. Each of the boxes represents an object type or class. Each class
has a specific role in life, which I've written near each box. The Screen type
is a full screen, and models the relationship between an abstract display and
the real physical display controller board. If you send a character to a
Screen object, it will pass those characters along to the display controller.
That's about all Screen can do.
Window adds a little pizzazz to Screen by being a rectangular chunk of the
screen, perhaps with a border. A Window has a size and a position within a
Screen, and may be dragged around the screen with the mouse or cursor pad, and
enlarged or reduced in size. That's about all that a Window object can do.
Note, however, that if you send characters to Window, it will pass them onto
the display controller. It does this using the same mechanisms built into type
Screen. Window inherits everything that Screen is, by virtue of being beneath
Screen in the object hierarchy.
Beneath Window are two different classes: Field and Menu. A Field has the
power to display and edit a value of some kind. A Menu displays several items
and chooses one from the group. Field and Menu are distinct from one another,
but they inherit everything that both Screen and Window have within them: Both
menus and fields have a size and a position and can be dragged around the
screen, and both send characters to the display controller.
At each level in the chart, the objects get a little more powerful, a little
more specific -- and a little more useful. Beneath Menu are two real varieties
of menu, PopUp and Matrix. Both inherit the essence of a menu -- to display
several items, then accept input from the user to choose one of those items --
but each does the actual work in its own way.
The huge mass of detail of a windowing environment API is distributed across
the hierarchy in a rational, graspable manner. Direct control of the display
hardware is gathered behind class Screen. Sizing and dragging of windows is
gathered behind class Window and so on.
In short, to use a pop-up menu object in this scheme, you narrow your
attention to one item, a software model of a real menu that draws together all
the disparate function calls and data items that you would otherwise have to
hunt up and pull together on your own from that mess of an API. I should point
out that there are other ways to simplify an API, but object-orientation
provides additional benefits related to the inheritance concept that I will
touch on in future columns.
There's a lot more to both MacApp and object-oriented programming than just
this, of course. Managing complexity is a major benefit of OOPs, but not the
only one, and inheritance is one mechanism within OOPs, but again, there are
more. If you want to know more from a Macintosh perspective, get the superb
Object-Oriented Programming for the Macintosh, by Kurt J. Schmucker (Hayden
Books, 1986.) The point I'm making is that objects are one way of managing
complexity by hiding details behind a software model that relates to the
problem at hand and no more. The hierarchy of MacApp allows a programmer to
see only what he needs to see at any given time in the development process
without tripping over unrelated details. It was Mr. Structure himself, Niklaus
Wirth, who defined structured programming as the artful hiding of details.
Object-oriented programming is a world-class method of hiding unnecessary
details, and it's about as artful as they come.
Which is fortunate, because the Presentation Manager API makes the Macintosh
look elegant and obvious by comparison. We may never really know how difficult
the NeXT machine would be to program in the absence of an OOPs, because the
operating system is the OOPs, and what structure MacApp imposes on the
underlying Mac API, Next Step imposes on itself. All the components of the
operating system and the elegantly-designed (and monochrome, hurrah!) user
interface are objects.
NextStep incorporates a new language called Objective C. Objective C
represents an opinion watershed for me -- allowing me to believe that, by
gully, this object stuff may civilize C yet. Still, this is the un-C column,
and I'll leave it to my cohorts here to fill you in on the details of both
Objective C and NextStep. To get a flavor for Objective C and OOPs at the same
time, read Object-Oriented Programming: An Evolutionary Approach by Brad Cox.
Cox defined Objective C, and he has an intriguing metaphor for objects:
Software IC's, plugable modules that may be considered "black boxes" and used
without alteration in many diverse applications. This is an important facet of
any OOPs, but by no means the only or even the most important one.

The point to carry away from the NeXT steamship is this: By the artful hiding
of details, NextStep makes the NeXT API much easier to grasp and manipulate
than Mac, Windows, or PM. The component-modelling nature of OOP makes such
detail hiding easy. If the OOPs paradigm shift does nothing else, it will have
earned its keep for that alone.


Steamship Seen Through Windows, Darkly


Neither Windows nor PM have anything resembling MacApp yet ... or do they? I
have fielded some tantalizing rumors indicating that Microsoft is about to go
head-to-head with Borland with an environment-based Pascal compiler.
If it's true, it will be an intriguing mix: An object-oriented Turbo Pascal
compatible Pascal compiler that (gasp) generates Windows applications.
Back in my PC Tech Journal days I told Microsoft that Windows would not hit
the big time until there were a development environment for it that
intelligently managed the complexity of the API. That advice sank like a
stone, as did my editorship at PC Tech Journal, as eventually did PC Tech
Journal itself. In the intervening years we've seen a handful of solid
applications appear for Windows -- and handfuls are emphatically not the big
time. The Windows API is daunting, and the Windows Software Development
Toolkit (SDK) is chaotic. Something needs to bring order to the Windows and PM
APIs, and it would be just too deliciously ironic if Microsoft, the
C-Is-The-Only-Way-To-Go (except maybe for Basic) company decided to use a
Pascal compiler to do the job.
By the time you read this, we should know the truth.


Board the SS Turbo!


By the time I heard the Microsoft rumor, the steamships were already getting
pretty thick around here. Then another mast heaved over the horizon, with
Philippe Kahn tooting the whistle. This one is for real: In May Borland
released Turbo Pascal 5.5, with new technology making Turbo Pascal fully
object-oriented in some interesting and innovative ways.
Contrary to conventional wisdom, it doesn't take a whole new and exotic
language specification to support object-oriented programming. Borland did it
by adding only four new reserved words to Turbo Pascal: OBJECT, VIRTUAL,
CONSTRUCTOR, and DESTRUCTOR. If your existing source code does not use those
four reserved words, all object-oriented extensions will be completely
transparent to you. You can use objects or ignore them; it's your choice. (If
you ignore them, however, you're nuts.)
There is no genuine standard for object-oriented Pascal, so Borland drew on
numerous existing languages: C++, Macintosh Object Pascal, and Oberon, the
latest language concocted by Niklaus Wirth. Unlike C++, the object extensions
are not a separate module or some sort of preprocessor. And unlike the older
OOPs like Smalltalk and Actor, you can choose the degree to which a program
will be object-oriented. In Turbo fashion there are mainstream objects and
optimized objects, and you the programmer get to decide which sort of object
is the best for a given application.
Welding code to data presents some novel problems in debugging, so Turbo
Debugger was upgraded simultaneously, with new inspectors built in
specifically to poke around inside objects. One nice touch is a hierarchy
inspector, which shows a little taxonomy chart of all object types present in
the module being debugged.
I haven't had enough time yet to write any significant code in object-style
Turbo Pascal, but what I've seen so far is clean to the point of being
elegant. I'll be spending a lot more time on Turbo Pascal objects in future
columns. Right now, I'd say, Upgrade. Without hesitation.


Already at the Dock


With all the steamships steaming into shore, it's easy enough to ignore the
ones that have been at the dock for some time. Apart from Zortech C++ (which
I'll let the C folks discuss) there are two longstanding reasonably priced
OOPs for DOS: Smalltalk/V and Actor. Both have been around since '85 or '86,
and I have been using both off and on since that time. Both are solid
products, well-documented, and a lot of fun.
Smalltalk/V, at $100, is the cheapest OOPs you can buy right now, and if you
have a disposable $100 it's one of the best ways to get comfortable with the
concepts of object-oriented programming. The reasons are two: Smalltalk is
totally object-oriented, in a way that permeates every corner of the language.
If you understand Smalltalk, you have OOPs in your hip pocket. Also, Smalltalk
is certainly the best documented OOPs in history, due to a superb line of
books from Addison-Wesley that came out of Xerox in the early 1980s.
Digitalk's own documentation is excellent, and with the Addison-Wesley books
you have everything you need.
Space is short, so I'll leave Smalltalk for the time being. My next column
will focus on Smalltalk, and I'll provide the whole list of Smalltalk titles,
along with information on the various versions of Smalltalk on the market
right now.


The Pioneers and the Arrows


Actor is a bit of a nonesuch. I've often felt that the old saw "Pioneers get
arrows" applies with considerable force to The Whitewater Group. They had the
vision to provide an easy-to-use development environment for Microsoft Windows
back in 1986, while Microsoft still forces developers to slog through the
swamp of their intolerable SDK. Yet for all that, Actor hasn't taken over the
Windows world as I was sure it would two years ago.
Unlike Smalltalk (with which it is often unfairly compared) Actor creates
stand-alone programs that do not require the presence of the Actor product to
execute. (All generated programs, however, are WinApps and require Microsoft
Windows.) The code is fast, the manual is very good, and aftermarket books are
beginning to appear. I've seen the manuscript for Object-Oriented Programming
With Actor by Marty Franz, and it will be well worth its cover price both as a
tutorial for Actor users and as a preview for those who want to see what the
language is like before they buy the product. Expect the book to appear in
October.
Apart from being the only rational development environment Windows has ever
had, much of Actor's value lies in the object hierarchy they give you, out of
the box, for managing Windows complexity. The leverage of the package has to
be experienced to be believed. You really can write a text editor window in
two Actor statements, just as their ads say, because Whitewater has already
created the text editor class. All you need to do is create an instance
variable of that class, and you have a text editor window. Like so:
 MyEditor := New(EditWindow, ThePort,"editmenu" "Editor",
nil); Show(MyEditor, 1);
If you need to gussy up the text editor window a little, you don't have to
start from scratch. You just hang a new class beneath the editor window class
on the hierarchy, and add what needs adding. Inheritance allows your new
editor class to be everything Whitewater's canned editor class is, with your
own custom extensions melted seamlessly in.


PRODUCTS MENTIONED


Actor V1.2 The Whitewater Group 906 University Place Evanston, IL 60201
$495.00
Object-Oriented Programming For The Macintosh by Kurt J. Schmucker Hayden
Books, 1986 ISBN 0-8104-6565-5 607pp. $34.95
Object-Oriented Programming With Actor by Marty Franz Scott, Foresman & Co.,
1989 ISBN 0-673-38641-4 $24.95
Smalltalk/V Digitalk, Inc. 9841 Airport Blvd. Los Angeles, CA 90045
213-645-1082 $99.95
Turbo Pascal V5.5 Borland International 1800 Green Hills Road Scotts Valley,
CA 95066 408-438-8400 Standard package: $149.95 Professional package: $250
Upgrade: $34.95 from TP 5.0 $59.95 from Pro Pack 5.0 $75 from TP 4.0
Any user interface component that you might want to put in a Windows
application is represented on the Actor object hierarchy. They're like
templates: You grab a template off the hierarchy and whack out a new copy of
an object for your use. The tangled web of the Windows API is way back there
somewhere, so far back you can forget about it and get the real work of
writing your application done.
This is amazing stuff. Why then hasn't Actor gotten the recognition it
deserves? As much as I like Actor (and in many ways I like it better than
Smalltalk, with which I have been acquainted for almost ten years) I see two
serious problems: First of all, Actor is a whole new language spec. It's
something like Pascal, a little like C, a very little like Forth, and quite a
bit like Smalltalk. These are strange bedfellows indeed, and while it's a
perfectly reasonable language spec, I have found no compelling reason why it
had to be as different as it is from Pascal or C. Life is too short to get
really, really good at more than one or two languages, and I think the
learning curve has driven away many potential users.
Second, and related to the first problem, is that Actor costs $500. This is
cheap if you want to develop for Windows, but if you want to see if a new
language spec is worth learning, there's no inexpensive way to try it out
short of stealing a copy somewhere. Baskin-Robbins makes those teeny spoons
for giving away samples of Garlic Avocado ice cream because most people are
not going to spend a buck-fifty on a cone full of stuff that they might or
might not want to swallow. If Actor were a WinApp-generating implementation of
Pascal or C this would not be the case. Alas, Actor needs a cheap version of
itself for the language tire-kickers.
I'll have more to say about Actor in a future column, particularly if rumors
of a Windows version of Quick Pascal turn out to be true. If you have any need
to develop for Windows, trust me: The $500 price tag is cheap compared to the
time and torn hair that you will save over using the SDK. Learning it takes a
week or so, but what's learned is learned -- and you will pick up plenty of
OOPs savvy along the way for free.


Steaming for Home


Last night I heard that a friend of mine had kicked off a cold fusion reaction
in a pickle jar full of laboratory-purity heavy water. Chemical reaction? Kind
of hard when there's only one chemical in the jar. (He washed the dill residue
out before using it, trust me.) OK, but is it fusion?

Who knows? But hey, who cares? If more energy is coming out of the jar than is
going in, it could be angels on treadmills and it wouldn't matter. If the
process is perfectible and scalable, this could be Year One of the Millennium.
My swimming pool contains about three gallons of heavy water (all water on
Earth, in fact, contains one part in 7000 of heavy water) which would heat my
house and run my car for about ten thousand years. Separating ordinary and
heavy water can be done with solar energy and requires no exotic materials or
equipment and does not involve radioactivity.
The point is this: You don't need Lawrence Livermore Labs to change the world.
No sir. You don't need a supercomputer to change the world. Uh-uh. The age of
Desktop Physics has returned, after a 70-year hiatus.
No, all you need is a pickle jar, a PC clone, and Turbo Pascal.



























































July, 1989
SWAINE'S FLAMES


The Information Revolution and the Morning News




Michael Swaine


Years ago, when I worked for a company whose name need not be mentioned, we
had a visit from a man with a thick accent. He would be returning to his
country shortly, he said, and wished us to sell him a certain item of computer
equipment to take back with him. There were some legal problems, yes, involved
in taking such equipment from our country to his, but he told us not to worry
about that. And the less we said about the transaction, he let us understand,
the better it would be for all concerned. I believe he paid in cash.
Yes, I'm confessing to taking part in a smuggling operation, but before you
turn me in, let me plead the extenuating circumstances. The computer was an
Exidy Sorcerer, the country to which the computer was going was Brazil, the
law being violated was Brazil's, and the smuggler only wanted to use the
computer for bookkeeping in his small business.
Brazil's protectionism laws, intended to stimulate the growth of an indigenous
computer industry, have rather held up the computerization of the Brazilian
economy, according to Louis Rossetto in his non-editorial in the April
Language Technology. Smuggling and the black market are becoming necessary
evils in the face of $2,000 Rio de Janeiro street prices for XT clones.
Rossetto cites evidence that the protectionism is crumbling, including
Brazil's decision to consider an Olivetti subsidiary "national." The damage it
has done, of course, won't go away quickly.
But Brazil's travail doesn't amount to a hill of coffee beans in the crazy
world of Pacific rim economics as described by James Fallows in the May
Atlantic.
Fallow describes a condition, one of whose syndromes is protectionism and one
symptom is the 47th Street Photo paradox: if 47th Street Photo in Manhattan
could export to Japan the Japanese-made goods it sells to New Yorkers, it
could make a fortune. Charges of protectionism fly in both directions, usually
couched in the polite terms that characterize U.S.-Japan relations. But the
facts are pretty one-sided. "How protectionistic can a country with a
$10-billion monthly trade deficit really be?" Fallows asks. Japan, on the
other hand, is keeping its people relatively poor to compete economically in
the world market. Fallows says that Japan's policies are ultimately ruinous
for everyone, including Japan, but sees little evidence that Japan will change
its ways soon.
As I was writing this column, I heard on National Public Radio of the suicide
of one Japanese bureaucrat involved in the corruption that is bringing down
the government. It's dramatic news, but even the fall of the government is not
expected to affect Japan's economic policy. Another story I heard on NPR while
writing this column concerned the possibility of the Supreme Court's
reconsidering Roe vs. Wade, the decision that legalized abortion. Whatever
your feelings about the legality of abortion, you should consider the
implications of a side issue that is also being brought before the Court, an
issue that touches on the relationship between information and power. The
issue is whether or not restrictions should be placed on access to abortion
information.
Everyone reading this magazine is intimately involved in a technological
revolution, one of the most radical effects of which will be a redistribution
of access to information of all sorts, combined with an increased dependence
on information in society. The fact that purchasing power now rests on the
ability to provide an acceptable credit card number and expiration date is
just one example of the "informatizing" of society. Those of us who are in the
vanguard of the revolution should be especially sensitive to attempts to
redefine access to information. If knowledge is power, then access to
information is enfranchisement.
If the Supreme Court should decide to overturn Roe vs. Wade the decision may
be made out of strong moral convictions regarding abortion. But it is also
true that the most effective way to stifle dissent is to keep the dissenters
ignorant. If access to information is enfranchisement, then restriction of
access to information is tyranny.
On the lighter side of tyranny, in the May Lotus magazine, Lindsy van Gelder
examines groupware, a software category some detractors have labelled fascist.
"Groupware," she quotes Institute for the Future fellow Paul Saffo as saying,
is software that "can be used effectively only by two or more people." Is it a
real opportunity for developers or just a buzzword? Van Gelder paints an
unpromising picture of an ill-researched concept ill suited to the way people
work today -- but that picture would also have fit the personal computer a
decade ago. One working model of groupware is the CASE model, without which
some large software projects would never get done.
Shopping for exotic fauna for my garden of typos, I recently unearthed
"terrabyte WORMs" in a computer magazine editorial. It should be "terrabite
worms," of course, an allusion to the appetites of annelids.







































July, 1989
OF INTEREST





Microsoft began recently shipping its QuickPascal Compiler, a Pascal
implementation that the company claims is Turbo Pascal source-code compatible.
The compiler is built around Microsoft's Quick-family technology (an
integrated compiler/editor/debugger environment) while further supporting
object-oriented extensions as defined by Object Pascal.
QuickPascal (QP) provides a window-based editor that supports multiple views
into a source file or simultaneous editing of multiple files. Windows can be
resized and overlap with up to nine text-editing windows opened at one time.
QP also provides mouse support. The QP debugger supports single-stepping into
and around procedures and object methods. Microsoft claims that QP is the
first Pascal compiler to support integrated debugging of assembler routines
from within the environment, allowing the display and modification of CPU
registers and flags. Like other Quick-family compilers, QP offers on-line
hypertext reference for context-sensitive help.
QP uses four non-standard Pascal key words: the object-oriented extensions of
Object, Override, and Inherited and the non-OPP keyword Cstring. (Note that
the four keywords new to Turbo Pascal 5.5 are different.) Except for the BGI,
QP is 100 percent compatible with Turbo Pascal; QP does have its own graphics
library that mimics the BGI. QP requires 512K of memory and DOS 2.1 or higher
and retails for $99.
In conjunction with Microsoft's QP announcement, several developers announced
third-party support. Turbo-Power Software says its Turbo Professional 5.0 (a
library of general purpose procedures and functions) and B-Tree Filer 5.0
(database-related procedures) are QP-compatible. Likewise, Blaise Computing
said that its Power Tools Plus 5.0 (general-purpose library of interrupt and
screen control procedures), Asynch Plus (communications routines), and Power
Screen (screen management library) also supports QP. Reader Service No. 20.
Microsoft Corporation 16011 NE 36th Way Box 97017 Redmond, WA 98073-9717
206-882-8080
The recently announced Intel 80486 is essentially a souped-up 80386,
delivering 2X-5X the performance at equivalent MHz. To support on-chip cache
and multiprocessing, the 486 has six new instructions, otherwise the 486 is
chiefly a tweak of the 386. According to Intel, the company studied all
available pure-386 software and optimized the most often-used instructions to
make them one-cyclers to boost performance. The 486 has an on-chip 80387 and
cache memory/control with 8K and five-level pipelining. Intel has working
silicon and expects production quantities by fall of this year. The first 486
machines, which should be available by winter, are expected to be
file/database servers and other high-performance boxes.
At the same time, Intel announced a C compiler for the 486 (along with other
software development tools), a 386SX, a low-power-consumption chip for laptops
that increases battery life by 30 percent and 33-MHz 80386 availability. Other
announcements included a 32-bit LAN coprocessor, EISA and MCA chip sets, and a
new programmable logic device.
Initial pricing for the 80486 will be $950, which is about $50 more than the
cost of a 80386, 80387, and supporting chips. Intel hinted about a price drop
in the 386 but didn't say when. The 486 is 100 percent binary compatible with
all previous Intel CPUs. Reader Service No. 23.
Intel Corporation 3065 Bowers Ave. Santa Clara, CA 95052-8065
A new set of routines designed to help in the development of user-interfaces
has been released by Maxx Data Systems Inc. The routines, called Creative
Interface Tools, allow programmers who use any of Borland's Turbo languages
(C, Pascal, Basic, and Prolog) to create bit-mapped screen fonts, bit-mapped
icons, graphic mouse cursors, and multi-level menus. Among the specific tools
are a Graphic Symbol Designer, Menu System, and Keyboard Processor with mouse
support. The complete package sells for $69.95. Reader Service No. 22.
Maxx Data Systems, Inc. 1126 S. Cedar Ridge, Ste. 115 Duncanville, TX 75137
Logitech Inc., a company usually associated with mice and Modula-2, has
started shipping a sophisticated source-level debugger for OS/2 environments
named the MultiScope Debugger. The program allows you to debug programs
written in Microsoft C, Pascal, MASM, Logitech Modula-2, and IBM C/2,
Pascal/2, and Macro Assembler/2. You can use MultiScope under either
Presentation Manager or OS/2 text mode and debug at run-time (using
conditional or unconditional breakpoints and memory or symbolic watchdogs to
monitor program execution) or in a post-mortem mode (to examine programs after
a run-time error).
One of MultiScope's unique features is a graphical representation of a
program's data structure. This lets you explore dynamically allocated data
structures and their internal relationships. The debugger also allows you to
look at your program from 13 different simultaneous views, including views
from the perspective of the source, data, memory, register, threads,
breakpoints, and so on. The debugger lets you evaluate expression, execute
function calls, debug child processes and dynamic link libraries, and perform
remote debugging. Although DDJ editors have not yet had the opportunity to
perform hands-on testing of the debugger, the demonstration provided for us
was very impressive. The debugger sells for $299. Reader Service No. 25.
Logitech, Inc. 6505 Kaiser Dr. Fremont, CA 94555
PHIGS is available on PCs with the release of Template Graphics Software's
Figaro/386. Figaro/386 is a version of its Figaro graphics software designed
for 80386-based PCs. The Figaro/386 offers standard PHIGS (Programmer's
Hierarchical Interactive Graphics System) features such as hierarchical data
structures, geometric modeling, and interactive input. The Figaro toolset
gives programmers the ability to render and update 2-D and 3-D graphics
objects in one or more views, deal with trivial and incremental updates to the
display, and to develop programs that use advanced input functions (full event
input, programmable triggers, and soft input devices like screen buttons and
sliders). To use the Figaro system, developers simply relink their
applications to a resident Figaro library. Figaro/386 sells for $1,595. Reader
Service No. 26.
Template Graphics Software 9685 Scranton Rd. San Diego, CA 92121-9810
Fortran programmers may find a new publication, The Fortran Journal, of
particular interest. According to editor Walt Brainerd, the journal contains
articles about people and organization involved with Fortran, product
announcements, product reviews, book reviews, programming hints and routines,
and reports on Fortran 88. The Fortran Journal is published six times per year
at the subscription rate of $28/year in the US and $36 elsewhere. Reader
Service No. 27.
Fortran Users Group P.O. Box 4201 Fullerton, CA 92634
The Open Software Foundation (OSF) has released to its membership for review a
complete set of specifications and plans for its OSF/1 offering, which
combines the OSF operating system and OSF/Motif. (See "The OSF Windowing
System" by Kee Hinckley, DDJ, March 1989.) The operating system architecture
provides a portable and interoperable applications platform that supports
industry standards and specifications such as POSIX and the X/Open Portability
Guide 3. And it provides a smooth upgrade path, allowing for the addition of
extensions with minimal disruption to the existing kernel. OSF is waiting for
member feedback on the specifications before it prepares the final version,
which will be released in four stages: The Vendor Kit, the Application Kit,
the University Platform, and the Commercial Platform. The OSF is an
international, not-for-profit organization dedicated to the development and
delivery of an open, portable software environment. Reader Service No. 32.
Open Software Foundation 11 Cambridge Center Cambridge, MA 02142
XVT, a cross-platform software development tool that allows programmers to
develop and maintain one set of application source code, now supports
Presentation Manager in addition to Windows and the Macintosh. (For an
in-depth description of XVT, see "The Portability Dream" by Margaret Johnson,
DDJ, March, 1989.) Also, XVT (which is short for "Extensible Virtual
Toolkit,") is now available from Graphic Software Systems. The tool creates
100 percent portability between the three graphical user-interface
environments. Additionally, the XVT enhances programmer productivity because
of the increased capacity for native toolkit calls and programming details. We
recently spent some more hands-on time with XVT and continue to be impressed
with this package. The XVT version for Windows supports the Microsoft
Windows/286 and Windows/386 software development kits (SDKs), including
Microsoft C. XVT for Presentation Manager supports Microsoft C and the
Microsoft or IBM OS/2 1.1 Toolkits. XVT for Macintosh supports Think C or the
MPW compiler. A Unix-based X Window System version is planned for later this
year. Each package costs $595. Reader Service No. 31.
Graphic Software Systems 9590 SW Gemini Drive Beaverton, OR 97005 503-641-2200
Alcyon Corporation has announced that its new software development utilities
package, the Regulus-386 Tool-Kit, fully automates the management of
80386-based real-time applications software development. The Regulus-386
Tool-Kit supposedly brings most Unix System V utilities to Alcyon's
Regulus-386 operating system, a real-time Unix-compatible operating system for
the 80386. This means PC-based real-time application developers can take
advantage of the standard Unix tools to speed up the program development
cycle. The Tool-Kit provides such utilities as: source code control system,
make, vi screen editor, Bourne shell, C shell, and more. The cost is $800.
Perpetual upgrades are available at additional cost. Reader Service No. 30.
Alcyon Corporation 6888 Nancy Ridge Drive San Diego, CA 92121 800-748-5858
619-587-9968 (from inside California)






























August, 1989
August, 1989
EDITORIAL


So Long, Good Friend




Jonathan Erickson


One of DDJ's more unique and endearing qualities is the affinity that exists
between the magazine's readers, writers, and editors. This kindred spirit is
one of the things that makes DDJ special. But the downside of a relationship
like this is that we're all touched much more deeply by any loss. Such is the
case with the death of Kent Porter, our senior editor who passed away the
first week in June. His unexpected death leaves all of us -- those who were
used to seeing him every day and those of you who had grown used to hearing
from him every month -- with an empty feeling.
Kent was a special person whose association with DDJ goes back many years --
first as a reader, then as a contributor, and finally as a staff editor. Most
recently, Kent wrote about graphics programming, an area that was a particular
interest of his. Before that, he authored the "Structured Programming" column
where he became known as an advocate for Modula-2, Pascal, and clear thinking
in general. And, over the years, he wrote dozens of articles for DDJ and just
about every technical magazine in the programming field. Add his more than 20
books (most of them on various aspects of programming, although his first book
was entitled Building Model Ships From Scratch) and you begin to see just how
prolific and disciplined he was. To a great extent, Kent's technical prowess,
sense of humor, and clarity of expression embodied the notion of what DDJ is
all about.
Programming was just one of Kent's many interests. He could speak six or seven
languages (nope, not programming languages, he started learning Italian
shortly before his death) and, believe it or not, he was an accomplished
"needlepointer" who made up his own designs. As lucky as we were to know and
work with Kent, he felt he was lucky too. He recently told former DDJ editor
Ron Copeland that he was doing what he had always dreamed of when he was
growing up -- writing books and magazine articles. He added that working at
DDJ was the most satisfying job he'd ever held. While that certainly makes us
feel better now, it's more gratifying to know that Kent was happy and
satisfied with what he was doing.
Although his column will end with this month's issue, Kent had a couple of
articles we hadn't gotten around to publishing, and those will see the light
of day over the coming months. He'd also just finished updating his excellent
book on Stretching Turbo Pascal, and I hope you'll be seeing it in the
bookstores before long.
As a lasting memorial to our friend, we're in the process of setting up a
scholarship fund in Kent's name. It will be an annual award to a deserving
computer science student, and I'll be providing more details on it in a future
issue.
In last month's editorial, I mentioned that we're providing listings via an
on-line service built by David Betz and Bill Garrison. The response was great,
with several hundred of you using the service to download files throughout the
month and we're continuing to expand the available material. Again, the number
is 603-882-1599. Dial it up and send us some e-mail with your comments.
Those Macintosh aficionados among us, like John Kirkpatrick of Houston and
Grant Schampel of St. Paul, have commented that it's been a couple of months
or so since we ran a Mac article. The reason for the dearth is that we've been
stockpiling Mac articles for another special issue named Dr. Dobb's Macintosh
Journal, which is due out next month. The first thing you'll notice about the
issue is that it is packed with code. One article, in fact, has over 1800
lines of code while another has about 1000 lines. The articles range from
discussions of device drivers and memory management to object-oriented and
32-bit color programming. In the process of putting the special issue
together, we've also been able to assemble a good selection of additional
code-intensive Mac articles that we'll be running in the regular DDJ just
about every month.









































August, 1989
LETTERS







More on Superlinearity -- and More


Dear DDJ,
I am writing in reference to Michael Swaine's column "Programming Paradigms,"
in the April 1989 issue of Dr. Dobb's Journal. I have always enjoyed reading
Swaine's columns and I am pleased to be writing to him.
I wanted to comment on his discussion of superlinearity. I share his
skepticism about the results of Rao and Kumar about superlinear speedups with
parallel processing. I feel that their example (in fact any example) of
superlinearity falls into Swaine's first case, that is, of a better algorithm
being used in the parallel case.
Nine processors, each running a depth-first search on a part of the search
tree, are not together doing a depth-first search. They are running a similar
(but different) algorithm that samples each part of the tree for solutions.
Suppose we wanted to run exactly the same algorithm on a sequential processor.
First we implement a simple light-weight process (or thread) system with nine
threads. Each thread simulates one of the nine processors running in parallel
in the parallel solution. This method will exactly track the parallel
algorithm at one-ninth the speed (plus a small constant factor for the
multiprogramming). The constant factor for the thread switching only requires
handling the timer interrupt and swapping register.
The exact value of the thread-switching overhead is not important anyway
because we know it will be a constant factor and we can make it arbitrarily
small by increasing the time slice value and considering larger trees. This
method will approach exact linearity in the limit and it is only the large
problems that one would benefit from using parallelism any way.
There are other ways to look at the same idea. We could dispense with actual
multiprogramming and instead just sequentialize the parallel algorithm. That
is, have the algorithm keep nine stacks and service them in rotation.
The reason that their solution seems to produce superlinearity is (as Swaine
mentioned) that there are several solutions and the depth-first search method
always goes left to right and will lose out to the parallel algorithm if the
solutions tend to cluster elsewhere. Another solution would be to modify the
depth-first algorithm to randomly decide to explore either the left or right
subtree first and stack the other subtree. This method would visit a random
leaf node first and then spread out on both sides of that node (randomly
spreading left or right at each step) until it reached both sides of the tree.
If it turned out that it would be better to visit leaf nodes randomly (and not
spread out from a randomly selected left node) you could pick randomly from
the stack rather than always taking the most recent inserted element (which
would make it a list, I guess).
As long as I am writing, I thought I might comment on Swaine's ideas for a
first course in computer science. I am interested in this because I am an
associate professor in the computer science department at the University of
New Mexico (although I am currently on paternity leave).
First, I might mention that we just added a course called "Programming
Paradigms" although at the senior level. The thing I liked best about Swaine's
proposal is the emphasis on reading programs rather than writing them. If the
analogies to reading and writing natural language or learning Morse code hold,
then it would be much easier to learn to read a program than to learn to write
one. As a result you could cover much more interesting and complex programs in
the first course. In addition, learning program reading would help later in
doing program maintenance, which we know is a major activity of working
programmers.
A related proposal I have been hearing for a while is that the first course in
computer science should (like the first course in other sciences) survey all
the important parts of the field rather than just teach programming. I would
tend to merge Swaine's proposal into that one since computer science is much
more than programming and even the more general area of programming paradigms
is just a part of the whole of computer science.
I do not see changes coming soon, however. I have been trying for several
years to get our department to teach Scheme in the first programming class.
(Scheme, rather than Lisp, so we can use Abelson and Sussman's book.) Then the
students could be writing interesting programs (like Eliza or symbolic
differentiation) rather than worrying about the problems of non-conditional
ANDs in Pascal.
Let me close by saying again how much I enjoy Michael Swaine's columns. Keep
them coming.
Charles Crowley
Albuquerque, New Mexico


Reading, Writing, and New Technologies


Dear DDJ,
Contrary to what editors of PC rags think, reading skill in many languages and
(I groan at the use of this too hip word) paradigms is not the most important
skill that one can and should learn from one's first CS course. True, reading
is a useful skill and one should learn it but it is hardly the most important
thing to learn about computers and computer languages. I think that if you
have to rank the items that you would most like your students to walk away
with from a programming class they would be:
1. Learn to type -- this skill can be used for many wonderful things and can
even be used in areas of life having nothing to do with computers.
2. Learn to read obtuse dreck out of manuals -- this skill helps one not only
over and over in CS, but also in the home when attempting to assemble a
Japanese bicycle, or Sears paraphernalia.
3. Learn to pay attention to detail -- this skill is co-titled "get the
semi-colons in the right place" but again it transfers to painting; writing
specs; designing automobiles; managing people, money, or time; playing piano
or video games. If you want to do any of these right you must be concerned
with detail.
4. Learn responsibility -- the computer does exactly what you tell it, no
more, no less. If you want it to do the right thing, you alone are responsible
for the product of your code.
You can get all of this by teaching any computer language, however, the
emphasis is on writing, not on reading. When reading you can skip over
details, when writing you can't. Reading is lazy in comparison to writing.
By the way, are high-level languages like Fortran a different paradigm from
assembly code? Is Quicksort a different sorting paradigm from, say, bubble
sort? Is OOP with multiple inheritance a different paradigm from the
traditional single inheritance OOP? Were big tail fins on cars the new
paradigm in car design or just style? Is the word paradigm coming to mean
anything that is a little different from something else but you want to
emphasize how hard it was for you to learn the new thing by claiming that the
new thing is revolutionary or radical? This hardly matches Khuns' [sic] use
(yes, I know he used the word in 187 different ways in the 203 uses in his
book) where he refers to one common mindset (Newtonian mechanics) that had to
be replaced in light of new knowledge (relativity). Note the words "common"
and "replaced." OOP is not replacing assembler and neither is parallel
computing replacing sequential computing. They have all been around for years
and are different fields of endeavor, they are not alternate views of one
reality. The way Swaine uses the word paradigm is cute, adds zest to his
articles (which I enjoy quite a bit, don't misunderstand me), and sparks
controversy because it is fundamentally wrong, but somewhat defensible due to
the vagueness of the word to begin with. It's a good way to spark reader
feedback but is on the same level as using cheesecake photos to spark sales.
If I may render the opinion of an old computer hack, message passing,
event-driven windows systems are new and an exciting territory to someone from
the Hi-Level App school but is just business as usual to us ASM systems hacks
who have been writing interrupt driven (message from the data unit #4 just
came in at a higher priority than the disk, right when we were in the middle
of ...) device handlers since the early 50s. We just wanted to share the joy
of systems hacking to all the Apps writers. Yes, it is graphical and sexy and
new and different from what you are used to, but revolutionary? Was structured
programming and indenting revolutionary? Yes, people wrote spaghetti code with
gotos but others of us felt is was unreadable and developed styles to limit
the code flow so we could understand what was going on. When those became
standardized and were given names in high-level languages was that the
revolution? And now OOPs are the great new snake oil of the 90s. Boy, when you
get objects you will really be set free! No more hard code to write. You want
to print a floating point number you just tell it DISPLAY YOURSELF and it
knows what to do. The fact that Basic has kept track of strings and integers
and floats and printed them appropriately for years is not germain. The fact
that for years people have kept fields in records to identify what type of
record it is so that they can CASE to the right code is beside the issue. In
OOPs it is all automatic. All your CASES are hidden from view, the code writes
itself! Well, actually you do have to write all the separate cases, but
instead of keeping them in one file of subroutines you can keep them in
separate little files with the object classes! And you can inherit from one
class to another, that was when it is appropriate to reuse it without even
rewriting it! Radical! Of course, if the record is really different you have
to write something new, but hey, you can use any of the other subroutines that
you've already written to make it really easy!
Of course, if there are bugs you still have to go find them.
Fortunately, since you are reusing all these wonderful already debugged
methods, the problem is always in the code you've just written, unless, of
course, you made a mistake in one of the old methods that you just never
exercised until just now. Also, if you design it wrong from the start, you may
not know it until you finish because you start at the bottom with simple
classes and build your way up (as opposed to top down) and you may have to
rebuild the whole thing anyway, but we'll fix that in our next paradigm shift,
eh? In short, my impression of the great progress in computer software in the
last decade is that we have discovered some important things:
1. There is a lot of money to be made in software. You make more money selling
Barbie's clothes than you make selling Barbie dolls.
2. Software is a fashion market. You must convince the user that if he doesn't
have fins on his machine he is just not where it is at. His old software just
isn't as cool as the new stuff.
3. Your look must be coordinated. That is, the jacket alone won't do it. You
have to have the Object-Oriented Compiler, and the OODebug, and the OOEditor,
and the OOFileSystem.
4. OO la la, look at the money just waiting to be made.
5. Mostly, we need lots of hype in the press, preferably of the type that
tells people the new stuff is magic -- that is, cures everything, is new,
revolutionary, very technical, and hard to understand. You'll have to change
your whole outlook cause this is one of those mind-bending paradigm shifts.
Marlin Eller
Seattle, Wash.


CRC Algorithms


Dear DDJ,
The basis of the algorithm is a simulation of the hardware tapped
shift-register used to generate CRCs. I show the registers shifting right
while your implementation shifts left. Your feedback constant 0x1021 is the
CCITT value 0x8408 bit-reversed. Your version of the implementation stores the
data byte into the lower 8 bits of the left-shifted CRC in order to affect the
data input which I show as a single bit input to the accum_crc function. My
CRC is 16-bits, while your implementation requires a 24-bit CRC variable, of
which the high-order 16-bits match my CRC variable.

I hope that this helps to clarify the CRC algorithm Al showed in his article.
Considering the amount of interest I found when I published, you will probably
get a number of letters from readers concerning the CRC function. There are
much faster CRC algorithms which use small, precomputed tables to do the CRC
calculation for an entire data byte at a time, eliminating the for loop
required in the bit-by-bit algorithms I have devised.
Meanwhile, keep the articles coming! You do the readership a great service by
publishing source code listings in the magazine. I've been a reader (and
occasional contributor) for over 12 years now.
Robert D. Grappel
Concord, Mass.


Mapping DOS Memory Revisited


Dear DDJ,
I am writing in response to a letter from Bruce Koivu that appeared in your
April issue. Bruce was concerned with converting Rob Moore's "Mapping MS-DOS
Memory Allocation" (November, 1988) program from Turbo C to Microsoft C. I
have a few suggestions that may help Bruce in this process.
1. Use pragmas to pack the MCB structure:
 #pragma pack(1) struct MCB { char chain; unsigned pid; unsigned
psize; char fill[11]; }; typedef struct MCB huge *PTRMCB; #pragma pack( )
The first pack pragma tells the compiler to perform byte alignment on any
following structures. The second pack pragma tells the compiler to default
back to word alignment.
2. Use the FP_SEG and FP_OFF macros found in <dos.h> instead of MK_FP: in
Turbo C:
 /* far pointer to segment address/
 /of first MCB * unsigned far *segmptr;/
 /* get pointer to segment address of first MCB */
 segmptr = MK_FP(sregs.es, regs.x.bx-2);
 /* get and return pointer to first MCB */
 return MK_FP(*segmptr, 0);
in Microsoft C:
/* far pointer to segment address of first MCB */
unsigned far * segmptr;
/* get pointer to segment address of first MCB */

FP_SEG(segmptr) = sregs.es;
FP_OFF(segmptr) = regs.x.bx-2;
/* get pointer to first MCB */
FP_SEG(segmptr) = *segmptr;
FP_OFF(segmptr) = 0;
/* return pointer */
return (segmptr);
This may seem strange at first (using a macro on the left side on an
assignment statement), but a look at the macros themselves reveals that this
is perfectly legal. Recall that a far pointer is stored in memory as an offset
followed by a segment. Both may be treated as unsigned (16-bits). Through a
series of casts to an unsigned pointer, these two macros access the segment
and offset portions of a far pointer.
3. If you are having problems with declaring MCBPTR to be huge, (my compiler
kept treating it as far), then you may need to change the pointer arithmetic
used in getting the next MCB pointer. In Turbo C:
/* huge pointer to an MCB */
PTRMCB ptrmcb;
/* get pointer to next MCB */
ptrmcb += ptrmcb->psize + 1;
in Microsoft C:
/* "huge"pointer to an MCB */
/* but, for some unknown reason at least to me) */
/* it is treated as far! */
PTRMCB ptrmcb;
/* temporary to hold segment address of MCB pointer */
unsigned mcbseg;
/* get pointer to next MCB */
mcbseg = FP_SEG(mcbptr) + mcbptr->size + 1;
FP_SEG(mcbptr) = mcbseg;
Remember, both huge and far pointers may access any location in memory.
However, when performing pointer arithmetic, such as the addition above, a far
pointer only uses 16-bit math while a huge pointer uses 32-bit math! Since far
and huge pointers are stored as offset followed by segment, one can see that
the segment of a far pointer cannot be changed via simple addition! Of course,
you may modify the segment directly as shown above.
4. Do not use the maximum optimization switch /Ox. Among other things, the /Ox
switch causes the compiler to disregard pointer aliasing in its optimizations.
As there is a lot of work with pointers in this program, this may cause the
program to behave unpredictably. On my computer, this results in an endless
loop, which continually prints out information on the first MCB only!
Steve Smith
Renton, Wash.


TUL Is Alive and Kicking



Dear DDJ,
Our experience is that the joint venture in India significantly strengthens
our global competitiveness. TUL has implemented scores of newly designed,
leading-edge applications, and not just conversions as Mr. Little seems to
imply. Increasingly, new software development projects are being undertaken
within India, thanks to improving communications and satellite links. This, in
my view, proves once again that clients will ultimately care more about
quality, timeliness, and cost, than where a project is performed.
Anil Shrikhande
Unisys Corporation
Blue Bell, Penn.

























































August, 1989
SMALLTALK + C: THE POWER OF TWO


Is Smalltalk + C > C++?




Dave Thomas and Randolph Best


Dave is the president of Computer Based Information Systems, Inc., based in
Ottawa, Ontario, Canada. He can be reached at 613-728-1558 or
dthomas@carleton.ca. Randolph is the executive vice president of engineering
and development for Digital Composition Systems, Inc. He can be reached at
1715 W Northern Ave., Phoenix, AZ 85021, 602-870-7667.


Sophisticated applications require the integration of many software components
and it has recently been argued that object-oriented technology is one of the
most effective ways to deal with this complexity. Although many programmers
advocate extending an existing language, such as C, into a hybrid language,
such as C++ or Objective C, these systems lack the flexibility, library,
environment, and memory management of pure object-oriented languages like
Smalltalk.
In this article, we describe a pragmatic approach which we call "Smalltalk +
C," that allows each language to be used where appropriate. Smalltalk, for
example, is used to represent and manipulate high-level information while C is
used to implement small, time/resource critical low-level facilities --an
approach we've found to be effective. First, we'll illustrate the power of
this unlikely marriage as implemented in dbPUBLISHER, a database publishing
program we developed using Digitalk's Smalltalk/V. Second, we'll examine an
embedded TCP/IP network application written in Smalltalk/V286.


The Strength of C


The major benefits and weaknesses of C are well known to most readers of DDJ.
C is a low-level language for systems programming such as device drivers. It
is portable, and in most cases, is the most efficient language available on a
given hardware platform. Unfortunately, when C is used in a large project, its
power leads to problems of reliability. (The market for hardware debuggers is
testimony to the problems of debugging large C applications.) We use C for the
necessary low-level routines for our applications. Because these routines
typically have a single function and can be written in a small number of
lines, they are readily understood and tested. Often these small modules can
be reused in several quite different applications.


The Power of Smalltalk


Smalltalk provides all of the facilities for programming-in-the-large that C
lacks, including a robust interactive environment for applications
development. Because everything in Smalltalk is an object, pointer errors like
those in C are eliminated. Smalltalk uses a sophisticated, high-speed, garbage
collector to manage memory so the programmer doesn't need to worry about the
complexities of storage management. Developers who work with Microsoft
Windows, Presentation Manager, and the Macintosh are only too familiar with
subtle storage management bugs, which are eliminated by using a garbage
collector.
The class structure of Smalltalk provides the organizational framework needed
for a large application. It allows the application programmer to concentrate
on the application classes and to ignore low-level details. The Smalltalk
language, however, doesn't allow the programmer to have direct access to the
underlying hardware. While Digitalk's implementation is extremely fast, there
are some things that require low-level programming, or at least access to a
program written in another language. There are two ways to do this in
Smalltalk/V, by "using the DOS shell" and through "user primitives." In
Smalltalk/V286, a more sophisticated "virtual machine" interrupt strategy is
available.


The dbPUBLISHER Challenge


Database publishing requires complete report generation, data conversion,
formatting, and tightly coupled page composition tools in one integrated
package, none of which is supported by traditional desktop publishing
software. Our primary goal in the development of dbPUBLISHER was to provide
these features in software that would run on 560K, 80286/386, MS-DOS/DOS
hardware. Our biggest challenge was to do so with limited development funds
and an outrageous 8-month delivery schedule.
Our existing library of C modules that perform page composition, image import,
font management, and report generation required a completely new user
interface. We needed a non-standard interface that used a small number of
screens for report generation, report layout, professional typesetting, and
markup tag insertion. Smalltalk/V proved to be the ideal tool for building the
user interface and the main program. It dispatches to external C and Pascal
programs for specialized services. Figure 1 illustrates dbPUBLISHER's
architecture.
Figure 1: dbPUBLISHER architecture

 Smalltalk/V

 dbPUBLISHER
 Application primitives

 Composition Output drivers Font management Report generation
 (Pascal) (C) (C) (C)

Our user interface supports a full windows-style look-and-feel. An outliner
style interface is used to define complex structured reports. This gives the
user full visibility of the entire report structure. The user can construct
complex nested SQL-style queries to compose a report directly from multiple
databases and/or spreadsheets. To accommodate the demands of complex
typography, dbPUBLISHER uses over 50 detailed dialog boxes. Page layout for
reports is performed using a WYSIWYG MacDraw-style application. Unlike most
drawing editors which use screen resolution, we use scaled pixels (1/4736287
inches). With this resolution dbPUBLISHER can be used for fine typography such
as forms and tables.


Invoking Another Program


The DOS shell in Smalltalk/V rapidly dumps the current memory image to disk
and runs an external program via COMMAND.COM. The resident stub is
approximately 40 Kbytes. Our enhancement reduces the stub size and allows
external programs outputting in text mode to pour properly into a Smalltalk/V
graphics window. These facilities were implemented using our own
Load-and-Execute primitives which also allow the current environment strings
to be read and written.
Primitives are simply Smalltalk methods written in another language. A user of
a primitive method doesn't see any difference in calling it versus calling a
normal Smalltalk method. The code in Listing One illustrates both the
definition and use of the primitive method doDCSPrimitive. In Smalltalk/V, you
are forced to use numbered primitives, but in Smalltalk/V286 and V/Mac, you
can have user primitives.
The Load-and-Execute primitive resides in low memory and swaps a block of high
memory to a temporary file, adjusts the DOS memory handles so that DOS thinks
there is enough memory available to run the application, and then performs a
normal DOS EXEC function call (int 21h, ax=4h00h). When control returns to the
primitive, the process is reversed; memory handles are reconstructed, the
block is read from the temporary file (which is then deleted), and control is
returned to Smalltalk. Unfortunately, this simple strategy does not work in
real-life on 80x86-based computers running MS-DOS. The primary reason this
approach fails is that the block of memory that is swapped out will quite
likely contain data and code required by both DOS and any asynchronous events
(notably, mouse events, control-break, and critical errors). In addition, the
parameters that are passed to the primitive by Smalltalk also reside in this
swappedout region.
In general, the solution is to move the offending objects to low memory where
they can be protected by the primitive, or (in the case of unwanted
asynchronous events such as the mouse event handler) be masked, disabled, or
replaced, as appropriate. Smalltalk parameters and the DTA pointer can be
moved into low memory, but asynchronous interrupt service routines (ISRs) are
not easily detached and moved about. Thus, the only solutions available are to
either disable them, or to replace the offending ISRs with dummy code that
prevents the system from crashing.
It is not acceptable, however, to arbitrarily re-vector the entire interrupt
vector table, nor is it acceptable to disable essential hardware interrupts
(that is, diskette, keyboard, real-time clock, and so on). Actually, the only
interrupts that need to be re-vectored are those that might get clobbered when
the application is executed. In other words, if an interrupt vector currently
points into the region of memory that we are about to make available, we
should first point that vector to a dummy ISR' then restore it when we are
through.

The source code in Listing Two illustrates the primitive implementation for an
interface to DOS, special purpose dbPUBLISHER primitives, and running external
C and Pascal programs.


A TCP/IP Ethernet Application


However unlikely it may seem, we used Smalltalk/V286 in a TCP/IP application.
It provided split execution of the graphical human interface on an IBM AT,
with a specialized Smalltalk/V286 embedded virtual machine running on a VME
bus Unix system. This application proved that the speed of the byte code
interpreter was up to the task.
This application required the processing of interrupts from an external
hardware and software source. Digitalk provided virtual machine interrupts for
this application. Listings Three and Four show source code that illustrates
the interface between a TCP/IP Ethernet resident driver (that runs in Real
mode) and Smalltalk/V286. (Listing Three shows a sample Smalltalk/V286 calling
method, while Listing Four provides the Smalltalk/V286 assembly language
primitive interface.) This interface allows a host PC to talk to an embedded
target real-time system.
One Smalltalk user primitive is provided: socketPrimitive dispatches the
appropriate code based on an opcode parameter. The arguments are passed in a
Smalltalk Array. The primitive runs in protected mode, sets up the parameter
block the installed driver expects, and switches to real mode (CALL_NETWORK
macro in sockprim.inc) using the INT 50H call. This call then runs the
doEthernetInt (in sockprim.asm) in real mode which does the INT 68H call to
the installed driver as would any other application. On return, the error code
is extracted from the parameter block and returned as an instance field of the
receiver (the object to which the message causing the primitive was sent).
This illustrates mode switching.
As part of initialization, the Smalltalk primitive calls the installed driver
(using the procedure described earlier) with the address of a small routine
(socket_event_handler). This address is in the segment of the primitive that
is to be called when a Smalltalk VM Interrupt should be generated. Any
hardware interrupts cause an automatic switch to real mode and a call to the
handler that was installed before Smalltalk was invoked. Thus, upon servicing
an interrupt from the Ethernet card, the installed driver can call
socket_event_handler in real mode to cause a Smalltalk VM Interrupt. Virtual
Machine Interrupts can be generated from either real or protected mode; both
are handled by different Digitalk macros.


Conclusion


These examples illustrate how two proven programming languages can be combined
into a single application. This approach enabled us to get a new product into
production quickly, in spite of a small engineering budget. We now have time
to do the OS/2 Presentation Manager version by re-directing Smalltalk/V286
primitives to the appropriate PM calls, and to improve performance with
multitasking techniques. We can port the application to the Macintosh II with
less trauma than typical Mac ports and with a guaranteed reliability factor.
Smalltalk's inherent object protection yielded a bullet-proof application
program for first release.
The success of Hypercard and Hypercard Xcommands (primitives) is another
convincing example of the benefits of small, single-function primitives
combined with a flexible user-programmable interpreter. The point is simple --
use the right language for the right job.


Availability


All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063, or call 800-356-2002 (from inside
Calif.) or 800-533-4372 (from outside Calif.). Please specify the issue number
and format (MS-DOS, Macintosh, Kaypro).


SMALLTALK + C: THE POWER OF TWO
by Dave Thomas and Randolph Best



[LISTING ONE]

Object subclass: #Dos
 instanceVariableNames:
 'registers temp1 temp2 '
 classVariableNames: ''
 poolDictionaries: '' !

!Dos methods !

doDCSPrimitive: opcode
 "PRIVATE - Call the DCS Primitives using an interrupt"
 <primitive: 96>
getEnvironmentValue: anEnvironmentString
 "Private- This is a method for getting the
 value of an environment variable(string). Answers
 the value if anEnvironmentString is valid or nil
 if not found.
 Here the instance variables are
 used as follows:
 temp1 = the name of the environment variable wanted
 temp2 = the value of the environment, if the
 environment variable exists. "
 temp1 := anEnvironmentString asAsciiZ.
 temp2 := String new: 128.
 (self doDcsPrimitive: -11)
 ifTrue:[ ^temp2 trimBlanks ]
 ifFalse:[ ^''].!







[LISTING TWO]

;*****************************************************************************
;ACCESS.USR - PRIMITIVE ENTRY MACRO - SMALLTALK/V
;Copyright (C) 1986 - Digitalk, Inc. - Reprinted by Permission
;*****************************************************************************
; It is essential that this macro appear at the beginning of
; the primitive in that it saves certain registers. Some of
; the registers may or may not have to be restored depending on
; whether or not the primitive is successful, and some
; must be restored before exiting the primitive.

enterPrimitive MACRO

; At the end of this macro the stack will appear as below:
; SP--> BP--> - saved BP - must be restored on exit
; +2 - saved BX - must be restored in case of failure
; +4 - saved DI - must be restored in case of failure
; +6 - saved SI - must be restored on exit
; +8 - saved DS - must be restored on exit
; +10 - IP
; +12 - CS
; +14 - FLAGS
; if there are any argument passed to the primitive they are found here
; +16 - last argument to primitive
; +18 - second last argument to primitive
; - etc...
; high address -
; Note: some of the following macros use BP assuming the value has not
; changed from what this macro sets it to. If you use BP be sure
; to restore it before using the macros that make use of BP.

 PUSH DS ;set up stack as shown above
 PUSH SI
 PUSH DI
 PUSH BX
 PUSH BP
 MOV BP,SP ;set BP to Top of Stack
 ENDM

; This macro must be used when the primitive must be exited and the
; primitive was SUCCESSFUL. The resulting pointer or small integer
; must be in BX before invoking this macro. The macro will:
; - mark the object (pointer) in BX so that the garbage collector
; will not collect it as garbage
; - certain registers are restored
; - the return address and flags are popped into temporary registers
; - the arguments are flushed and replaced with the result in BX
; - the return address and flags are put back on the stack
; - AX is set to zero (AH=0 indicates that the primitive was successful)
; - and the IRET instruction is executed

exitWithSuccess MACRO numOfArgs


; On entry BX must contains the result to be pushed
; on the stack.

 markPtr BX,ES ;mark result object in BX
 POP BP ;restore BP
 ADD SP,4
 POP SI ;restore DS:SI pair
 POP DS
 POP CX ;pop the offset,segment and
 POP DX ;flags into temp registers
 POP AX
IF numOfArgs
 ADD SP,numOfArgs * 2 + 2 ;flush all args of primitive
ENDIF
 PUSH BX ;push result on stack
 PUSH AX ;push flags, segment and
 PUSH DX ;offset back onto stack
 PUSH CX
 XOR AX,AX ;AH to 0 (prim was successful)
 IRET ;interrupt return
 ENDM

; This macro must be used when the primitive must be exited and the
; primitive FAILED. The macro will:
; - restore all saved registers
; - set AH to 1 indicating failure condition and AL to the number
; of arguments passed to the primitive
; - and the IRET instruction is executed

exitWithFailure MACRO numOfArgs

 POP BP ;restore all saved registers
 POP BX
 POP DI
 POP SI
 POP DS
 MOV AX,256 + numOfArgs ;AH to 1 (prim failed)
 IRET ;interrupt return
 ENDM

; This macro will return the address of the object with object pointer
; in objectReg. The address will be returned in the register pair
; segmentReg:offsetReg. segmentReg must be a segment register.

getObjectAddress MACRO objectReg,offsetReg,segmentReg

; All the arguments must be different registers.
; BP must not have changed from the value it was
; set to in the enterPrimitive macro.

 MOV segmentReg,[BP+12]
 ROR objectReg,1
 MOV offsetReg,segmentReg:[objectReg]
 ROL objectReg,1
 MOV segmentReg,SS:[objectReg]
 AND offsetReg,MASK offsetMask
 ENDM

; This macro will mark the object whose object pointer is in objectReg

; so the garbage collector will not collect as garbage. It is to be used
; whenever an object pointer is stored. If the object in objectReg is
; a small integer, no marking occurs.

markPtr MACRO objectReg,segmentReg
 LOCAL done

; BP must not have changed from the value it was
; set to in the enterPrimitive macro.

 ROR objectReg,1
 JNC done
 MOV segmentReg,[BP+12]
 OR BYTE PTR segmentReg:[objectReg],MASK grayMask
done: ROL objectReg,1

 ENDM

; This macro will get the class (pointer) of the object whose pointer
; is in objectReg. If the object in objectReg is a small integer then
; the class pointer is set to ClassSmallInt.

getClass MACRO objectReg,classReg,segmentReg
 LOCAL done
;all the arguments must be different registers

 MOV classReg,ClassSmallInt
 TEST objectReg,1
 JZ done
 MOV classReg,SS
 SHR classReg,1
 MOV segmentReg,classReg
 MOV classReg,segmentReg:[objectReg]
done:
 ENDM

; This macro will set the zero condition flag as to whether the object size
; is an even or odd number of bytes. This test should be performed on
; byte-addressable objects. A zero condition means that the object is an
; even number of bytes (actual size = object header - 2). A non-zero
; condition means that the object is an odd number of bytes
; (actual size = object header - 3).

isSizeEven MACRO objectReg,workReg,segmentReg

; BP must not have changed from the value it was
; set to in the enterPrimitive macro.

 MOV workReg,objectReg
 ROR objectReg,1
 MOV segmentReg,[BP+12]
 TEST BYTE PTR segmentReg:[objectReg],MASK oddMask
 MOV objectReg,workReg
 ENDM

;*****************************************************************************
; PRIMITIVE ENTRY POINT SAMPLE CODE FRAGMENTS
;*****************************************************************************


 DW getCountryEnt
 DW getEnvironEnt
 DW exeApplicEnt
 DW exeProgramEnt
 DW setNoXlatEnt
 DW setXlatEnt
 DW atEndEnt
 DW bufFlushEnt
 DW bufWriteEnt
 DW bufReadEnt
 DW initOutputEnt
 DW initInputEnt
jumpTable DW interruptEnt
 DW inWordEnt
 DW inByteEnt
 DW outWordEnt
 DW outByteEnt
 DW peekEnt
 DW pokeEnt
 DW blockMoveEnt

 *
 *
 *

dcsPrims proc far

; This is code that will be executed everytime this primitive
; is invoked from "Smalltalk/V"

 enterPrimitive ;enter primitive macro

 mov bx,[bp+16] ;get function code from first instance
 test bl,1 ; variable of receiving object. if it
 jnz failure ; isn't a number then something's very
 ; rotten in the state of Denmark.

 cmp bx,16 ;the function code is already shifted
 jge failure ; left by one (courtesy of smalltalk's
 cmp bx,-26 ; way of identifying integers), so do
 jle failure ; some range checks and abort if the
 ; value is out of bounds.

 jmp cs:[bx+jumpTable] ;otherwise, jump to the code associated
 ; with this function number.
 *
 *
 *

 mov cs:stackOfs,sp ;save the current Smalltalk stack, and
 mov cs:stackSeg,ss ; replace it with a local stack in low
 mov ax,cs ; (protected memory).
 cli
 mov ss,ax
 lea sp,cs:_stack
 sti

 mov ah,2fh ;save the DOS DTA pointer, just in case
 int 21h ; it gets clobbered by the application.

 mov cs:dtaOfs,bx
 mov cs:dtaSeg,es

 call exec ;load and execute the program
 mov bx,truePtr ; the errorlevel
 jnc exec01 ; code is returned in "retCode". If the
 mov bx,falsePtr ; carry was set, then an error occurred
exec01: mov retVal,bx ; and we answer "false", otherwise we
 ; answer "true".
 mov ah,1ah ;restore the DOS DTA pointer.
 lds dx,cs:dtaPtr
 int 21h

 cli ;and, restore the Smalltalk stack with
 mov ss,cs:stackseg ; the saved pointer.
 mov sp,cs:stackofs
 sti

 mov ax,retCode ;convert the errorlevel to a Smalltalk
 shl ax,1 ; integer format, and return it in the
 les bx,receiverPtr ; fifth instance variable of receiver
 mov es:[bx+12],ax ; object.

 mov bx,retVal ;the Smalltalk convention is to put the
 jmp success1 ; answer for this obj in bx, and call
 ; the "leavePrimitive" macro. a jump
 ; to success will do this quite nicely
 ; thank yew!






[LISTING THREE]

;queue an interpreter interrupt (in protected mode)
; AL=interrupt number to queue
interruptVM MACRO
 CALL DWORD PTR SS:[queueVMinterrupt]
 ENDM
;queue an interpreter interrupt (in real mode)
ISVinterruptVM MACRO
 MOV ES,CS:[realParmSeg]
 CALL DWORD PTR ES:[ISVqueueVMinterrupt]
 ENDM
The code to call the socket primitive in Smalltalk is:
socketPrimitiveOpcode: opcode withArguments: argumentArray
 "PRIVATE: Call the socket primitive."
 &lt;primitive: socketPrimitive&gt;
 ^self error: 'Network Primitive failed - is sockprim.bin loaded?'






[LISTING FOUR]


;*****************************************************************************
;* FIXDPTRS.USR
;*****************************************************************************

;fixed segments

plusSmallSeg = 6 ;segment of small positive integers
nilSegment = 106H ;segment of nil object
minusSmallSeg = 116H ;segment of small negative integers
booleanSeg = 10EH ;segment for true and false
characterSeg = 11EH ;segment for all character objects
fixedPtrSeg = 126H ;segment for all fixed ptr objects

;fixed offsets

nilOffset = 106H ;offset of nil object
trueOffset = 0fff3H ;offset of true object
falseOffset = 0fff1H ;offset of false object
firstCharOffset = 2 ;offset of ascii char 0

;all of the following objects are in the segment fixedPtrSeg
;what is given below are their offsets
 ;array of classes in system
classArrayOffset equ nilOffset+size objectHeader
Smalltalk equ classArrayOffset + size assoc
ErrorCode equ Smalltalk + size assoc

;*****************************************************************************
; OBJECTS.USR
;*****************************************************************************

;Object header structure

objectHeader STRUC
ClassPtrHash DW ? ;see below for values for fixed classes
ObjectPtrHash DW ? ;usually contains object hash
GCreserved DW ?
NumberFixed DW ? ;number of named instance variables
ObjectSize DB 3 DUP(?) ;stored as low,middle,high order
 ; size is stored as # of instance variables
ObjectFlags DB ? ;defined below
objectHeader ENDS

;object flags (contained in objectFlag byte of objectHeader)
PointerBit EQU 10H ;Object contains pointers
IndexedBit EQU 8 ;Object has indexed instance variables
;other bits in byte are reserved

;Array Object
arrayObj STRUC
 DB size objectHeader DUP (?)
arrayObj ENDS

;Character Object
charObj STRUC
 DB size objectHeader DUP (?)
asciiValue DD ? ;ascii value
charObj ENDS
;Note that this is 16 bytes in size


; Association Object
assoc STRUC
 DB size objectHeader DUP (?)
assocKey DD ?
assocValue DD ?
assoc ENDS

; Point object
pointObj STRUC
 DB size objectHeader DUP (?)
pointX DD ?
pointY DD ?
pointObj ENDS

; Hash values for classes
SmallIntegerHash equ 0
emptySlotHash equ SmallIntegerHash + 8
StringHash equ emptySlotHash + 8
MessageHash equ StringHash + 8
SymbolClassHash equ MessageHash + 8
LargePosIntHash equ SymbolClassHash + 8
HomeContextHash equ LargePosIntHash + 8
LargeNegIntHash equ HomeContextHash + 8
ContextHash equ LargeNegIntHash + 8
PointHash equ ContextHash + 8
ArrayHash equ PointHash + 8
LinkHash equ ArrayHash + 8

; This is a useful struc for accessing arguments in primitives
; For example, to load the receiver into DS:SI
; LDS SI,[BP+receiverPtr]

;stack after enterPrimitive macro
primitiveFrame STRUC
savedBP DW ?
returnAddr DD ?
receiverPtr DD ?
arg1Ptr DD ?
arg2Ptr DD ?
arg3Ptr DD ?
primitiveFrame ENDS

;This struc defines the beginning of a user primitive load module
primLoadModule STRUC
installEntry DW ? ; 0 entry point for installation routine
reserved1 DW 0 ; 2
 DW 0 ; 4
realCodeSeg DW ? ; 6 after loading, will contain real mode addr
primTableOffset DW ? ; 8 offset of table of primitive subroutines
realParmSeg DW ? ; A after loading, will contain real mode addr
 ; of virtual machine communication area.
reserved2 DW 0 ; C
 DW 0 ; E
primLoadModule ENDS

;*****************************************************************************
;* ACCESS.USR -PRIMITIVE ENTRY MACRO - SMALLTALK/V286
;* Copyright (C) 1988 - Digitalk, Inc. - Reprinted by Permission

;*****************************************************************************

; It is essential that this macro appear at the beginning of
; the primitive in that it saves certain registers. Some of
; the registers may or may not have to be restored depending on
; whether or not the primitive is successful, and some
; must be restored before exiting the primitive.

enterPrimitive MACRO

; At the end of this macro the stack will appear as below:
;
; SP--> BP--> - saved BP
; +2 - return addr (offset)
; +4 - return addr (segment)
; If there are any argument passed to the primitive they are found here.
; All arguments and the receiver are passed as 32 bit pointers
; +6 - receiver of primitive (offset)
; +8 - receiver of primitive (segment)
; +10 - first argument to primitive (offset)
; +12 - first argument to primitive (segment)
; - :
; high address - :
;
; Note: some of the following macros use BP assuming the value has not
; changed from what this macro sets it to. If you use BP be sure
; to restore it before using the macros that make use of BP.

 PUSH BP ;save old BP
 MOV BP,SP ;set BP to Top of Stack
 ENDM

; This macro must be used when the primitive must be exited and the
; primitive was SUCCESSFUL. The resulting pointer or small integer
; must be in DX,AX (DX=segment, AX=offset) before invoking this macro.

exitWithSuccess MACRO

; On entry DX,AX must contains the result to be pushed
; on the stack.
 MOV SP,BP
 POP BP ;restore BP
 RETF ;far return
 ENDM

; This macro must be used when the primitive must be exited and the
; primitive FAILED.

exitWithFailure MACRO

 XOR AX,AX ;AX=DX=0, for failure return
 XOR DX,DX
 MOV SP,BP
 POP BP ;restore BP
 RETF ;far return
 ENDM

;Object testing macros
;

;in the following testing macros,
;the result is returned in the zero flag as follows: z=no, nz=yes

;Object has pointers?? jz=no, jnz=yes
isPointerObject MACRO objSeg,objOff
 TEST objSeg:[objOff+objectFlags],PointerBit
 ENDM

;Object is indexable?? jz=no, jnz=yes
isIndexedObject MACRO objSeg,objOff
 TEST objSeg:[objOff+objectFlags],IndexedBit
 ENDM

;Object is contained in single segment?? jz=no, jnz=yes
isSmallObject MACRO objSeg,objOff
 OR objOff,objOff
 ENDM

;**** size extraction macros ****

;Object size is expressable in elements or bytes.
; elements is the number of smalltalk objects it contains
; bytes is the number of bytes it occupies (including header)
; note that objects always occupy an even number of bytes
;For example:
; #( 1 2 3 ) is an array with three elements and it occupies 24 bytes
; 'hello' is a string with 5 elements and it occupies 18 bytes

;extract the size in elements
getElementSize MACRO objSeg,objOff,resultLowWord,resultHighByte
 MOV resultHighByte,byte ptr objSeg:[objOff+objectSize+2]
 MOV resultLowWord,word ptr objSeg:[objOff+objectSize]
 ENDM

;compute the size in bytes
getBigByteSize MACRO objSeg,objOff,resultLowWord,resultHighByte
 LOCAL addHeader
 getElementSize objSeg,objOff,resultLowWord,resultHighByte
 isPointerObject objSeg,objOff
 JZ addHeader
 ADD resultLowWord,resultLowWord
 ADC resultHighByte,resultHighByte
 ADD resultLowWord,resultLowWord
 ADC resultHighByte,resultHighByte
addHeader: ADD resultLowWord,size objectHeader+1
 ADC resultHighByte,0
 AND resultLowWord,0FFFEH
 ENDM

;user calls to interpreter routines

;routine vector offsets
ISVqueueVMinterrupt equ 0FFF0H - 4
queueVMInterrupt equ ISVqueueVMinterrupt-4
oldToNewStore equ queueVMinterrupt-4 ;used in oldToNewUpdate macro

;queue an interpreter interrupt (in protected mode)
; AL=interrupt number to queue
interruptVM MACRO

 CALL DWORD PTR SS:[queueVMinterrupt]
 ENDM

;queue an interpreter interrupt (in real mode)
ISVinterruptVM MACRO
 MOV ES,CS:[realParmSeg]
 CALL DWORD PTR ES:[ISVqueueVMinterrupt]
 ENDM

;miscellaneous but usefull macros

;is object a small positive integer -- je=yes, jne=no
;(only segment needs to be tested)
isSmallPosInt MACRO segmentExpression
 CMP segmentExpression,plusSmallSeg
 ENDM

;is object a small negative integer -- je=yes, jne=no
;(only segment needs to be tested)
isSmallNegInt MACRO segmentExpression
 CMP segmentExpression,minusSmallSeg
 ENDM

;is object a character -- je=yes, jne=no
;(only segment needs to be tested)
isCharacter MACRO segmentExpression
 CMP segmentExpression,characterSeg
 ENDM

;is object static, i.e. constant, no stores allowed -- ja=no, jbe=yes
;(only segment needs to be tested)
isStaticObject MACRO segmentExpression
 CMP segmentExpression,characterSeg
 ENDM

;****** This macro must be called after EVERY pointer store *******
;****** Failure to do so will invalidate the garbage collector ****
;****** leading to catastrophic and unpredicable results **********

;This macro detects old space to new space pointer stores,
;and updates the GC data base accordingly.
;
;macro arguments are as follows:
; segReg = seg reg of object stored into
; offReg = offset reg of object stored into
; valueSeg = segment of pointer that was stored
; workReg = a work register
;
;example of use: store ptr BX:AX into object ES:DI at slot 'contents'
; MOV word ptr es:[di+contents],AX ;store offset
; MOV word ptr es:[di+contents+2],BX ;store segment
; OldToNewUpdate es,di,bx,ax
OldToNewUpdate macro segReg,offReg,valueSeg,workReg
 LOCAL done
 MOV workReg,segReg
 CMP workReg,92EH
 JAE done
 CMP valueSeg,92EH
 JB done

 PUSH segReg
 PUSH offReg
 CALL DWORD PTR SS:[oldToNewStore]
 POP offReg
 POP segReg
done:
 ENDM

;****************************************************************************
; SOCKPRIM.ASM
; The V286 Socket Primitives V2.0
; In this implementation, recv() and accept() only operate in
; a nonblocking fashion, returning errno EWOULDBLOCK if the
; operation can not be completed immediately. All other calls
; block until completion or error.
; Opcode 0 is implemented in this version as Socket closeAll.
; Opcode 18 is implemented in this version as Socket version.
; An additional non-standard errno EDRIVER (254) is returned if
; installation fails or the installed driver behaves strangely.
;****************************************************************************
;
TITLE Socket Primitives
.286P
 INCLUDE fixdptrs.usr
 INCLUDE objects.usr
 INCLUDE access.usr

 INCLUDE sockprim.inc

code SEGMENT PUBLIC 'CODE'
 ASSUME CS:code,DS:nothing,ES:nothing

PGROUP GROUP CODE

;****************************************************************************
; Smalltalk/V286 Reserved Area
;****************************************************************************
;
 ORG 0
 DW OFFSET install ;installation routine entry point

 DW 0 ;reserved for future use
 DW 0

 DW 0 ;real mode segment of this code

 DW OFFSET primTable ;addr of table of primitives and entry points
 DW 0 ;real mode segment address of protected mode
 ;parameter area

 DW 0 ;cell used for real mode calls to VM
 DW 0

;****************************************************************************
; Local Data
;****************************************************************************
;
initialized_flag DW 0
Socket_PB DB 32 DUP (0)

Name_Buffer DB MAX_NAME_SIZE DUP (0)
Data_Buffer DB MAX_BUFFER_SIZE DUP (0)

;****************************************************************************
; The socketPrimitive operation dispatcher
;****************************************************************************
;
socketPrimitive PROC FAR
;
; Dispatch the socket primitive specified by opcode.
; PARAM(1) is the opcode
; PARAM(2) is the argumentArray
;
 enterPrimitive
 isSmallPosInt &lt;WORD PTR [BP+arg1ptr+2]&gt; ;opcode SmallInteger?
 JE ok_opcode
 FAIL ;FAIL if not
ok_opcode:
 MOV AX,initialized_flag ;perform initialization
 OR AX,AX
 JNE initialized
 PUSH BP
 CALL socket_install
 POP BP
 CMP AX,0
 JE initialized
 SUCCEED_ERROR
initialized:
 MOV AX, WORD PTR [BP+arg1ptr] ;opcode
;
; check opcode bounds
;
 CMP AX,0
 JL exit_FAIL
 CMP AX,MAX_OPCODE
 JG exit_FAIL
 SHL AX,1
 MOV SI,AX
;
; dispatch operation
;
 MOV AX,WORD PTR DS:[Socket_Primitives+SI]
 JMP AX
 SUCCEED_POSITIVE_INTEGER

exit_FAIL:
 FAIL

socketPrimitive ENDP

;****************************************************************************
; The socket event handler - called by resident driver in real mode
;****************************************************************************
;
socket_event_handler PROC FAR
 MOV AX,Network_VMInterrupt
 ISVinterruptVM
 RET
socket_event_handler ENDP


;****************************************************************************
; General purpose success exit
; AX = integer value to be returned (positive or negative)
;****************************************************************************
;
;SUCCEED_INTEGER PROC NEAR
; CMP AX,0
; JL SUCCEED_NEGATIVE
; SUCCEED_POSITIVE_INTEGER
;SUCCEED_NEGATIVE:
; SUCCEED_NEGATIVE_INTEGER
;SUCCEED_INTEGER ENDP

;****************************************************************************
; Install Socket Event Handler - called automatically by dispatch
;****************************************************************************
;
socket_install PROC NEAR
 MOV AX,1
 MOV SI,OFFSET initialized_flag
 MOV DS:[SI],AX
;
 SET_PB_FOR OPCODE_register_event_handler
 MOV WORD PTR DS:PB_Event_Handler[BX],OFFSET socket_event_handler
 MOV AX,DS:[realCodeSeg]
 MOV WORD PTR DS:PB_Event_Handler+2[BX],AX
;
 MOV BYTE PTR DS:PB_errno[BX],EDRIVER
 MOV WORD PTR DS:PB_Return_Code[BX],-1
;
 CALL_NETWORK
 RET
socket_install ENDP

;****************************************************************************
; The socket operations
;****************************************************************************

op_unimplemented PROC NEAR
 MOV AX,EINVAL
 SET_errno
 SUCCEED_ERROR
op_unimplemented ENDP

op_closeAll PROC NEAR
 SET_PB_FOR OPCODE_socket_close_all
;
 CALL_NETWORK
 SUCCEED_INTEGER
op_closeAll ENDP

;****************************************************************************
; deinstall() - perform required cleanup before Smalltalk/V exit
;****************************************************************************
;
op_deinstall PROC NEAR
 MOV AX,0
 MOV SI,OFFSET initialized_flag

 MOV DS:[SI],AX
;
 SET_PB_FOR OPCODE_register_event_handler
 MOV WORD PTR DS:PB_Event_Handler[BX],0
 MOV WORD PTR DS:PB_Event_Handler+2[BX],0
;
 CALL_NETWORK
 SUCCEED_INTEGER
op_deinstall ENDP

;****************************************************************************
; socket()
; ARG 1 Address_Format
; ARG 2 Type
; ARG 3 Protocol
;****************************************************************************
;
op_socket PROC NEAR
 SET_PB_FOR OPCODE_socket
;
 GET_POSITIVE_INTEGER_ARG 1
 MOV DS:PB_Address_Format[BX],AX
;
 GET_POSITIVE_INTEGER_ARG 2
 MOV DS:PB_Type[BX],AX
;
 GET_POSITIVE_INTEGER_ARG 3
 MOV DS:PB_Protocol[BX],AX
;
 CALL_NETWORK ;sets errno, result in AX
 SUCCEED_INTEGER
op_socket ENDP

 *
 *
 *

;=================================================================
; doEthernetInt
; This procedure executes in REAL MODE. The parameter block has
; been filled. setup es:bx to point to the parameter block and
; call the ethernet driver.

doEthernetInt PROC FAR
 PUSH AX ; Save registers
 PUSH BX ;
 PUSH ES ;
 MOV AX,CS ;
 MOV ES,AX ; es points to this segment
 MOV BX, OFFSET Socket_PB ; bx contains offset to pblock
 INT Resident_Driver_Interrupt ; call driver
 POP ES ; restore registers
 POP BX ;
 POP AX ;
 RET
doEthernetInt ENDP

;table of primitive names and entry points
primTable:

 DB 'socketPrimitive' ;Smalltalk name of primitive
 DB 0
 DW offset socketPrimitive ;offset of entry point
;
; more entries can go here

 DW 0 ;end of table

;installation routine, called at the time the module is loaded
install PROC FAR
 ret ;we have nothing to do, so return
install endp

code ENDS
 END















































August, 1989
MAKING THE C-TO-FORTRAN CONNECTION


No matter which camp you're from, the C-to-Fortran connection can add
functionality to your programs




Michael A. Floyd


Mike is a technical editor for DDJ and can be contacted at 501 Galveston
Drive, Redwood City, CA 94063. On CompuServe 76703,4057, or on MCI Mail as
MFLOYD.


If you've ever had to make a connecting flight to get to your final
destination, you know it takes more than just getting off one plane and onto
another. Among other things, someone has to check that your baggage follows,
your arriving flight has to be on time, and you have to get to the right gate
at the right time.
Making the C-to-Fortran connection is much the same. As a programmer, you must
ensure that arguments get from one language to the next, and that any excess
baggage is handled. Ultimately, you gain the benefits of both languages. With
huge Fortran libraries such as IMSL and the wealth of available third-party C
tools, there's no telling how far you'll be able to take it.
This article shows you how to call a Lahey FORTRAN subprogram from a C main
program, and how to call a C function from a Fortran program. For the purposes
of this article, I'm using Borland's Turbo C 1.5 with Lahey's F77L. This
connection works equally well with other C compilers, including Microsoft C.


Calling Fortran from C


The procedure for calling Fortran subprograms varies slightly between the
different versions of C. In particular, Microsoft C main programs cannot call
F77L subroutines and functions directly. Instead, you must create a main
program in F77L that immediately calls the MSC main( ) as if it was a
subprogram. Once loaded, main( ) can call the appropriate Fortran subprograms.
Turbo C and Lattice C, on the other hand, can call F77L subprograms directly.
Because F77L references data by address, all arguments passed by the C program
must be pointers. In the case of numerical constants, for instance, it is
necessary to assign the constant's value to a C variable and reference the
variable's address. Table 1 provides a list of C data types that may be passed
to F77L along with their corresponding Fortran data types.
Table 1: Data types that can be passed from C functions to Lahey FORTRAN
subprograms

 C Type Fortran Type
-----------------------------------------------------------------
 int * INTEGER*2
 long int * INTEGER*4
 float * REAL*4
 double * REAL*8 (double precision)
 float[2] (real portion is stored in the COMPLEX
 first array element)

 Struct Complex * COMPLEX

 double[2] (real portion is stored in COMPLEX*16
 the first array element)

 Struct DoubleComplex* COMPLEX*16
 char * LOGICAL*1

 struct CHARACTER { CHARACTER*(*)
 char *text;
 int length;
 }

In addition, Fortran handles strings differently than C does. In F77L,
CHARACTER*(*) refers to a descriptor structure that contains the address (4
bytes) and size (2 bytes) of the string. The string size, however, may or may
not be specified at compile time. Therefore, the CHARACTER*(*) declaration
must indicate that a variable-length string is being passed, the length of
which is determined by the caller. Example 1 shows how to pass a string, as
well as an integer and a real, from C to F77L.
Example 1: Passing a string from C to F77L

 /* Shows how to pass integer, float, and string parameters
 from C to F77L. */

 typedef
 struct {
 char *text;

 int length;
 } Str;

 extern void fortran_sub (int *, float *, Str *);

 main()
 {
 int number;
 float realNumber;
 Str charStr;

 number = 20;
 realNumber = 23.9;
 charStr.text = "Passing a text string";

 fortran_sub (&number, &realNumber, &charStr);
 }

 c
 c Fortran subroutine to take three arguments (integer, real,
 c and character string) passed from C, and print the results.
 c
 SUBROUTINE FORTRAN_SUB (IntVal, RVal, CharStr)
 BCEXTERNAL FORTRAN_SUB
 INTEGER*2 IntVal
 REAL RVal
 CHARACTER*(*) CharStr

 PRINT *, Intval, RVal, CharStr
 END

On the Fortran side of Example 1, notice the BCEXTERNAL statement immediately
following the subroutine header. Lahey provides three external statements,
BCEXTERNAL, MSCEXTERNAL, and LCEXTERNAL, to resolve C's external references to
names at link time. Your brand of C determines which external statement you'll
use. Example 1 assumes that Turbo C is being used. Replace the BCEXTERNAL
statement with MSCEXTERNAL if you're using Microsoft C, or LCEXTERNAL if
you're using Lattice C.
One of the real strengths of the Fortran Language is its ability to handle
complex numbers. It's an ideal time to think of calling an F77L subprogram
from C. The C-to-F77L interface supports two complex data types; COMPLEX and
COMPLEX*16. The corresponding C structure for a single precision complex
number is as follows:
 struct Complex {
 float real;
 float imaginary;
}
Double precision complex numbers are handled using the following C structure:
 struct DoubleComplex {
 double real;
 double imaginary;
}
Alternatively, you can reference complex numbers using a two-element array
where the first element holds the real portion and the second element contains
the imaginary portion of the complex number.
You can also access the return value of a Fortran function from your C
programs. In C, you do this by passing the address of where the function value
is to be stored as the first parameter in the call. Example 2 demonstrates how
to access the return value of an F77L function.
Example 2: Accessing the return value of an F77L function

 /* Shows how to access the return value of a Fortran function. */

 extern void cube(int *, int *);

 main()
 {

 int InVal, ReturnVal;
 InVal = 2;

 cube (&ReturnVal, &InVal);
 }


 c
 c Fortran function to calculate the cube of a given number.
 c
 FUNCTION CUBE (X)
 BCEXTERNAL CUBE

 INTEGER*2 X, CUBE
 CUBE = X*X*X
 END



Going the Other Way


As you might have guessed, it's also possible to call C routines from F77L.
There are, however, a few considerations in designing the Fortran program.
First, an external statement must declare the names of all C functions used in
the module. This guarantees that F77L will use the C calling conventions. In
Addition, the external statement tells the compiler to prefix each function
name with an underbar to correctly resolve externals at link time. Therefore,
be sure to leave the "Generate Underbars" switch on (the default for both MSC
and TC) when compiling on the C side.
Passing arguments from F77L to a C function can be a little tricky, so watch
out! The first potential problem is that C does not check to see if the
correct number of arguments have been passed to it. An illegal number of
parameters can have unpredictable results, and it is the programmer's
responsibility to check this.
Also, C gives you the option of passing by value or by reference. Because you
are working with a copy, passing by value in a sense puts up a "fire wall"
that prevents you from incorrectly altering the actual value of an argument.
You lose this benefit in the Fortran-to-C connection because F77L requires
that all arguments be passed by reference. Lahey has therefore included the
CARG function to allow the Fortran program to pass most arguments by value.
Table 2 provides a list of Fortran data types that can be passed to C.
Table 2: Data types that can be passed from Lahey FORTRAN subprograms to C
functions

 Fortran Type C Type
---------------------------------------------------------------------------
 INTEGER*2 int *
 CARG(INTEGER*2) int
 INTEGER*4 long int *
 CARG(INTEGER*4) long int
 REAL*4 float *
 CARG(REAL*4) float (double pushed on stack)
 REAL*8(double precision) double *
 CARG(REAL*8) double
 COMPLEX struct complex *

 COMPLEX float[2](real portion stored in first
 array element)

 COMPLEX*16 struct

 COMPLEX*16 double[2]

 LOGICAL*1 char * (Boolean value)
 LOGICAL*4 --
 CHARACTER char[]
 CARG(CHARACTER) char[] (null-terminated)
 label --
 EXTERNAL --

You should be aware of a couple of restrictions. Most importantly, CARG cannot
be used with C function return values. Also, CARG cannot be used with
LOGICAL*1 nor COMPLEX data types.
You'll find CARG useful in one particular case. As mentioned earlier, F77L
references a string through a descriptor structure, while C expects a pointer
to a null-terminated sequence of characters. But F77L does not include a null
byte at the end of a string. You can, however, use CARG to create a
null-terminated copy of the string and pass it by value.
One final consideration involves the use of the prototypes in C. Because F77L
defines both single- and double-precision arguments as DOUBLE, floats must be
handled differently. For example, consider the following prototype:
 void func (double x)
In order to handle a float instead of a double, the definition for func must
be changed to:
 void func(x)
 float x;
It is important to note that the following definition is not allowed in the
Fortran-to-C connection:
 void func(float x)


Arrays



The difference in the way the two languages handle arrays can be the cause for
some confusion. In particular, C and Fortran treat array subscripting and
sequential order differently. C begins its subscripting at element 0 while
Fortran normally begins subscripting at element 1. F77L, however, allows you
to create arrays that are 0 relative. Therefore, you must remember to declare
all appropriate arrays to begin at element 0.
In addition, arrays in F77L behave much like the pointers in a parameter list.
For example, consider the following code fragment:
 BCEXTERNAL
 INTEGER*2 I(0:9)

 ...
 CALL C_FUNCTION(I, I(3))

In C, the corresponding definition is as follows:
 void c_function(i, i3)
 int i[10], *i3; ...

C and Fortran also handle multi-dimensional arrays differently. Fortran stores
arrays in column-major format while C stores arrays in row-major format. The
solution is to simply reverse the subscripts in one of the two languages. For
example, consider the following two-dimensional array declared in Fortran:
 integer*2 A(3:0, 4:0)
The array is sequentially referenced as A(0, 0), A(1, 0), A(2, 0), A(0, 1),
A(1, 1), A(2, 1), and so on. To ensure compatibility, the corresponding C
definition must be as follows:
 int A[4][3];
Obviously, the language you choose to reverse subscripts in is not important.
To avoid confusion, however, you may wish to choose one language to
consistently switch subscripts throughout all of your programs. That way you
will never doubt whether your arrays are row-major or column-major.


File I/0


You must also be aware of differences in the way files are handled between the
two languages. F77L, in particular, adds header and indexing information to a
file. Normally this information is transparent to the programmer. C, on the
other hand, adds no such header nor indexing information. In addition, F77L
uses unit numbers to access files, while C uses file handles and stream
pointers. The heuristic, then, is to open, read, write, and close a file all
from within the same language. The exception, of course, is the standard
device I/O performed on unformatted files. In this case, it is important to
remember that a given file cannot be opened simultaneously by both languages.


Who's the Boss?


When it's time to compile and link your modules together, the first question
you should ask yourself is "who's the boss?"
The language in which the main program is written is the boss. The boss
controls the environment that your program will run in. Environmental
considerations include how memory is managed and language calling conventions.
When the main program is written in C, all of the F77L runtime routines are
available except SYSTEM and CHAIN. Of course, all of the C runtime routines
are available. When the main is written in F77L, all of the Fortran run-time
routines are available. On the C side, functions not requiring the C
environment are accessible. Some memory management functions such as malloc,
calloc, and free are supported. Many others, including farmalloc and sbrc, are
not.
Now it's time to bring the components of your program together into an
executable file. F77L uses the large memory model, so remember to compile your
C modules under the large model as well. This also means that far pointers are
always used. And be sure that stack checking is turned off when compiling on
the C modules. On the Fortran side, use the /NI switch to turn off interface
checking. And because floating-point calculations are handled by C, the NDP
compiler option is unnecessary. If you need floating-point emulation, select
the /E option.
For the link step, Lahey provides a set of interface routines in two object
modules that coordinate the different language environments. BCF77L is used
when C is the boss, and F77LBC is used when Fortran is the boss. In addition,
you'll have to include C's initialization module, COL, when C is the boss. No
matter who the boss is, you'll always include both languages' run-time
libraries. The connection relies on C's libraries for floating-point
calculations, so you'll either use C's floating point or emulation library.
I've included the link command line for the C to F77L connection at the top of
Listing One and the F77L to C connection at the top of Listing Three.


Graphically Speaking


DOT.C (Listing One) is a Turbo C program that uses the Borland Graphics
Interface (BGI) routines to randomly plot pixels on the screen. The C module
calls FRAND.FOR (Listing Two) to generate the random locations.
The BGI initialization routine Initialize( ) was taken directly from Borland's
BGIDEMO program, and demonstrates an easy method for supporting multiple
hardware configurations.
Specifically, RANDDOT performs hardware detection and supports Hercules
graphics, CGA, EGA, and VGA hardware. The interface demonstrates how both a
Fortran subroutine and a function can be called from C.
After initializing the system to graphics mode, RandomDot( ) is called and the
viewport settings [established by Initialize( )] are retrieved. Next, the
Fortran subroutine SEED_RAND is called to get the initial seed value that will
be used by F77L's random number generator. SEED_RAND calls RRAND, which
references the system clock to generate the initial pseudorandom value. RRAND
also generates and stores the seed value, which the random number generator
function RND will later use. The initial value generated by RRAND is returned
to the caller in RandomDot( ).
Next, a for loop is used to generate and plot 500 randomly placed (and
colored) dots on the screen. The random values are gotten from the Fortran
function FRAND which, in turn, calls RND. RND retrieves the seed value
generated by RRAND. This process of seeding RND is completely transparent to
the programmer, and is one of the side benefits of using the Fortran RND
function as opposed to C's random( ) function. It is also worth noting that,
from the C side, it is difficult to distinguish a subroutine from a function
because the Fortran function must return its value as the first argument in
the parameter list.


Sorted Partners


As a final example, consider SORT.FOR (see Listing Three). SORT uses RND to
generate an array of random integer values that are then passed to C's qsort
routine (see Listing Four) for sorting in ascending and descending order. Once
sorted, the values are displayed and the user is prompted for a value to
search for. The input value is passed to C's bsearch function and the results
of the search are displayed. The sort could have been done using a bubble sort
in Fortran, but a quick sort is much faster. C's bsearch function is used for
similar reasons. Add to that C's ability to manipulate pointers, and things
start moving significantly faster. Finally, qsort and bsearch are part of the
run-time library, so there's no need to reinvent the wheel.


Final Note on Versions


As mentioned at the beginning of this article, I used Borland's Turbo C 1.5.
TC 2.0 was not supported at the time of this writing, although Lahey plans to
release a new version of F77L by the time this article reaches print. Lahey
also supports Microsoft C 3.2 and up, although there are problems passing
character lengths in MSC 4.0. The connection also works with early versions of
Lattice C, but Lahey no longer supports that compiler. On the Fortran side,
Lahey provides a 32-bit version of F77L (F77L-EM/32) that is compatible with
Metaware's High C 386. On the low end, Lahey provides a student version, LP77,
that is compatible with the C compilers mentioned earlier. LP77, however,
requires routines from the LP77 toolkit, which is available separately from
Lahey Computer Systems.
Author's Note: I want to thank everyone on the Lahey technical staff for their
assistance while preparing this article.


Availability



All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063, or call 800-356-2002 (from inside
Calif.) or 800-533-4372 (from outside Calif.). Please specify the issue number
and format (MS-DOS, Macintosh, Kaypro).


THE C-TO-FORTRAN CONNECTION
by Michael Floyd


[LISTING ONE]

/* DOT.C - uses Lahey FORTRAN's random number generator to randomly
 place pixels on the screen.
 This example demonstrates how to call F77L functions and
 subroutines from a C program.
 Uses Initialize() from Borland's BGIDEMO example to
 perform hardware detection, load the appropriate BGI
 driver and initialize the system to graphics mode.
 Link line using Borland's TLINK is as follows:
 link bcf77l.150+c0l+frand+do_frand,frand,,emu+mathl+cl+f77l
*/

#include <stdio.h>
#include <stdlib.h>
#include <graphics.h>

char *DriverNames[] = {
 "Detect",
 "CGA",
 "EGA",
 "HercMono",
 "VGA"
};

struct PTS {
 int x, y;
}; /* Structure to hold vertex points */

int GraphDriver; /* The Graphics device driver */
int GraphMode; /* The Graphics mode value */
double AspectRatio; /* Aspect ratio of a pixel on the screen*/
int MaxX, MaxY; /* The maximum resolution of the screen */
int MaxColors; /* The maximum # of colors available */
int ErrorCode; /* Reports any graphics errors */
struct palettetype palette; /* Used to read palette info */

/* */
/* Function prototypes */
/* */

extern void frand (int *, int *);
extern void seed_rnd (int *);
void Initialize(void);
void RandomDot(void);

/* Begin main() */

main()
{

 Initialize(); /* Set system into Graphics mode */
 RandomDot(); /* Place pixels at random locations */
 closegraph(); /* Return the system to text mode */

} /* End main() */

/* INITIALIZE: Initializes the graphics system and reports */
/* any errors which occured. */

void Initialize(void)
{
 int xasp, yasp; /* Used to read the aspect ratio*/

 GraphDriver = DETECT; /* Request auto-detection */
 initgraph( &GraphDriver, &GraphMode, "" );
 ErrorCode = graphresult(); /* Read result of initialization*/
 if( ErrorCode != grOk ){ /* Error occured during init */
 printf(" Graphics System Error: %s\n", grapherrormsg( ErrorCode ) );
 exit( 1 );
 }

 getpalette( &palette ); /* Read the palette from board */
 MaxColors = getmaxcolor() + 1; /* Read maximum number of colors*/

 MaxX = getmaxx();
 MaxY = getmaxy(); /* Read size of screen */

 getaspectratio( &xasp, &yasp ); /* read the hardware aspect */
 AspectRatio = (double)xasp / (double)yasp; /* Get correction factor */

} /* End Initialize */

void RandomDot(void)
{
 int seed;
 int i, x, y, height, width, rand_val, color, temp;
 struct viewporttype vp;

 getviewsettings( &vp );
 height = vp.bottom - vp.top;
 width = vp.right - vp.left;

 seed_rnd( &seed ); /* Seed F77L's Random # Gen. Output discarded */

 for( i=0 ; i<1000 ; ++i ){ /* Put 1000 pixels on screen */
 temp = width - 1;
 frand( &rand_val, &temp ); /* Call F77L's RND function */
 x = rand_val + 1;
 temp = height - 1;
 frand( &rand_val, &temp );
 y = rand_val + 1;
 frand( &rand_val, &MaxColors );
 color = rand_val;
 putpixel( x, y, color );
 } /* End for loop */

} /* End RandomDot() */







[LISTING TWO]

c
c FRAND.FOR - Calls F77L's random Number generator RND.
c Demonstrates how to call a FORTRAN function from C
c
c Inputs : None
c Outputs: RETVAL

 FUNCTION FRAND(N)

 BCEXTERNAL FRAND
 INTEGER*2 N, FRAND

 FRAND = INT(RND() * N )
 RETURN
 END

c
c SEED_RND - Used to seed F77L's random Number generator.
c Demonstrates how to call a FORTRAN subroutine from C
c
c Inputs : None
c Outputs: RETVAL

 SUBROUTINE SEED_RND(RETVAL)

 BCEXTERNAL SEED_RND
 INTEGER*2 RETVAL

 RETVAL = INT(RRAND())
 RETURN
 END







[LISTING THREE]

c
c SEARCH.FOR uses rnd() to generate a list of random values
c that are then passed to C's qsort routine for sorting in
c ascending and descending order. Once sorted, the values
c are dislayed and the user is prompted for a value to search
c for. The input value is passed to C's bsearch function and
c the results of the search are displayed
c
c To link, use the following command line:
c
c tlink f77lbc.150+search+do_srch,search,,emu+mathl+cl+f77l
c
 PROGRAM SEARCH


 BCEXTERNAL q_sort, bin_search
 INTEGER*2 A(0:20), B(0:20), C(0:20), I, J
 INTEGER*2 FOUND, bin_search, VAL, R

 DO 10 I = 0, 19
 A(I) = 0
 B(I) = 0
 C(I) = 0
10 CONTINUE

 R = rrand()
 PRINT *, R
 DO 20 I = 0, 19

 A(I) = 32767.0 * rnd()
 B(I) = A(I)
 C(I) = A(I)

20 CONTINUE

 PRINT *, 'Input Ascending Descending'
 PRINT *, 'MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM'
 call q_sort(A,20,0)
 call q_sort(B,20,1)
 DO 30 J = 0, 19
 PRINT 40, C(J), A(J), B(J)

30 CONTINUE
40 FORMAT(I6,I13,I13)
 PRINT *, 'Enter value to search for: '
 READ *, VAL
 FOUND = bin_search(A, 20, VAL)
 IF (FOUND .NE. 0) THEN
 PRINT *, VAL, ' found in list!'
 ELSE
 PRINT *, VAL, ' NOT found!'
 ENDIF

 END







[LISTING FOUR]

/* do_srch.c
** q_sort()
** This function will take a one dimensional array of length n and
** integer width and will sort it in ascending or descending order.
** Nothing is returned -- status always equals zero if any checking
** is done by the FORTRAN calling program.
** inputs: array ptr to array
** length number of elements in array
** order 0 = ascending
** 1 = descending

** output: none
*/
#include <stdio.h>
#include <stdlib.h>

int q_sort (void *array,int *length,int *order);
int ascending (void *first,void *second);
int descending (void *first,void *second);
int bin_search (void *array, int *length, int *key);

int q_sort (void *array,int *length,int *order)
{
int status = 0; /* return value */

qsort (array,(size_t) *length,sizeof (int),
 (int(*)(const void *,const void *)) ((*order == 0) ? ascending :
descending));
return (status);
} /* end of do_sort() */

/* ascending()
** This function is used by qsort and/or bsearch to return a
** value based on the comparison of two inputs. qsort uses this
** function to perform an ascending sort.
** inputs: first ptr to first element
** second ptr to second element
** return: result of comparison
*/

int ascending (void *p1, void *p2)
{
return ((*(int *) p1 < *(int *) p2) ? (-1) : (*(int *) p1 == *(int *) p2) ?
(0) : (1));
}

/* descending()
** This function is used by qsort and/or bsearch to return a
** value based on the comparison of two inputs. qsort uses this
** function to perform a descending sort.
** inputs: first ptr to first element
** second ptr to second element
** return: result of comparison
*/

int descending (void *p1, void *p2)
{
return ((*(int *) p1 < *(int *) p2) ? (1) : (*(int *) p1 == *(int *) p2) ? (0)
: (-1));
}

/* bin_search()
** This function takes a sorted FORTRAN array and a key and
** attempts to locate the key value using C's bsearch(). The
** function passes back the value if found, or 0 if not found.
** inputs: array ptr to an array
** length length of the array
** key ptr to a key value
** return: result of search
*/

int bin_search (void *array, int *length, int *key)
{

 int *ptr;

 ptr = (int *) bsearch(key, array, (size_t) *length, sizeof(int), ascending);
 return(ptr != NULL);

}

[EXAMPLE 1]


/* Passing a string to FORTRAN from a C main() */
typedef struct {
 char *text;
 int length;
} CHARACTER;

extern void f_subroutine(int *, float *, CHARCTER *);

main() {
 int ival;
 float fval;
 CHARACTER cval;

 cval.text = "contents of variable";
 cval.length = strlen(cval.text);
 f_subroutine(&ival, &fval, &cval);
} /* End of C example */

c
c FORTRAN subroutine to accept and print a string
c
SUBROUTINE F_SUBROUTINE(I, F, C)
 BCEXTERNAL F_SUBROUTINE
 INTEGER*2 I
 REAL F

 CHARACTER*(*) C
 PRINT I, F, C
END


[EXAMPLE 2]

/* C module to get FORTRAN function return value */
extern void f_function(double *, double *);

c_module_main() {
 double, dval, return_val;

 f_function(&return_val, &dval);
}

c
c FORTRAN function to calculate the cube of the input number
c
function f_function(x)
MSCEXTERNAL f_function
double precision x, f_function


f_function = x * x * x
return
end



























































August, 1989
TRANSLATING PCX FILES


It's possible with PostScript and C




Kent Quirk


Kent is a free-lance writer and president of Totel Systems, Inc., a company
that develops device drivers, custom software, and embedded systems. Kent can
be reached at 489 Groton Rd., Westford, MA 01886 or on BIX or MCI Mail as
kquirk.


PCX files have become somewhat of a standard these days. Developed by ZSoft
(Marietta, Georgia) for use with their PC Paintbrush graphics editor, PCX is a
reasonably compact, yet simple, method of storing graphics data. The PCX file
format gained popularity because PC Paintbrush was packaged with the Microsoft
Mouse, and most mouse users tinkered with it. Nowadays, it seems most
scanners, fax boards, and desktop publishing systems process PCX files. I
found, however, that I had no handy way of printing these images on a
PostScript printer except by reading them into Lotus Manuscript first, which
wasn't always convenient (particularly when the image came from a fax card).
PostScript, on the other hand, is a page-definition language (PDL), a
programming language enhanced with a set of operators that allow the
specification of marks on a page, including images, line-drawing, and text. It
is a stack-oriented language whose most salient (and interesting) feature is
that it can be read and processed in linear order, without ever backing up.
This makes it ideal for use in a printer, which typically has a unidirectional
connection to a computer. PostScript is probably the most powerful way
currently available for specifying the description of an output page in a
printer.
The problem with PostScript is that everything a printer does has to be
written in PostScript. You can't send it straight text (PrintScreen doesn't
work at all), and you certainly can't send it an image without working pretty
hard.
So how do you print an image in PostScript? Using the image operator is most
common. This function takes a stream of data and converts it to a rectangular
bitmap of arbitrary size, with 1, 2, 4, or 8 bits per pixel. But PostScript
can't accept binary data (it reserves and traps certain ASCII control
characters), so all its input must be converted to hex or some other printable
form, which then has to have PostScript code added to it to make it print.
In the course of my work, I needed a program to perform a PCX-to-PostScript
translation. Because C is my language of choice for most projects, I decided
to write the program, PRPCX, in C. In the process of writing PRPCX, I learned
a lot about PCX files and imaging in PostScript, and codified some of my
thinking about C.


C Idioms


In my seven years of programming in C, I've developed certain idioms -- common
bits of code which fairly type themselves into the program. These are not
subroutines or include files; they are more like mental templates that guide
the generation of code specific to the application I'm working on. I find that
good C programmers tend to develop many such idioms, perhaps in self-defense.
Because of the expressive power of C, there are usually several different ways
to perform a given task. C has been accused of being a write-only language. I
feel this is true only for programmers who fail to develop and follow the
particular idioms of the language. C can be written (badly) to look like
Pascal or any of a number of other languages. A good programmer learns and
develops a "C-ish" programming style, that becomes a standard code that
requires a little effort to write or understand. A classic example is the
traditional counting loop:
 for(i=0; i<last; i++)
 do_something( );
I've developed larger idioms as well, several of which appear in this program.
The first problem in writing the translation is developing a set of tools for
reading and manipulating the PCX files. The two files PCX.C (Listing One) and
PCX.H (Listing Two) provide enough facilities to read PCX files, but not to
create them. They are written for generality and clarity, not efficiency.
Application of better buffering or converting the data to final form on the
fly, could speed up the process greatly, but would complicate matters
unnecessarily for our purposes. Michael Swaine would point out a Programming
Paradigm: First get it right, then get it fast. Although there are exceptions
to any rule, it certainly applies in this case.


PCX File Characteristics


A PCX file has a 128-byte header that describes the contents of the rest of
the file. ZSoft has not been lazy -- there have been at least five versions of
PC Paintbrush, and PCX files have changed to keep up. What's good is that the
same header format applies to all PCX versions to date. What's bad is that the
data formats that follow the header vary wildly -- unfortunately for
portability, PCX files are tied fairly closely to the video board on which
they were created.
Some video boards (such as the EGA and VGA in certain modes) arrange their
memory in the form of bit planes. This is a way of looking at graphics memory
three-dimensionally. The x and y dimensions define an array of bits that
corresponds to the dimensions of the video display. The z axis defines the
number of bits per pixel for the video board. One pixel is spread among
several different memory banks. Provided you have the help of a graphics
controller, this memory organization allows for great flexibility in graphics
processing.
In the PCX format, if the video board uses bit planes, the data is stored in
bit plane form. One pixel is stored across multiple bytes of data. Rearranging
these bit planes into single-pixel values, then cross-referencing them so they
can be printed sensibly on a monochrome printer might be an interesting
exercise, but is beyond the scope of this article.
Monochrome boards (or monochrome modes on color boards) just store the data as
a stream of bits. Since the printer is monochrome anyway, I chose to limit
PRPCX to using only files created in monochrome mode.
The first routine (pcx_read_header) reads the PCX header into a data area. It
allocates a data area if one is not specified. Here's the first idiom: When
passing a pointer to a structure as an argument to a subroutine, I take a NULL
pointer to mean that the user is requesting automatic allocation of the data
area.
The pcx_header fields include information on the version of PCX used to create
the file, the data compression method (only RLE is supported to date), and the
number of bits per pixel in the image. The PCX file defines a rectangle of
data which always contains an even number of bytes per line and a number of
lines that is a multiple of eight. The upper-left and lower-right parameters
define the actual size and position of the image data within that rectangle.
If the video board uses a palette, it is stored in the palette area if it fits
(it fits for up to 16 colors). If it doesn't fit, it is stored at the end of
the file, with the type of palette indicated by paletteinfo.
The print_pcx_header function prints the data to a given file in user-readable
form. For someone trying to use sizing and scaling to position an image, this
can be helpful.
I broke out the pcx_alloc_line() routine because it might be desirable to call
it individually in the case where you want to build an array of data lines for
faster response or for image editing.
The pcx_next_line() function is the key to the whole process. It reads the
next line from the PCX file and returns it, in unpacked raw data form, to the
caller. Understanding it requires an understanding of the PCX file format and
its method of data compression.
PCX files use run length encoding (RLE) for compression -- basically, any
consecutive data bytes which have the same value are replaced by a count byte
followed by the value. Count bytes are distinguished from data bytes by
setting the top two bits; the bottom six bits represent the number of
repetitions, from 0 to 63. If a data byte needs to use a value with data in
this range, it can do so by specifying a repeat count of 1.
RLE is sometimes an excellent way to store image data, especially if (like
most pictures generated with PC Paintbrush) the image contains large areas of
solid colors. However, on random data or gray scale images, where there are
few repeated data bytes, the PCX format is likely to be some 25 percent larger
than raw data size because of the escape code necessary for data bytes with
the top two bits set. For more information on RLE, see "Run Length Encoding"
by Robert Zigon (DDJ, February 1989) and "RLE Revisited" by Phil Daley (DDJ,
May 1989).
The file PRPCX.C (Listing Three) contains the rest of the code while Listing
Four is the Make file. The general flow is to first initialize the map
structure to default values. This structure is used to contain the scaling and
positioning data for placing the PCX image on the PostScript page. Next,
process the command line and set up the values for scaling and sizing. The .PS
file is then copied to the output. The scaling and sizing variables are
emitted, and finally the .PCX file is read and emitted as hex data.
The command line parser is another idiom that just flies from my fingertips
whenever I need it. Basically, it's a simple checker that looks for a leading
hyphen (-) or slash (/) in each argument. If it sees either one, it does a
switch on the second character. Matches generate either string assignments or
a conversion, as in x = atoi(argv[i]+2);.
If no leading slash is seen, assume that the argument is a filename and assign
the filename variable to it. This is easy, but limits the program to one
filename per run. Because there are good reasons not to generate more than one
image per PostScript file, this isn't really a problem.
I've used this command line parser so often that it just writes itself, with
no concentration required to make it work. Although it's not as flexible as a
general-purpose parsing subroutine, it's small, fast and easy. Only rarely
have I had to get more sophisticated.
Normally, this program sends its output to STDOUT, so it can be piped to a
disk file, another filter, or directed to the printer. Direct to the printer,
however, may be the most commonly used form. Because people's printers don't
change names, it made sense to allow the user to put the printer name in the
environment. Once set, it stays set until a reboot or it is explicitly
changed. This is ideal for a utility likely to use the same arguments over and
over. Those of you who prefer to use the command line can redirect the output,
or use the -o switch (which makes the most sense in batch files).
If no arguments are specified, there's nothing to do, so a usage message is
printed. This too is important. I believe that a user should always be able to
figure out something about what the program does and how to get started
without having to read the manual (or the article). Now that everything is set
up, the program calls dofile(), which does the rest of the job.
The first function dofile() does is open the PCX file. Another idiom: If the
filename doesn't contain a period, concatenate .PCX to the end of the name.
This way the user doesn't have to type the obvious, but can override any
assumptions.
dofile() opens the file, and if the user asks for it, dumps the header and
returns without doing further work. Next read the PostScript prologue file.
Many programmers aren't aware that the first element in the argv array (for
DOS versions later than 3.0) is a pointer to the complete path and filename of
the program. For earlier versions of DOS, the program name is available (put
there by the compiler), but the path from which it is run is not. Use this
information to find the prologue file -- and simply require it to have the
same path and name as the executable file, but with the .PS extension. Again a
simple rule that doesn't restrict the user.


The PostScript Code



I'm using a fairly standard PostScript technique. First, create a prologue
file PRPCX.PS (Listing Five) that contains PostScript code that defines all
the functions needed to handle the image generation. Copy this file to the
beginning of the output file. This way, the C program doesn't have to know
much about PostScript, and an experienced user can modify the prologue to do a
different job without having to recompile the software.
The prologue in this case contains functions for building the transform
matrix, reading the image data from the file, doing scaling and image
generation, and printing the page after the image is complete.
Adobe (Mountain View, Calif.) has defined a comment convention for PostScript
files. Files conforming to the specification can be specially handled by print
servers, or read into desktop publishing programs as Encapsulated PostScript
(EPS) files. The specification is laid out in Adobe's Red Book, Appendix C.
Basically, minimally conforming programs contain information that defines the
fonts used and image size.
Normally, this information is placed at the beginning of the file. However,
Adobe allowed the instruction (atend) to mean that the actual data is found at
the end of the file. My original intention for PRPCX was to place the
BoundingBox item (which describes the boundaries of all the black marks on a
page) at the end of the file. Unfortunately, I discovered that Ventura
Publisher, in violation of the EPS specifications, requires BoundingBox to be
placed at the beginning of the file.
So for ease of programming, PRPCX is limited to processing one file at a time,
and there is a special hack to fill in the BoundingBox entry in PRPCX.PS. The
file is copied to the output until it sees the line containing BoundingBox. It
then generates a new BoundingBox line based on the header read from the data
file, and continues with the file copy.
The calculations used are the same as those used by PostScript's image
operator, which is capable of scaling and flipping bitmaps. The syntax of the
image operator is as follows:
 width height bpp transform readproc image
For those who don't know PostScript, the first thing to learn is that it's a
stack-oriented language. Any operator takes its arguments on the stack, so
they come before the operator does. In this case, the width and height
operators are found deepest on the stack; they are the width and height of the
bitmap in pixels. The bpp operand tells the number of bits per pixel in the
image. 1, 2, 4, and 8 are allowed; this program only handles images with 1 bit
per pixel.
PostScript allows objects of any type on the stack. The [] (mark) operators
define an array, which (like lists in Lisp) simply collect everything between
them into a single object. In the case of the transform operand, image expects
a six-element array containing the transform matrix, which is used as a
multiplier to convert the image into the output area.
Finally, image accepts a procedure (again on the stack) which it calls to get
image data. Each time this procedure is called it should return a string
containing image data. When width * height data elements have been consumed,
image is finished processing. Returning a zero-length string aborts image
early. Use the readhexstring operator (see the readproc function in PRPCX.PS)
to read a hex string of data from the input and convert it to binary.
The Adobe Red Book talks of a convention that speeds up the generation of an
image; if the image conversions yield one input pixel per device pixel, all
scaling and transforming is turned off, for a large increase in speed. The
default behavior of PRPCX is to generate the bitmap at this size.
Adobe recommends handling the transformation in two parts. First, use the
transform matrix operand (part of the image operator) to convert the image to
a 1 x 1 unit square with the image upright. Then use translate and scale to
move the image to the shape and location desired.
Translating to the unit square is slightly complicated by the fact that a
PostScript is measured from the bottom of the page, while a PCX image starts
at the top. Fortunately, the transform matrix takes care of this neatly. A
normal transformation to the unit square uses the following matrix:
 [width 0 0 height 0 0]
The transform to invert the Y axis is:
 [width 0 0 -height 0 height]
In PostScript, that last matrix is written:
 [width 0 0 height neg 0 height]
The neg (negate) operator inverts the sign of the object on top of the stack
(which is height, in this case). The next step is to convert to device
resolution. This requires that you first translate the coordinate axes to the
desired position using standard resolution, which is 72 dpi (one point). This
way the position of the image is independent of its size.
Next, make the image its original shape (assuming square pixels). To do this,
scale the unit square to the width and height of the image in pixels. Then
scale the coordinate system to device resolution, using the command
 72 res div 72 res div scale
The scale operator takes two operands, the scale factor in X and Y. Divide 72
by the actual resolution of the device. This makes one unit in device space
equivalent to one unit in user space. Finally, to allow the user to change the
scaling, add one more scale factor. All of these transformations take place in
the scaleit routine within the prologue.
The image operator takes an argument which is a procedure to supply it with
data. Because the PostScript data stream has certain reserved characters, use
the readhexstring operator to accept hex data. This doubles the number of
bytes sent to the printer, but that's the way it's generally done; it's
certainly the easiest to code. The readhexstring operator fills a string with
data from the input file. In order to ensure that it doesn't read past the end
of the file, you have to build a string just large enough to hold one line of
input. The imagedata routine in PRPCX.PS contains code to do this.
Now we start generating PostScript code. First define and give values to all
the variables used by the prologue. Then tell PostScript to actually do the
scaling and translation calculations. Finally, tell it to start accepting an
image.
The image is processed by expanding each line to raw data using the PCX
routines, inverting it if necessary, then emitting the data as a stream of hex
bytes. Each line of input data is separated by a newline in the output file.
At the end of the file, emit the code to get PostScript to generate the image
(the single operator showpage), and you're done. The grestore operator
restores the image context that existed before you started this whole process.
This is simply good manners; a good idea when generating EPS files.
The result of all this work has been the conversion of a PCX file, a useful
and standard data format for images, into PostScript, another useful and
standard format, with C as the medium of exchange. In the process, we've
learned to make efficient use of all three.


Acknowledgments


I'd like to thank ZSoft for its booklet of information on PCX files, and
Laser-Go for the use of GoScript, a program which allows the use of PostScript
code with printers that don't support it in native mode.


Availability


All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063, or call 800-356-2002 (from inside
Calif.) or 800-533-4372 (from outside Calif.). Please specify the issue number
and format (MS-DOS, Macintosh, Kaypro).


TRANSLATING PCX FILES
by Kent Quirk


[LISTING ONE]

/*+
 Name: pcx.c
 Author: Kent J. Quirk
 Abstract: This file contains subroutines to read PCX files.
-*/

#include <stdio.h>
#include <stdlib.h>
#include <malloc.h>
#include <memory.h>

#include "pcx.h"

/**** p c x _ r e a d _ h e a d e r ****
 Abstract: Reads the header of a PCX file.
 Parameters: A data storage area for the header, an open file.
 Returns: The pointer to the data storage area passed, or NULL if error.
 Comments: The file should be opened in binary mode.
****************************/
PCX_HDR *pcx_read_header(PCX_HDR *hdr, FILE *f)
{
 fseek(f, 0L, SEEK_SET); /* header is at top of file */
 if (fread(hdr, 1, sizeof(PCX_HDR), f) != sizeof(PCX_HDR))
 return(NULL);
 else
 return(hdr);
}

/**** p c x _ p r i n t _ h e a d e r ****
 Abstract: Printf's a PCX file header data to a given file.
 Parameters: The PCX file header, the file to write the data to.
 Returns: Nothing
****************************/
void pcx_print_header(PCX_HDR *hdr, FILE *f)
{
 char *t;
 if (hdr->pcx_id != 0x0A)
 {
 fprintf(f, "Not a PCX file.\n");
 return;
 }
 switch (hdr->version) {
 case 0: t="2.5"; break;
 case 2: t="2.8 with palette info."; break;
 case 3: t="2.8 without palette info."; break;
 case 5: t="3.0"; break;
 default: t="unknown."; break;
 }
 fprintf(f, "PCX Version %s\n", t);
 fprintf(f, "Compression: %s\n",
 hdr->encoding==1 ? "Run length" : "Unknown");
 fprintf(f, "%d bits per pixel\n", hdr->bpp);
 fprintf(f, "Image from (%d, %d) to (%d, %d) pixels.\n",
 hdr->upleftx, hdr->uplefty, hdr->lorightx, hdr->lorighty);
 fprintf(f, "Created on a device with %d x %d dpi resolution.\n",
 hdr->display_xres, hdr->display_yres);
 fprintf(f, "The image contains %d image planes.\n", hdr->nplanes);
 fprintf(f, "There are %d bytes per line of data after decompression.\n",
 hdr->bytesperline);
 fprintf(f, "There are %ld total bytes in the image.\n",
 ((long)hdr->lorighty - (long)hdr->uplefty + 1) *
 (long)hdr->bytesperline);
 return;
}

/**** p c x _ a l l o c _ l i n e ****
 Abstract: Allocates enough space to store a complete scan line from the
 current PCX file.
 Parameters: The header pointer.
 Returns: A pointer to a "big enough" block of allocated space, or

 null if not enough space or an error occurred.
****************************/
BYTE *pcx_alloc_line(PCX_HDR *hdr)
{
 return(calloc(hdr->nplanes, hdr->bytesperline));
}

/**** p c x _ n e x t _ l i n e ****
 Abstract: Reads the next line from the PCX file.
 Parameters: Pointer to the header, a pointer to the line area (or NULL
 for automatic allocation), and the open data file.
 Returns: A pointer to the line area, or NULL if there was a problem.
 Comments: To read the first line, call pcx_read_header() first.
 This sets the file position to the beginning of the data.
****************************/
BYTE *pcx_next_line(PCX_HDR *hdr, BYTE *line, FILE *f)
{
 int c, len;
 BYTE *dp;
 WORD linesize = hdr->nplanes * hdr->bytesperline;
 WORD i;
 if (line == NULL)
 if ((line = pcx_alloc_line(hdr)) == NULL)
 return(line);
 dp = line; /* point to data */
 for (i=0; i<linesize; )
 {
 if ((c = fgetc(f)) == EOF)
 return(NULL);
 if ((c & PCX_COMPRESSED) == PCX_COMPRESSED)
 {
 len = (c & PCX_MASK);
 if ((c = fgetc(f)) == EOF)
 return(NULL);
 memset(dp, (BYTE)c, len);
 dp += len;
 i += len;
 }
 else
 {
 *dp++ = (BYTE)c;
 i++;
 }
 }
 return(line);
}






[LISTING TWO]

/*+
 Name: pcx.h
 Author: Kent J. Quirk
 Abstract: This file contains information required when handling
 PCX files.

-*/

/********************
 Need these to handle the PCX data below.
*********************/
typedef unsigned char BYTE;
typedef unsigned int WORD;

/********************
 This is the definition of the PCX header.
*********************/
typedef struct {
 BYTE pcx_id; /* Always 0x0A for PCX files */
 BYTE version; /* Version of the PCX format */
 BYTE encoding; /* 1 = RLE (RLL) compression */
 BYTE bpp; /* Number of bits per pixel */
 WORD upleftx, uplefty; /* position of upper left corner */
 WORD lorightx, lorighty; /* lower right corner (inclusive) */
 WORD display_xres, display_yres; /* resolution in dpi of display */
 BYTE palette[48]; /* palette data if it fits */
 BYTE reserved;
 BYTE nplanes; /* number of bit planes of data */
 WORD bytesperline; /* # bytes in an uncompressed line */
 WORD palletteinfo;
 BYTE reserved2[58];
} PCX_HDR;

/********************
 These two definitions are used to decompress data in the PCX file.
 (The compressed count byte has the top two bits set).
*********************/
#define PCX_COMPRESSED 0xC0
#define PCX_MASK 0x3F

/********************
 These prototypes declare the PCX read subroutines.
*********************/
PCX_HDR *pcx_read_header(PCX_HDR *hdr, FILE *f);
BYTE *pcx_alloc_line(PCX_HDR *hdr);
BYTE *pcx_next_line(PCX_HDR *hdr, BYTE *line, FILE *f);
void pcx_print_header(PCX_HDR *hdr, FILE *f);






[LISTING THREE]

/*+
 Name: prpcx.c
 Author: Kent J. Quirk
 Abstract: This program prints .PCX files (as created by PC Paintbrush
 and other software) on a PostScript printer by converting
 them to a PS-compatible image. The user can scale and
 position the image.
-*/

#include <stdio.h>

#include <stdlib.h>
#include <memory.h>
#include <string.h>

#define BUFSIZE 100

#include "pcx.h"

typedef struct {
 int xpos;
 int ypos;
 int width;
 int height;
 int scale;
 int invert;
 int prt_res;
 int dumphdr;
} MAPPING;

/**** c o p y _ p s _ h e a d e r ****
 Abstract: Opens the PS header file and copies it to the output.
 Parameters: Filename of the current file (the .PS extension is added
 here) and the output file pointer.
 Returns: 0 if successful, 1 if failure.
****************************/
char *copy_ps_header(char *name, FILE *outfile, char *stop)
{
 static char buf[BUFSIZE];
 char *bp;
 static FILE *f = NULL;
 if (f == NULL)
 {
 strcpy(buf, name);
 if ((bp = strchr(buf, '.')) != NULL)
 *bp = 0;
 strcat(buf, ".ps"); /* open file with this name but .ps ext */
 if ((f = fopen(buf, "r")) == NULL)
 {
 fprintf(stderr, "Unable to open PostScript header file '%s'\n",
 buf);
 return(NULL);
 }
 }
 else
 {
 fputs(buf, outfile);
 }
 while (fgets(buf, BUFSIZE, f) != NULL)
 {
 if ((stop != NULL) && (strncmp(buf, stop, strlen(stop)) == 0))
 return(buf); /* bail out right now */
 fputs(buf, outfile);
 }
 fclose(f);
 f = NULL;
 return(NULL);
}

/**** d o f i l e ****

 Abstract: Processes a single PCX file.
 Parameters: char *filename - the input PCX filename (.PCX optional)
 MAPPING *map - the structure containing page position info
 char *psname - the PostScript prologue (.PS will be forced)
 FILE *outfile - the open output file
 Returns: 0 if successful, 1 if no file generated
****************************/
int dofile(char *filename, MAPPING *map, char *psname, FILE *outfile)
{
 FILE *f;
 PCX_HDR hdr;
 WORD i, j, xsize, ysize;
 BYTE *lineptr = NULL;
 char *t;
 char buf[BUFSIZE];
 long bbox_x, bbox_y;
 strcpy(buf, filename);
 if (strchr(buf, '.') == NULL)
 strcat(buf, ".pcx"); /* add .PCX if needed */
 if ((f = fopen(buf, "rb")) == NULL)
 {
 fprintf(stderr, "Unable to open '%s'\n", buf);
 return(1);
 }
 if (pcx_read_header(&hdr, f) == NULL)
 {
 fprintf(stderr, "Unable to read header for file '%s'.\n", buf);
 fclose(f);
 return(1);
 }
 if (map->dumphdr)
 {
 pcx_print_header(&hdr, stdout);
 return(1);
 }
 if (hdr.nplanes != 1)
 {
 fprintf(stderr, "Only able to read monochrome .PCX files.\n");
 fclose(f);
 return(1);
 }
 xsize = hdr.lorightx - hdr.upleftx + 1;
 ysize = hdr.lorighty - hdr.uplefty + 1;

 t = copy_ps_header(psname, outfile, "%%BoundingBox");
 bbox_x = (long)xsize * (long)map->width * (long)map->scale / 10000L;
 bbox_y = (long)ysize * (long)map->height * (long)map->scale / 10000L;
 bbox_x += map->xpos;
 bbox_y += map->ypos;
 sprintf(t, "%%%%BoundingBox: %d %d %ld %ld\n", map->xpos, map->ypos,
 bbox_x, bbox_y);
 t = copy_ps_header(psname, outfile, NULL);

 fprintf(outfile, "/bmap_wid %d def\n", xsize);
 fprintf(outfile, "/bmap_hgt %d def\n", ysize);
 fprintf(outfile, "/bpp %d def\n", hdr.bpp);
 fprintf(outfile, "/res %d def\n\n", map->prt_res);
 fprintf(outfile, "/x %d def\n", map->xpos);
 fprintf(outfile, "/y %d def\n\n", map->ypos);

 fprintf(outfile, "/scy %d 100 div def\n", map->height);
 fprintf(outfile, "/scx %d 100 div def\n", map->width);
 fprintf(outfile, "/scg %d 100 div def\n\n", map->scale);
 fprintf(outfile, "scaleit\n");
 fprintf(outfile, "imagedata\n\n");

 for (i=0; i<ysize; i++)
 {
 lineptr = pcx_next_line(&hdr, lineptr, f);
 if (map->invert) /* invert if necessary */
 for (j=0; j < xsize/8; j++)
 lineptr[j] = ~lineptr[j];
 for (j=0; j < xsize/8; j++)
 fprintf(outfile, "%02X", lineptr[j]);
 fprintf(outfile, "\n");
 }
 fprintf(outfile, "\nshowit\n");
 free(lineptr);
 fclose(f);
 return(0);
}

/**** u s a g e ****
 Abstract: Prints a usage message and dies.
 Parameters: None
 Returns: Never returns.
****************************/
void usage()
{
 printf("PRPCX: by Kent Quirk\n");
 printf(" Given a .PCX file, this program creates a PostScript file \n");
 printf(" which will print the image.\n");
 printf(" PRPCX [-wW] [-hH] [-xX] [-yY] [-sS] [-rR] [-d] [-i] filename\n");
 printf(" Options include: (units) [default]\n");
 printf(" -sSCA set overall scale factor (percent) [100]\n");
 printf(" -wWID set horizontal scale factor (percent) [100]\n");
 printf(" -hHGT set vertical scale factor (percent) [100]\n");
 printf(" -xPOS set horizontal position (points from left) [0]\n");
 printf(" -yPOS set vertical position (points from bottom) [0]\n");
 printf(" -rRES set printer resolution (dpi) [300]\n");
 printf(" -d dump PCX file info to stdout [off]\n");
 printf(" -i invert image [off]\n");
 printf(" -oFIL set output filename, or use SET PRPCX=filename\n");
 printf(" The defaults print the image at one pixel per device pixel\n");
 printf(" at the lower left corner of the page.\n");
 printf(" PRPCX.PS must be in the same directory as PRPCX.EXE.\n");
 exit(1);
}

/**** m a i n ****
 The main routine for PRPCX. Sets defaults, parses command line,
 and calls dofile().
****************************/
int main(int argc, char *argv[])
{
 int i;
 MAPPING map;
 FILE *outfile = stdout;
 char *outfname = NULL;

 char *filename = NULL;
 map.xpos = map.ypos = 0;
 map.width = map.height = map.scale = 100;
 map.invert = 0;
 map.prt_res = 300;
 map.dumphdr = 0;
 if (argc < 2)
 usage();
 for (i=1; i<argc; i++)
 {
 if (argv[i][0] == '-' argv[i][0] == '/')
 {
 switch (argv[i][1])
 {
 case 'x': case 'X':
 map.xpos = atoi(argv[i]+2);
 break;
 case 'y': case 'Y':
 map.ypos = atoi(argv[i]+2);
 break;
 case 'h': case 'H':
 map.height = atoi(argv[i]+2);
 break;
 case 'w': case 'W':
 map.width = atoi(argv[i]+2);
 break;
 case 's': case 'S':
 map.scale = atoi(argv[i]+2);
 break;
 case 'r': case 'R':
 map.prt_res = atoi(argv[i]+2);
 break;
 case 'i': case 'I':
 map.invert = !map.invert;
 break;
 case 'd': case 'D':
 map.dumphdr = 1;
 break;
 case 'o': case 'O':
 outfname = argv[i]+2;
 break;
 case '?':
 usage();
 break;
 default:
 fprintf(stderr, "Unknown option %s\n", argv[i]);
 usage();
 break;
 }
 }
 else /* process a file */
 {
 filename = argv[i];
 }
 }
 if ((outfname != NULL) ((outfname = getenv("PRPCX")) != NULL))
 {
 if ((outfile = fopen(outfname, "w")) == NULL)
 {

 fprintf(stderr,"Unable to open output file %s", outfname);
 exit(1);
 }
 }
 i = dofile(filename, &map, argv[0], outfile);
 fclose(outfile);
 return(i);
}







[LISTING FOUR]

#
# Program: PRPCX
#

.c.obj:
 cl -c -W2 -Zid -Od -AS $*.c

pcx.obj : pcx.c pcx.h

prpcx.obj : prpcx.c pcx.h

prpcx.exe : prpcx.obj pcx.obj
 echo prpcx.obj+ >prpcx.lnk
 echo pcx.obj >>prpcx.lnk
 echo prpcx.exe >>prpcx.lnk
 echo nul >>prpcx.lnk
 link /NOI $(LDFLAGS) @prpcx.lnk;







[LISTING FIVE]

%!PS-Adobe 1.0
%%Title:
%%Creator:
%%Pages: 1
%%BoundingBox:
%%EndComments

gsave

 % the next item translates the image from
 % top-to-bottom .PCX format to PS bottom-to-top
/xform
{
 [ bmap_wid 0 0 bmap_hgt neg 0 bmap_hgt ]
} def


/readproc
{
 { currentfile picstr readhexstring pop }
} def

/scaleit
{
 x y translate
 bmap_wid bmap_hgt scale
 72 res div 72 res div scale
 scx scg mul scy scg mul scale

 /picstr bmap_wid 8 idiv string def

} def

/imagedata
{
 bmap_wid bmap_hgt bpp xform readproc image
} def

/showit
{
 grestore
 showpage
} def

% Will generate <bmap_hgt> lines of image data,
% each with <bmap_wid>/8 bytes of data (in hex).
%%EndProlog
































August, 1989
BUILDING YOUR OWN C INTERPRETER


Here's the source code for an interpreter of your own




Herbert Schildt


Herb is the author of more than two dozen computer books, with topics ranging
from C to Modula-2. This article is based on his book Born to Code in C
(Osborne/McGraw-Hill). He can be reached at RR #1, Box 130, Mahomet, IL 61853.


In this article, I develop a C interpreter that can execute a subset of K&R
ANSI C. The interpreter not only is functional as presented, but is designed
so that you can easily enhance and extend it. In fact, you can even add
features not found in ANSI C if you choose. By the time you finish reading
this article, you'll have a C interpreter you can use and enlarge, and you
will have gained considerable insight into the structure of the C language
itself.
Although ANSI C has only 32 keywords, it is a rich and powerful language. It
would, of course, take far more than a single article to fully describe and
implement an interpreter for the entire C language. Instead, this C
interpreter understands a fairly narrow subset of the language. Table 1 lists
the features that are implemented in it.
Table 1: Features provided by this C interpreter

 Parameterized functions with local
 variables

 Recursion

 The if statement

 The do-while, while, and for loops

 Integer and character variables

 Global variables

 Integer and character constants

 String constants (limited implementation)

 The return statement, both with and
 without a value

 A limited number of standard library functions.

 These operators: +, -, *, /, %, <, >,
 <=, >=, ==, !=, unary -, and unary +.

 Functions returning integers

 Comments

In order to simplify and shorten the source code to the interpreter, I have
imposed one small restriction on the C grammar: The targets of the if, while,
do, and for must be blocks of code surrounded by beginning and ending curly
braces. You may not use a single statement. For example, my interpreter will
not correctly interpret code in Example 1; instead, you must write code like
that in Example 2. Because the objects of the program control statements are
often blocks of code anyway, this restriction does not seem too harsh. (With a
little effort, though, you can remove this restriction.)
Example 1: This C interpreter will not interpret this code

 for(a=0; a<10; a=a+1)
 for(b=0; b<10; b=b+1)
 for(c=0; c<10; c=c+1)
 puts("hi");

Example 2: This C interpreter will interpret this code


 for(a=0; a<10; a=a+1) {
 for(b=0; b<10; b=b+1) {
 for(c=0; c<10; c=c+1) {
 puts("hi");

 }
 }
 }



Reducing the Source Code to its Components


To create the C interpreter, we first construct the expression evaluator and
the piece of code that reads and analyzes expressions, the expression parser.
There are several different ways to design an expression parser for C. Many
commercial compilers use a table-driven parser that is generally created by a
parser-generator program. While table-driven parsers are usually faster than
other methods, they are hard to create by hand. For this C interpreter I will
use a recursive-descent parser, which is essentially a collection of mutually
recursive functions that process an expression. If the parser is used in a
compiler, then its function is to generate the proper object code that
corresponds to the source code. In an interpreter, however, the object of the
parser is to evaluate a given expression.
Fundamental to all interpreters (and compilers, for that matter) is a special
function that reads the source code and returns the next logical symbol from
it. For historical reasons, these logical symbols are generally referred to as
tokens, and computer languages in general (and C in particular) define
programs in terms of tokens. This C interpreter categorizes tokens as shown in
Table 2. The function that returns tokens from the source code for the C
interpreter is called get_token() and is shown in Listing One, lines 313
through 423. The get_token() function uses the global data and enumeration
types in Figure 1.
Figure 1: The get_token( ) function uses the global data and enumeration types
as shown here

 char*prog;/* points to current location in source code*/
 extern char* p_buf;/* points to start of program buffer*/

 char token[80];/* holds string representation of token*/
 char token_type;/* contains the type of token*/
 char tok;/* holds the internal representation of token if it is a
 keyword*/

 enum tok_types {DELIMITER, IDENTIFIER, NUMBER, KEYWORD,
 TEMP, STRING, BLOCK};

 enum double_ops {LT=1, LE, GT, GE, EQ, NE};

 /* These are the constants used to call sntx_err( ) when a syntax error
 occurs. Add more if you like.
 NOTE: SYNTAX is a generic error message used when nothing
 else seems appropriate.

 */
 enum error_msg
 {SYNTAX, UNBAL_PARENS, NO_EXP, EQUALS_EXPECTED,
 NOT_VAR, PARAM_ERR, SEMI_EXPECTED,
 UNBAL_BRACES, FUNC_UNDEF, TYPE_EXPECTED,
 NEST_FUNC, RET_NOCALL, PAREN_EXPECTED,
 WHILE_EXPECTED, QUOTE_EXPECTED, NOT_TEMP,
 TOO_MANY_LVARS};

Table 2: The interpreter's token categories

 Token Includes
 Type

 delimiters punctuation and
 operators

 keywords keywords

 strings quoted strings

 identifiers variable and function

 names

 number numeric constant

 block {or}

The current location in the source code is pointed to by prog. The p_buf
pointer is unchanged by the interpreter and always points to the start of the
program being interpreted. The get_token( ) function begins by skipping over
all white space, including carriage returns and line feeds. Because no C token
(except for a quoted string or character constant) contains a space, spaces
must be bypassed. The get_token( ) function also skips over comments. Next,
the string representation of each token is placed into token, its type (as
defined by the tok_types enumeration) is put into token_type, and, if the
token is a keyword, its internal representation is assigned to tokvia the
look_up( ) function (shown in the full parser listing, Listing One). You can
see by looking at get_token( ) that it converts C's two-character relational
operators into their corresponding enumeration value. Although not technically
necessary, this step makes the parser easier to implement. Finally, if the
parser encounters a syntax error, it calls the function sntx_err( ) with an
enumerated value that corresponds to the type of error found. The sntx_err( )
function is also called by other routines in the interpreter whenever an error
occurs. The sntx_err() function is shown here in Listing One, lines 274
through 311.
The entire code for the C interpreter recursive descent parser is shown in
Listing One (PARSER.C), along with some necessary support functions, global
data, and data types. As listed, this code is designed to go into its own
file.
The atom( ) function and the functions that begin with eval_exp implement the
production rules for C expressions. To verify this, you might want to mentally
execute the parser using a simple expression.
The atom( ) function finds the value of an integer constant or variable, a
function, or a character constant. The two kinds of functions that may be
present in the source code are user-defined or library. If a user-defined
function is encountered, its code is executed by the interpreter in order to
determine its value. (The calling of a function will be discussed in the next
section.) If the function is a library function, however, then its address is
first looked up by the internal_func( ) function and is then accessed via its
interface function. The library functions and the addresses of their interface
functions are held in the internal_func array in lines 56 through 73.


The C Interpreter


The heart of the C interpreter is developed in this section. Before the
interpreter can actually start executing a program, however, a few clerical
tasks must be performed. Some method must be devised to allow execution to
begin at the right spot, and all global variables must be known and accounted
for before main( ) begins executing. It is important, though not technically
necessary, that the location of each function defined in the program be known
so that a call to a function can be as fast as possible. If this step is not
performed, a lengthy sequential search of the source code will be needed to
find the entry point to a function each time it is called.
The solution to these problems is the interpreter prescan. Prescanners (or
preprocessors, as they are sometimes called, though they have little
resemblance to a C compiler's preprocessor) are used by many commercial
interpreters regardless of what language they are interpreting. A prescanner
reads the source code to the program before it is executed and performs
whatever tasks can be done prior to execution. In this C interpreter, the
prescanner performs two important jobs: First, it finds and records the
location of all user-defined functions including main( ). Second, it finds and
allocates space for all global variables. Here, the function that performs the
prescan is called prescan( ) and is shown in Listing Two, LITTLEC.C, lines 190
through 228.
Global variables are stored in a global variable table called global_vars by
decl_global( ), lines 43 through 50 and lines 240 through 253. The integer
gvar_index(line 76) holds the location of the next free element in the array.
The location of each user-defined function is put into the func_table array,
shown in lines 52 through 55. The func_index variable (line 75) holds the
index of the next free location in the table.
The main( ) function to the C interpreter, shown in lines 92 through 118,
loads the source code, initializes the global variables, calls prescan( ),
primes the interpreter for the call to main( ), and then executes call( ),
which begins execution of the program.
The interp_block( ) function is the heart of the interpreter. This function
decides what action to take based upon the next token in the input stream. It
is designed to interpret one block of code and then return. The interp_block(
) function is shown in lines 119 through 174.
Ignoring calls to functions like exit( ), a program ends when the last curly
brace (or a return) in main( ) is encountered. It does not necessarily end
with the last line of source code. This is one reason that interp_block( )
executes only a block of code, and not the entire program. Another reason is
that, conceptually, C consists of blocks of code. Therefore, interp_block( )
is called each time a new block of code is encountered. This includes both
function calls and blocks begun by various C statements, such as an ifblock.
This means that the interpreter may call interp_block( ) recursively in the
process of executing a program.
When the interpreter encounters an int or char keyword, it calls decl_local( )
to create storage for a local variable. Each time a local variable is
encountered, its name, type, and value (initially zero) are pushed onto the
stack using local_push( ). The global variable lvartos indexes the stack.
(There is no corresponding "pop" function. Instead, the local variable stack
is reset each time a function returns.) The decl_local and local_push( )
functions are shown in lines 254 through 353.
All function calls (except the initial call to main( )) take place through the
expression parser from the atom( ) function by a call to call( ). It is the
call( ) function that actually handles the details of calling a function. The
call( ) function is shown in Listing Two, lines 269 through 337, along with
two support functions.
Let's return to the expression parser for a moment. When an assignment
statement is encountered, the value of the right side of the expression is
computed and assigned to the variable on the left using a call to assign( ).
Given a program like the one in Example 3, how does the assign( ) function
know which variable is being assigned a value in each assignment? The answer
is simple: First, in C, local variables take priority over global variables of
the same name. Second, local variables are not known outside their own
function. To see how we can use these rules to resolve the above assignments,
examine the assign( ) function shown in lines 369 through 388.
Example 3: Assigning values to a variable

 int count;
 main ()
 {
 int count;
 count = 100;
 f();
 }
 f()
 {
 int count;
 count = 99;
 }



Executing Statements and Loops


Now that the basic structure of the C interpreter is in place, it is time to
add some control statements. Each time a keyword statement is encountered
inside of interp_block( ), an appropriate function is called, which processes
that statement. One of the easiest is the if. The if statement is processed by
exec_if( ), shown in lines 419 through 438.
Like the if, it is easy to interpret a while loop. The function that actually
performs this task is called exec_while( ) and is shown in lines 439 through
454.
A do/while loop is processed much like the while. When interp_block( )
encounters a do statement, it calls exec_do( ), shown in lines 455 through
469.
The main difference between the do/ while and the while loops is that the
do/while always executes its block of code at least once because the
conditional expression is at the bottom of the loop. Therefore, exec_do( )
first saves the location of the top of the loop into temp and then calls
interp_block( ), recursively, to interpret the block of code associated with
the loop. When interp_block( ) returns, the corresponding while is retrieved
and the conditional expression is evaluated. If the condition is true, prog is
reset to the top of the loop; otherwise, execution continues.
The interpretation of the for loop poses a more difficult challenge than the
other constructs. This is partly because the structure of the C for is
definitely designed with compilation in mind. The main trouble is that the
conditional expression of the for must be checked at the top of the loop, but
the increment portion occurs at the bottom of the loop. Therefore, even though
these two pieces of the for loop occur next to each other in the source code,
their interpretation is separated by the block of code being iterated. With a
little work, however, the for can be correctly interpreted.
When interp_block( ) encounters a for statement, exec_for( ) is called. This
function is shown in lines 482 through 514.


Library Functions


Because the C programs executed by the interpreter are never compiled and
linked, any library routines they use must be handled directly by the
interpreter. The best way to do this is to create an interface function, which
the C interpreter calls when a library function is encountered. This interface
function sets up the call to the library function and handles any return
values.
Due to space limitations, the interpreter contains only five library
functions: getche( ), putch( ), puts( ), print( ), and getnum( ). Of these,
only puts( ) is described by the ANSI standard, and it outputs a string to the
screen. The putch( ) function is a common extension to C for interactive
environments. It waits for and returns a key struck at the keyboard. Unlike
most implementations of putchar( ) it does not line-buffer input. This
function is found in many compilers, such as Turbo C, QuickC, and Lattice C.
The putch( ) is also defined by many compilers designed for use in an
interactive environment. It outputs a single character argument to the console
and does not buffer output.
The functions getnum( ) and print( ) are my own creations. The getnum( )
function returns the integer equivalent of a number entered at the keyboard.
The print( ) function is a handy function that can output either a string or
an integer argument to the screen. It is an example of a function that would
be difficult to implement in a compiled environment, but is easy to create for
an interpreted one. The five library functions are shown in Table 3 in their
prototype forms. The C interpreter library routines are listed in Listing
Three (LCLIB.C).
Table 3: The five library functions


 int getche(void); /* read a character from keyboard and return
its value */

 int putch(char ch); /* write a character to the screen */

 int puts(char*s); /* write a string to the screen */

 int getnum(void); /* read an integer from the keyboard and return
its value */
 int print(char*s); /* write a string to the screen */

 or

 int print(int i); /* write an integer to the screen */

To add additional library functions, first enter their names and addresses of
their interface functions into the internal func( ) array. Next, following the
lead of the functions shown here, create appropriate interface functions.


Compiling and Linking the C Interpreter


Once you have compiled all three files that make up this interpreter, compile
and link them together. If you use Turbo C, you can use a sequence like this:
 tcc -c parser.c tcc -c lclib.c tcc littlec.c parser.obj lclib.obj
If you use Microsoft C, use this sequence:
 cl -c parser.c cl -c lclib.c cl littlec.c parser.obj lclib.obj
If you use a different C compiler, follow the instructions that come with it.
The program in Listing Four demonstrates the various features of my C.


Improving and Expanding the C Interpreter


The C interpreter presented here was designed with transparency of operation
in mind. The goal was to develop an interpreter that could be easily
understood with the least amount of effort, and to design it so that it could
be easily expanded. As such, the interpreter is not particularly fast or
efficient. The basic structure of the interpreter is correct, however, and you
can increase its speed of execution in the following ways.
One potential improvement is with the lookup routines for variables and
functions. Even if you convert these items into integer tokens, the current
approach to searching for them relies upon a sequential search. You could,
however, substitute some other, faster method, such as a binary tree or some
sort of hashing method.
Two general areas in which you can expand and enhance the C interpreter are C
features and ancillary features. Among the C statements you can add to the
interpreter are additional action statements, such as the switch, the goto,
and the break and continue statements.
Another type of C statement you can add is new data types. The interpreter
already contains the basic hooks for additional data types. For example, the
var_type structure already contains a field for the type of variable. To add
other elementary types (that is, float, double, long), increase the size of
the value field to the size of the largest element you wish to hold.
Supporting pointers is no harder than supporting any other data type; however,
you will need to add support for the pointer operators to the expression
parser. This will involve some lookahead. Once you have implemented pointers,
arrays will be easy. The space of an array should be allocated dynamically
using malloc( ), and a pointer to the array should be stored in the value
field of var_type.
The addition of structures and unions poses a more difficult problem. The
easiest way to handle them is to use malloc( ) to allocate space for them, and
to use a pointer to the object in the value field of the var_type structure.
(You will also need special code to handle the passing of structures and
unions as parameters.) To handle different return types for functions, add a
type field to the func_type structure, which defines what type of data a
function returns.


Availability


All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063, or call 800-356-2002 (from inside
Calif.) or 800-533-4372 (from outside Calif.). Please specify the issue number
and format (MS-DOS, Macintosh, Kaypro).


BUILDING YOUR OWN C INTERPRETER
by Herbert Schildt



[LISTING ONE]

/* Recursive descent parser for integer expressions
 which may include variables and function calls. */
#include "setjmp.h"
#include "math.h"
#include "ctype.h"

#include "stdlib.h"
#include "string.h"
#include "stdio.h"

#define NUM_FUNC 100
#define NUM_GLOBAL_VARS 100
#define NUM_LOCAL_VARS 200
#define ID_LEN 31
#define FUNC_CALLS 31
#define PROG_SIZE 10000
#define FOR_NEST 31

enum tok_types {DELIMITER, IDENTIFIER, NUMBER, KEYWORD, TEMP,
 STRING, BLOCK};
enum tokens {ARG, CHAR, INT, IF, ELSE, FOR, DO, WHILE, SWITCH,
 RETURN, EOL, FINISHED, END};
enum double_ops {LT=1, LE, GT, GE, EQ, NE};
/* These are the constants used to call sntx_err() when
 a syntax error occurs. Add more if you like.
 NOTE: SYNTAX is a generic error message used when
 nothing else seems appropriate.
*/
enum error_msg
 {SYNTAX, UNBAL_PARENS, NO_EXP, EQUALS_EXPECTED,
 NOT_VAR, PARAM_ERR, SEMI_EXPECTED,
 UNBAL_BRACES, FUNC_UNDEF, TYPE_EXPECTED,
 NEST_FUNC, RET_NOCALL, PAREN_EXPECTED,
 WHILE_EXPECTED, QUOTE_EXPECTED, NOT_TEMP,
 TOO_MANY_LVARS};
extern char *prog; /* current location in source code */
extern char *p_buf; /* points to start of program buffer */
extern jmp_buf e_buf; /* hold environment for longjmp() */
/* An array of these structures will hold the info
 associated with global variables.
*/
extern struct var_type {
 char var_name[32];
 enum variable_type var_type;
 int value;
} global_vars[NUM_GLOBAL_VARS];
/* This is the function call stack. */
extern struct func_type {
 char func_name[32];
 char *loc; /* location of function entry point in file */
} func_stack[NUM_FUNC];
/* Keyword table */
extern struct commands {
 char command[20];
 char tok;
} table[];
/* "Standard library" functions are declared here so
 they can be put into the internal function table that
 follows.
*/
int call_getche(void), call_putch(void);
int call_puts(void), print(void), getnum(void);

struct intern_func_type {
 char *f_name; /* function name */

 int (* p)(); /* pointer to the function */
} intern_func[] = {
 "getche", call_getche,
 "putch", call_putch,
 "puts", call_puts,
 "print", print,
 "getnum", getnum,
 "", 0 /* null terminate the list */
};
extern char token[80]; /* string representation of token */
extern char token_type; /* contains type of token */
extern char tok; /* internal representation of token */
extern int ret_value; /* function return value */
void eval_exp(int *value), eval_exp1(int *value);
void eval_exp2(int *value);
void eval_exp3(int *value), eval_exp4(int *value);
void eval_exp5(int *value), atom(int *value);
void eval_exp0(int *value);
void sntx_err(int error), putback(void);
void assign_var(char *var_name, int value);
int isdelim(char c), look_up(char *s), iswhite(char c);
int find_var(char *s), get_token(void);
int internal_func(char *s);
int is_var(char *s);
char *find_func(char *name);
void call(void);
/* Entry point into parser. */
void eval_exp(int *value)
{
 get_token();
 if(!*token) {
 sntx_err(NO_EXP);
 return;
 }
 if(*token==';') {
 *value = 0; /* empty expression */
 return;
 }
 eval_exp0(value);
 putback(); /* return last token read to input stream */
}
/* Process an assignment expression */
void eval_exp0(int *value)
{
 char temp[ID_LEN]; /* holds name of var receiving
 the assignment */
 register int temp_tok;
 if(token_type==IDENTIFIER) {
 if(is_var(token)) { /* if a var, see if assignment */
 strcpy(temp, token);
 temp_tok = token_type;
 get_token();
 if(*token=='=') { /* is an assignment */
 get_token();
 eval_exp0(value); /* get value to assign */
 assign_var(temp, *value); /* assign the value */
 return;
 }
 else { /* not an assignment */

 putback(); /* restore original token */
 strcpy(token, temp);
 token_type = temp_tok;
 }
 }
 }
 eval_exp1(value);
}
/* This array is used by eval_exp1(). Because
 some compilers cannot initialize an array within a
 function it is defined as a global variable.
*/
char relops[7] = {
 LT, LE, GT, GE, EQ, NE, 0
};
/* Process relational operators. */
void eval_exp1(int *value)
{
 int partial_value;
 register char op;
 eval_exp2(value);
 op = *token;
 if(strchr(relops, op)) {
 get_token();
 eval_exp2(&partial_value);
 switch(op) { /* perform the relational operation */
 case LT:
 *value = *value < partial_value;
 break;
 case LE:
 *value = *value <= partial_value;
 break;
 case GT:
 *value = *value > partial_value;
 break;
 case GE:
 *value = *value >= partial_value;
 break;
 case EQ:
 *value = *value == partial_value;
 break;
 case NE:
 *value = *value != partial_value;
 break;
 }
 }
}
/* Add or subtract two terms. */
void eval_exp2(int *value)
{
 register char op;
 int partial_value;
 eval_exp3(value);
 while((op = *token) == '+' op == '-') {
 get_token();
 eval_exp3(&partial_value);
 switch(op) { /* add or subtract */
 case '-':
 *value = *value - partial_value;

 break;
 case '+':
 *value = *value + partial_value;
 break;
 }
 }
}
/* Multiply or divide two factors. */
void eval_exp3(int *value)
{
 register char op;
 int partial_value, t;
 eval_exp4(value);
 while((op = *token) == '*' op == '/' op == '%') {
 get_token();
 eval_exp4(&partial_value);
 switch(op) { /* mul, div, or modulus */
 case '*':
 *value = *value * partial_value;
 break;
 case '/':
 *value = (*value) / partial_value;
 break;
 case '%':
 t = (*value) / partial_value;
 *value = *value-(t * partial_value);
 break;
 }
 }
}
/* Is a unary + or -. */
void eval_exp4(int *value)
{
 register char op;
 op = '\0';
 if(*token=='+' *token=='-') {
 op = *token;
 get_token();
 }
 eval_exp5(value);
 if(op)
 if(op=='-') *value = -(*value);
}
/* Process parenthesized expression. */
void eval_exp5(int *value)
{
 if((*token == '(')) {
 get_token();
 eval_exp0(value); /* get subexpression */
 if(*token != ')') sntx_err(PAREN_EXPECTED);
 get_token();
 }
 else
 atom(value);
}
/* Find value of number, variable or function. */
void atom(int *value)
{
 int i;

 switch(token_type) {
 case IDENTIFIER:
 i = internal_func(token);
 if(i!= -1) { /* call "standard library" function */
 *value = (*intern_func[i].p)();
 }
 else
 if(find_func(token)){ /* call user-defined function */
 call();
 *value = ret_value;
 }
 else *value = find_var(token); /* get var's value */
 get_token();
 return;
 case NUMBER: /* is numeric constant */
 *value = atoi(token);
 get_token();
 return;
 case DELIMITER: /* see if character constant */
 if(*token=='\'') {
 *value = *prog;
 prog++;
 if(*prog!='\'') sntx_err(QUOTE_EXPECTED);
 prog++;
 get_token();
 }
 return;
 default:
 if(*token==')') return; /* process empty expression */
 else sntx_err(SYNTAX); /* syntax error */
 }
}
/* Display an error message. */
void sntx_err(int error)
{
 char *p, *temp;
 int linecount = 0;
 register int i;
 static char *e[]= {
 "syntax error",
 "unbalanced parentheses",
 "no expression present",
 "equals sign expected",
 "not a variable",
 "parameter error",
 "semicolon expected",
 "unbalanced braces",
 "function undefined",
 "type specifier expected",
 "too many nested function calls",
 "return without call",
 "parentheses expected",
 "while expected",
 "closing quote expected",
 "not a string",
 "too many local variables"
 };
 printf("%s", e[error]);
 p = p_buf;

 while(p != prog) { /* find line number of error */
 p++;
 if(*p == '\r') {
 linecount++;
 }
 }
 printf(" in line %d\n", linecount);
 temp = p;
 for(i=0; i<20 && p>p_buf && *p!='\n'; i++, p--);
 for(i=0; i<30 && p<=temp; i++, p++) printf("%c", *p);
 longjmp(e_buf, 1); /* return to save point */
}
/* Get a token. */
get_token(void)
{
 register char *temp;
 token_type = 0; tok = 0;
 temp = token;
 *temp = '\0';
 /* skip over white space */
 while(iswhite(*prog) && *prog) ++prog;
 if(*prog=='\r') {
 ++prog;
 ++prog;
 /* skip over white space */
 while(iswhite(*prog) && *prog) ++prog;
 }
 if(*prog=='\0') { /* end of file */
 *token = '\0';
 tok = FINISHED;
 return(token_type=DELIMITER);
 }
 if(strchr("{}", *prog)) { /* block delimiters */
 *temp = *prog;
 temp++;
 *temp = '\0';
 prog++;
 return (token_type = BLOCK);
 }
 /* look for comments */
 if(*prog=='/')
 if(*(prog+1)=='*') { /* is a comment */
 prog += 2;
 do { /* find end of comment */
 while(*prog!='*') prog++;
 prog++;
 } while (*prog!='/');
 prog++;
 }
 if(strchr("!<>=", *prog)) { /* is or might be
 a relation operator */
 switch(*prog) {
 case '=': if(*(prog+1)=='=') {
 prog++; prog++;
 *temp = EQ;
 temp++; *temp = EQ; temp++;
 *temp = '\0';
 }
 break;

 case '!': if(*(prog+1)=='=') {
 prog++; prog++;
 *temp = NE;
 temp++; *temp = NE; temp++;
 *temp = '\0';
 }
 break;
 case '<': if(*(prog+1)=='=') {
 prog++; prog++;
 *temp = LE; temp++; *temp = LE;
 }
 else {
 prog++;
 *temp = LT;
 }
 temp++;
 *temp = '\0';
 break;
 case '>': if(*(prog+1)=='=') {
 prog++; prog++;
 *temp = GE; temp++; *temp = GE;
 }
 else {
 prog++;
 *temp = GT;
 }
 temp++;
 *temp = '\0';
 break;
 }
 if(*token) return(token_type = DELIMITER);
 }
 if(strchr("+-*^/%=;(),'", *prog)){ /* delimiter */
 *temp = *prog;
 prog++; /* advance to next position */
 temp++;
 *temp = '\0';
 return (token_type=DELIMITER);
 }
 if(*prog=='"') { /* quoted string */
 prog++;
 while(*prog!='"'&& *prog!='\r') *temp++ = *prog++;
 if(*prog=='\r') sntx_err(SYNTAX);
 prog++; *temp = '\0';
 return(token_type=STRING);
 }
 if(isdigit(*prog)) { /* number */
 while(!isdelim(*prog)) *temp++ = *prog++;
 *temp = '\0';
 return(token_type = NUMBER);
 }
 if(isalpha(*prog)) { /* var or command */
 while(!isdelim(*prog)) *temp++ = *prog++;
 token_type=TEMP;
 }
 *temp = '\0';
 /* see if a string is a command or a variable */
 if(token_type==TEMP) {
 tok = look_up(token); /* convert to internal rep */

 if(tok) token_type = KEYWORD; /* is a keyword */
 else token_type = IDENTIFIER;
 }
 return token_type;
}
/* Return a token to input stream. */
void putback(void)
{
 char *t;
 t = token;
 for(; *t; t++) prog--;
}
/* Look up a token's internal representation in the
 token table.
*/
look_up(char *s)
{
 register int i;
 char *p;
 /* convert to lowercase */
 p = s;
 while(*p){ *p = tolower(*p); p++; }
 /* see if token is in table */
 for(i=0; *table[i].command; i++)
 if(!strcmp(table[i].command, s)) return table[i].tok;
 return 0; /* unknown command */
}
/* Return index of internal library function or -1 if
 not found.
*/
internal_func(char *s)
{
 int i;
 for(i=0; intern_func[i].f_name[0]; i++) {
 if(!strcmp(intern_func[i].f_name, s)) return i;
 }
 return -1;
}
/* Return true if c is a delimiter. */
isdelim(char c)
{
 if(strchr(" !;,+-<>'/*%^=()", c) c==9 
 c=='\r' c==0) return 1;
 return 0;
}
/* Return 1 if c is space or tab. */
iswhite(char c)
{
 if(c==' ' c=='\t') return 1;
 else return 0;
}






[LISTING TWO]


/* A Little C interpreter. */

#include "stdio.h"
#include "setjmp.h"
#include "math.h"
#include "ctype.h"
#include "stdlib.h"
#include "string.h"

#define NUM_FUNC 100
#define NUM_GLOBAL_VARS 100
#define NUM_LOCAL_VARS 200
#define NUM_BLOCK 100
#define ID_LEN 31
#define FUNC_CALLS 31
#define NUM_PARAMS 31
#define PROG_SIZE 10000
#define LOOP_NEST 31

enum tok_types {DELIMITER, IDENTIFIER, NUMBER, KEYWORD,
 TEMP, STRING, BLOCK};
/* add additional C keyword tokens here */
enum tokens {ARG, CHAR, INT, IF, ELSE, FOR, DO, WHILE,
 SWITCH, RETURN, EOL, FINISHED, END};
/* add additional double operators here (such as ->) */
enum double_ops {LT=1, LE, GT, GE, EQ, NE};
/* These are the constants used to call sntx_err() when
 a syntax error occurs. Add more if you like.
 NOTE: SYNTAX is a generic error message used when
 nothing else seems appropriate.
*/
enum error_msg
 {SYNTAX, UNBAL_PARENS, NO_EXP, EQUALS_EXPECTED,
 NOT_VAR, PARAM_ERR, SEMI_EXPECTED,
 UNBAL_BRACES, FUNC_UNDEF, TYPE_EXPECTED,
 NEST_FUNC, RET_NOCALL, PAREN_EXPECTED,
 WHILE_EXPECTED, QUOTE_EXPECTED, NOT_TEMP,
 TOO_MANY_LVARS};
char *prog; /* current location in source code */
char *p_buf; /* points to start of program buffer */
jmp_buf e_buf; /* hold environment for longjmp() */

/* An array of these structures will hold the info
 associated with global variables.
*/
struct var_type {
 char var_name[ID_LEN];
 int var_type;
 int value;
} global_vars[NUM_GLOBAL_VARS];
struct var_type local_var_stack[NUM_LOCAL_VARS];
struct func_type {
 char func_name[ID_LEN];
 char *loc; /* location of entry point in file */
} func_table[NUM_FUNC];
int call_stack[NUM_FUNC];
struct commands { /* keyword lookup table */
 char command[20];
 char tok;

} table[] = { /* Commands must be entered lowercase */
 "if", IF, /* in this table. */
 "else", ELSE,
 "for", FOR,
 "do", DO,
 "while", WHILE,
 "char", CHAR,
 "int", INT,
 "return", RETURN,
 "end", END,
 "", END /* mark end of table */
};
char token[80];
char token_type, tok;
int functos; /* index to top of function call stack */
int func_index; /* index into function table */
int gvar_index; /* index into global variable table */
int lvartos; /* index into local variable stack */
int ret_value; /* function return value */
void print(void), prescan(void);
void decl_global(void), call(void), putback(void);
void decl_local(void), local_push(struct var_type i);
void eval_exp(int *value), sntx_err(int error);
void exec_if(void), find_eob(void), exec_for(void);
void get_params(void), get_args(void);
void exec_while(void), func_push(int i), exec_do(void);
void assign_var(char *var_name, int value);
int load_program(char *p, char *fname), find_var(char *s);
void interp_block(void), func_ret(void);
int func_pop(void), is_var(char *s), get_token(void);
char *find_func(char *name);

main(int argc, char *argv[])
{
 if(argc!=2) {
 printf("usage: c <filename>\n");
 exit(1);
 }
 /* allocate memory for the program */
 if((p_buf=(char *) malloc(PROG_SIZE))==NULL) {
 printf("allocation failure");
 exit(1);
 }
 /* load the program to execute */
 if(!load_program(p_buf, argv[1])) exit(1);
 if(setjmp(e_buf)) exit(1); /* initialize long jump buffer */
 /* set program pointer to start of program buffer */
 prog = p_buf;
 prescan(); /* find the location of all functions
 and global variables in the program */
 gvar_index = 0; /* initialize global variable index */
 lvartos = 0; /* initialize local variable stack index */
 functos = 0; /* initialize the CALL stack index */
 /* setup call to main() */
 prog = find_func("main"); /* find program starting point */
 prog--; /* back up to opening ( */
 strcpy(token, "main");
 call(); /* call main() to start interpreting */
}

/* Interpret a single statement or block of code. When
 interp_block() returns from it's initial call, the final
 brace (or a return) in main() has been encountered.
*/
void interp_block(void)
{
 int value;
 char block = 0;
 do {
 token_type = get_token();
 /* If interpreting single statement, return on
 first semicolon.
 */
 /* see what kind of token is up */
 if(token_type==IDENTIFIER) {
 /* Not a keyword, so process expression. */
 putback(); /* restore token to input stream for
 further processing by eval_exp() */
 eval_exp(&value); /* process the expression */
 if(*token!=';') sntx_err(SEMI_EXPECTED);
 }
 else if(token_type==BLOCK) { /* if block delimiter */
 if(*token=='{') /* is a block */
 block = 1; /* interpreting block, not statement */
 else return; /* is a }, so return */
 }
 else /* is keyword */
 switch(tok) {
 case CHAR:
 case INT: /* declare local variables */
 putback();
 decl_local();
 break;
 case RETURN: /* return from function call */
 func_ret();
 return;
 case IF: /* process an if statement */
 exec_if();
 break;
 case ELSE: /* process an else statement */
 find_eob(); /* find end of else block
 and continue execution */
 break;
 case WHILE: /* process a while loop */
 exec_while();
 break;
 case DO: /* process a do-while loop */
 exec_do();
 break;
 case FOR: exec_for();
 break;
 case END:
 exit(0);
 }
 } while (tok != FINISHED && block);
}
/* Load a program. */
load_program(char *p, char *fname)
{

 FILE *fp;
 int i=0;
 if((fp=fopen(fname, "rb"))==NULL) return 0;
 i = 0;
 do {
 *p = getc(fp);
 p++; i++;
 } while(!feof(fp) && i<PROG_SIZE);
 *(p-2) = '\0'; /* null terminate the program */
 fclose(fp);
 return 1;
}
/* Find the location of all functions in the program
 and store global variables. */
void prescan(void)
{
 char *p;
 char temp[32];
 int brace = 0; /* When 0, this var tells us that
 current source position is outside
 of any function. */
 p = prog;
 func_index = 0;
 do {
 while(brace) { /* bypass code inside functions */
 get_token();
 if(*token=='{') brace++;
 if(*token=='}') brace--;
 }
 get_token();
 if(tok==CHAR tok==INT) { /* is global var */
 putback();
 decl_global();
 }
 else if(token_type==IDENTIFIER) {
 strcpy(temp, token);
 get_token();
 if(*token=='(') { /* must be assume a function */
 func_table[func_index].loc = prog;
 strcpy(func_table[func_index].func_name, temp);
 func_index++;
 while(*prog!=')') prog++;
 prog++;
 /* prog points to opening curly brace of function */
 }
 else putback();
 }
 else if(*token=='{') brace++;
 } while(tok!=FINISHED);
 prog = p;
}
/* Return the entry point of the specified function.
 Return NULL if not found.
*/
char *find_func(char *name)
{
 register int i;
 for(i=0; i<func_index; i++)
 if(!strcmp(name, func_table[i].func_name))

 return func_table[i].loc;
 return NULL;
}
/* Declare a global variable. */
void decl_global(void)
{
 get_token(); /* get type */
 global_vars[gvar_index].var_type = tok;
 global_vars[gvar_index].value = 0; /* init to 0 */
 do { /* process comma-separated list */
 get_token(); /* get name */
 strcpy(global_vars[gvar_index].var_name, token);
 get_token();
 gvar_index++;
 } while(*token==',');
 if(*token!=';') sntx_err(SEMI_EXPECTED);
}
/* Declare a local variable. */
void decl_local(void)
{
 struct var_type i;
 get_token(); /* get type */
 i.var_type = tok;
 i.value = 0; /* init to 0 */
 do { /* process comma-separated list */
 get_token(); /* get var name */
 strcpy(i.var_name, token);
 local_push(i);
 get_token();
 } while(*token==',');
 if(*token!=';') sntx_err(SEMI_EXPECTED);
}
/* Call a function. */
void call(void)
{
 char *loc, *temp;
 int lvartemp;
 loc = find_func(token); /* find entry point of function */
 if(loc==NULL)
 sntx_err(FUNC_UNDEF); /* function not defined */
 else {
 lvartemp = lvartos; /* save local var stack index */
 get_args(); /* get function arguments */
 temp = prog; /* save return location */
 func_push(lvartemp); /* save local var stack index */
 prog = loc; /* reset prog to start of function */
 get_params(); /* load the function's parameters with
 the values of the arguments */
 interp_block(); /* interpret the function */
 prog = temp; /* reset the program pointer */
 lvartos = func_pop(); /* reset the local var stack */
 }
}
/* Push the arguments to a function onto the local
 variable stack. */
void get_args(void)
{
 int value, count, temp[NUM_PARAMS];
 struct var_type i;

 count = 0;
 get_token();
 if(*token!='(') sntx_err(PAREN_EXPECTED);
 /* process a comma-separated list of values */
 do {
 eval_exp(&value);
 temp[count] = value; /* save temporarily */
 get_token();
 count++;
 }while(*token==',');
 count--;
 /* now, push on local_var_stack in reverse order */
 for(; count>=0; count--) {
 i.value = temp[count];
 i.var_type = ARG;
 local_push(i);
 }
}
/* Get function parameters. */
void get_params(void)
{
 struct var_type *p;
 int i;
 i = lvartos-1;
 do { /* process comma-separated list of parameters */
 get_token();
 p = &local_var_stack[i];
 if(*token!=')') {
 if(tok!=INT && tok!=CHAR) sntx_err(TYPE_EXPECTED);
 p->var_type = token_type;
 get_token();
 /* link parameter name with argument already on
 local var stack */
 strcpy(p->var_name, token);
 get_token();
 i--;
 }
 else break;
 } while(*token==',');
 if(*token!=')') sntx_err(PAREN_EXPECTED);
}
/* Return from a function. */
void func_ret(void)
{
 int value;
 value = 0;
 /* get return value, if any */
 eval_exp(&value);
 ret_value = value;
}
/* Push local variable */
void local_push(struct var_type i)
{
 if(lvartos>NUM_LOCAL_VARS)
 sntx_err(TOO_MANY_LVARS);
 local_var_stack[lvartos] = i;
 lvartos++;
}
/* Pop index into local variable stack. */

func_pop(void)
{
 functos--;
 if(functos<0) sntx_err(RET_NOCALL);
 return(call_stack[functos]);
}
/* Push index of local variable stack. */
void func_push(int i)
{
 if(functos>NUM_FUNC)
 sntx_err(NEST_FUNC);
 call_stack[functos] = i;
 functos++;
}
/* Assign a value to a variable. */
void assign_var(char *var_name, int value)
{
 register int i;
 /* first, see if it's a local variable */
 for(i=lvartos-1; i>=call_stack[functos-1]; i--) {
 if(!strcmp(local_var_stack[i].var_name, var_name)) {
 local_var_stack[i].value = value;
 return;
 }
 }
 if(i < call_stack[functos-1])
 /* if not local, try global var table */
 for(i=0; i<NUM_GLOBAL_VARS; i++)
 if(!strcmp(global_vars[i].var_name, var_name)) {
 global_vars[i].value = value;
 return;
 }
 sntx_err(NOT_VAR); /* variable not found */
}
/* Find the value of a variable. */
int find_var(char *s)
{
 register int i;
 /* first, see if it's a local variable */
 for(i=lvartos-1; i>=call_stack[functos-1]; i--)
 if(!strcmp(local_var_stack[i].var_name, token))
 return local_var_stack[i].value;
 /* otherwise, try global vars */
 for(i=0; i<NUM_GLOBAL_VARS; i++)
 if(!strcmp(global_vars[i].var_name, s))
 return global_vars[i].value;
 sntx_err(NOT_VAR); /* variable not found */
}
/* Determine if an identifier is a variable. Return
 1 if variable is found; 0 otherwise.
*/
int is_var(char *s)
{
 register int i;
 /* first, see if it's a local variable */
 for(i=lvartos-1; i>=call_stack[functos-1]; i--)
 if(!strcmp(local_var_stack[i].var_name, token))
 return 1;
 /* otherwise, try global vars */

 for(i=0; i<NUM_GLOBAL_VARS; i++)
 if(!strcmp(global_vars[i].var_name, s))
 return 1;
 return 0;
}
/* Execute an IF statement. */
void exec_if(void)
{
 int cond;
 eval_exp(&cond); /* get left expression */
 if(cond) { /* is true so process target of IF */
 interp_block();
 }
 else { /* otherwise skip around IF block and
 process the ELSE, if present */
 find_eob(); /* find start of next line */
 get_token();
 if(tok!=ELSE) {
 putback(); /* restore token if
 no ELSE is present */
 return;
 }
 interp_block();
 }
}
/* Execute a while loop. */
void exec_while(void)
{
 int cond;
 char *temp;
 putback();
 temp = prog; /* save location of top of while loop */
 get_token();
 eval_exp(&cond); /* check the conditional expression */
 if(cond) interp_block(); /* if true, interpret */
 else { /* otherwise, skip around loop */
 find_eob();
 return;
 }
 prog = temp; /* loop back to top */
}
/*Execute a do loop. */
void exec_do(void)
{
 int cond;
 char *temp;
 putback();
 temp = prog; /* save location of top of do loop */
 get_token(); /* get start of loop */
 interp_block(); /* interpret loop */
 get_token();
 if(tok!=WHILE) sntx_err(WHILE_EXPECTED);
 eval_exp(&cond); /* check the loop condition */
 if(cond) prog = temp; /* if true loop; otherwise,
 continue on */
}
/* Find the end of a block. */
void find_eob(void)
{

 int brace;
 get_token();
 brace = 1;
 do {
 get_token();
 if(*token=='{') brace++;
 else if(*token=='}') brace--;
 } while(brace);
}
/* Execute a while loop. */
void exec_for(void)
{
 int cond;
 char *temp, *temp2;
 int brace ;
 get_token();
 eval_exp(&cond); /*initialization expression */
 if(*token!=';') sntx_err(SEMI_EXPECTED);
 prog++; /* get past the ; */
 temp = prog;
 for(;;) {
 eval_exp(&cond); /* check the condition */
 if(*token!=';') sntx_err(SEMI_EXPECTED);
 prog++; /* get past the ; */
 temp2 = prog;
 /* find the start of the for block */
 brace = 1;
 while(brace) {
 get_token();
 if(*token=='(') brace++;
 if(*token==')') brace--;
 }
 if(cond) interp_block(); /* if true, interpret */
 else { /* otherwise, skip around loop */
 find_eob();
 return;
 }
 prog = temp2;
 eval_exp(&cond); /* do the increment */
 prog = temp; /* loop back to top */
 }
}






[LISTING THREE]

/****** Internal Library Functions *******/

/* Add more of your own, here. */

#include "conio.h" /* if your compiler does not
 support this header file,
 remove it */
#include "stdio.h"
#include "stdlib.h"


extern char *prog; /* points to current location in program */
extern char token[80]; /* holds string representation of token */
extern char token_type; /* contains type of token */
extern char tok; /* holds the internal representation of token */

enum tok_types {DELIMITER, IDENTIFIER, NUMBER, COMMAND, STRING,
 QUOTE, VARIABLE, BLOCK, FUNCTION};
/* These are the constants used to call sntx_err() when
 a syntax error occurs. Add more if you like.
 NOTE: SYNTAX is a generic error message used when
 nothing else seems appropriate.
*/
enum error_msg
 {SYNTAX, UNBAL_PARENS, NO_EXP, EQUALS_EXPECTED,
 NOT_VAR, PARAM_ERR, SEMI_EXPECTED,
 UNBAL_BRACES, FUNC_UNDEF, TYPE_EXPECTED,
 NEST_FUNC, RET_NOCALL, PAREN_EXPECTED,
 WHILE_EXPECTED, QUOTE_EXPECTED, NOT_STRING,
 TOO_MANY_LVARS};
int get_token(void);
void sntx_err(int error), eval_exp(int *result);
void putback(void);
/* Get a character from console. (Use getchar()) if
 your compiler does not support getche().) */
call_getche()
{
 char ch;
 ch = getche();
 while(*prog!=')') prog++;
 prog++; /* advance to end of line */
 return ch;
}
/* Put a character to the display. (Use putchar()
 if your compiler does not support putch().) */
call_putch()
{
 int value;
 eval_exp(&value);
 printf("%c", value);
 return value;
}
/* Call puts(). */
call_puts(void)
{
 get_token();
 if(*token!='(') sntx_err(PAREN_EXPECTED);
 get_token();
 if(token_type!=QUOTE) sntx_err(QUOTE_EXPECTED);
 puts(token);
 get_token();
 if(*token!=')') sntx_err(PAREN_EXPECTED);
 get_token();
 if(*token!=';') sntx_err(SEMI_EXPECTED);
 putback();
 return 0;
}
/* A built-in console output function. */
int print(void)

{
 int i;
 get_token();
 if(*token!='(') sntx_err(PAREN_EXPECTED);
 get_token();
 if(token_type==QUOTE) { /* output a string */
 printf("%s ", token);
 }
 else { /* output a number */
 putback();
 eval_exp(&i);
 printf("%d ", i);
 }
 get_token();
 if(*token!=')') sntx_err(PAREN_EXPECTED);
 get_token();
 if(*token!=';') sntx_err(SEMI_EXPECTED);
 putback();
 return 0;
}
/* Read an integer from the keyboard. */
getnum(void)
{
 char s[80];
 gets(s);
 while(*prog!=')') prog++;
 prog++; /* advance to end of line */
 return atoi(s);
}







[LISTING FOUR]

/* C Interpreter Demonstration Program
 This program demonstrates all features
 of C that are recognized by this C interpreter.
*/
int i, j; /* global vars */
char ch;

main()
{
 int i, j; /* local vars */
 puts("C Demo Program.");
 print_alpha();
 do {
 puts("enter a number (0 to quit): ");
 i = getnum();
 if(i < 0 ) {
 puts("numbers must be positive, try again");
 }
 else {
 for(j = 0; j < i; j=j+1) {
 print(j);

 print("summed is");
 print(sum(j));
 puts("");
 }
 }
 } while(i!=0);
}
/* Sum the values between 0 and num. */
sum(int num)
{
 int running_sum;
 running_sum = 0;
 while(num) {
 running_sum = running_sum + num;
 num = num - 1;
 }
 return running_sum;
}
/* Print the alphabet. */
print_alpha()
{
 for(ch = 'A'; ch<='Z'; ch = ch + 1) {
 putch(ch);
 }
 puts("");
}
/* Nested loop example. */
main()
{
 int i, j, k;
 for(i = 0; i < 5; i = i + 1) {
 for(j = 0; j < 3; j = j + 1) {
 for(k = 3; k ; k = k - 1) {
 print(i);
 print(j);
 print(k);
 puts("");
 }
 }
 }
 puts("done");
}
/* Assigments as operations. */
main()
{
 int a, b;
 a = b = 10;
 print(a); print(b);
 while(a=a-1) {
 print(a);
 do {
 print(b);
 }while((b=b-1) > -10);
 }
}
/* This program demonstrates recursive functions. */
main()
{
 print(factr(7) * 2);

}
/* return the factorial of i */
factr(int i)
{
 if(i<2) {
 return 1;
 }
 else {
 return i * factr(i-1);
 }
}
/* A more rigorous example of function arguments. */
main()
{
 f2(10, f1(10, 20), 99);
}
f1(int a, int b)
{
 int count;
 print("in f1");
 count = a;
 do {
 print(count);
 } while(count=count-1);
 print(a); print(b);
 print(a*b);
 return a*b;
}
f2(int a, int x, int y)
{
 print(a); print(x);
 print(x / a);
 print(y*x);
}
/* The loop statements. */
main()
{
 int a;
 char ch;
 /* the while */
 puts("Enter a number: ");
 a = getnum();
 while(a) {
 print(a);
 print(a*a);
 puts("");
 a = a - 1;
 }
 /* the do-while */
 puts("enter characters, 'q' to quit");
 do {
 ch = getche();
 } while(ch!='q');
 /* the for */
 for(a=0; a<10; a = a + 1) {
 print(a);
 }
}
































































August, 1989
C MULTIDIMENSIONAL ARRAYS AT RUN TIME


Organizing the heap for run-time multidimensional arrays isn't easy, but it is
possible




Paul Anderson


Paul is a consultant and co-author of Advanced C Tips and Techniques,
published by Howard W. Sams, from which this article was adapted. Paul can be
reached at 1212 Eolus Ave., Leucadia, CA 92024.


When you declare arrays in C, their sizes remain fixed. You have to decide
before your program runs how large an array will be, regardless of whether you
declare it as automatic or static. What about declaring arrays at run time? In
this case, you need to call the C library routine malloc( ) or calloc( ) to
build dynamic arrays from memory (called the heap). This approach eliminates
anticipating array bounds at compile time and makes programs allocate only as
much memory as they need.
One-dimensional arrays are easy because a call to malloc( ) or calloc( )
returns a pointer to a chunk of heap memory that you may use as a
one-dimensional array. You cast the heap pointer to an appropriate data type
and use either array notation or pointers to reference the allocated elements.
Multidimensional arrays, however, are more difficult. There's no standard C
library routine that sets up the heap so that you may use it as a
multidimensional array. To the heap manager, heap storage is merely a block of
consecutive bytes with no notion of rows, columns, and so on. You need a way
to organize the heap for run-time multidimensional arrays. Moreover, it would
be nice to retain the concepts of rows and columns so that heap memory appears
like a multidimensional array to your programs.
In this article, I'll discuss C functions that create run-time two-dimensional
and three-dimensional arrays of any data type (including arrays of structures
and unions). I'll also discuss a technique to mimic the subroutine calling
conventions of Basic and Fortran with two-dimensional arrays. Along the way,
I'll review two-dimensional and three-dimensional arrays and the relationships
between pointer expressions and array references. This should give you the
information you need to implement these concepts in your own C programs.


The Basic Rule


Let's start with how C views arrays. For a one-dimensional array of any data
type, the following equivalence exists between an array reference and a
pointer expression:
 a[i] = (*(a + i))
I call this the basic rule because it applies to many complicated pointer
expressions, as you will see later on. Note that you may omit the outer
parentheses most of the time except for expressions where C's precedence rules
require them. The basic rule helps explain simple relations such as &a[i] = a
+i, where I apply C's address operator (&) to both sides of the relation and
use C's precedence rules to simplify the result. Using the same method, the
basic rule helps derive the relations a[0] = *a and &a[0] = a.
One of the surprising things about C arrays is that array references don't
really exist. If you don't believe this, try running the portable program in
Example 1 , which compiles without error. You'll discover the program displays
5 (the sixth element of the array) four times. Despite the fact that the last
two array references must be the ultimate in job security, C translates the
array references to pointer expressions according to the basic rule. Although
no one (hopefully) would use expressions such as these in programs, they're
conclusive proof that C handles arrays differently from the way other
languages do.
Example 1: A portable program that compiles without error

 #include <stdio.h>
 main ()
 {
 static char a[] = "0123456789";
 int i = 5;

 printf ("%c %c %c %c\n", a[i], a[5], i[a], 5[a]);
 }

Now let's move on to multidimensional arrays. Two-dimensional arrays are easy
because you can visualize them as grids with rows and columns. (By the way,
I'll use the term grid for a two-dimensional array from now on. Think of it as
a checkerboard in which rows and columns locate unique elements.) In C, the
declaration double mint[3][5]allocates storage for 15 doubles, arranged as 3
rows by 5 columns. Figure 1 shows how to visualize the array mint. Array
references have the format mint[row][col]. Once again, C allows you to use
expressions with pointer indirection and array reference notation that are not
immediately obvious. mint[1], for instance, is a pointer to a double located
in the first column of the second row. Likewise, *mint is a pointer to a
double in the first row and first column. The name of the array (mint) is a
pointer to an array of five doubles, located in the first row. The expression
mint + 1, therefore, is a pointer to the array of five doubles in the second
row.
The basic rule comes in handy for deciphering two-dimensional array
references. The notation mint[1][2], for example, is equivalent to *(mint[1] +
2), according to the following derivation.
Let:
 p = mint[1]
Then:
 mint[1][2] = p[2]
Marking mint[1]as p makes it easier to apply the basic rule:
 p[2] = (*(p + 2))
Now substitute mint[1] for p in both sides of the previous expression:
 mint[1][2] = (*(mint[1] + 2)) mint[1][2] = *(mint[1] + 2)
(Here I drop the outer parentheses because they're unnecessary.)
mint[1][2] is also equivalent to (*(mint + 1))[2], although this is
considerably more obscure. Here are the substitutions with the basic rule:
 mint[1] = (*(mint + 1)) mint[1][2] = (*(mint + 1))[2]
This time, however, you cannot remove the outer parentheses because C's
precedence rules require them ([] has higher precedence than *). Without the
parentheses, the expression *(mint + 1)[2] evaluates to **(mint + 3). (Can you
derive this with the basic rule?) Both expressions are wrong because they
locate an element outside the bounds of the mint array.
Three-dimensional arrays add another level to the same concept. The C
declaration double vision[2][3][5], for example, allocates storage for 30
doubles, arranged as two grids of 3 rows and 5 columns. Think of it as a stack
of two-dimensional arrays on top of each other. Figure 2 shows how to
visualize the array vision. Array references have the format
vision[grid][row][col]. C permits pointer expressions for arrays in three
dimensions as well as it does for two dimensions, but their meanings are
different. vision[1] is now a pointer to an array of five doubles located at
the first row of the second grid. Likewise, vision[1][1] is a pointer to a
double in the second grid, second row, first column. The name of the array
(vision) is a pointer to a 3 by 5 array of doubles (the first grid), and
vision + 1 is a pointer to the second grid. I'll return to these types of
pointer expressions when I apply them later on to run-time arrays.


Storage Map Equations



Viewing multidimensional arrays as grids with rows and columns is for our
benefit. Unfortunately, the compiler has a harder job than we do making the
connection between arrays and grids. Physical memory is accessed as a
one-dimensional array, so the compiler maps multidimensional arrays to blocks
of memory. Each time you reference a multidimensional array element, the
compiler calculates an address in the memory block.
Figure 3 shows the memory layout for the declaration double mint[3][5]. C
stores two-dimensional arrays in memory by rows (this is called row major
form). The second subscript varies faster than the first one. Therefore,
mint[0] points to the first double in the first row, mint[1] points to the
first double in the second row, and so forth. The compiler calculates the
address for an array reference by locating the appropriate row and accessing
the correct column within that row.
When you use the array reference mint[i][j] in a C program, the compiler
implements the following pointer expression: *(&mint[0][0] + 5 * i + j).
Subscript i is the row number, subscript j is the column number, and
&mint[0][0] is the base address of the array. I call this above expression a
storage map equation. Every multidimensional array declaration in a program
has one. This particular storage map equation is for the two-dimensional array
double mint[3][5].
Note that the number of rows in a two-dimensional array declaration is not
used in the storage map equation. This helps explain why the declarations in
Example 2 are legal in C.
Example 2: The number of rows in a two-dimensional array declaration is not
used in the storage map equation

 main ()
 {
 extern double mint [][5]; /* OK */
 . . . mint [i][j] . . .
 f(mint);
 }
 f(a)
 double a [][5]; /* OK */
 {
 . . . a[i][j] . . .
 }

To generate the storage map equation for a[i][j] inside f( ), the compiler
requires the number of columns but not the number of rows. Likewise, the
extern statement provides the compiler with the necessary information to
reference mint[i][j], even though the array is declared in another file.
Storage map equations apply to three-dimensional arrays as well. Figure 4
shows the memory layout for the declaration double vision[2][3][5]. The second
3 by 5 grid follows the first. Memory layout within each grid has the same
organization as in the two-dimensional example.
When you use the array reference vision[i][j][k], the C compiler uses the
following storage map equation: *(&vision[0][0][0] + 15 * i + 5 * j + k).
Subscripts i, j, and k are the grids, rows, and columns, respectively, and
&vision[0][0][0] is the base address of the array. The number 15 is the
product of the number of rows (3) and the number of columns (5). The storage
map equation for a three-dimensional array reference does not use the number
of grids.
Why be concerned with storage map equations, anyway? It turns out that many C
compilers generate multiply instructions in assembly code for multidimensional
array references. Depending on the compiler you use, this may affect
performance of a running C program. Suppose, for example, you declare mint as
follows: double mint[3][4]; The storage map equation for mint[i][j] is now
*(&mint[0][0] + 4 * i + j). Some compilers generate shift instructions in
place of multiplies. Multiplying an integer by a power of 2, for instance, is
the same as shifting the bits left by the value of the power. In this example,
a compiler could shift i left by 2 bits instead of multiplying it by 4. If the
number of columns is not a power of 2 (like our original declaration of
mint[3][5]), some compilers may even generate a set of "shift-and-add"
instructions. In other words, shifting i left by 2 bits and adding i is the
same as multiplying i by 5. In the following sections, I'll create two- and
three-dimensional arrays that don't use storage map equations to access array
elements. In many cases, this improves a program's performance.


Two-Dimensional Arrays at Run Time


Now let's put all this information to work and create a function that
allocates two-dimensional arrays at run time. Suppose you need a 2 by 3 array
of integers in a program. Instead of declaring int a[2][3], let's allocate
memory from the heap with the statements in Example 3.
Example 3: Allocating memory from the heap

 int *p;
 int **a;

 p = (int *) calloc(2 * 3, sizeof(int)); /* pointer to data */
 a = (int **) malloc(2 * sizeof(int *)); /* pointer to rows */

 a[0] = p; /* first row */
 a[1] = p + 3; /* second row */

Figure 5 shows the memory arrangement. There are two pointers to heap storage.
The first pointer (p) points to the array of data (six integers). The second
pointer (a) points to a pointer array containing the addresses of the rows of
the data array. Note that p is a pointer to an int and a is a pointer to a
pointer to an int. I use calloc( ) to zero fill the data array and malloc( )
to allocate heap storage for the row array.
With the pointer array a, you may now use two-dimensional array references in
a program. a[1][2], for instance, refers to the last data item, or the third
column of the second row of the two-dimensional array. From the basic rule,
it's as if you typed *(a[1] + 2). However, a[1] is now a pointer to the first
integer in the second row, and a[1] + 2 is a pointer to the third column in
the same row. The compiler uses pointer indirection in place of a storage map
equation to access array elements.
Listing One contains the C source code for two functions based on this
technique. Function dim2( ) creates two-dimensional arrays of any data type at
run time and function free2( ) frees them. Example 4 shows how to create a 3
by 5 array of integers and a 4 by 6 array of structures. The statements
free2(a) and free2(s) release the heap memory allocated for each array.
Example 4: Creating two-dimensional arrays of integers and structures

 int **a; /* 2D array of integers */
 struct something **s; /* 2D array of structures */

 a = (int **) dim2 (3, 5, sizeof(int));
 s = (struct something **) dim2 (4, 6, sizeof(struct something));

Function dim2( ) allocates heap memory for the row pointer array (pointed to
by prow) and the data array (pointed to by pdata). The forloop connects the
row pointer array to the data array. Function free2( ) makes two calls to
free( ) to release heap memory. The first call frees the data array and the
second call releases the row pointer array. Note that the order for the calls
to free( ) is significant!


Three-Dimensional Arrays at Run Time


Three-dimensional arrays at run time extend the same concepts. Implementing
the compile time declaration of int a[2][3][2], for instance, requires two
pointer arrays in addition to the data array. One pointer array contains
pointers to the rows of heap data and the other one contains pointers to the
grids. Figure 6 shows the memory arrangement. Pointer a points to the grid
pointer array, p2 points to the row pointer array, and p points to the data.
Once you initialize the grid and row pointer arrays, you access the data with
a. Here are the declarations for the three pointers: int ***a, **p2, and *p.
Note that three-dimensional array references with a use pointers with triple
indirection. The array reference a[0][2][1], for example, becomes *(*(*a + 2)
+ 1), according to the basic rule. As with the two-dimensional technique,
three-dimensional array references use pointer indirection in place of a
storage map equation with multiplies or shifts and adds.
Listing Two contains the C source code for two functions based on the
three-dimensional technique. Function dim3( ) creates the three-dimensional
arrays of any data type at run time and free3( ) frees them. Example 5 shows
how to create a 3 by 4 by 5 array of integers and a 4 by 6 by 8 array of
structures. Function calls free3(a) and free3(s) release the heap memory
allocated for each array.

Example 5: Creating three-dimensional arrays of integers and of structures

 int ***a; /* 3D array of integers */
 struct something ***s; /* 3D array of structures */

 a = (int ***) dim3 (3, 4, 5, sizeof(int));
 s = (struct something ***) dim3 (4, 6, 8, sizeof(struct something));

Function dim3( ) allocates heap memory for the grid pointer array (pointed to
by pgrid), the row pointer array (pointed to by prow), and the data array
(pointed to by pdata). Two for loops connect the grid pointer array to the row
pointer array and to the data array. Function free3( ) makes three calls to
free( ) to release heap memory for the data array, the row pointer array, and
the grid pointer array, respectively.


Function Calling Conventions


Languages such as Fortran and Basic allow you to pass different-size
multidimensional arrays as arguments to the same function and use array
notation to reference elements. In Fortran, for example, the statements:
 SUBROUTINE FUNC(A, M, N)
 DIMENSION A(M, N)
 . . .
 END
allow a subroutine called FUNC( ) to access a two-dimensional array called A
with run-time values for M rows and N columns. Programs call FUNC( ) with
different-size arrays. Fortran libraries use subroutines such as FUNC( ) to
invert matrices or calculate mathematical items, such as determinants and
eigenvalues. This convention is useful because subroutines may reference
elements by rows and columns.
C doesn't provide such a built-in feature. Consider what happens, for example,
when you pass the address of a multidimensional array to a function. Inside
the function, the compiler uses a storage map equation with fixed sizes. To
illustrate, suppose you declare the following two-dimensional array of
integers: int data[5][9].
The statements in Example 6 call a C function, func( ), to pass the address of
the two-dimensional array along with the number of rows and columns. The
compiler requires 9 in the declaration for a to properly address elements of
the array with the storage map equation.
Example 6: Passing the address of a two-dimensional array

 func(data, 5, 9);

 Inside func(), you have the following:

 func(a, rows, cols)
 int a[][9];
 int rows, cols;
 {
 register int i, j;

 for (i = 0; i < rows; i++)
 for (j = 0; j < cols; j++)
 . . . a[i][j] . . . /* array references OK */

Suppose you have another two-dimensional array called int moredata [4][10];
You can't use func( ) to access data in this array. The function call
func(more data, 4, 10) won't work because func( ) is compiled with the number
of columns set to 9 and not 10. Attempts to use it make func( ) reference
memory incorrectly.
Another alternative is to pass the address of the first element of the
two-dimensional array. The statements:
 func(&data[0][0], 5, 9);
 func(&moredata[0][0], 4, 10);
make func( ) access the two-dimensional arrays as a one-dimensional array. The
code inside func( ) changes, however. Now you have the code shown in Example
7.
Example 7: The effect of passing the first address of an element of a
two-dimensional array

 func(a, rows, cols)
 int *a;
 int rows, cols;
 {
 register int *p = a, *end = a + row * cols;
 while (p < end) {
 . . . *p++ . . . /* Can't use a[i][j] references */
 }
 }

Here, parameter a is a pointer to an integer (previously, it was a pointer to
an array of integers). This allows you to use compact pointer expressions such
as *p++ to access memory as a series of rows*cols integers. Although this is
relatively fast, the concept of rows and columns disappears. Functions that
compute matrix inversions and determinants need this information, though.
In situations such as this, we'd like to have C behave like Fortran or Basic.
This would help us translate Fortran and Basic subroutines to C more easily.
Let's apply the previous techniques to solve this problem.
Suppose you want to pass the address of different-sized, two-dimensional
arrays to a function that calculates a mathematical quantity called a
determinant. The details of how you calculate determinants do not concern us
here, but this problem serves as a good example of an algorithm that requires
rows and columns for the calculation. Listing Three is a C program that
calculates determinants for two-dimensional arrays of doubles.
Function det( ) calls sdim2( ) to construct pointer arrays to the data using
the two-dimensional technique presented earlier. Function sdim2( ) is similar
to dim2( ), except that it doesn't have to create the data array. Instead, it
allocates heap memory for the row pointer array, connects it to the data array
(pointed to by pdata) and returns the heap memory address. This allows the
det( ) function to access the array elements of the different size arrays
passed to it using array notation with rows and columns. Before det( )
returns, it calls free( ) to release heap memory for the pointer array.



Performance Pointers


What about performance issues? The techniques I've shown you for run-time
multidimensional arrays substitute pointer indirections for storage map
equations. (Remember, storage map equations typically generate integer
multiplies or shifts and adds in assembly code.) I've benchmarked these
programs and others using Microsoft's C compiler (under DOS and Xenix for 286
and 386 machines) and discovered that pointer indirections are no worse in
execution time than integer multiplies (and often execute faster). In the
two-dimensional case, the overhead of allocating heap memory for the row
pointer array isn't too bad, but remember that three-dimensional arrays
require two pointer arrays. This approach uses more heap memory and can take
more time to set up. It's also possible to eliminate function call overhead by
using macros to generate the arrays (the solutions are in the bibliography).
Ultimately, you'll have to judge whether the overhead of setting up run-time
multidimensional arrays is worth the effort for your application.


Bibliography


Anderson, Paul and Anderson, Gail; Advanced C: Tips and Techniques.
Indianapolis, Ind.: Howard W. Sams & Company, 1988.


Acknowledgments


I'd like to thank Gail Anderson, Marty Gray, and Tim Dowty for their
assistance.


Availability


All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063; or call 800-356-2002 (from inside
Calif.) or 800-533-4372 (from outside Calif.). Please specify the issue and
format (MS-DOS, Macintosh, Kaypro).


C MULTIDIMENSIONAL ARRAYS AT RUN TIME
by Paul Anderson


[LISTING ONE]


#include <stdio.h>
#include <malloc.h>

char **dim2(row, col, size) /* creates 2D array */
int row, col;
unsigned size;
{
 int i;
 char **prow, *pdata;

 pdata = (char *) calloc(row * col, size);
 if (pdata == (char *) NULL) {
 fprintf(stderr, "No heap space for data\n");
 exit(1);
 }
 prow = (char **) malloc(row * sizeof (char *));
 if (prow == (char **) NULL) {
 fprintf(stderr, "No heap space for row pointers\n");
 exit(1);
 }

 for (i = 0; i < row; i++) {
 prow[i] = pdata; /* store pointers to rows */
 pdata += size * col; /* move to next row */
 }
 return prow; /* pointer to 2D array */

}

void free2(pa) /* frees 2D heap storage */
char **pa;
{
 free(*pa); /* free the data */
 free(pa); /* free pointer to row pointers */
}






[LISTING TWO]


#include <stdio.h>
#include <malloc.h>

char ***dim3(grid, row, col, size) /* creates 3D array */
int grid, row, col;
unsigned size;
{
 int i;
 char ***pgrid, **prow, *pdata;

 pdata = (char *) calloc(grid * row * col, size);
 if (pdata == (char *) NULL) {
 fprintf(stderr, "No heap space for data\n");
 exit(1);
 }
 prow = (char **) malloc(grid * row * sizeof (char *));
 if (prow == (char **) NULL) {
 fprintf(stderr, "No heap space for row pointers\n");
 exit(1);
 }
 pgrid = (char ***) malloc(grid * sizeof (char **));
 if (pgrid == (char ***) NULL) {
 fprintf(stderr, "No heap space for grid pointers\n");
 exit(1);
 }

 for (i = 0; i < grid * row; i++) {
 prow[i] = pdata; /* store pointers to rows */
 pdata += col * size; /* move to next row */
 }

 for (i = 0; i < grid; i++) {
 pgrid[i] = prow; /* store pointers to grid */
 prow += row; /* move to next grid */
 }
 return pgrid; /* pointer to 3D array */
}

void free3(pa) /* frees 3D heap storage */
char ***pa;
{
 free(**pa); /* free the data */

 free(*pa); /* free the row pointers */
 free(pa); /* free the grid pointers */
}






[LISTING THREE]


/* det.c - find determinant of a two-dimensional array of doubles */

#include <stdio.h>
#include <malloc.h>

main()
{
 double det();

 static double f[4][4] = {
 1, 3, 2, 1,
 4, 6, 1, 2,
 2, 1, 2, 3,
 1, 2, 4, 1
 };
 static double g[5][5] = {
 1, 3, 2, 1, 7,
 4, 6, 1, 2, 6,
 2, 1, 2, 3, 5,
 1, 2, 4, 1, 4,
 8, 5, 4, 1, 3
 };

 printf ("determinant of f = %g\n", det(f, 4));
 printf ("determinant of g = %g\n", det(g, 5));
}

double det(arg, n) /* calculate determinant for n by n matrix */
char *arg;
int n;
{
 register int i, j, k;
 double **a; /* this is the array name */
 char **sdim2();
 double ret; /* determinant */
 double x; /* temp */

 /* dynamically create 2 dimensional "array" a from arg */
 a = (double **) sdim2(arg, n, n, sizeof(double));

 /* determinant algorithm using rows and columns */
 for (k = 0; k < n - 1; k++)
 for (i = k + 1; i < n; i++){
 x = a[i][k]/a[k][k];
 for (j = k; j < n; j++)
 a[i][j] = a[i][j] - x * a[k][j];
 }


 for (ret = 1, i = 0; i < n; i++)
 ret *= a[i][i];

 free(a); /* free heap storage */

 return ret;
}

char **sdim2(pdata, row, col, size) /* "creates" 2D array */
char *pdata;
int row, col;
unsigned size;
{
 int i;
 register char **prow;

 prow = (char **) malloc(row * sizeof (char *));
 if (prow == (char **) NULL) {
 fprintf(stderr, "No heap space for row pointers\n");
 exit(1);
 }

 for (i = 0; i < row; i++) {
 prow[i] = pdata; /* store pointers to rows */
 pdata += size * col; /* move to next row */
 }
 return prow; /* pointer to 2D array */
}

































August, 1989
C DYNAMIC MEMORY USE


Finding and exterminating C's dynamic memory errors




Randall Merilatt


Randall is co-founder and chief scientist at Raima Corporation and can be
contacted at 3245 146th Place SE, Ste. 230, Bellevue, WA 98007.


A gas is defined as a substance capable of expanding to completely fill a
container and take on the shape of the container. When you consider software
and a computer's memory in light of this definition, it is only reasonable to
conclude that software is a gas. The situation has always been that no matter
how large a container you make (that is, how much memory your computer has),
software expands to fill it completely. We've been increasing the size of our
containers for more than 40 years now, and it still seems as though we never
have enough. Because of this, we have had to be quite miserly in our use of
memory just to get the most out of what has always been and, apparently,
always will be a precious commodity.
Memory is used to store a program's executable code and the data that is
manipulated by that code. Overlays are one method used to limit the amount of
memory used for storage of executable code. A program can store its data in
static or dynamic memory. Static memory is allocated at the time the program
is first loaded into memory by the operating system, and the amount of static
memory used by a program is fixed. Dynamic memory, on the other hand, is
allocated as needed at the request of the program during execution. Dynamic
memory is assigned from a pool of available memory managed by the operating
system that is often referred to as the heap. When the program finishes using
a block of dynamic memory, it can return the memory to the heap. Note that
static and dynamic memory are strictly software differentiations based on how
memory is used. The kind of physical memory, be it standard RAM, extended, or
expanded, is immaterial.
Prudent use of memory is enhanced by minimizing the software's reliance on
static memory and maximizing its use of dynamic memory. The choice of
programming language can have an important impact on the ability to do this,
and C is particularly well suited for extensive use of dynamic memory. "Gas"
is also defined as "any substance that produces a poisonous, irritating, or
asphyxiating atmosphere." Software does not match this definition because it
actually seems to create an atmosphere in which bugs thrive. Which takes me to
the primary subject of this article -- bugs, specifically the kind that result
from the use of C's dynamic memory features, and how to find and exterminate
them.


C's Dynamic Memory Capabilities


The standard C library provides four functions for managing dynamic memory:
malloc, calloc, realloc, and free. A summary of the use of these functions is
provided in Table 1. These functions are roughly equivalent to those provided
in other languages (for example, Pascal/Modula-2's new and dispose). The real
power comes from C's pointer and array concepts.
Table 1: C's dynamic memory allocation functions

 C Declaration Function Description
 -------------------------------------------------------------------------

 char *malloc (size) Allocates a block of size bytes of memory returning
 unsigned int size; a pointer to the start of the block or NULL if
 there is not enough memory available.

 char *calloc Allocates and clears num contiguous blocks of memory
 (num, size) each size bytes in length returning a pointer to the
 unsigned int num; start of the block or NULL if there is not enough
 unsigned int size; memory available.

 char *realloc Adjusts size of allocated block. Returns a pointer
 (ptr, size) to the block of size bytes or NULL if there is not
 char *ptr; enough memory available. The original block may
 unsigned int size; need to be moved to accommodate the new size.

 free (ptr) Frees a block of dynamic memory pointed to by ptr
 char *ptr; and allocated by a previous call to malloc or
 calloc.

In C, arrays can be either static or dynamic. Static arrays are declared with
a constant size; dynamic arrays are declared as pointer variables. The size of
the array is specified at run time when the memory for the array is allocated
(using either malloc or calloc). In C, both static and dynamic arrays can be
referenced using exactly the same syntax, so there is no requirement to
statically define array sizes. The specification of the size of a needed array
can be deferred until program execution time, when the program has suitable
information regarding the intended use.
Languages without this capability force you to impose arbitrary limits on the
user because of the need to fix array sizes at compile time. The end result is
a large program with table sizes set up to accommodate extreme usage
situations.


Types of Dynamic Memory Misuse in C


The danger inherent in C's dynamic memory and pointer manipulation features
comes from the possibility that pointers can be assigned errant values without
detection by the compiler or run-time system. Errant pointers often result in
an address reference outside the limits of the memory space assigned to the
program. In protected environments, such as Unix and OS/2, this results in an
error called a segmentation violation, which is detected by the use of a
debugger that can easily reveal the location of the offending statement. Both
MS-DOS and the Macintosh's OS, however, are unprotected environments that
allow programs to address any part of memory. In these environments, an errant
pointer could do anything from causing no damage at all to destroying the
entire file system! Debugging in these environments can be a most painstaking
experience, even with the use of a debugger.
The insidious nature of another type of problem that can occur is alluded to
in the following warnings:
". . . any program that changes memory that is not allocated to it stands a
chance of destroying a DOS memory management control block. This causes
unpredictable results that don't show up until an activity is performed where
DOS uses its chain of control blocks (the normal result is a memory allocation
error, for which the only corrective action is to restart the system). -- "
From page 11-3 of the DOS Version 3.30 Technical Reference.
"Attempting to free an invalid pointer may affect subsequent allocation and
cause errors." -- From page 290 of the Microsoft C 5.1 Run-Time Library
Reference.

These comments refer to a common manifestation of an errant pointer bug that
results in the system "hanging" on an ensuing call to malloc. The specific
call that reveals the problem may actually occur after many other malloc calls
have succeeded, compounding the debugging problem.
In my experience, the list in Table 2 identifies some of the most common
programming errors that lead to these dynamic memory corruption problems.
Table 2: Common dynamic memory programming errors in C

 1. Writing beyond the end of an allocated block of memory.
 A typical example is calling function strcpy with a source
 string that is missing the sentinel NULL byte. It can also occur
 from an "off by one" subscripting error.

 2. Freeing a pointer to unallocated memory.
 Have you ever tried freeing static memory? Sometimes, it is not
 that unusual to assume a character pointer points to a dynamic string
 when, in fact, it points to a string constant.

 3. Freeing a pointer to a previously freed block of memory.
 Often, a system has a variety of dynamic data structures, such as
 inverted lists, containing pointers to the same dynamic memory.
 This error happens when an attempt is made to free the common
 memory pointer from multiple locations.

 4. Continued use of freed dynamic memory.
 Some dynamic structures are allocated and freed as needed during
 execution. The need for allocation is often determined by whether
 or not a particular pointer is NULL. This error occurs when you
 forget to assign a NULL to that pointer when the structure is freed
 so that a later allocation that should occur, in fact, doesn't.



Management and Debugging of Dynamic Memory


The need for the use of dynamic memory in Raima's database software, db_VISTA
and db_QUERY, is particularly acute. Both packages are C linkable libraries
providing full-featured database management and query engines. As such, they
need to use as little memory as possible while still providing the flexibility
needed to support a wide variety of applications. Yet, because of the dangers
inherent in C's dynamic memory capabilities, and the critical need for
reliability in a DBMS, a disciplined approach to dynamic memory usage had to
be developed so that this insidious class of bugs could be easily detected and
corrected.
Because most C implementations of malloc and free provide little, if any,
error checking, it was necessary to build our own extended dynamic memory
control module, which we've dubbed Xmem and which is shown in Listing One.
Xmem keeps track of all pointers to dynamically allocated memory and performs
checks for the four errors delineated in Table 2. Defined in the module are
functions x_malloc, X_calloc, and x_free, which are to be called instead of
malloc, calloc, and free. One additional function, called X_chkfree, checks to
ensure that all blocks allocated using X_malloc and x_calloc have been freed
and, reports and frees any that have not been freed.
A global int variable, memtrace, can be set to determine the type of dynamic
memory control to be used. If memtrace is 0, no error checking is performed --
that is, x_malloc, X_calloc, and x_free simply call malloc, calloc, and free.
A memtrace value of 1 enables pointer tracking and checking for errors 1, 2,
and 3. Setting memtrace to 2 additionally checks for error 4.
Xmem maintains two global long variables that can be referenced by the
application program when memtrace is nonzero. Variable tot_memory contains the
total amount of dynamic memory that is currently allocated; variable tot_alloc
contains the total number of calls to x_malloc and x_calloc.
A table of all allocated pointers is maintained by Xmem. When a request for a
block of memory is made through a call to x_malloc, a gap of extra space is
allocated at the end of the block to be used to check for overwrites. The
extra space is filled with some predefined character (FILLCHAR) and checked
for changes when the block is freed (error 1). The size of the allocated block
and the pointer are stored in the table.
The memory allocation tracking table simplifies checking for an invalid
pointer. When x_ free is called, if the pointer is in the table, the space is
freed and the pointer's entry is removed. If the pointer is not in the table,
then it is either invalid or it has already been freed (errors 2 and 3).
Setting memtrace to 2 enables checking for changes to a freed block of memory
(error 4). Instead of actually freeing the block, x_ free allocates a copy of
the block (if the copy already exists, then we know that the block has been
previously freed -- error 3). Function x_chkfree compares the copy with the
original and reports any changes. Note that this option does not free any
memory but uses virtually twice as much. The errors detected by this method
are particularly treacherous, however. Note that it is not really necessary to
maintain a copy of the block in order to detect changes. A cyclic redundancy
check (CRC) value (for example, check-sum) could be used instead. This would
indeed identify that a change has occurred. The disadvantage of using a CRC,
however, is that you do not know what was changed. With a copy, you can know
the exact location of the changes and use a debugger (for example, Microsoft's
CodeView, placing a trace point on the changed location) to find the offending
code.
The memory allocation tracking table is constructed as a hash table. A hash
table is a structure that stores an item in a location in the table that is
computed based on the value of the item. In our case, a hash index is computed
from the value of the pointer by casting the pointer to a long and computing
its modulo based on the size of the hash table array. A hash table was chosen
because of the amount and frequency of needed allocations in db_QUERY.
Depending on the complexity of the query, as few as ten but up to as many as
several hundred allocations can occur during the setup and processing of a
query. Thus, it is important to be able to locate a pointer's position in the
table quickly.
The hash table consists of an array of pointers to buckets. A bucket stores
information about each allocated pointer. Lines 39 through 55 in Listing One
contain the hash table declarations. Field alloc in struct bucket is a dynamic
array of struct alloc_entry. This array will store up to bucketsize (line 28)
allocated pointer entries. Each alloc entry contains the size of the allocated
block, the pointer (ptr) to the allocated block, and a pointer to the copy of
the block allocated when the block is freed. The size of the hash table is
specified by a global int variable hashsize (line 27). This value must be odd
(preferably prime) or the odd-numbered buckets will never be used.
The hash table is a dynamic array of bucket pointers called ptrhash (line 55).
Because more than bucketsize pointers could have the same hash value, the
buckets contain a next pointer to allow additional buckets to be allocated and
chained to the same ptrhash entry. Figure 1 illustrates the structure of the
memory allocation tracking table.
Xmem defines two local functions that manage the hash table and report errors.
Function sto_ptr stores the specified pointer and the size in the hash table.
This function allocates the dynamic memory for ptrhash the first time it is
called. Line 75 computes the hash table index, which is used in line 78 to
subscript the ptrhash array. Any buckets that exist for the computed ptrhash
entry are searched in lines 77 - 80 for an available alloc array entry. If
there is no available space, a new bucket is allocated and connected to the
linked list (lines 82 - 102). The pointer information is then stored into the
next available alloc array slot in the bucket (lines 103 - 107). Function
del_ptrdeletes the specified pointer from the hash and frees the dynamic
memory referenced by it. When the pointer is located (lines 131 - 134), the
gap is checked and, if changed, the error is reported (line 139). If memtrace
is 1, the pointer is removed from the bucket and the dynamic memory is freed
(lines 145 - 161). If memtrace is 2, a copy of the allocated memory is made,
unless one already exists, for comparison by function x_chkfree (lines 164 -
171).
When memtrace is nonzero, function x_malloc calls malloc to allocate the
requested memory, augmented by the extra gap space. It then calls sto_ptr to
store the pointer and size in the hash. Function x_calloc calls x_malloc to
allocate the requested memory and then calls the standard C function memset to
clear the allocated block.
Function x_free checks for a NULL pointer (something that, for some unknown
reason, many implementations of free do not do), and if memtrace is nonzero,
it calls del_ptr to remove the pointer from the hash and free the space. If
memtrace is 0, x_free simply calls free.
Function x_chkfree searches the hash table, reporting and freeing any pointers
that remain. If memtrace is 2, it checks for any changes to freed blocks. The
hash table itself is also freed by x_chkfree.
When errors are reported, a debugger is needed to track down the specific
source of the problem. For the case in which the gap has been over-written,
you can record the value of the pointer and rerun the exact test scenario,
setting a breakpoint, for example, at line 211 in x_malloc to find out when
and where that particular allocation occurred. You can discover how the memory
is to be used from the function calls stack. This will often be sufficient to
reveal the problem.
You could also use the debugger to break whenever the gap is modified, which
would reveal the specific statement that is causing the problem.
Unfortunately, this kind of operation is very slow in most debuggers,
primarily because of the lack of good hardware support for debugging.
Setting a breakpoint at line 184 in function del_ptr and then inspecting the
function calls stack when the breakpoint occurs will often explain why a
pointer to be freed is not in the table. Perhaps it has already been freed, or
you see that the space being freed is actually static.


Enhancements


Many enhancements could be made to the functionality presented here. For
example, it is often valuable to be able to free not only a single pointer but
also all other dynamic memory that was since allocated. This could be
accomplished by adding an allocation number to the alloc_entry struct that is
assigned the current tot_alloc value when stored in the table. A new function
called x_release could be added that would free all pointers in the table with
an allocation number greater than that associated with the specified pointer.
The CRC method of detecting changes to allocated memory described earlier
could form the basis of a change-tracking system whereby those dynamic
variables that had been modified by a given function could be reported. This
could be implemented by adding a string argument to x_malloc and x_calloc that
contained the name of the pointer variable being allocated. The
alloc_entry-struct would store this string as well as the current CRC value. A
function that would be called as desired (for example, on exit from each
function in the application program) would report the names of all variables
that had different CRC values. If the application's function name were printed
first, then the list could be inspected to ensure that the proper variables
were modified.


Conclusion



Because memory has always been and still is a precious commodity, we have had
to be frugal in our use of memory in the development of our programs. The
ability in C to maximize the use of dynamic memory has allowed us to develop
sophisticated software systems to run on small computers. C has often been
criticized, however, for the ease with which catastrophic errors can occur
resulting from the very use of these powerful features. Some people have even
written off the use of C because of these problems.
This article has presented proven techniques to detect and identify errors in
the use of dynamic memory in C: As C and its supporting cast of development
tools, particularly debuggers, continue to evolve, there is no doubt in my
mind that these problems will be much more easily avoided.


Availability


All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063, or call 800-356-2002 (from inside
Calif.) or 800-533-4372 (from outside Calif.). Please specify the issue number
and format (MS-DOS, Macintosh, Kaypro).


C DYNAMIC MEMORY USE
by Randall Merilatt


[LISTING ONE]

/*---------------------------------------------------------------
 xmem.c -- Extended Dynamic Memory Control Module.
---------------------------------------------------------------*/
/* ********************** INCLUDE FILES ********************** */
#include <stdio.h>

/* ******************** EXTERNAL FUNCTIONS ******************* */
extern char *malloc(unsigned int);
extern char *calloc(unsigned int, unsigned int);
extern void free(char *);

/* ********************* GLOBAL FUNCTIONS ******************** */
char *x_malloc(unsigned int);
char *x_calloc(unsigned int, unsigned int);
void x_free(char *);
void x_chkfree();

/* ********************* GLOBAL VARIABLES ******************** */
/* memtrace usage:
 = 0 => simple calls to malloc & free
 = 1 => tracking of all allocations using hash table
 = 2 => checking for changes to previously freed blocks
*/
int memtrace = 1; /* memory tracing control variable */
long tot_memory = 0L; /* total amount of allocated memory */
long tot_alloc = 0L; /* total # of allocations */
int hashsize = 47; /* size of hash table */
int bucketsize = 10; /* number of entries per hash bucket */

/* ********************* LOCAL VARIABLES ********************* */
/* memory allocation tracking table */

/* amount of extra allocation for overhead */
#define OVHDSIZE 2

/* fill character for overhead gap */
#define FILLCHAR '\377'

/* allocated entry information */
typedef struct alloc_entry {
 int size; /* size of allocated area */

 char *ptr; /* pointer to allocated area */
 char *freed; /* pointer to copy of allocated area */
} ALLOCATION;

typedef struct bucket {
 struct bucket *next; /* pointer to next bucket when filled */
 int entries; /* number of used entries */
 ALLOCATION *alloc; /* allocated entry array */
} BUCKET;

#define NUL_BUCKET ((BUCKET *)0)

/* dynamic pointer hash table */
static BUCKET **ptrhash = (BUCKET **)0;

/* ==============================================================
 Store pointer in hash table
*/
static char *sto_ptr(p, b)
char *p; /* pointer to be stored */
unsigned b; /* size of area */
{
 register BUCKET *bp, *bq; /* bucket pointers */
 register int bno; /* bucket/entry number */

 if ( ! ptrhash ) {
 /* allocate pointer hash table */
 ptrhash = (BUCKET **)calloc(hashsize, sizeof(BUCKET *));
 if ( ! ptrhash )
 return(NULL);
 tot_memory = hashsize * sizeof(BUCKET *);
 }
 /* compute hash table index */
 bno = (int)((unsigned long)p % hashsize);

 /* find first bucket with available entries */
 for (bq = bp = ptrhash[bno]; bp && bp->entries == bucketsize;
 bp = bp->next)
 bq = bp;

 /* allocate new bucket if necessary */
 if ( bp == NUL_BUCKET ) {
 if ( ! (bp = (BUCKET *)malloc(sizeof(BUCKET))) )
 return(NULL);
 bp->next = NUL_BUCKET;
 bp->entries = 0;
 if ( bq )
 /* connect to end of bucket chain */
 bq->next = bp;
 else
 /* initial bucket for this hash entry */
 ptrhash[bno] = bp;

 /* allocate bucket's allocation entry array */
 bp->alloc = (ALLOCATION *)calloc(bucketsize,
 sizeof(ALLOCATION));

 /* memory total includes space used by hash table */
 tot_memory += sizeof(BUCKET) +

 bucketsize*sizeof(ALLOCATION);
 }
 /* store pointer to allocated block */
 bno = bp->entries++;
 bp->alloc[bno].ptr = p;
 bp->alloc[bno].freed = NULL;
 bp->alloc[bno].size = b;

 /* update total allocation */
 tot_memory += b;

 /* increment total number of allocations */
 ++tot_alloc;

 return(p);
}

/* ==============================================================
 Delete pointer from hash table
*/
static void del_ptr(p)
char *p; /* pointer to be freed */
{
 int gap; /* index into overhead space */
 register BUCKET *bp, *bq; /* bucket pointers */
 register int bno, i; /* bucket/entry number */

 /* compute hash table index */
 bno = (int)((unsigned long)p % hashsize);

 /* search bucket(s) for pointer */
 for (bq = NUL_BUCKET, bp = ptrhash[bno]; bp; bp = bp->next) {
 for ( i = 0; i < bp->entries; ++i ) {
 if ( bp->alloc[i].ptr == p ) {
 /* check integrity of gap */
 for (gap=bp->alloc[i].size-OVHDSIZE;
 gap<bp->alloc[i].size; ++gap ) {
 if ( p[gap] != FILLCHAR ) {
 printf("WARNING overwrite, addr: %lx\n",
 (long)p);
 break;
 }
 }
 if ( memtrace == 1 ) {
 /* remove entry from bucket */
 if ( --bp->entries == 0 ) {
 /* free this bucket */
 if ( bq )
 bq->next = bp->next;
 else
 ptrhash[bno] = bp->next;
 free((char *)bp->alloc);
 free((char *)bp);
 tot_memory -= (sizeof(BUCKET) +
 bucketsize*sizeof(ALLOCATION));
 }
 else if ( i < bp->entries ) {
 /* move last entry into current spot */
 bp->alloc[i] = bp->alloc[bp->entries];

 }
 free(p);
 }
 else {
 /* memtrace == 2
 => save copy to check for bad mods */
 if ( bp->alloc[i].freed )
 printf("WARNING freeing free ptr, addr: %lx\n",
 (long)p);
 else if (bp->alloc[i].freed = malloc(bp->alloc[i].size))
 memcpy(bp->alloc[i].freed, bp->alloc[i].ptr,
 bp->alloc[i].size);

 }
 /* update total allocated memory count */
 tot_memory -= bp->alloc[i].size;

 /* normal return */
 return;
 }
 }
 bq = bp;
 }
 if ( ! bp )
 printf("WARNING freeing bad pointer, addr: %lx\n", (long)p);
}

/* ==============================================================
 Allocate b bytes of memory
*/
char *x_malloc( b )
unsigned int b; /* number of bytes to allocate */
{
 register char *mptr;

 if ( memtrace ) {
 /* add gap space */
 b += OVHDSIZE;

 /* allocate memory */
 if ( mptr = malloc(b) ) {
 /* fill gap */
 memset(mptr+b-OVHDSIZE, FILLCHAR, OVHDSIZE);

 /* store mptr in ptrhash */
 mptr = sto_ptr(mptr, b);
 }
 }
 else
 mptr = malloc(b);

 return(mptr);
}

/* ==============================================================
 Allocate and clear i*s bytes of memory
*/
char *x_calloc( i, s )
unsigned int i; /* number of blocks to be allocated */

unsigned int s; /* size (in bytes) of each block */
{
 register unsigned int amt;
 register char *mptr;

 /* allocate requested space */
 if ( mptr = x_malloc(amt = i*s) ) {
 /* clear requested space */
 memset(mptr, '\0', amt);
 }
 return (mptr);
}

/* ==============================================================
 Free allocated memory
*/
void x_free( p )
char *p; /* pointer to block to be freed */
{
 if ( p == NULL )
 printf("WARNING freed a null pointer\n");
 else if ( memtrace )
 del_ptr(p);
 else
 free((char *)p);
}

/* ==============================================================
 Check to ensure all blocks have been freed
*/
void x_chkfree()
{
 ALLOCATION *ap; /* allocation entry pointer */
 register int bno, i; /* bucket/entry number */
 register BUCKET *bp, *bq; /* bucket pointers */

 if ( memtrace ) {
 /* check for unfreed variables */
 for ( bno = 0; bno < hashsize; ++bno ) {
 for ( bp = ptrhash[bno]; bp; bp = bq ) {
 for (i = 0; i < bp->entries; ++i) {
 ap = &bp->alloc[i];
 if ( memtrace == 2 && ap->freed ) {
 /* check for changes to freed blocks */
 if ( memcmp(ap->ptr, ap->freed, ap->size) )
 printf("WARNING block chgd after free, addr: %lx\n",
 (long)ap->ptr);
 }
 /* free unfreed block */
 printf("WARNING freeing unfreed block, addr: %lx\n",
 (long)ap->ptr);
 free(ap->ptr);
 }
 bq = bp->next;

 /* free bucket */
 free((char *)bp->alloc);
 free((char *)bp);
 }

 }
 /* free pointer hash pointer array */
 free((char *)ptrhash);
 ptrhash = (BUCKET **)0;

 tot_memory = 0L;
 }
}






















































August, 1989
C PROCEDURE TABLES


Calling functions by the number




Tim Berens


Tim Berens is president of Back Office Applications, Inc., a contract software
house. He can be reached at 1250 W Dorothy Ln., Ste. 301, Dayton OH 45409.


The first time I heard of storing data in tables, it seemed completely
unnecessary. Drawing on my full two weeks of programming experience, I came to
the conclusion that arrays were just too much work.
Since that rather premature conclusion, I have found arrays to be one of the
most useful tools for handling large quantities of data. I remember drawing a
similarly premature conclusion when I first heard about an underused feature
of the C language -- arrays of pointers to functions.
A pointer to a function contains the starting address of a subroutine. Just
like a character pointer points to a character in memory, a function pointer
points to a function, which can be executed using the pointer. A function
executed in this way performs exactly the same as a function executed using a
standard call. Arguments can be passed to it, and a value can be returned from
it.
A function pointer by itself can be a handy tool, but it takes on a great deal
more significance when stored with others in an array. We call such an array a
"procedure table."
A procedure table is a powerful tool for software design. It is a method of
storing procedures (functions and subroutines) in a table so that the stored
data can be accessed by using a numeric offset. This gives you a method for
systematically controlling procedures, as in a loop, for example.
To illustrate the use of procedure tables, I have chosen a task that I have
had to tackle in many systems and one that I have always found particularly
cumbersome. At some point, a program usually prompts the user for a series of
responses in order to gather parameters. These parameters may be used as query
criteria for a report, for narrowing down a potential problem, as input for a
graph, and so on.
Gathering these parameters is complicated by the validation of the responses,
and by multiple possible paths through the prompts. For example, if the first
prompt asks the user if a report is to be printed for a single account, a
range of accounts, or all accounts, the second prompt depends on the user's
response. If the second and third prompts get the starting and ending account
numbers for a range, the program must validate that the ending account number
is larger than the starting account number. This goes on ad nauseam.
It always bothered me to take the obvious approach to this problem. Gathering
the parameters in this manner resulted in a mass of nested if/else phrases.
This type of code is difficult to read and maintain and is not reusable.
Procedure tables provide a better way.


prompter( )


Listings One and Two illustrate prompter( ). The goal of prompter( ) was to
develop a reusable routine that prompts a user, validates responses, and
performs actions based on those responses. prompter( ) must be able to handle
unique validations and multiple paths through the dialog. It does this by
executing functions stored in a procedure table, which is implemented as an
array of data structures.
The basic building block of prompter( ) is the data structure question, which
is declared as:
 struct question {
 char * text;
 char * response;
 int (*validate)( );
 int (*doit)( );
 int (*set)( );
 };
The application that calls prompter( ) defines one or more arrays of this
structure. prompter( ) loops through the array(s) of structures according to
the algorithm shown in Example 1. Let's look at each member of question in
detail.
Example 1: Looping through an array

 loop{
 display current_question->text
 get response from user
 execute current_question->validate
 if (no error on validate){
 execute current_question->doit
 }
 copy response to current_question->response

 execute current_question->set
 if (error from validate){
 call error handler
 }
 }

question.text is the text of the prompt, such as "Do you want this report for
one account, a range of accounts, or all accounts?"
question.response is a pointer to the variable that receives the user's
response. This allows prompter( ) to gather the parameters needed by the
application. If question.response is set to NULL, prompter( ) assumes the
application no longer needs the response and throws it away.
question.validate is the address of the routine that validates the response.
For example, it may check to see that the account number entered is valid. If
an error occurs, the function returns a unique non-zero code, and the error is
handled downstream by the handle_error( ) routine.

question.doit is the address of the routine that will perform the action, if
any, associated with this question. For example, it might convert an ASCII
account number to an integer and store it in the appropriate variable.
question.set is the address of a routine that determines what the next
question will be. It can tell prompter( ) to go on to the next question in the
array or to jump to a new array of questions, depending on the response given
by the user. For example, if the user requests a report for a range of
accounts, this routine will tell prompter( ) to ask the questions stored in
the range_of_accounts array. If an error was encountered, the routine can tell
prompter( ) to ask the current question again, or to back up and restart the
questions at a previous point in the array.
The prompter( ) routine is itself quite small. It is nothing more than a
triggering mechanism for the functions stored in procedure tables, that is, a
routing control to the proper routine.
After each function executes, it returns a status code that indicates if it
was successful. prompter( ) uses this value to route control to the error
handler if an error is encountered.
prompter( ) makes no decisions about which question structure to use as its
basis for prompting the user. The application defines the arrays of question
structures, and it is responsible for deciding what their contents are and the
order they will be in.
Several routines are provided to simplify this job. These routines are called
by the set member of question (question.set( )), which decides what the next
question will be based on the response given by the user.


The prcontrol Structure


The prcontrol structure is the master control structure for prompter( ). It
contains all information regarding the current state of the prompting. A
pointer to this structure is passed to every function called from prompter( ).
This allows the functions to examine and modify the current state of prompter(
).
The prcontrol structure is declared:
 struct prcontrol {
 int current_question;
 struct question * current_group;
 int group_stack_ptr;
 char response[121];
 int errstat;
 struct errormess * errormess;
 }
Let's look at each member in detail:
prcontrol.current_question is the offset of the current question in the array
of question-data structures pointed to by prcontrol.current_group.
prcontrol.current_question is set to zero on entry to prompter( ) and is
typically incremented by question.set in order to go on to the next question.
prcontrol.current_group is a pointer to an array of question structures. This
is the group of questions that is currently being asked.
prcontrol.group_stack_ptr is a part of the mechanism that allows prompter( )
to jump easily from one array (or group) of questions to the next. This
permits prompter( ) to follow multiple paths through the prompts. See the
discussion of multiple paths later in this article.
prcontrol.response is a buffer for holding the response entered by the user.
prcontrol.errstat is the error status returned from question.validate( ) or
question.doit( ). Typically this value is examined by question.set( ) before
deciding what route to tell prompter( ) to take. It is also passed to
handle_error( ) to display the proper error message.
prcontrol.errormess is a pointer to an array of errormess structures. This
pointer is passed to handle_error( ) when an error is encountered so it can
display the proper error message.


Multiple Paths


The ability to follow multiple paths is handled through the use of a stack,
which is implemented as an array of group_stack structures.
The group stack enables prompter( ) to jump to another group of questions
without losing its place. It operates in a way similar to the stack in a C or
assembler program.
When a question.set( ) routine decides it must jump to another array of
questions, it calls the routine start_group( ). start_group( ) calls
push_group( ) to push prcontrol.current_group and prcontrol.current_question
onto the group stack.
When this group of questions is finished executing, a question.set( ) routine
calls end_group( ). end_group( ) calls pop_group( ), which pops
prcontrol.current_group and prcontrol.current_question off of the group stack.
end_group( ) then increments prcontrol.current_question to continue at the
next question, past the point where the other group of questions was called.
This method allows a group of questions to start another group of questions,
which can start another group of questions, and so on, and prompter( ) can
still easily return to the point at which it started.


Error Handling


Wouldn't computer programs be much easier to write if we could assume that
users never make mistakes? But of course we cannot assume this, so prompter( )
has an error handling mechanism.
Any time an error is encountered by question.validate( ) or question.doit( ),
the routine that detects the error returns a unique, non-zero value to
prompter( ). This value is saved in prcontrol.errstat( ). The routine in
question.set( ) then decides how this error will effect the direction of the
questions.
This error status is then passed to the routine handle_error( ), whose
responsibility is to build and display the proper error message. handle_error(
) works with an array of struct errormess. This array is declared:
 struct errormess{
 int errstat;
 char * message;
 int (*build)( );
 };
errormess.errstat is the value that identifies the error, and
errormess.message is the message that appears when this error is encountered.
errormess.build is the address of the function that will perform any extra
formatting of errormess.message. For example, if the error ACCOUNT_NOT_IN_FILE
is encountered, this routine might turn the message Account %s not in file
into "Account 101 not in file."
The application program defines an array of these structures. When
handle_error( ) is called, it searches through this array until it finds a
match. When a match is found it executes errormess.build if errormess.build is
not NULL. Then it displays the message.


Is This Too Much Work?


Before we dig into a specific example, let me try to address a question that
many will have. Yes, this is too much work to go through to ask two questions.
But this is not too much work to go through to ask 200 or even 20 questions.

A method like this produces real savings in development time. The reason is
that system prompts occur in patterns. For example, a system we recently
developed had about 20 reports. Each report required that the user specified
an output destination: printer, screen, or disk. The disk selection requires
additional entry of a filename. If the file exists, the user has the option of
overwriting that file, choosing a new file or appending the report to the
file.
The savings in development time is realized after the printer, screen, or file
procedure tables have been built and the routines coded for the first time. At
this point, all routines become data that is fed to prompter( ). Any program
that needs to access this particular series of questions as a part of its
prompts simply calls start_group( ) from one of its question.set routines to
start this group of questions. The printer, screen, or file prompts appear on
the screen and when they have completed, the prompts go on from where they
left off. The programmer never again has to worry about this series of
prompts.
As the number of this type of pattern of prompts increases, the value of
prompter( ) as a development tool increases. prompter( ) can be even used to
do a rudimentary form of reasoning by having it chew its way through a series
of prompts from a user. It will eventually reach a conclusion.


A Specific Example


The example I have chosen to illustrate prompter( ) is a series of prompts
that gathers parameters for a mythical account report. The parameters that
must be gathered are:
The account query criteria -- the account number or range of accounts to be
included.
The display parameter's record that is to be used.
Should the Over/Short report be printed automatically?
The report destination: printer, screen, or disk.
Refer to Listing Three for the code that handles this set of prompts.
account_parms is the array of struct question that provides the main flow of
prompts. Its address goes into the prcontrol structure from main( ) before the
call to prompter( ).
You can see the flow of the prompts by reading the initialization of this
array. Let's look at the initialization of the first member in detail:
account_parms[0].text points to "Do you want this report for a single account
or a range of accounts? (S or R)." This text will be displayed when the user
is being prompted.
account_parms[0].response points to single_or_range (which is an array of
char). The response entered by the user will be copied here so the report
program knows if the report is to be for a single account or a range of
accounts.
account_parms[0].validate is set to the address of the routine
account_or_range_val( ). This routine will make sure that the response is S or
R. If not, account_or_range_val( ) returns the error status ENTER_S_OR_R and
handle_error( ) prints the appropriate error message.
account_parms[0].doit is set to the address of no_op( ). This routine does
nothing but return(O);. no_op( ) is used as a place holder, because no doit
action is required for this prompt. It is necessary because prompter( ) will
execute the function specified in account_parms[0].doit, so a function address
must be stored there.
account_parms[0].set points to account_or_range_set( ). This function
determines if the user entered S, R, or an erroneous response. If an error
occurred, account_or_range_set( ) does nothing. This will cause the question
"Do you want this report for a single account or a range of accounts? (S or
R)" to be asked again. If the user entered S, account_or_range_set( ) starts
the group that requests the single account number. If the user entered R,
account_or_range_set( ) starts the group that requests the range of account
numbers.
As you look through the code in Listing Two, you will see calls to several
functions that assist with control of flow. These functions include:
start_group( ), which starts a new group of questions.
end_group( ), which ends the current group of questions. It calls pop_group( )
to restore the prompting to its previous state.
checkerror_next_question( ), which causes prompter( ) to go on to the next
question unless an error was encountered.
checkerror_end_group( ), which ends the current group of questions unless an
error was encountered.
restart_group( ), which restarts the current group of questions.
Notice that checkerror_next_question( ) and checkerror_end_group( ) can be
used as a question.set( ) routine in many cases (look at
account_parms[1].set). It is precisely this type of function reuse that saves
development time over the long run.
As you develop more and more code that uses prompter( ), you begin to notice
patterns. Once a pattern has been discovered, you write a generalized routine
to handle the pattern, and this routine can be used over and over.
I suggest that the simplest way to get a clear picture of how prompter( )
works its way through the procedure tables is to single-step your way through
prompter( ) with a debugger like CodeView.


Suggestions for Improving prompter( )


The first step to improve prompter( ) is to remove the printf( )/gets( )
interface, and attach a windowing-type interface. To do this, I suggest that
you add an element to the question data structure. This element
(question.form) is a pointer to a function that formats the output. Its job is
to place the question text on the screen in its proper position, adjusting
such attributes as color. If you are careful to isolate all screen positioning
to only the question.form routines, you can later port the system to a
different display by just rewriting these functions.
Next, add a better keyboard input routine. Most production systems do not use
gets( ) for input.
Finally, as you use prompter( ), you will notice that there is room for
improvement in the area of moving backwards in the prompts. You can develop a
more elegant approach by having the routine prompter( ) automatically call
pop_group( ) when it is at the end of a group.


Back to Procedure Tables


The point of this article was not to demonstrate how to prompt users for input
but, to demonstrate the use of procedure tables. Procedure table techniques
that are similar to those used in prompter( ) can be applied to a wide variety
of tasks. We have used these techniques for the development of file
maintenance programs, communications programs, parsers, menus, report
generators, keyboard input validation routines, and others.
We have found procedure tables to be extremely helpful for developing software
that is flexible, bug free, and highly maintainable. Using procedure tables
allows us to treat functions as if they were data, and this opens up a new
world to system design.


Availability


All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063, or call 800-356-2002 (from inside
Calif.) or 800-533-4372 (from outside Calif.). Please specify the issue number
and format (MS-DOS, Macintosh, Kaypro).


C PROCEDURE TABLES
by Tim Berens



[LISTING ONE]



/****************************************************************************
 Name : prompter.c

 Description : A routine for prompting a user for a series of answers.
****************************************************************************/
#include<stdio.h>
#include"prompter.h"

struct group_stack group_stack[GROUP_STACK_SIZE];

prompter(pc)
 struct prcontrol * pc;
{
 int errstat;

 pc->current_question = 0;
 pc->group_stack_ptr = 0;

 for(;;){
 pc->errstat = 0;

 display_current_question(pc);

 gets(pc->response);

 if(*pc->response == 0){
 continue;
 }

 if(!(pc->errstat =
 (*pc->current_group[pc->current_question].validate)(pc))){

 if(pc->errstat =
 (*pc->current_group[pc->current_question].doit)(pc)){
 if(pc->errstat == EXIT_NOW){
 return(0);
 }
 }
 }

 if(pc->current_group[pc->current_question].response != NULL){
 strcpy(pc->current_group[pc->current_question].response,
 pc->response);
 }

 (*pc->current_group[pc->current_question].set)(pc);

 if(pc->current_group[pc->current_question].text == NULL){
 return(0);
 }

 if(pc->errstat){
 handle_error(pc->errstat,pc->errormess);
 }

 }

}


display_current_question(pc)
 struct prcontrol * pc;
{
 printf("\n%s\n",pc->current_group[pc->current_question].text);
 printf("--->");

}

handle_error(errstat,errormess)
 int errstat;
 struct errormess * errormess;
{
 int i;
 int emess_offset = -1;
 char * message,messagebuff[100];

 for(i = 0 ; errormess[i].errstat != -1 ; ++i){
 if(errormess[i].errstat == errstat){
 emess_offset = i;
 break;
 }
 }
 message = messagebuff;
 if(emess_offset != -1){
 strcpy(message,errormess[emess_offset].message);
 if(errormess[emess_offset].build){
 (*errormess[emess_offset].build)(message);
 }
 }
 else{
 sprintf(message,"Error %d.",errstat);
 }

 puts("\n");
 puts(message);
 return(0);
}


/***************************************
 Flow control routines
***************************************/
no_op()
{
 return(0);
}

next_question(pc)
 struct prcontrol * pc;
{
 ++pc->current_question;
 return(0);
}


pop_group(pc)
 struct prcontrol * pc;
{

 --pc->group_stack_ptr;
 pc->current_group = group_stack[pc->group_stack_ptr].group;
 pc->current_question = group_stack[pc->group_stack_ptr].current_question;
 return(0);
}

push_current_group(pc)
 struct prcontrol * pc;
{
 group_stack[pc->group_stack_ptr].group = pc->current_group;
 group_stack[pc->group_stack_ptr].current_question = pc->current_question;
 ++pc->group_stack_ptr;
 return(0);
}

start_group(newgroup,pc)
 struct question * newgroup;
 struct prcontrol * pc;
{
 push_current_group(pc);
 pc->current_group = newgroup;
 pc->current_question = 0;
 return(0);
}

restart_group(pc)
 struct prcontrol * pc;
{
 pc->current_question = 0;
 return(0);
}


end_group(pc)
 struct prcontrol * pc;
{
 pop_group(pc);
 ++pc->current_question;
 return(0);
}

checkerror_end_group(pc)
 struct prcontrol * pc;
{
 if(pc->errstat){
 return(0);
 }
 end_group(pc);
 return(0);
}

checkerror_next_question(pc)
 struct prcontrol * pc;
{
 if(pc->errstat){
 return(0);
 }
 next_question(pc);
 return(0);

}






[LISTING TWO]

/****************************************************************************
 Name : prompter.h

 Description : Declarations for prompter
****************************************************************************/

struct prcontrol {
 int current_question;
 struct question * current_group;
 int group_stack_ptr;
 char response[121];
 int errstat;
 struct errormess * errormess;
 };

struct question {
 char * text;
 char * response;
 int (*validate)();
 int (*doit)();
 int (*set)();
 };

struct group_stack {
 struct question * group;
 int current_question;
 };

/************************
 errormess data structure
************************/

struct errormess {
 int errstat;
 char * message;
 int (*build)();
 };

#define GROUP_STACK_SIZE 50
#define NO_ERROR 0
#define EXIT_NOW 2001

int pop_group(),end_group(),no_op(),next_question();
int checkerror_end_group(),checkerror_next_question();








[LISTING THREE]

/****************************************************************************
 Name : prsample.c

 Description : A sample that uses the prompter() routine
****************************************************************************/
#include<stdio.h>
#include"prompter.h"
#include<ctype.h>

/**************************
 The report parameter variables
***************************/
char report_destination[2];
char dest_filename[30];
char single_or_range[2];
char start_account[20],end_account[20];
int account_number;
char display_parmname[50];
char include_overshort[2];

/*********************
 Error Values
*********************/
#define ENTER_S_OR_R 1
#define ENTER_Y_OR_N 2
#define START_ACCOUNT_LARGER 3
#define BAD_PARM_NAME 4
#define BAD_ACCOUNT_NUMBER 5
#define ENTER_P_S_OR_D 6
#define FILE_EXISTS 7

/************************
 Report to printer, screen or disk routines
************************/
int filename_val();
struct question report_filename[] = {
 { "What is the name of the disk file?",
 dest_filename,filename_val,no_op,checkerror_end_group},
 { NULL,NULL,NULL,NULL,NULL }
 };

filename_val(pc)
 struct prcontrol * pc;
{
 FILE * fp,*fopen();
 /* you should put a routine to validate that the response
 entered is a legal file name here */
 if(fp = fopen(pc->response,"r")){
 fclose(fp);
 return(FILE_EXISTS);
 }
 return(0);
}

reportdest_val(pc)
 struct prcontrol * pc;

{
 char * strchr();
 if((!strchr("PpSsDd",pc->response[0])) (strlen(pc->response) != 1)){
 return(ENTER_P_S_OR_D);
 }
 return(0);
}

reportdest_set(pc)
 struct prcontrol * pc;
{
 char destination;
 destination = islower(*pc->response) ? *pc->response-32 : *pc->response;
 switch(destination){
 case 'P' :
 case 'S' : next_question(pc);
 break;
 case 'D' : start_group(report_filename,pc);
 break;
 }
 return(0);
}

/***************************
 Account routines
***************************/
int account_val(),end_account_set(),end_account_val();

struct question account_range[] = {
 {"Enter the starting account.",
 start_account,account_val,no_op,checkerror_next_question},
 {"Enter the ending account.",
 end_account,end_account_val,no_op,end_account_set},
 { NULL,NULL,NULL,NULL,NULL }
 };

int save_account_doit(),account_set();
struct question account[] = {
 {"Enter the account.",
 start_account,account_val,save_account_doit,checkerror_end_group},
 {NULL,NULL,NULL,NULL,NULL}};

account_or_range_val(pc)
 struct prcontrol * pc;
{
 char * strchr();
 if((!strchr("SsRr",pc->response[0])) (strlen(pc->response) > 1)){
 return(ENTER_S_OR_R);
 }
 return(0);
}

account_or_range_set(pc)
 struct prcontrol * pc;
{
 char account_or_range;
 account_or_range = islower(*pc->response) ? *pc->response-32 :
 *pc->response;
 if(pc->errstat){

 return(0);
 }
 if(account_or_range == 'S'){
 start_group(account,pc);
 }
 if(account_or_range == 'R'){
 start_group(account_range,pc);
 }
 return(0);
}

save_account_doit(pc)
 struct prcontrol * pc;
{
 account_number = atoi(pc->response);
 return(0);
}

account_val(pc)
 struct prcontrol * pc;
{
 if((atoi(pc->response) < 100) (atoi(pc->response) > 1000)){
 return(BAD_ACCOUNT_NUMBER);
 }
 return(0);
}

end_account_val(pc)
 struct prcontrol * pc;
{
 int errstat;
 if(errstat = account_val(pc)){
 return(errstat);
 }
 if(atoi(start_account) >= atoi(pc->response)){
 return(START_ACCOUNT_LARGER);
 }
 return(0);
}

end_account_set(pc)
 struct prcontrol * pc;
{
 switch(pc->errstat){
 case NO_ERROR : end_group(pc);
 break;
 case START_ACCOUNT_LARGER : restart_group(pc);
 break;
 case BAD_ACCOUNT_NUMBER : break;
 }
 return(0);
}

/***************************
 Get display parameters routines
****************************/
char * legal_parmnames[] = { /* In a "real" system, this table */
 "default", /* would probably be stored in a file */
 "daily", /* and parmname_val would check to see */

 "weekly", /* if the name entered is in this file. */
 "yearly",
 NULL
 };

parmname_val(pc)
 struct prcontrol * pc;
{
 int i;
 for(i = 0 ; legal_parmnames[i] != NULL ; ++i){
 if(strcmp(pc->response,legal_parmnames[i]) == 0){
 return(0);
 }
 }
 return(BAD_PARM_NAME);
}

bld_bad_parmname(message)
 char * message;
{
 sprintf(message + strlen(message)," %s, %s, %s, or %s.",
 legal_parmnames[0],legal_parmnames[1],legal_parmnames[2],
 legal_parmnames[3]);
 return(0);
}

/**************************
 yesno validation
***************************/
yesno_val(pc)
 struct prcontrol * pc;
{
 char * strchr();
 if((!strchr("YyNn",pc->response[0])) (strlen(pc->response) != 1)){
 return(ENTER_Y_OR_N);
 }
 return(0);
}

/**************************************
 Main question array procedure table
***************************************/
struct question account_parms[] = {
 {"Do you want this report for a single account or a range of accounts? (S or
R)",
 single_or_range,account_or_range_val,no_op,account_or_range_set },
 {"Enter the name of the display parameter record.",
 display_parmname,parmname_val,no_op,checkerror_next_question},
 {"Do you want to include the Over/Short Report? (Y/N)",
 NULL,yesno_val,no_op,checkerror_next_question},
 {"Do you want this report on the printer, screen, or saved to disk?(P,S or
D)",
 report_destination,reportdest_val,no_op,reportdest_set},
 { NULL,NULL,NULL,NULL,NULL }
 };

struct errormess account_errormess[] = {
 { ENTER_S_OR_R,"Please enter S or R.",NULL },
 { ENTER_Y_OR_N,"Please enter Y or N.",NULL },
 { START_ACCOUNT_LARGER,"The starting account must be smaller than the ending
account.",NULL },
 { BAD_ACCOUNT_NUMBER,"The account number must be between 100 and 1000",NULL
},

 { BAD_PARM_NAME,"Choose one of the following :",bld_bad_parmname },
 { ENTER_P_S_OR_D,"Please enter P, S or D",NULL },
 { FILE_EXISTS,"That file already exists.",NULL },
 { -1,NULL,NULL }
 };

main(argc,argv)
 int argc;
 char * argv[];
{
 int errstat;
 struct prcontrol prcontrol;

 prcontrol.current_group = account_parms;
 prcontrol.errormess = account_errormess;

 if(errstat = prompter(&prcontrol)){
 handle_error(errstat,account_errormess);
 }
 /* Print the report with the gathered parameters */
}




[EXAMPLE 1]


 loop{
 display current_question->text

 get response from user

 execute current_question->validate

 if(no error on validate){
 execute current_question->doit
 }

 copy response to current_question->response

 execute current_question->set

 if(error from validate){
 call error handler
 }
 }















August, 1989
GOING FROM K&R TO ANSI C


Extending and codifying the C language




Scott Robert Ladd


Scott Robert Ladd is a full-time computer journalist, who lives in the
mountains of Colorado. He can be contacted through MCI Mail (ID: 369-4376) or
at 705 W Virginia, Gunnison, CO 81230.


Last September, the ANSI X3J11 committee finalized the ANSI standard for C and
sent it up one level for final approval (which should occur this year).
Culminating five years of study and discussion, the standard is becoming the
basis for C compilers on many hardware platforms. In the MS-DOS world, every
compiler has incorporated at least some of the ANSI features and many are
coming close to full compliance with the standard.
The committee began with the classic "White Book" of C, Brian Kernighan and
Dennis Ritchie's The C Programming Language, also known by the author's
initials of K&R. This thin white book defined the C language and was used by
both implementors and users of C. Unfortunately, it was an incomplete
standard. Many decisions about the language's facilities were left up to the
compiler implementor. As C became more popular, it became clear that certain
aspects of the language caused problems when debugging or porting programs.
Because of these factors, different C compilers included disparate extensions
and features.
In designing a standard for C, the ANSI committee had several goals in mind.
First, they wanted to maintain compatibility with K&R. This meant minimizing
changes that would invalidate existing K&R-compatible programs. In addition,
they wanted to codify and include common language extensions and programming
practices. Finally, they wanted to improve the utility and portability of the
language. This was not an easy task.
In the end they did an admirable job. While some old-line C programmers may
grumble about having "their" language changed, the end result is that the ANSI
standard is compatible with K&R and it enhances the language.
This article is not designed to be a complete tutorial on ANSI C. Its goal is
to explain how ANSI has added to C, and what changes may affect existing
programs and programming practices. Several new facilities have been added to
the language and can be exploited to improve program functionality,
readability, and maintainability. In spite of the committee's best efforts,
there are some subtle changes that may occasionally catch the programmer who
is unaware of them.
The article is divided into three sections that focus on the preprocessor, the
language, and the standard library. In each section I'll discuss subtle
changes and additions to that part of the language. Subtle changes are those
that may "sneak" up on the unsuspecting programmer. Additions are new ANSI
features that the programmer should be aware of.


Preprocessor


The preprocessor was changed significantly, and some of these changes can lead
to problems when porting code between ANSI and K&R compilers. Many compiler
vendors have not yet fully implemented the ANSI changes for fear that they
would break existing code.
A new preprocessor operator has been added: defined(name). Supplied with the
name of a macro, defined( ) returns a true value if that macro has been
defined, or false if it has not. This duplicates the function of the ifdef
directive but allows for the use of Boolean operators for checking the
definition of multiple macros.
Two more preprocessor operators are new with ANSI: # and ## which affect the
replacement of tokens in the replacement lists of macros. When a parameter in
the replacement list of a function-like macro is preceded by a # operator, the
corresponding argument is inserted at that point as a string literal. The ##
operator concatenates adjacent tokens in the replacement list; these
newly-built tokens are then available for further macro replacement.
Due to operating system differences, the ANSI standard does not define a
specific search method for a file specified in a #include directive. K&R used
Unix-style directory structures and file names when specifying how files were
located, but many operating systems do not have similar or equivalent
facilities.
Under K&R it was possible to define the same macro in several places. How this
was handled was implementation-dependent -- most compilers merely used the
last definition encountered. This practice can lead to problems, especially
when the definitions are stored in header files. ANSI's solution was to
disallow any macro redefinitions, unless they were identical.
K&R allowed for recursive macros that contained their own names in their
expansion. This was a dangerous practice, leading to run-away preprocessing
and logic problems. ANSI has made recursive macros illegal; the macro's own
definition is ignored while it is being expanded. This change can cause
problems in code, which relied upon recursive macros.
Two new directives were added: #error is used to output an error message
during preprocessing. The standard strongly suggests that #error should also
halt compilation. To pass configuration information to the compiler the
#pragma directive was invented. Whatever follows the #pragma is interpreted in
an implementation-defined manner. Any #pragmas that are not understood by the
compiler are ignored.
Another facility added was the #elif (for else if) directive. It eliminates
many of the massive if ... endif structures found in some programs.
Some compilers allowed white space around the #, introducing a preprocessor
statement, but others did not. ANSI determined that the white space didn't
interfere with anything, so they decided to allow it. This means that white
space can exist on either side of the # and can be used to format nested
preprocessor statements.


Language


The C language itself has undergone many changes. Most of the changes are
additions, but some are clarifications of ambiguities in K&R or changes that
may affect programming practices.
The keywords entry, fortran, and asm have been deleted by ANSI. No one ever
used entry, and both fortran and asm were considered non-portable. ANSI does
list fortran and asm in the common extensions, though.
An ANSI-conformant compiler must allow for case-sensitive internal
identifiers, which can be unique through their 31st character. The original C
compilers could only handle six to eight character identifiers. Because of
linkages to other languages and older hardware architectures, however,
external names are only required to be six characters long and are not case
sensitive. In a future ANSI standard, the restriction on external names will
be lifted. Programmers should be aware of this change because some older
programs may rely upon a lesser number of significant characters in an
internal identifier.
One area in which several changes have occurred is the declaration and
definition of functions. Using a concept from C++, the ANSI committee added
function prototypes to C. A prototype is a function declaration that defines
the types of parameters for a function. For example
 int func1(char * str, unsigned int 1amount);
This statement declares that the function func1( ) returns an int and accepts
two parameters: a char pointer and an unsigned integer. The identifiers str
and amount are optional; their inclusion can improve the understanding of the
nature of the function. When the compiler processes a call to func1( ), it can
check to see if the types of the actual arguments match those specified in the
prototype. This check prevents many common C errors, which occur when
incorrect data types are passed to functions. It should be noted that this
type checking can be circumvented through the use of proper casts.
Prototypes have made minor changes in how some parameters work. In K&R C,
float values were automatically promoted to doubles when passed as parameters.
Function prototypes can be used to force a float to be passed as a float. With
the new rules that allow floats to be used in calculations without being
promoted to doubles, library functions can be created to work entirely with
floats. This eliminates the possible overhead of doubles when the extra
precision is not needed.
The prototype style can also be used in function definition headers. This
makes the headers resemble those from a Pascal or Modula-2 program. Under the
new style func1( ) would be defined as
 int func1(char * str, unsigned int amount)
 {
 /* program code */
 }
whereas under the old style (still acceptable under ANSI) it would have been
written as
 int func1(str, amount)
 char * str;
 unsigned int amount;
 {
 /* program code */

 }
Under ANSI, if the parameter list in a function prototype ends with ellipses,
that function can accept a variable parameter list. This function
 int func2(int val,...);
accepts one int parameter (type-checked) and an implementation-defined number
of untyped parameters. The macros defined in stdarg.h are used to extract the
untyped parameters.
A function prototyped with an empty parameter list can accept any number of
parameters of any type, and calls to it are not type-checked. You should be
aware that many ANSI compilers enforce proper prototyping through warnings and
errors. It is to your advantage to use the type-checking capabilities found in
ANSI C.
ANSI has added some new types. The most significant of these additions is the
void type. void represents the NULL set, in other words, an object of type
void has no values. While this might seem to be a rather senseless type it
does have some important uses.
For example, a function that does not return a value can be defined with a
return value of void. This eliminates the old problem in K&R where these
functions implicitly returned a random int, the value of which was
meaningless. Also, a function can be prototyped with void in its parameter
list, indicating that the function does not accept any parameters.
A void pointer is a pointer to anything. K&R C used char pointers for generic
pointers. Any type of pointer can be assigned to a void * -- and vice versa --
without a cast. void pointers can be used to make functions that access
generic-area memory more conveniently. Under K&R C, malloc( ) returned a char
* pointer, which then needed to be cast to the pointer type it was assigned
to. The ANSI version of malloc( ) returns a void *thereby eliminating the need
for a cast.
The keyword signed has been added by ANSI. This was in response to the need to
explicitly declare a signed char (or in this case, a very small integer) type
on implementations where the default char type was unsigned. To keep the
language consistent, signed is now allowed as a qualifier for all integral
types. ANSI also made official the ability to apply the unsigned qualifier to
long and short types.
A new floating-point type is the long double. It must be at least as precise
as a double. Some MS-DOS implementations have already included this type in
10-byte IEEE format. long float is, however, no longer a valid synonym for
double.
const and volatile are two new qualifiers. Adopted from C++, the const
qualifier locks in the value of a data item, preventing its value from being
modified. For example,
 const double pi = 3.1415927;
would prevent the value of pi from being changed. When used in a function
prototype and definition, the const qualifier prevents the function from
changing that argument. Several ANSI prototypes use const to safeguard the
data accessed through pointer arguments.
Under some circumstances, a data item may be changed by forces outside the
scope of a program. The volatile keyword was invented to inform the compiler
that a value may change asynchronously. This is required for optimizing C
compilers that can make assumptions about the value of a data item; volatile
prevents these assumptions from being made.
The enumeration type has been in common use for years in some C dialects.
Designated by the enum keyword, an enumerated type defines a special set of
related integer values. For example
 enum rank (first, second, third);
 enum rank my_rank;

 my_rank = first;
The value of type rank can be one of three possible values, first, second, and
third. first has an integral value of 0, second a value of 1, and third a
value of 2. An enumerated value can be used anywhere an int can. The compiler
can make checks to be sure that a value of type rank is assigned only those
values listed in the definition of rank (known as enumeration constants).
By default, enumeration constants are assigned consecutive integral values
beginning with zero. Explicit assignments can be made, however, to preset the
values of the constants. The enumeration type coin shows this facility:
 enum coin (penny=1, nickel=5, dime=10, quarter=25);
Several changes have been made in numeric constants. Integral constants are
automatically stored as int, long, or unsigned long values. The smallest
integral type that can hold the constant is used. A new constant suffix, U(or
u), can be used to specify that a constant is unsigned. As in K&R, a suffix of
L(or l) can be used to force the constant into a long value.
Floating point constants are automatically stored as doubles, unless the
constant has an F (or f) suffix. The F suffix forces the constant to be stored
as a float.
Many pre-ANSI implementations of C extended the uses of structured types.
These additions have been adopted in the ANSI standard, and structures may now
be passed in function parameters by value. K&R did not allow this, and all
structures had to be referenced by pointer parameters. In addition, ANSI
allows functions to have structures as return values, and assignments can be
made between structures of the same type.
Early C compilers allowed both the <operator>= and = <operator> forms of the
short-cut assignment operators. The latter form is somewhat ambiguous because,
for example, the = operator may indicate either subtraction or the assignment
of a negative value. Therefore, the ANSI committee has specified the
<operator>= format as the only valid forms of these operators.
You will find that there are several defined types in ANSI C. These types were
created to aid in portability. The K&R version of the sizeof operator returned
an int; ANSI's sizeof returns a value defined as size_t. size_t is defined as
an implementation-specific integral type.
K&R defined a loose relationship between pointers and integers. Pointer values
could be assigned to integers and back again. Comparisons between pointers and
integers were allowed but the results from such comparisons varied from
implementation to implementation.
The ANSI standard no longer defines integers and pointers as interchangeable.
The only integral value that has any validity when compared to a pointer is 0.
In fact, the standard explicitly defines a NULL pointer as having an integral
value of 0. Programs that rely upon pointer arithmetic and point integer
conversions may not be ANSI compatible.
The const qualifier can be used to control changes to the pointer and the data
it points to. A const int*, for example, would be a pointer to a constant
integer. The pointer can be changed but not the value it points to. The
declaration int * const denotes an unmodifiable pointer to a changeable
integer value.
The only significant change made by ANSI to executable statements affects the
switch statement. Under K&R, only int values were allowed for the controlling
expression, and now they may be of any integral type, including long and
unsigned values.


Library


K&R did not specify a complete library for C. It discusses the standard I/O
functions commonly found in stdio.h and several functions from the Unix
library. The latter primarily consists of low-level I/O functions, which have
been dropped from the ANSI standard for portability reasons. An example of a
dynamic memory allocator is included but malloc( ) and company are absent. No
math or string functions are discussed.
In order to promote portable programs, ANSI created a standard library and a
standard set of headers to go with it. There are a total of 15 ANSI headers.
Some functions that were defined by K&R in stdio.h are now found in other
headers like stdlib.h. The following is a list of the headers and a short
description of the functions found in them.
assert.h -- Diagnostics macro
The assert macro is commonly defined for most existing C implementations. It
is used to place diagnostic tests in programs. assert accepts a single
parameter. When the parameter is false (a zero value), assert displays the
name of the source file and the current line number to the standard error
device. The abort( ) function is then called.
ctype.h -- Character handling
These functions not only test characters to see if they are in a certain range
but also change the case of certain characters. For example, the isalpha( )
function tests to see if its parameter is an alphabetic character. toupper( )
converts a lowercase letter to an uppercase one. The locale (see locale.h
later in this text) setting can affect how these functions work. In K&R, these
facilities were defined in stdio.h.
errno.h -- Standard errors
Several library functions return error code in a value called errno, which is
defined in this header along with its possible values. These error values are
highly implementation-dependent.
float.h -- Floating-point limits
There are several macros in this header that expand to values for limits and
ranges for floating-point values.
limits.h -- Integer limits
This header is similar to float.h but defines limits and ranges for integral
values.
locale.h -- Localization
In making C an international language, it became clear that a mechanism was
needed to allow country- or location-specific information to be set. Things
such as decimal point characters and currency characters change from place to
place. The macros, structures, and functions found in locale.h help make
writing portable programs easier.
math.h -- Mathematics
Mathematical functions have always been a part of C but until ANSI they were
largely implementation-defined. This header declares functions for
trigonometric, hyperbolic, logarithmic, and utility operations.
setjmp.h -- Non-local jumps
Using the setjmp( ) macro and the longjmp( ) function, a program can literally
jump from any location within itself to any other. While the capability may
violate the principles of structured program design, it can be useful when an
exception condition occurs in a deeply-nested portion of a program.
signal.h -- Signal handling
A signal is an exception condition. The functions signal( ) and raise( )
provide a portable method for handling exceptions, such as program breaks and
floating-point errors. These ANSI functions are based on those defined for
Unix. raise( ) replaces the Unix kill( ) and eliminates the latter's support
for multiprocessing. ANSI implemented a subset of the signals defined for Unix
and allows each implementation to define signals of their own.
stdarg.h -- Variable arguments
Defined in this header are a set of macros that can be used to write portable
functions accepting variable numbers of parameters. Many compilers provide a
different set of macros based on the Unix V compile. In my experience, the
ANSI macros are clearer and easier to use than the Unix macros.
stddef.h -- Common definitions

ANSI defined a number of standard types and macros most of them are here and
some may be repeated in other headers as well. Among the definitions in
stddef.h are: NULL, size_t(the return value type of the sizeof operator), and
the offsetof macro. An invention of ANSI, offsetof is a macro that determines
the byte-offset of a member within a structure.
stdio.h -- output
This header defines and prototypes most of the functions listed in Chapter 7
of K&R. Few, if any, changes were made in the definitions found in K&R but
several new functions have been added. There are now functions for working
with temporary files (with random, unique names). remove( ) deletes files and
replaces the Unix-specific unlink( ). Using setvbuf( ) (borrowed from Unix V),
the programmer can control the buffering of I/O streams. Two new file
positioning functions, fgetpos( )and fsetpos( ), have been added to handle
files longer than fseek( ) can cope with.
stdlib.h -- General utilities
This is the "kitchen sink" header, containing definitions and prototypes for a
wide variety of unrelated functions. Included are numeric-to-string
conversions: the malloc( ) family of memory allocation functions, random
number functions, abort and exit facilities, binary search and quicksort
functions, multibyte character functions (for working with foreign alphabets),
and miscellaneous integer functions. Many existing compilers have their own
headers for these functions, and you may need to determine how many of the
non-standard headers are still required.
string.h -- String handling
Here is where you'll find the functions for manipulating strings. Most of
these functions have been around as long as C has, and several are defined in
K&R. Also included in the header are prototypes for the memory functions,
which work similar to the string copy/move/set functions. The memory functions
are designed for modifying memory through generic (non-character) pointers.
time.h -- Date and time
While date and time functions exist in most C libraries, there is almost no
standardization between implementations. ANSI designed the functions declared
in this header to provide a full spectrum of date and time facilities.
Conversion functions are provided to convert integral times to structures to
text strings.
Converting to ANSI may mean making minor changes in the library facilities you
use and how you use them.


Conclusions


I hope that this overview gives you a sense of the effects the ANSI standard
will have on your programming style and existing applications. Converting old
programs to ANSI C should not be an arduous task. We should all be grateful to
the X3J11 committee for the work that they have done; their standard will help
move C into the 1990s.


BENCHMARKING TURBO C AND QUICKC
by Scott Robert Ladd

[GRIND.C]

/*
 Program: Grind

 Version: 1.11
 Date: 26-Oct-1988

 Language: ANSI C

 Tests all aspects of a C compiler's functions, including disk i/o,
 screen i/o, floating point, recursion, prototyping, and memory
 allocation. It should be a large enough program to test the advanced
 optimizers in some compilers.

 Developed by Scott Robert Ladd. This program is public domain.
*/

#include <stdio.h>
#include <stdlib.h>
#include <math.h>

#define MAXFLOATS 1000

struct tabent
 {
 double val, vsum, vsqr, vcalc;
 };

struct tabent table[MAXFLOATS];

char *errmsg[] = {
 "GRIND.TBL cannot be created",
 "GRIND.TBL cannot be closed properly",
 "GRIND.IN cannot be found",
 "GRIND.IN has been truncated",
 "GRIND.IN cannot be closed properly"
 };


/* function prototypes */
short main(void);
void readfloats(void);
void sortfloats(void);
void quicksort(struct tabent *, short, short);
void maketable(void);
void writetable(void);
void error(short);

short main(void)
 {
 puts("\nGrind (C) v1.10 -- A Benchmark Program\n");

 readfloats();

 sortfloats();

 maketable();

 writetable();

 puts("\7End run GRIND!!!");

 return(0);
 }

void readfloats()
 {
 register short i;
 char buf[12];
 FILE *fltsin;

 printf("--> Reading in floats. At # ");
 if (NULL == (fltsin = fopen("GRIND.IN","r")))
 error(2);

 for (i = 0; i < MAXFLOATS; ++i)
 {
 printf("\b\b\b\b\b%5d",i);
 if (NULL == fgets(buf,12,fltsin))
 error(3);
 table[i].val = atof(buf);
 }

 if (fclose(fltsin))
 error(4);

 printf("\n");
 }

void sortfloats()
 {
 puts("--> Sorting data");
 quicksort(table,0,MAXFLOATS-1);
 }

void quicksort(struct tabent * item,
 short left,

 short right)
 {
 register short i, j;
 struct tabent x, y;

 i = left;
 j = right;
 x = item[(i+j)/2];

 do
 {
 while (item[i].val < x.val && i < right) i++;
 while (x.val < item[j].val && j > left) j--;
 if (i <= j)
 {
 y = item[i];
 item[i] = item[j];
 item[j] = y;
 i++;
 j--;
 }
 }
 while (i <= j);

 if (left < j) quicksort(item,left,j);
 if (i < right) quicksort(item,i,right);
 }

void maketable()
 {
 register short i;
 double sum = 0.0;

 puts("--> Calculating table values");

 for (i = 0; i < MAXFLOATS; ++i)
 {
 sum = sum + table[i].val;
 table[i].vsum = sum;
 table[i].vsqr = table[i].val * table[i].val;
 table[i].vcalc = sqrt(fabs(table[i].val)) * log10(fabs(table[i].val));
 }
 }

void writetable()
 {
 FILE *fd;
 register short i;

 if (NULL == (fd = fopen("GRIND.TBL","w+")))
 error(0);

 puts("--> Writing Table to File");

 for (i = 0; i < MAXFLOATS; i = i + 10)
 {
 fprintf(fd,
 "val = %5.2f, sum = %5.2f, sqr = %5.2f, calc = %5.2f\n",
 table[i].val,

 table[i].vsum,
 table[i].vsqr,
 table[i].vcalc);
 }

 if (fclose(fd))
 error(1);

 }

void error(short err_no)
 {
 printf("\n\7GRIND ERROR: %s\n",errmsg[err_no]);
 exit(err_no);
 }

[DMATH.C]

/*
 Benchmark: DMath
 Version: 1.00 23-Jan-1989

 Language: ANSI C

 Computes all of the sines of the angles between 0 and 360 degrees using
 doubles.

 Developed by Scott Robert Ladd. This program is public domain.
*/

#define dabs(a) (((a) < 0.0) ? (-(a)) : (a))

/* conversion factor for converting radian to degrees */

#define deg2rad 57.29577951

/* prototypes */
void main(void);
double fact(double);
double power(double, double);

void main()
 {
 double angle, radians, sine, worksine, temp, k;

 for (angle = 0.0; angle <= 360.0; angle += 1.0)
 {
 radians = angle / deg2rad;
 k = 0.0;
 worksine = 0.0;

 do {
 sine = worksine;
 temp = (2.0 * k) + 1.0;
 worksine += (power(-1.0,k) / fact(temp)) * power(radians,temp);
 k += 1.0;
 }
 while (dabs(sine - worksine) > 1E-8);
 }

 }

/* Note: this function is designed for speed; it ONLY works when n is integral
*/
double fact(double n)
 {
 double res;

 res = 1.0;

 while (n > 0.0)
 {
 res *= n;
 n -= 1.0;
 }

 return res;
 }

/* Note: this function is designed for speed; it ONLY works when p is integral
*/
double power(double n, double p)
 {
 double res;

 res = 1.0;

 while (p > 0.0)
 {
 res *= n;
 p -= 1.0;
 }

 return res;
 }


[FXREF.C]

/*
 Program: FXREF (File Cross-Reference)

 Version: 1.10
 Date: 21-Sep-1988

 Language: ANSI C

 Reads a file from standard input, and sorts and organizes each
 token (word) found using a binary tree, keeping track of the number
 of occurences of each token and their location by line and column.
 It then prints a report to stdout.

 Released into the public domain for "educational" purposes.
*/

#include "stdio.h"
#include "string.h"
#include "ctype.h"
#include "stdlib.h"

/* type definitions */

typedef unsigned short line_no;

typedef struct loc_s
 {
 line_no line;
 struct loc_s * next;
 }
 location;

typedef struct tok_s
 {
 struct tok_s * less, * more;
 char * text;
 struct loc_s *loc, *last;
 }
 token;

token * root;

char * err_msg[] = {
 "token table root",
 "token text",
 "location references",
 "token record"
 };

/* function prototypes */
int main(void);
void parse_tokens(char *, line_no);
void add_tree(char *, line_no);
token * find_tree(char *);
void show_tree(token *);
void error(short);

int main()
 {
 char buf[256];
 line_no line=0;

 if (NULL == (root = ( token *)malloc(sizeof( token))))
 error(0);

 root->less = NULL;
 root->more = NULL;
 root->text = NULL;
 root->loc = NULL;

 while (NULL != (fgets(buf,256,stdin)))
 {
 ++line;
 printf("%5u: %s",line,buf);
 parse_tokens(buf,line);
 }

 printf("\x0C\n");

 show_tree(root);

 return 0;

 }

void parse_tokens(char * buf, line_no line)
 {
 char tok[256];
 line_no pos;

 while (1)
 {
 while ((!isalpha(*buf)) && (*buf != 0))
 ++buf;
 if (*buf == 0)
 return;
 pos = 0;
 while (isalpha(*buf))
 tok[pos++] = *buf++;
 tok[pos] = 0;
 add_tree(tok,line);
 }
 }

void add_tree(char * tok, line_no line)
 {
 token *temp_tok, *new_tok;
 location *temp_loc;
 short comp;

 if (root->text == NULL)
 {
 if (NULL == (root->text = (char *)malloc((unsigned)strlen(tok)+1)))
 error(1);
 strcpy(root->text,tok);
 if (NULL == (root->loc = ( location *)malloc(sizeof( location))))
 error(2);
 root->loc->line = line;
 root->loc->next = NULL;
 root->last = root->loc;
 return;
 }

 temp_tok = find_tree(tok);

 if (comp = strcmp(tok,temp_tok->text))
 /* comp is true (non-zero) if they don't match */
 {
 if (NULL == (new_tok = ( token *)malloc(sizeof( token))))
 error(3);
 if (NULL == (new_tok->text = (char *)malloc((unsigned)strlen(tok)+1)))
 error(1);
 new_tok->less = NULL;
 new_tok->more = NULL;
 strcpy(new_tok->text,tok);
 if (NULL == (new_tok->loc = ( location *)malloc(sizeof( location))))
 error(2);
 new_tok->loc->line = line;
 new_tok->loc->next = NULL;
 new_tok->last = new_tok->loc;
 if (comp < 0)
 temp_tok->less = new_tok;

 else
 temp_tok->more = new_tok;
 }
 else
 /* if comp is false (0), the tokens match */
 {
 if (NULL == (temp_loc = ( location *)malloc(sizeof( location))))
 error(2);
 temp_loc->line = line;
 temp_loc->next = NULL;
 temp_tok->last->next = temp_loc;
 temp_tok->last = temp_loc;
 }
 }

 token *find_tree(char * tok)
 {
 short comp;
 token *node;

 node = root;

 while (1)
 {
 if (0 == (comp = strcmp(tok,node->text)))
 return node;
 if (comp < 0)
 if (node->less == NULL)
 return node;
 else
 node = node->less;
 else
 if (node->more == NULL)
 return node;
 else
 node = node->more;
 }
 }

void show_tree(token * node)
 {
 location *lloc;
 short pos;

 if (NULL == node) return;

 show_tree(node->less);
 printf("%-32s: ",node->text);
 pos = -1;
 lloc = node->loc;
 while (lloc != NULL)
 {
 if (++pos == 7)
 {
 pos = 0;
 printf("\n%32s: "," ");
 }
 printf("%5d ",lloc->line);
 lloc = lloc->next;

 }
 printf("\n");
 show_tree(node->more);
 }

void error(short err_no)
 {
 printf("\nFXREF ERROR: Cannot allocate space for %s\n",err_msg[err_no]);
 exit(err_no+1);
 }

[GRAPH_QC.C]

/*
 Benchmark: GRAPH_QC
 Version: 1.00 29-Jan-1989

 Language: Microsoft QuickC v2.00

 Purpose: Tests the speed of QuickC's graphics functions.

 Written by Scott Robert Ladd. Released into the public domain.
*/

#include "time.h"
#include "stdio.h"
#include "graph.h"
#include "stddef.h"

short main(void);
void initialize(void);
void draw_lines(void);
void draw_ellipses(void);
void draw_and_fill(void);
void finalize(void);

short max_x, max_y;
short mid_x, mid_y;

struct videoconfig scrn_cfg ;

clock_t start;
float line_time, ellipse_time, fill_time;

short main()
 {
 initialize();

 draw_lines();
 draw_ellipses();
 draw_and_fill();

 finalize();

 return 0;
 }

void initialize()
 {

 _setvideomode(_ERESCOLOR);

 _getvideoconfig(&scrn_cfg);

 max_x = scrn_cfg.numxpixels - 1;
 max_y = scrn_cfg.numypixels - 1;

 mid_x = max_x / 2;
 mid_y = max_y / 2;
 }

void draw_lines()
 {
 short i, x, y;

 start = clock();

 for (i = 0; i <= 5; ++i)
 {
 if ((i % 2) == 0)
 _setcolor(_BRIGHTWHITE);
 else
 _setcolor(_BLACK);

 for (x = 0; x <= max_x; x += 4)
 {
 _moveto(x,0);
 _lineto(max_x - x,max_y);
 }

 for (y = 0; y <= max_y; y += 2)
 {
 _moveto(max_x,y);
 _lineto(0,max_y - y);
 }
 }

 line_time = (clock() - start) / CLK_TCK;

 _clearscreen(_GCLEARSCREEN);
 }

void draw_ellipses()
 {
 short x, y;

 _setcolor(_BRIGHTWHITE);

 start = clock();

 for (x = 6; x < mid_x; x += 6)
 for (y = 10; y < mid_y; y += 10)
 _ellipse(_GBORDER, mid_x - x, mid_y - y, mid_x + x, mid_y + y);

 ellipse_time = (clock() - start) / CLK_TCK;

 _clearscreen(_GCLEARSCREEN);
 }


void draw_and_fill()
 {
 short i, color;

 _moveto(0,0);

 _lineto(20,0);
 _lineto(30,20);
 _lineto(10,40);
 _lineto(10,50);
 _lineto(100,50);
 _lineto(100,52);
 _lineto(50,52);
 _lineto(50,54);
 _lineto(102,54);
 _lineto(102,10);
 _lineto(630,120);
 _lineto(500,150);
 _lineto(620,180);
 _lineto(510,200);
 _lineto(630,250);
 _lineto(400,340);
 _lineto(5,300);
 _lineto(0,0);

 _setfillmask(NULL);

 start = clock();

 for (i = 0; i < 4; ++i)
 {
 for (color = 1; color < 15; ++color)
 {
 _setcolor(color);
 _floodfill(mid_x, mid_y, _BRIGHTWHITE);
 }
 }

 fill_time = (clock() - start) / CLK_TCK;

 _clearscreen(_GCLEARSCREEN);
 }

void finalize()
 {
 _setvideomode(_DEFAULTMODE);

 printf("line time = %.1f\n",line_time);
 printf("ellipse time = %.1f\n",ellipse_time);
 printf("fill time = %.1f\n",fill_time);
 }


[GRAPH_TC.C]

/*
 Benchmark: GRAPH_TC
 Version: 1.00 29-Jan-1989


 Language: Borland Turbo C v2.0

 Purpose: Tests the speed of QuickC's graphics functions.

 Written by Scott Robert Ladd. Released into the public domain.
*/

#include "time.h"
#include "stdio.h"
#include "graphics.h"

short main(void);
void initialize(void);
void draw_lines(void);
void draw_ellipses(void);
void draw_and_fill(void);
void finalize(void);

short max_x, max_y;
short mid_x, mid_y;

clock_t start;
float line_time, ellipse_time, fill_time;

int driver = EGA,
 mode = EGAHI;

short main()
 {
 initialize();

 draw_lines();
 draw_ellipses();
 draw_and_fill();

 finalize();

 return 0;
 }

void initialize()
 {
 initgraph(&driver, &mode, "D:\\TC\\BGI");

 max_x = 639;
 max_y = 349;

 mid_x = max_x / 2;
 mid_y = max_y / 2;
 }

void draw_lines()
 {
 short i, x, y;

 start = clock();

 for (i = 0; i <= 5; ++i)
 {

 if ((i % 2) == 0)
 setcolor(WHITE);
 else
 setcolor(BLACK);

 for (x = 0; x <= max_x; x += 4)
 {
 moveto(x,0);
 lineto(max_x - x,max_y);
 }

 for (y = 0; y <= max_y; y += 2)
 {
 moveto(max_x,y);
 lineto(0,max_y - y);
 }
 }

 line_time = (clock() - start) / CLK_TCK;

 cleardevice();
 }

void draw_ellipses()
 {
 short x, y;

 setcolor(WHITE);

 start = clock();

 for (x = 6; x < mid_x; x += 6)
 for (y = 10; y < mid_y; y += 10)
 ellipse(mid_x, mid_y, 0, 360, x, y);

 ellipse_time = (clock() - start) / CLK_TCK;

 cleardevice();
 }

void draw_and_fill()
 {
 short i, color;

 moveto(0,0);

 lineto(20,0);
 lineto(30,20);
 lineto(10,40);
 lineto(10,50);
 lineto(100,50);
 lineto(100,52);
 lineto(50,52);
 lineto(50,54);
 lineto(102,54);
 lineto(102,10);
 lineto(630,120);
 lineto(500,150);
 lineto(620,180);

 lineto(510,200);
 lineto(630,250);
 lineto(400,340);
 lineto(5,300);
 lineto(0,0);

 start = clock();

 for (i = 0; i < 4; ++i)
 {
 for (color = 1; color < 15; ++color)
 {
 setfillstyle(SOLID_FILL,color);
 floodfill(mid_x, mid_y, WHITE);
 }
 }

 fill_time = (clock() - start) / CLK_TCK;

 cleardevice();
 }

void finalize()
 {
 closegraph();

 printf("line time = %.1f\n",line_time);
 printf("ellipse time = %.1f\n",ellipse_time);
 printf("fill time = %.1f\n",fill_time);
 }
































August, 1989
A GENERIC HEAPSORT ALGORITHM IN C


Heapsort is an excellent sorting algorithm for many applications




Stephen Russell


Stephen is a member of the Basser Department of Computer Science at the
University of Sydney. He can be reached at the Madsen Building, F09,
University of Sydney, N.S.W., Australia 2006.


When faced with the problem of sorting data, many programmers choose a simple
algorithm, such as insertion or selection sort, or maybe Shellsort. Quicksort
is another common choice, at least for languages that support recursion: The
qsort library function makes this choice particularly easy for C programmers.
Surprisingly few programmers choose the Heapsort algorithm, perhaps because it
is not an intuitive algorithm: The other algorithms are certainly easier to
understand. Heapsort, however, has some advantages over its competitors.
With Heapsort, the time taken to sort n items is proportional to n log n,
regardless of the initial order of the data. In comparison, although Quicksort
has an average performance of n log n, some data cause it to execute in time
proportional to n{2}. The other algorithms also have a worst-case performance
of n{2}. When sorting large amounts of data, the difference between a
guaranteed speed of n log n versus a worst-case of n{2} is significant.
In addition, Heapsort is not a recursive algorithm, making it suitable for
languages--such as Basic and Fortran -- that do not support recursion. Even
for languages, such as Pascal and C, eliminating the recursion may lead to a
faster algorithm, depending on the cost of procedure calls. The cost of saving
and restoring registers for each call may be significant on some machines.
Heapsort, however, has its disadvantages. For small sets of data, the simpler
algorithms may actually perform better, due to their lower overheads. Choosing
the right algorithm for a particular application requires careful analysis and
knowledge of the tradeoffs involved. Books on algorithm design, such as
Sedgewick's Algorithms (Addison-Wesley, 1983) and the Gonnet's Handbook of
Algorithms and Data Structures (Addison-Wesley, 1984), provide useful guides
to help make the right choice.
Even with these caveats, Heapsort is a strong candidate for inclusion in a
programmer's library of standard tools. The rest of this article describes a
generic Heapsort function, modelled on the C qsort function. This function
allows an arbitrary array of data to be sorted, when provided with a function
to compare elements of the array.


How Heapsort Works


The Heapsort algorithm is based on an interesting data structure -- the heap.
Heaps are typically used for applications in which the maximum or minimum
value is needed from a changing collection of data. When an item is added to
or removed from a heap, the heap can be rearranged to find the new maximum (or
minimum) value in log n steps. This process is a very efficient way to
implement priority queues, and it forms the basis of Heapsort.
A heap is a partially ordered array of data, organized as a special form of
binary tree, with the following properties. First, for each node in the tree,
the node's value is greater than the values of either of its child nodes,
which implies that the root of the tree contains the maximum value of all the
nodes. If the minimum value is needed instead, each node must have a lesser
value than its children. Second, the tree is fully balanced, and each level of
the tree is filled from left to right. This property allows the tree to be
constructed using an array, rather than explicit pointers. If the first item
of a heap H is H[1], then its two children are H[2] and H[3]. Similarly, the
children of H[2] are H[4] and H[5]. In general, the children of H[i] are H[2i]
and H[2i+1], and the parent of H[k] is H[k/2].
Converting an array of arbitrary data to a heap involves rearranging the
values in the array until the ordering property of a heap is satisfied. This
conversion is usually realized by working through the tree from the bottom up.
For array H containing n items, start by considering the last item that could
have children -- H[k], where k is n/2. H[k] is compared to its children H[2k]
and H[2k+1]. If H[k] is less than one or both of its children, it is swapped
with the greater child, say H[j]. Then compare the new value in H[j] with its
two children, and perform another swap if needed. Eventually, you will reach a
node that has no children, or a spot where no swap is needed. At this point,
the subtree starting at H[k] is organized as a heap. This process is performed
for all values of k from n/2 to 1. After the last iteration, the array is a
heap, and H[1] contains the maximum value. Figure 1 shows the process of
constructing a heap from a collection of numbers. The tree structure of the
heap is shown in Figure 2.


The Heapsort Algorithm


Heapsort works in two phases. In the first phase, the data is rearranged in
the array to form a heap; this moves the maximum data value to the first
position of the array. In the second phase, the maximum value is swapped with
the last element in the heap, and the size of the heap is reduced by one. This
phase puts the maximum data value in its final position in the array. However,
the element now occupying the first position may not be the next maximum: The
next maximum is found by rearranging the remaining elements of the heap. The
process of finding the maximum value in the heap, and moving it to its final
position, continues until the heap is reduced to a single element. Figure 3
shows Heapsort in operation for the heap constructed in Figure 1.
Listing One shows an implementation of the Heapsort algorithm for sorting an
array of integer values. Most of the work is performed by the function call
fixheap(h,i,n), which ensures that the subheap starting at position h[i] is
correctly ordered. As I described previously, the value of i goes from n/2 to
1 while constructing the heap. In the second phase, the value of i is always
1, with the value of n decreasing as each maximum value is extracted from the
heap.
The fact that arrays in C are indexed from 0, rather than 1, is adjusted for
in the first statement in hsort( ) by decrementing the pointer to the base of
the array. This adjustment allows the rest of the algorithm to consider the
array as starting at h[1].


A Generic Heapsort


The hsort( ) function in Listing One can be easily changed to sort, for
example, arrays of float or char values. With a little extra effort, hsort( )
can also sort data that are accessed indirectly through an array of pointers,
or data with multiple sort keys. Having to modify the algorithm each time a
particular sort is needed, though, is inconvenient and error-prone. A generic
Heapsort is needed that can sort arbitrary data when provided with an
appropriate function to compare values. Such a function is shown in Listing
Two.
Modelled on the C qsort( ) library function, the version of hsort( ) shown in
Listing Two sorts an array of n items, with each element "size" bytes in
length. The cmp argument is a pointer to a comparison function. Each time
fixheap( ) needs to compare elements, it calls this function, passing the
addresses of the two items as arguments. The function then returns a value
less than, equal to, or greater than zero, depending on whether the first item
is less than, equal to, or greater than the second.
The version of hsort( ) in Listing Two is derived directly from the simple
version in Listing One. Unfortunately, each calculation of the address of an
element involves a multiplication by the size of the elements. On many
computers, multiplication instructions are slow. It is possible, however, to
rewrite the algorithm to eliminate these multiplications completely. This
leads to a significant speed improvement. A further increase in speed can be
gained by expanding the calls to fixheap( ) and swap( ) inline.
The final version of Heapsort is given in Listing Three. It is derived from
the algorithm in Listing Two by performing induction variable elimination, as
might be done by an optimizing compiler. The critical optimization is
calculating the distance in bytes between the addresses of the elements h[0],
h[i], and h[2i]. For a given value of i, the distance is i * size, where size
is the size of each element. Table 1 shows the relationship between the heap
elements and expressions for their addresses. The variable gap is the
calculated distance, and base0 is the address of the element h[0].
Table 1: The relationship between the heap elements and expressions

 Element Address
 -----------------------------------
 h[0] base0
 h[1] base0 + size
 h[i] base0 + gap
 h[2i] base0 + gap + gap
 h[2i+1] base0 + gap + gap + size

The next step is to note that as i decreases during the heap construction
phase, the value of gap is reduced by size for each iteration. The initial
value for gap is n * size / 2, and it is the only step where multiplication is
required. The address of h[n] is also calculated and stored in the variable
hi. During the second phase of the sort, the value of hi is decreased by size
for each iteration, which corresponds to decrementing n. Although these
optimizations result in a more complex algorithm, they are worthwhile given
its intended use as a standard library function.



Using hsort( )


Listing Four shows an example that uses hsort( ) to sort an array of strings,
which are read from input. Each element of the array is of type char *. The
cmp( ) function is passed two char ** pointers and uses strcmp( ) to compare
the strings. After the strings are sorted, they are written to standard
output.


Conclusion


The algorithms presented in this article cover a range of complexity and
versatility. For some applications, the simple algorithm given in Listing One
may perform best, and it provides a skeleton that can be extended for special
purposes. The generic Heapsort given in Listing Three provides a useful
library function, which can be used at any time data need sorting. Its
generality, however, does incur a small cost in performance compared to a
special-purpose sort. This cost will be acceptable for most applications.
Choosing the right approach is, of course, part of the craft of programming.


Availability


All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063; or call 800-356-2002 (from inside
Calif.) or 800-533-4372 (from outside Calif.). Please specify the issue number
and format (MS-DOS, Macintosh, Kaypro).


A GENERIC HEAPSORT ALGORITHM IN C
by Stephen Russell




[LISTING ONE]

/*
 * The Heapsort to sort an array of n integers.
 */

static
fixheap(h, i, n)
int *h;
unsigned i, n;
{
 unsigned k;
 int tmp;

 while ((k = 2 * i) <= n) /* h[k] = left child of h[i] */
 {
 /* Find maximum of left and right children */

 if (k != n && h[k+1] > h[k])
 ++k; /* right child is greater */

 /* Compare greater of children to parent */

 if (h[i] >= h[k])
 return;

 /* Parent is less than child, so swap */

 tmp = h[k]; h[k] = h[i]; h[i] = tmp;

 i = k; /* move down tree */
 }
}



hsort(h, n)
int *h;
unsigned n;
{
 unsigned i;
 int tmp;

 --h; /* adjust for zero-origin arrays in C */

 for (i = n/2; i > 1; --i)
 fixheap(h, i, n); /* build heap, except for h[1] */

 while (n > 1)
 {
 fixheap(h, 1, n); /* move max to h[1] */
 tmp = h[1]; /* move max to final position */
 h[1] = h[n];
 h[n] = tmp;
 --n; /* reduce size of heap */
 }
}




[LISTING TWO]


/*
 * Generic Heapsort, derived from listing 1.
 */


#define H(k) (h + k * size)

static
swap(p1, p2, n) /* swap n bytes */
char *p1, *p2;
unsigned n;
{
 char tmp;

 while (n-- != 0)
 {
 tmp = *p1; *p1++ = *p2; *p2++ = tmp;
 }
}

static
fixheap(h, size, cmp, i, n)
char *h;
unsigned size, i, n;
int (*cmp)();
{
 unsigned k;

 while ((k = 2 * i) <= n)
 {
 if (k != n && (*cmp)(H(k+1), H(k)) > 0)

 ++k;

 if ((*cmp)(H(i), H(k)) >= 0)
 return;

 swap(H(i), H(k), size);
 i = k;
 }
}

hsort(h, n, size, cmp)
char *h;
unsigned n, size;
int (*cmp)();
{
 unsigned i;

 h -= size;

 for (i = n/2; i > 1; --i)
 fixheap(h, size, cmp, i, n);

 while (n > 1)
 {
 fixheap(h, size, cmp, 1, n);
 swap(H(1), H(n), size);
 --n;
 }
}



[LISTING THREE]

/*
 * Generic Heapsort.
 *
 * Synopsis:
 * hsort(char *base, unsigned n, unsigned size, int (*fn)())
 * Description:
 * Hsort sorts the array of `n' items which starts at address `base'.
 * The size of each item is as given. Items are compared by the function
 * `fn', which is passed pointers to two items as arguments. The function
 * should return < 0 if item1 < item2, == 0 if item1 == item2, and > 0
 * if item1 > item2.
 * Version:
 * 1988 April 28
 * Author:
 * Stephen Russell, Department of Computer Science,
 * University of Sydney, 2006
 * Australia.
 */

#ifdef INLINE

#define swap(p1, p2, n) {\
 register char *_p1, *_p2;\
 register unsigned _n;\
 register char _tmp;\

\
 for (_p1 = p1, _p2 = p2, _n = n; _n-- > 0; )\
 {\
 _tmp = *_p1; *_p1++ = *_p2; *_p2++ = _tmp;\
 }\
}\

#else

/*
 * Support routine for swapping elements of the array.
 */

static
swap(p1, p2, n)
register char *p1, *p2;
register unsigned n;
{
 register char ctmp;

 /*
 * On machines with no alignment restrictions for int's,
 * the following loop may improve performance if moving lots
 * of data. It has been commented out for portability.

 register int itmp;

 for ( ; n > sizeof(int); n -= sizeof(int))
 {
 itmp = *(int *)p1;
 *(int *)p1 = *(int *)p2;
 p1 += sizeof(int);
 *(int *)p2 = itmp;
 p2 += sizeof(int);
 }

 */

 while (n-- != 0)
 {
 ctmp = *p1; *p1++ = *p2; *p2++ = ctmp;
 }
}

#endif

/*
 * To avoid function calls in the inner loops, the code responsible for
 * constructing a heap from (part of) the array has been expanded inline.
 * It is possible to convert this common code to a function. The three
 * parameters base0, cmp and size are invariant - only the size of the
 * gap and the high bound of the array change. In phase 1, gap decreases
 * while hi is fixed. In phase 2, gap == size, and hi decreases. The
 * variables p, q, and g are only used in this common code.
 */

hsort(base, n, size, cmp)
char *base;
unsigned n;

unsigned size;
int (*cmp)();
{
 register char *p, *q, *base0, *hi;
 register unsigned gap, g;

 if (n < 2)
 return;

 base0 = base - size; /* set up address of h[0] */

 /*
 * The gap is the distance, in bytes, between h[0] and h[i],
 * for some i. It is also the distance between h[i] and h[2*i];
 * that is, the distance between a node and its left child.
 * The initial node of interest is h[n/2] (the rightmost
 * interior node), so gap is set accordingly. The following is
 * the only multiplication needed.
 */

 gap = (n >> 1) * size; /* initial gap is n/2*size */
 hi = base0 + gap + gap; /* calculate address of h[n] */
 if (n & 1)
 hi += size; /* watch out for odd arrays */

 /*
 * Phase 1: Construct heap from random data.
 *
 * For i = n/2 downto 2, ensure h[i] is greater than its
 * children h[2*i] and h[2*i+1]. By decreasing 'gap' at each
 * iteration, we move down the heap towards h[2]. The final step
 * of making h[1] the maximum value is done in the next phase.
 */

 for ( ; gap != size; gap -= size)
 {
 /* fixheap(base0, size, cmp, gap, hi) */

 for (p = base0 + (g = gap); (q = p + g) <= hi; p = q)
 {
 g += g; /* double gap for next level */

 /*
 * Find greater of left and right children.
 */

 if (q != hi && (*cmp)(q + size, q) > 0)
 {
 q += size; /* choose right child */
 g += size; /* follow right subtree */
 }

 /*
 * Compare with parent.
 */

 if ((*cmp)(p, q) >= 0)
 break; /* order is correct */


 swap(p, q, size); /* swap parent and child */
 }
 }

 /*
 * Phase 2: Each iteration makes the first item in the
 * array the maximum, then swaps it with the last item, which
 * is its correct position. The size of the heap is decreased
 * each iteration. The gap is always "size", as we are interested
 * in the heap starting at h[1].
 */

 for ( ; hi != base; hi -= size)
 {
 /* fixheap(base0, size, cmp, gap (== size), hi) */

 p = base; /* == base0 + size */
 for (g = size; (q = p + g) <= hi; p = q)
 {
 g += g;
 if (q != hi && (*cmp)(q + size, q) > 0)
 {
 q += size;
 g += size;
 }

 if ((*cmp)(p, q) >= 0)
 break;

 swap(p, q, size);
 }

 swap(base, hi, size); /* move largest item to end */
 }
}






[LISTING FOUR]


/*
 * Use hsort() to sort an array of strings read from input.
 */

#include <stdio.h>


#define MAXN 500
#define MAXSTR 1000


cmp(p1, p2)
char **p1, **p2;
{
 return strcmp(*p1, *p2);

}

static char *string[MAXN];
static char buf[MAXSTR];

extern char *gets();
extern char *malloc();


main()
{
 char *p;
 int i, n;

 for (n = 0; gets(buf); ++n)
 {
 if (n == MAXN)
 {
 fprintf(stderr, "Too many strings\n");
 exit(1);
 }

 p = malloc(strlen(buf) + 1);
 if (p == (char *)NULL)
 {
 fprintf(stderr, "Out of memory\n");
 exit(2);
 }

 strcpy(string[n] = p, buf);
 }

 hsort(string, n, sizeof string[0], cmp);

 for (i = 0; i < n; ++i)
 puts(string[i]);

 exit(0);
}























August, 1989
BENCHMARKING TURBO C AND QUICKC


Wrapping up the Comparison




Scott Robert Ladd


Scott Ladd is a full-time, free-lance computer journalist. You can reach him
at 705 W Virginia, Gunnison, CO 81230.




Wrapping up the comparison


This month Scott Ladd reports the benchmark results as part of a follow-up on
the review he presented in the May issue: "QuickC versus Turbo C." In
addition, the Doctor examines CompuView's "VEdit Plus," Magna Carta's "C
Windows Toolkit," and Genus's "PCX Programming Toolkit."
This benchmark report is a follow-up to the comparative review of Microsoft's
QuickC 2.0 and Borland's Turbo C 2.0 that appeared in the May issue. That
article was a qualitative comparison. This one is quantitative and measures
the performance of the two under similar conditions. As I've said before and
I'll say again, no set of benchmarks can truly reveal how good a product is.
The only purpose benchmarks serve is to give a general idea of how well a
given compiler performs on certain types of code. (And speaking of code,
because of space constraints, I'm not including the programs used to generate
the results. If you'd like to see them, though, they'll be posted on DDJ's
CompuServe forum and are available through the DDJ editorial offices.)
For this comparison, I used five tests whose results appear in Table 1.
Dhrystone 2 is an updated version of the famous Dhrystone benchmark, which
represents the statistical "average" program. Because Dhrystone comes in three
files, it made an excellent test for the make facilities.
Table 1: QuickC versus Turbo C benchmark results

 Benchmark Test Borland Microsoft
 Turbo C 2.0 QuickC 2.0
 ----------------------------------------------

 timings (seconds)
 compile 11.80 12.42
 link 5.28 9.56
 run 19.45 18.67
 Dhrystones/second 2,571 2,678
 .EXE size (bytes) 9,692 11,405

 DMATH
 timings (seconds)
 compile 7.31 9.28
 link 7.90 10.82
 run
 emulator 170.21 194.76
 coprocessor 5.44 5.38
 .EXE size (bytes)
 emulator 20,546 17,578
 coprocessor only 10,706 9,786

 FXREF
 timings (seconds)
 compile 9.81 11.54
 link 5.16 9.84
 overall run 31.47 26.90
 .EXE size (bytes) 8,710 9,461

 GRAPH
 timings (seconds)
 compile 9.28 11.59
 link 10.16 23.20

 emulator run 158.85 53.61
 coprocessor run 158.52 53.28
 .EXE size (bytes)
 emulator 38,720 43,312
 coprocessor 28,864 35,520

 GRIND
 timings (seconds)
 compile 9.10 11.26
 link 7.91 15.38
 run
 emulator 28.10 30.43
 coprocessor 17.74 15.00
 .EXE size (bytes)
 emulator 25,976 27,340
 coprocessor 16,136 19,548

FXREF is a filter. It reads in a text file from standard input, and builds a
binary tree of the text tokens and their line-number references. Once the file
has been read and displayed, a cross reference is printed from the binary
tree. I have used FXREF in the past as a useful benchmark of I/O and dynamic
memory allocation speed.
Both compilers provide a graphics library, and thus was born the GRAPH
benchmark. It is actually a simple, three-part test. Part one tests
line-drawing speed, part two looks at the quickness of drawing ellipses, and
part three times the rapidity of filling an irregular area with a solid color.
Two separate programs were created, one for each compiler, due to the
differences in initialization code and function names.
GRIND is a report program. It reads 1000 floating point numbers from disk and
sorts them, then calculates a table of values, which it writes to disk.
Many floating point benchmarks test the speed of library functions more than
the quality of floating point code generated. DMATH solves this problem by
avoiding the use of any library functions. All variables in the program,
including loop counters, are doubles. DMATH calculates the sines of all of the
angles from 0 to 359 degrees, using a simple series.
The benchmarks were conducted on an Intel-motherboard 80386 machine running at
16 MHz. The computer was equipped with an 80387 coprocessor, 2 Mbytes of
memory, and an EGA card and monitor. Timing was performed by a program that
resets the interrupt timer to 1/100th of a second of accuracy, and each timed
test result is the average of five iterations. Tests were run for both the
emulator and coprocessor floating-point packages, whenever possible. All times
for compiles and links are for the command-line versions. QuickC's incremental
compilation and linking would speed it up immensely on developmental program
builds. Compile speeds were tested with all debug options and warnings turned
off.
The command-line options used for QuickC were -Ox -FPi. For Turbo C, the
options were -Z -0 -G -d. The -FPi87 flag was used with QuickC to generate the
coprocessor tests, and the -f87 flag was used for the same purpose with Turbo
C.
Some immediate trends can be seen by looking at the benchmark table. The
compile times for QuickC and Turbo C are very close together, but the Turbo C
linker runs much faster by far. It's important to note that these compiles are
for the entire modules; using QuickC's incremental compilation and linking
features make it considerably faster than Turbo C.
The most surprising run-time result is Turbo C's poor performance on the GRAPH
test. The ellipse() and flood fill() functions perform more than twice as
slowly as their Microsoft counterparts. Borland claims that "professional
programmers" use polyfill() -- which draws an automatically-filled polygon --
rather than floodfill(), but I don't agree with that logic. Most filling is
done in regions bounded by objects that may or may not be drawn in one piece.
Drawing conclusions from these tests can be risky. In general, Turbo C's
compile/link cycle is shorter than QuickC's (although, use of QuickC's
incremental compile/link would probably reverse this trend), but QuickC
programs tend to run a little faster. Nevertheless, the overall performance of
the two C compilers in these benchmarks is too close to reveal any clear
winner.
And that's good news for the vendors and -- even more important -- for us
programmers. No matter which product you choose, you can be assured that it is
of excellent quality.



































August, 1989
VEDIT PLUS


Professor T.A. Elkins


Professor T.A. Elkins is a consultant who specializes in strategy, policy, and
matters of wide-ranging academic interest.




Product Information


VEdit Plus, Version 3, CompuView Products, Inc., 1955 Pauline Blvd., Ann
Arbor, MI 48103; 313-996-1299. Requirements: 156 Kbytes of RAM, hard disk not
required. Supports: IBM PC, XT, AT, PS/2, and compatibles running under
MS-DOS, CP/M-86, FlexOS, Xenix, and OS/2. Also DESQview, MS Windows,
PC-MOS/386, and Concurrent DOS. Price: $185.
VEdit Plus, Version 3, can deal simultaneously with up to 37 files in multiple
windows, and can exchange material between all of the files and windows. It
provides multiple views of a single file, and a full-fledged text-programming
language which includes macros. VEdit Plus can record hundreds of undo levels,
provide on-line help for itself and a compiler or a collateral program, run a
compiler with hooks for competent error matching or run another child process,
check structure for items, such as the proper nesting of brackets, automate
graphic character drawings or illustrate the screen character set, and provide
a middling integer calculator -- all EXE.PAC in less than 68 Kbytes of disk
space.
The program has always set a standard for configurability -- the entire
keyboard can be internally reassigned and a number of keyboard macros can be
built into VEdit Plus. In Version 3, all the functional keyboard reassignments
are dynamically detected by the help system, which reports the current command
keys. Multiple copies of the program can be configured with individual sign-on
messages; extremely complex key redefinitions and macros can be automatically
loaded by a separate initiation file; scores of tab locations can be assigned,
control characters can be shown graphically or as control characters; and all
of the system defaults for switches, colors, cursor operation (including blink
rate and line-ending characters) can be individually set for each VEdit Plus
configuration.
For several popular word processors, VEdit Plus provides a partial emulation
through preset key sequences that approximate many individual word processor
commands or key stroke sequences. These emulations do not alter VEdit Plus's
operation or its display screen, nor do they add to or change text
manipulations. Accordingly, the emulations are a convenience feature to bring
certain word processor operations into a familiar context, but even this much
help will be important to some users.
VEdit Plus offers three user modes of operation, starting as a full-screen
editor or starting in command mode. The modes are user selectable, with the
screen-edit mode accepting abbreviated commands or providing complete menus.
Screen mode also offers auto-indenting and other programmer aids, the usual
block operations supplemented by helpful column-block operations, horizontal
scrolling for spreadsheet and other long line files, and blinding speed for
55-Kbyte or smaller files. Finally, there is the command macro mode in which
several self-running macros are supplied for activities such as mailing list
sorts and search and replace operations.
VEdit Plus is available for a variety of operating systems; I examined only
MS-DOS. The DOS system comes with a preconfigured .EXE file that is ready to
run, but VEdit Plus includes an install program that will allow easy set-up
and customization as well as a reconfiguration program that provides for very
minute control of the system's operations. For my operation, I inhibited the
sign-on message, relocated the help files to another drive, turned on insert
mode, set block operations to column mode, changed the tabs to every five
spaces, and altered the page buffer size to improve performance. All of these
adjustments were straightforward and well documented.
Adding keyboard macros is also easy, and it should be noted that each of these
changes goes into the VEdit Plus .COM file. (One simple macro I added provides
for the command Alt-Q to instantly quit the program.) Far too often a system
generates a host of tiny collateral files to set various internal matters,
disregarding the fact that each of these files requires a full disk cluster of
storage space. Because my system is perpetually full, I am grateful to
CompuView for this nice touch. The need for numerous small files was one of my
main objections to BRIEF, a major competitor to VEdit Plus.
The HELP system provides two more nice touches. As mentioned, the current
keyboard control key assignments are sensed by the HELP system so that any
assignment changes a user may make are automatically reflected when the HELP
screen displays. This procedure has decided advantages over the templates used
by the generality of programs because it is always up-to-date, never bent,
dirty, lost, or in the way!
The rest of the optional HELP system employs three ASCII files with some
well-documented, intelligent indexing commands. Any of these files can be
edited to add notes that users find necessary. With a bit more sophistication,
two or more of the standard or modified HELP files can be merged and an
entirely new help file can be written for anything that might be run with
VEdit Plus and that needs on-line help. A fairly remarkable amount of
intelligence can be built into this new HELP file, allowing, for example, the
automated look-up of a host of sub-program command switches. As a top
programmer's text editor, VEdit Plus naturally has internal support for
compilers, but the addition of on-line compiler help is frosting indeed.
Column mode operation has begun to creep into more systems, but for those who
don't know this wonderful option, I'll illustrate. Consider the following
characters:
 qwertyuiop asdfghjkl zxcvbnm
Suppose that the er, df, and cv all needed to be removed. With Column mode,
this task is trivial; just Block the rectangle from the e to the v and delete.
Column mode is slower to execute than normal Block operations, but the
convenience is obvious. I've used Column mode to remove entire segment
references from .LST files in seconds -- an activity that would take hundreds
or even thousands of manual keystrokes.
Finally, the VEdit Plus menu operation is utterly easy. The command to bring
up the menus is not shown on the screen as it really should be, but I'll
concede that anyone can remember F1 after only minutes of work with VEdit
Plus.
Not much is wrong with VEdit Plus, but the menu system leads me to one set of
oddities. On a monochrome monitor, the menus are pleasantly shown in inverse
video, but the item selected is then shown in normal video. I found two-item
menus disconcerting because I am accustomed to the highlight indicating
selection, and decided to reconfigure the program for normal video and
pull-down menus, a standard alternative on the configuration list. When I did
this, nothing happened.
After a goodly amount of work, and more than a reasonable amount of head
scratching, I called CompuView. After some checking, I was told that when the
attribute for normal video was set at 2 it inhibited all other changes. Make
this value 7, they told me, and all would be well. But all was not well. With
this value set at 7 the cursor disappeared! Finally, we learned that the
screen erase setting had to be changed as well. At last I could reconfigure
the pull-down menus to normal video, and the problem was solved.
Another minor matter concerns file size. VEdit Plus's .EXE file can be
EXEPACKed to save 4 Kbytes of disk space. But the packed version can no longer
be changed by the configured program. I recommend a well-documented archive
copy for a VEdit Plus file that is packed.
Finally, the only authentic problem with VEdit Plus concerns the edit buffers
it assigns and its internal virtual memory system. With CP/M antecedents,
VEdit Plus never uses more than one memory segment for any single activity.
This limits the largest buffer space to about 55 Kbytes, and other buffers may
get less if memory is short. On a positive note, the system simply dumps to
disk any part of a file it can't fit into the RAM buffer space. VEdit Plus
can, accordingly, deal with multi-megabyte files. Unfortunately, even when
memory is available, the virtual system still operates under the single
segment limitation, and the "Wait for File" message can quickly become
bothersome. Even when running on a RAM disk, the delays for virtual activity
are long enough to provoke resentment.
Of course, the payoff comes when huges files are being edited. Then the
smaller VEdit Plus printers allow vastly superior speed in such routine
activities as search-and-replace. On 1 Mbyte or larger files VEdit Plus flies,
while other text editors I've seen creep if they can run at all.




























August, 1989
C WINDOWS TOOLKIT


Tom Castle


Tom Castle is a chemist who frequently writes programming articles. He can be
reached at 8734 Merrimac, Richland, MI 49083, or on MCI Mail: tcastle.




Product Information


C Windows Toolkit, Version 2.0; Magna Carta Software, PO Box 475594, Garland,
TX 75047, 214-226-6909. FAX 214-226-0386. BBS (for downloading demo and
examples of applications) 214-226-8088. Requirements: IBM PC, XT, AT, PS2, or
compatible. Compilers: Turbo C 2.0, Microsoft C 5.0, MS QuickC, and Mix Power
C. Price: $99.95 (includes source code).
There are quite a few windowing toolboxes available to C programmers. One kit
in particular, C Windows Toolkit from Magna Carta, is an exceptional value.
Aside from a generous library, the package includes a font editor, a pop-up
ruler for window alignment, the library source code, and a 359-page manual
with a tutorial covering video architectures and the toolbox functions.
The guts of C Windows Toolkit, of course, is window management. The toolbox
separates the creation of a window from its presentation so that construction
is transparent and the display event is instantaneous. Many of the window
attributes, such as colors, size, location, borders, and shadowing, are set by
individual functions, so you may have quite a bit of code after building a
window with all the works. Also, you may wind up with a fairly hefty header
file because all the information used to construct windows and menus is kept
in a series of data structures.
By assigning priorities to your windows, the active (highest priority) window
will be in the foreground -- overlapping the underlying elements. If you bring
a window to the foreground, the priorities of the other windows will be
adjusted automatically.
You can settle for writing formatted output to windows, but the Toolkit also
includes facilities for virtual screens. With virtual screens, limited in size
only by free memory, your window can act like a viewing port for the
underlying text.
The Toolkit contains some nice surprises for window handling. You have the
ability to load and store windows on disk, zoom and contract windows from
various vantage points and speeds, and even produce an opening curtain
presentation.
The Toolkit provides all the necessary functions to display and make
selections from pop-up menus. You create scrolling menus by defining a menu
box of a certain size and then using a virtual screen to hold the actual text
of the menu items. Pretty slick. As another option, menus can be sized
automatically by not specifying any dimensions. Functions for the creation of
line menus or the top bars of pulldown menus aren't included in this version
of the Toolkit. Those functions, however, can be downloaded from Magna Carta's
BBS.
Menu selection can be accomplished by scrolling highlight bars, or optionally
by key press of a designated highlighted letter in the menu name. The mouse
isn't supported yet. Because separate functions are available for adding and
removing highlighting from a menu item, multiple selections are possible from
your menus.
Magna Carta probably felt a need to round out the package, so they threw in
the kitchen sink. Several keyboard input routines are available. They also
threw in all sorts of system and hardware detection functions: Enhanced
keyboards, Hercules RAMfont support, video adapters, EGA ROM parameters, and
active ANSI.SYS files. There is also a lot of support for EGA and VGA options
like the number of lines per screen, palette selection, split screens, smooth
scrolling and panning, and ROM fonts.
In addition, they give you TSR clocks, stopwatches, and alarm clocks you can
display. Speaker control and delay timers are also squeezed into the library.
There are two major new additions to the library in version 2.0. One set of
functions is for data entry fields, including parsing and validation. The
other set is a collection of functions for building a text editor.
Two considerations, in addition to the function library and documentation,
make C Windows Toolkit very attractive. First, Magna Carta doesn't require
royalties on the run-time package or any example code included in your
application. Second, at no extra cost, they include the source code to the
entire library -- of which about 98 percent is written in C for easy
inspection and modification. Like it or not, windows are with us. With that
stark fact in mind, you have a choice to make; your time or your money. Magna
Carta's C Windows Toolkit makes that choice a little easier.


































August, 1989
PCX PROGRAMMER'S TOOLKIT


Tom Castle


Tom Castle is a chemist who frequently writes programming articles. He can be
reached at 8734 Merrimac, Richland, MI 49083, or on MCI Mail: tcastle.




Product Information


PCX Programmer's Toolkit, Version 3.52; Genus Microprogramming, 11315 Meadow
Lake, Houston, TX 77077; 713-870-0737 or 800-227-0918. Requirements: IBM PC,
XT, AT, PS2, or compatible; DOS 2.1 or higher; 128K RAM. Supported Compilers:
Microsoft C, MS QuickC, Turbo C, Lattice C, Turbo Pascal, Microsoft Pascal,
Microsoft Basic, QuickBasic, Microsoft Fortran, Clipper, Microsoft ASM, (soon
Turbo ASM). Video Adapters: Hercules, CGA, EGA, VGA, Paradise Professional
VGA, Orchid ProDesigner, Video 7 VRAM, STB Extra/EM. Also supports: Expanded
memory conforming to LIM 4.0. Price: $195. Source code available for $300.
In my opinion, there is an overabundance of bit-mapped graphics file formats
for MS-DOS machines. All too often for the user, files created by one program
won't be usable by another program. From a programmer's vantage, we have to
deal with all these different beasties.
Fortunately, a ubiquitous bit-mapped image file format has emerged from the
mire, ZSoft's PCX format. Almost every drawing and desktop publishing program,
along with FAX and scanner software, uses or translates PCX-format files.
A PCX file consists of a 128-byte header section followed by a compressed data
section. By using runlength encoding (RLE) to compress the data, bit-mapped
graphics files can be squeezed to a small fraction of their original size.
This is obviously important when you get up into extended VGA with resolutions
of 800 x 600 pixels in 16 colors (240 Kbytes) or even EGA 640 x 350 pixels in
16 colors (112 Kbytes).
The first part of the PCX Programmer's Toolkit is a collection of stand-alone
programs that include the screen capture, display, clip, print, and library
utilities. Other programs locate pixel coordinates, retrieve the PCX header
information, translate captured text screens to a bit-mapped graphics image,
and fix older versions of PCX files to conform to the latest specification set
by ZSoft.
The second part of the Toolkit is a collection of function libraries to
include in your applications. There are a total of 61 functions, and the
libraries can be used with a wide array of languages and compilers (see
product box).
This Toolkit lets you do much more than display and save PCX files. A number
of the library functions deal with transferring a file from a storage location
to either another storage area or to an output device. You also have functions
to query about file types, headers, palettes, and hardware. Two functions
directly display bit-mapped images on the display with logical operations for
animation or superimposing images.
The Toolkit includes library management routines that let you manipulate
entire image libraries and extract single images for presentation. By using
libraries of images instead of individual PCX files, your graphics images are
less available for scrutiny or hazard from the end user.
The Toolkit lets you use two different types of image buffers when working
with PCX files. The traditional buffer is used for storing compressed PCX
image files. A second type, a virtual buffer, stores the entire uncompressed
bit image.
There are two advantages to using virtual buffers. First, images larger than
the screen can be stored in a virtual buffer. The image then can be quickly
panned or scrolled by scanning the virtual buffer. The virtual buffer is
limited in size only by free memory, and can be placed in expanded memory if
available. Second, displaying or printing an image from a virtual buffer are
about five to ten times faster than the same operations from a conventional
buffer containing a compressed PCX image file. The Toolkit, Version 3.52,
supports printing to a Hewlett-Packard LaserJet II. This is true both for the
stand-alone print utility and for the print functions in the programmer's
library. Support for Epson/IBM dot-matrix printers is scheduled for the next
release, but Genus has voiced reluctance about becoming too bogged down with
printer driver support. This could be a weak point in an otherwise excellent
package. You'll probably need to buy an additional graphics print driver
toolkit.




































August, 1989
PROGRAMMING PARADIGMS


Background on Backprop




Michael Swaine


Last month I reported on a conversation with engineer Hal Hardenbergh about
his interest in neural networks. I've also been talking with Hardenbergh's
software cohort in neuraldom, Tom Waite, as well as with neural net algorist
Dave Parker and engineer/programmer/writer Jurgen Fey about neural nets,
transputers, and the Occam language. Next month I'll report on those
conversations and take a more algorithmic look at neural nets.
This month I'm stepping out of the interview format to present some background
that I hope will put last month's and next month's columns in perspective.
Last month's column touched only tantalizingly on some issues, such as the
present-day practical uses of neural net technology. There are some remarkably
mundane as well as some cutting-edge uses to which nueral net insights have
been put in the past twenty-five years.
Also, the interview format may have made it hard to be sure, in reading last
month's column, just what was historical or technological fact and what was
Hardenbergh's perspective (however valid and interesting that perspective
might be). This month's neural net backgrounder should clear that up. The very
fact that a hardware engineer would get interested in what has been referred
to as "the parapsychology of AI," a fact that I presented as somewhat
surprising, in fact has a history of its own, and in the perspective of that
history is not so surprising after all.
Lately in this column I have been asking software developers (and last month
an engineer) why they are pursuing certain approaches to software development
rather than other approaches. This month I guess I'm asking myself "Why neural
networks?" You'll find only one reference listed at the end of this column,
because all the articles I drew upon can be found in the omnibus collection by
Anderson and Rosenfeld. I recommend it to anyone interested in the history and
present state of neural network research and development.


Where Did It Begin?


The perception model proposed by F. Rosenblatt in 1958 was the beginning of
all modern neural network research. It already contained most of the
interesting elements present in today's neural nets.
It described an artificial nervous system. It combined cells into several
connected layers: A "retina" where input signals arrived, an "association
layer" where retinal cells connected, and a "response layer." Connections
between association-level and response-level cells were bidirectional,
permitting feed-back that allowed the perceptron to learn. The goal of the
operation of the perceptron was to learn to activate the right response layer
for the given input.
Rosenblatt also focused on the kind of problem that occupies most neural
networks today: The classification of interesting patterns of inputs. It was a
sufficiently difficult problem that it still challenges neural net developers;
it was also sufficiently difficult to cause serious problems for Rosenblatt's
perceptron when two AI experts subjected the perceptron to rigorous analysis.


Where are the Practical Applications?


As I was working on this column, I got a call from Hal Hardenbergh. You know
what new explosives detector they're using at JFK International Airport? he
wanted to know. Yep. Well, it's a neural net. Coming as it did two days before
I would be standing in a line at JFK to board a flight to Europe,
Hardenbergh's bulletin took on a personal significance for me.
There are more neural net devices in use than meet the eye. A fair amount of
neural net research and development is DoD work, and we either don't hear
about it or hear about it only obliquely. Sometimes presentations at neural
network conferences seem to be delivered in an obscure code. What is this
about recognizing faces in the fog? Wait, if you substitute "smoke" for "fog"
and "tank" for "face," does it begin to make more sense?
One of the biggest success stories to come out of neural network research is
adaptive switching circuits, described by Bernard Widrow and Marcian E. Hoff
in 1960. Hoff is Ted Hoff, the inventor (if you live west of the Rio Grande)
of the microprocessor.
The perceptron learned by changing its coupling coefficients in response to
error feedback regarding its immediate past classifications. But many of the
proposed perceptron learning rules were impractically slow in converging to
the coefficients that would give correct classifications. Widrow and Hoff
developed what they called an adaptive neuron, related to perceptrons, that
converged to correct classification quickly. One novel feature of Widrow and
Hoff's neuron was that it continued to learn even when it was emitting correct
responses.
Widrow and Hoff built a lunchbox-sized adaptive pattern classification machine
to demonstrate their adaptive neuron's learning behavior. They originally
called the box Adaline, which stood for either Adaptive Linear Neuron or
Adaptive Linear Element, depending on how comfortable they felt about neural
net research when they were discussing it.
But it was not learning lunchboxes that proved the value of Widrow and Hoff's
technique. The error correction algorithm they used is called "least mean
squares," or LMS, because it involves minimizing the square of the error, and
LMS has been used extensively in signal processing and has seen wide use in
error correction in modems.
Widrow and Hoff also laid some groundwork for current neural net development.
Neural net research focuses on how to hook up networks of artificial neurons
so they can learn from experience. The best-known current algorithm for
implementing "experience" in neural nets is back propagation, which is a
generalization of the Widrow/Hoff rule. As Hardenbergh pointed out here last
month, back propagation had to be discovered three times before the discovery
stuck.


Why Three Times?


Rosenblatt first described the perceptron in 1958, but the machinery he first
proposed is still in use in neural network research and development. For years
it was a hot area of research, with hundreds of papers published.
Then, suddenly, the bottom dropped out, and so did the funding. The very use
of the word "neuron" became unpopular in AI work. The reason was the
fundamental inability of elementary perceptrons to classify certain kinds of
patterns. Psychological researchers called the relevant kind of pattern
classification problem concept attainment, and the interesting patterns in
concept attainment were those involving discontiguous sets, exclusive ors, and
problems that could not be solved by partitioning an input space with planes.
This limitation of simple perceptrons was brilliantly spelled out in Minsky &
Papert's 1969 book Perceptrons.
Minsky and Papert made their point clearly and emphatically. Inability to
handle problems of the concept formation-type was a serious problem, and they
treated it as such: They dismissed the bulk of the hundreds of perceptron
papers as "without scientific value."
In retrospect, it appears that Minsky and Papert were a little precipitous in
concluding that the problem they identified could not be solved. Simple
single-layer perceptrons may have been proved to be without scientific value,
but the same could not be said for multilayer perceptrons. Adding a couple
more layers would allow perceptrons to classify all sorts of problems,
although it complicated the learning problem seriously. What was needed was a
practical learning algorithm for multilayer perceptrons. But so effective was
Minsky and Papert's demolition job that nobody took the discovery of such an
algorithm seriously at first. Or at second.


Is Backprop the Algorithm of Choice?


It is if you are doing multilevel neural net work. The only other algorithm
that works with multilayer nets, the Boltzmann machine, is much slower.
Backprop is a generalization of the Widrow/Hoff error correction rule. The
Widrow/Hoff rule compared the actual output with what the output was supposed
to be and used the magnitude of the error to adjust strengths of the
connections between cells. For situations in which the correct response was
known, it worked well. Adding additional layers introduces a difficulty. How
do you compute the correct output for the hidden intermediate layers in order
to adjust the connection strengths that led to these outputs? The problem is
complicated by the fact that adjusting the connection strengths actually
changes the topology of the network.
The solution used in back propagation is to run the connections backward, to
ascertain the strengths. Back propagation involves a forward pass through the
layers to estimate the error and a backward pass to modify the connection
strengths and decrease the error.
Backprop currently looks like one of the most promising, if not the most
promising area of neural net research, and could generate some interesting
results. One of the intriguing ideas about neural nets, particularly Boltzmann
machines and backprop nets, is the notion that deep insights into the nature
of the information being processed and the effective representation of it can
be derived from looking at the internal layers. A neural net that learns to
classify patterns effectively, it is argued, contains in its hidden layers a
representation of the input. If the output classifications are adequate to our
needs, then the hidden-layer representations are also adequate, and we could
send only these representations, dispensing with the input.
One benefit could be the use of backprop neural nets to develop new data
compression algorithms.


Why is This of Interest to a Hardware Engineer?



It's not so odd that Hardenbergh, an engineer, was attracted to this domain of
artificial intelligence. Perceptron research and neural network research have
always had enormous appeal for engineers. "Much of the later work on
perceptrons and successors was done by engineers and physicists," Anderson and
Rosenfeld say, "a situation still true today in research on neural networks."
The perceptron was a learning machine, potentially capable of complex adaptive
behavior. It's easier to conceive of it as a device than as an approach to
developing AI software systems; and until you see the algorithms spelled out,
it's easier to see neural nets as a mathematical or engineering challenge than
as a programming problem.
Hardenbergh and Waite see them as all of the above. Although I don't mean this
to be a plug for Vicom or its employees, there are several reasons why I am
going to keep a journalist's eye on Hardenbergh and Waite and Vicom.
The canonical problem for neural networks is the classification of visual
figures, pattern classification. The earliest work in the neural net
tradition, Walter Pitts and Warren S. McCulloch's research in the 1940s,
focused on problems like recognizing squares wherever they appeared in the
visual field. The basic problem remains unsolved today. It's particularly
apparent to anyone who has to process image data. Vicom is in the
image-processing business.
Image-processing companies are at something of an impasse: They all have the
same algorithms, nobody has any technological edge. The time is ripe for a new
approach.
Vicom is not exclusively wrapped up in DoD work, so that smokescreens will not
obscure their results.
The amount of money required to fund a real breakthrough in neural nets for
image processing is probably not enormous, not beyond the reach of potential
customers of a company like Vicom.
I like the way Waite and Hardenbergh are approaching this. They are pragmatic
enough to be discussing using neural nets as a component of an image
processing system, not over-loading the network, not forcing it to solve
problems for which there are already good image processing solutions. They are
bringing hardware and software knowledge into the process at the start. And
they seem focused, which is good if their approach is the right one.
Next month: Tom Waite (and others) on back propagation (and other topics).


Reference


Anderson, James A. and Rosenfeld, Edward, Neurocomputing: Foundations of
Research. MIT Press, Cambridge, MA, 1988.














































August, 1989
C PROGRAMMING


OPPs to the Left, FOPs to the Right, and a View from the Center




Al Stevens


It turns out that I have been misusing a C language feature, the power of
which I have only just come to understand. The feature is the typedef
statement. Let's examine how my minor transgression came about by looking at
how typedef can be used and how it should be used.
Suppose you have a structure like this one:
struct empl_rcd {
 int emplno;
 char emplname[25];
 unsigned date_hired;
 int empl_category;
 long salary;
 };
You can define a brand new data type for this structure with the typedef
statement. Of such is the extensible nature of C. It works like this:
 typedef struct empl_rcd EMPLOYEE;
All declarations of the structure can now use the new EMPLOYEE data type
rather than the struct empl_rcd type. These are examples of how you can
declare instances of this data type.
 EMPLOYEE newhire;
 EMPLOYEE *retired;
 EMPLOYEE chiefs[5];
Why do this? Subsequent references to the structures might gain nothing from
the use of the typedef. In practice, you could not tell by looking at the code
that accessed one of these structures that the EMPLOYEE data type had been
defined at all. Witness these expressions.
 printf(newhire.emplname);
 total_payroll += chiefs[i].salary;
 return retired->empl_category;
In none of these cases is the code insulated from the format of the data
structure. In all cases, the code is exactly the same as if the typedef had
not been used. The strongest advantage of the typedef, its ability to hide
information, is not realized by these uses of it. If no other software in your
system needs to know that EMPLOYEEs exist, the typedef is wasted. What, then,
do you gain from the EMPLOYEE typedef provided in these examples? Nothing,
unfortunately, unless you consider the reduced keystrokes involved in coding
the name.
Now consider the C standard input/ output library FILE type. This is a typedef
defined in stdio.h that identifies a stream file definition. The structure
itself, if such there be, is defined in stdio.h, and its name and format are
implementation-dependent. A designer of a standard C input/output library can
define a FILE to be anything from a simple integer to the file-defining
structure itself. This seems to be a more correct use of typedef. There is
something called a FILE, and its implementors know its internals. You know
only what you need to know to use a FILE, and you do not care whether it is a
structure, union, integer, pointer, or what. All you know is that you can use
one.
In an application program you typically do three things with a FILE data type.
You declare pointers to the type, assign values to the pointers by calling
library functions that return the address of a FILE, and pass the FILE
pointers to other functions. The library functions know what to do with the
pointers because they know the format of the FILE data type. The calling
program does not (or does not need to) know that. The information is thus
hidden. You can peek and learn it if you want, and you could code specific
references to the data structures that underlie it, but you would surely
sacrifice portability, not only among compilers but possibly between versions
of the same compiler.
How then could we use the EMPLOYEE typedef in ways that would benefit us? One
way would be to define it in a header file and tuck all the employee-related
functions away in their own function library, much like the stdio.h file and
standard library are used. Then, applications programs that need to do things
to employee records could call these functions with EMPLOYEE pointers in the
same way we use the FILE pointer for stream input/ output. The functions in
the library would have to be aware of the formats of the data structures, but
the using functions would not. That way payroll, personnel, and project
management systems could all use the same employee function library without
needing to know the internal formats and methods of how employee records are
stored. Almost object-oriented, wouldn't you say?
How have I abused the typedef? Last autumn, in the window function library
that supports the SMALLCOM project, I defined MENU and FIELD data types and
then required the applications programs, programs that you might write, to
initialize the structures under the typedef definitions. Nothing is hidden. In
order to use the menu and data-entry function library, you must know the
format of those structures. Change the format and you must change your
programs. A better approach would have been to provide initializing functions
or macros with initializing data values as parameters. Then, when the
underlying structures and functions change to accommodate new requirements,
only the applications programs that deal with the changes need to be looked
at.
Why do I need to hide information from myself? I know the formats of those
structures, I understand the probability that they will change and the
implications of such change, and, besides, initializations are more efficient
at compile time than at run-time, anyway. (Insert here your favorite argument
for continuing to do things the way you have always done them.)
A compiler must hide the details of the FILE data type to preserve the
standard method for implementing it. But how well hidden is it? Look into
stdio.h and you will see the whole enchilada. The FILE structure format is
there. Some of the standard library functions expand into macros that directly
address members in the FILE structure, and the macros are there. How well
hidden is information that I can find in well-commented source code by using
my text editor?
The answer, of course, is that the standard input/output information
hidey-hole is a benign safe-store, benign in that a curious seeker can find it
and peruse its contents without difficulty, benign in that only casual
measures protect its secrets from the voyeur. You would not care to regard, on
a daily basis, the details of the garbage disposal in your kitchen or the
destination of its fare, but you know where the information can be found if
you really need it. The same is true of the information hidden in readily
accessible stashes such as stdio.h.
These are not new ideas. Information hiding has been talked about and used for
years. But it has come to be viewed in a new light in recent years, one called
"object-oriented programming," and I began to revisit my own practices with
such things as typedef when I began studying OOP. The new light is still dim
and my vision myopic, but we'll break out of the clouds soon, I think. One
thing is clear: OOP encourages you to do a better job of hiding the hideable
information.


OOPs


I think it was Robert Benchley who observed that the world is divided into two
kinds of people -- those who divide the world into two kinds of people and
those who do not. The world of programmers is dividing. There are those who
embrace object-oriented programming and those who do not. Let's wonder why.
We are told that to learn OOP we must abandon what we know about conventional
programming, that the biggest obstacle to learning is one of unlearning. The
prospect of returning to elementary school will discourage a lot of
conventional programmers who feel that they know how to write programs just
fine, thank you.
My pal at DDJ, Kent Porter, suggested a two-syllable word for the notion that
OOP is totally new and different, a word that means what you get when you run
your favorite straw hat through a bull. Kent had been toying with the
object-oriented extensions coming to Turbo Pascal and had crossed the fence to
the greener pastures of OOP. Kent was too old to start learning all over
again. And he was too young to leave when he did. I will miss him.
For me, I am studying C++ now, and it has been suggested that the Turbo Pascal
extensions are a smooth bridge from C to C++. I intend to explore that
possibility. My prophesy is that the prevailing OOP language will be C++, and
that its full acceptance will occur when and if Borland and Microsoft
introduce C++ compilers. If anything has ever screamed for OOP support, the
applications program interfaces to DOS Windows and OS/2 Presentation Manager
have. Their apparently object-oriented architectures sure do need a C++ with
related object libraries behind them.
I tend toward C++ because it is a superset of C, and one can always retreat to
C when the OOP view does not seem to fit. Some OOP purists suggest that C++ is
not really OOP. Maybe, maybe not, and maybe the paradigm has not settled in,
but I'll still bet C++ is the wave of tomorrow if there is going to be a new
wave.
Whether or not C++ replaces C as a primary development language depends on how
much it is hyped by the moving forces in language development and what it does
for the programmer. Certainly if Borland and Microsoft get behind it, lots of
programmers will give it a try. But programmers will not stick with C++ unless
there is a reason to.
We measure the value of a language by its ability to express a complex
algorithm with few lines of code in a way that can be read and plainly
understood. Most programmers took to C because it could do those things
without getting too far away from the hardware and without rigidly enforcing
anything. To win C programmers over, C++ has to get things running with fewer
and easier lines of code, make for programs that are easier to modify, or
both. As long as the code is readable later, why code two when one will do?
C++ has to improve our ability to write programs, or it is not necessary.
So what is different about OOP? Perhaps it is that the OOP view of a
programming problem has a different perspective than the one we enjoy now. But
is that really so?
The OOP view looks toward the data structures and the functions specific to
those structures. You describe classes of data, objects that are instances of
those classes, and methods to process the objects. Then, to make things
happen, you send a message to an object, and things happen. Consider that old
chestnut, hello.c, in conventional C.
 /* ------ hello.c in C ------ */
 #include <stdio.h>
 main( )
 {

 printf("hello, world\n") ;
 }
This program is oriented to the function it performs. It is a
function-oriented program (FOP). The FOP programmer thinks about solutions in
terms of the procedure, the functions, the steps one takes to solve something.
Now, consider the same program in C++
 \\ ------ hello.c in C++
 #include <stream.h >>
 main( )
 {
 cout << "hello, world \ n";
 }
This program, we are told, is oriented to its objects, the data structures it
supports, and the methods it uses to support them.
They look similar, don't they? In conventional C, you called the printf
function, passing it the address of the string to display. The stdio.h header
file provided the prototype for the printf function. In C++ you send a message
to cout, which is an object of class ostream as defined in stream.h. The
message consists of the address of the string. The << operator, also defined
in stream.h as a part of the class, is the method the object will use.
The << operator is not itself an integral part of C++ any more than the printf
function is an integral part of the C language. The << operator is associated
with the ostream class within stream.h. You can associate custom operators
with classes in C++. The greater-than operator, for example, can mean
different things to different classes. Moreover, it can mean different things
in the same class when taken in the context of the types of what it operates
on.
Now let's write the "hello, world" message to a file. First, we'll use
conventional C:
 #include <stdio.h>
 main()
 {
 FILE *fp;
 fp = fopen("hello.dat", "w");
 fputs("hello, world\n", fp);
 fclose(fp);
 }
In standard FOP C, you call three functions. The first one opens the file and
returns a pointer to a FILE. The second function accepts a data address and a
FILE pointer and sends the data string to the file. The third function closes
the file.
Now let's look at the same operation with C++:
 #include <stream.h>
 main()
 {
 filebuf bf;
 bf.open("hello.dat", output);
 ostream op(&bf);
 op.put("hello, world\n");
 }
In the C++ OOP view, you declare an object of class filebuf named bf. This
object is the data buffer itself. Next you connect the buffer to a stream file
with the filebuf::open function. The other way of saying that is that you send
the string address and the output flag as messages to the bf object, telling
it to use its open method. Then you declare an object of type ostream named op
with the filebuf as an initializing argument. Finally you send the string
message to the op object telling it to use its put method.
At this level, the two views are still similar. A C programmer could make the
transition to C++ with little problem if this was as far as it went. In fact,
you might wonder, why bother? It appears to be a syntax change, nothing more.
The big difference, however, is in what is hidden. The definitions of the
filebuf and ostream classes are significantly different from their FILE
counterparts in standard C.
"So what?" you might ask. "The differences are hidden. Who cares how different
they are?"
Think back to the EMPLOYEES typedef. That one is a part of the application, is
your responsibility, and, if it is to be an object, needs a class, hidden
member data structures, and public member method functions. To design these
things you have to be as object-oriented as the language. If OOP has anything
to offer, the apparent different statement syntax is not it. There must be
merit in the taking of that different perspective.
One of the problems we will confront in writing our first C++ programs is
deciding what should be classes and objects and what should not. When do we
send messages and when do we call functions? What is the most appropriate way
to send messages? Suppose your program posts error conditions by opening a
window and displaying the message. Should the error processor be an
old-fashioned C function like this?
 error("Too many Chiefs");
Or should the error processor be an object of a class of output like this?
 class error: public window {
 . . .
 };
 error errs;
 errs.post("Too many Chiefs");
Should the error processor use a member function as just shown, or should it
use its own operator to post errors like this?
 errs << "Too many Chiefs";
You might post errors with the error class's constructor and destructor
functions. The constructor function is called when the object is declared, and
the destructor function is called when the object goes out of scope.
 if (chiefs > MAXCHIEFS) {
 error errs("Too many chiefs");
 }
This OOP stuff is beginning to make sense. The code in this column may have a
few mistakes, but I'm just learning C++ myself. In the next few months I"ll
explore it in more depth, and we'll learn together.


Are People Object-Oriented?


For a programming paradigm to take hold, it must work like people think. The
solution to a problem should resemble the problem. Are we object-oriented
creatures? I think we might be. People know how to do many complex things but
usually only within the context of the objects involved. A plumber's methods
have meaning only when considered in the context of sinks, bathtubs, and the
like. Take away an understanding of the objects and the methods would be
nonsense.
So what is more important and deserves more emphasis, the object or the
methods, the bathtub or the plumber's tools and procedures? Where should the
perspective be? And what should be saved?
With traditional FOP, you preserve general-purpose, reusable functions in
libraries, and you keep the applications data structures in header files. The
two are not associated except within the context of programs that tie them
together. The applications-specific functions are generally not retained for
reuse. In OOP you can catalog the reusable stuff just as with FOP, but you
also closely associate data-oriented functions with the data structures as
objects, and you can save these objects for reuse if appropriate.

So what should be in the library, the functions or the objects? What should be
designed to be reusable? Obviously, only those things that are likely to be
reused should be so designed, and sometimes it goes one way and sometimes the
other. Without knowing any better, I am drawn to C++ because of its subset
ANSI C. You can go either way even within the same program. Only time will
tell if that is a good idea.
My pal Bill Chaney tells me to think again; a FOP is a dandy, and OOPs means a
mistake has been made.


Book Report: The C++ Programming Language


The C++ Programming Language by Bjarne Stroustrup tries a lot to be the K&R
book of C++. It resembles the "white book." It has the same size, same style
and presentation, and (almost) the same title. But K&R is a well-crafted
description of C aimed at the programmer who does not yet know the language.
As you read K&R, you are led carefully through the features of the language
with well-organized examples and a well-designed sequence for the presentation
of information. The C++ book does not measure up to this high standard.
If anything is going to spell the demise of C++ or OOP in general, it is that
the language and the paradigm are or seem to be difficult to describe.
Programmers do not readily embrace something that the experts cannot explain.
Maybe it isn't so difficult; maybe no one has described them correctly yet.
You will hear that many programmers have difficulty making the transition from
C to C++. Perhaps they are trying to do it by reading this book. Believe me,
Stroustrup's book is not the place to begin.
Once it gets past a brief description of the C subset, the book dives into
details way too deep for the newcomer. For example, it introduces classes with
a discussion of how ostream is implemented without explaining what ostream is.
Then it sinks into an arcane description of how operator overloading is used
to implement expressions like A << B << C well before it fully explains how
you can extend the language with operators for objects. This book suffers from
the malady that most treatments of OOP have -- abstruse examples. Most
programmers will not relate to the implementation of stream input/output
classes and objects because they do not do that kind of programming. Most
programmers will not associate with examples that use complex numbers, because
relatively few real-world programming problems use them. The book renames C's
arrays, calling them vectors, and never explains that. Then, to further
confuse you, the discussion of vectors makes you think that there is a vector
class by building one. Not until later do you learn that you are being told
how to use the features of C++ to do your own vector bounds checking and not
using C++'s unbounded vectors at all. The entire operator topic uses the
implementation of improved vectors as an example. Most programmers would not
cozy up to these examples even if the examples were well-organized and
-presented because most programmers do not write systems programs where such
techniques are applied. The approach taken by this book resembles the typical
university computer sciences curriculum, where emphasis is on language design
and compiler development rather than how and why they are used.
That's about as far as I've gotten with The C++ Programming Language. I'm
looking for another book.
My first impression of C was taken from K&R. The organization and clarity of
that work engendered confidence in the language. It was easy to believe that
the language would live up to its promise; Dennis Ritchie, one of the
coauthors of the book, designed the language.
If the same confusion and lack of organization found in the C++ "definitive
reference and guide" is in the language (C++ was designed by Stroustrup), we
are in deep sushi. First impressions die hard, but I hope to be wrong. I still
believe that C++ will be the next major programming language.
As the work of the ANSI X3J11 committee winds down, they are considering the
standardization of C++. If this undertaking gets going, it is a sure sign that
C++ will establish and maintain a strong presence for a long time.
















































August, 1989
GRAPHICS PROGRAMMING


The First 67 Miles of Curves




Kent Porter


Driving south from Carmel, California, along the wild, vertical cliffs of the
Big Sur coastline, you pass a sign saying "Curves Next 67 Miles." A couple of
hours and 67 miles later, you come to another sign saying "Curves Next 42
Miles." I guess some bureaucrat figured that 67 miles' worth of bad news was
all your average motorist could handle at one time.
The drive through Big Sur is one of the most spectacular collections of curves
in the world. In one respect, however, there's nothing unusual about it.
Nearly everything in nature consists of curves. Look around. Just about the
only straight lines you see are man-made.
The reason we've concentrated so far on straight lines in computer graphics is
that they're easy to deal with and easy to understand. However, if graphics is
to depict real-world objects it has to draw curves.
This is a difficult subject. Whole careers in computer science are devoted to
the investigation of curves. So I'll be more honest than that nameless
bureaucrat and tell you up front that there are a full 109 miles of curves
ahead. However, we're going to take them the same way most people travel
through Big Sur: In stages, with stops along the way.
We begin this month with two relatively simple curves that can be used to
depict most rounded objects. The first is the ellipse, which can also draw a
circle. The other is the conic spline, a remarkably supple object forming an
arc. The road through Big Sur can be likened to a connected string of conic
splines. In due course later, we'll consider cubic and Bezier splines, which
are useful for concisely expressing complex curves in two and three
dimensions. But first let's do the easier stuff.


'Round and 'Round We Go


Remember Bresenham? He's the guy we talked about in March, who invented a way
to draw straight lines efficiently. Well, Bresenham didn't stop there to rest
on his laurels. He went on to figure out how to draw ellipses without using
floating point arithmetic. If you want a good exposition of the Bresenham
method, see Algorithms 14 and 15 in James Blinn's article "How Many Ways Can
You Draw a Circle?" (DDJ, September 1997). That article will also give you an
insight into the complexities of even something as apparently simple as a
circle.
The easiest way to make a circle is to rely on classical trigonometry. Given
any angle, you find the corresponding point on the circle as follows:
The X is at the cosine of the angle times the radius, plus the center X.
The Y is at the sine of the angle times the radius, plus the center Y.
This seems so simple that one wonders why circle-making is a big deal.
The problem is that it's computationally intensive. The sine and cosine are
periodic floating point functions that take a lot of resources (that is, time)
to calculate. It wouldn't matter much if every computer was equipped with
floating point hardware such as an '87 chip. However, few are, relying instead
on time-hungry floating point emulation in software. Integer operations are
much faster, and that's why Bresenham and others have quested for better
methods.
With so many learned folks studying the problem, it's inevitable that variants
on the Bresenham algorithm have appeared. The one we discuss here is called
the midpoint algorithm. Wilton's PROGRAMMER'S GUIDE TO PC & PS/2 VIDEO SYSTEMS
(1987, Microsoft Press) devotes most of Chapter 7 to this algorithm, so I
won't go into much depth; if you're serious about graphics, get this book.
In nature, an ellipse is a circle elongated in some direction. To simplify
matters, the Bresenham algorithm and its derivatives impose a limit: An
ellipse can be elongated either horizontally or vertically, but not on the
diagonal. (The conic splines discussed later remove this restriction.) If we
can describe an ellipse as a stretched circle, then it's also fair to say that
a circle is an ellipse in which both axes have the same length. That's why the
method works for both true circles and ellipses, and why the terms are
interchangeable.
The midpoint algorithm assumes that we start at point {0, y} and proceed
through one quadrant, or 90 degrees of angular motion, to point {x, 0} (where
y is the vertical semiaxis and x is the horizontal). In other words, it only
worries about the upper-right quadrant of the ellipse. Corresponding points
for the other three quadrants are found by adding to and subtracting from the
center coordinates.
As the curve forms the exact edge advances, moving among and between pixel
positions on the raster display, just like a slanting straight line. To draw a
reasonably accurate representation, then, it's necessary to find out which
fixed pixel is closest to the real curve at any given point. We could do this
in the classic sine/cosine method by rounding the results of the calculations.
To save that overhead, the Bresenham algorithm uses a decision variable
called, by convention, d.
In general, if d is positive, then the curve lies below the midpoint between
two vertically adjacent pixels, so the pixel below is selected. Otherwise the
pixel above is closer. When the curve has advanced through half its arc (first
octant) the tendency of motion becomes more downward than rightward. Thus the
algorithm switches its orientation to select between horizontally adjacent
pixels. A negative value for d means that the nearer pixel is outside the
ellipse. The algorithm steps on a pixel-by-pixel basis, assuring an unbroken
curve on the display. (Gaps sometimes occur using the "pure" Bresenham
method.) The first octant advances the X in each iteration and selects the
nearest Y. The second octant advances by Y and picks the appropriate X. The
value of d changes each time by a pair of values called dx and dy by Wilton.
The ellipse( ) function implementing this algorithm (Listing One) refers to
them as xd and yd to avoid conflict with the virtual coordinate functions.
The values of xd and yd, using the function's names, are updated at each step
by values that remain constant for the duration of the call. These constants
reflect the rate of change in the curve, taking into account the tension
between the X and Y semiaxes. Thus, xd and yd maintain a running total of the
effects of angular motion along the curving path. The value of d is controlled
by applying xd and yd, and so the sign of d selects the nearest pixel to each
successive point.
Because the polynomial constants of angular motion are computed outside the
loop and all values are 16- or 32-bit integers, the midpoint algorithm is
efficient while drawing a smooth, unbroken, and accurate representation of the
ellipse.
Want to see? First compile ELLIPSE.C and add it to your copy of GRAFIX.LIB
with the command
 LIB grafix +ellipse;
Next, add the two prototypes from Listing Twoto GRAFIX.H. And then compile,
link, and run CIRCLES.C from Listing Three. It draws a true circle and two
ellipses.


Just Partway 'Round, Thanks


Graphics literature sometimes explains that the term spline derives from the
loftsman's spline, a piece of thin flexible material used to round inside
edges. The analogy is lost on me inasmuch as I don't know what a loftsman
does, nor do I care. A more useful analogy is to the fanciful curlicued French
curves used by draftsmen (which is perhaps as meaningless to you). At any
rate, spline is a term often used in computer graphics to mean an open curve.
There are several kinds of splines. The simplest is the conic spline, so
called because it exists inside a triangular outline (or "hull").
Conceptually, the hull is a V-shaped set of three points. The curve itself
goes between the two end points of the V. As it crosses the open space, the
intersection of the hull's two sides -- the point of the V -- attracts the
curve, causing it to bend inward. The amount of curvature depends on the depth
of the enclosing cone.
Figure 1 illustrates a couple of conic splines. Note that the points where the
curve meets the sides of the cone are called knots, and the attractor is
called the control point. As you'll see shortly, the orientation of one of the
sides doesn't have to be horizontal despite what the drawings suggest.
The curve in Figure 1a bends through more than 90 degrees, while that in
Figure 1b falls somewhat short of a quarter-circle. Nevertheless, both curves
represent one quadrant of an ellipse. Figure 2 shows how. Consequently, the
conic spline isn't really attracted to its control point, but is instead
rotating around the ellipse's center. In doing so it travels between the
opposite corners of a parallelogram completed by the center and control point.
So, why not just use the midpoint algorithm to plot a conic spline, drawing
only the affected quadrant? Because the ellipses defining a spline don't
necessarily have vertical and horizontal axes. Remember, Bresenham imposes
that as a condition on his ellipses. We could draw conic splines with the
midpoint method, but the sides would always have to be at perpendicular
angles. Thus, we couldn't draw the curves in Figure 1.
The spline in Figure 1a illustrates the problem. Two-thirds of the way between
Knots A and B, the curve bends beyond 90 degrees and begins moving back in the
opposite horizontal direction. It started out going right, and now it's going
left. If we rotated the whole thing, a similar problem would exist with going
first up, then down, or vice versa. The midpoint algorithm simply can't handle
this because it expects the axes to be orthogonal.
We need a more capable algorithm. That's what the arc() function in ARC.C
(Listing Four) provides. This algorithm uses fixed point arithmetic to
approximate floating point without the overhead. The fixed point values are
long integers with a fractional value in the lower 16 bits. This accounts for
the left shifts when setting up the control values, and also for the constant
HALFPI. The arc sweeps through a quarter-turn, or PI/2 radians, so we
calculate HALFPI as (3.1415927 / 2) << 16, which works out to 102,944.
The whole idea of this algorithm is somewhat like that of Bresenham, though it
doesn't look like it. The values vx and vy give offsets for the current point
relative to the center of the ellipse (x0 and y0). The ux and uy variables
accumulate angular motion as the curve progresses. Because vx, vy, and ux, uy
interact with each other, in effect updating each others values at the bottom
of the loop, the angular motion to the right (when drawing Figure 1a)
gradually slows and eventually backtracks.
Everything depends on the pixel density variable called "den," which provides
a power of 2 by which to multiply and divide the control values. The sense of
this rests upon the effect of shifting an integer. Every time you shift one
bit to the right, you divide by 2. Three shifts to the right is division by 8,
four is division by 16, and so on. Multiplication by a power of 2 similarly
occurs when shifting left.
The algorithm draws a predictable number of pixels depending on the value of
den. When den is 10, it draws 804 pixels between the knots. The number of
pixels decreases by half each time den decrements by one. In computing the
pixel density factor, then, the algorithm determines approximately how far the
curve has to travel, and assigns the appropriate power of 2 to control its
motion.
The exact number of pixels needed to draw an unbroken curve depends on the
depth of the cone and the distance between knots. The algorithm might attempt
to write a pixel in the same position more than once, so the loop tracks the
previous position and only allows a write when the new position is different;
the comparison is computationally cheaper than a rewrite.
See, I told you curves aren't easy. But because the conic algorithm uses only
integers with fractional values in the lower 16 bits, it draws splines
efficiently.
Compile ARC.C, then put it into the library with
 LIB grafix +arc;
Now you're ready to see some conic splines in action.

The first example is CONICS.C in Listing Five. This program vividly
illustrates the relationship between a curve and its enclosing cone. The
curves are in white, their hulls in red.
The next example, HANGER.C in Listing Six draws a coat hanger to demonstrate
how conic splines can be joined with each other and with lines to construct
real-world images. The secret lies in forming continuous joints. You do this
by placing one knot and the control point on the same plane as the line that
continues the curve, and starting or ending the line at the knot. For example,
the bottom of the hanger is on Y coordinate 260, and its right end is at
X-500. The right spline's knot is also at {500, 260} and its control point
similarly lies on Y=260. The result is a smooth joint; the spline appears to
be a continuation of the wire that bends around to the right shoulder of the
hanger. The hook at the top joins four conic curves. In each case, the joints
occur at coincident knots and the control points of any two adjacent curves
lie on the same plane such that a line passing through them also passes
through the common knot.
The final example, BALL.C in Listing Seven combines ellipses, curves, and
floodfills to form the shaded 3-D image of a ball resting on its shadow. This
is a brute-force approach to shading, a subject that still lies in the far
distant future of this column, but the visual result is the roughly the same.
The flow of the program should be apparent; the message is that we have now
reached the point of being able to produce reasonably good graphics with the
tools developed here.
Forty-two more miles of curves to go. Time for a rest stop.
Kent completed this installment of his "Graphics Programming" column shortly
before he passed away. Even though the series will end with this issue, we'd
like to hear how you're using some of the ideas Kent's introduced in this
space. Drop us a note about your program or, better yet, write an article and
share your applications with others. -- eds


GRAPHICS PROGRAMMING COLUMN
by Kent Porter



[LISTING ONE]

/* ELLIPSE.C: Midpoint algorithm for drawing an ellipse */
/* Based on Wilton's "VIDEO SYSTEMS," pp 230-231 */
/* For inclusion in GRAFIX.LIB */
/* K. Porter, DDJ Graphics Programming Column, 8/89 */

#include "grafix.h"

void far ellipse (int cx, int cy, /* center x, y */
 int horiz_rad, int vert_rad) /* radii ("semi-axes") */
{
int x = 0, y = vert_rad; /* starting coords for curve */
long asq = (long) horiz_rad * horiz_rad; /* a^2 */
long a2sq = asq + asq; /* 2a^2 */
long bsq = (long) vert_rad * vert_rad; /* b^2 */
long b2sq = bsq + bsq; /* 2b^2 */
long d, xd, yd; /* control values */

 /* Initialize control values */
 d = bsq - asq * vert_rad + asq / 4L; /* b^2 - a^2b + a^2/4 */
 xd = 0L;
 yd = a2sq * vert_rad; /* 2a^2b */

 /* Loop to draw first half of quadrant */
 while (xd < yd) {
 draw_point (cx+x, cy+y); /* set pixels in all quadrants */
 draw_point (cx-x, cy+y);
 draw_point (cx+x, cy-y);
 draw_point (cx-x, cy-y);
 if (d > 0L) { /* if nearest pixel is toward the center */
 --y; /* move toward center */
 yd -= a2sq; /* update control values */
 d -= yd;
 }
 ++x; /* next horiz point */
 xd += b2sq; /* update control values */
 d += bsq + xd;
 }

 /* Loop to draw second half of quadrant */
 d += (3L * (asq-bsq) / 2L - (xd+yd)) / 2L;
 while (y >= 0) { /* do until y = 0 */
 draw_point (cx+x, cy+y); /* set pixels in all quadrants */
 draw_point (cx-x, cy+y);
 draw_point (cx+x, cy-y);
 draw_point (cx-x, cy-y);

 if (d < 0L) { /* if nearest pixel is outside ellipse */
 ++x; /* move away from center */
 xd += b2sq; /* update control values */
 d += xd;
 }
 --y; /* next vertical point */
 yd -= a2sq; /* update control values */
 d += asq - yd;
 }
} /* --------------- */







[LISTING TWO]

Caption: Add these entries to your copy of GRAFIX.H


/* From August '89 */
/* --------------- */
void far ellipse /* draw ellipse */
 (int cx, int cy, /* at center x, y */
 int horiz_rad, int vert_rad); /* radii (semi-axes) */

void far arc (int x1, int y1, int x2, int y2, int xc, int yc);
 /* draw conic spline, where knots are at x1, y1 and */
 /* and x2, y2, control point is at xc, yc */






[LISTING THREE]

/* CIRCLES.C: Draws several ellipses to demo ellipse() function */
/* K. Porter, DDJ Graphics Programming Column, Aug '89 */

#include <conio.h>
#include "grafix.h"

void main ()
{
 if (init_video (EGA)) {
 setcoords (-400, 300, 400, -300);

 /* draw a true circle in upper center of screen */
 ellipse (dx(0), dy(100), dxunits (150), dyunits (150));

 /* tall skinny ellipse to left */
 set_color1 (13); /* light magenta */
 ellipse (dx(-300), dy(0), dxunits (50), dyunits (250));

 /* short fat ellipse lower right */
 set_color1 (10); /* light green */

 ellipse (dx(150), dy(-200), dxunits (200), dyunits (30));

 /* hold for keypress, then quit */
 getch();
 }
}






[LISTING FOUR]

/* ARC.C: Draws a conic spline using fixed point math */
/* Knots are at x1, y1 and x2, y2, control pt at xc, yc */
/* For inclusion in GRAFIX.LIB */
/* K. Porter, DDJ Graphics Programming Column, Aug '89 */

#include "grafix.h"
#include <math.h>
#define HALFPI 102944L /* fixed-point PI/2 */

void far arc (int x1, int y1, int x2, int y2, int xc, int yc)
{
int i, x, y, prevx = -1, prevy = -1;
int delta_x, delta_y, delta_t, dt = 804, den = 10;
long vx, vy, ux, uy, x0, y0;

 /* fixed-point control values for this arc */
 vx = (long)(xc - x2) << 16; /* distance to start */
 vy = (long)(yc - y2) << 16;
 ux = (long)(xc - x1) << 16; /* current point adjustment factor */
 uy = (long)(yc - y1) << 16;
 x0 = ((long)x1 << 16) - vx; /* center of arc */
 y0 = ((long)y1 << 16) - vy;

 /* compute pixel density factor (2^den) */
 delta_x = (labs (vx) + labs (ux)) >> 16;
 delta_y = (labs (vy) + labs (uy)) >> 16;
 delta_t = delta_x + delta_y;
 while (dt > delta_t) {
 dt /= 2;
 --den;
 }

for (i = (int)((HALFPI << den) >> 16); i >= 0; --i) {
 x = (int)((x0 + vx) >> 16); /* current position */
 y = (int)((y0 + vy) >> 16);
 if ((x != prevx) (y != prevy)) /* if not same as last point */
 draw_point (x, y); /* draw new point */
 prevx = x; /* remember this position */
 prevy = y;
 ux -= vx >> den; /* advance arc */
 uy -= vy >> den; /* division by 2^den */
 vx += ux >> den;
 vy += uy >> den;
 }
}








[LISTING FIVE]

/* CONICS.C: Draws several conic splines and their enclosing hulls */
/* K. Porter, DDJ Graphics Programming Column, Aug '89 */

#include <conio.h>
#include "grafix.h"

void main ()
{
 if (init_video (EGA)) {

 /* Fig. 1a */
 set_color1 (12); /* red hull */
 draw_line ( 5, 20, 250, 20);
 draw_line (50, 180, 250, 20);
 set_color1 (15); /* white curve */
 arc (5, 20, 50, 180, 250, 20);

 /* Fig. 1b */
 set_color1 (12); /* red hull */
 draw_line (630, 20, 520, 20);
 draw_line (470, 120, 520, 20);
 set_color1 (15); /* white curve */
 arc (630, 20, 470, 120, 520, 20);

 /* acute conic in the center */
 set_color1 (12);
 draw_line (200, 330, 370, 40);
 draw_line (340, 200, 370, 40);
 set_color1 (15);
 arc (200, 330, 340, 200, 370, 40);

 /* wait for keypress and quit */
 getch();
 }
}






[LISTING SIX]

/* HANGER.C: Draws a coat hanger with lines and conic splines */
/* K. Porter, DDJ Graphics Programming Column, Aug '89 */

#include <conio.h>
#include "grafix.h"

void main ()

{
 if (init_video (EGA)) {

 /* draw body of hanger */
 draw_line (320, 180, 140, 240); /* left top */
 arc (140, 240, 140, 260, 80, 260); /* left end */
 draw_line (140, 260, 500, 260); /* bottom */
 arc (500, 260, 500, 240, 560, 260); /* right end */
 draw_line (500, 240, 320, 180); /* right top */

 /* draw hook at top */
 arc (320, 180, 310, 160, 320, 168);
 arc (310, 160, 300, 140, 300, 154);
 arc (300, 140, 320, 120, 300, 120);
 arc (320, 120, 340, 140, 340, 120);

 /* wait for keypress and quit */
 getch();
 }
}






[LISTING SEVEN]

/* BALL.C: Ball throwing a shadow */

#include "grafix.h"
#include <conio.h>

void main ()
{
 if (init_video (EGA)) {
 setcoords (-320, 240, 319, -239);

 /* Draw the backdrop in dark blue */
 set_color1 (1);
 fill_rect (0, 0, 639, 150);

 /* Draw the floor in light blue */
 set_color1 (9);
 fill_rect (0, 150, 639, 199);

 /* Create shadow on floor */
 set_color1 (8); /* dark gray */
 ellipse (dx(0), dy(-160), 160, 25);
 setfillborder (8);
 floodfill (dx(0), dy(-130));

 /* Draw the ball */
 set_color1 (7); /* light gray */
 ellipse (dx(0), dy(0), dxunits (150), dyunits (150));

 /* Shade the ball underneath */
 arc (dx(-150), dy(0), dx(150), dy(0), dx(0), dy(-50));
 setfillborder (7);

 floodfill (dx(0), dy(-25));

 /* Shade the ball white on top */
 set_color1 (15);
 floodfill (dx(0), dy (145));

 /* Hold for keypress and quit */
 getch();
 }
}




















































August, 1989
STRUCTURED PROGRAMMING


Thinking Big, Talking Small




Jeff Duntemann, K16RA


There's a definition of the word "legendary" that I favor: Something that
everybody talks about but which has never had any basis in fact. (The
legendary Loch Ness monster comes to mind, as well as that legendary IBM
service and support.) There's a computer language that comes perilously close
to being legendary, and that language is the (almost-legendary) Smalltalk.
Amidst the dusty stacks of computer magazines filling my two walls of Hundavad
bookshelves is the October 1980 issue of the late and lamented Creative
Computing. On the cover is a Halloween witch boarding her broomstick, cackling
in a cartoon balloon: "Come with me on a journey to the mysterious world of
Smalltalk!" Such is the stuff of which legends are made.
Though it tried gamely, Creative Computing did little to chase the smoke
surrounding the language. At best, they made it sound like an infix Forth
width a graphics user interface, and that comes closer to the truth than the
PARC folks would care to admit. The problem was that Ted Nelson and the other
gurus of the time were so taken by the ivory tower PARC mystique and the
dazzlingly precocious Xerox graphics workstations that they mistook the user
interface for the language, muttered things about animation scripts and
notebook-sized Dynabooks, and never really got around to answering the serious
question: Why is Smalltalk special?
It is special because it is the ultimate object-oriented language. It was
easily 15 years ahead of its time in many ways, and now that a world --
screaming for Object-Oriented Anything (OOA) -- is ready for Smalltalk, the
language revealed falls far short of the mystical, magical otherworldliness
that ten years of yearning have coated it with. If you've never actually got
down and hacked in Smalltalk, what I'd like you to do is adopt the Firesign
Theater attitude toward it: Everything you know is wrong. I'm going to try to
explain Smalltalk from the other direction -- as a perfectly of language
within a perfectly remarkable framework -- and in the process take a stab at
showing you what object-oriented programming is about.


At the Language Nudist Park


You'd recognize Smalltalk if you ran into it at the language nudist park,
stripped bare of overlapping windows and mouse cursors and all that other
folderol. Here's an assignment statement in Smalltalk:
 fudgeFactor:= 42.
Man, it's just like back home in Pascalville! A numeric variable called
fudgeFactor takes on a value of 42. The period is the local equivalent of
Pascal's notorious (if not legendary) semicolon, and indicates the end of a
statement. Like Pascal and C and Basic and all but the most bizarre languages
like Prolog, a Smalltalk program is nothing more than a series of statements
that do something in sequence.
Smalltalk, it seems, has everything all the other languages have, and most of
its parts look familiar in an odd, polyglot kind of way. Generating the
character equivalent of an integer is done this way:
 68 asCharacter
Excuse me, Mr. Forth ... no, it really is Smalltalk, and Smalltalk at the
lowest level is just another collection of railroad diagrams, like any
language you could name. The syntax is new, but the concepts are utterly
traditional. If you're looking for a FORloop, look no further:
 4 timesRepeat:[ Turtle go: 100; turn: 90]
Smalltalk sets off compound statements within pairs of square brackets, just
as in Pascal we use BEGIN/END and in C {/}. Saying 4 timesRepeat: [] is just
about precisely equivalent to FOR I := 1 TO 4 DO BEGIN END. Nothing magical or
legendary about that at all.


The Message is the Medium


Yet another reason Smalltalk leans toward the legendary is that the PARC
people, in designing it, made up new names for many traditional computer
science concepts. What in most languages we'd call passing parameters in
Smalltalk is called "passing messages." The distraction makes sense once
you've grokked the fullness of the language, but for newcomers the term
message passing promises more exotica than it delivers, and the result is
gross confusion.
Here's why: The Smalltalk expression mentioned earlier, 68 asCharacter,
returns the ASCII character 'D.' However, in the Smalltalk jargon, what is
happening is that a message called asCharacter is passed to the value 68. The
value 68 responds by sending back a message consisting of the value 'D.' Does
this confuse you? It sure confused the hell out of me when I first encountered
it. In Pascal you'd pass the value 68 to the standard function Chr (as in
Chr(68)) and the standard function would return the value 'D.' In Smalltalk,
it seems like you pass the procedure to the parameter, rather than the other
way around. Bizarre? No more so than Forth, and some of my best friends use
Forth all the time.
Forth uses postfix notation ("reverse Polish" -- now there's a legend for
you!) because it serves the pathologically stack-centered architecture of the
Forth language. Smalltalk uses its message-passing notation because message
passing serves the architecture of Smalltalk, which, like Forth, differs from
that of Pascal, C, and Basic. The thing to remember is that, like Forth,
everything about Smalltalk makes perfect sense if you take it in the spirit of
the language.
On a statement level, then, Smalltalk is just a variation on a common theme.
The uncommon aspects of Smalltalk appear when it starts to put its clothes
back on. The magic is in the framework, not the syntax.


Who's the Boss?


Smalltalk's architecture is not easily described in the legendary 25 words or
fewer. I like a challenge, so I'll try. These are Smalltalk's three
architectural principles:
1. Data Is Boss. 2. Data Knows What To Do. 3. Data Bequeaths Everything It Has
To Its Children.
First of all, Data Is Boss. To Pascal programmers, Data Is Clay, and we spend
all our time fiddling up code-ish widgets to squeeze, shape, spindle, and
mangle that data. Smalltalk moves data to center stage. We tend to think of a
Pascal program as a series of code-clumps passing control from one to another.
Data gets passed around as well, passively being beaten about the ears and
pounded into new shapes and sizes. In a Smalltalk program, the active parties
are not sequences of code but items of data.
Weird? Well, consider ... what's the more valuable and lasting entity: The act
of a dog barking or the dog itself? As Confucius might say: There is still a
dog even when the dog is silent. Smalltalk leans toward a philosophy which
says: Mind the dog, and the bark will take care of itself.


When is Data More than Data?


This is why the statement 68 asCharacter is seen as a message being passed to
a data item. The data item is the active party, because Data Knows What To Do.
Data in Smalltalk is more than data. For every data item in Smalltalk, there
is a list of actions that the data item can take. One of the things that a
number knows how to do, for example, is to convert itself to an ASCII
character and pass the character back as a message. That's what the number 68
does when it receives the message asCharacter. Other messages (three out of a
great many) that an integer value understands, and can respond appropriately
to, are:
factorial -- The value calculates and returns its own factorial
reciprocal -- The value divides itself into 1 and returns its own reciprocal
negated -- The value subtracts itself from 0 and returns its negated value
These may look and sound like procedures to you, and they are in fact the
"code" portion of a Smalltalk program. But the critical difference is this:
They are considered to be part of the integer value. You cannot somehow reach
in and pull a procedure called factorial out of an integer value. The two are
welded together at the hip, hand in hand together for all time, forever and
ever amen.
This more-than-data concept in Smalltalk has its own name, and that name is
object. An object in Smalltalk is a piece of data and the things it knows how
to do.

This is only weird for the first 13 minutes you think about it. (I timed it.)
Why weld the code to the data? Easy: To keep the code out of trouble. Can a
dog whistle? Can a teakettle bark? No. Yet we Pascal guys get in trouble with
the compiler all the time, trying to pass character values to the Abs standard
function, trying to take the cosine of True, things like that. Matching data
types to code that can legally manipulate data of that type is lots of trouble
-- so Smalltalk ends the problem by gluing the two together.
Those actions an object can take in response to messages are called "methods."
Every object has its crisply-defined suite of methods. And in Smalltalk,
everything (bar nothing!) is an object.
This is what I was hinting at when I implied that Smalltalk statements looked
normal -- until they started putting their clothes on. The arithmetic
expression 17 + 42 looks the same in Basic, Fortran, Pascal ... and Smalltalk.
However, through Smalltalk-colored glasses, this is what's really happening:
The + message (arithmetic addition) is sent to the value 17. Hot on the heels
of the + message is an argument -- in this case -- the value 42. The + message
tells 17, "add yourself to the next thing coming your way, and return the
sum." The next thing down the pike is the value 42. 17 adds itself to 42, and
sends the value 59 back out again.
The value 59 is an object too. So, if you have something like this:
 (17 + 42) * 3
Smalltalk sees it as sending the + message and the argument 42 to the value
17, and then sending the * (arithmetic multiplication) message and the
argument 3 to the value 59, which was obligingly returned by the 17.
Cynics might argue that this is all word play, and that an arithmetic
expression is an arithmetic expression, not a bunch of numbers playing
ping-pong with plus signs. And I'd have to admit, it is word play -- just as
any computer language is an interplay of a set of symbols, a set of syntactic
rules, and a semantic architecture. Smalltalk uses standard symbols (unlike
some truly weird languages like APL) and a familiar set of syntactic rules.
What's different is the semantic architecture -- but if you refuse to accept
that architecture at face value, you're not playing by the rules, and I can
only advise you to go sit in the corner.


My Object All Sublime


A lot of Smalltalk's reputation for weirdness comes from this tendency to
anthropomorphize things like integer values. Objects know what to do -- their
suite of methods is the collection of things they can accomplish -- and they
do what they do in response to messages sent to them. One conjures up visions
of a little purple number 7 reading a telegram and doing some quick pocket
calculator work before sending a sum back by return wire. Just as we sometimes
use the metaphor of a stack of china plates when speaking of stack-oriented
languages like Forth, in Smalltalk we use the anthropomorphic zap on inanimate
(nay, disembodied) entities like forty-twos and screen windows and text
editors. It's a mnemonic device to remind us that data now runs things and
takes action through the code, and not the other way around.
The anthropomorphic metaphor was stronger in the old days, when languages like
Smalltalk were called actor languages because objects were seen as actors,
each performing a script on cue. The term "actor" has fallen into disuse
except in academe and in the title of another object-oriented language that
I'll deal with in a future column.
No, the term at the center of the maelstrom these days is object. An object is
the same concept in Smalltalk, Actor, C++, or the new Quick Pascal and Turbo
Pascal 5.5: A data structure consisting of some number of fields (rather like
the fields of a record) bound up with a suite of procedures that act on or
will those fields in performing the work that the object must accomplish.


Bugs Sealed in Amber


This welding together of code and data is called "encapsulation." In Smalltalk
the term is quite literal: An object's fields (called "instance variables" in
Smalltalk jargon) are so thoroughly encapsulated within the object that other
objects cannot directly perceive them. The closest familiar analog is the
implementation section of a Pascal unit, where data can be defined that cannot
be perceived from outside the unit, but only accessed by the code contained in
the unit.
Smalltalk enforces this as strictly as Pascal units do. Only an object's
methods may even know the names of an object's instance variables. To read the
value of some instance variable, a method must be defined to return a copy of
that instance variable, and a message must be sent to the object requesting a
copy of that variable. No method, no copy, no knowledge that the instance
variable even exists!
Now that's encapsulation.
Other object-oriented languages, as I'll explain in later columns, do not
erect quite such impenetrable walls around their object's inner fields. The
reason is pretty simple: Speed. Smalltalk imposes a tremendous amount of
overhead in enforcing encapsulation. The benefits are significant -- side
effects and "sneak paths" almost literally cannot exist in Smalltalk code --
but the costs in performance are high.
Encapsulation in Smalltalk is rather like potting instance variables in
milspec black epoxy resin. You get into and out of the module through its
terminal strip, period. C++ and Object Pascal do something a little more like
blister-packaging under transparent plastic: The goodies can be seen and felt
by the consumer, but direct manipulation is discouraged.


Family Resemblances


Encapsulation is a nice idea, but there's nothing in it (at least in the
C++/Object Pascal sense) that can't be accomplished by traditional C
implementations and good extended Pascals like Turbo Pascal. Code and data can
be combined in Turbo Pascal just by placing procedural types as fields in a
record along with data fields. This works well, and I've used it as a means of
organizing programs in the past.
What really sets Smalltalk and other true object-oriented languages apart from
the old school is that third Smalltalk architectural principle: Data Bequeaths
Everything It Has To Its Children. This is the notion of inheritance, and I'd
call it the single, most important aspect of object-oriented programming.
Pascal has something a little like inheritance. When you want to limit a data
type to some subset of the values of another type, you can define a subrange:
 TYPE CharCodes = 0..255;
Here, we've defined a subrange of type Integer that embraces only the first
256 integer values. Values of type CharCode, however, really are integers, in
that they may take place in integer calculations and be passed as actual
parameters in formal parameters defined as Integer. CharCode variables inherit
their integer-ness from type Integer, while taking on a new characteristic
specific to type CharCode: The limiting of values to those between 0 and 255.
Now, broaden this notion by an order of magnitude and you'll begin to get the
idea. A Smalltalk object can have child objects that inherit everything the
parent object has. Typically, however, child objects either add to or somehow
modify the instance variables or methods of the parent objects. You literally
write code in Smalltalk by choosing an existing object or objects as the
foundation of your application and creating child objects that modify the
parent objects in a way that gets your work done.
Where Pascal has data types, Smalltalk has object classes, and inheritance
works on classes rather than on individual objects. The real work of Smalltalk
programming lies in defining new classes and writing their methods. A new
class defined on the foundation of an existing class is called "a subclass;"
the class from which a subclass is defined is the subclass's "superclass."
A class, like a data type in Pascal, is a template. You create Smalltalk
objects by grabbing a class template and whacking out a new instance of that
object class. That's Pascal-think, though -- in keeping with Smalltalk's
anthropomorphic metaphor, it's more correct to say that new instances are
created by sending a message to the class in question, requesting that it
create a new instance of itself. Poof! The new instance happens.
Inheritance allows a second-level structure to be imposed on a program. Object
themselves are structures, and object classes are related to one another
within a structure-of-structures, called "an object class hierarchy." A
portion (a small portion) of the Smalltalk object class hierarchy is shown in
Figure 1. At the top of the tree is the class Object. Everything in Smalltalk
is descended from Object.
Figure 1: A portion of Smalltalk/V's class hierarchy

 Magnitude
 Association
 Character
 Date
 Number
 Floater
 Fraction
 Integer
 LargeNegativeInteger
 LargePositiveInteger
 SmallInteger
 Time

One such something is Magnitude, including all objects that may take values
that may be equal to, greater than, or less than other values of a similar
class. The children of Magnitude include characters, numbers, times, and
dates.


Distributed Functionality



This is all very handy for showing relationships among classes, but what is
actually handed down through the hierarchy? The answer is object behavior;
primarily methods that dictate what an object may do. "Object behavior is
distributed throughout the object hierarchy at appropriate levels." This is a
subtle, sneaky concept that won't necessarily make the lights come on until
you've done some work in Smalltalk. Generalized behavior is defined early on
in the hierarchy, up near the top. Objects modify the behavior of their parent
classes as they need to, but modifying only what they need to, leaving general
behavior intact where it is still valid.
As an example, consider Magnitude. Its methods define ordering and comparing
functions that embrace anything that can be said to take on values that may be
greater than or less than one another. One date or time can be greater than
another, as can one number. The general behavior that all magnitudes can share
is defined for class Magnitude. Behavior specific to dates or times is defined
within class Date and Time. Numeric functions such as reciprocal, cosine,
tangent, and so on would be meaningless as applied to time or date values, so
they are defined in class Number and inherited by the different numeric
classes such as Float and Integer.
The idea is not to duplicate any code needlessly. Internally, Smalltalk looks
a lot like a threaded-code Forth system. Methods perform specific behavior,
and then call parent methods to perform more general behavior, after which the
parent methods call their parent methods to perform even more general
behavior, and so on. As with Forth, there is a kernel of primitive methods
written in assembly language upon which the rest of the language is built.
There are other, even more subtle consequences of inheritance such as
polymorphism, which may in fact require a column all to itself. I'll come back
to inheritance again and again; it is the backbone of object orientation and
has more wrinkles than a cotton shirt in a hot dryer.


Talking Small


With very little fanfare, a product called Methods appeared in 1985 from Jim
Anderson's Digitalk Inc. in L.A. Methods was, remarkably enough, a textbased
implementation of Xerox's Smalltalk-80 specification. It may have been the
first low-cost object-oriented language to ever appear on the PC, and not one
programmer in a hundred had ever heard of it.
Methods grew into graphics overshoes and became Smalltalk/V a couple of years
later. (The V is a vee, not a five ... ) At $99 it remains the least expensive
object-oriented language of which I am aware. (Rumor holds that Quick Pascal
will come in at $99, but I have no hard information on it yet.) The
Smalltalk/V manual is excellent, and I think that the product represents one
of the best ways to come to understand object-oriented programming. It's
graphics-based, and the demo programs are very visual and lots of fun, with
animated dogs (of which I am inordinately fond) bouncing around the screen in
response to messages sent from the keyboard.
The Smalltalk/V product is 86-generic and runs on any DOS machine with CGA,
EGA, VGA, or Hercules graphics. Digitalk also has a more advanced Smalltalk
product for 286 and 386 machines, Smalltalk/V286, which provides better
performance and a richer feature set (including much more room to work) and
sells for $199. A Mac version is available, and provides an intriguing
portability path between the two hostile camps.
The only other DOS-based Smalltalk that I know of is offered by ParcPlace
Systems, a Xerox spinoff that is finally making some effort to put a Xerox
Smalltalk implementation in the hands of the DOS developer. The Smalltalk-80
Development System runs steep ($995) and requires a 386. In fairness, I must
admit that I've been using Smalltalk/V for two years and the ParcPlace product
for only about a month, so I'll refrain from detailed comparisons. The price
alone (and I feel price is important) takes the ParcPlace product down a few
notches in my esteem. It's good, but it's not a thousand dollars good. It's a
workstation product, ported from Unix, that has to stoop a little to make it
under DOS, whereas Smalltalk/V was designed from the ground up to run in a DOS
environment.
On the other hand, for those who care, Smalltalk-80 is the Real Thing, born
out of the primordial soup that Xerox continually cooks but rarely allows
others to taste. Its adherence to the Smalltalk-80 books (see product box) is
closer than that of Smalltalk/V, and in fact ParcPlace considers those books
its "real" user documentation. (The 3-ring binder document, sold with the
product, is reference-oriented, heavily technical, and fragmented.)
If you want a taste of Smalltalk, or a taste of OOP, pick up Smalltalk/V. It's
cheap and it works like a charm. The 286 product is there if you want more
room and more speed. I'm hoping (but not expecting) that ParcPlace will port
Smalltalk-80 to Presentation Manager soon, at which point the price becomes
less of an issue. Anything that manages the complexity of an API like PM's is
valuable, and for PM development Smalltalk-80 would almost certainly be worth
the considerable price.


Those Legendary Smalltalk Books


Smalltalk is an anomaly in that it had superb documentation on the market long
before there was an implementation that anyone could buy. A series of three
books from Addison-Wesley appeared in the early 1980s, and two of the three
are required reading for anyone interested in Smalltalk. The first,
Smalltalk-80: The Language and its Implementation is the "white book" for
Smalltalk, written by Adele Goldberg and David Robson. Beautiful, interesting,
literate, and huge, the book defines the language and puts you on a sound
theoretical footing. The other book, Smalltalk-80: The Interactive Programming
Environment, by Adele Goldberg, describes the standard Smalltalk-80
environment implemented completely by the ParcPlace product and closely by
Digitalk's. It's about browsers and editors and form tools, and is essential
if you intend to work in the language. The Goldberg/Robson text, on the other
hand, is sufficient if you're interested only in familiarizing yourself with
the language's principles.
A third book, Smalltalk-80: Bits of History, Words of Advice, by Glenn
Krasner, is meta-Smalltalk, that is, smalltalk about Smalltalk. It provides
some fascinating history about the origins of the language, and liberal doses
of hacker-heavy lore on how to bring up your very own implementation -- which
is not something I would try to do in ten thousand years. The book is notable
for its photo of the PARC NoteTaker machine, which was a 256K 8086-based
spitting image of the Osborne 1, in regular use in 1978. Xerox really did
invent and throw away the personal computer ... over and over and over again.
Addison-Wesley has since published a few additional Smalltalk texts, but none
of them come close to any of these three in quality or completeness. Highly
recommended.


The Downside


There's a lot more to say about Smalltalk, and I'll touch on it from time to
time in these columns. I like the language a lot, and I credit it with
preparing me technically for the arrival of this crowd of OOP steamships that
I cataloged last issue.
On the other hand, Smalltalk will probably never cross the line to become a
mainstream language, as the weavers of its legend have been harping for many
years. The reason is purely practical: Smalltalk is by nature an interpreter,
and unless everybody has the interpreter, the grass-roots critical mass of
support among recreational hackers and part-time programmers just won't be
there. IBM put Basic on the map by throwing a solid interpreter in the box
with every machine they sold. Had they done that with Smalltalk, Smalltalk
might be where Basic is today, or close.
Unfortunately, it's tough to write a Smalltalk, just as it's fairly easy to
write a Basic. Furthermore, Smalltalk is sluggish on 8088 machines, in part
because of its interpreted nature, but mostly because it is inescapably
graphics-based. Digitalk might have done well to preserve their Methods
product in dry ice for the current OOP craze as a text-based $49 loss-leader
to get people knowledgeable about OOPs in general and Smalltalk in particular.
(You listening, Jim?) Given a $99 price and some superb documentation from
Addison-Wesley, Smalltalk/V is perhaps the best current environment to learn
OOP principles ... but to apply those principles broadly you're going to have
to move to a mass market language like Turbo Pascal.
Smalltalk does well as a prototyping tool, rather like a thinking man's Dan
Bricklin for graphics apps. And if you've got a fast machine and you're in a
position to work entirely within the Smalltalk environment, you can create a
lot of powerful tools quickly. Still, a mainstream language it isn't, and I
caution those of you who have been dazzled by the legend not to expect
effort-free programming. Smalltalk is a computer language, really. Tools is
tools. The magic, if anywhere, is in you.


Products Mentioned


The Smalltalk-80 Programming System ParcPlace Systems 1550 Plymouth Str.
Mountain View, CA 94043 415-691-6700 $995.00 (requires 386)
Smalltalk/V Digitalk, Inc. 9841 Airport Blvd. Los Angeles, CA 90045
213-645-1082 General 86-family version $99 286/386 version $199
Smalltalk-80: Bits of History, Words of Advice by Glenn Krasner
Addison-Wesley, 1983 ISBN 0-201-11669-3 Softcover, 344 pp. $19.95
Smalltalk-80: The Interactive Programming Environment by Adele Goldberg
Addison-Wesley, 1984 ISBN 0-201-11372-4 Hardcover, 516 pp. $29.95
Smalltalk-80: The Language and its Implementation by Adele Goldberg and David
Robson Addison-Wesley, 1983 ISBN 0-201-11371-6 Hardcover, 714 pp. $34.95



















August, 1989
SWAINE'S FLAMES


Unbundled Integration




Michael Swaine


Amanda Hixson was one of the more savvy writers we relied upon to review
software back in the early 1980s when I was an editor at Info World. We knew
we could count on her to hold software to high standards, to evaluate it from
the user's viewpoint, and to get quickly to the essence of what was right or
wrong with the product. When, years later, she went to work for Apple, I
wondered if it was the right move; writers should write, I thought.
Over the years, Hixson demostrated to the satisfaction of any observer that
her decision was a savvy one, working her way up to a position reporting to
Randy Battat, the VP of Product Marketing. Early this year she acquired a new
title, when she presented Battat with the idea that the Macintosh System
Software should be marketed. It wasn't a difficult sale; although Apple
doesn't have to sell System Software on its own, the company understands that
it's the System Software that sells the little gray toasters. Battat agreed
that it was a good idea, but pleaded that he had no time. "I do," she said,
and he made her Product Marketing Manager for System Software. The first big
step Apple has taken in marketing its System Software was the press day at the
May Developers' Conference, when Apple spelled out which of the rumored new
features were in fact going into the next major system release. The
announcements, along with other announcements coming out of the Developers'
Conference, are worthy of note to anyone remotely considering developing a Mac
product but not already in on all the former secrets.
Some of the features Apple will be folding into System Version 7.0 are obvious
and have obvious developer consequences. The virtual memory and 32-bit
addressing, if cleanly implemented, should be beneficial for everybody. The
nice feature of VM for users is the ability to buy memory for average need,
rather than for maximum need. Of course, System Version 7.0 will require a
minimum of 2 Mbytes of memory. The outline fonts and layout manager, giving
Apple real device-independent typographic-quality text, will be very
interesting for applications that can make effective use of it. And the
database access mechanism looks interesting.
But I found the InterApplication Communications architecture (IAC) the most
intriguing.
The IAC gives the developer a choice of ways to get information from one
application to another. Beyond the existing Clipboard for copying and pasting
text and pictures, there will be a Live Copy/Paste facility that programmers
can incorporate into new applications. It uses a publish/subscribe model: The
user of one application selects some data in a spreadsheet he's working on
(for example) and "publishes" it. Another user can "subscribe" to the
published data, which brings it into the word-processing document he's working
on (for example), and the data will change in the subscriber's document
whenever the publisher changes the original. Apple is providing developers
with a toolbox and user-interface guidelines for implementing Live Copy/Paste.
Then there are the Event Manager extensions. Because the Mac system is
event-driven, extending the Event Manager is a logical way to allow
inter-application communication: Just let one application message another. The
trick is that the other application has to be able to recognize the message,
and that requires some guidance from Apple. What Apple is providing is a
protocol of standard messages for inter-application communication: Generic
spreadsheet messages, for example, that only need some conforming spreadsheet
program (not necessarily Excel) to be on the receiving end.
The low-level Program-to-Program Communication mechanism (PPC) is the tool the
higher-level components use to get their jobs done; it's also the tool a
developer would use to develop more subtle inter-application links than the
higher-level tools allow. It also permits desk accessories, control panels,
and other chunks of code to communicate.
Not too long ago, a lot of people put their money and time into integrated
software packages: Omnibus programs that melded spreadsheet, text, graphic,
and other kinds of processing. Unfortunately, not enough of these people were
customers. One theory about the failure of integrated software is that it was
the bundling, not the integration, that didn't work. If this theory is
accurate, then users would love it if they could pick the applications they
wanted to use and know that they would work together as tightly as the
components of one of the integrated packages. Or more tightly.
That's what Apple has in mind; there was a lot of talk at the conference about
not reinventing the wheel, about having an application signal another
application when it needs something done outside its area of specialization.
I'd like to see that.








































August, 1989
OF INTEREST





A specification describing a general approach to object-oriented software
design has been published by Interactive Development Environments Inc. The
specification, called Object-Oriented Structured Design (OOSD), is a
non-proprietary notation for design of software systems that is
language-independent (supporting languages such as C++, Eiffel, Smalltalk,
Fortran, C, and Pascal) and synthesizes traditional top-down design with
modern concepts for object-oriented design into a comprehensive approach to
modeling software architecture. In addition to providing improved support for
the pivotal architectural design step of the software development process,
OOSD supports automated generation of code for multiple programming languages,
provides partitioning of a system into a coherent software architecture,
facilitates reuse of design elements, and provides a clear notation for
communication among designers and reviewers. The key building blocks in OOSD
are modules, lasses, and monitors. Reader Service No. 20.
Interactive Development Environments, Inc.
595 Market St.,
12th Floor
San Francisco, CA 94105
415-543-0900
Hamilton C shell, recently released by Hamilton Laboratories is an interactive
language for OS/2. The company says that its key benefit is that it allows you
to describe what you want the machine to do much more quickly and easily, even
if what you want is fairly complex. Some of its features include: Fully
nestable programming constructs for iteration and condition-testing;
variables, arrays, and a wide range of expression operators and built-in
functions; a powerful history mechanism for recalling and editing past
commands; language constructs for I/O redirection, piping, background
execution, and parallel threading; and more. Hamilton C shell complies with
the Berkeley 4.3 Unix Programmer's Manual. C shell requires 286- or 386-based
AT or PS/2 or compatible with a minimum of 2 Mbyte of RAM and a 2-Mbyte hard
disk. All executables will run properly inside a Presentation Manager window.
The cost is $350. Reader Service No. 22.
Hamilton Laboratories
13 Old Farm Road
Wayland, MA 01778
508-358-5715
VM-DEBUG (The Virtual Machine Debugger), a debugging tool for PCs, XTs, and
ATs, has been released by Wendin Inc. VM-DEBUG is an interpreter whose
language is 8088 machine code extended with the real-mode instructions of an
80286. It can stop the execution of a program at any point, examine or alter
memory or registers, examine the program, and determine where the program has
been. Unlike normal debuggers, these functions are accomplished by a program
outside the addressing space of the program or system under test, so that
VM-DEBUG can never be altered or destroyed by an errant program. VM-DEBUG also
has the ability to trace DOS itself or the ROMs, and set breakpoints within
ROM. It retails for $99. Reader Service No. 23.
Wendin, Inc.
P.O. Box 3888
Spokane, WA 99220-3888
509-624-8088
A new book that describes how assembler macros are used with the 80386 will be
released next month by Tab Professional and Reference Books. The book,
entitled 80386 Macro Assembler With Toolkit and written by Penn Brumm and Don
Brumm, discusses the Microsoft Macro Assembler (MASM) operands and operators,
program structure and file control directives, global, conditional, and macro
directives, and interfacing MASM with high-level languages. The 608-page book
retails for $25.95. Reader Service No. 28.
Tab Professional and Reference Books
Blue Ridge Summit, PA 17294-0850
800-822-8138
Lattice C 6.0, for DOS and OS/2, is now available from Lattice Inc. The system
uses global optimizing technology (see "Optimization Technology" by Keith
Rowe, DDJ, June, 1989 for details on this technology) and includes a
source-level, native, and cross debugger and an integrated screen editor. The
compiler also supports automatic register variable and registerized parameter
support, built-in function support, and support for precompiled header files.
The system includes more than 800 prewritten functions, multithreaded
libraries, DLL support, and also comes with special graphics, database,
communications, and screen management libraries.
Lattice C 6.0 runs on IBM PC/XT/AT and compatibles and requires DOS 2.1 or
OS/2 1.0 or later. The price for version 6.0 is $250, although owners of
version 3.4 can upgrade for $75. Owners of all other versions can upgrade for
$115. Reader Service No. 32.
Lattice, Inc.
2500 S Highland Ave.
Lombard, IL 60148
312-916-1600
The new hardware-assisted model of the Periscope debugger is now available
from the Periscope Company. Periscope Model IV runs on 80286 and 80386 systems
with CPU speeds up to 25MHz and zero wait states. It works on any
PC-compatible with a standard AT-style bus. Periscope IV is designed to help
software developers achieve optimum performance from their own software, and
can monitor program execution in real-time. It can track down memory
overwrites because it does not slow down program execution as software-based
debuggers do. The Periscope software provides source-level and symbolic
support for Borland, IBM, Lattice, Manx, Microsoft, and other languages.
Prices range from $2195 to $2995. Reader Service No. 26.
The Periscope Company
1197 Peachtree St.
Plaza Level
Atlanta, GA 30361
404-875-8080
Microsoft Corporation has released a number of programming tools for the OS/2
Presentation Manager that complement its OS/2 language compiler family. At the
centerpiece is the Microsoft OS/2 Presentation Manager Toolkit, a complete
"one box" answer to Presentation Manager developer's needs, and it includes a
collection of graphical tools, books, Quickhelp documentation, sample code,
and electronic support. These components are available individually or in
various combinations. Upgrade pricing available. Minimum system requirements
are an 80286 or 80386 processor running OS/2 1.1; 2.5 Mbytes of user memory; a
high-density 5.25-inch, or a 3.5-inch drive; and a hard disk. The toolkit
supports EGA and VGA graphics adapters and is compatible with any Microsoft
OS/2 language compiler. The complete Presentation Manager Toolkit is $500.
Upgrade pricing is $200 for registered owners of the OS/2 Programmer's
Toolkit. Reader Service No. 24.
Microsoft Corporation
16011 NE 36th Way
Box 97017
Redmond, WA 98073-9717
206-882-8080
Microsoft Corporation has updated its Fortran compiler with the recent release
of the MS Fortran Optimizing Compiler, Version 5.0, which supports DOS and
OS/2 1.1 with Presentation Manager. The company claims that Fortran 5.0 offers
the broadest VAX and IBM VS Fortran compiler syntax available on a PC,
supporting virtually all VAX Fortran syntax other than VAX operating system
calls. Fortran 5.0 comes with a graphics library, the CodeView source-level
debugger, the MS Editor, LINK, NMake, LIB, and OS/2 support and extended
syntax. Minimum system requirements are a PC with 320K of available user
memory and DOS 3.0 or OS/2 1.1. The suggested retail price for Fortran 5.0 is
$450, although registered owners of MS Fortran 4.1 who bought the compiler
after April 1, 1989, can upgrade for free. Other 4.1 registered owners can
upgrade for $100, 4.0 owners for $150, and $250 for owners of earlier
versions. Reader Service No. 27.
Microsoft Corporation
16011 NE 36th Way
Box 97017
Redmond, WA 98073-9717
206-882-8080
A protocol specification called the Virtual Control Program Interface (VCPI),
which is designed to prevent conflicts between 386 software from different
companies, has been sanctioned by a number of different hardware and software
companies. Originally sponsored by Phar Lap Software Inc., VCPI addresses
technical issues between control programs and DOS extenders that arise due to
the nature of the 386. These conflicts include CPU mode switching, hardware
interrupt processing, and the sharing of extended memory. Left unresolved,
these conflicts force the user to turn off control programs in order to run an
extended application. The agreement of a programming standard allows the new
categories of software brought on by 386 PCs to play together. In addition to
Phar Lap, other sponsors of the VCPI specification are Quarterdeck, Auadram,
Lotus, A.I. Architects, Qualitas, and Rational Systems. The new VCPI
specification will be available on June 1, 1989, at no cost through Phar Lap
Software. Reader Service No. 29.
Phar Lap Software, Inc.
60 Aberdeen Ave.
Cambridge, MA 02138
617-661-151O
Clear Software Inc. has recently introduced Clear+ for C, a product that helps
developers understand C code. It is designed to instantly produce high-quality
system documentation and to clarify the logic of C programs and applications.
According to Sandy Rudy, who designed the language portions of the program,
"Clear+ acts like a compiler but the output doesn't run, it shows [on the
screen]." Clear President Yadim Yasinovsky went on to tell DDJ that "Clear+ is
ideal in an environment when you inherit someone else's code."

Clear reads the source code of any C application and instantly produces the
system tree chart, function flow charts, formatted source listings, function
cross references, and prototype files. As flow charts and tree charts are
output to a printer, screen, or a file, the program automatically calculates
the spacing, number of pages (or screens) required, and the placement of
symbols for the flow chart. And diagrams look exactly the same on screen as
they do in print. Clear also features powerful hardcopy controls, various
graphics options, and options to directly invoke a text editor or compiler of
the user's choice. Clear can analyze C source code as is or after it has been
preprocessed with either an external or internal preprocessor.
Clear has been designed for use with any IBM PC/XT/AT or compatible and
supports Hercules, CGA, EGA, VGA, most dot-matrix printers, and HP LaserJet+
printer. Minimum RAM requirement is 512K. Clear currently supports Microsoft
C, QuickC, Turbo C, and any generic C compiler. Clear+ for C retails for
$199.95 plus $5.00 for shipping and handling. Clear+ for dBase is also
available for the same price. A combination package (Clear+ for C and Clear+
for dBase) is sold for $310.00 plus $10.00 for shipping and handling. Reader
Service No. 31.
Clear Software, Inc.
637 Washington St.,
Ste. 105
Brookline, MA 02146
617-232-4720
The QuickC family has grown with Microsoft Corporation's introduction of the
QuickC Compiler with QuickAssembler 2.01. This package is an integrated
C-and-assembler environment which includes an integrated editor, compiler, and
debugger. The C portion of the compiler has not changed from QuickC 2.0. The
assembler portion consists of a macro-assembler add-in module built around
MASM 5.1, which features single-pass assembly technology.
Additionally, the package provides incremental compiling/linking and
recompiles/reassembles and relinks only modules that have been changed since
the previous compilation. The debugger can be used to debug both assembler and
mixed-language programs. The package is supported by an on-line reference
system in addition to books and manuals.
System requirements are MS-DOS 2.1 or higher and 512K of RAM; a mouse is
optional. The QuickC compiler and QuickAssembler package sells for $199
although registered QuickC Compiler 2.0 owners can buy the assembler addin
module directly from Microsoft for $75. Reader Service No. 33.
Microsoft Corp
16011 NE 36th Way
Box 97017
Redmond, WA 98073-9717
206-882-8080
A page makeup system that supports C compilers is now available through
Quality Software. The system, called FutureComp Laserline, is a MS-DOS version
of the company's FutureComp Page Makeup Environment which is primarily
designed to produce custom page makeup programs for directories, catalogues,
parts lists, technical manuals, and similar applications.
The Laserline version of FutureComp can output to most printers and
typesetters that support PCL or PostScript. (The Proline version of the system
supports high-resolution typesetters like those from Mergenthaler and
Autologic.)
In addition to font handling and hyphenation dictionary utilities, FutureComp
provides C functions for composing blocks of text, placing illustrations and
tables, and for positioning these blocks anyway on a page. Users can also
construct logical fonts for special applications.
The system supports any compiler that can call ANSI C functions, and
applications programs created with FutureComp can be recompiled on any
computer system that supports C. The system runs on any IBM PC/XT/AT using DOS
2.0 or higher; Microsoft C 5.1 is also required. FutureComp sells for $695 and
includes PostScript and HP PCL translators. Reader Service No. 34.
Quality Software
60 Lewis St.
Newton, MA 02158
617-965-2231
As C portability tools become increasingly important, new ones continue to pop
up, the most recent being Abraxas Software's CodeCheck. What this package does
is analyze C source code in terms of its portability between PC-DOS, OS/2,
Macintosh, Unix, and VMS. "In essence, CodeCheck is an expert system that
looks at the code and tells if it is portable or not," Abraxas president
Patrick Conley told DDJ. During set-up, the system loads a set of rules for a
particular platform (DOS, 0S/2, Mac, and so on), then checks the code against
those rules. It then identifies any code that is not portable to or from any
environment, including C++ and ANSI C. It also quantifies code maintainability
with user-defined measures at all levels and identifies unacceptable style or
usage.
CodeCheck supports all C compilers from major vendors and requires 512K of
memory. It sells for $295 and Conley said it would be shipping in mid-August.
Reader Service No. 35.
Abraxas Software, Inc.
7033 SW Macadam Ave.
Portland, OR 7219
503-244-5253
HCR Corporation has introduced the packaged version of an advanced Unix C++
compiler based on C++, Release 2.0, from AT&T. HCR/C++ provides all of the key
features of C++, such as type safe linkages, default membership
initialization, and the ability of each class to define its own operators. It
will run on most 386-based systems. HCR's dbXtra, based on dbx Version 3 from
Berkeley 4.3 BSD Unix, adds the ability to operate through windows, permitting
users to review their output and source code easily, even on standard
terminals. HCR/C++ allows direct debugging of C++ and window access to the
translated C source code. All C++ code is translated into C before execution
so programmers can apply dbXtra to examine either C or C++ code during
debugging.
Initial copies will be available at an introductory price of $499 (50 percent
off the list price of $995). HCR's standard support and upgrade options are
available; each user of HCR/C++, Version 1, also will have the option to
upgrade to Version 2 for a delivered price of $99. Reader Service No. 36.
HCR Corporation
130 Bloor Street
West Toronto, Ontario, Canada
M5S 1N5
416-922-1937
Quibus Enterprises has updated its Fortran Development Tools package, which is
designed to help Fortran programmers maintain their code. Release 2 of the
package automatically deals with poorly formatted or heavily modified code
using a pretty printer that indents, renumbers, and generally cleans up the
code. The program also converts GOTOs to structured IF-THEN-ELSE blocks.
A preprocessor for supporting conditional compilation and code sharing is
included, along with a utility to extract subroutines from source files. All
tools accept Fortran 77 plus extensions from VMS, Lahey, Microsoft Fortran.
The tools run on IBM PC/XT/AT compatibles with 512K of memory and sell for
$129. Reader Service No. 37.
Quibus Enterprises, Inc.
106 N. Draper Ave.
Champaign, IL 61821
217-356-8876




















September, 1989
September, 1989
EDITORIAL


Just Around the Corner




Jonathan Erickson


Whether you're ready or not, the end of this year (or the beginning of the
next, depending on how you look at it) is right around the corner. 1990 and
we're on a downhill run into the twenty-first century, and writing the correct
year on your checks will be the least of your problems. We'll have to worry
not only about the right decade, but, before long, the right century, too.
Somehow that still sounds a little unreal. For DDJ, the new decade will bring
the beginning of our fifteenth year, spanning three decades. That's not bad
for any magazine, let alone one that deals with computers.
For our part, we're lining up next year/next decade articles and, once again,
we need your help to make sure we're getting out the right kind of articles
for you. When we published the 1989 editorial calendar, we heard from a lot of
you who sent in suggestions and/or wrote articles (thanks from us and from
your fellow readers), and we'd like to hear what's on your mind this year.
So in the first part of 1990, our monthly themes will be:
January Real-time Programming
February Windowing Systems
March Assembly Language Programming
April Neural Nets
May Memory Management
June Hypertext
Later in the year, we'll cover topics such as graphics programming, our annual
C issue, structured languages, operating systems, and object-oriented
programming. I'll post more specifics at a later date.
As always, we're looking for focused, task-specific articles that solve a
particular programming problem, present a new or unique technique or utility,
and list your program code. If a subject that interests you isn't mentioned
above, that's okay, the list isn't meant to be all-inclusive. Only about half
the articles in any issue are theme related; the others are on all kinds of
programming-related topics. Any language -- C, Basic, Pascal, Modula-2, C++,
Fortran, Forth, Smalltalk, assembly, and so on -- is fine, as is any operating
system platform, including Unix, DOS, OS/2, Macintosh, or whatever. (One
really popular article we ran not long ago was on the Amiga and I can already
hear "I told you so...." echoing from Amiga fans.)
So, if you have an idea for an article that you'd like to write or to see
someone else write, give Mike or me a call or drop us a letter (E-mail or
regular) describing what you have in mind. (For CompuServe, address to
76704,50, MCI Mail care of DDJ, BIX to jerickson, or if you're using the DDJ
listing service, jerickson.) For regular mail, be sure to send it to DDJ, 501
Galveston Dr., Redwood City, CA 94063. We quite often get letters addressed to
us in care of our subscription service in Boulder, Colo. The letters
eventually get here, but sometimes weeks later.
In other news, AT&T has finally released its 2.0 specification for C++. It's
been a long time coming and everyone, particularly compiler vendors who've
been waiting for the other shoe to drop, can now get down to the business of
forging ahead with their C++ plans. (See the "Of Interest" column in this
issue for details.)
While it's great that the specification is finally available, it was the 2.0
"language system" (don't call it "preprocessor") license arrangements that
caught my eye. To license C++ 1.02, all you had to do was pony up $2000, not
necessarily a big deal either for little developers or the big guys. That's
not the story with 2.0, however. If you currently hold a 1.02 license, a 2.0
license will cost you $10,000, and if you don't hold a 1.02, but want a 2.0
license, the cost is $20,000. (Okay, okay, I know AT&T has to cover all those
development costs and I wouldn't expect the company just to give it away.)
If you fall into the former category (that is, you want the 2.0 but don't have
1.02), here's a tip: Get a license to 1.02 for $2000, then immediately upgrade
to 2.0 for $10,000, saving yourself $8000 in the process. I checked with an
AT&T rep who said they would do this, but for how long, she didn't know. If
you take advantage of this money-saving tip, don't forget about who put you
onto it. You might want to consider sharing your savings; 10 percent is a nice
round figure, made payable to the editors of DDJ, of course.

































September, 1989
LETTERS







Good-bye, Again


Dear DDJ,
We were all shocked to hear about the death of Kent Porter. Having seen him so
recently at a press conference made the news all the more incredible. What can
we say?
As an industry leader, a writer, programmer, and technical editor, Kent taught
us a lot. His opinion was respected. He inspired us. It's hard for us to
believe he's not here. Little things, like leafing through Dobb's only to
stumble across the "Graphics Programming" column, become painful reminders. We
remember how precious life is and how seldom we stop to appreciate those
around us -- both coworkers, and friends -- for the special people they are.
This is hard for us in many ways. Most of us know Kent through his words. We
learned about the intricacies of VGA graphics while we laughed at an anecdote
about Kent's wife having a salesman match wall paint against her wet
washcloth. We can't pick up a copy of Turbo Technix without seeing Kent's
name. His wide range of interests and his good humor made his appeal broad.
Whether we are interested in C or Pascal, bezier curves or binary trees, we
always find Kent teaching us something.
We shared a common vision with Kent and each other that has bound us since the
early days. We saw the world around us changing and felt we had a hand in
those changes. Computer Lib, Dream Machines, the Community Memory Project, and
the People's Computer Company, with Dr. Dobb's cheering us all on from its
leaves of scratchy newsprint. We never minded that Dobb's looked more like the
first Whole Earth Catalog than the slick and polished journal that would lead
a revolution, changing the way we work and play forever. We thought that we
could do anything. Through our intense commitment and energy we could shape
and build a world that our parents and our parents' parents couldn't even
imagine.
Kent's death brings us back to earth. Dreams are smashed, a friend is lost, we
remember our mortality. Tributes, like funerals, are for those left behind. We
don't know how to ease the way we feel about Kent. Maybe for a moment we can
put aside the petty rivalries and the intense competition, and remember that
we all live now on a tiny planet, in the global village that shrinks daily in
size. And maybe we can remember that the greatest thing we can do for our
friends is appreciate and affirm them now when a kind word can carry them
through a hard day rather than waiting until our voice can no longer be heard.
We will miss Kent.
Philippe Kahn
Greg Voss
Robert Dickerson
Anders Hejlsberg
Dick O'Donnell
Tom Wu
Gene Wang
Brad Silverberg
David Intersimone
Gary Whizin
Rick Shell
-- Borland International
Scotts Valley, Calif.


Another View from the Trenches


Dear DDJ,
When I read the letter from Mr. Rick Rodman in the "Letters" column (DDJ, July
1989), I was surprised. I enjoy the bit-twiddling articles just as much as the
next reader. However, I feel that there are options other than bit-twiddling
and that they should be explored in a forum such as DDJ.
I disagree with Mr. Rodman's opinion, "... all those object-oriented paradigms
aren't worth a plugged nickel in the real world. All that structured
programming data abstraction is the wrong way to go, too." Those statements
are very general. While those techniques have been shown to improve many
aspects of programmer productivity, they are not necessarily useful in every
situation and may not produce shorter program execution times. Perhaps the
type of environment he is working in would not be suitable for the use of
structured, object-oriented, or other advanced techniques.
I enjoy the current format of DDJ very much. I applaud the addition of the
"Graphics Programming" column, the OS/2 coverage, and the coverage of other
environments such as the Mac and Amiga. I am looking forward to the coverage
of OOP in general and Smalltalk in the "Structured Programming" column. I
always manage to read my copy of DDJ within two days, starting with "Swaine's
Flames," of course. I just can't put it down! Keep up the good work!
John H. Critchfield, Jr.
Duenweg, Missouri


keyhit( ) without MASM


Dear DDJ,
In addition to Al Stevens' discussion of the Control-Break abort handling with
Microsoft C in the February issue, here is a version of Al's keyhit( )
function that is purely written in MSC and therefore doesn't require MASM to
be compiled.
The following version of keyhit( ) uses the ROM BIOS keyboard buffer head and
buffer tail pointers: If the pointers are equal, no input from the keyboard is
pending; otherwise, a key has been pressed and its ASCII value put into the
keyboard buffer by BIOS. The pointers are located in the BIOS data area at
0040:001A and 0040:001C, and can easily be accessed using Al's peek( )
function. Here's all there is to it:
 int keyhit(void)
 {
 return(peek(0x40,0x1a)!= peek(0x40,0x1c));
 }
Ralph Langner
Langner Expertensysteme
Hamburg, W Germany



Fanning Mohr's Flames


Dear DDJ,
Metz's response to Mohr's flames (DDJ, June 1989) seems to me to have missed
the most important lesson inherent in Mohr's commentary: When we ignore
history, we are condemned not merely to relive it, but to suffer living in an
inferior version. RT-11, and its big brother TOPS-10, are well-designed
operating systems that run on well-designed hardware. They owe their success
to a combination of features that are much too rare in today's micro world:
1. Well-designed, orthogonal instruction sets invoked by consistent mnemonics.
Useful repertoires of addressing modes, operating over linear address spaces.
2. Uniform subroutine calling conventions across a variety of high-level
languages, and consistent, well-documented, operating system calls.
3. A consistent, highly intuitive command-line syntax, with highly mnemonic
key words, each of which can be abbreviated to the fewest characters which
make it unique. A well-chosen set of options for each command, combined with
well-chosen defaults.
4. Facilities for user profiling, file protection, access control, and usage
accounting that make it easy to offer the convenience of a command-line
interface for the programmer and the security of an idiot-proof interface for
the naive user.
Although some of these features have been incorporated in micro operating
systems, I've yet to see anything that approaches the seamless implementation
of the DEC products. And let there be no doubt that the quality of the
operating system strongly affects a programmer's productivity. Over a period
of some twelve years, I watched a shop running TOPS-10 outproduce a
neighboring shop running VM/CMS, typically by a factor of about ten to one.
Can we afford to discard such capable tools?
Similar arguments apply to TECO. Yes, Borland's integrated programming
environment is cozy, but on the whole, full screen editors waste too much time
in navigation and screen painting. For a simple demonstration, try a global
search and replace with CP/M's ED, and then do the same job with Wordstar.
It's faster to copy a large DOS file to a CP/M disk, do the replacement with
ED, and then move the file back to the DOS machine.
ED is, of course, a greatly simplified version of TECO. It lacks TECO's
conditionals, flow control, Q registers, pushdown list, bounded searches,
wildcard searches, etc. TECO is as much a programming language as an editor. A
small subset of TECO is adequate for a majority of editing jobs, but very
powerful tools are available whenever they are needed. And although TECO is an
extremely terse language, most of the commands have highly mnemonic one-or
two-character "key words."
One of the Mohr important points in Doug's letter is that a program as
powerful as TECO can run on a machine as small as a 64K PDP-11. In the CP/M
world, ED wastes quite a bit of memory because it was written in a high-level
language, but it still runs in less than 8K. An assembly language rewrite
could roughly halve the RAM requirement, leaving plenty of room for some of
the more esoteric features of TECO. I know of at least two people who were
convinced that writing their own versions of TECO for DOS machines was a sound
investment in future productivity, and I am currently working on a version for
CP/M.
On a related subject, Jeff Duntemann's column in the June issue leads me to
wonder why, after all these years, we still toy with clever kludges instead of
coding the obvious calendar algorithm. Granted, the calendar display was
intended only to illustrate screen management techniques, but as Jeff himself
asks, is this angst really necessary? Given a suitable choice of offset for
the year, 16-bits will hold 179 years' worth of days, and whose code will
survive that long? Jeff's simple function to test for leap years (certainly
justified in this application) suggests that he has no such expectations. Why
not stick to integer arithmetic? If one really needs an archaeological time
scale, a LONG-INT will handle five million years. And in making the primary
test for leap years, one can save a few machine cycles by coding
 IF (Year AND 3) = 0 THEN
 Is LeapYear:= TRUE
If the full leap year algorithm is required, this logic can be applied twice,
once before and once after dividing by 100.
And finally, endorsing Mark Pickerill's views in June "Letters," do we build
bigger and faster machines, not because we need more computing power, but
because it's more comfortable to repeat the familiar mistakes than to risk
making new ones?
Yours for the more efficient use of resources.
Arpad Elo, Jr.
St. Johnsbury, Vermont


Hal's A Hit


Dear DDJ,
I have decided to renew my subscription to Dr. Dobb's solely on the basis of
Michael Swaine's interview with Hal Hardenbergh. I have followed him over the
years through DTACK Grounded, and have always admired him for the fact that
(as I knew the facts) he was right MOST of the time. This, for a writer, is
quite a record and generally unsurpassed by any other writer whom I read in
the computer field.
He is a first-rate engineer, clearly, and I cannot say the same for any other
engineer who writes for the public. It was a pleasure to see his existence
recognized by Dr. Dobb's, does this mean Dr. Dobb's is "cutting through the
crap" also? I can now hope to learn more about AI through Hardenbergh's
incisive analytical powers: either in Dr. Dobb's or Programmer's Journal.
Swaine's was a very good interview -- keep up the good work! Maybe there is
hope after all.
John Griffith
Yorktown Hts., N.Y.


Superlinearity, Smoke or Mirrors


Dear DDJ,
In the July DDJ "Letters" department was a letter entitled "Superlinearity
without Mirrors." Phil might not be using mirrors, but he is certainly using
slight-of-hand [sic]. The example he described is not a demonstration of
superlinearity, but a reasoning fallacy.
Reviewing the example, Phil says that one processor will require (j+x) steps,
and ten processors will require x steps per processor, or 10x total.
But this has nothing to do with multiple processors, and everything to do with
the fact that he changed the search order. If he had one processor searching
the first element in each of ten partitions, then the second element in each,
etc., he would get the same results as his multiprocessor system. Likewise, if
his first processor gets elements 0, 10, 20,... the second gets 1, 11, 21,...
etc., he would get the same results as his uniprocessor.
The point is that the location of the match is random. What he actually
demonstrated is that for some values, searching in one order will find the
value sooner than searching in another order.
It does not matter what order is used, since they all average out anyway. So
you will use a search order that minimizes the overhead of the search.
John M. Dlugosz
Plano, Texas















September, 1989
AUTOROUTING WITH THE A* ALGORITHM


Searching for the best PC board layout


 * This article contains the following executables: NEVIN.ZIP


Randy Nevin


Randy holds a BS in computer science from Oregon State University and an MS in
computer science from the University of Washington. He has worked for
Microsoft since 1983 on various programming languages and networking products.
He can be reached at 1731 211th PL NE, Redmond, WA 98053.


A few years ago, a friend of mine designed an adapter board for the IBM PC.
The tools he used were blue and red strips of tape, a sharp knife, large
sheets of clear plastic, and a generous amount of patience. It took him
several weeks, and after the first board was tested he discovered that some of
the traces were incorrect and had to be cut with the knife and rerouted with a
solder and wires. This caused me to start thinking about ways to use the power
of the computer to simplify this tedious, error-prone job.
The design of a printed circuit board implements an electronic circuit. First
you create a schematic. From this you derive a list of chips and other
components that perform the required functions, and a list of the pins that
need to be connected. Together, these lists are referred to as the "net-list."
As long as the connections are made correctly, you usually don't care where
the traces (the wires embedded in the board) are placed.
As you can imagine (or may already know), designing a PC board is a complex
search problem with a seemingly infinite number of possible solutions.
Luckily, there are algorithms from the field of artificial intelligence that
we can use to design computer programs called "autorouters" that do this
searching for you. In this article, I'll look at two algorithms: The
breadth-first and A* (pronounced as "A Star") search algorithms. This article
is actually based on an application I wrote to layout, view, and laser-print
circuit board designs. Because of the length of that application (nearly 2500
lines of C code), I'll focus my discussion here on the pseudocode that
implements the two algorithms mentioned and the source code that implements
the A* algorithm. The entire C source code that implements the printed circuit
board layout system is available on DDJ's Forum on CompuServe, DDJ's on-line
service, and on the disks mentioned at the end of this article.


What Is Autorouting?


Autorouting is one of a class of global optimization problems that are
difficult to solve. A good circuit board layout, for example, minimizes things
like:
Physical problems (trace lengths, board size, number of routing holes, holes
that transfer a trace from one side of the board to the other, also called
vias)
Signal crosstalk
Number of layers
At the same time, the layout maximizes things like signal strength,
reliability, and ease of debugging. The overall value of a board design is a
function of all of these often conflicting variables. It is usually acceptable
to find a solution that satisfies a set of constraints, because finding the
globally optimal solution is infeasible for all but the most trivial problems.
Autorouting can also be viewed as a collection of search problems. The
individual problems deal with finding a route and laying down a trace between
two locations. There are many algorithms for solving search problems, each
with different running time characteristics and data space requirements.
Autorouting search algorithms typically operate in two phases{1}, treating the
board as a matrix of cells. The first phase starts at the source cell and
searches for the target cell, usually by going in several directions at the
same time. The algorithm builds an auxiliary data structure to keep track of
how each cell was reached (this is referred to as "Pred" in the algorithms in
Figure 1 and Figure 2 ). The first phase ends when the target cell has been
found, and the second phase begins. If the first phase exhausts all
possibilities without reaching the target cell, then no route exists between
them, and there is no reason to do the second phase.
Figure 1: Pseudocode for the breadth-first algorithm

 BFS Algorithm (* breadth-first search *)
 (* Search a graph or state space, depending on the problem
 definition. *)
 (* S is the start node, T is the goal node. *)
 (* Open is an ordered list of nodes (ordered by arrival time;
 nodes enter at the tail and leave at the head), also called a
 queue. Closed is a set of nodes (order doesn't matter). In
 general, nodes that need to be searched are put on Open. As they
 are searched, they are removed from Open and put in Closed. *)
 (* Pred is defined for each node, and is a list of "came from"
 indications, so when we finally reach T, we traverse Pred to
 construct a path to S. *)

 1 Open <- {S} (* a list of one element *)
 Closed <- {} (* the empty set *)
 Pred [S] <- NULL, found <- FALSE
 WHILE Open <> {} and not found DO
 5 x <- the first node on Open
 Open <- Open - {x} (* remove x from Open *)
 Closed <- Closed + {x} (* put x in Closed *)
 IF x = T THEN found <- TRUE (* we're done *)
 ELSE (* continue search through node x *)
 10 let R be the set of neighboring nodes of x
 FOR each y in R DO
 IF y is not on Open or in Closed THEN

 Pred[y] <- x (* remember where we came from *)
 Open <- Open + {y} (* put y on Open (at the tail) *)
 15 IF found THEN
 use Pred[T] to find Pred[Pred[T]] and so on until S is reached
 (* this traces out the solution path in reverse *)
 ELSE T cannot be reached from S

The second phase uses the auxiliary data structure to trace the route from the
target cell back to the source cell, actually laying down the electrical
connections. The second phase is identical for the breadth-first and A* search
algorithms. But the first phase is different, and it is this phase that gives
these algorithms different behaviors.
The main data structures used in the first phase are the Open queue and the
Closed set, which hold cell coordinates. Because a cell's coordinates uniquely
identify it, we'll say that the Open queue and Closed set contain cells. Cell
coordinates will be represented as r2c3s1 for the cell at row 2, column 3,
side 1, or as r2c3 when it is understood that all cells are on the same side
of the board. To remind ourselves that Open is a queue and Closed is a set,
when we talk about adding cells to them, we will put the cells "on" the queue
and "in" the set. Initially, the Open queue contains the source cell and the
Closed set is empty.
The first phase is a loop, which removes an element from the head of the Open
queue, puts it in the Closed set (which indicates that it has been searched),
and checks to see if it is the target cell. If so, the first phase is done.
Otherwise, the neighbors of the cell (those adjacent to it) are placed on the
Open queue, and the loop continues. As we'll see later, the essential
difference in the breadth-first and A* search algorithms is the order in which
the neighbors are placed on the Open queue.


Breadth-First Search


Figure 1 contains pseudocode for the breadth-first search algorithm. This
algorithm works by processing a first in/first out (FIFO) queue of open cells;
that is, cells that have been reached, but not yet searched. Initially, the
open queue contains only the source cell. A cell is removed from the head of
the open queue, placed in the set of closed cells (cells that have been
searched), and checked to see if it is the target cell. If not, its neighbors
are placed at the tail of the open queue. Neighboring cells that have already
been reached are ignored. (If a cell's coordinates are on the open queue or in
the closed set, then it has been reached, otherwise, it has not.) This
continues until one of two things happens:
The target cell has been found
The open queue is empty, in which case the target cannot be reached from the
source cell
A version of breadth-first search known as Lee's algorithm{2} has served as
the basis for some autorouters since the early 1960s. The original algorithm
does not consider diagonally adjacent cells as neighbors, and consequently,
the back-tracking phase can create only horizontal and vertical traces. We'll
enhance the algorithm so that diagonally adjacent cells are neighbors, thus
enabling it to produce diagonal traces. Unfortunately, Lee's algorithm suffers
from a behavior inherent in the breadth-first search technique, which limits
its application to problems of relatively small size. As the distance between
the source and target cells increases by a factor of N, the number of cells
processed by Lee's algorithm -- and therefore processing time -- increases by
the square of N.
Figure 2 shows the behavior of Lee's algorithm while searching for a path
between the source cell S (r5c5) and the target cell T (r8c8). Lee's algorithm
does not specify the order in which neighboring cells are placed on the open
queue, but we'll use the compass directions north, east, south, and west,
followed by northeast, southeast, southwest, and northwest. This order tends
to produce traces with a minimal number of turns.
In Figure 2a, the source cell (r5c5) has been searched, and its eight
neighbors have been placed on the open queue. The arrows indicate the
direction from which each cell was reached, and correspond to the Pred data
structure. After the first eight cells on the open queue have been reached and
moved to the closed set, the algorithm searches the configuration in Figure
2b, where there are sixteen cells on the open queue. Once these sixteen cells
have been searched, the configuration in Figure 2c is reached. Now the target
cell (r8c8) is fourth from the end on the open queue, and a solution is
imminent. Searching r8c8, the algorithm recognizes it as the target cell, and
uses the Pred data structure to construct a trace back to the source cell.
You can see that the search progresses outward from the source cell in all
directions, like ripples when you throw a pebble into the water. If we double
the size of the problem so that S and T are six cells apart, the number of
cells searched and therefore the processing time will be about four times as
great. If we triple the size of the problem, the number of cells searched will
be roughly nine times more. Thus, the behavior of Lee's algorithm is quadratic
in the size of the problem, which makes it infeasible for large problems.


A* Search


Figure 3 gives pseudocode for the A* search algorithm, while Listing One shows
it implemented in C. This method also works by processing a queue of open
cells, which initially contains only the source cell. But this is a priority
queue, which means cells are inserted according to the estimated distance to
the target{3}, not just at the end. Cells that are on the shortest estimated
path from source to target go to the head of the queue. The A* algorithm
removes the cell from the head of the open queue and checks to see if it's the
target. If not, the neighboring cells are put on the open queue at the proper
position. The algorithm checks neighboring cells that have already been
searched to see if the new path between them and the source is shorter than
the previous one. If it is, they are repositioned on the open queue according
to the new estimated path length from source to target. As in breadth-first
search, this continues until the target cell has been found or the open queue
is empty.
Figure 3: Pseudocode for the A* algorithm

 A* Algorithm (* heuristic search *)
 (* Search a graph or state space, depending on the problem
 definition. *)
 (* S is the start node, T is the goal node. *)
 (* Open is an ordered list of nodes (ordered by lowest F value;
 see below), also called a priority queue. Closed is a set of
 nodes (order doesn't matter). In general, nodes that need to be
 searched are put on Open (at the proper position). As they are
 searched, they are removed from Open and put in Closed.
 Occasionally a newer, better route will be found to a node after
 it has already been searched, in which case we remove it from
 Closed and put it back on Open to be reconsidered. *)
 (* G[x] is the distance already traveled to get from S to node x,
 and is known exactly. H(x) is a function (heuristic) which
 returns an estimate of the distance from node x to T. F[x] is
 the estimated distance from S to T by going through node x, and
 is computed by F[x] = G[x] + H(x). H(x) can be calculated for
 any node, but F[x] and G[x] only become defined when node x is
 visited. *)
 (* Pred is defined for each node, and is a list of "came from"
 indications, so when we finally reach T, we traverse Pred to
 construct a path to S. *)
 (* Distance (x,y) is a function for calculating the distance
 between two neighboring nodes. *)
 1 Open <- {S} (* a list of one element *)
 Closed <- {} (* the empty set *)
 G[S] <- 0, F[S] <- 0, Pred[S] <- NULL, found <- FALSE
 WHILE Open <> {} and not found DO
 5 x <- the first node on Open (* node with smallest F value *)

 Open <- Open - {x} (* remove x from Open *)
 Closed <- Closed + {x} (* put x in Closed *)
 IF x = T THEN found <- TRUE (* we're done *)
 ELSE (* continue search through node x *)
 10 let R be the set of neighboring nodes of x
 FOR each y in R DO
 IF y is not on Open or in Closed THEN
 G[y] <- G[x] + Distance (x,y)
 F[y] <- G[y] + H(y) (* estimate solution path
 length *)
 15 Pred[y] <- x (* remember where we came from *)
 Open <- Open + {y} (* put y on Open *)
 ELSE (* y is on Open or in Closed *)
 IF (G[x] + Distance (x,y)) < G[y] THEN
 (* we've found a better route to y *)
 20 G[y] <- G[x] + Distance (x,y)
 F[y] <- G[y] + H(y)
 Pred[y] <- x (* remember where we came from *)
 IF y is on Open THEN
 reposition y according to F[y]
 25 ELSE (* y is in Closed *)
 Closed <- Closed - {y} (* remove y from
 Closed *)
 Open <- Open + {y} (* put y on Open *)
 IF found THEN
 use Pred[T] to find Pred[Pred[T]] and so on until S is reached
 30 (* this traces out the solution path in reverse *)
 ELSE T cannot be reached from S

A* depends on being able to estimate the distance between a cell and the
target cell. In the case of autorouting, a simple measure of this distance is
available, and this helps A* to concentrate the search in the direction most
likely to succeed. The more accurate the estimate, the faster the search.
In practice, A* does not suffer from the quadratic behavior of Lee's
algorithm, it solves similar problems faster and can be applied to larger
problems where Lee's algorithm performs poorly. As the distance between the
source and target cells increases, the number of cells processed by A*
increases, but not as dramatically as with Lee's algorithm.
Figure 4 shows the behavior of the A* search algorithm. A* does not specify
whether new cells go in front of or behind cells already on the open queue
that evaluate to identical estimated path lengths. We use the convention that
they are placed in front. This minimizes the time to insert a cell on the open
queue.
In Figure 4a, the source cell (r3c3) has been searched, and its eight
neighbors are on the open queue. Each cell on the open queue also includes the
estimated length of the shortest path from S to T that goes through that cell.
After the first cell (r4c4) has been searched and moved to the closed set, the
configuration in Figure 4b is reached, where there are 12 cells on the open
queue. After searching the next cell (r5c5), the algorithm reaches the
configuration in Figure 4c. Now the target cell (r6c6) is at the head of the
open queue, and a solution will be found on the next iteration of the loop.
Searching r6c6, A* recognizes it as the target and uses the Pred data
structure to construct a trace back to the source cell.
You can see that the search progresses more directly toward the target cell.
The target draws the search much as the earth's gravity pulls objects toward
the center of mass. If we double the size of the problem, the search will
process about twice as many cells, and if we triple its size, the search will
run through three times as many. This linear behavior makes A* more attractive
for autorouting than the quadratic Lee's algorithm. With the incorporation of
the heuristic -- the rule, that guides the search in the direction most likely
to succeed -- it is difficult to estimate worst case behavior. However, A*
will never take more time than Lee's algorithm, and it will never search any
cells that Lee's algorithm could avoid.


Optimizations and Generalizations


The algorithms in Figures 1 and 3 solve the general search problem. When these
algorithms are implemented and customized to a particular application, there
are ways to speed them up.
The A* algorithm in Figure 3 recomputes the heuristic H(y) when it discovers a
better way to reach a cell. Depending on how difficult this heuristic is to
compute, you can probably save some work at the expense of complicating the
algorithm. When lines 20 and 21 of Figure 3 are executed, the previous values
of G[y] and F[y] are destroyed. But F[y] = G[y] + H(y), so you could save F[y]
- G[y] (which is H(y)) in a temporary variable, and use that variable instead
of recomputing H(y) on line 21. Also, the common sub-expression G[x] +
Distance(x,y) should be placed in a temporary variable, instead of being
computed twice (lines 18 and 20).
Often, rather than searching for a path between two individual cells, what is
really desired is a path between one of a set of source cells and one of a set
of target cells (as when connecting power and ground pins). Both algorithms
can be modified by adding the entire set of source cells to the initial open
queue, and checking for a member of the set of target cells on each iteration.
When this is done, the heuristic used by the A* algorithm becomes more
complicated. It must estimate the minimum distance from the current cell to
any one of the target cells.
For breadth-first search, once the target cell is placed on the open queue, it
is pointless to add any more cells to the open queue because when this
happens, the problem is solved. An appropriate shortcut would be to insert a
check before line 13 in Figure 1 to see if y is the target cell. If it is, use
Pred[y] to construct the trace back to the source cell, and return.


Memory Requirements


Both search algorithms use quite a bit of memory to solve problems of
non-trivial size. The breadth-first search algorithm needs memory to represent
the board, the predecessor structure, and the closed set. The A* search
algorithm needs these also, plus structures for F[x] and G[x] (see Figure 3).
In addition, both algorithms dynamically allocate memory for the open cell
queue.
The board is represented as a pair of two-dimensional arrays -- one for the
front side, the other for the back -- in which the dimensions are the number
of rows and columns of cells. Not counting holes and traces relating to holes
(Figure 5, groups A, B, and C), there are 30 possible cell contents, which can
be represented with 5 bits per cell.
The hole-related cells are more difficult to enumerate; they can be combined
in many ways. If we simply assign 1 bit to each of the eight traces in groups
B and C, and add one more bit to indicate a hole, 14 bits will be sufficient
to represent any cell. On a board of N rows and M columns, we'll need N*M*28
bits total.
The predecessor structure is also a pair of two-dimensional arrays, where an
entry must be able to represent one of the eight compass directions or an
indication for the opposite side of the board. This takes 4 bits per cell, or
N*M*8 bits total.
The closed set can be represented by a pair of two-dimensional, single-bit
arrays, where a bit is one if the corresponding cell has been searched, and
zero otherwise. This will take N*M*2 bits total.
F[x] and G[x] will be similar to the board arrays, but they must contain a
16-bit integer for each cell, requiring N*M*64 bits total. Note that if memory
usage needs to be minimized at the cost of increased processing time, we could
omit the F[x] arrays, and calculate the F values as they are needed from the
G[x] arrays and the heuristic function, H(x).
Breadth-first search thus requires N*M*38 bits, and A* needs N*M*102 bits of
static memory. For a printed circuit board 4 x 13 inches (80 cells x 260
cells), breadth-first search will need 98,800 bytes and A* will need 265,200
bytes. Different algorithms that solve the same problem often trade off memory
against processing time to achieve better performance. This is the case with
A* versus the breadth-first search.</entry>
More Details.


Locality of Reference



Despite the fact that A* requires more memory than breadth-first search, A*
exhibits better memory usage patterns. This is because it shows better
locality of reference than breadth-first search. Locality of reference deals
with the sequence in which memory locations are used, and consists of two
rules of thumb: 1. The memory location currently being referenced is likely to
be referenced again in the near future, and 2. Memory locations near the one
currently being referenced are likely to be referenced in the near future.
When the first rule holds true for a given program, that program can probably
benefit from a memory cache. When the second rule holds true, the program can
probably benefit from a virtual memory environment with a least-recently-used
page preemption policy. Most computer systems with virtual memory and caches
apply them to both code and data, so programs that exhibit good locality of
reference should benefit from both rules.
This becomes a factor when solving large problems (say, routing a printed
circuit board that is 10 inches in both dimensions). In a virtual memory
environment, improved locality of reference can minimize swapping. In an
environment with cache memory, improved locality of reference can increase the
cache hit rate. Both of these tend to decrease the total running time.
The memory references in the breadth-first search algorithm go around and
around in circles of constantly increasing size, and do not reflect a common
locality of reference. Thus, the breadth-first search algorithm is not able to
take good advantage of virtual memory or a memory cache. The memory references
of A* tend to be from the same area of the printed circuit board for extended
periods of time, taking better advantage of these mechanisms. In a large
problem, this helps to offset the extra memory that A* requires by adding
speed beyond that provided by the basic algorithm. Improved locality of
reference by itself may not be a sufficient reason to select A* over
breadth-first search, but it is icing on the cake.


References


1. Stephen E. Belter, "Computer-aided Routing of Printed Circuit Boards: an
Examination of Lee's Algorithm and Possible Enhancements," BYTE, (June 1987),
199 - 208.
2. C.Y. Lee, "An Algorithm for Path Connections and Its Applications," IRE
Transactions on Electronic Computers," (September 1961), 346 - 365.
3. Steven L. Tanimoto, The Elements of Artificial Intelligence, (1987,
Rockville, Maryland: Computer Science Press), 148 - 164. This covers the
breadth-first and A* search algorithms.


Distance Calculations


The A* search algorithm uses a heuristic to estimate the distance between the
current cell and the target cell. As implemented in the autorouting program,
the heuristic is a simple geometric distance approximation. Figure 5
illustrates all the possible cell types used to construct a trace, grouped by
type. For each group, the distance of that cell type is also given. These
distances are calculated based on a cell size of 50 x 50 mils. (A mil is
1/1000 inch, so the autorouter uses 20 cells/inch. A typical full-length
adapter board for an IBM PC is 4-inches high and 13-inches long, or 80-cell
rows x 260-cell columns.)
The group B and C traces can coexist in the same cell, so a hole can have up
to 16 traces connecting it with other cells (eight on each side of the board).
Also, the parallel traces of group F can coexist in the same cell (on the same
side of the board), as shown by group J. This allows the routing of two traces
through the same cell, providing the higher density required by some circuits
(memory arrays, for example). Aside from these exceptions, cells can only
contain one trace type (on each side of the board).
To determine the approximate distance of a trace that will connect two holes,
view the board as a matrix, the differences in cell coordinates are three rows
and five columns. The shortest path between them will use a diagonal trace
across three cells and a horizontal trace across two cells. Using the cell
types in Figure 5, the length of the trace will be 23 + (2 * 71) + 60 + 50 +
12 = 287 mils. A trace that uses a routing hole to go from one side of the
board to the other covers a greater distance than one that goes diagonally
across a cell (group E in Figure 5) and stays on the same side of the board
because part of its path goes around the edge of a circle.
A typical hole is 25 mils in diameter, and is at the center of a cell. To
calculate the distance of a trace through a routing hole, measure the section
of the hole between the two connecting traces. For instance, an entering trace
can connect to a hole at a point A, and possible exiting traces on the
opposite side of the board at points B, C, D, and E. The distances between A
and each of these points are 10, 20, 29, and 39 mils, respectively. To
calculate these, use the geometric formula Circumference = PI * Diameter
(approximately 78.5 mils) and divide by 8 (a 1/8 section of a hole is
approximately 9.8 mils), add 1, 2, 3, and 4 of these sections, then round off
to an integer.
The heuristic in the autorouting program includes a penalty when a trace takes
a turn or switches to the other side of the board through a routing hole
because turns are often the weak points in a circuit, and traces are more
likely to break at a turn than in a straight part. The penalty encourages A*
to use straight lines, and even allows a slightly longer trace to be selected
over one with too many turns. The amount of penalty depends on the kind of
turn; sharper turns are assessed a larger penalty. Routing holes incur a
significant penalty, since overusing them early in the routing process can
make later traces more difficult or even impossible to construct because a
routing hole dedicates a cell exclusively to a single trace, for both sides of
the board. Such a cell is not available to later routing, thus reducing the
total number of cells that can be used. -- R.N.




_AUTOROUTING WITH THE A* ALGORITHM_
by Randy Nevin


* PC BOARD LAYOUT SYSTEM TO ACCOMPANY _AUTOROUTING WITH THE A* ALGORITHM_ BY
RANDY NEVIN IS
ATTACHED TO THE END OF THE FOLLOWING LISTINGS



[LISTING ONE]

/*
** printed circuit board autorouter, Copyright (C) Randy Nevin 1989.
** you may give this software to anyone, make as many copies as you
** like, and post it on public computer bulletin boards and file
** servers. you may not sell it or charge any fee for distribution
** (except for media and postage), remove this comment or the
** copyright notice from the code, or claim that you wrote this code
** or anything derived from it.
** the author's address is: Randy Nevin, 1731 211th PL NE, Redmond,
** WA 98053. this code is available directly from the author; just
** send a floppy and a self-addressed floppy mailer with sufficient
** postage. however, you should first attempt to get a copy of this
** software package from the Dr. Dobb's Journal BBS.
*/
/* the low-order bit indicates a hole */
#define HOLE 0x00000001L /* a conducting hole */
/* traces radiating outward from a hole to a side or corner */
#define HOLE_NORTH 0x00000002L /* upward */
#define HOLE_NORTHEAST 0x00000004L /* upward and right */
#define HOLE_EAST 0x00000008L /* to the right */

#define HOLE_SOUTHEAST 0x00000010L /* downward and right */
#define HOLE_SOUTH 0x00000020L /* downward */
#define HOLE_SOUTHWEST 0x00000040L /* downward and left */
#define HOLE_WEST 0x00000080L /* to the left */
#define HOLE_NORTHWEST 0x00000100L /* upward and left */

/* straight lines through the center */
#define LINE_HORIZONTAL 0x00000002L /* left-to-right line */
#define LINE_VERTICAL 0x00000004L /* top-to-bottom line */

/* lines cutting across a corner, connecting adjacent sides */
#define CORNER_NORTHEAST 0x00000008L /* upper right corner */
#define CORNER_SOUTHEAST 0x00000010L /* lower right corner */
#define CORNER_SOUTHWEST 0x00000020L /* lower left corner */
#define CORNER_NORTHWEST 0x00000040L /* upper left corner */

/* diagonal lines through the center */
#define DIAG_NEtoSW 0x00000080L /* northeast to southwest */
#define DIAG_SEtoNW 0x00000100L /* southeast to northwest */

/* 135 degree angle side-to-far-corner lines */
#define BENT_NtoSE 0x00000200L /* north to southeast */
#define BENT_NtoSW 0x00000400L /* north to southwest */
#define BENT_EtoSW 0x00000800L /* east to southwest */
#define BENT_EtoNW 0x00001000L /* east to northwest */
#define BENT_StoNW 0x00002000L /* south to northwest */
#define BENT_StoNE 0x00004000L /* south to northeast */
#define BENT_WtoNE 0x00008000L /* west to northeast */
#define BENT_WtoSE 0x00010000L /* west to southeast */

/* 90 degree corner-to-adjacent-corner lines */
#define ANGLE_NEtoSE 0x00020000L /* northeast to southeast */
#define ANGLE_SEtoSW 0x00040000L /* southeast to southwest */
#define ANGLE_SWtoNW 0x00080000L /* southwest to northwest */
#define ANGLE_NWtoNE 0x00100000L /* northwest to northeast */

/* 45 degree angle side-to-near-corner lines */
#define SHARP_NtoNE 0x00200000L /* north to northeast */
#define SHARP_EtoNE 0x00400000L /* east to northeast */
#define SHARP_EtoSE 0x00800000L /* east to southeast */
#define SHARP_StoSE 0x01000000L /* south to southeast */
#define SHARP_StoSW 0x02000000L /* south to southwest */
#define SHARP_WtoSW 0x04000000L /* west to southwest */
#define SHARP_WtoNW 0x08000000L /* west to northwest */
#define SHARP_NtoNW 0x10000000L /* north to northwest */

/* directions the cell can be reached from (point to previous cell) */
#define FROM_NORTH 1
#define FROM_NORTHEAST 2
#define FROM_EAST 3
#define FROM_SOUTHEAST 4
#define FROM_SOUTH 5
#define FROM_SOUTHWEST 6
#define FROM_WEST 7
#define FROM_NORTHWEST 8
#define FROM_OTHERSIDE 9

#define TOP 0
#define BOTTOM 1

#define EMPTY 0
#define ILLEGAL -1

/* visit neighboring cells in this order
** (where [9] is on the other side):
** +---+---+---+
** 1 2 3 
** +---+---+---+
** 4 [9] 5 
** +---+---+---+
** 6 7 8 
** +---+---+---+
*/

static int delta[8][2] = { /* for visiting neighbors on the same side */
 { 1, -1 }, /* northwest */
 { 1, 0 }, /* north */
 { 1, 1 }, /* northeast */
 { 0, -1 }, /* west */
 { 0, 1 }, /* east */
 { -1, -1 }, /* southwest */
 { -1, 0 }, /* south */
 { -1, 1 } /* southeast */
 };

static int ndir[8] = { /* for building paths back to source */
 FROM_SOUTHEAST, FROM_SOUTH, FROM_SOUTHWEST,
 FROM_EAST, FROM_WEST,
 FROM_NORTHEAST, FROM_NORTH, FROM_NORTHWEST
 };

/* blocking masks for neighboring cells */
#define BLOCK_NORTHEAST ( DIAG_NEtoSW BENT_StoNE BENT_WtoNE \
 ANGLE_NEtoSE ANGLE_NWtoNE \
 SHARP_NtoNE SHARP_EtoNE )
#define BLOCK_SOUTHEAST ( DIAG_SEtoNW BENT_NtoSE BENT_WtoSE \
 ANGLE_NEtoSE ANGLE_SEtoSW \
 SHARP_EtoSE SHARP_StoSE )
#define BLOCK_SOUTHWEST ( DIAG_NEtoSW BENT_NtoSW BENT_EtoSW \
 ANGLE_SEtoSW ANGLE_SWtoNW \
 SHARP_StoSW SHARP_WtoSW )
#define BLOCK_NORTHWEST ( DIAG_SEtoNW BENT_EtoNW BENT_StoNW \
 ANGLE_SWtoNW ANGLE_NWtoNE \
 SHARP_WtoNW SHARP_NtoNW )
struct block {
 int r1, c1; long b1, h1;
 int r2, c2; long b2, h2;
 };
static struct block blocking[8] = { /* blocking masks */
 { 0, -1, BLOCK_NORTHEAST, HOLE_NORTHEAST,
 1, 0, BLOCK_SOUTHWEST, HOLE_SOUTHWEST },
 { 0, 0, 0, 0, 0, 0, 0, 0 },
 { 1, 0, BLOCK_SOUTHEAST, HOLE_SOUTHEAST,
 0, 1, BLOCK_NORTHWEST, HOLE_NORTHWEST },
 { 0, 0, 0, 0, 0, 0, 0, 0 },
 { 0, 0, 0, 0, 0, 0, 0, 0 },
 { 0, -1, BLOCK_SOUTHEAST, HOLE_SOUTHEAST,
 -1, 0, BLOCK_NORTHWEST, HOLE_NORTHWEST },
 { 0, 0, 0, 0, 0, 0, 0, 0 },

 { -1, 0, BLOCK_NORTHEAST, HOLE_NORTHEAST,
 0, 1, BLOCK_SOUTHWEST, HOLE_SOUTHWEST }
 };
static int selfok[5][8] = { /* mask for self-blocking corner effects */
 { 1, 1, 1, 1, 1, 1, 1, 1 },
 { 0, 0, 0, 0, 1, 0, 1, 0 },
 { 0, 0, 0, 1, 0, 0, 1, 0 },
 { 0, 1, 0, 0, 1, 0, 0, 0 },
 { 0, 1, 0, 1, 0, 0, 0, 0 }
 };
static long newmask[5][8] = { /* patterns to mask in neighbor cells */
 { 0, 0, 0, 0, 0, 0, 0, 0 },
 { 0, 0, 0, 0, CORNER_NORTHEAST CORNER_SOUTHEAST, 0,
 CORNER_SOUTHEAST CORNER_SOUTHWEST, 0 },
 { 0, 0, 0, CORNER_SOUTHWEST CORNER_NORTHWEST, 0, 0,
 CORNER_SOUTHEAST CORNER_SOUTHWEST, 0 },
 { 0, CORNER_NORTHEAST CORNER_NORTHWEST, 0, 0,
 CORNER_NORTHEAST CORNER_SOUTHEAST, 0, 0, 0 },
 { 0, CORNER_NORTHEAST CORNER_NORTHWEST, 0,
 CORNER_SOUTHWEST CORNER_NORTHWEST, 0, 0, 0, 0 }
 };
/* board dimensions */
extern int Nrows;
extern int Ncols;

void Solve () { /* route all traces */
 int r1, c1, r2, c2, r, c, s, d, a, nr, nc, skip;
 register int i;
 char far *n1;
 char far *n2;
 long curcell, newcell, buddy;
 int newdist, olddir, success, self;

 /* go until no more work to do */
 for (GetWork( &r1, &c1, &n1, &r2, &c2, &n2 ); r1 != ILLEGAL;
 GetWork( &r1, &c1, &n1, &r2, &c2, &n2 )) {
 if (r1 == r2 && c1 == c2) /* already routed */
 continue;
 success = 0;
 InitQueue(); /* initialize the search queue */
 /* get rough estimate of trace distance */
 a = GetApxDist( r1, c1, r2, c2 );
 SetQueue( r1, c1, TOP, 0, a, r2, c2 );
 SetQueue( r1, c1, BOTTOM, 0, a, r2, c2 );
 /* search until success or we exhaust all possibilities */
 for (GetQueue( &r, &c, &s, &d, &a ); r != ILLEGAL;
 GetQueue( &r, &c, &s, &d, &a )) {
 if (r == r2 && c == c2) { /* success! */
 /* lay traces */
 Retrace( r1, c1, r2, c2, s );
 success++;
 break;
 }
 curcell = GetCell( r, c, s );
 if (curcell & CORNER_NORTHWEST)
 self = 1;
 else if (curcell & CORNER_NORTHEAST)
 self = 2;
 else if (curcell & CORNER_SOUTHWEST)

 self = 3;
 else if (curcell & CORNER_SOUTHEAST)
 self = 4;
 else
 self = 0;
 /* consider neighbors */
 for (i = 0; i < 8; i++) {
 /* check self-block */
 if (!selfok[self][i])
 continue;
 if ((nr = r+delta[i][0]) < 0
 nr >= Nrows
 (nc = c+delta[i][1]) < 0
 nc >= Ncols)
 /* off the edge */
 continue;
 newcell = GetCell( nr, nc, s );
 /* check for non-target hole */
 if (newcell & HOLE) {
 if (nr != r2 nc != c2)
 continue;
 }
 else {
 newcell &= ~(newmask[self][i]);
 /* check for traces */
 if (newcell)
 continue;
 }
 /* check blocking on corner neighbors */
 if (delta[i][0] && delta[i][1]) {
 /* check first buddy */
 buddy = GetCell( r+blocking[i].r1,
 c+blocking[i].c1, s );
 if (buddy & HOLE) {
 if (buddy & (blocking[i].h1))
 continue;
 }
 else if (buddy & (blocking[i].b1))
 continue;
 /* check second buddy */
 buddy = GetCell( r+blocking[i].r2,
 c+blocking[i].c2, s );
 if (buddy & HOLE) {
 if (buddy & (blocking[i].h2))
 continue;
 }
 else if (buddy & (blocking[i].b2))
 continue;
 }
 olddir = GetDir( r, c, s );
 newdist = d+CalcDist( ndir[i], olddir,
 (olddir == FROM_OTHERSIDE)
 ? GetDir( r, c, 1-s ) : 0 );
 /* if not visited yet, add it to queue */
 if (!GetDir( nr, nc, s )) {
 SetDir( nr, nc, s, ndir[i] );
 SetDist( nr, nc, s, newdist );
 SetQueue( nr, nc, s, newdist,
 GetApxDist( nr, nc, r2, c2 ),

 r2, c2 );
 }
 /* we might have found a better path */
 else if (newdist < GetDist( nr, nc, s )) {
 SetDir( nr, nc, s, ndir[i] );
 SetDist( nr, nc, s, newdist );
 ReSetQueue( nr, nc, s, newdist,
 GetApxDist( nr, nc, r2, c2 ),
 r2, c2 );
 }
 }
 /* consider other side of board */
 /* check for holes or traces on other side */
 if (newcell = GetCell( r, c, 1-s ))
 continue;
 skip = 0;
 /* check for nearby holes */
 for (i = 0; i < 8; i++) {
 if ((nr = r+delta[i][0]) < 0
 nr >= Nrows
 (nc = c+delta[i][1]) < 0
 nc >= Ncols)
 /* off the edge */
 continue;
 if (GetCell( nr, nc, s ) & HOLE) {
 /* neighboring hole */
 skip = 1;
 break;
 }
 }
 if (skip) /* neighboring hole? */
 continue; /* yes, can't drill one here */
 olddir = GetDir( r, c, s );
 newdist = d+CalcDist( FROM_OTHERSIDE, olddir,
 (olddir == FROM_OTHERSIDE)
 ? GetDir( r, c, 1-s ) : 0 );
 /* if not visited yet, add it to queue */
 if (!GetDir( r, c, 1-s )) {
 SetDir( r, c, 1-s, FROM_OTHERSIDE );
 SetDist( r, c, 1-s, newdist );
 SetQueue( r, c, 1-s, newdist,
 a, r2, c2 );
 }
 /* we might have found a better path */
 else if (newdist < GetDist( r, c, 1-s )) {
 SetDir( r, c, 1-s, FROM_OTHERSIDE );
 SetDist( r, c, 1-s, newdist );
 ReSetQueue( r, c, 1-s, newdist,
 a, r2, c2 );
 }
 }
 if (!success)
 printf( "\t*!* UNSUCCESSFUL *!*\n" );
 /* clear direction flags */
 for (r = 0; r < Nrows; r++) {
 for (c = 0; c < Ncols; c++) {
 SetDir( r, c, TOP, EMPTY );
 SetDir( r, c, BOTTOM, EMPTY );
 }

 }
 }
 }
/* this table drives the retracing phase */
static long bit[8][9] = { /* OT=Otherside */
 /* N, NE, E, SE, S, SW, W, NW, OT */
/* N */ { LINE_VERTICAL, BENT_StoNE, CORNER_SOUTHEAST, SHARP_StoSE,
 0, SHARP_StoSW, CORNER_SOUTHWEST, BENT_StoNW,
 (HOLE HOLE_SOUTH) },
/* NE */ { BENT_NtoSW, DIAG_NEtoSW, BENT_EtoSW, ANGLE_SEtoSW,
 SHARP_StoSW, 0, SHARP_WtoSW, ANGLE_SWtoNW,
 (HOLE HOLE_SOUTHWEST) },
/* E */ { CORNER_NORTHWEST, BENT_WtoNE, LINE_HORIZONTAL, BENT_WtoSE,
 CORNER_SOUTHWEST, SHARP_WtoSW, 0, SHARP_WtoNW,
 (HOLE HOLE_WEST) },
/* SE */ { SHARP_NtoNW, ANGLE_NWtoNE, BENT_EtoNW, DIAG_SEtoNW,
 BENT_StoNW, ANGLE_SWtoNW, SHARP_WtoNW, 0,
 (HOLE HOLE_NORTHWEST) },
/* S */ { 0, SHARP_NtoNE, CORNER_NORTHEAST, BENT_NtoSE,
 LINE_VERTICAL, BENT_NtoSW, CORNER_NORTHWEST, SHARP_NtoNW,
 (HOLE HOLE_NORTH) },
/* SW */ { SHARP_NtoNE, 0, SHARP_EtoNE, ANGLE_NEtoSE, BENT_StoNE,
 DIAG_NEtoSW, BENT_WtoNE, ANGLE_NWtoNE,
 (HOLE HOLE_NORTHEAST) },
/* W */ { CORNER_NORTHEAST, SHARP_EtoNE, 0, SHARP_EtoSE,
 CORNER_SOUTHEAST, BENT_EtoSW, LINE_HORIZONTAL, BENT_EtoNW,
 (HOLE HOLE_EAST) },
/* NW */ { BENT_NtoSE, ANGLE_NEtoSE, SHARP_EtoSE, 0, SHARP_StoSE,
 ANGLE_SEtoSW, BENT_WtoSE, DIAG_SEtoNW,
 (HOLE HOLE_SOUTHEAST) }
 };

void Retrace ( rr1, cc1, rr2, cc2, s )
 /* work from target back to source, laying down the trace */
 int rr1, cc1, rr2, cc2, s; /* start on side s */
 {
 int r0, c0, s0, r1, c1, s1, r2, c2, s2;
 register int x, y;
 long b;

 r1 = rr2;
 c1 = cc2;
 s1 = s;
 r0 = c0 = s0 = ILLEGAL;
 do {
 /* find where we came from to get here */
 switch (x = GetDir( r2 = r1, c2 = c1, s2 = s1 )) {
 case FROM_NORTH: r2++; break;
 case FROM_EAST: c2++; break;
 case FROM_SOUTH: r2--; break;
 case FROM_WEST: c2--; break;
 case FROM_NORTHEAST: r2++; c2++; break;
 case FROM_SOUTHEAST: r2--; c2++; break;
 case FROM_SOUTHWEST: r2--; c2--; break;
 case FROM_NORTHWEST: r2++; c2--; break;
 case FROM_OTHERSIDE: s2 = 1-s2; break;
 default:
 fprintf( stderr, "internal error\n" );
 exit( -1 );

 break;
 }
 if (r0 != ILLEGAL)
 y = GetDir( r0, c0, s0 );
 /* see if target or hole */
 if ((r1 == rr2 && c1 == cc2) (s1 != s0)) {
 switch (x) {
 case FROM_NORTH:
 OrCell( r1, c1, s1, HOLE_NORTH ); break;
 case FROM_EAST:
 OrCell( r1, c1, s1, HOLE_EAST ); break;
 case FROM_SOUTH:
 OrCell( r1, c1, s1, HOLE_SOUTH ); break;
 case FROM_WEST:
 OrCell( r1, c1, s1, HOLE_WEST ); break;
 case FROM_NORTHEAST:
 OrCell( r1, c1, s1, HOLE_NORTHEAST ); break;
 case FROM_SOUTHEAST:
 OrCell( r1, c1, s1, HOLE_SOUTHEAST ); break;
 case FROM_SOUTHWEST:
 OrCell( r1, c1, s1, HOLE_SOUTHWEST ); break;
 case FROM_NORTHWEST:
 OrCell( r1, c1, s1, HOLE_NORTHWEST ); break;
 case FROM_OTHERSIDE:
 default:
 fprintf( stderr, "internal error\n" );
 exit( -1 );
 break;
 }
 }
 else {
 if ((y == FROM_NORTH
 y == FROM_NORTHEAST
 y == FROM_EAST
 y == FROM_SOUTHEAST
 y == FROM_SOUTH
 y == FROM_SOUTHWEST
 y == FROM_WEST
 y == FROM_NORTHWEST)
 && (x == FROM_NORTH
 x == FROM_NORTHEAST
 x == FROM_EAST
 x == FROM_SOUTHEAST
 x == FROM_SOUTH
 x == FROM_SOUTHWEST
 x == FROM_WEST
 x == FROM_NORTHWEST
 x == FROM_OTHERSIDE)
 && (b = bit[y-1][x-1])) {
 OrCell( r1, c1, s1, b );
 if (b & HOLE)
 OrCell( r2, c2, s2, HOLE );
 }
 else {
 fprintf( stderr, "internal error\n" );
 exit( -1 );
 }
 }
 if (r2 == rr1 && c2 == cc1) { /* see if source */

 switch (x) {
 case FROM_NORTH:
 OrCell( r2, c2, s2, HOLE_SOUTH ); break;
 case FROM_EAST:
 OrCell( r2, c2, s2, HOLE_WEST ); break;
 case FROM_SOUTH:
 OrCell( r2, c2, s2, HOLE_NORTH ); break;
 case FROM_WEST:
 OrCell( r2, c2, s2, HOLE_EAST ); break;
 case FROM_NORTHEAST:
 OrCell( r2, c2, s2, HOLE_SOUTHWEST ); break;
 case FROM_SOUTHEAST:
 OrCell( r2, c2, s2, HOLE_NORTHWEST ); break;
 case FROM_SOUTHWEST:
 OrCell( r2, c2, s2, HOLE_NORTHEAST ); break;
 case FROM_NORTHWEST:
 OrCell( r2, c2, s2, HOLE_SOUTHEAST ); break;
 case FROM_OTHERSIDE:
 default:
 fprintf( stderr, "internal error\n" );
 exit( -1 );
 break;
 }
 }
 /* move to next cell */
 r0 = r1; c0 = c1; s0 = s1;
 r1 = r2; c1 = c2; s1 = s2;
 } while (!(r2 == rr1 && c2 == cc1));
 }

int GetApxDist ( r1, c1, r2, c2 ) /* calculate approximate distance */
 int r1, c1, r2, c2;
 {
 register int d1, d2; /* row and column deltas */
 int d0; /* temporary variable for swapping d1 and d2 */
 /* NOTE: the -25 used below is because we are not going */
 /* from the center of (r1,c1) to the center of (r2,c2), */
 /* we are going from the edge of a hole at (r1,c1) to */
 /* the edge of a hole at (r2,c2). holes are 25 mils in */
 /* diameter (12.5 mils in radius), so we back off by 2 */
 /* radii. */
 if ((d1 = r1-r2) < 0) /* get absolute row delta */
 d1 = -d1;
 if ((d2 = c1-c2) < 0) /* get absolute column delta */
 d2 = -d2;
 if (!d1) /* in same row? */
 return( (d2*50)-25 ); /* 50 mils per cell */
 if (!d2) /* in same column? */
 return( (d1*50)-25 ); /* 50 mils per cell */
 if (d1 > d2) { /* get smaller into d1 */
 d0 = d1;
 d1 = d2;
 d2 = d0;
 }
 d2 -= d1; /* get non-diagonal part of approximate route */
 return( (d1*71)+(d2*50)-25 ); /* 71 mils diagonally per cell */
 }
/* distance to go through a cell */
static int dist[10][10] = { /* OT=Otherside, OR=Origin (source) cell */

 /* N, NE, E, SE, S, SW, W, NW, OT, OR */
/* N */ { 50, 60, 35, 60, 99, 60, 35, 60, 12, 12 },
/* NE */ { 60, 71, 60, 71, 60, 99, 60, 71, 23, 23 },
/* E */ { 35, 60, 50, 60, 35, 60, 99, 60, 12, 12 },
/* SE */ { 60, 71, 60, 71, 60, 71, 60, 99, 23, 23 },
/* S */ { 99, 60, 35, 60, 50, 60, 35, 60, 12, 12 },
/* SW */ { 60, 99, 60, 71, 60, 71, 60, 71, 23, 23 },
/* W */ { 35, 60, 99, 60, 35, 60, 50, 60, 12, 12 },
/* NW */ { 60, 71, 60, 99, 60, 71, 60, 71, 23, 23 },

/* OT */ { 12, 23, 12, 23, 12, 23, 12, 23, 99, 99 },
/* OR */ { 99, 99, 99, 99, 99, 99, 99, 99, 99, 99 }
 };

/* distance around (circular) segment of hole */
static int circ[10][10] = { /* OT=Otherside, OR=Origin (source) cell */
 /* N, NE, E, SE, S, SW, W, NW, OT, OR */
/* N */ { 39, 29, 20, 10, 0, 10, 20, 29, 99, 0 },
/* NE */ { 29, 39, 29, 20, 10, 0, 10, 20, 99, 0 },
/* E */ { 20, 29, 39, 29, 20, 10, 0, 10, 99, 0 },
/* SE */ { 10, 20, 29, 39, 29, 20, 10, 0, 99, 0 },
/* S */ { 0, 10, 20, 29, 39, 29, 20, 10, 99, 0 },
/* SW */ { 10, 0, 10, 20, 29, 39, 29, 20, 99, 0 },
/* W */ { 20, 10, 0, 10, 20, 29, 39, 29, 99, 0 },
/* NW */ { 29, 20, 10, 0, 10, 20, 29, 39, 99, 0 },

/* OT */ { 99, 99, 99, 99, 99, 99, 99, 99, 99, 0 },
/* OR */ { 99, 99, 99, 99, 99, 99, 99, 99, 99, 0 }
 };

/* penalty for routing holes and turns, scaled by sharpness of turn */
static int penalty[10][10] = { /* OT=Otherside, OR=Origin (source) cell */
 /* N, NE, E, SE, S, SW, W, NW, OT, OR */
/* N */ { 0, 5, 10, 15, 20, 15, 10, 5, 50, 0 },
/* NE */ { 5, 0, 5, 10, 15, 20, 15, 10, 50, 0 },
/* E */ { 10, 5, 0, 5, 10, 15, 20, 15, 50, 0 },
/* SE */ { 15, 10, 5, 0, 5, 10, 15, 20, 50, 0 },
/* S */ { 20, 15, 10, 5, 0, 5, 10, 15, 50, 0 },
/* SW */ { 15, 20, 15, 10, 5, 0, 5, 10, 50, 0 },
/* W */ { 10, 15, 20, 15, 10, 5, 0, 5, 50, 0 },
/* NW */ { 5, 10, 15, 20, 15, 10, 5, 0, 50, 0 },

/* OT */ { 50, 50, 50, 50, 50, 50, 50, 50, 100, 0 },
/* OR */ { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 }
 };

/*
** x is the direction to enter the cell of interest.
** y is the direction to exit the cell of interest.
** z is the direction to really exit the cell, if y=FROM_OTHERSIDE.
** return the distance of the trace through the cell of interest.
** the calculation is driven by the tables above.
*/

int CalcDist ( x, y, z )
 /* calculate distance of a trace through a cell */
 int x, y, z;
 {
 int adjust;


 adjust = 0; /* set if hole is encountered */
 if (x == EMPTY)
 x = 10;
 if (y == EMPTY)
 y = 10;
 else if (y == FROM_OTHERSIDE) {
 if (z == EMPTY)
 z = 10;
 adjust = circ[x-1][z-1] + penalty[x-1][z-1];
 }
 return( dist[x-1][y-1] + penalty[x-1][y-1] + adjust );
 }



Figure 1: Pseudo-code for the breadth-first search algorith

BFS Algorithm (* breadth-first search *)
 (* Search a graph or state space, depending on the problem definition. *)
 (* S is the start node, T is the goal node. *)
 (* Open is an ordered list of nodes (ordered by arrival time; nodes enter
 at the tail and leave at the head), also called a queue. Closed is a set
 of nodes (order doesn't matter). In general, nodes that need to be
 searched are put on Open. As they are searched, they are removed from
 Open and put in Closed. *)
 (* Pred is defined for each node, and is a list of "came from" indications,
 so when we finally reach T, we traverse Pred to construct a path to S. *)
1 Open <- {S} (* a list of one element *)
 Closed <- {} (* the empty set *)
 Pred[S] <- NULL, found <- FALSE
 WHILE Open <> {} and not found DO
5 x <- the first node on Open
 Open <- Open - {x} (* remove x from Open *)
 Closed <- Closed + {x} (* put x in Closed *)
 IF x = T THEN found <- TRUE (* we're done *)
 ELSE (* continue search through node x *)
10 let R be the set of neighboring nodes of x
 FOR each y in R DO
 IF y is not on Open or in Closed THEN
 Pred[y] <- x (* remember where we came from *)
 Open <- Open + {y} (* put y on Open (at the tail) *)
15 IF found THEN
 use Pred[T] to find Pred[Pred[T]] and so on until S is reached
 (* this traces out the solution path in reverse *)
 ELSE T cannot be reached from S



Figure 3: Pseudo-code for the A* algorithm

A* Algorithm (* heuristic search *)
 (* Search a graph or state space, depending on the problem definition. *)
 (* S is the start node, T is the goal node. *)
 (* Open is an ordered list of nodes (ordered by lowest F value; see below),
 also called a priority queue. Closed is a set of nodes (order doesn't
 matter). In general, nodes that need to be searched are put on Open (at
 the proper position). As they are searched, they are removed from Open
 and put in Closed. Occasionally a newer, better route will be found to a

 node after it has already been searched, in which case we remove it from
 Closed and put it back on Open to be reconsidered. *)
 (* G[x] is the distance already traveled to get from S to node x, and is
 known exactly. H(x) is a function (heuristic) which returns an estimate
 of the distance from node x to T. F[x] is the estimated distance from S
 to T by going through node x, and is computed by F[x] = G[x] + H(x).
 H(x) can be calculated for any node, but F[x] and G[x] only become
 defined when node x is visited. *)
 (* Pred is defined for each node, and is a list of "came from" indications,
 so when we finally reach T, we traverse Pred to construct a path to
 S. *)
 (* Distance(x,y) is a function for calculating the distance between two
 neighboring nodes. *)
1 Open <- {S} (* a list of one element *)
 Closed <- {} (* the empty set *)
 G[S] <- 0, F[S] <- 0, Pred[S] <- NULL, found <- FALSE
 WHILE Open <> {} and not found DO
5 x <- the first node on Open (* node with smallest F value *)
 Open <- Open - {x} (* remove x from Open *)
 Closed <- Closed + {x} (* put x in Closed *)
 IF x = T THEN found <- TRUE (* we're done *)
 ELSE (* continue search through node x *)
10 let R be the set of neighboring nodes of x
 FOR each y in R DO
 IF y is not on Open or in Closed THEN
 G[y] <- G[x] + Distance(x,y)
 F[y] <- G[y] + H(y) (* estimate solution path length *)
15 Pred[y] <- x (* remember where we came from *)
 Open <- Open + {y} (* put y on Open *)
 ELSE (* y is on Open or in Closed *)
 IF (G[x] + Distance(x,y)) < G[y] THEN
 (* we've found a better route to y *)
20 G[y] <- G[x] + Distance(x,y)
 F[y] <- G[y] + H(y)
 Pred[y] <- x (* remember where we came from *)
 IF y is on Open THEN
 reposition y according to F[y]
25 ELSE (* y is in Closed *)
 Closed <- Closed - {y} (* remove y from Closed *)
 Open <- Open + {y} (* put y on Open *)
 IF found THEN
 use Pred[T] to find Pred[Pred[T]] and so on until S is reached
30 (* this traces out the solution path in reverse *)
 ELSE T cannot be reached from S


















September, 1989
SIMULATED ANNEALING


This algorithm may be one of the best solutions to the problem of
combinatorial optimization




Michael P. McLaughlin


Dr. McLaughlin is a member of the technical staff, Air Transportation Systems,
for Mitre Corp. He can be reached at 1740 Westwind Way, McLean, VA 22102.


In science and engineering, it is a truism that, in any real system,
everything is connected. Indeed, you could do far worse than to define science
as the search for such connections. Scientists accorded the highest esteem,
those names nearly everyone knows, are generally individuals whose insight
allowed them either to forge links between a large number of separate concepts
or to find a connection that no one had suspected.
Occasionally, something similar happens on a smaller scale. This article
concerns the application of statistical mechanics to the solution of a
difficult problem in computer science. The problem is combinatorial
optimization; the technique is called "simulated annealing."
Combinatorial optimization is the task of taking a finite number of
indivisible "objects" and arranging them in a configuration that is "best"
according to some stipulated criteria. For instance, how should 15 swimmers be
selected from a team of 30, for an upcoming meet, if no one is allowed to
enter more than one event? Or how should aircraft, preparing to land, be
sequenced in order to minimize delays? What is the best routing for a
telephone call from Boston to San Francisco at any given moment? What is the
best parse for a given English-language sentence? What these and similar
problems have in common are 1. an extremely large number of discrete choices,
2. a complicated set of optimizing criteria and 3. numerous constraints. In
such circumstances, simulated annealing has proven to be an excellent tool for
seeking out the best solution.
Simulated annealing is a relatively new form of stochastic search, familiar
primarily to VLSI chip designers. As you can imagine, determining the best
geometrical arrangement for thousands of circuits is a formidable task. To
take an oversimplified example, suppose you have to partition 1000 circuits
into five regions on a microchip. Further suppose that permutations within a
region do not matter and that every chip configuration can be given a
numerical score ased upon an appropriate set of (possibly conflicting)
criteria. Prior to assessing constraints on placement, you have five choices
for each circuit for a total of 5{1000} (about 10{699}) configurations.
With numbers such as these, it really doesn't matter what kind of computer is
available; checking each possibility is out of the question. Nevertheless, the
task remains and neither guessing nor giving up are viable options. An answer
is usually found with the help of a little computer assistance, often using
simulated annealing. Before looking at the usefulness of this simulated
annealing, however, let's examine its main competitor.


The Greedy Algorithm


Perhaps the simplest general method for trying to locate a global optimum
(maximum or minimum) in a discrete space is a technique often referred to as
the "greedy" algorithm. Described in English, it goes something like this:
1. You randomize the starting configuration.
2. If possible, you find a better configuration by making repeated (usually
random) changes to the current configuration.
3. Make this better configuration the current one, then go back to step 2.
4. If a better configuration is not found after a reasonable amount of effort,
conclude that you have obtained a local optimum. Save it in the variable
OPTIMUM.
5. Repeat from step 1, replacing OPTIMUM if appropriate, until your patience
or computing resources are exhausted. Hope that the final value of OPTIMUM is
global.
Until you try a few experiments with this algorithm on large problems with
known answers, there is a strong temptation to believe that it might actually
work. However, unless the size of the problem (the number of elements to be
configured) is small or the configuration space exceptionally smooth, the
chances of finding a global optimum in this fashion are remote.
The deficiencies of the greedy algorithm are not obscure. Imagine a mountain
goat traipsing about the Rocky Mountains looking for the deepest valley (where
the best grass is). This goat is very hungry (greedy?) and, consequently,
never takes an upward step. Unless the goat starts out on a smooth slope of
the deepest valley, it will never reach that valley but will, instead, come to
a dead end in some higher valley, that is, in a local minimum, which may be
much higher than the global minimum. Moreover, a space as rough as the Rockies
contains thousands of valleys, some only a few feet deep, and repeating the
exercise (step 5, above) may yield improvement but probably not the best
result.


A Great Deal of Difficulty


The difficulties of combinatorial optimization can be illustrated in the game
of poker solitaire where the object is to rearrange the 25 cards of the
tableau, like that in Figure 1, so that the 12 hands of straight poker formed
from the 5 rows, 5 columns, and 2 diagonals give the highest total score.
Various scoring schemes are popular, but I shall use the one listed in Table 1
in which each hand of a pair or better has a value inversely proportional to
its probability in a standard deck, normalized to that of a single pair. All
hands of a given type (for example, full house) are considered equal, and an
ace can be high or low.</entry>
Table 1: Scoring table

 Hand Score
 ---------------------------

 Straight flush 27,456

 Four of a kind 1,760

 Full house 293

 Flush 215

 Straight 108

 Three of a kind 20

 Two pair 9


 Pair 1

Compared to the task of optimizing circuit placement on a microchip, this is a
small problem indeed. There are only 25! (about 10{25}) configurations and
even fewer if symmetry (rotations, reflections, and so on) is taken into
account. Nevertheless, this is such a huge number that, even if a computer
could evaluate a million configurations per second without repetition, the
execution time required to test all of them would be comparable to the present
age of the universe. To accommodate this situation, both simulated annealing
and the greedy algorithm employ stochastic techniques, evaluating only a
fraction of the possible configurations. Consequently, these algorithms
provide no absolute guarantee that they will find the global optimum every
time. For large problems, it is usually sufficient that a technique provides a
reasonable chance of coming close to the global optimum nearly every time.
Then, a few runs will yield as good an answer as you are likely to require.
The tableau in Figure 1 contains 7 pairs for a score of 7, although there are
obviously 4 aces and 4 kings, so it is reasonable to expect that a good score
would be at least 4000. Finding a really high score is not easy, however. The
12 poker hands are tightly interlocked, and even a small change in
configuration can result in a drastic change in score. This is another way of
saying that the configuration space is very rough. Informal experiments have
shown it to be more than a match for a human player, and it provides a good
test for any combinatorial optimization technique.


Simulated Annealing


The term annealing originally referred to a process employed in the
fabrication of objects constructed of metal or glass. When these materials are
shaped, small, often microscopic, regions of stress develop in response to
deformations at the atomic level and cause the object to be prone to fracture.
Annealing corrects this defect.
From a chemical standpoint, regions of stress have relatively high energy and
thus are "almost fractured" already. An object would be more stable (of lower
energy) if the stress were absent. However, atoms and molecules in a solid at
room temperature do not have enough energy, on average, to move about and
relieve the stress. (If they did, the object wouldn't be very solid.) When an
object is annealed, it is first heated enough to provide the constituent atoms
with sufficient energy to relax any stress but not enough to initiate melting.
It is then cooled very slowly. During the cooling phase, the atoms are
gradually "frozen" in place. If the annealing is done properly, the resulting
object will be without stress. If the cooling is too rapid, there will be
insufficient time for the atomic structure to relax completely, and the object
will end up with excess energy and fracture too easily.
In simulated annealing, score is associated with energy such that minimizing
the energy means optimizing the score. In the poker solitaire example, high
score signifies low energy. The tableau in Figure 1 can be thought of as a
collection of "atoms" frozen into a configuration that has too much energy.
Proceeding with this analogy, the tableau may be subjected to simulated
annealing by raising its "temperature" until it is almost "melted," then
cooling it slowly until it is again "frozen." Although the connection between
real and simulated annealing appears to be little more than a metaphor, this
system (25 playing cards) responds in the same way as does a physical system
-- and with the same results.
More Details.
In any physical system, the average </entry> energy, <E>, of the constituent
atoms is a function of the absolute temperature, T. As the temperature of a
system decreases, the average energy of its atoms also decreases. A typical
cooling curve is shown in Figure 2. The slope of this curve, d<E>/dT, is the
heat capacity, C, of the system. In regions where the curve is steep, the heat
capacity is high. This usually indicates a phase change -- water into ice, for
example. Quite often, these regions also display large values of C/T. This
quantity is equal to dS/dT, the rate of change of entropy, S, with
temperature, which is related to a change in the orderliness of the atoms.
Therefore, regions of a cooling curve that are steep usually imply a
significant reconfiguration of the atoms (or molecules) at the corresponding
temperatures. In an annealing process where temperature is being regulated,
you would have to slow down the cooling in such regions in order to allow the
atoms time to rearrange themselves and to permit their average energy to
decrease gradually. Annealing works best when this is done.
Simulated annealing also produces cooling curves much like that of Figure 2.
In fact, the literature of simulated annealing demonstrates that these
analogies run very deep and it is not at all easy to decide just where to draw
the line between metaphor and methodology.


The Connection


The logical link between combinatorial optimization and annealing is the
Boltzmann distribution, one of the basic laws of statistical mechanics. Up to
now, I've referred to the average energy of a collection of atoms. In most
large populations, there is significant variation from individual to
individual. So it is with atoms and molecules. They are constantly moving, and
their mutual collisions result in continual transfers of energy from one atom
(or molecule) to another. Much of the science of statistical mechanics
involves techniques for extracting macroscopic properties of a system from the
collective behavior of enormous numbers of atoms and molecules. This
information may be obtained, to a large extent, by determining how energy is
partitioned among the atoms and molecules making up the sample.
At any temperature, there are generally more atoms at lower energies than at
higher energies. The Boltzmann distribution, (see Example 1, equation 1)
describes the relative number of atoms (or molecules) populating any two
energy states, E [hi] > E [lo], at some temperature, T, where N[x] is the
number of atoms with energy E[x], T is the absolute temperature (degrees
Kelvin), and k is Boltzmann's constant. When equation 1 is adapted to the
problem of combinatorial optimization, the constant is ignored, energy is
equated to score (cost), and the left-hand side of equation 1 is interpreted
as a probability. Simulated annealing uses such probabilities to direct a
stochastic search of the configuration space by employing equation 2, Example
1 where S[x] is a score and T is a scaling factor.
Example 1: The Boltzmann distribution in chemistry and computer science

 Equation 1: N [hi]/N [lo] = exp ((E [lo]-E [hi])/kT)
 Equation 2: Prob (accept worse score) = exp ((S [lo]-S [hi])/T)

In other words, you construct a new configuration at random and determine its
score. If this score is at least as good as that of the current configuration,
then the old configuration is forgotten and the new configuration becomes the
current configuration, as in the greedy algorithm. If the new score is worse
(here, lower), however, then equation 2 is used to generate the probability of
accepting the new configuration anyhow. A uniform random number in the
interval (0,1) is then generated, and if it is less than the probability
computed with equation 2, the new configuration is made the current
configuration. The net effect is to allow the system to extricate itself from
local optima (that is, to let the goat climb uphill sometimes).
As equation 2 suggests, choosing a worse configuration is a function of
temperature. The higher the temperature, the higher the "energy" of the
system. The higher the energy of the system, the more readily it will do the
work of moving in an unfavorable direction -- just like real atoms and
molecules. As the temperature decreases, however, any given expenditure of
energy becomes less likely. Consequently, simulated annealing automatically
adapts to the magnitude of the features of a good configuration. The most
profitable features are found first and the smaller, more subtle, features
later. Starting with the tableau in Figure 1, simulated annealing first finds
the four kings and four aces. If the temperature is very high, these features
may be forgotten from time to time, but as the temperature decreases,
eventually they are frozen into every acceptable tableau. As the temperature
continues to fall, hands of lower and lower scores are successively frozen
until, finally, the entire tableau is frozen. Simulated annealing is
terminated at this point.
It is clear that the annealing schedule -- the program for reducing the
temperature -- is critical; a large drop in temperature at the wrong time may
freeze in undesirable features. (Ideally, all reconfigurations should be
reversible.) The annealing schedule answers the following four questions:
1. What should be the value of the starting temperature?
2. When should the temperature be changed?
3. How should the temperature be changed?
4. When should the search be stopped?
A quick-and-dirty procedure is to pick a starting temperature high enough so
that virtually all scores are deemed acceptable, hold that and each successive
temperature for (100*SIZE) reconfigurations or (10*SIZE) successful
reconfigurations, whichever comes first, then decrease the temperature by a
constant factor -- for example, T(new) = 0.9*T(old) -- stopping when T reaches
some very low value (say, 0.1). Such a schedule is often employed for an
exploratory run on a new problem in order to determine the shape of the
cooling curve. Given the cooling curve, you can then create a proper annealing
schedule by arranging for the temperature to decrease most slowly where the
curve is steepest.


Implementation Details


The ANNEAL.C program (see Listing One) addresses both the poker solitaire
problem and computes the average and standard deviation of the scores accepted
at any temperature. These statistics help in answering the questions posed
above.
To choose a starting temperature, begin by picking one that seems high
relative to the scores expected and run the program for that temperature only.
There should be very few reconfigurations rejected. After equilibrium is
reached at this temperature (see below), statistics will be output, one of
which is SIGMA, the standard deviation of the acceptable scores. As the
temperature approaches infinity, SIGMA approaches SIGMA(inf), the standard
deviation of a large set of random configurations. Set the starting
temperature equal to 20*SIGMA(inf).
There is no point in trying further reconfigurations at a given temperature
once a system has reached thermal equilibrium. (The method used here for
recognizing equilibrium is adapted from a procedure published by Huang, Romeo,
and Sangiovanni-Vincentelli.)
ANNEAL.C keeps track of the number of reconfigurations accepted (nsuccesses)
and rejected (nfailures) at a given temperature as well as the number of
acceptable scores less than or more than SIGMA/2 from the average acceptable
score (incount and outcount, respectively). The code in Listing Two is then
used to decide whether or not equilibrium has been reached. (Uppercase
variables are parameters.)
Once equilibrium has been established, the temperature is decreased, provided
that the current configuration has a score no worse than SIGMA/2 from the
average score. The temperature is not changed until this is true. The idea is
to avoid transmitting a poor outlier to a new, lower temperature. As shown,
there is no explanation for delaying the transition to a lower temperature.
The simplest and most flexible method for computing a new temperature is to
use a table, matched to the cooling curve, listing the desired sequence of
temperature ratios, T(new)/T(old), and the temperatures at which they take
effect. Typically, these ratios fall in the range (0.5, 1.0). ANNEAL.C permits
up to ten values for these ratios in addition to the initial value. Table 2
gives the actual ratios used in this example.</entry>
Table 2: Temperature ratios

 Temperature t_ratio
 Less Than
 -------------------------

 -- 0.7
 360 0.8
 215 0.7
 85 0.8
 60 0.9
 30 0.95
 15 0.9

 7 0.8
 3 0.7
 0 0.7

The configuration is considered frozen if, at equilibrium, the largest single
change in score observed at the current temperature is equal to the difference
between the best and worst scores at this temperature. This is taken to
indicate that all accessible scores are of comparable value and there is no
further need for simulated annealing. In ANNEAL.C, this test is carried out
only after the temperature reaches a preset limit, t_low. As a failsafe,
another parameter, t_min, is stipulated as the temperature at which the
configuration is always assumed to be frozen. Once the configuration is
frozen, it is "quenched" by setting the temperature equal to zero (greedy
algorithm).
In the interest of expediency, the best configuration found is always
remembered separately from the current configuration. Ideally, the final
configuration should be the best. This is often not the case, however. At the
end of the program, this best-found configuration is also quenched.


Giant Steps and Baby Steps


Although a proper annealing schedule is crucial to the success of simulated
annealing, there is another consideration that is nearly as important
--namely, the manner in which you make a random change to the current
configuration. This is true for simulated annealing and, to a lesser extent,
for the greedy algorithm. To find a global optimum, any search technique must
be able to access the entire configuration space. To do this effectively
requires both "giant steps" and "baby steps," the former for moving quickly to
a distant part of the space and the latter for fine-tuning.
In this example, baby steps are easy -- just interchange two cards. This is
the smallest reconfiguration possible. Finding good giant steps is much more
difficult. Of course, you could simply execute several baby steps at once but
experience shows that this strategy yields poor results. One reason is that,
in making a giant step, it is essential to retain as much of the current score
as possible. At moderate to low temperatures, any arbitrary giant step would
almost certainly cause a substantial worsening of the score and the new
configuration would simply be rejected. All rejected moves are wasted and an
efficient algorithm must try to keep such moves to a minimum.
In ANNEAL.C, a giant step consists of interchanging 2 of the 12 hands of the
tableau, selected at random from a uniform distribution. A baby step, on the
other hand, is implemented using a "rotation." A random integer in the range
[2,5] is chosen from an exponential distribution (see Table 3). This number of
cards is then selected from the tableau using a uniform distribution. The
first card selected is temporarily set aside and the resulting hole filled
with card number 2, and so on. The last hole is filled with card number 1. The
ratio of giant steps to baby steps is a parameter, exg_cutoff, here set equal
to 0.2. Using only giant steps or baby steps generally gives poor
performance.</entry>
Table 3: Rotation distribution

 # of Cards Probability
 to Rotate
 -----------------------------

 2 0.33333
 3 0.27018
 4 0.21899
 5 0.17750



Experimental Results


Simulated annealing and the greedy algorithm are both stochastic techniques.
Therefore, they do not give the same results every time. The proper way to
evaluate a stochastic technique is to apply it several times and determine its
average behavior (actually, its behavior distribution). One of the reasons
such a small example was chosen for this article was precisely to allow such a
test to be done in a reasonable amount of time.
The simulated annealing algorithm was carried out 30 times, starting with the
tableau in Figure 1 each time, using a random-number generator with a period
much longer than that required by the experiment. The greedy algorithm was
then applied to the same tableau for the same amount of CPU time. Apart from
the rule for accepting or rejecting a new configuration, the two programs are
essentially identical. In particular, all the relevant parameter values are
the same (see A.DAT, Listing Three). The best result from either technique is
a score of 4516 (see Figure 3). This apparent optimum, found once by the
greedy algorithm and twice by simulated annealing, is degenerate (that is,
there are several tableaux with this score, which are not symmetry-related).
The distribution of results for simulated annealing is shown in Figure 4 and
that for the greedy algorithm in Figure 5.
The number of iterations used to generate Figure 4 and Figure 5 should be
enough to give an approximate indication of the average performance of the two
techniques. No serious attempt was made to find the combination of parameters
that would optimize the simulated annealing results because this would have
constituted a bias against the greedy algorithm and such a combination would
have applied too specifically to this particular initial tableau. The
parameter values in A.DAT appear to be adequate for the poker solitaire
problem in general although the annealing schedule will have to be altered
when the cards are changed, especially if the new initial tableau can produce
a straight flush (see accompanying box). For other combinatorial optimization
problems, both the parameter values and the annealing schedule will have to be
changed. The big questions, of course, are, "Does simulated annealing work?"
and "Is it worth the effort?" Taken together, the results shown in Figure 4
and Figure 5 suggest that simulated annealing is definitely worth the effort.
Although you could argue that, given equal amounts of computation time, both
techniques find what appears to be the global optimum, this would be overly
simplistic. For stochastic techniques, the important criterion is their
average behavior.</entry>
More Details.
For this example, the probability that any given iteration of the greedy
algorithm yields a score greater than or equal to the average simulated
annealing result is 6/1583. Thus, 183 iterations of the greedy algorithm
afford a 50 percent chance of doing at least as well as simulated annealing
(607 iterations for a 90 percent chance). The greedy algorithm, however,
converges only 53 times faster in this example than does simulated annealing.
Clearly, there is a higher payoff with simulated annealing.


Final Thoughts


Although I've compared simulated annealing to the well-known greedy algorithm,
it would be incorrect to suggest that the latter is the only alternative.
Recently, neural networks and various genetic algorithms have been applied to
combinatorial optimization problems with some success. If the problems are
small enough, then classical integer programming techniques (that is,
branch-and-bound) may also prove fruitful. Moreover, when evaluating the
material presented here, there are several points to bear in mind. First, not
every application of simulated annealing will exhibit better performance than
a greedy algorithm to the extent observed in this case, so you should not
extrapolate too much from this single example.
Another important consideration is that simulated annealing is very
time-consuming, and almost any serious application of this sort is best
written in assembly language. The program ANNEAL.C was coded in C solely in
the interests of pedagogy and portability.
Finally, both algorithms described in this article are stochastic in nature.
Even though the tableau in Figure 1 was subjected to hundreds of simulated
annealing runs, there is still no guarantee that the optimum found is, in
fact, the global optimum. It's possible that there exists a tableau with a
score greater than 4516 but that the path through the configuration space
leading to it is so narrow that even the millions of reconfigurations carried
out are insufficient to find it.
In this regard, it's interesting that all the tableaus found having scores of
4516 contained only 11 scoring hands, not 12. This being the case, there may
be "room" for an even higher score. Better yet, what if a 13th hand were
defined, consisting of the 4 corners plus the center card. Would there be any
point in adding the extra degree of freedom? This intriguing thought is left
as an exercise for DDJ readers.


Bibliography


Huang, M. D.; Romeo, F.; and Sangiovanni-Vincentelli, S. "An Efficient Cooling
Schedule for Simulated Annealing." Proc. Int. Conf. Computer-Aided Design
(ICCAD86) (1986): 381 - 384.
Kirkpatrick, S.; Gelatt, C. D. Jr.; and Vecchi, M. P. "Optimization by
Simulated Annealing." Science, vol. 220, no. 4598 (1983): 671 - 680.
Metropolis, N.; Rosenbluth, A;, Rosenbluth, M;, Teller, A;, and Teller, E.
"Equations of State Calculations by Fast Computing Machines." J. Chemical
Physics, vol. 21 (1953): 1087 - 1091.
Nash, L.K. Elements of Statistical Thermodynamics. Reading, Mass.:
Addison-Wesley, 1968.
Press, W. H.; Flannery, B. P.; Teukolsky, S. A.; and Vetterling, W. T.
Numerical Recipes, The Art of Scientific Computing. New York: Cambridge
University Press, 1986: 326 - 334.
Sechen, C. and Sangiovanni-Vincentelli, A. "TimberWolf 3.2: A New Standard
Cell Placement and Global Routing Package." Proc. 23rd Design Automation Conf.
(1986): 432 - 439.


Availability



All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063, or call 800-356-2002 (from inside
Calif.) or 800-533-4372 (from outside Calif.). Please specify the issue number
and format (MS-DOS, Macintosh, Kaypro).</entry>


ANNEAL.C Data Structures, Functions, and Variables


The playing cards are represented as 2-byte integers. The card values, 2 to
Ace (high), are coded as the numbers 2 to 14 and the suits {spades, hearts,
diamonds, clubs} are coded as {2048, 1024, 512, 256}, respectively. The
tableau is intrinsically a two-dimensional array but is implemented as a
one-dimensional array to avoid the computational burden of double indexing
(see HAND[12][5] below). ANNEAL.C ignores the symmetry of the tableau with
respect to the scoring function because it is, computationally, more trouble
than it is worth.
rand1( ) -- takes a 31-bit integer, SEED, and returns another one. This
generator yields a full-cycle sequence without repeating any numbers.
uni(z) -- returns a double-precision float in the range (O,z) via rand1( ).
initialize( ) -- reads in the starting data from A.DAT and sets up the data
structures, and so on.
init( ) -- resets a number of global counters and variables, chiefly to aid in
the collection of the statistics necessary to detect equilibrium.
get_new_temperature( ) --computes a temperature ratio, if necessary, and a new
temperature (see Table 2).
check_equilibrium( ) -- uses the code given in the article to detect the
equilibrium point.
update( ) -- computes those statistics, which need not be found until
equilibrium has been achieved. (see SCALE).
selectwr( ) -- is a general routine to perform N selections without
replacement.
reconfigure(direction) -- makes a new configuration, or restores the tableau
to its previous configuration if the last move was rejected. (Typically, more
than 90 percent of the reconfigurations are rejected.) This routine uses giant
steps or baby steps.
get_score(crds) -- is called by evaluate1( ) and returns the score of a single
hand as defined in HAND[12][5].
evaluate1( ) -- passes each hand, one by one, to get_score(crds) and tallies
up the total SCOR (which becomes the current SCORE, if the reconfiguration is
accepted). Along with rand1( ) and reconfigure( ), the functions evaluate1( )
and get_score( ) are all written in MC68020 assembly language in the working
code. This code runs at a speed of approximately 2035 attempted
reconfigurations/sec. on an AMIGA 1000 equipped with a 68020/68881 chip set,
plus 32-bit RAM (a standard Amiga, executing ANNEAL.C, does 70/sec.), and
finishes an average simulated annealing run in less than 3 minutes with the
results as shown in Figure 4. Without this sort of speed, the long series of
experiments needed to obtain good parameter values, and to compare with the
greedy algorithm, would have required more patience than this author has ever
possessed.
decide( ) -- answers the question "To accept or not to accept?" and uses the
formula shown Example 1, equation 2. It also accumulates statistics like
INCOUNT and OUTCOUNT that must be computed at every move.
report( ) -- produces the output statistics and the final tableaux.
try_new_temperature( ) -- is a calling function that uses a newly computed
temperature. It also checks that the tableau passed on to a new temperature is
not a low outlier.
main( ) -- q.v. HAND[12][5] -- stipulates the 12 hands (5 cards each) in the
TABLEAU and describes the various hands in terms of the appropriate indices of
TABLEAU.
PARTITION[6], OVERLAP[66], HEX [12], CINDEX[25] -- are used to generate a new
configuration. PARTITION implements Table 3 in long integers (pre-multiplied
by MODULUS). OVERLAP is a triangular array giving the index, in TABLEAU, of
the intersection of two HANDs (-1 if no overlap). HINDEX and CINDEX are arrays
used to perform selection without replacement and to remember those selections
should the new configuration be rejected.
SCORES[18] -- is an implementation of Table 1. After evaluate1( ) has
determined the index to SCORES, SCOR is assigned the corresponding value.
TEMPERATURE, T_LOW, T_MIN --is the scaling factor. T_LOW is the temperature
below, which the tableau can be tested to see if it is FROZEN or not. If the
temperature ever goes below T_MIN, the tableau is automatically FROZEN.
AVG_SCORE, SIGMA -- are the average and standard deviation of the scores of
all the tableaux accepted at the current temperature.
SCALE -- is a typical SCORE when the temperature is getting cold. This
variable has no function in the algorithm per se, but is incorporated so that
SIGMA may be computed using single-precision arithmetic. (This is especially
important in the absence of a floating-point chip.) By subtracting off a good
score before accumulating the square-of-the-score, the number of digits to be
squared is reduced. If this is not done, then a single-precision number will
lose too much precision at a time in the annealing process when the
average-of-the-squares and the square-of-the-average are nearly equal. When
this happens, it is possible to get SIGMA = 0 erroneously. This has a
deleterious effect on the program.
T_RATIO -- T(new)/T(old). The initial value (here, 0.7) and 10 other values
are listed in A.DAT.
EXG_CUTOFF -- is the ratio Prob (giant step)/Prob(baby step).
SUCC_MIN, INCOUNT_LIMIT, OUT-COUNT_LIMIT, FIRST_LIMIT, ULTIMATE_LIMIT -- are
parameters used to determine the equilibrium point.
-- M.M.



Cellular Automata: A New Way of Simulation


Jean F. Coppola and Dr. Francis T. Marchese
Jean Coppola is a master's degree student in computer science and Dr. Francis
Marchese is an associate professor in the computer science department at Pace
University. They can be reached at Pace University, 1 Pace Plaza, New York, NY
10038.
In the annealing process, glass, certain metals, and alloys are heated and
cooled in a controlled way to allow the atoms to arrange themselves into a
stable state. Normally, the process of annealing is simulated by means of a
set of complex differential equations. Alternatively, annealing can be
simulated by an intuitively simpler method known as "cellular automata."
Cellular automata are discrete dynamical systems in which the behavior of the
system as a whole is defined in terms of a set of local rules. The simplest
and best-known cellular automata is the game of LIFE conceived by John Conway
in the early 1970s. LIFE is a computer simulation that consists of a grid of
cells upon which a collection of creatures are born, live and die. The
life/death process is governed by a neighborhood rule that defines the
behavior of any creature at any cell located in terms of the behavior of its
neighbors. This is a significant concept because it implies that for
collective systems, no member of that collection of entities (or creatures)
can exist independently of the rest. Moreover, the rules that govern the
system as a whole are only the rules of the local neighborhoods. Hence, for
example, in a two-dimensional life game consisting of a 100 x 150 grid, the
rules are only defined for each local 3 x 3 matrix! Even though this is the
case, the complexity of the life process produced can be quite profound.
As a result, cellular automata has been applied in areas such as physics,
chemistry, and computer science. For example, particle dynamics, galactic
evolution, molecular kinetics, or parallel distributed processing are all
fields where local laws dictate global behavior. At Pace University, we've
applied cellular automata to the real-time study of the dynamics of crystal
growth. Because of the large number of local interactions that must be
accommodated in these simulations, even a mainframe cannot provide the
computing power required for real-time visualization. However, assisting in
the simulations, is an IBM PC-compatible, high-performance hardware option
called CAM-6 (Cellular Automata Machine) that runs under MS-DOS. CAM-6 is a
special-purpose coprocessor designed specifically for cellular automata
simulations by researchers at the MIT Laboratory for computer science. Its
pipelined architecture allows software to run at speeds comparable to that of
a CRAY-1, therefore allowing the CAM board to update and display a 256 x 256
array of cells 1/60 of a second, thus producing real-time animation. Each cell
in CAM is represented by a pixel, and cells can assume 1 of 16 possible states
signified by 16 different colors.
The CAM-6 machine is programmable in a high-level language called "CAM Forth,"
a special version of Forth-83. Because Forth was originally developed for
real-time applications and meets the demands of cellular automata by its code
efficiency.
We have used this combination of hardware and software to simulate crystal
growth. Presently, we wish to discover the local rules which govern the growth
of sodium chloride (common table salt). This is accomplished by programming
neighborhoods that work with the four major neighbors on all the planes. The
difficult part in programming "rules" is keeping in mind that one cannot
change the state of a neighbor but only the cell currently in question. For
example, you cannot indicate if a cell has three or more neighbors at state 1,
and turn all neighbors at state 0 to 1. Therefore, programming has to be done
around this constraint, which at times can be extremely difficult. We have
also simulated complex solid liquid equilibria. This involved programming
minor neighborhoods which allowed states to be compared from a second CAM
machine that was programmed as a random noise generator.
By watching and studying these simulations, we hope to gain insight into the
synthesis of crystals and fundamental patterns of molecular growth. The models
of cellular automata could one day be a major breakthrough in the
understanding of nature's rules, which in turn could affect every aspect of
life.


--J.C., F.T.M.


_SIMULATED ANNEALING_
by Michael P. McLaughlin


[LISTING ONE]

/* ANNEAL.C -- Author: Dr. Michael P. McLaughlin
*/

#include <stdio.h>
#include <math.h>

#define ABS(x) ((x<0)?(-(x)):(x))
#define BOOLEAN int
#define TRUE 1
#define FALSE 0
#define FORWARD 1
#define BACK 0

struct cardtype {
 char card[3];
 int code;
 } cards[25];
float ratios[10][2];
int tableau[25],best_tableau[25];
float temperature,best_temperature,t_low,t_min,t_ratio;
long seed,score,best_score,tot_successes,tot_failures,scor,exg_cutoff,
 report_time,report_interval,modulus = 2147483399,step = 1;
int next;
BOOLEAN equilibrium,frozen = FALSE,quench=FALSE,final=FALSE;

/* variables needed to assess equilibrium */
float totscore,totscore2,avg_score,sigma,half_sigma,score_limit;
long bscore,wscore,max_change,nsuccesses,nfailures,scale,
 incount,outcount;
int incount_limit,outcount_limit,succ_min,first_limit,ultimate_limit;

/* poker hands in tableau */
int hand[12][5] = {0,1,2,3,4, /* rows */
 5,6,7,8,9,
 10,11,12,13,14,
 15,16,17,18,19,
 20,21,22,23,24,
 0,5,10,15,20, /* columns */
 1,6,11,16,21,
 2,7,12,17,22,
 3,8,13,18,23,
 4,9,14,19,24,
 0,6,12,18,24, /* diagonals */
 4,8,12,16,20};
 /* 0,4,12,20,24 = corners */

long rand1()
/* Get uniform 31-bit integer -- Ref. CACM, June 1988, pg. 742 */
{
 register long k;
 k = seed/52774;
 seed = 40692*(seed-k*52774)-k*3791;
 if (seed < 0)
 seed += modulus;
 return(seed);
}
double uni(z)
int z;
{
 return((double) z*rand1()/modulus);
}
void initialize()

/* Set up entire simulated annealing run. */
{
 char label[20];
 double exgc;
 register i;
 FILE *fp;
 fp = fopen("a.dat","r");
 fscanf(fp,"%f %s\n",&temperature,label);
 fscanf(fp,"%f %s\n",&t_low,label);
 fscanf(fp,"%f %s\n",&t_min,label);
 fscanf(fp,"%f %s\n",&avg_score,label);
 fscanf(fp,"%ld %s\n",&scale,label);
 fscanf(fp,"%f %s\n",&t_ratio,label);
 fscanf(fp,"%f %s\n",&sigma,label);
 fscanf(fp,"%lf %s\n",&exgc,label);
 fscanf(fp,"%d %s\n",&succ_min,label);
 fscanf(fp,"%d %s\n",&incount_limit,label);
 fscanf(fp,"%d %s\n",&outcount_limit,label);
 fscanf(fp,"%d %s\n",&first_limit,label);
 fscanf(fp,"%d %s\n",&ultimate_limit,label);
 fscanf(fp,"%ld %s\n",&report_interval,label);
 fscanf(fp,"%ld %s\n",&seed,label);
 half_sigma = 0.5*sigma; report_time = report_interval;
 score_limit = 20; exg_cutoff = modulus*exgc;
 for (i=0;i<10;i++)
 fscanf(fp,"%f %f\n",&ratios[i][0],&ratios[i][1]);
 for (i=0;i<25;i++) {
 fscanf(fp,"%d %s\n",&cards[i].code,cards[i].card);
 tableau[i] = cards[i].code;
 }
 fclose(fp);
 printf(" TOTAL TOTAL CURRENT AVERAGE\n");
 printf("STEP TEMPERATURE SUCCESSES FAILURES SCORE SCORE ");
 printf(" SIGMA\n\n");
}
void init()
/* set up for new temperature */
{
 nsuccesses = 0; nfailures = 0; equilibrium = FALSE; incount = 0;
 outcount = 0; bscore = 0; wscore = 100000; max_change = 0;
 totscore = 0; totscore2 = 0;
 if (temperature < t_min)
 frozen = TRUE;
}
void get_new_temperature()
{
 if (temperature < ratios[next][0]) {
 t_ratio = ratios[next][1];
 next++;
 }
 temperature = t_ratio*temperature;
}
void check_equilibrium()
/* Determine whether equilibrium has been reached. */
{
 if ((nsuccesses+nfailures) > ultimate_limit)
 equilibrium = TRUE;
 else if (nsuccesses >= succ_min) {
 if (incount > incount_limit)

 equilibrium = TRUE;
 else {
 if (outcount > outcount_limit) {
 if (nsuccesses > first_limit)
 equilibrium = TRUE;
 else {
 incount = 0;
 outcount = 0;
 }
 }
 }
 }
}
void update()
/* Compute statistics, etc. */
{
 float ascore,s;
 if (nsuccesses > 0) {
 ascore = totscore/nsuccesses;
 avg_score = ascore+scale;
 s = totscore2/nsuccesses-ascore*ascore;
 if (s > 0.0) {
 sigma = sqrt(s); half_sigma = 0.5*sigma;
 score_limit = avg_score-half_sigma;
 }
 }
 if ((temperature < t_low)&&((bscore-wscore) == max_change))
 frozen = TRUE;
}
void selectwr(array,mode,nchoices)
/* Select from array without replacement. */
int *array,mode,nchoices;
{
 int i,temp,size,index;
 size = mode==1 ? 12 : 25;
 for (i=0;i<nchoices;i++) {
 index = uni(size);
 temp = array[--size];
 array[size] = array[index];
 array[index] = temp;
 }
}
void reconfigure(direction)
/* If direction is FORWARD, make a new move; if it is BACK, restore
 the tableau to its previous configuration. */
int direction;
{
 static long partition[6] = {0,0,715827799,1296031795,1766307855,
 2147483400};
 static int hindex[12] = {0,1,2,3,4,5,6,7,8,9,10,11};
 static int cindex[25] = {0,1,2,3,4,5,6,7,8,9,10,11,12,13,
 14,15,16,17,18,19,20,21,22,23,24};
 static int overlap[66] = {-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,0,5,10,15,
 20,1,6,11,16,21,-1,2,7,12,17,22,-1,-1,3,
 8,13,18,23,-1,-1,-1,4,9,14,19,24,-1,-1,-1,
 -1,0,6,12,18,24,0,6,12,18,24,4,8,12,16,20,
 20,16,12,8,4,12};
 static int mode,nchoices,common,last; /* remember changes */
 int temp,hi,lo,i,j,k;

 if (direction == FORWARD) {
 if (rand1() < exg_cutoff) { /* giant step */
 mode = 1; nchoices = 2;
 selectwr(hindex,mode,nchoices);
 hi = hindex[11]; lo = hindex[10]; /* order hand indices */
 if (hi < lo) {
 hi = lo;
 lo = hindex[11];
 }
 common = overlap[hi*(hi-1)/2+lo]; /* triangular matrix */
 }
 else { /* baby step */
 mode = 0; nchoices = 2;
 rand1(); /* How many cards to rotate? */
 while (seed > partition[nchoices])
 nchoices++;
 selectwr(cindex,mode,nchoices);
 temp = tableau[cindex[24]]; /* rotate forward */
 for (i=1,j=24;i<nchoices;i++,j--)
 tableau[cindex[j]] = tableau[cindex[j-1]];
 tableau[cindex[j]] = temp;
 last = j;
 }
 }
 if (mode == 1) { /* swapping hands is symmetrical */
 register first,second,c;
 first = hindex[11]; second = hindex[10];
 k = (common<0) ? 5 : 4;
 for (c=0,i=0,j=0;c<k;c++,i++,j++) {
 if (hand[first][i] == common)
 i++;
 if (hand[second][j] == common)
 j++;
 temp = tableau[hand[first][i]];
 tableau[hand[first][i]] = tableau[hand[second][j]];
 tableau[hand[second][j]] = temp;
 }
 }
 else if (direction == BACK) { /* rotation is not */
 temp = tableau[cindex[last]];
 for (i=1,j=last;i<nchoices;i++,j++)
 tableau[cindex[j]] = tableau[cindex[j+1]];
 tableau[cindex[24]] = temp;
 }
}
long get_score(crds)
/* Return score of one hand. */
int *crds;
{
 static long scores[18] = {0,1,1,20,1,9,20,1760,1,9,9,
 293,20,293,1760,108,215,27456};
 int flush,i,index = 0;
 flush = crds[0]&crds[1]&crds[2]&crds[3]&crds[4]&0xff00;
 for (i=0;i<5;i++)
 crds[i] = crds[i]&0xff; /* ignore suits */
 {
 register b,k,t; /* sort cards from high to low */
 for (b=4;b>0;b--)
 for (k=0;k<b;k++)

 if (crds[k] < crds[k+1]) {
 t = crds[k];
 crds[k] = crds[k+1];
 crds[k+1] = t;
 }
 }
 if (!flush) { /* multiple cards ? */
 if (crds[0] == crds[1]) index += 8;
 if (crds[1] == crds[2]) index += 4;
 if (crds[2] == crds[3]) index += 2;
 if (crds[3] == crds[4]) index += 1;
 }
 if (!index) { /* straight ? */
 if (((crds[0]-crds[4]) == 4)((crds[0] == 14)&&(crds[1] == 5)))
 index = 15;
 if (flush)
 index = (index == 15) ? 17 : 16;
 }
 return(scores[index]);
}
long evaluate1()
/* Return total score of tableau (no corners). */
{
 int temp[5],h,c;
 long scr = 0;
 for (h=0;h<12;h++) { /* h<13 for corners */
 for (c=0;c<5;c++)
 temp[c] = tableau[hand[h][c]];
 scr += get_score(temp);
 }
 return(scr);
}
void decide()
/* Either accept new configuration or restore old configuration */
{
 float pdt1,pdt2,sscor;
 long s;
 BOOLEAN acceptable = FALSE;
 scor = evaluate1();
 if (scor >= score) {
 if (scor > best_score) {
 register i;
 best_score = scor;
 best_temperature = temperature;
 for (i=0;i<25;i++)
 best_tableau[i] = tableau[i];
 }
 acceptable = TRUE;
 }
 else if (temperature > 0.0) { /* uphill movement? */
 if (uni(1) < exp((scor-score)/temperature)) /* Equation 2 */
 acceptable = TRUE;
 }
 if (acceptable) { /* statistics, etc. */
 s = ABS(score-scor);
 if (s > max_change)
 max_change = s;
 score = scor;
 sscor = scor-scale; /* to aid precision of sigma */

 totscore += sscor;
 totscore2 += sscor*sscor;
 nsuccesses++;
 if (ABS(avg_score-score) < half_sigma)
 incount++;
 else
 outcount++;
 if (score > bscore) /* maximization */
 bscore = score;
 else if (score < wscore)
 wscore = score;
 }
 else { /* unacceptable */
 reconfigure(BACK);
 nfailures++;
 }
}
void report()
{
 tot_successes += nsuccesses; tot_failures += nfailures;
 printf("%3ld %10.1f %9ld %9ld %7ld %7.0f %5.1f\n",step,temperature,
 tot_successes,tot_failures,score,avg_score,sigma);
 report_time += report_interval;
 if (frozen) {
 int temp[25],k,kk;
 register i,j;
 BOOLEAN ok;
 if (final)
 printf("\nFINAL -- ");
 else if (quench)
 printf("\nQUENCH -- ");
 else
 printf("\nFROZEN -- ");
 printf("BEST SCORE = %5ld BEST TEMPERATURE = %5.1f\n\n",
 best_score,best_temperature);
 for (i=0;i<25;i++) {
 k = best_tableau[i];
 j = 0;
 ok = FALSE;
 while (!ok) {
 if (cards[j].code == k) {
 temp[i] = j;
 ok = TRUE;
 }
 else j++;
 }
 }
 for (i=0;i<25;i+=5) {
 for (j=i;j<(i+5);j++) {
 k = 4-strlen(cards[temp[j]].card);
 for (kk=0;kk<k;kk++) printf(" ");
 printf("%s",cards[temp[j]].card);
 }
 printf("\n");
 }
 printf("\n");
 }
}
void try_new_temperature()

{
 init();
 while (!equilibrium) {
 reconfigure(FORWARD);
 decide();
 check_equilibrium();
 }
 update();
 if (!quench) { /* ratchet */
 while ((float) score < score_limit) { /* maximization */
 reconfigure(FORWARD);
 decide();
 }
 }
}
main()
{
 initialize();
 while (!frozen) {
 try_new_temperature();
 update();
 if ((step == report_time)(frozen))
 report();
 step++;
 if (temperature > 0.0)
 get_new_temperature();
 }
/* Quench frozen configuration. */
 temperature = 0;
 quench = TRUE;
 try_new_temperature();
 update();
 report();
 step++;
/* Quench best configuration (rarely useful). */
 {
 register i;
 final = TRUE;
 for (i=0;i<25;i++)
 tableau[i] = best_tableau[i];
 }
 try_new_temperature();
 update();
 report();
 printf("SEED = %ld\n",seed); /* seed of next run */
}







[LISTING TWO]

 if ((nsuccesses+nfailures) > ULTIMATE_LIMIT)
 equilibrium = true;
 else if (nsuccesses >= SUCC_MIN) {
 if (incount > INCOUNT_LIMIT)

 equilibrium = true;
 else {
 if (outcount > OUTCOUNT_LIMIT) {
 if (nsuccesses > FIRST_LIMIT)
 equilibrium = true;
 else {
 incount = 0;
 outcount = 0;
 }
 }
 }
 }






[LISTING THREE]

2150 temperature
1.5 t_low
0.1 t_min
40 avg_score
4400 scale
0.7 t_ratio
150 sigma
0.2 exg_cutoff
200 succ_min
250 incount_limit
400 outcount_limit
2700 first_limit
10000 ultimate_limit
1 report_interval
1234567890 seed
360 0.8
215 0.7
85 0.8
60 0.9
30 0.95
15 0.9
7 0.8
3 0.7
0 0.7
0 0.7
1032 8H
2061 KS
270 AC
526 AD
522 10D
1029 5H
265 9C
1035 JH
1037 KH
525 KD
515 3D
263 7C
2058 10S
2062 AS

518 6D
1036 QH
267 JC
259 3C
2053 5S
269 KC
520 8D
1028 4H
1038 AH
519 7D
1026 2H



[EXAMPLE 1]

Equation 1: N /su/hi//N /su/lo/ = exp((E lo-E hi)/kT)

Equation 2: Prob(accept worse score) = exp((S /su/lo/ -S /su/hi/)/T)


Example 1: Equations that describe the Boltzman distribution








































September, 1989
FORCE-BASED SIMULATIONS


C++ makes it easy to find out how one body in a universe influences all others




Todd King


Todd is a programmer/analyst with the Institute of Geophysics and Planetary
Physics at UCLA. He is also associated with the NASA/JPL Planetary Data
Systems project. Todd can be reached at 1104 N Orchard, Burbank, CA 91506.


You've probably heard it said that the languages that best model the real
world are object-oriented. When you try to build a simulation system you
realize just how true this is. This article presents a force-based simulation
system written in Zortech's C++ that demonstrates how and why an
object-oriented language like C++ is a natural choice for implementing
simulation systems.
The most common type of simulation systems are probably circuit type
simulations, where components have inputs and perform transformation of the
input into some form of output. The output is then fed to another component.
With force-based systems, you have a collection of bodies that forms a
universe. Each body exerts a force on every other body in the universe. This
means that just the presence of a body has an influence on all other bodies.
This is significantly different from circuit-based simulation, where
influences are channeled from component to component.


Building a Forced-based System


Probably the most difficult and critical element in a simulation system is
managing the objects that exist in the simulation. The function of the object
manager is to maintain the communication between bodies and to enforce the
physical laws. Listing One contains the source code for the administrator for
the system presented in this article. In this listing, the administrator
method is called big_bang( ), of the object class UNIVERSE.
It is crucial to maintain the integrity of each body. In C++, this is best
accomplished by making the physical parameters private and providing methods
to return the physical parameters. Because the physical parameters for any
body in the system are the same, the class for bodies is called BODY (also in
Listing One). The method UNIVERSE::service( ) registers a body with the
UNIVERSE. The collection of bodies to be serviced is maintained as an array of
objects of class BODY, and is repeatedly stepped through when you call
UNIVERSE::big_bang( ).
In the class of BODY, the most notable method is apply_ force( ), which takes
an outside influence on the body and converts it into a change in physical
parameters. The physical law that determines how bodies influence each other
is encoded in this method. In the present system, we are using the derivative
of the force law of gravity, which results in an instantaneous velocity
vector. This velocity vector is then applied to the body's current velocity
vector to determine its new vector. The other member functions of the BODY
class are simple and I'll refer you to the source for details.
Viewing the results of the simulations can be done in a variety of ways. In
some cases, a table of numbers is sufficient, but in most cases the results
are best seen in some animated form. Listing Two contains those methods and
functions that are directly related to the display of a simulation. The method
BODY::update( ) displays a body on the screen. It also updates the position of
the body based on its velocity vector. Even though I choose to display things
as characters (so everyone can watch the simulation), it should be relatively
easy to alter the code to work with graphic images.


Simulating the Earth and Moon


Let's look at some examples. Example 1 is a simulation of the Earth and Moon.
The source is straight forward. We create the bodies, define their physical
parameters (mass, location, and velocity), and then let the UNIVERSE take care
of the rest. The numbers used in all the methods are real. So if the physical
laws on which the system is built match reality, the Moon should orbit the
earth every 27.3 days. In the example, one day is equal to one tick.
Example 1: Simulation of the Earth and the Moon

 /*------------------------------------------------
 Simulates the orbital dynamics of the Earth
 and Moon.
 Todd King
 --------------------------------------------------*/

 #include "simul.hpp"
 #include "simulscr.hpp"

 main() {
 BODY earth;
 BODY moon;
 UNIVERSE universe;

 earth.set_mass(5.98e24);
 earth.set_position(5.0e8, 5.0e8);
 earth.set_icon('E');
 moon.set_mass(7.36e22);
 moon.set_position (5.0e8, 8.8e8);
 moon.set_icon('M');
 moon.set_velocity(-1020.0, 0.0);

 universe.service(&earth);
 universe.service(&moon);

 universe.big_bang();

Another thing you will observe when you run this example is that the
Earth/Moon pair moves gradually to the left. While trying to understand this
result, I realized that the simulation system had revealed the notion of
"center of mass" for systems, even though this notion was not bred into the
system. The center of mass for a system is the point that two (or more)
orbiting bodies revolve around. It is this point that moves at the velocity of
the system as a whole. So when the Moon was given its initial velocity, the
center of mass was given the same velocity. Hence the entire system moves to
the left. Figure 1 illustrates this system.
Example 1 is a verification-type simulation that demonstrates that the
simulation system produces substantiated results. This allows us to perform a
hypothetical simulation and trust the results. Let's look at what would happen
if Planet X passed through our system. Assume that the planet is twice as big
as the Moon. Listing Three is the source for this simulation. The system is
the same as that presented in Example 1, with the addition of a third body
that moves through the system from the lower-left corner to the upper-right
corner. When running this example you'll see just how disruptive such an
occurrence would be, even if Planet X doesn't collide with any other body.
Figure 2, shows the path of Planet X through the Earth/Moon system.


Conclusions


Simulation systems have broader applications than the model presented here.
They are useful in developing and testing new theories. With simulation
systems you can change the physical laws and constants and observe the
results. One such constant is G, the gravitational constant, which can be
found in Listing Four. Simulations also allow us to duplicate events in nature
at a faster rate. For example, it takes the moon a lot less than 27.3 days of
wall clock time to revolve about the earth in Example 1.
Games are a form of simulation systems, and with a little work you can create
a game with this simulation system by adding a space ship body to one of the
examples and allowing the user to control the rockets on it. The goal would be
to obtain an orbit about one of the other bodies. Such a skill could be useful
as we begin to move off the surface of our planet.
The system presented here has a few limitations. First, the maximum distance
allowed between any two objects is the square root of the largest possible
value for a double. On a PC, this value is 1.3e154. Next, the number of bodies
that can exist in a UNIVERSE are limited. This is set with the parameter
MAX_BODIES and has an upper boundary that is the same as for an int, and is
limited because we use an array to track the bodies. Converting this to a link
list would eliminate this limitation.


Bibliography


Rusnick, Robert, and Halliday, David. Physics, Part 1. New York: John Wiley &
Sons, 1977.
Negoita, Constantin Virgil, and Ralescu, Dan. Simulation, Knowledge-Based
Computing and Fuzzy Statistics. New York: Van Nostrand Reinhold Company, Inc.,
1987.


Availability


All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063, or call 800-356-2002 (from inside
Calif.) or 800-533-4372 (from outside Calif.). Please specify the issue number
and format (MS-DOS, Macintosh, Kaypro).


_FORCED-BASED SIMULATIONS_
by Todd King



[LISTING ONE]

/*-------------------------------------
 Basic object classes and method
 definitions for simulation software
 File: simul.hpp
 Todd King
----------------------------------------*/
#include <stream.hpp>
#include <disp.h>
#include <conio.h>
#include <math.h>
#include <time.h>
#include <math.h>
#include "simconst.h"

#define ESC 27
#define MAX_BODY_POOL 100

typedef struct VECTOR_2D {
 double x, y;
};

class BODY {
 VECTOR_2D world;
 VECTOR_2D velocity;
 double mass;

 double gmass;
 char icon;
public:
 BODY();
 set_mass(double m);
 set_velocity(double x, double y);
 set_position(double x, double y);
 apply_force(VECTOR_2D from, double amount);
 VECTOR_2D position();
 double get_gmass();
 update();
 set_icon(char c);
};

/* Distance and time units are converted to screen units and ticks */
BODY::set_mass(double m) {
 double pow();

 mass = m;
 gmass = G * m;
};

BODY::set_velocity(double x, double y) {
 velocity.x = x;
 velocity.y = y;
};

BODY::set_position(double x, double y) {
 world.x = x;
 world.y = y;
};

BODY::apply_force(VECTOR_2D from, double gmass) {
 VECTOR_2D d;
 double rs;
 double v;
 double r;

 d.x = world.x - from.x;
 d.y = world.y - from.y;

 rs = (d.x * d.x) + (d.y * d.y);
 if(rs != 0.0) { // there's a seperation
 r = sqrt(rs);
 v = (gmass / rs) * SECS_PER_TIC;
 velocity.x += v * d.x / r;
 velocity.y += v * d.y / r;
 }
};

BODY::BODY() {
 world.x = 0;
 world.y = 0;
 velocity.x = 0;
 velocity.y = 0;
 icon = '*';
};

VECTOR_2D BODY::position() {

 VECTOR_2D vec;

 vec.x = world.x;
 vec.y = world.y;
 return(vec);
};

double BODY::get_gmass() {
 return(gmass);
}

BODY::set_icon(char c) { icon = c; }

class UNIVERSE {
 unsigned int body_cnt;
 BODY *body_pool[MAX_BODY_POOL];
public:
 UNIVERSE();
 service(BODY *bptr);
 big_bang();
};

UNIVERSE::UNIVERSE() {
 body_cnt = 0;
};

UNIVERSE::service(BODY *bptr) {
 if(body_cnt >= MAX_BODY_POOL) return(0);
 body_pool[body_cnt] = bptr;
 body_cnt++;
};

UNIVERSE::big_bang() {
 int i, j;

 init_screen();

 print_message(" Press ESC to stop.");
 for(;;) {
 print_tick();
 if(kbhit()) {
 switch(getch())
 {
 case ESC:
 return(0);
 }
 }
 sleep(0.1);

/* Let each body influence all others */
 for(i = 0; i < body_cnt; i++) {
 for(j = 0; j < body_cnt; j++) {
 if(j == i) continue; // don't apply force to self
 body_pool[i]->apply_force(body_pool[j]->position(),
 body_pool[j]->get_gmass());
 }
 }

/* Display all bodies */

 for(i = 0; i < body_cnt; i++) {
 body_pool[i]->update();
 }

 }
 deinit_screen();
};

sleep(double seconds) {
 time_t ltime1, ltime2;

 time(&ltime1);
 time(&ltime2);
 while(difftime(ltime1, ltime2) < seconds) {
 time(&ltime2);
 }
}








[LISTING TWO]

/*--------------------------------------------
 Methods which are related to screen I/O.
 File: simulscr.hpp
 Todd King
----------------------------------------------*/
#include "simconst.h"

#define DISPLAY_Y 24
#define DISPLAY_X 80

BODY::update() {
 extern VECTOR_2D Extent;

 VECTOR_2D screen;

 screen.x = DISPLAY_X * (world.x / EXTENT_X);
 screen.y = DISPLAY_Y * (world.y / EXTENT_Y);
 if( (screen.x < DISPLAY_X && screen.x >= 0.0) &&
 (screen.y < DISPLAY_Y && screen.y >= 0.0) ) {
 disp_move(DISPLAY_Y - (int) screen.y, (int) screen.x);
 disp_printf(" ");
 }

 world.x += velocity.x * SECS_PER_TIC;
 world.y += velocity.y * SECS_PER_TIC;

 screen.x = DISPLAY_X * (world.x / EXTENT_X);
 screen.y = DISPLAY_Y * (world.y / EXTENT_Y);
 if( (screen.x < DISPLAY_X && screen.x >= 0.0) &&
 (screen.y < DISPLAY_Y && screen.y >= 0.0) ) {
 disp_move(DISPLAY_Y - (int) screen.y, (int) screen.x);
 disp_printf("%c", icon);

 }
 disp_move(0,0);
};

init_screen() {
 disp_open();
 disp_move(0, 0);
 disp_eeop();
}

deinit_screen() {
 disp_close();
}

print_tick() {
 static unsigned int Tick = 0;

 disp_move(0, 0);
 disp_printf("Tick: %u", Tick);
 Tick++;
}

print_message(char mesg[]) {
 disp_move(24,0);
 disp_printf(mesg);
}






[LISTING THREE]

/*------------------------------------------------
 Simulation of what would happen if a planet
 about twice the size of the Moon passed
 close to the earth within the Moon's orbit.
 Todd King
--------------------------------------------------*/
#include "simul.hpp"
#include "simulscr.hpp"

main() {
 BODY earth;
 BODY moon;
 BODY planet_x;
 UNIVERSE universe;

 earth.set_mass(5.98e24);
 earth.set_position(5.0e8, 5.0e8);
 earth.set_icon('E');
 moon.set_mass(7.36e22);
 moon.set_position(5.0e8, 8.8e8);
 moon.set_icon('M');
 moon.set_velocity(-1020.0, 0.0);
 planet_x.set_mass(14.8e22);
 planet_x.set_position(1.0, 1.0e4);
 planet_x.set_icon('X');

 planet_x.set_velocity(1800, 2000);

 universe.service(&earth);
 universe.service(&moon);
 universe.service(&planet_x);
 universe.big_bang();
}








[LISTING FOUR]

/*-----------------------------------------------------
 Various constants which affect the simulation
 system. Units are in meters, seconds and Kilograms.
 File: simconst.h
 Todd King
-------------------------------------------------------*/
#define G -6.67e-11 /* Gravitational constant */
#define SECS_PER_TIC 86400 /* One day */
#define EXTENT_X 10e8 /* Width of displayed universe */
#define EXTENT_Y 10e8 /* Hieght of displayed universe */



[EXAMPLE 1]

/*-----------------------------------------------
 Simulates the orbital dynamics of the Earth
 and Moon.
 Todd King
-------------------------------------------------*/
#include "simul.hpp"
#include "simulscr.hpp"

main() {
 BODY earth;
 BODY moon;
 UNIVERSE universe;

 earth.set_mass(5.98e24);
 earth.set_position(5.0e8, 5.0e8);
 earth.set_icon('E');
 moon.set_mass(7.36e22);
 moon.set_position(5.0e8, 8.8e8);
 moon.set_icon('M');
 moon.set_velocity(-1020.0, 0.0);

 universe.service(&earth);
 universe.service(&moon);
 universe.big_bang();
}

































































September, 1989
SETTING PRECEDENCE


Improved search strategies mean high performance




Mark C. Peterson


Mark Peterson is a contract computer programmer in the Hartford, Conn.-area.
He can be reached by calling 203-754-1162, or on CompuServe at 70441,3353.


Programmers now bask in the luxury of huge expanses of computer memory. To
take full advantage of the situation, they need high-speed algorithms capable
of managing random access to correspondingly huge amounts of data. One
solution involves the use of binary trees (B-trees) or some extension such as
B+ trees. Another solution that I prefer, the precedence tree (Ptree), is both
simple and elegant.
Traditional methods of B-tree construction tend to degenerate into linear
lists or other unbalanced states. There are routines for balancing a B-tree,
but they are often time-consuming, at times having to traverse through every
element in the tree. In addition, these routines tend to be code intensive.
One reason is the many different tree configurations possible for the same set
of numbers (see Figure 1). The final configuration is dependent upon the order
in which the numbers are entered. And as you'll soon see, inefficiencies are
created by this order of entry.
In addition to the Ptree, this article explores the use of pointer references
to create elegant code. The majority of code for the article is contained in
five listings. SPARRAY (see Listings One and Two) constructs and maintains a
sparse array utilizing a Ptree. DIAGARRA (see Listings Three and Four)
diagrams the Ptree of the sparse array and provides statistics. PTREE.C (see
Listing Five) inserts numbers into and deletes them from a Ptree and utilizes
DIAGARRA to observe the different configurations. All code is written in ANSI
standard C.


Precedence Trees


The Ptree builds an ordered B-tree in such a way that only one configuration
is possible for a given set of numbers. This is done by either placing numbers
in the middle of an existing tree or as a new leaf to create a tree. This
technique resolves inefficiencies due to order of entry and creates a
configuration that is always reasonably balanced. Furthermore, these trees
tend to become more balanced as more numbers are entered.
The entry of sequential lists of numbers, which creates a linear list using
traditional methods, instead creates a perfectly balanced B-tree using
techniques of precedence. Also, a Ptree uses only one chain of traversals from
the root to a given leaf for placement of a new number in the tree. Removal of
an old number is accomplished with a single pass through the Ptree along two
chains without recursion. Highspeed construction with a tendency toward
perfect balance means a lightning-fast algorithm.


Theory


One problem with ordered B-trees is the many different possible states for a
given set of numbers. The reason is that traditional binary-tree
implementations only place new numbers as additional leaves. This produces
configurations that vary depending upon the order in which the numbers are
entered (see Figure 1). Note that most of the configurations in Figure 1 are
not balanced. Therefore, if construction of the tree is left to chance, which
is the traditional method, the odds of ending up with an efficient tree are
not good.
The net result is that efficiency liabilities created by the order of entry
are additive and never resolved without balancing the tree. For example, if
the first numbers entered form a linear list, such as 0, 1, 2, and 3, all
successive searches and placements must first traverse this list (see Figure
2).
An ordered B-tree containing all possible numbers for a given number of bits
is referred to as a full B-tree. Only two configurations are perfectly
balanced for a full 4-bit B-tree. One configuration is shown in Figure 3; the
other is the trivial exchange of 1 and 0.
A close look at the bit patterns shows a convergence, meaning that a traversal
in the proper direction on a binary search results in a number with a bit
pattern more closely resembling the number being sought. In this case, this is
a leading convergence where the matching of bit patterns occurs from the most
significant bit to the least. B-trees configured with the matching bit pattern
occurring from the least significant bit to the most (a lagging convergence)
cannot be ordered. Diagraming the tree so that lower numbers are to the right
and higher numbers are down not only makes it easier to see this leading
convergence but is also easier to implement.
Note that the numbers closer to the root contain more factors of 2 than those
that are further away. This point is key to the Ptree algorithm. The Ptree
algorithm compares the number being placed with the number in the current
position, using the factors of 2 to determine which has precedence (that is,
which should be positioned closer to the root). The number with the greater
factors of 2 has higher precedence. If the two numbers initially have the same
factors of 2, they are bit-shifted to the right until one has more or becomes
0. By definition, 0If the two numbers initially have the same factors of 2,
they are bit-shifted to the right until one has more or becomes 0. has the
lowest precedence. As an example, Figure 4 compares the precedence of the
numbers 11 and 15. Because 11 has more factors of 2 (after bit shifting), it
has precedence over 15. Figure 4 also compares the numbers 12 and 28. Because
12 bit-shifts to 0 before 28 does, it has precedence over 28.
Figure 4: Comparing the numbers 11 and 15 to determine which has precedence

 11 = 1011, 15 = 1111 (each has no factors of 2)
 5 = 0101, 7 = 0111 (each has no factors of 2)
 2 = 1001, 15 = 0011 (2 has one factor, 3 has none)

 --------------------------------------------------------

 12 = 01100, 28 = 11100 (each has two factors of 2)
 6 = 00110, 14 = 01110 (each has one factor of 2)
 3 = 00011, 7 = 00111 (each has no factors of 2)
 1 = 00001, 3 = 00011 (each has no factors of 2)
 0 = 00000, 1 = 00001

This added dimension of precedence checking creates a B-tree that is "rigid"
(Figure 5) with only one possible configuration for a given set of numbers.
This configuration is always reasonably balanced.
When I say "reasonably balanced," I mean that the Ptree algorithm strikes a
compromise between perfection and practicality. Keeping a tree perfectly
balanced ensures the fastest search time but requires a great deal of trouble
to maintain. Ignoring balance invites rampant linear listing. The compromise
produces a reasonable balance.
There is some listing for certain sets of numbers that are separated by orders
of 2 -- for example the numbers 0, 1, 3, 5, 9, 17, 33, 65, and so on, form a
linear list. The maximum length of this linear list is dependent on the
largest number capable of being entered into the tree. That is, if the tree
can hold only 32-bit numbers, the maximum length of any linear list is 32.
This, however, is a very fragile state because entered numbers not conforming
to this pattern break down the list into a more balanced state.


Implementation


A quick algorithm for comparing the factors of 2 in a pair of numbers is to
XOR each number with itself minus 1 and compare the results (see Chk-PrecGT( )
in Listing Two). Subtracting 1 from a number turns all the trailing binary
0s(the factors of 2) to 1s and the least significant binary 1 to a 0, leaving
the rest of the binary digits untouched. When this result is exclusively-OREd
with the original number, all the unchanged leading binary digits become 0s,
leaving a result that is representative of the factors of 2.

The (x - 1) ^ x representative numbers shown in Figure 6 can then be compared
to see which x has more factors of 2. The expression 0- 1 is undefined because
the numbers involved are whole numbers (positive integers) and therefore the
calculated (x- 1) ^ x of 0 is 0 by definition.
The definition of a Ptree states that all numbers either greater or less than
a given number will have a lower precedence. This means that putting a number
in the Ptree usually requires placement somewhere in the middle of the tree
rather than at the end by adding another leaf. This requires that the Ptree be
"normalized" after insertions and deletions. To demonstrate, consider the
insertion of the value 64 into the Ptree in Figure 1. Because 64 contains five
factors of 2 and 108 (the root value) only contains two such factors, 64 has a
higher precedence and should therefore be the new root.
After linking 108 to the left side of 64, the tree is no longer ordered
because several numbers on the "greater" side of 64 are actually less (Figure
2). These lesser numbers must be "pruned" away from the left side of the tree
and relinked to the right side.
Traversals are made through the tree using PruneLower( ) and PruneHigher( ).
Because 108 was linked to the "higher" (left) side of 64, PruneLower( ) is
called first. If the new linkage had been made to the "lower" (right) side of
64, PruneHigher( ) would have been called first. PruneLower( ) traverses along
the lower linkages until it reaches a section that is less than the reference
value 64. PruneLower( ) breaks this linkage and makes a call to PruneHigher(
). PruneHigher( ) traverses along the higher linkages of this section of lower
numbers until it reaches a section greater than 64. It breaks that linkage and
makes a call to PruneLower( ). This process continues until a leaf is reached.
Prior to return from the recursion, the Ptree is divided into sections of
numbers higher and lower than 64. The return from the recursion relinks these
sections to their proper locations to create an ordered B-tree (see Figure 9).
As shown in Figure 10, the removal of 64 from the Ptree creates a hole that
must be filled. The removal effectively leaves two sets of numbers
corresponding to the left and right sides of the tree. In essence, there are
two subtrees: one with values greater than 64 and another with values less
than 64. The two sections need to be joined along an interface line, as shown
in Figure 11. A number on the interface line from the greater section is
compared with one on the interface line of the lower section and linked
according to precedence. Linkages unrelated to the interface line are
unaffected. The result is the original Ptree configuration (see Figure 7. The
use of pointer references allows this to be done as a single, iterative-style
pass through the Ptree without recursion.


Application


The SPARRAY structure (Listing Two) is a sparse array that handles elements up
to the limit of unsigned long. For most compilers this equates to an
addressing capability of 4,294,967,296 locations. SPARRAY uses a Ptree to
manage the elements in the sparse array.
MakeSpArray( ) and DeleteSpArray( ) are used to allocate memory dynamically.
MakeSpArray( ) returns a null pointer if there is insufficient memory, and
DeleteSpArray( ) sets the pointer referenced by RefPtr to NULL to prevent
further usage of the memory space after deletion.
FindSpElem( ) is a standard binary search routine that returns a null pointer
if the element is not found. The function reads:
Starting with the root, while this element is not a null pointer and is not
pointing to the desired location, continue looking either higher or lower
depending on whether this current location is greater or less than the desired
one. Return the "this" pointer (either NULL or the one sought).
PlaceElement( ) assumes that the element's location being placed is not
already in the array. The function first sets up a place for the "new"
location by checking to see if there are any empty elements (that is, elements
removed by RemoveSpElem( )). If there are none, the function chooses the next
highest element that has been used. If there is no more room in the array
(Array->HighestUsed== Array->Size), the function returns NULL.
The next section uses a binary search similar to FindSpElem( ) with two
exceptions: a check is made for precedence, and this is a pointer reference
rather than a pointer. The use of a pointer reference (as opposed to a
pointer) allows setting the new linkage based on previous events. In other
words, the linkage can be set "in context" rather than explicitly checking the
new linkage to see if it is a higher, lower, or a new root pointer. The
algorithm reads as follows:
First start with the address of the root. While "this" address is not
referring to a null pointer, check to see if the "Loc" being sought has
precedence over the referenced location. If so, then set either the "Higher"
or "Lower" linkage of "New" to the reference pointer and call the appropriate
pruning function based on whether "this" referenced location is greater or
less than the "New" location. If the referenced location has precedence over
the "New" location, then traverse either higher or lower as appropriate. Break
out of the loop when either a null pointer is referenced or a precedence check
returns true, and set whatever was referring to "this" displaced element
(either a "Higher" or "Lower" linkage pointer or the root) to the "New"
element. Return the "New" element pointer.
The pruning functions could be written iteratively setting the linkages on the
fly. Using several pointer references rather than saving them on the stack,
however, is a clumsy solution. The time spent dereferencing the various
pointer references far exceeds the time spent performing a recursion, and it
also takes up more code space.
RemoveSpElem( ) first sets the pointer reference this to the linkage pointing
to the element to be removed by starting at the root and performing a binary
search. The use of the pointer reference again allows for removal of either
higher, lower, or the root linkage without explicit code. If the element is
not found, a null pointer is returned; otherwise, the hole in the Ptree is
filled. There are, in this case, only four possible linkages:
1. A number from the higher interface can maintain its original linkage.
2. A number from the higher interface can be linked to a number from the lower
interface.
3. A number from the lower interface can maintain its original linkage.
4. A number from the lower interface can be linked to a number from the higher
interface.
Of these four cases, there are only two variables in contention: Higher
Inter->Lower and LowerInter->Higher.
To further illustrate the linkages, consider Figure 11. Here, Hole is set to
64, HigherInter to the number on the higher side of 64 (in this case, 108),
and LowerInter to the number on the lower side of 64 (46), and the reference
pointer to this is set to whatever was pointing to 64 (in this case,
Array->Root). The values 46 and 108 are compared, and because 108 has the
higher precedence, the pointer to 64 (Array->Root) is now set to 108, this is
set to the address of the lower side of 108 (82), and Higher is set to 82.
Then, 82 and 46 are compared, and because 82 has precedence, the pointer
referenced by this maintains the original linkage. In other words:
since: this = &HigherInter->Lower and: HigherInter = HigherInter->Lower

 then: *this = HigherInter means: HigherInter->Lower = HigherInter->Lower
Next, 73 is compared to 46, and because 46 has precedence, the pointer
referenced by this is set to 46. This links 46 to the lower side of 82. The
pointer reference this is now set to the address of LowerInter->Higher, and
LowerInter is set to LowerInter->Higher (62). The process continues until
either LowerInter or HigherInter is set to a null pointer and the final
linkage made accordingly. The last part of the routine links Hole into the
beginning of the chain of empty elements and updates the number of elements
used.
Listing Two contains routines for diagraming the Ptree and providing
statistics. Listing Three has the user program for manually entering and
deleting numbers from the Ptree and observing the configuration.


Winning Compromise


The Ptree algorithm rapidly constructs ordered B-tree configurations that are
always reasonably balanced by striking a winning compromise between perfection
and practicality. In this scenario, linear listing is virtually nonexistent.
Sequential entry, which results in a linear list using standard B-tree
construction methods, produces perfectly balanced trees using the Ptree
algorithm. Although the code in this article is written for a sparse array,
the Ptree algorithm can easily be applied to any application requiring an
ordered B-tree.


Availability


All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063; or call 800-356-2002 (from inside
Calif.) or 800-533-4372 (from outside Calif.). Please specify the issue number
and format (MS-DOS, Macintosh, Kaypro).

_SETTING PRECEDENCE_
by Mark C. Peterson


[LISTING ONE]

/* SPARRAY.H - Header file for SPARRAY.C */
/* Copyright (C) 1988, Mark C. Peterson */

#ifndef SPARRAY_H
#define SPARRAY_H

union VARIABLE {
 long Num;
 void *Ptr;
};


typedef struct _SPELEM {
 struct _SPELEM *Higher, *Lower;
 unsigned long Loc;
 union VARIABLE Element;
} SPELEM;

/* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
 * Loc The location of the element within the sparse array. *
 * Higher A pointer to a SPELEM with a higher "Loc" of a lower *
 * precedence. *
 * Lower A pointer to a SPELEM with a lower "Loc" of a lower *
 * precedence. *
 * Element A union VARIABLE for storage of either a "long" or "void*" *
 * element. "Element" is initialize to a NULL (either 0L or *
 * (void*)0) by MakeSpArray() or when an element is removed by *
 * RemoveSpElem(). *
 * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
*/

typedef struct _SPARRAY {
 SPELEM *SpElem, *Root, *EmptyElem;
 unsigned Size, ElemUsed, HighestUsed;
} SPARRAY;

/* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
 * SpElem An array of SPELEM structures. The number of SPELEM *
 * structures in the array is stored in "Size". *
 * Root A pointer to an SPELEM structure that is the root to the *
 * Ptree. *
 * EmptyElem A pointer to a chain of SPELEM structures (along the *
 * "Higher" pointer) that has been removed from the sparse *
 * array. *
 * Size The number of SPELEM structures in SpElem array. *
 * ElemUsed Keeps track of the number of elements in the sparse array. *
 * "ElemUsed" should be checked by the programmer periodically *
 * to see if the sparse array is full (i.e "ElemUsed" == *
 * "Size"). *
 * HighestUsed Used by PlaceElement(). If there are no empty elements *
 * ("EmptyElem" == NULL) and "HighestUsed" == "Size" the *
 * function returns a NULL. *
 * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
*/

/* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
 * Note that the only variable that should be changed directly by the *
 * programmer is the "Element" member in the SPELEM structure. *
 * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * */

SPARRAY *MakeSpArray(unsigned NumElem);
/* Creates the SPARRAY structure with "NumElem" number of elements,
 initializes all the elements to zero, and returns a pointer to the
 SPARRAY structure. */

SPELEM *FindSpElem(SPARRAY *Array, unsigned long Loc);
/* Returns a pointer to the SPELEM structure associated with "Loc".
 if "Loc" is not in the sparse array the function returns a NULL. */

SPELEM *PlaceSpElem(SPARRAY *Array, unsigned long Loc);
/* Places the "Loc" in the Ptree structure. Returns a NULL if the sparse
 array is full.

 WARNING: This function should only be called if FindElement() returns
 a NULL. Attempts to place a "Loc" that is already there will corrupt
 the Ptree. */

SPELEM *SpArray(SPARRAY *Array, unsigned long Loc);
/* Attempts to locate "Loc" using FindElement(). If not found the "Loc"
 is placed using PlaceElement(). Returns a NULL if "Loc" is not found
 and the sparse array is full. */

void ClrSpArray(SPARRAY *Array);
/* Removes all the elements from the sparse array. */

void DeleteSpArray(SPARRAY **ArrayRef);
/* Frees the memory allocated by MakeSpArray and sets the SPARRAY pointer
 referenced by ArrayRef to NULL. */

void RemoveSpElem(SPARRAY *Array, unsigned long Loc);
/* Removes "Loc" from the sparse array. */

#endif







[LISTING TWO]

/* SPARRAY.C - Routines for maintaining a sparse array using a Ptree */
/* Copyright (C) 1988, Mark C. Peterson */

#include <stdlib.h>
#include "SPARRAY.H"

static SPELEM
 *PruneLower(SPELEM *this, unsigned long Loc),
 *PruneHigher(SPELEM *this, unsigned long Loc);

static unsigned
 ChkPrecGT(unsigned long Loc, unsigned long ThisLoc);
/* Check "Loc" precedence greater than "ThisLoc" precedence. */

SPARRAY *MakeSpArray(unsigned NumElem) {
 SPARRAY *Array;

 if(Array = calloc(1, sizeof(SPARRAY))) {
 Array->Size = NumElem;
 if(!(Array->SpElem = (SPELEM*)calloc(NumElem, sizeof(SPELEM)))) {
 free(Array);
 Array = (SPARRAY*)0;
 }
 }
 return(Array);
}

static SPELEM *PruneLower(SPELEM *this, unsigned long Loc) {
 SPELEM *RetPtr;


 while(this->Lower && (this->Lower->Loc > Loc)) this = this->Lower;
 if(RetPtr = this->Lower) this->Lower = PruneHigher(this->Lower, Loc);
 return(RetPtr);
}

static SPELEM *PruneHigher(SPELEM *this, unsigned long Loc) {
 SPELEM *RetPtr;

 while(this->Higher && (this->Higher->Loc < Loc))
 this = this->Higher;
 if(RetPtr = this->Higher) this->Higher = PruneLower(this->Higher, Loc);
 return(RetPtr);
}

static unsigned ChkPrecGT(unsigned long Loc, unsigned long ThisLoc) {
 unsigned long LocPrec, ThisPrec;

 while(Loc && ThisLoc) {
 LocPrec = (Loc - 1) ^ Loc;
 ThisPrec = (ThisLoc - 1) ^ ThisLoc;
 if(LocPrec != ThisPrec) return(LocPrec > ThisPrec);
 Loc >>= 1;
 ThisLoc >>= 1;
 }
 return(Loc > ThisLoc);
}

SPELEM *PlaceSpElem(SPARRAY *Array, unsigned long Loc) {
 SPELEM **this, *New;

/* This section sets up a "New" SPELEM pointer */
 if(!Array->EmptyElem) {
 if(Array->HighestUsed < Array->Size)
 New = &Array->SpElem[Array->HighestUsed++];
 else return((SPELEM*)0);
 }
 else {
 New = Array->EmptyElem;
 Array->EmptyElem = Array->EmptyElem->Higher;
 }
 Array->ElemUsed++;
 New->Higher = New->Lower = (SPELEM*)0;
 New->Loc = Loc;

/* This section places the "New" SPELEM pointer into the Ptree */
 this = &Array->Root;
 while(*this) {
 if(ChkPrecGT(Loc, (*this)->Loc)) {
 if((*this)->Loc > Loc) {
 New->Higher = *this;
 New->Lower = PruneLower(*this, Loc);
 }
 else {
 New->Lower = *this;
 New->Higher = PruneHigher(*this, Loc);
 }
 break;
 }
 else this = (*this)->Loc > Loc ? &(*this)->Lower : &(*this)->Higher;

 }
 *this = New;
 return(*this);
}

void RemoveSpElem(SPARRAY *Array, unsigned long Loc) {
 SPELEM *Hole, *HigherInter, *LowerInter, **this;

 /* Find the element to be removed */
 this = &Array->Root;
 while(*this && ((*this)->Loc != Loc))
 this = (*this)->Loc > Loc ? &(*this)->Lower : &(*this)->Higher;

 if(Hole = *this) {
 /* Fill the hole in the Ptree if the element was found */
 LowerInter = Hole->Lower; /* Lower Interface Pointer */
 HigherInter = Hole->Higher; /* Higher Interface Pointer */
 while(LowerInter && HigherInter) {
 if(ChkPrecGT(LowerInter->Loc, HigherInter->Loc)) {
 *this = LowerInter;
 this = &LowerInter->Higher;
 LowerInter = LowerInter->Higher;
 }
 else {
 *this = HigherInter;
 this = &HigherInter->Lower;
 HigherInter = HigherInter->Lower;
 }
 }
 if(LowerInter) *this = LowerInter;
 else *this = HigherInter;

 /* Link the unused spot into the chain of empty elements along
 the "Higher" linkage. */
 Hole->Higher = Array->EmptyElem;
 Array->EmptyElem = Hole;
 Array->ElemUsed--; /* Update the number of elements used */
 }
}

SPELEM *FindSpElem(SPARRAY *Array, unsigned long Loc) {
 SPELEM *this;

 this = Array->Root;
 while(this && (this->Loc != Loc))
 this = this->Loc > Loc ? this->Lower : this->Higher;
 return(this);
}

SPELEM *SpArray(SPARRAY *Array, unsigned long Loc) {
 SPELEM *Found;

 Found = FindSpElem(Array, Loc);
 if(!Found)
 Found = PlaceSpElem(Array, Loc);
 return(Found);
}

void ClrSpArray(SPARRAY *Array) {

 unsigned n;

 Array->EmptyElem = Array->Root = (SPELEM*)0;
 for(n = 0; n < Array->Size; n++)
 Array->SpElem[n].Element.Num = 0L;
 Array->HighestUsed = Array->ElemUsed = 0;
}

void DeleteSpArray(SPARRAY **ArrayRef) {
 SPARRAY *Array;

 if(Array = *ArrayRef) {
 free(Array->SpElem);
 free(Array);
 *ArrayRef = (SPARRAY*)0;
 }
}







[LISTING THREE]


/* DIAGARRA.H - Header file for DIAGARRA.C */
/* Copyright (C) 1988, Mark C. Peterson */

#ifndef DIAGARRA_H
#define DIAGARRA_H

#include "SPARRAY.H"

extern unsigned MaxTrav, NumElem, NumLines;
extern unsigned long TotalTrav;
/* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
 * MaxTrav - Maximum number of traversals from the root to any *
 * number in the Ptree, set by SumSpArray(). *
 * NumElem - Number of elements in the Ptree, set by SumSpArray(). *
 * NumLines - Number of lines printed, set by DiagSpArray(). *
 * TotalTrav - Total number of linkages in the Ptree, set by *
 * SumSpArray(). *
 * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * */

void DiagSpArray(SPARRAY *this);
/* Diagram the Ptree used by the SPARRAY */

void DumpSpArray(SPELEM *this);
/* Dump an ordered listing of the elements of the SPARRAY */

void SumSpArray(SPARRAY *Array);
/* Determine various statistics about the Ptree used by the SPARRAY */

#endif







[LISTING FOUR]

/* DIAGARRA.C - Routines for diagraming the Ptree of a SPARRAY */
/* Copyright (C) 1988, Mark C. Peterson */

#include <string.h>
#include <stdlib.h>
#include <stdio.h>
#include "DIAGARRA.H"

static char String[8000];
unsigned MaxTrav, NumElem, NumTrav, StrLength, NumLines;
unsigned long TotalTrav;

static void RecDiagSpArray(SPELEM *this) {
 unsigned NumLength;

 NumLength = printf("%ld", this->Loc);
 if(this->Higher) strcat(String, " ");
 else strcat(String, " ");
 StrLength += 2;
 memset(String + StrLength, ' ', NumLength);
 StrLength += NumLength;
 String[StrLength] = '\0';
 if(this->Lower) {
 printf("--");
 RecDiagSpArray(this->Lower);
 }
 StrLength -= (NumLength + 2);
 if(this->Higher) {
 printf("\n%s\n", String);
 NumLines++;
 String[StrLength] = '\0';
 printf("%s", String);
 RecDiagSpArray(this->Higher);
 }
 else String[StrLength] = '\0';
}

void DiagSpArray(SPARRAY *Array) {
 String[0] = '\0';
 NumLines = StrLength = 0;
 if(Array->Root) RecDiagSpArray(Array->Root);
 else printf("<NULL>");
}

void DumpSpArray(SPELEM *this) {
 if(this->Lower) DumpSpArray(this->Lower);
 printf("%lu\t", this->Loc);
 if(this->Higher) DumpSpArray(this->Higher);
}

static void RecSumSpArray(SPELEM *this) {
 NumElem++;
 if(NumTrav > MaxTrav) MaxTrav = NumTrav;
 TotalTrav += (long)NumTrav;

 if(this->Lower) {
 NumTrav++;
 RecSumSpArray(this->Lower);
 NumTrav--;
 }
 if(this->Higher) {
 NumTrav++;
 RecSumSpArray(this->Higher);
 NumTrav--;
 }
}

void SumSpArray(SPARRAY *Array) {
 MaxTrav = NumElem = NumTrav = 0;
 TotalTrav = 0l;
 RecSumSpArray(Array->Root);
}






[LISTING FIVE]

/* PTREE.C - Program for constructing and diagraming a Ptree */
/* Copyright (C) 1988, Mark C. Peterson */

#include <stdio.h>
#include <stdlib.h>
#include "SPARRAY.H"
#include "DIAGARRA.H"

#define INPUT_SIZE 15
#define ARRAY_SIZE 100

char *InputNum(char *Input, unsigned size) {
 static char PromptStr[] = "\n\nNumber? ";

 printf(PromptStr);
 return(fgets(Input, size, stdin));
}

void main(void) {
 char Input[INPUT_SIZE];
 SPARRAY *Array;
 unsigned long Number;

 if(Array = MakeSpArray(ARRAY_SIZE)) {
 puts("Enter a number at the prompt. If it is not in the");
 puts("Ptree it will be added. If the number is in the");
 puts("Ptree it will be deleted. To end the program enter a");
 puts("blank line.\n");

 while(*InputNum(Input, INPUT_SIZE - 1) != '\n') {
 putchar('\n');
 Number = atol(Input);
 if(FindSpElem(Array, Number)) RemoveSpElem(Array, Number);
 else {

 if(!PlaceSpElem(Array, Number)) {
 puts("Array Full");
 break;
 }
 }
 DiagSpArray(Array);
 }
 DeleteSpArray(&Array);
 }
 else puts("Error making SpArray");
}










</entry>








































September, 1989
ROLL YOUR OWN MINILANGUAGES WITH MINI-INTERPRETERS


Customizing assembler code for speed and readability




Michael Abrash and Dan Illowsky


Michael Abrash and Dan Illowsky are responsible for new products development
at Orion Instruments, a Redwood City-based manufacturer of innovative PC-based
engineering and instruments. They can be reached at 702 Marshall St., Ste.
#420, Redwood City, CA 94063.


When you sit down to program, do you ever think about how nice it would be to
have a language written just for your particular application? Although C,
Pascal, assembler, and the like are undeniably powerful, they're
general-purpose languages; as a result, you spend most of your time matching
the general-purpose constructs of those languages to the needs of your
particular applications. What if instead you had languages with commands such
as "Draw centered text at bottom of screen," "Beep speaker three times and
wait for key," or "Animate image left for specified distance?" In that case,
simply put, you could concentrate on the functionality rather than the
implementation of your applications -- and your code would be one heck of a
lot shorter, too.
Although customized languages sound farfetched, in truth they're not, at least
not on a small scale. In this article we'll look at mini-interpreters, which
let you define small languages designed for specific tasks. Mini-interpreters
are easy to create, make for extremely compact programs, are very flexible,
and are easily maintained and modified.
We'll start by defining what a mini-interpreter is. After that we'll explore
the pros and cons of mini-interpreters, and we'll finish up by looking at a
fully functional mini-interpreter. The sample mini-interpreter can draw text
and lines and perform text-mode animation, all in less than 700 bytes.


The Flexibility of Assembler-Defined Data


Before you can understand anything else about mini-interpreters, you must
understand that the data-definition capabilities of assembler are vastly
superior to those of other languages. Only in assembler can you readily create
data sequences that consist of arbitrarily intermixed 8-bit unsigned values,
16-bit signed values, 16-bit pointers, 32-bit pointers, and even 64-bit
floating-point values.
As an example, the following code defines the contents of memory starting at
label AsmData to be a mix of 8-bit signed values, 16-bit pointers, and text:
AsmData label byte
 db SetXY,79,20
 db SubProg
 dw Box4x4$
 db TextUp,'Enter your name',O
 db Done
It's important to understand that AsmData in this example isn't just some sort
of structure or variant record. The following would be an equally valid
definition of AsmData:
AsmData label byte
 db SubProg
 dw SpinAround$
 db TextUp,'Ten seconds until exit...',O
 db Wait,10
 db Cls
 db TextUp,'Exited.',O
 db Done
AsmData is simply a free-form mix that can contain, in any order, signed and
unsigned values of various sizes and near and far pointers. No regularity,
repetition, or structure is required, for the point of using assembler-defined
data with mini-interpreters is to be able to mix data in any way,
unconstrained by the limitations of high-level-language data structures. This
makes possible both compact data encoding and complete flexibility in data
definition.
Try doing that in C!


How Mini-Interpreters Work


At heart, a mini-interpreter is nothing more than a table-lookup loop driven
by a jump table and a free-form sequence of data. The interpreter reads the
next function number from the data sequence, advances the data sequence
pointer, and calls the jump table entry corresponding to the function number
read, as shown in Figure 1. The function called via the jump table may
optionally read any number of parameters of any type from the data sequence,
advancing the data pointer each time it does so and preserving the data
pointer in any case. When the function ends and returns to the
mini-interpreter, the next number in the data sequence indicates the next
function to be executed. The whole process is repeated until the function that
terminates the interpreter is reached.
The key to the operation of a mini-interpreter is that each function knows the
number and type of its own parameters, so each function is responsible for
reading data from the data sequence and advancing the data sequence pointer
properly. The mini-interpreter itself needs to know nothing more than how to
read the next function number from the data sequence and call the
corresponding function.
In Figure 2, which shows an annotated sample data sequence, note that the
mini-interpreter reads only the function number bytes, calling the
corresponding function immediately after doing so. Each function is
responsible for obtaining and handling its own parameters, thereby leaving the
data sequence pointer pointing to the function number for the next function.
The basic operation of a mini-interpreter is simple, then: A data sequence, or
"miniprogram," provides function numbers, which are used to vector through a
jump table to functions. The function numbers are basically commands in the
"minilanguage" defined by the functions in the jump table. The functions then
acquire their own parameters from the data sequence as needed.
Another way to view a mini-interpreter is as a control program that allows you
to call various functions in any order and with any parameters. The jump table
defines the functions that can be called -- in effect defining a minilanguage
-- and the data sequence defines the calling order and the parameters, thereby
serving as a miniprogram. The same result could, of course, be accomplished
simply by writing code that calls the desired routine with the desired
parameters. The great advantage of using a mini-interpreter over writing
equivalent code is that a mini-interpreter makes for more compact code that's
also easier to write and maintain.
Now, consider this: Pointers to the jump table and/or the data sequence can be
parameters passed to the interpreter, so the operation of the interpreter can
be changed instantly. By passing in a pointer to a different data sequence,
the functions in the jump table can be combined in different orders and with
different parameters; in other words, a different miniprogram can be run. By
passing in a pointer to a different jump table, the very minilanguage that the
mini-interpreter supports can be altered.
In other words, not only the m0iniprogram that the mini-interpreter is running
but also the minilanguage that it supports can easily be changed, even in
mid-program -- the ultimate in flexibility.


Benefits of Mini-Interpreters



The benefits of mini-interpreters are many and varied, with flexibility,
compact code, and ease of use being high on the list. Let's look at these
benefits in more detail.
As noted earlier, a mini-interpreter is extremely flexible because it consists
of nothing more than a function-vectoring loop that's driven by a jump table
that defines a minilanguage and a data sequence that defines a miniprogram. A
different miniprogram can be executed by running a different data sequence
through the mini-interpreter. The function set and/or operation of the
functions that make up a minilanguage can be changed at any time simply by
altering or replacing the jump table currently driving the mini-interpreter.
By the same token, the minilanguage supported by a mini-interpreter can be
extended simply by adding additional functions to the jump table.
Not only can a mini-interpreter switch from one minilanguage or miniprogram to
another, but it can also nest minilanguages and miniprograms. A minilanguage
command can easily save the current data sequence pointer and recursively
start the mini-interpreter with a new miniprogram, then restore the original
data pointer when the new miniprogram finishes. In effect, this allows
mini-interpreters to support subroutines. Similarly, a minilanguage command
can save the current jump table pointer and start the mini-interpreter with a
new jump table (and a new miniprogram as well, if desired), thereby
temporarily switching to another minilanguage altogether.
There really aren't any limitations on the types of commands minilanguages can
support. The only rule is that the code that implements any given minilanguage
command must preserve the data sequence pointer (the pointer to the current
miniprogram), advancing it past any parameters the command uses.
The flexibility of minilanguage function definition leads directly to the next
benefit of mini-interpreters, which is compact code. Functions are subject to
virtually no limitations, so it's easy to tailor them to perform precisely the
tasks that a given application demands, with no wasted code. This means that
miniprograms can be very efficient because the commands available in the
minilanguage are matched to the task at hand.
Then, too, the function numbers in a minilanguage can easily be encoded in a
single byte. This means that all the commands in a minilanguage can be 1 byte
long, a claim that not even assembler instructions can make. Again, this is
made possible by the narrow focus of a minilanguage. Only the functions needed
for a specific task are implemented in a minilanguage, in contrast to
general-purpose languages, which must support a wide range of general-purpose
programming constructs.
The ability to support 1-byte commands is one area in which mini-interpreters
are superior to code that simply calls the desired functions one after
another. A function call, as used in C or assembler code, takes a minimum of 3
bytes, in contrast to the 1-byte command encoding used by mini-interpreters.
Another area in which mini-interpreters are superior to code that calls
functions is that of passing parameters. Mini-interpreters are extremely
compact because there's no overhead involved in passing parameters. A C
compiler would (at best) generate the following 8-byte code sequence in order
to pass a pointer to the text string HelloMsg as a parameter to the function
TextUp%:
 mov ax,offset HelloMsg
 push ax ;pass the pointer as a parameter
 call TextUp%
 pop ax ;clear the parameter from the stack
Assembler code could be smaller -- but still bulky -- at 6 bytes:
 mov ax,offset HelloMsg
 call TextUp%
In a miniprogram, however, a mere byte of miniprogram code would suffice, with
the text built right into the miniprogram:
 db 15,'Hello',O
Here, 15 is the entry number of TextUp% in the jump table that defines the
current minilanguage. The last example could be made considerably more
readable as follows:
 TextUp equ 15

 db TextUp,'Hello',O
We will return to the topic of making miniprograms readable shortly.
Mini-interpreters lend themselves to compact code in every respect. The code
in the functions that implement a minilanguage tends to be reused heavily
because those functions make up the commands available in the minilanguage.
Programs written in the minilanguage can readily be reused in the form of
subprograms, which are essentially subroutines, particularly as subprograms
can be nested. Any command or subprogram can easily be repeated any number of
times simply by defining a function in the minilanguage that starts a nested
mini-interpreter the desired number of times.
The code of a mini-interpreter itself can be extremely small and usually is.
After all, mini-interpreters don't actually do much; they're just the glue
that lets a miniprogram call the functions that make up a minilanguage, as
defined by the jump table. Here's the entire mini-interpreter from Listing One
which we'll discuss later:
 cld
GetNextCommand:
 lodsb
 mov bl,al
 xor bh,bh
 shl bx, 1
 call [bx+Function_Table]
 jmp short GetNextCommand
The functions that make up a minilanguage do take up code space, of course,
but then you'd have to write that code anyway in order to accomplish the
desired task. One of the main points of using a mini-interpreter is to let you
sequence those functions and pass those parameters as efficiently as possible.
By the way, although we're only going to discuss assembler mini-interpreters
in this article, don't think that mini-interpreters can't be useful in the
context of high-level-language programs. For one thing, mini-interpreters can
easily be called from high-level-language programs to carry out specific tasks
at the cost of far fewer bytes than the high-level language could manage.
What's more, mini-interpreters can even be implemented in high-level
languages, albeit not quite so efficiently as in assembler. All that's
required to write a mini-interpreter is the availability of pointers and the
ability to support jump tables, requirements that C meets admirably. One
caveat regarding high-level-language mini-interpreters, though: Always create
your miniprograms in assembler, even when your mini-interpreters, jump tables,
and minilanguage functions are all written in high-level languages. As we saw
earlier, high-level languages can't come close to matching assembler where
flexible data definition is concerned -- and flexible data definition is
absolutely essential when you want to create the most compact and powerful
miniprograms possible.


Ease of Creation and Maintenance


We've yet to cover one characteristic of mini-interpreters, and that's ease of
use. You've already seen some of the reasons why mini-interpreters are easy to
use: The capabilities of minilanguages are matched to the task at hand, so the
available commands are intuitive and fit in without any fuss, and
minilanguages are table-driven, so it's easy to add new commands as the need
arises. Still and all, at this moment miniprograms might not seem particularly
easy to write and maintain, but that's because you haven't yet seen the last
piece of the puzzle.
That last piece is the translation of function numbers in miniprograms from
numbers to symbols by way of either the equ directive or the macro directive.
Assume, for example, that the function SetXY% is the second entry in the jump
table for a minilanguage. The miniprogram
 db 2,79,20
isn't particularly easy to write or read -- in fact, it's pretty much
incomprehensible. On the other hand, the equivalent miniprogram
 SetXY equ 2
 :
 db SetXY,79,20
is easy to write and perfectly readable. Basically, equated symbols are used
with mini-interpreters in much the same way that mnemonics are used with
assemblers: To allow programmers to work with human-oriented symbols rather
than numbers.
If you don't mind losing some assembly speed, you can use macros to make
miniprograms still easier to use. Macros can be used both to check the number
and type of parameters and to make miniprograms more readable -- for example,
the last example can be implemented as follows:
 SetXY macro X,Y,EfforCheck
 ifnb <ErrorCheck>
 %out Too many parameters to SetXY
 .err
 endif
 ifb <Y>
 %out Too few parameters to SetXY
 .err
 endif

 db 2,X,Y
 endm
 :
 SetXY 79,20
The macro SetXY not only defines the appropriate data for the SetXY command
but also checks to make sure that there are exactly two parameters. If
necessary, the if directive can even be used to check the magnitude and/or
types of the parameters. Once the macro is defined, the actual code of the
miniprogram -- SetXY 79, 20 -- is intuitive and easy to read or write. A
program written with such macros would look something like
 SetXY 0,0
 SetXYTextDirection Down
 TextUp 'START',
 End0fText
which is certainly straightforward enough.
We're not going to use macros in the example program, both because macros take
more source code space and because they tend to obscure the basic operation of
the mini-interpreter, which is what we're exploring right now. Macros are,
however, the best way to go if you need to implement long miniprograms and/or
a minilanguage with many commands because parameter error checking can help
avoid bugs and the improved readability of macro-based miniprograms can help
you find your way around your code.


Limitations of Mini-Interpreters


You might well think that the primary limitation of mini-interpreters is speed
-- but you'd be wrong. Interpreters are normally slow, but mini-interpreters
aren't normal interpreters in that, unlike most interpreters,
mini-interpreters don't have to do any parsing. Miniprograms are already
parsed because they consist only of function number bytes and function
parameters already in the form -- binary, text, what have you -- that each
function expects. The only overhead incurred by the mini-interpreter is
reading each function number from the miniprogram and branching to the
corresponding entry in the jump table, and that just doesn't take long.
Besides, when you create the functions that make up a minilanguage, you can
pack as much functionality as you want into any one function. If there's
something that just has to be done as fast as possible, you can create a
function that does that task from start to finish as a single mini-interpreter
command.
No, speed isn't the major limitation of the mini-interpreter approach; that
dubious honor falls instead to decision making. Mini-interpreters can branch
in limited ways -- for example, by executing a subprogram or repeating a
command multiple times. Mini-interpreters, however, don't lend themselves
especially well to the more general sorts of conditional branching and code
structures that are needed for decision-making code.
A command that conditionally branches to another miniprogram location is
possible, but such a command couldn't easily handle generalized condition
testing with relational, logical, and arithmetic operations and would surely
lead to cryptic spaghetti code. Code structures such as if. . . then. . .
else, for, and do. . . while aren't impossible, but they certainly wouldn't be
easy to implement.
Even if they were easy to implement, however, complex control structures are
contrary to the reason for using mini-interpreters in the first place, which
is efficient implementation of well-defined tasks. If you're going to bother
with general-purpose control structures, expression evaluation, and the like,
you might as well use a general-purpose language -- that's what those
languages are designed for. Mini-interpreters work best when you need to
perform tasks that can be expressed as a sequence of parameterized actions.
Don't confuse decision making with complexity -- mini-interpreters are
excellent for many sorts of complex tasks. Mini-interpreters save
proportionally more space when used for lengthy tasks, and the ease of writing
and reading miniprograms matters most when the task is complex.


Applications


So exactly what sort of complex tasks are mini-interpreters suited for? They
are suited for complex sound generation, for one, because a miniprogram that
could tweak the speaker in various ways and for various periods of time would
be far easier to write than, say, assembler code that did the same with a
series of out instructions and timer reads. Manipulation of structured data,
for another; a minilanguage could readily be built to control insertion,
deletion, and modification of records in a database and fields in those
records, for example. Parsing text is yet another example, for
mini-interpreters lend themselves well to tasks that can be expressed as state
machines.
We've saved the most obvious mini-interpreter applications, screen control and
animation, for last. Next, we're going to implement a sample mini-interpreter
designed for precisely those applications.


A Sample Mini-Interpreter


Listing One shows a fully functional mini-interpreter in action. The sample
miniprogram run by this mini-interpreter does quite a bit: It clears the
screen, displays the text START and END, draws a complex maze, and animates
the movement of an arrow through that maze.
Even though all screen output in Listing One is done through BIOS functions,
the entire screen is drawn instantaneously on a PC AT and just a bit more
slowly on a PC. In fact, the animation actually must be slowed down
considerably by way of the DELAY_COUNT value, so you can see that this
num-interpreter provides better than adequate performance. What's more,
Listing One assembles to a program just 684 bytes long, with some of those
bytes taken by functions that aren't even used in the sample miniprogram.
Better yet, the sample miniprogram itself, which starts at DemoScreen$ and
ends at SpinAround$, is just 302 bytes long in its entirety.
Mini-interpreters do indeed make for compact programs.
Listing One illustrates many of the desirable features of mini-interpreters.
The main miniprogram starting at DemoScreen$ uses subprograms (started with
the SubProg command) to make the program still more compact and modular. For
example, SpinAround$ is used twice as a subprogram to cause the arrow to spin
around a square two characters on a side. If instead we wanted the arrow to
spin around a square one character on a side, all we'd need to do is change 2
to 1 in SpinAround$, and the animation of the arrow spinning would be changed
through out the program.
It's also interesting to note that DemoScreen$ uses the minilanguage's DoRep
command. In fact, it's DoRep that's used to repeat SpinAround$ five times in
order to make the arrow spin at the beginning and end of the maze, so you can
see that this particular minilanguage supports repetition of subprograms.
It's hard to overstate the flexibility of mini-interpreters such as the one in
Listing One. Because the entire drawing and animation sequence can be run by
interpreting DemoScreen$, the whole demo could be repeated simply by executing
DemoScreen$ as a subprogram, with DoRep repeating everything the desired
number of times. If the interpreter were to be started with the following
miniprogram:
 db DoRep,3,SubProg
 dw DemoScreen
 db Done
then the entire demo would be repeated three times -- at a cost of just 6
extra bytes.
The miniprogram in Listing One is easy to read, too. The equated names for the
various commands are clear, are documented where they're defined with equ, and
could be made clearer, if necessary, by lengthening them. As we saw earlier,
the minilanguage commands could be implemented as macros if either still
greater clarity or parameter checking were needed.
Relatively few commands were required to support the functionality required by
this application, so we chose to implement just one minilanguage, and to
hard-wire the interpreter Interp for that one minilanguage by directly
addressing the minilanguage's jump table Function_Table in Interp. If we had
wanted to, however, we could easily have passed the address of the jump table
into Interp, thereby allowing Interp to support whatever minilanguage the
calling routine chooses, just as it already supports whatever miniprogram the
calling routine passes in. Had we done that, we could just as easily have used
three minilanguages as one: One minilanguage for text drawing, one for maze
drawing, and one for animation.
By the way, Interp can be called recursively; it is in fact called recursively
from the SubProg% function, which is invoked with the SubProg miniprogram
command. All Interp ever looks at is the current byte in the miniprogram,
which is pointed to by si, so starting a subprogram is a simple matter of
calling Interp with a new pointer in si. SubProg% pushes the miniprogram
pointer -- the pointer to the miniprogram containing the SubProg command --
before calling Interp with a pointer to a subprogram in si. That means that
when the subprogram has finished and Interp returns to SubProg%, SubProg% can
simply pop si and return to Interp in order to continue execution of the
original miniprogram with the command following SubProg.
Function_Table and the functions following Interp in Listing One completely
define the minilanguage used in this program and provide all the capabilities
available to the miniprogram DemoScreen$. Additional miniprogram code that
needed commands not available in Function_Table could be supported by writing
the needed functions and adding an entry for each to Function_Table.
Additional miniprogram code that could make do with just the functions already
in Function_Table would be extremely compact because it could be implemented
as nothing more than minilanguage commands, without the need for a single byte
of new assembler code.
As a final note, take a look at how the functions following Interp obtain
their parameters. Each function gets its own parameters, if any, directly from
the miniprogram data pointed to by si, advancing si so that it points to the
function number for the next function. Each function is free to obtain
parameters in the most efficient possible way; for example, SetXY% loads both
the X and Y coordinates from the miniprogram data sequence with a single
lodsw.


Conclusion


Mini-interpreters provide a superb way to implement many sorts of well-defined
tasks in a minimum of space. Mini-interpreter-based miniprograms are easy to
create and maintain, and are extremely flexible. Think of mini-interpreters
whenever program size is a consideration; even when space is not an issue, you
may want to take advantage of their ease of use.


Availability



All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063; or call 800-356-2002 (from inside
Calif.) or 800-533-4372 (from outside Calif.). Please specify the issue number
and format (MS-DOS, Macintosh, Kaypro).


_ROLL YOUR OWN MINILANGUAGES WITH MINI-INTERPRETERS_
by Michael Abrash and Dan Illowsky



[LISTING ONE]

; This program demonstrates the use of a mini-interpreter to produce
; code that is compact, flexible and easy to modify. The mini-
; program draws and labels a maze and animates an arrow through
; the maze.
;
; Note: This program must be run in 80-column text mode.
;
; Tested with TASM 1.0 and MASM 5.0.
;
; By Dan Illowsky & Michael Abrash 2/18/89
; Public Domain
;
Stak segment para stack 'stack' ;allocate stack space
 db 200h dup (?)
Stak ends
;
_TEXT segment para public 'code'
 assume cs:_TEXT, ds:_TEXT
;
; Overall animation delay. Selected for an AT: set higher to slow
; animation more for faster computers, lower to slow animation less
; for slower computers.
;
DELAY_COUNT equ 30000
;
; Equates for mini-language commands, used in the data
; sequences that define mini-programs. The values of these
; equates are used by Interp as indexes into the jump table
; Function_Table in order to call the corresponding subroutines.
;
; Lines starting with ">>" describe the parameters that must
; follow the various commands.
;
Done equ 0 ;Ends program or subprogram.
 ;>>No parms.
SubProg equ 1 ;Executes a subprogram.
 ;>>Parm is offset of subprogram.
SetXY equ 2 ;Sets the cursor location (the location at
 ; which to output the next character).
 ;>>Parms are X then Y coordinates (both
 ; bytes).
SetXYInc equ 3 ;Sets the distance to move after displaying
 ; each character.
 ;>>Parms are X then Y amount to move after
 ; displaying character (both bytes).
SetX equ 4 ;Sets the X part of the cursor location.
 ;>>Parm is the X coordinate (byte).
SetY equ 5 ;Sets the Y part of the cursor location.

 ;>>Parm is the Y coordinate (byte).
SetXInc equ 6 ;Sets the X part of the amount to move after
 ; displaying each character.
 ;>>Parm is the X amount to move after
 ; character is displayed (byte).
SetYInc equ 7 ;Sets the Y part of the amount to move after
 ; displaying each character.
 ;>>Parm is the Y amount to move after
 ; character is displayed (byte).
SetAtt equ 8 ;Sets the screen attribute of characters to
 ; be displayed.
 ;>>Parm is attribute (byte).
TextUp equ 9 ;Displays a string on the screen.
 ;>>Parm is an ASCII string of bytes,
 ; which must be terminated by an EndO byte.
RepChar equ 10 ;Displays a single character on the screen
 ; a number of times.
 ;>>Parms are char to be displayed followed
 ; by byte count of times to output byte.
Cls equ 11 ;Clears screen and makes text cursor
 ; invisible.
 ;>>No parms.
SetMStart equ 12 ;Sets location of maze start.
 ;>>Parms are X then Y coords (both bytes).
Mup equ 13 ;Draws maze wall upwards.
 ;>>Parm is byte length to draw in characters.
Mrt equ 14 ;Draws maze wall right.
 ;>>Parm is byte length to draw in characters.
Mdn equ 15 ;Draws maze wall downwards.
 ;>>Parm is byte length to draw in characters.
Mlt equ 16 ;Draws maze wall left.
 ;>>Parm is byte length to draw in characters.
SetAStart equ 17 ;Sets arrow starting location.
 ;>>Parms are X then Y coordinates
 ; (both bytes).
Aup equ 18 ;Animates arrow going up.
 ;>>No parms.
Art equ 19 ;Animates arrow going right.
 ;>>No parms.
Adn equ 20 ;Animates arrow going down.
 ;>>No parms.
Alt equ 21 ;Animates arrow going left.
 ;>>No parms.
DoRep equ 22 ;Repeats the command that follows
 ; a specified number of times.
 ;>>Parm is repetition count (one byte).
;
EndO equ 0 ;used to indicate the end of a
 ; string of text in a TextUp
 ; command.
;********************************************************************
; The sequences of bytes and words between this line and the next
; line of stars are the entire mini-program that our interpreter will
; execute. This mini-program will initialize the screen, put text on
; the screen, draw a maze, and animate an arrow through the maze.
;
DemoScreen$ label byte ;this is the main mini-program that our
 ; interpreter will execute
; Initialize the screen

 db SubProg
 dw InitScreen$
; Put up words
 db SetXY,0,0, SetXYInc,0,1, TextUp,'START',EndO
 db SetXY,79,20, TextUp,'END',EndO
; Draw the maze
 db SetMstart,4,0, Mrt,8, Mdn,4, Mrt,4, Mup,3, Mrt,4, Mdn,3
 db Mrt,4, Mdn,8, Mrt,3, Mup,3, Mrt,5, Mup,9, Mrt,17, Mdn,9
 db Mrt,5, Mdn,3, Mrt,4, Mup,10, Mrt,12, Mdn,18, Mrt,6
 db SetXY,4,2, Mrt,4, Mdn,2, Mlt,4, Mdn,18, Mrt,12, Mup,4
 db Mrt,4, Mdn,4, Mrt,11, Mup,11, Mrt,5, Mup,9, Mrt,9, Mdn,9
 db Mrt,5, Mdn,11, Mrt,12, Mup,4, Mrt,4, Mdn,4, Mrt,10
 db SetXY,8,6, SubProg
 dw Box4x6$
 db SetXY,8,14, SubProg
 dw Box4x6$
 db SetXY,24,14, SubProg
 dw Box4x6$
 db SetXY,54,14, SubProg
 dw Box4x6$
 db SetXY,62,4, SubProg
 dw Box4x6$
 db SetXY,16,6, SubProg
 dw Box4x4$
 db SetXY,16,12, SubProg
 dw Box4x4$
 db SetXY,62,12, SubProg
 dw Box4x4$
; Animate the arrow through the maze.
 db SetAStart,3,0, Alt,2, Adn,2, Art,2, Aup,2
 db SetXY,0,0
 db DoRep,5,SubProg
 dw SpinAround$
 db Alt,2, Adn,1, Art,9, Adn,4, Alt,4, Adn,8, Art,8, Adn,8
 db Alt,8, Aup,8, Art,8, Aup,2, Art,8, Adn,2, Art,7, Aup,3
 db Art,5, Aup,9, Art,13, Adn,9, Art,5, Adn,11, Art,8, Aup,10
 db Art,8, Aup,8, Alt,8, Adn,8, Art,8, Adn,10, Art,8, Adn,1
 db Art,2, Aup,2, DoRep,5,SubProg
 dw SpinAround$
 db Alt,2, Adn,1, Art,1
 db Done
; Subprogram to clear the screen and initialize drawing variables.
InitScreen$ db SetXY,0,0, SetAtt,7, SetXYInc,1,0, Cls, Done
; Subprograms to draw boxes.
Box4x4$ db Mrt,4, Mdn,4, Mlt,4, Mup,4, Mrt,2, Done
Box4x6$ db Mrt,4, Mdn,6, Mlt,4, Mup,6, Mrt,2, Done
; Subprogram to spin the arrow around a square.
SpinAround$ db Alt,2, Adn,2, Art,2, Aup,2, Done
;********************************************************************
; Data for outputting text characters to the screen.
Text_Out_Data label byte
Cursor_X_Coordinate db 0
Cursor_Y_Coordinate db 0
Cursor_X_Increment db 1
Cursor_Y_increment db 0
Character_Attribute db 7
Last_Maze_Direction db 0ffh ;0-up, 1-rt, 2-dn, 3-lt
 ; 0ffh-starting
AnimateLastCoordinates dw 0 ;low byte is X, high byte is Y

;
; Jump table used by Interp to call the subroutines associated
; with the various function numbers equated above. The functions
; called through this jump table constitute the mini-language
; used in this program.
;
Function_Table label word ;list of function addresses
 dw Done% ; which correspond one for
 dw SubProg% ; one with the commands defined
 dw SetXY% ; with EQU above
 dw SetXYInc%
 dw Set% ;Set%, MOut%, and Animate% all use
 dw Set% ; the function number to determine
 dw Set% ; which byte to set or which
 dw Set% ; direction is called for
 dw Set%
 dw TextUp%
 dw RepChar%
 dw Cls%
 dw SetMStart%
 dw MOut%
 dw MOut%
 dw MOut%
 dw MOut%
 dw SetAStart%
 dw Animate%
 dw Animate%
 dw Animate%
 dw Animate%
 dw DoRep%
;
; Program start point.
;
Start proc far
 push cs ;code and data segments are the
 pop ds ; same for this program
 mov si,offset DemoScreen$ ;point to mini-program
 call Interp ;execute it
 mov ah,1 ;wait for a key before clearing the
 int 21h ; the screen and ending
 mov ah,15 ;get the current screen mode
 int 10h ; so it can be set to force
 sub ah,ah ; the screen to clear and the
 int 10h ; cursor to reset
 mov ah,4ch
 int 21h ;end the program
Start endp
;
; Mini-interpreter main loop and dispatcher. Gets the next
; command and calls the associated function.
;
Interp proc near
 cld
GetNextCommand:
 lodsb ;get the next command
 mov bl,al
 xor bh,bh ;convert to a word in BX
 shl bx,1 ;*2 for word lookup
 call [bx+Function_Table] ;call the corresponding

 ; function
 jmp short GetNextCommand ;do the next command
;
; The remainder of the listing consists of functions that
; implement the commands supported by the mini-interpreter.
;
; Ends execution of mini-program and returns to code that
; called Interp.
;
Done%:
 pop ax ;don't return to Interp
 ret ;done interpreting mini-program or subprogram
 ; so return to code that called Interp
;
; Executes a subprogram.
;
SubProg%:
 lodsw ;get the address of the subprogram
 push si ;save pointer to where to
 ; resume the present program
 mov si,ax ;address of subprogram
 call Interp ;call interpreter recursively
 ; to execute the subprogram
 pop si ;restore pointer and resume
 ret ; the program
;
; Sets the screen coordinates at which text will be drawn.
;
SetXY%:
 lodsw
 mov word ptr [Cursor_X_Coordinate],ax
 ret
;
; Sets the amount by which the cursor will move after each
; character is output to the screen.
;
SetXYInc%:
 lodsw
 mov word ptr [Cursor_X_Increment],ax
 ret
;
; Sets individual X coordinate, Y coordinate, X movement after
; character is output to the screen, Y movement, or character
; attribute depending on function number.
;
Set%:
 shr bx,1 ;calculate the command number
 lodsb ; get the new value
 mov [bx+Text_Out_Data-SetX],al ;store in location
 ; corresponding to
 ; the command number
Return:
 ret
;
; Displays a string of text on the screen.
;
TextUp%:
GetNextCharacter:
 lodsb ;get next text character

 or al,al ;see if end of string
 je Return ;if so, next command
 call OutputCharacter ;else output character
 jmp short GetNextCharacter ;next character
;
; Displays a single character on the screen multiple times.
;
RepChar%:
 lodsw ;get the character in AL
 ; and the count in AH
RepCharLoop:
 push ax ;save the character and count
 call OutputCharacter ;output it once
 pop ax ;restore count and character
 dec ah ;decrement count
 jne RepCharLoop ;jump if count not now 0
 ret
;
; Clears the screen and turns off the cursor.
;
Cls%:
 mov ax,600h ;BIOS clear screen parameters
 mov bh,[Character_Attribute]
 xor cx,cx
 mov dx,184fh
 int 10h ;clear the screen
 mov ah,01 ;turn off cursor
 mov cx,2000h ; by setting bit 5 of the
 int 10h ; cursor start parameter
 ret
;
; Sets the start coordinates for maze-drawing.
;
SetMStart%:
 lodsw ;get both X and Y coordinates and store
 mov word ptr [Cursor_X_coordinate],ax
 mov [Last_Maze_Direction],0ffh ;indicate no
 ret ; last direction

;
; Maze-drawing tables.
;
XYincTable db 0,-1, 1,0, 0,1, -1,0
 ;X & Y increment pairs for the 4 directions
CharacterGivenDirectionTable db 179,196,179,196
 ;vertical or horizontal line character to use
 ; for a given direction
FirstCharGivenNewAndOldDirectionTable label byte
 db 179,218,179,191, 217,196,191,196 ;table of corner
 db 179,192,179,217, 192,196,218,196 ; characters
;
; Outputs a maze line to the screen.
;
MOut%:
 sub bx,Mup+Mup ;find new direction word index
 mov ax,word ptr [bx+XYincTable] ;set for new
 mov word ptr [Cursor_X_Increment],ax ; direction
 shr bx,1 ;change to byte index from word index
 mov al,[bx+CharacterGivenDirectionTable] ;get char for

 ; this direction
 mov ah,al ;move horizontal or vert
 mov dl,[Last_Maze_Direction] ; character into AH
 mov [Last_Maze_Direction],bl ;if last dir is 0ffh then
 or dl,dl ; just use horiz or vert char
 js OutputFirstCharacter ;look up corner character
 shl dl,1 ; in table using last
 shl dl,1 ; direction*4 + new direction
 add bl,dl ; as index
 mov al,[bx+FirstCharGivenNewAndOldDirectionTable]
OutputFirstCharacter:
 push ax ;AL has corner, AH side char
 call OutputCharacter ;put out corner character
 pop ax ;restore side char to AH
 lodsb ;get count of chars for this
 dec al ; side, minus 1 for corner
 xchg al,ah ; already output
 jmp short RepCharLoop ;put out side char n times
;
; Table of arrow characters pointing in four directions.
;
AnimateCharacterTable db 24,26,25,27
;
; Animates an arrow moving in one of four directions.
;
Animate%:
 sub bx,(Aup+Aup) ;get word dir index
 mov ax,word ptr [XYIncTable+bx] ;set move direction
 mov word ptr [Cursor_X_Increment],ax
 lodsb ;get move count
 shr bx,1 ;make into byte
 mov ah,[bx+AnimateCharacterTable] ; index and get
 xchg al,ah ; char to animate
NextPosition: ; into AL, AH count
 mov dx,[AnimateLastCoordinates] ;coords of last arrow
 ;move cursor to where last
 ; character was output
 mov word ptr [Cursor_X_Coordinate],dx
 push ax ;save char and count
 mov al,20h ;output a space there
 call OutputCharacter ; to erase it
 pop ax ;restore char in AL, count in AH
 push ax ;save char and count
 mov dx,word ptr [Cursor_X_Coordinate] ;store new coords
 mov [AnimateLastCoordinates],dx ; as last
 call OutputCharacter ;output in new
 mov cx,DELAY_COUNT ; location then
WaitSome: ; wait so doesn't
 loop WaitSome ; move too fast
 pop ax ;restore count and
 ; character
 dec ah ;count down
 jne NextPosition ; if not done
 ret ; do again
;
; Sets the animation start coordinates.
;
SetAStart%:
 lodsw ;get both X & Y

 mov [AnimateLastCoordinates],ax ; coordinates and
 ret ; store
;
; Repeats the command that follows the count parameter count times.
;
DoRep%:
 lodsb ;get count parameter
NextRep:
 push si ;save pointer to command
 ; to repeat
 push ax ;save count
 lodsb ;get command to repeat
 mov bl,al ;convert command byte to
 xor bh,bh ; word index in BX
 shl bx,1 ;
 call [bx+Function_Table] ;execute command once
 pop ax ;get back the count
 dec al ;see if it's time to stop
 je DoneWithRep ;jump if done all repetitions
 pop si ;get back the pointer to the
 ; command to repeat, and
 jmp NextRep ; do it again
DoneWithRep:
 pop ax ;clear pointer to command to
 ; repeat from stack, leave
 ; SI pointing to the next
 ; command
 ret
;
Interp endp
;
; Outputs a text character at the present cursor coordinates,
; then advances the cursor coordinates according to the
; X and Y increments.
;
OutputCharacter proc near
 push ax ;save the character to output
 mov ah,2 ;set the cursor position
 mov dx,word ptr [Cursor_X_Coordinate]
 xor bx,bx ;page 0
 int 10h ;use BIOS to set cursor position
 pop ax ;restore character to be output
 mov ah,9 ;write character BIOS function
 mov bl,[Character_Attribute] ;set attribute
 mov cx,1 ;write just one character
 int 10h ;use BIOS to output character
 ;advance X & Y coordinates
 mov ax,word ptr [Cursor_X_Coordinate] ;both x & y Incs
 add al,[Cursor_X_Increment] ; can be negative
 add ah,[Cursor_Y_Increment] ; so must add bytes
 ; separately
 mov word ptr [Cursor_X_Coordinate],ax ;store new X & Y
 ; coordinates
 ret
OutputCharacter endp
;
_TEXT ends
 end Start ;start execution at Start
































































September, 1989
80386 PROTECTED MODE AND MULTITASKING


Putting the 386 to work under MS-DOS




Tom Green


Tom is an engineer at Central Data Corp. in Champaign, Ill. He can be reached
at 217-359-8010.


Protected mode on the 80386 (and on the 80286 as well) provides built-in
memory protection and hardware support for multitasking. There are not many
examples of protected-mode programming because most of the world uses the
8Ox86 family to run MS-DOS. I recently built a small symbolic debugger and
multitasking kernel for the 80386, and learned a great deal about
protected-mode programming in the process. For this article, I developed some
basic 80386 tools that can be used with most MS-DOS C compilers. I used Turbo
C 2.0 and TASM 1.0, and the code should be portable to most compilers and
assemblers. (The assemblers must support the 80386 instruction set.
The biggest drawback to this code is that it runs in 16-bit mode. The 80386
allows a segment to be marked as either a 32-bit segment or a 16-bit segment.
Turbo C 2.0 produces code that uses only 16-bit registers and does not take
full advantage of the 32-bit registered CPU. (When you develop a large
application, this factor is a big drawback.) The code does allow you to learn
more about 80386 protected mode without resorting to lots of assembly language
or investing in expensive new tools.
The 80386 has 32-bit registers and allows access to a 4-gigabyte address
space. Both of these features offer a big improvement over the 8086. I think
that the most important improvement in the 80386 is its hardware support for
multitasking. The ability of the 80386 to save and restore the state of a task
is a big plus when you write operating systems and debuggers. All of these new
features support 16-bit and 32-bit code. As a result, the code presented in
this article has few limitations. You could use this code to develop a small
multitasking kernel just by adding a timer interrupt and the necessary
algorithms.
A warning before you get started: This program only runs on an MS-DOS machine
with an 80386 CPU that is running MS-DOS in real mode. (Some 80386 machines
run MS-DOS as a virtual-mode task in protected mode. You should be able to
turn this off, so check the documentation for your machine and operating
system.) The machine must support and use video modes 2, 3, or 7. (This
program writes directly to video RAM and locates video RAM via the mode. If
the program finds a video mode it cannot use, the program simply exits.)


The Global Descriptor Table


When the 80386 runs in protected mode, the segment registers (CS, DS, ES, SS,
FS, and GS) are loaded with selectors. Each selector points to an 8-byte
descriptor in either the Local Descriptor Table or the Global Descriptor
Table. Each selector also contains a privilege level. In this case, all of the
code and data run at privilege level 0( the highest privilege level). The
process of calculating the selector for a given descriptor is easy when
privilege level 0 is used and a Local Descriptor Table is not used. The
selector is the offset of the descriptor in the Global Descriptor Table.
(Remember that a descriptor is an 8-byte data structure.) To use descriptor 2,
for example, load the segment register with selector 10h. To determine the
appropriate selector, multiply the descriptor by 8, starting with descriptor
1. 8h, 10h, 18h, 20h and 28h are examples of typical Global Descriptor Table
selectors. Descriptor 0 is the NULL descriptor and cannot be used.
Figure 1 shows the layout of the three types of descriptors in this code. To
avoid confusion, a Local Descriptor Table is not used. The Global Descriptor
Table, which can reside anywhere in physical memory, is located within an
array of structures in the C data space. The structure called "descriptor" in
Listing One (386.h) shows how the descriptors in Figure 1 look when written in
C. An array of ten descriptors, called gdt, is declared in Listing Two
(task.c). This array can be larger or smaller, depending upon how many
descriptors are needed for the code. The Global Descriptor Table can contain
up to 8192 descriptors.
Next, set up the descriptors to run the C program. The program will start
running in real mode, so the segment addresses of the C code and data must be
translated to physical addresses. This program will run in the 8086 Small
model, which means it will have 64K of code space and 64K for data, heap, and
stack. (See Figure 2 for the layout of a Turbo C Small model program.) The
segment addresses in this code and data can easily be converted into 20-bit
physical addresses. To do so, left shift each 16-bit segment by 4.
The code uses the three types of descriptors shown in Figure 1: Data Segment
Descriptors, Executable Segment Descriptors, and TSS Descriptors. To
initialize a descriptor, call the routine init_gdt_descriptor. This routine is
located in the file task.c (Listing Two). The first parameter passed to this
routine is a descriptor structure pointer, which is the address of one of the
descriptors in the gdt array. The second parameter passed to the routine
is</entry> a 32-bit (unsigned long) physical base for the segment. This 32-bit
physical address is the location where the segment that is being created will
start. The third parameter passed to the routine is a 32-bit (unsigned long)
limit of the segment. This limit indicates the length of the segment measured
in bytes. The process of setting up the limit in a descriptor gets a little
complicated because only 20 bits are available to contain a 32-bit value. In
this situation, use the G bit or "granularity bit." This process (which would
require a long description) is handled by the C routine. The fourth parameter
passed to the routine is an 8-bit (unsigned char) descriptor type (either a
code, data, or TSS descriptor).
Six segments -- a code segment, a data segment, three Task State Segments
(TSS), and a video memory segment -- must be created in order to run this
code. First, the Turbo C pseudovariables _CS and _DS, which contain the
segment addresses of both the code and the data segments, are used to create
the code and data segments. During the process, the segment addresses are left
shifted 4 bits and turned into physical addresses. A 64K limit is assigned to
each of these descriptors, so that each descriptor appears to the program just
like a real-mode segment would appear. Next, the three Task State Segments are
set up. (Task State Segments are described in more detail later in this
article.) Each Task State Segment is a global data structure with a 64h-byte
limit. The physical address of each Task State Segment is determined by adding
the offset in the corresponding data segment to that data segment's physical
address.
The last segment that must be created is the video memory segment. This
4000-byte data segment allows you to write directly to video RAM. When the
program runs in protected mode, no MS-DOS functions can be accessed, so the
print routine must write directly to video RAM. After BIOS is called to
determine the video mode, a segment is set up for color or monochrome memory.
The vid_mem_putchar routine writes characters and attributes to video RAM by
using a far pointer that contains the selector for the newly created video RAM
segment.


Task State Segments


A Task State Segment is used by the 80386 when a task switch occurs. The 80386
has a special task register that holds the selector for the TSS that
corresponds to the task that is currently running. Figure 3 shows a TSS.
When a task switch occurs, the current values of all of the CPU registers are
saved into the task's TSS. As you can see in Figure 3, there is a place to
store all general-purpose registers and segment registers. The complete state
of the task can be saved so that the next time the task executes, it can pick
up where it left off. After the current task's state is saved in that task's
TSS, the state of the new task is loaded into the CPU registers from the new
task's TSS. The CPU begins execution at the address loaded into cs:eip from
the TSS. The new task picks up right where it left off the last time it was
switched out (or it picks up the values to which the TSS is initialized).
Listing One contains a structure, called "tss," that can be used to set up a
TSS. This structure contains several fields that begin with fill and are not
used by the CPU, but must be zeroed. With this structure a TSS can be
initialized and installed in the Global Descriptor Table.
The routine init_tss in Listing Two creates a TSS for a task. This routine
sets up the starting selector value for all segment registers with the code
and data selectors passed (cseg and dseg parameters). The eip (instruction
pointer) is set up to point to the code of the task. The esp and ebp registers
are set up to point to the stack for the new task. At this point, TSS has
starting values for execution, data, and stack. The func_ptr ip parameter,
which contains the address in the code segment where execution begins, is
passed. In this code, pass the address of the C function that will be used for
the tasks (task1 and task 2).
A TSS must be installed in the Global Descriptor Table just as a code or data
segment is installed. To install a TSS, find its physical address in memory
and set up the size limit of 64h bytes. The type field of the descriptor
indicates to the CPU that a TSS is a special kind of segment.


Assembly Language Routines


Four assembly language routines (see Listing Three) are required in order to
use the 80386 code. These routines are described later, and are called from C
with parameters passed on the stack. It would be a good idea to examine
Listing Three carefully while you read these descriptions.
void protected_mode(unsigned long gdt_ptr,unsigned int cseg, unsigned int
dseg). This routine sets everything up and makes the switch to protected mode.
The first parameter is gdt_ptr, which is the 32-bit physical address of the
Global Descriptor Table. (Pass the physical address of the descriptor array
gdt in this parameter.) The next two parameters, cseg and dseg, are the
selectors for the descriptors set up for the selector in the array gdt. (In
these parameters, pass the selector for the code segment and for the data
segment, respectively.) Next, the cs register is loaded by a jmp instruction
with the code segment selector. The rest of the segment registers are loaded
with the data segment selector. During the entire process, the code and data
descriptors have been set up to match the way that the segments would appear
if the CPU was running in real mode. At this point, the routine can set
everything up and return to the C code. The code, data, and stack are all the
same as they would be in real mode. The jmp DWORD PTR p_mode instruction
flushes the CPU instruction prefetch queue and ensures that the cs register is
loaded with the correct selector. Because the code jumps through a pointer,
the data p_mode is set up to point to the protect label. It may seem strange
to jump to the next instruction, but it must be done.
void real_mode(unsigned int dseg). This routine switches back to real mode.
The routine reloads all of the segment registers with the correct real-mode
segments, and assumes that the code is executing from a 64K segment. (A
complete description of the process of switching back to real mode, is
presented on page 14-4 of the Intel 80386 Programmer's Reference Manual, so I
will not discuss this process in detail here.) Each segment register must be
loaded with the selector of a segment that has a 64K limit before returning to
real mode. The parameter dseg is loaded into all of the segment registers
except the segment register cs. The routine then performs another jump to the
next instruction via jmp FAR PTR flush. This step flushes the instruction
prefetch queue, and loads the cs register with the original code segment that
runs in real mode.
void load_task_register(unsigned int tss_selector). This routine loads the CPU
task register with the TSS selector that is passed. When the CPU performs the
first task switch, it must have an available TSS where the current task state
can be stored. Before the routine switches to the first task, the task
register must be loaded with a valid TSS selector.
void jump_to_task(unsigned int task). This routine switches to a new task.
Pass the selector of the TSS you want to switch to. If the task that calls
jump_to_task is switched back to, the task resumes execution immediately after
the jmp DWORD PTR new_task instruction and will return to the C code that
called jump_to_task. The code can then switch back and forth between tasks,
starting where it left off at the task switch. In this routine, the code is
jumping through a pointer, so new_task data must be set up with the selector
of the new task. The address offset to jump to is not used for new tasks. The
task gets the address to start running from its TSS when the registers are
loaded.


Using the Code


The code is contained in three files: task.c, 386.h, and mode.asm. The
following Turbo C 2.0 and TASM 1.0 instructions assemble, compile, and link
the code:
 tasm /MX mode.asm tcc -I. .\include -L. .\lib task.c mode.obj
This code can easily be ported to other MS-DOS C compilers. The Turbo C
pseudovariables _CS and _DS are used to create the Global Descriptor Table and
Task State Segments. If your C compiler does not support these
pseudovariables, you can create two small assembly language routines to place
the values into the cs and ds registers. If your compiler expects the return
value of a function to be located in the ax register (which is usually the
case) the routines can be as simple as this:

 Get current value of cs register

 mov ax,cs ret

 Get current value of ds register

 mov ax,ds ret

Some Turbo C-specific screen I/O routines have also been used in this code. If
you port the code to another C compiler, check that compiler's manual for
similar routines.
The assembly language portion of the code must be used with an assembler that
supports the 80386 instruction set and some of the new features in Microsoft
MASM 5.0. Parameters are expected on the stack, as described for the Turbo C
Small C memory model. You may have to adjust these parameters before the code
can be used with your compiler.
The code in its present form must be compiled under the Small C memory model
in order to work properly. In addition, the code expects only two segments: A
code segment and a data segment. If you use a memory model with multiple code
segments and a function tries to perform a far jump to another function, the
first function will jump to a different segment, which will be an invalid
selector. With a lot of work, descriptors could be created for all code and
data segments for larger C memory models, but I would recommend starting with
the Small model.
To run the program, type TASK. Note that the program prompts you to press
Return to enter protected mode. After you press Return, several hello messages
from task 1 and task 2 appear on the screen. Next, you are returned first to
the main task and finally to real mode. You are then prompted to press Return
in order to exit to DOS.
The 80386 allows a task switch to occur when an interrupt handler is called.
(This feature comes in handy when you develop a debugger.) When a processor
exception occurs, the interrupt handler can cause a task switch. To determine
exactly what was contained in all of the registers at the time of the
exception, examine the TSS of the task that caused the exception. In addition,
because the 80386 supports breakpoints in hardware, your debugger can also use
an interrupt handler that causes a task switch to the debugger. Breakpoint
handling is easy when supported by hardware breakpoints and multitasking. I
recommend a complete reading of the Intel 80386 Programmer's Reference Manual
to see all of the features of the 80386. Although this code does not do very
much, it provides a set of tools that can be easily expanded into a
multitasking kernel.
As mentioned before, there are some drawbacks to this code. Because Turbo C
2.0 cannot produce real 32-bit code, the 32-bit registers in the 80386 are not
used. A solution is to write assembly language subroutines that take advantage
of the 80386 capabilities. In addition, you cannot use many of the C library
routines or BIOS or MS-DOS calls with this code. These routines and calls
expect information to be located in certain segments, and will load the
segment registers with invalid selectors in protected mode. Also, when the
switch is made to protected mode, interrupts are turned off; the 80386 then
handles interrupts with a special Interrupt Descriptor Table. The process of
implementing this step would have added too much to the code, so interrupts
must be turned off when the code runs in protected mode. To run the code in
protected mode, new interrupt handlers would need to be developed. If you
develop an Interrupt Descriptor Table, use the code in a Global Descriptor
Table as a guide (the two tables are similar).
I hope that this article has provided some insights into 80386 protected-mode
programming. If you examine this code with the help of the article and the
Intel 80386 Programmer's Reference Manual, you should be able to use protected
mode. In general, most examples of protected-mode routines are written in
assembly language and are difficult to understand. When the Global Descriptor
Table and Task State Segments are created using C, protected-mode programs are
much simpler to set up and to install.


Suggested Reading


80386 Programmer's Reference Manual, Intel Corporation, 1987.
Nelson, Ross. "Programming on the 80386," Dr. Dobbs Journal: October 1986.


Availability


All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063, or call 800-356-2002 (from inside
Calif.) or 800-533-4372 (from outside Calif.). Please specify the issue number
and format (MS-DOS, Macintosh, Kaypro).


_80386 PROTECTED MODE AND MULTITASKING_
by Tom Green


[LISTING ONE]

/************************************************************************/
/* 386.H - structures etc. for the 80386 */
/* By Tom Green */
/************************************************************************/

/* all of these structures are processor dependant, so */
/* you must set code generation for byte alignment */


/* generic descriptor - data, code, system, TSS */

typedef struct descriptor{
 unsigned int limit_lo;
 unsigned int base_lo;
 unsigned char base_mid;
 unsigned char type_dpl;
 unsigned char limit_hi;
 unsigned char base_hi;
}descriptor;

/* call, task, interrupt, trap gate */

typedef struct gate{

 unsigned int offset_lo;
 unsigned int selector;
 unsigned char count;
 unsigned char type_dpl;
 unsigned int offset_hi;
}gate;

/* this is the layout for a task state segment (TSS) */
/* the fill fields of structures are not used by the 80386 */
/* but must be there */

typedef struct tss{
 unsigned int back_link; /* selector for last task */
 unsigned int fill1;
 unsigned long esp0; /* stack pointer privilege level 0 */
 unsigned int ss0; /* stack segment privilege level 0 */
 unsigned int fill2;
 unsigned long esp1; /* stack pointer privilege level 1 */
 unsigned int ss1; /* stack segment privilege level 1 */
 unsigned int fill3;
 unsigned long esp2; /* stack pointer privilege level 2 */
 unsigned int ss2; /* stack segment privilege level 2 */
 unsigned int fill4;
 unsigned long cr3; /* control register 3, page table */
 unsigned long eip; /* instruction pointer */
 unsigned long eflags;
 unsigned long eax;
 unsigned long ecx;
 unsigned long edx;
 unsigned long ebx;
 unsigned long esp;
 unsigned long ebp;
 unsigned long esi;
 unsigned long edi;
 unsigned int es;
 unsigned int fill5;
 unsigned int cs;
 unsigned int fill6;
 unsigned int ss;
 unsigned int fill7;
 unsigned int ds;
 unsigned int fill8;
 unsigned int fs;
 unsigned int fill9;
 unsigned int gs;
 unsigned int filla;
 unsigned int ldt;
 unsigned int fillb;
 unsigned int tbit; /* exception on task switch bit */
 unsigned int iomap;
}tss;

#define TSS_SIZE (sizeof(tss))
#define DESCRIPTOR_SIZE (sizeof(descriptor))
#define GATE_SIZE (sizeof(gate))
#define DPL(x) (x<<5)
#define TYPE_CODE_DESCR 0x18
#define TYPE_DATA_DESCR 0x10
#define TYPE_TSS_DESCR 0x09

#define TYPE_CALL_GATE 0x0c
#define TYPE_TASK_GATE 0x05
#define TYPE_INTERRUPT_GATE 0x0e
#define TYPE_TRAP_GATE 0x0f
#define SEG_WRITABLE 0x02
#define SEG_READABLE 0x02
#define SEG_EXPAND_DOWN 0x04
#define SEG_CONFORMING 0x04
#define SEG_ACCESSED 0x01
#define SEG_TASK_BUSY_BIT 0x02
#define SEG_PRESENT_BIT 0x80
#define SEG_GRANULARITY_BIT 0x80
#define SEG_DEFAULT_BIT 0x40
#define SELECTOR_MASK 0xfff8







[LISTING TWO]

/************************************************************************/
/* TASK.C - this code creates and sets up the Global Descriptor */
/* Table and Task State Segments. the code switches to protected */
/* mode, runs tasks, and returns to real mode. */
/* Compile with Turbo C 2.0 */
/* By Tom Green */
/************************************************************************/

#include <stdio.h>
#include <dos.h>
#include <conio.h>
#include <stdlib.h>
#include "386.h"

/* selectors for entries in our GDT */
#define CODE_SELECTOR 0x08
#define DATA_SELECTOR 0x10
#define TASK_1_SELECTOR 0x18
#define TASK_2_SELECTOR 0x20
#define MAIN_TASK_SELECTOR 0x28
#define VID_MEM_SELECTOR 0x30

/* physical address of video ram, mono and color */
#define COLOR_VID_MEM 0xb8000L
#define MONO_VID_MEM 0xb0000L

/* video modes returned by BIOS call */
#define MONO_MODE 0x07
#define BW_80_MODE 0x02
#define COLOR_80_MODE 0x03

/* pointer to a function */
typedef void (func_ptr)(void);

/* extern stuff in mode.asm */
void protected_mode(unsigned long gdt_ptr,unsigned int cseg,unsigned int
dseg);

unsigned int load_task_register(unsigned int tss_selector);
void real_mode(unsigned int dseg);
void jump_to_task(unsigned int tss_selector);


/* prototypes for local functions */
void task1(void);
void task2(void);
void init_tss(tss *t,unsigned int cs,unsigned int ds,unsigned char *sp,
 func_ptr ip);
void init_gdt_descriptor(descriptor *descr,unsigned long base,unsigned long
 limit,unsigned char type);
void print(unsigned int x,unsigned int y,char *s);
void vid_mem_putchar(unsigned int x,unsigned int y,char c);

/* this array of descriptors will be our Global Descriptor Table */
descriptor gdt[10];

/* these are the TSS's for our tasks */
tss main_tss;
tss task_1_tss;
tss task_2_tss;

/* seperate stacks for each task */
unsigned char task_1_stack[1024];
unsigned char task_2_stack[1024];


/* global y location for protected mode screen writes */
/* using descriptor for video ram */
unsigned int y=0;

void main(void)
{
 unsigned long base;
 unsigned char type;
 union REGS r;

 /* setup code and data descriptors in GDT */

 /* code GDT entry 1 */

 /* turn code segment into 20 (and 32) bit physical base address */
 base=((unsigned long)_CS)<<4;
 /* set descriptor type for a readable code segment */
 type=TYPE_CODE_DESCR SEG_PRESENT_BIT SEG_READABLE;
 init_gdt_descriptor(&gdt[1],base,0xffffL,type);

 /* data GDT entry 2 */

 /* turn data segment into 20 (and 32) bit physical base address */
 base=((unsigned long)_DS)<<4;
 /* set descriptor type for a writeable data segment */
 type=TYPE_DATA_DESCR SEG_PRESENT_BIT SEG_WRITABLE;
 init_gdt_descriptor(&gdt[2],base,0xffffL,type);

 /* set up TSS's for tasks here */

 /* set descriptor type for a TSS */

 type=TYPE_TSS_DESCR SEG_PRESENT_BIT;

 /* put a descriptor for each TSS in the GDT */

 /* TSS GDT entry 3, TSS for task1 */
 /* turn segment:offset of task1 TSS into physical base address */
 base=(((unsigned long)_DS)<<4)+(unsigned int)&task_1_tss;
 init_gdt_descriptor(&gdt[3],base,(unsigned long)TSS_SIZE-1,type);

 /* TSS GDT entry 4, TSS for task2 */
 /* turn segment:offset of task2 TSS into physical base address */
 base=(((unsigned long)_DS)<<4)+(unsigned int)&task_2_tss;
 init_gdt_descriptor(&gdt[4],base,(unsigned long)TSS_SIZE-1,type);

 /* TSS GDT entry 5, TSS for main starting task */
 /* turn segment:offset of main TSS into physical base address */
 base=(((unsigned long)_DS)<<4)+(unsigned int)&main_tss;
 init_gdt_descriptor(&gdt[5],base,(unsigned long)TSS_SIZE-1,type);

 /* init the TSS with starting values for each task */

 /* task 1 */
 init_tss(&task_1_tss,CODE_SELECTOR,DATA_SELECTOR,task_1_stack+
 sizeof(task_1_stack),task1);

 /* task 2 */
 init_tss(&task_2_tss,CODE_SELECTOR,DATA_SELECTOR,task_2_stack+
 sizeof(task_2_stack),task2);

 /* video ram descriptor GDT entry 6 */

 /* set descriptor for a writeable data segment */
 type=TYPE_DATA_DESCR SEG_PRESENT_BIT SEG_WRITABLE;
 r.h.ah=15; /* get video mode BIOS */
 int86(0x10,&r,&r);
 /* check if mono mode */
 if(r.h.al==MONO_MODE)
 init_gdt_descriptor(&gdt[6],MONO_VID_MEM,3999,type);
 /* check if color mode */
 else if(r.h.al==BW_80_MODE r.h.al==COLOR_80_MODE)
 init_gdt_descriptor(&gdt[6],COLOR_VID_MEM,3999,type);
 else{
 printf("\nThis video mode is not supported.");
 exit(1);
 }

 /* we are now ready to enter protected mode */

 clrscr();
 cprintf("\nPress return to enter protected mode.");
 getchar();

 /* turn segment:offset of GDT into 20 bit physical address */
 base=(((unsigned long)_DS)<<4)+(unsigned int)&gdt;

 /* this puts us in protected mode */
 protected_mode(base,CODE_SELECTOR,DATA_SELECTOR);

 /* this loads the task register for the first task */

 load_task_register(MAIN_TASK_SELECTOR);

 y=3; /* this is line we will start printing on in protected mode */
 /* using our descriptor to write to video ram */

 print(0,y++,"Entered protected mode in main task");

 /* this jumps to first task (which will jump back here eventually) */
 jump_to_task(TASK_1_SELECTOR);

 print(0,y++,"Returned to main task, leaving protected mode");

 /* return us to real mode */
 real_mode(DATA_SELECTOR);

 gotoxy(1,22);
 cprintf("Returned to real mode. Press return to exit to DOS");
 getchar();
 clrscr();
}

/* code for task 1 */

void task1(void)
{
 while(1){
 print(0,y++,"Hello from task1");
 jump_to_task(TASK_2_SELECTOR);
 /* return to original task (in main()) after several task switches */
 if(y>18)
 jump_to_task(MAIN_TASK_SELECTOR);
 }
}

/* code for task 2 */

void task2(void)
{
 while(1){
 print(0,y++,"Hello from task2");
 jump_to_task(TASK_1_SELECTOR);
 }
}

/* this initializes a TSS */
/* inits segment registers, eip, and stack stuff to starting values */

void init_tss(tss *t,unsigned int cs,unsigned int ds,unsigned char *sp,
 func_ptr ip)
{
 t->cs=cs; /* code selector */
 t->ds=ds; /* set these to the data selector */
 t->es=ds;
 t->ss=ds;
 t->fs=ds;
 t->gs=ds;
 t->eip=(unsigned int)ip; /* address of first instruction to execute */
 t->esp=(unsigned int)sp; /* offset of stack in data */
 t->ebp=(unsigned int)sp;

}

/* this initializes a descriptor in the Global Decsriptor Table */
/* sets up the base, limit, type, and granularity */

void init_gdt_descriptor(descriptor *descr,unsigned long base,unsigned long
 limit,unsigned char type)
{
 descr->base_lo=(unsigned int)base;
 descr->base_mid=(unsigned char)(base >> 16);
 descr->type_dpl=type;
 /* if limit > 0xfffffL then we have to set granularity bit and shift */
 if(limit > 0xfffffL){
 limit = limit >> 12;
 descr->limit_hi=((unsigned char)(limit >> 16) & 0xff) 
 SEG_GRANULARITY_BIT;
 }
 else
 descr->limit_hi=((unsigned char)(limit >> 16) & 0xff);
 descr->limit_lo=(unsigned int)limit;
 descr->base_hi=(unsigned char)(base >> 24);
}

/* this routine prints a string using vid_mem_putchar */

void print(unsigned int x,unsigned int y,char *s)
{
 while(*s)
 vid_mem_putchar(x++,y,*s++);
}

/* this routine writes a character directly to video ram */
/* uses the selector for the descriptor we set up in main */
/* for video ram */

void vid_mem_putchar(unsigned int x,unsigned int y,char c)
{
 register unsigned int offset;
 char far *vid_ptr;

 offset=(y*160) + (x*2);
 /* make our far pointer use our special video ram descriptor */
 /* yes, we can even use far pointers with selectors */
 vid_ptr=MK_FP(VID_MEM_SELECTOR,offset);
 *vid_ptr++=c; /* write character */
 *vid_ptr=0x07; /* write attribute byte */
}






[LISTING THREE]

;*****************************************************************
; MODE.ASM
; Routines for switching to protected and real mode. Also includes
; routines for loading task register and jumping to a task.

; Assemble with Turbo TASM 1.0
; By Tom Green
;*****************************************************************

 .MODEL SMALL

 .386P

 .DATA

;this is where we stuff address of GDT that is passed
gdtptr LABEL PWORD
 dw 50h ;size in bytes of GDT, enough for 10 entries
 dd ? ;this is where we will put physical address of GDT

;this is where we will store the address of a task (the selector) that
;we will jump to in jump_to_task
new_task LABEL DWORD
 dw 00h
new_select LABEL WORD
 dw 00h

;this is where we store address to jump to when we enter protected mode
;in protected_mode
;offset of code where we will jump
p_mode LABEL DWORD
 dw OFFSET protect
;put selector of protected mode code segment here
p_mode_select LABEL WORD
 dw 0

 .CODE

 PUBLIC _real_mode,_protected_mode,_jump_to_task
 PUBLIC _load_task_register

;*****************************************************************
; void protected_mode(unsigned long gdt_ptr,unsigned int cseg,
; unsigned int dseg) - puts 386 in protected mode and loads segment
; registers with code and data selectors passed (cs and ds
; parameters). pass this routine a 32 bit physical address of the
; Global Descriptor Table (gdt_ptr parameter). Turns interrupts
; off while we run in protected mode.
;*****************************************************************
_protected_mode PROC NEAR
 push bp
 mov bp,sp
 mov ax,[bp+4] ;get low word of address of GDT
 mov dx,[bp+6] ;get high word of address of GDT
 mov WORD PTR gdtptr+4,dx ;store high word of address of GDT
 mov WORD PTR gdtptr+2,ax ;store low word of address of GDT
 mov ax,[bp+8] ;get selector for code descriptor
 mov dx,[bp+10] ;get selector for data descriptor
 mov p_mode_select,ax ;put code selector in our jmp pointer
 mov eax,0 ;prepare to zero out eflags
 push eax
 popfd ;zero out eflags
 lgdt PWORD PTR gdtptr ;load gdt register with limit and ptr
 mov eax,cr0

 or eax,1
 mov cr0,eax ;turn protected mode on
 jmp DWORD PTR p_mode ;this will jump to protect through ptr
 ;(cs will be loaded with code selector)
protect:
;we are now running in protected mode, and we will load segment registers
;with selectors that look like our code is still in real mode
 mov ss,dx ;load segment registers with data selector
 mov ds,dx
 mov es,dx
 mov fs,dx
 mov gs,dx
 mov ax,0
 lldt ax ;make sure ldt register has 0
 pop bp
 ret
_protected_mode ENDP

;*****************************************************************
; void load_task_register(unsigned int tss_selector) -
; loads task register with TSS selector
;*****************************************************************
_load_task_register PROC NEAR
 push bp
 mov bp,sp
 ltr [bp+4] ;load task register with selector for current task
 pop bp
 ret
_load_task_register ENDP

;*****************************************************************
; void real_mode(unsigned int dseg) -
; returns 386 to real mode. pass this routine the selector for
; a 64k data segment so we can return to real mode. this
; routine assumes we are executing from a 64k data segment.
; (80386 segment registers must have selector of segment with
; 64k limit to return to real mode)
;*****************************************************************
_real_mode PROC NEAR
 push bp
 mov bp,sp
 mov ax,[bp+4] ;get selector for data segment
 mov ds,ax ;now make sure all segment registers
 mov es,ax ;contain selector to 64k segment
 mov fs,ax ;must have this to return to real mode
 mov gs,ax
 mov ss,ax
 mov eax,cr0
 and eax,07ffffffeh
 mov cr0,eax ;protected mode off
 jmp FAR PTR flush ;flush queue and set cs for real mode
 ;now cs will be loaded with correct
 ;segment for real mode
flush:
 mov ax,DGROUP ;restore data seg registers for real mode
 mov ds,ax
 mov ss,ax
 mov es,ax
 sti ;interrupts back on for real mode

 pop bp
 ret
_real_mode ENDP

;*****************************************************************
; void jump_to_task(unsigned int tss_selector) -
; jumps to 386 TSS task. pass this routine the selector of the
; TSS of the task you want to jump to.
;*****************************************************************
_jump_to_task PROC NEAR
 push bp
 mov bp,sp
 mov ax,[bp+4] ;get selector of new task
 mov new_select,ax ;store it in pointer
 jmp DWORD PTR new_task ;jump to task through selector:offset ptr
 pop bp
 ret
_jump_to_task ENDP

 END










































September, 1989
WATCOM C7.0


Good things do get better




John M. Dlugosz


John is a free-lance writer and software developer and can be reached at P.O.
Box 867506, Plano TX 75086, or on CompuServe at 74066,3717.


You'll notice right away that Watcom is unique. As you open the box you'll
find a library-quality magazine holder containing several slim books, a
package of disks, and a foam block that prevents the contents from rattling
around during shipping. Throw the foam away and you've got a convenient caddy,
the perfect size for storing books. This kind of functionality sums up Watcom
C7.0.


Documentation


Watcom's documentation consists of a Library Reference Manual, Language
Reference (including a Programmer's Guide), Optimizing Compiler and Tools
User's Guide, Express C User's Guide, and Graphics Library Reference. Among
the smaller documents are an Addendum to the Library Reference, Watcom Editor
User's Guide, a Read Me First, and a set of 8.5- x 3-inch fanfold reference
cards (one covers the compiler and tools use, including C syntax, another the
library, and a third the graphics library), that fits nicely under the
keyboard.
The no-nonsense installation instructions are written with the assumption that
you know what you're doing, unlike other instructions I've recently seen. They
tell you what is on the disks and how the installation program (which is
excellent) will arrange the files on the hard disk. You can do without the
install program if you like.
The User's Manual tells you how to run the compiler, debugger, make utility,
and associated tools, and about the pragmas, calling conventions, and other
important information. The Language Reference manual is a reference to the C
syntax, and provides compiler-dependent information (bit field organization,
for instance). Everything you need to know about the compiler and the language
implementation seems to be listed, except how multi-byte character constants
(such as 'ab' -- Watcom, is that Ox6162 or Ox6261?) are handled. One
shortcoming, however, is that the books neither reference each other nor have
a common index.
The information in the User's Guide for the compiler and the debugger is
disorganized, though the index is excellent. I thought the book suffered from
an attempt to arrange sections in a "tutorial" order instead of as a "pure"
reference manual. Nevertheless, as a reference for someone who knows what he
is looking for, the documentation is a winner.


Utilities


The driver WCL is a typical C compiler driver. You can list file names (with
wild cards) and options on the command line to compile and link the files. You
can give the program OBJ files, LIB files, and C source files, but not ASM
files. Unfortunately the options apply to all the files, rather than just
those files that are listed after the option switch, which means that you
cannot give different options to different files.
You can reduce memory requirements and increase speed by using WCC, the C
compiler, in make files or anywhere else the driver is not needed or wanted.
Because the linker, WLINK, does not use the same commands as Microsoft's
linker (and those that imitate it), you'll have problems with your existing
linker response files. (WLINK is modeled after PLINK.) I took an existing
project and tried to switch to Watcom's compiler: first switched to the WCC
command in the make file (with the proper switches), keeping my old linker so
I would not have to rewrite the linker response file. I ran into two problems:
First, other linkers cannot handle Watcom's OBJ modules that contain debugging
information. Second, because Watcom's OBJ files use a different format for
storing the default library search, other linkers will not find it, and you
will have to add the library names to the linker response file.
The linker does, however, support multiple overlays, and lets you include or
exclude debugging information on a module-by-module basis.
Also included is a window-based debugger called WVIDEO. I'm not sure how much
memory it needs, but I know it requires much less than CodeView. Because it
loads symbols from the disk module by module, you can include debug
information on everything and not run out of memory. All windows are the full
width of the screen, can be moved or resized vertically, and can overlap.
The debugger is powerful, and has a macro language that controls all the
features and functions. You can use the command line to type in commands or
select options (using the Alt key) from a menu along the top of the screen.
You must use the up- and down-arrow keys, not the initial keys, and press
Return to select choices from submenus, or you can choose items with a mouse.
The Tab key moves the cursor from window to window; with a source or assembly
listing window, for instance, you can move the cursor and inspect different
parts of the program.
Commands (or a sequence of commands) can be assigned to a single key, and are
context-sensitive based on the window you are in. So you can assign a key to
set a breakpoint at the cursor position in the source window, another to
single step one step, or whatever you like. The file called PROFILE.DBG is
loaded when the session is started and contains all the Hot-key assignments,
window layout, and everything else you start up with. You can modify this file
to arrange things the way you like.


The Debugger


Debugger macros can refer to variables in your program, or to variables
defined by the debugger. A printf-like function displays values, an If
statement makes conditional choices, and long scripts can be stored in disk
files. What I liked best, however, is that breakpoints can trigger macro
execution.
The debugger is not perfect, however, as there is no "watch" window. (I'd like
a window that would display the results of a number of expressions that I want
to keep an eye on.) A "command" window, however, can be configured to function
as a watch window. The command window is considerably more powerful than a
typical "watch." It will execute a statement list, which presumably contains
some PRINT statements, every time the window is updated.
The other thing I don't like about the debugger is that an array of chars is
always displayed as the address of the first element. If I want to see the
value of a string, I must use a format string like ? {%s} name to display the
contents of name in a meaningful way. Unfortunately the same thing happens
when displaying an entire structure -- if you print a structured type
variable, all the fields are printed. You can also bring up an interactive
inspector, which shows the fields, and then select any of them to display. In
both forms, strings show up as their address.
When debugging is complete, the WSTRIP utility will remove debugging
information from the EXE file. All in all, the debugger is good enough for you
to invest the time it takes to learn how to use it well.


The Compiler


Though the compiler does not generate a compiled listing, a separate utility,
WDISASM, disassembles an OBJ file and correlates it with the original C
source. This is the best such utility I have seen. You can specify many
details, like putting register names in all caps, using [BX-2] form instead of
2[BX], including symbol references, and producing an assembled listing or
assembly source code only. With full symbolic information in the OBJ file, it
does a great job. Other tools included are a library manager, a touch utility,
a make utility, and a patch utility.
The compiler itself is the heart of the package. It is somewhat slower than
other compilers, but it finds all the errors before it starts crunching away
on the code generation part, so the development turn-around time is very fast.
One nice touch is that if you see a warning you don't like, the Break key
stops compilation and aborts instantly.


The Code Generator



The code generator is terrific. It passes parameters in registers, which can
do wonders for both size and speed. In some cases the object code produced was
a third of the size of Microsoft's. For short and simple functions, much of
the overhead of a function call is eliminated, and it becomes more of an
assembly language subroutine than a high-level language function call. Figure
1, for instance, shows a simple function that has been optimized. Notice that
a stack frame is not generated and that the resulting function is a simple
subroutine that doesn't have the overhead of a high-level language call.
Figure 1: A simple function that has been optimized

 Function

 const int c= 5;

 int foo (int x)
 {
 int y;
 y=2*x+c;
 return y;
 }

 int d;

 void bar( )
 {
 d= foo(d);
 }

 Code generated by Watcom C7.0

 foo_ shl AX,1
 add AX,_c
 ret

 bar_ mov AX,_d
 call foo_
 mov _d,AX
 ret

 Compare this with:

 foo push BP
 mov BP,SP
 mov AX,[BP+4]
 shl AX,1
 add AX,_c
 pop BP
 ret

 bar push BP
 mov BP,SP
 mov AX,_d
 push AX
 call foo
 add SP,2
 mov _d,AX
 pop BP
 ret

With optimization disabled, the code generator still generates fair code, so
some simple improvements must be built into the code generator on a primitive
level. Without optimization, the symbolic debug information is superb. All the
line numbers and local symbol information are exactly where I would expect,
without quirks. Even normal optimization is easy to follow in the debugger,
with the exception of loop optimization, where invariant expressions are
hoisted out.
There are numerous ways to fine-tune code generation. You can use a pragma to
modify function attributes, and you can specify the calling sequence in some
detail (including how parameters are passed, how the call is made, how the
value is returned, who clears any parameters from the stack, which registers
are not preserved by the function, what name the linker will know it by, and
other information). You can even have a "function" that generates a sequence
of bytes (which should correspond to some meaningful assembly language)
instead of a call instruction. For example, I had a function called bios_int(
) that was written in assembly language. It took four unsigned parameters and
loaded them into the AX, BX, CX, and DX registers, did an INT 10h call, and
returned the value from AX. With Watcom 7.0, I can define a pragma saying that
int_bios receives its parameters in AX, BX, CX, DX, returns a value in AX, and
generates an INT 10h instruction when the function is called instead of
calling a function. In other words, the assembly language function was
completely eliminated and replaced by intrinsic code generation that does the
same thing.
The amount of information that can be specified by pragmas is enormous. You
can fine-tune a program's performance, call functions that were compiled by a
different compiler, and fix linkage problems.
One interesting feature is that you can use a pragma to specify exactly what
"cdecl," "pascal," and "fortran" modifiers mean. For example, you can have
cdecl mean the Microsoft C calling convention, or you could have it mean
something else if you were using libraries produced for another compiler. This
can be set up to use existing assembly language routines. There is one quirk,
though. In large models, the DS register is used freely and does not
necessarily point to DGROUP when your assembly language function is called. A
compiler switch can be used to eliminate this behavior, and DS points to
DGROUP throughout the program. It would be better to allow the pragma to
specify that DS be loaded before the function is called.
As I said before, errors in the source are quickly found. You can even call
the compiler with a switch for syntax-check only. For the most part, the check
finds mistakes in the source that you would expect a modern day compiler to
find, though it can miss some. Figure 2 illustrates that it did not warn me
that I was passing a far pointer where a long was expected. Neither does it
always catch type errors where typedefs are involved.
Figure 2: Example demonstrating how a parameter of incorrect type is passed
without warning


 typedef long BIG;

 void foo (BIG);

 void bar ()
 {
 char *buf;
 foo (buf);
 }

In addition to this compiler, the package includes an Express C integrated
environment compiler, though most professional programmers would rather have a
regular version.


ANSI Conformance


Because the packaging proudly proclaimed "100% ANSI," I decided to delve into
the conformance issue. I tried the home version of the Plum-Hall validation
suite, and found that the symbol HUGE_VAL caused a linker error because, it
seems, the value is actually defined in the floating point library. The
library name was not included for linking, even though a variable of type
double was defined and a floating point comparison was made. This shows that
you might define a double variable and call printf( ), and have it not work
because the printf( ) linked in was taken from the regular library. This
probably would have worked on a more realistic program. I brought this to
Watcom's attention, and they indicated that they'll move the symbol into the
regular library. (Watcom ought to link the floating point library in if any
floating point variable is defined, as the method they are using does not
catch all the cases where it is needed.)
For things like the order of translation, double-slash comments, line
continuation, required headers, and even trigraphs, C7.0 came through with
flying colors.
While the const and volatile key words are recognized, the semantics are not
all available. A direct constant variable is treated properly, but pointers to
constants are not. In a case like const char *s;, the assignment *s=c; is not
caught as an error. As for volatile, it does not always prevent the kinds of
optimization it is supposed to. Figure 3, which is part of a const semantics
test I ran, indicates that Watcom 7.0 does not implement pointers to const
values.
Figure 3: This test indicated that Watcom C7.0 does not implement pointers to
const values

 void f101( )
 {
 const char *const s=g;
 *s= c;//error, but allowed by Watcom
 s= p;//error
 }

I looked at the standard library and found all the standard headers. I then
checked some of the more exotic aspects of ANSI-ism. The library has locale
functions (probably the single most difficult thing to implement) but only the
"C" locale is supported, so you can't call setlocale( ) (or if you do call it
with C, it doesn't actually have to do anything!). Likewise, localeconv( ) can
be a trivial function, and any functions that depend on the current locale
don't have to do anything special because the locale never changes.
There are multi-byte character handling functions like mblen( ), which gives
the number of bytes in the multi-byte character pointed to, and mbtowc( ),
which converts a multi-byte character to a wide character. But these functions
are trivial because the library does not actually support character sets that
have characters comprising more than one byte. Many of the new functions (40
to be exact) are listed in the Addendum, but not in the Library Reference nor
in the quick reference card.
The standard library is rich, having all the ANSI required functions and all
the DOS expected functions, like FP_SEG( ) and intdos( ). Functions like
_splitpath( ), _heapchk( ), and _heap-shrink( ) (which frees any unused memory
back to DOS in preparation for a system( ) call) are welcome additions to this
version. The library supports most of the niceties that I've encountered on
other compilers with one exception: I would like to be able to automatically
expand wild card file names in the argv[ ] list passed to main( ).


Wrapping It Up


Watcom 7.0's strong suit is its code generation. With its efficient calling
conventions, good optimization, and powerful pragmas, it can generate tight
and fast code. It has a rich library, and is a good first cut at an
ANSI-conforming implementation. The linker handles overlays well and the
debugger can set up complex debugging scenarios. I would be wary of the flaws
in compile time error checking, though. That Watcom 7.0 occasionally misses
bad parameter types and doesn't fully implement const detracts from the
overall evaluation. Watcom 7.0 is worth using for the code generator if that
is what is important to you.


Product Information


Watcom C7.0: requires an IBM PC/XT/AT and PS/2 and compatibles with 512K
memory, DOS 2.0, or later.
Watcom C7.0/386: is exactly the same as C7.0 but (obviously) requires a
386-based machine. C7.0/386 compiles 32-bit 80386 native code. The initial
release does not include a linker or debugger, although future releases will.
This release does use Phar Lap DOS Extender Tools and AI Architect's OS/386.
$895. Watcom Products Inc., 415 Phillip Street, Waterloo, ON N2L 3X2, Canada;
1-800-265-4555.


















September, 1989
PROGRAMMING PARADIGMS


Nasal Nets




Michael Swaine


In a short story by Nikolay Gogol, an officer misplaces his nose, which is
subsequently spotted here and there about Petersburg, masquerading as an
officer of higher rank. While the officer is trying to place an advertisement
offering a reward for his nose's return, the press official solicitously but
injudiciously offers him snuff. Insulted, the officer retorts that he is
missing precisely what is required to appreciate snuff, and besides he would
never use that cheap brand.
The point is ... but first I think I need to backtrack a bit.


Slow Learner


Hal Hardenbergh called to clarify a point in the interview with him that was
published in the June issue. In that interview, I left the speed issue
ambiguous: It's not the execution speed of trained neural nets, but the
convergence of the learning algorithm, that is slow. One other correspondent
knowledgeable in neural net techniques also caught the slip.
In Hardenbergh's words, "While it's true that multi-level, perceptron-based
artificial neural networks are extremely slow, this is only true for the
training phase. When you actually run them, they are much faster. This is even
true for toy problems. I have been playing with neural nets on my three Atari
STs and have achieved speedups from training to practice of one million to
one."
Tom Waite, who is working on neural nets with Hardenbergh at Vicom in San
Jose, Calif., pointed out that the performance of the system as it learns is
dramatic. Unlike adaptive filters, which just get better as they learn, neural
nets do a flip-flop to which it is hard not to attach anthropomorphic labels.
You see the system spontaneously "gets the idea," and develops "insight" into
the problem. "It will be stuck in a rut and all of a sudden go wild and drop
into the right answer," Waite said.
But that insight may take a while.
Slow or not, multi-level, perceptron-based artificial neural networks converge
to solutions more rapidly than some competitors. Hardenbergh said, "the
Boltzmann machine training method is generally regarded as more robust than
multi-level perceptrons, but it is also considered to be two orders of
magnitude slower than back propagation, so it's not used for practical work."
Anderson and Rosenfeld, in their massive and important book Neuro-computing,
say "[Backprop] is ... the most popular learning algorithm for working with
multi-layer networks. It is considerably faster than the other learning
algorithm that is successful with multi-layer networks, the Boltzmann
machine."
So, if your response to the interview was that you were missing precisely what
was required to appreciate the stuff (algorithms, code, hands-on examples),
and besides you would never use such a slow technique, I hope I have corrected
the impression I gave about the speed of neural nets. As for the algorithms,
code, and hands-on examples, that will have to wait until I can devote a full
column to it, because a backprop algorithm doesn't make any sense without a
fairly complete description of the network to which it is applied. At the end
of this column, though, are references to a book and an article where backprop
is well exemplified.


The Explosive Market for Neural Nets


Hardenbergh also brought up to date a reference in that article to a possible
development in the commercial neural nets market." At the time of the
publication of your interview with me there were no commercial applications of
multi-level artificial neural nets. Since then SAIC has signed a multimillion
dollar contract to provide explosives detectors to airports.
"The way their device works is that the explosives are subjected to
bombardment from a gamma ray source and the molecules making up the explosive
substance have characteristic signatures under gamma rays. This results in a
diffraction pattern that identifies the explosive. And SAIC uses a
multi-level, perceptron-based artificial neural network to detect the
characteristic patterns.
"I call a one-hundred-million-dollar contract real activity." So do we, Hal.


Closing the Circle


The management at Vicom is supportive of Waite and Hardenbergh's neural net
explorations because Vicom is in the image processing business, and the
applications of neural nets to image processing are evident. But there are
many powerful techniques already developed for processing images, dating back
at least to Fourier's development, in 1807, of the Fourier transform, which
has had applications Fourier could not have imagined, such as transforming
medical diagnosis by way of the CAT scan.
Waite and Hardenbergh see neural nets as needing to fit in with existing, more
conventional algorithms, so they find themselves regularly thinking in
different paradigms. Lately, Hardenbergh has been treading the well-trod path
of efficient algorithms for circle drawing.
"The past three weekends I've been playing with plotting curves. Back when I
was [working on another project] I plotted curves and pulled out three
curve-plotting algorithms, all in fact from DDJ. When I looked at them
recently, I found to my surprise that all of them used multiplication for each
point. Then I came across Bresenham's algorithm."
In fact, DDJ published an implementation and discussion of Bresenham's
algorithm in September, 1987. It was by Jim Blinn, a graphics expert at JPL. I
used Blinn's version of Bresenham's algorithm in my book on HyperTalk, to show
beginning HyperTalk programmers three ways to draw a circle, but of course
drawing circles in an interpreted language doesn't tell you a whole lot about
algorithmic efficiency. I was interested in Hardenbergh's take on Bresenham's
algorithm, because 1. Hardenbergh is interested in algorithmic efficiency and
usually has an interesting timing result or two to report, and 2. I knew how
he hated wading through academic explanations of technical issues. I suspected
that, if he hadn't seen the Blinn piece, he had only seen academic
explications, and would have impatiently taken off on his own, possibly
interesting tangent.
Sure enough, he told me, "The Bresenham algorithm doesn't use multiplication,
but Bresenham is an academician and so the article I read was full of
derivations and math. Normally, people doing this kind of work keep it as a
trade secret, but Bresenham is an academic, trying to make PhD Brownie points,
so he just made it obscure.
"So I put away the book and set about trying to write a good circle-drawing
algorithm myself. Then I found out that Vicom does not know how to do fast
ellipses, so I worked on ellipses. Now I have a circle algorithm, an algorithm
that does ellipses oriented to the axes, and also a tilted ellipse algorithm,
all without multiplies."
At that point Hardenbergh went back to the books. "I picked up the book in
which I had read the Bresenham algorithm and found to my surprise that I had
not reinvented Bresenham; my algorithm is just not Bresenham's. Mine is
simpler. It is ridiculously simple. And the ellipse and tilted ellipse
algorithms are just extensions of the circle algorithm, so I suspect that my
ellipse algorithms are also different from Bresenham's. I tried to check this
but could find no book there that contained the Bresenham algorithm [for
ellipses]."
Hal didn't give me his algorithms, but he did give me a number to test
against: one thousand radius-100 circles drawn in 5.84 seconds with a 0.02
second loop overhead. The hardware was an Atari ST.


The 48th Parallel Processor


In the Bavarian countryside outside Munich, I met with Jurgen Fey, almost
exactly on the 48th parallel, roughly the same latitude as Poltava, where the
young student Gogol was mercilessly teased for his beak-like nose a century
and a half earlier.
Jurgen turned up his nose when I told him that Hardenbergh had criticized the
Transputer for its lack of registers (Jurgen has developed a Transputer
board). "It's true that it's not a register machine. You can't do parallel
processing on a parallel machine. I challenge him to find a register-based
machine that supports parallel processing."
But Jurgen did agree that the Transputer's "native" language, occam, was on
its way out. There are now, he explained, several more or less viable
alternatives in the works, including ".Lisp, SC-Prolog, Parallel Prolog,
Modula-2, C, C, C, C, C, and C, Ada, assembler, and Forth." Jurgen is working
on something compatible with Turbo Pascal 4.0.


Antisocial Behavior in the Society of Mind



What has been called "the Minsky smoke bomb" continues to smell.
In The Structure of Scientific Revolutions, Thomas Kuhn wrote: "But paradigm
debates are not really about relative problem-solving ability, though for good
reasons they are usually couched in those terms. Instead, the issue is which
paradigm should in the future guide research on problems many of which neither
competitor can yet claim to resolve completely ... that decision must be based
less on past achievement than on future promise."
And Kuhn wrote: "... the defenders of traditional theory and procedure can
almost always point to problems that its new rival has not solved but that for
their view are no problems at all.... if a new candidate for paradigm had to
be judged from the start by hard-headed people who examined only relative
problem-solving ability, the sciences would experience very few major
revolutions."
And: "... The member of a mature scientific community is, like the typical
character of Orwell's 1984, the victim of a history rewritten by the powers
that be."
In Perceptrons, Marvin Minsky and Seymour Papert presented an impeccably
reasoned argument showing certain limits on the results you could get out of
any single-layer perceptron network, along with something quite different: An
"intuitive judgment that the extension [to multi-layer networks] is sterile."
Together, the reasoned argument and the unsupported (and mistaken) intuitive
judgment undermined support for this neural net paradigm that was contending
for AI research funding with their favored Lisp-based work at MIT. In 1969,
Minsky and Papert were the hard-headed establishment of artificial
intelligence and neural nets was the new candidate paradigm that got written
out of the history of AI work by their efforts. Two decades later, the
dramatic comeback of neural nets is one of those heartening success stories,
like "Nikolay Gogol, the odd, beak-nosed boy, had turned obscurity and failure
into triumph, and was well on his way to becoming one of the most famous
authors in Russia" (Dick Penner). Heartening once you get to the success part,
but discouraging through the years of obscurity and failure. Were those years
unnecessary?
Some interesting public documents recently came into my possession, and anyone
interested in the history of invention in this area ought to take a look at
them.
The problem that formed the crux of Minsky and Papert's denunciation of
perceptron-based neural net research was linearly separable functions;
basically, the exclusive-OR problem of classifying unlike objects into the
same category. For example, given a collection of red and green squares and
circles, you are to classify the red squares and green circles together in one
group, and the red circles and green squares in the other group. You can't
form the grouping with a straight line through the space whose dimensions are
red-green and circle-square. But it's a perfectly logical grouping, expressed
by the exclusive-OR expression "red or circle (but not both)." The first group
contains all objects satisfying the expression, the second group all objects
not satisfying it. The expression perfectly assigns the objects to the
required groups. Single-layer perceptrons, and thus neural nets, Minsky and
Papert said, couldn't do that.
Although Minsky and Papert clarified the full consequences of the problem with
single-layer perceptrons in a way that had never been done, this basic problem
had been perceived years earlier, and many people had given thought to its
solution.
At the very least, the problem was under attack. Here is what one researcher
wrote in 1963: "Two layers were used ... because one layer can only be trained
successfully on linearly separable functions [Minsky and Papert's point]....
Having two layers of variable weights was our initial intention, but we do not
yet know of an algorithm leading to a convergent training process." The writer
went on to discuss his group's search for such a convergent algorithm, as well
as practical measures they had taken.
How close were neural net researchers to solving the problem? Had the problem
been solved, perhaps accidentally, nearly a decade before Minsky and Papert
published their book? And was Minsky aware of the solution? I don't have the
answers, but the issue is apparently more complex than anyone has yet
admitted. I refer you to the work on graphical data processing done at SRI in
the early 1960s (references are at the end of this article).


Recommended Reading


Hal Hardenbergh has been buying all the books he can find on neural nets. He
called me to recommend strongly Neural Computing: Theory and Practice, by
Philip D. Wasserman, Van Nostrand Rinehart, 1989.
"I now know enough about back propagation to judge that the chapter on back
propagation is good, so by extension I suspect that the rest is good. I'm
particularly interested in the chapter on stochastic methods. I know someone
who knows the author and who says he's more interested in the Cauchy machine
than in back propagation. He says the training is better with simulated
annealing. The Cauchy machine is another stochastic method based on the
Boltzmann machine, so I'm currently studying the stochastic chapter in
Wasserman's book."
Hardenbergh considers the Wasserman book a good place to start learning about
neural nets: The right level and the right size. "It's not a large book. The
problem is that when you are starting to get into a new field you need to
cover a lot of ground so you need a big book but you're not really ready to
handle a big book."
A much more elementary but very broad presentation of this area can be found
in Cognizers: Neural Networks and Machines that Think, by R. Colin Johnson and
Chappell Brown, John Wiley, 1988. Light reading.
Hardenbergh and Waite have published their own version of the back propagation
algorithm in the June issue of Programmer's Journal.
The SRI work mentioned had to do with the development of several pieces of
parallel processing hardware (including MINOS I and MINOS II) and both
theoretical and practical work on multi-level neural net algorithms. The work
is described in several reports under SRI project number 3192. Quarterly
Reports 4 and 5 are particularly interesting, showing MINOS I doing
exclusive-OR classification. The quote is from the final report, Report No.
12, by A.E. Brain.
Many Russian writers have written stories about noses. Gogol's The Nose,
however, may be the only one in which the curious Russian fascination with
noses has survived translation. It was published in The Diary of a Madman and
Other Stories, Nikolay Gogol, translated by Andrew R. MacAndrew, New American
Library 1960; and was reprinted in Fiction of the Absurd: Pratfalls in the
Void, edited by Dick Penner, New American Library, 1980. The latter book
contains several other works that offer insights into the software development
process, including a selection from Catch-22 and Camus' essay on software
engineering, The Myth of Sisyphus.






































September, 1989
C PROGRAMMING


C++: Of Books, Compilers, and a Window Object




Al Stevens


Last month I looked at Bjorne Stroustrup's The C++ Programming Language and
decided to find another book to be an introduction to C++. The Stroustrup
book, which I will hereafter affectionately call BS (in the tradition of K&R),
will serve as a reference once I understand the C++ language better. But
because I still needed a C++ tutorial text, I scoured the bookstores.
I found a book that I can recommend for learning C++ but only with strict
reservations. The book is The Waite Group's C++ Programming by John Berry and
is for C programmers who want to learn C++. Berry is a good writer and a good
teacher. The organization and presentation of the material is just right for
teaching the extensions that C++ brings to C.
The C++ Programming Language is not, however, for programmers who do not
already know C. While Berry writes effectively about C++, he does not attempt
to teach ancestral C, so if you don't already know it, you aren't going to
learn it here. That omission is understandable and even appropriate -- there
are plenty of good texts on C already. But this book has one unforgivable
problem, one that most programmers will readily identify with and some will
condemn. The code is replete with errors. Most of the errors are ones that the
C programmer will spot right away. But someone else might assume that the code
works, might assume that C works that way, and might actually try to get the
examples to work, an exercise that the author himself should have undertaken
before going to print. Some of the programs in this book would not compile,
much less execute. Others are built with conventions that might work in the
limited contexts of the examples, but that are generally recognized by
seasoned C programmers to be undesirable coding practices.
Following are examples of the errors I found in the book. I am about half way
through now and can only assume that the second half is no better than the
first.
First, the author insists on putting executable code into header files. The
typical .h file contains class definitions and their member functions.
Programs that have several source code files all using the same classes will
run into link problems with multiply-defined functions. C programmers long ago
established a convention of style where header files are for definitions and
prototypes only, while all statements that reserve or use memory go into .c
files. As a newcomer to C++ I am assuming that C++ will have the same
convention. If it doesn't, it should.
Next, there are several programs in the book that use the length of a string,
as returned by strlen, to allocate a new buffer. Then strcpy copies the string
to the buffer. Every C programmer knows that strlen does not count the null
terminator character but that strcpy copies it. The examples might work, but
only when nothing critical is on the heap just past the new buffer.
The book uses a stack data structure to illustrate C++ class definitions. The
stack functions do not use the first element in the stack array, but they do
use the element one past the last one. Once again, the example program
probably runs OK, but if you use it in a bigger program, you are likely to
trash something else. In at least one other place Berry uses a subscript of 2
to address the second element of an array, a technique that does not work in
any context. The example code would not deliver the stated results at all.
These subscripting errors are typical of those made by new C programmers,
particularly programmers coming to C from Basic, and they are the kind of
errors that the author would have found if he had taken the time to try out
some of his code.
The give_date function allocates a new buffer every time you call it. Nothing
ever deletes the buffer. This error occurs several places in the book. Make
enough calls to give_date and the heap (called the "free store" in C++)
exhausts. BS defines a technique for capturing such errors, but the book
ignores it.
The Julian date functions ignore leap years. I suppose that's forgivable, but
the book should point it out. Some unsuspecting and trusting programmer might
just try to use some of this stuff.
The nth_token function returns the '\0' character when the function is clearly
defined to return a character pointer.
Finally, the discussion of function overloading confuses long and int. The
examples of operator overloading use strcat improperly.
If you are not already a veteran C programmer, stay away from this book -- it
sets bad examples. If you are one, I can recommend it for its textual content.
After recognizing the sloppy code and code conventions, I made allowances and
proceeded to enjoy the work. It does what I wanted it to do. It clearly
explains the C++ extensions in ways that I can understand. It is a shame that
the code examples are so bad. They mar an otherwise excellent book.
The book's zeal for C++ clouds its objectivity. In several places it attempts
to justify the language and the object-oriented extensions that it brings to C
by citing advantages of the new and disadvantages of the old. And in every one
of these cases there are adjacent advantages in C and adjacent disadvantages
in C++. Perhaps Berry does not understand C as well as he should.


The Worth of the C++ Extensions


As I get further into C++ and the object-oriented paradigm, the real advantage
they offer to the C programmer becomes evident. The OOPs proponents might not
agree, but I believe that what C++ offers is not a better way to declare and
define application-specific objects of information, but a marvelous method for
extending the C language with advanced and complex data types.
C and C++ are language environments with no intrinsic input/output operations.
Early on, C programmers evolved a standard library of functions that manage
file and console I/O. Other standard functions came along as well -- string
handlers, math, memory management, and so on. C++, as a superset of ANSI C,
inherits all those functions, but it has the potential for another layer of
standard extensions to the language. C++ lets you add data types -- objects.
This means that you can extend the language to include types that have
universal application to programming problems. For example, you might easily
define a string object that has most of the functionality of the string
operations that Basic programmers have always enjoyed. Other possible object
extensions are dollar fields, dates, names, and addresses. These extensions
are not complex object trees such as you might build in a custom database
application, but common data types that you can define in ways that everyone
can use. Besides the obvious ones just mentioned, there could be generic data
structures that most programmers use, such as linked lists, stacks, binary
trees, AVL trees, B-trees, and many others. C++, far more than C, is
extensible.
But beware. A programmer can let his imagination overload his ability,
overload the heck out of operators and functions in C++, and the result can be
the same kind of incomprehensible code that plagues everything from Cobol to
4GLs.
As C++ overtakes C as the language of choice, we will need C++ standards that
go beyond standard functions and type extensions. We need a set of guidelines
to use in the design of overloaded operators and functions, lest we all write
unreadable code. You could, for example, easily design an object that
overloads the plus operator to perform subtraction. That would be a silly
thing to do and is an exaggeration of the problem, but C++ offers the
potential for some really dense operations, and we need direction in the use
of this powerful new tool.


The Zortech C++ Compiler


Last year I began an occasional foray into C++ by using the Zortech C++
compiler. Zortech graciously provided the compiler, which is up to Version
1.07 now. It is a good product, being the descendent of the Datalite C
compiler. Zortech C++ includes a full ANSI C compiler as well as the C++
product.
The weaknesses in the Zortech package are in places that do not matter much to
me. The ZED editor is a visual joke, but I use Brief. The memory-resident help
system is buggy, will freeze your PC, and you should not use it. The MAKE
utility program blows up predictably when it finds certain compile errors. The
compiler, however, works great. It does not have an integrated development
environment after the fashion of Turbo C and QuickC, and it has no debugger.
You can compile with certain switches and link with the Microsoft LINK program
so that you can use Codeview. Unfortunately, the Turbo Debugger utility
program that makes Codeview programs work with TD does not work with Zortech
C++ programs, so I am denied the use of my favorite debugger.
I have read that Zortech C++ is not compatible with other C++ language
environments, but I do not yet know enough about C++ to judge, other than to
say that everything I have learned from BS and the Berry book have worked just
fine. Until there are standards for C++ and until the big boys publish C++
compilers, Zortech will do for me.


C++ Windows


To wring out my new C++ skills, I decided to attempt a simple language
extension and build a video window data type. Most of my PC programs involve
pop-up windows and menus, so the Window data type is a natural jumping-off
place.
Listing One is window.h, which defines the Window class and three derived
classes. In addition, window.h defines certain global values that are used by
these new classes. A C++ program that uses the new classes would include this
header file and would link to the object module that is compiled from Listing
Two.
Window.h and window.c create a new class, called "Window." Programs can now
use objects, which are instances of this class, just as they can use int,
float, struct, and the other standard C data types. A Window appears on the
screen when it comes into scope and disappears when it leaves scope, so a
using program does not open and close the window in the way it would with
traditional C programs. This convention imposes a certain amount of structure
on programs that use the Window type, not altogether a bad idea.
C++ uses constructor and destructor functions that accompany a new class. The
constructor automatically executes when the class comes into scope, and the
destructor function executes when the class goes out of scope. The Window's
constructor function establishes and displays the window, and the destructor
function erases it. When you declare an object as an instance of the class,
you include parameters in parentheses that the constructor function uses to
construct the object. To declare a Window, you use this format:
 Window wname(lf,tp,rt,bt,fg,bg);
The first four parameters are character coordinates of the four corners of the
Window, with the top left corner of the screen being 0,0. The last two
parameters are enum color constants as defined in window.h that specify the
foreground and background colors of the window.
A class definition specifies the private and public parts of the class. An
evolving convention has the data values of the class in the private part and
the functions that operate on the class (called "methods" in OOP-speak,
"member functions" in C++ terminology) in the public part. Only the class's
member functions can read and write the private parts (sorry, but that's what
the C++ founders call them) of the class.
The Window's private parts include its colors, screen position, pointers to
video memory save buffers, cursor position, and a pointer to a body of text
that can be assigned to the Window. The public parts include the constructor
and destructor functions, a function to assign an optional title to the
Window, overloaded << operators to write characters, lines, and blocks of text
to the Window, and functions to manipulate the text in a Window. The comments
in window.h tell you what these functions are.


Derived Window Classes



Window.h and window.c contain three classes derived from the Window class. In
C++ a derived class inherits the characteristics of the class from which it is
derived. The three derived classes provide for Notice Windows, Error Windows,
and a YesNo Window. These utility Windows pop up, display a message, wait for
a keystroke, and pop down. In each case the Window displays the initializing
message that is specified when the class is declared and waits for a keystroke
before it goes away. The YesNo Window requires a Y or N keystroke and returns
a true value if the key is the Y. The difference between the other two are the
Window colors. As published, the Error Window is red with blinking yellow
letters, and the Notice Window has black letters on a cyan background. YesNo
Windows are white on green.


Console Functions


As with all such systems, we must deal with the console hardware. I have
defined a group of functions that manage the screen displays, cursor, and the
keyboard. Most of them have corresponding functions in the Zortech display
library. Listing Three , console.h contains macros for the Zortech functions
and prototypes for the others. Listing Four is console.c, which contains the
functions that we need but that are not represented by Zortech display
functions. Users of other compilers must substitute macros or functions for
those supported by Zortech functions. Following are the console management
functions needed by the Window class and provided either as C functions in
console.c or macros to Zortech functions in console.h:
hidecursor -- This function hides the cursor so that its position still
affects text displays but the cursor itself is invisible.
unhidecursor -- This function makes the cursor visible again.
savecursor -- This function saves the cursor's current configuration on a
stack.
restorecursor -- This function restores the cursor's configuration to the one
most recently pushed on the savecursor stack.
getkey -- This function gets a keystroke by avoiding DOS calls to avoid
Ctrl-Break interrupts on MS-DOS computers. It also translates function
keypresses into unique 8-bit values as defined in console.h.
initconsole and closeconsole -- These functions initialize and close the
console functions. Many video packages require such functions for setting up
system parameters and flushing buffers.
savevideo and restorevideo -- These functions transfer video memory characters
to and from a save buffer. They are used to save and restore what a window
covers when it comes into scope. The function parameters specify the buffer
address and the window's character corner coordinates.
box -- This function draws a single-line box on the screen. The Window class
uses it to make a window border.
colors -- This function establishes the foreground and background colors for
subsequent screen displays.
setcursor -- This function positions the cursor to the specified character
coordinates.
window_printf and window_putc -- These are window-oriented versions of printf
and putchar.
videoscroll -- This is a window scrolling function. Its parameters include the
direction and number of lines to scroll and the character corner coordinates.


Look.c


The program in Listing Five (page xx) is look.c. It demonstrates the Window
classes with a procedure that lets you view a text file in a full-screen
Window. Look.c begins by calling the set_new_handler function, a BS technique
supported by Zortech that calls your function get when the new operator cannot
get any more memory from the free store. Next, the program declares a Window
that occupies the entire screen and uses the title method to write a title in
the Window's top border. The program sets up a filebuf object, opens a file by
using the name on the command line, and associates an instream object with the
filebuf.
The C++ stream convention for files is one that I find less than intuitive.
Buffers are objects and streams are objects. You declare a buffer and tell it
to open a file. Then you declare a stream and associate it with the buffer. To
read and write the file, you send messages to the methods of the stream. It
seems to me that all that could be done with one object making it simpler and
easier to understand. But because the two-object convolution is a BS
technique, compiler vendors will, no doubt, perpetuate it forevermore in the
name of conformity.
The look.c program reads all the lines of text from the file, puts each line
in a new buffer, and puts the addresses of the buffers into an array of
character pointers. Then the text goes to the Window by way of the overloaded
<< operator.
To demonstrate the use of the YesNo derived class, the program uses a YesNo
window to ask if you want to continue. If so, the program calls the page
member function to let you scroll and page through the text.
The program uses Error objects to tell you that there was no file name on the
command line or that it cannot find the file that you specified.


A Long Way Up the C++ Slope


The learning curve I climbed to get these window extensions working was steep
indeed, and it was made worse by the dearth of good OOPs and C++ tutorial
literature. Things would have been smoother if the Berry book had been
available for the C++ introduction, but what was really missing was a decent
explanation of the OOP paradigm. To fill that gap for you I am about to break
tradition for the "C Programming" column and recommend that you beg, borrow,
or buy Turbo Pascal 5.5. It does not matter to me if you never use the
software; the little book that explains the object-oriented extensions to TP
is what you want. It is head and shoulders above any introduction to OOPs I
have seen so far.
Next month we'll use the Window class as a base from which to derive some menu
classes, thus getting deeper into the inheritance features of C++.


Availability


All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063; or call 800-356-2002 (from inside
Calif.) or 800-533-4372 (from outside Calif.). Please specify the issue and
format (MS-DOS, Macintosh, Kaypro).


_C PROGRAMMING COLUMN_
by Al Stevens



[LISTING ONE]



// -------------- window.h

#ifndef WINDOWS
#define WINDOWS


// ---------- screen dimensions
#define SCREENWIDTH 80
#define SCREENHEIGHT 25

// --------- atrribute values for colors
enum color {
 BLACK, BLUE, GREEN, CYAN, RED, MAGENTA, BROWN, LIGHTGRAY,
 GRAY, LIGHTBLUE, LIGHTGREEN, LIGHTCYAN, LIGHTRED,
 LIGHTMAGENTA, YELLOW, WHITE, BLINK = 128
};

// ------------ spaces per tab stop (text displays)
#define TABS 4
// ------------ color assignments for window types
#define YESNOFG WHITE
#define YESNOBG GREEN
#define NOTICEFG BLACK
#define NOTICEBG CYAN
#define ERRORFG (YELLOW BLINK)
#define ERRORBG RED

// ------------ a video window
class Window {
 unsigned bg, fg; // window colors
 unsigned lf,tp,rt,bt; // window position
 unsigned *wsave; // video memory save buffer
 unsigned *hsave; // hide window save buffer
 unsigned row, col; // current cursor row and column
 int tabs; // tab stops, this window
 char **text; // window text content
public:
 Window(unsigned left, unsigned top,
 unsigned right, unsigned bottom,
 color wfg, color wbg);
 ~Window(void);
 void title(char *ttl);
 Window& operator<<(char **btext);
 Window& operator<<(char *ltext);
 Window& operator<<(char ch);
 void cursor(unsigned x, unsigned y);
 void cursor(unsigned *x, unsigned *y)
 { *y = row, *x = col; }
 void clear_window(void);
 void clreos(void); // clear to end of screen
 void clreol(void); // clear to end of line
 void hidewindow(void); // hide an in-scope window
 void restorewindow(void); // unhide a hidden window
 void page(void); // page through the text
 void scroll(int d); // scroll the window up, down
 void set_colors(int cfg, int cbg) // change the colors
 { fg = cfg, bg = cbg; }
 void set_tabs(int t) // change the tab stops
 { if (t > 1 && t < 8) tabs = t; }
};

// ---------- utility notice window
class Notice : Window {
public:

 Notice(char *text);
 ~Notice(){}
};

// ---------- utility yes/no window
class YesNo : Window {
public:
 YesNo(char *text);
 ~YesNo(){}
 int answer;
};

// ---------- utility error window
class Error : Window {
public:
 Error(char *text);
 ~Error(){}
};

#define max(x,y) (((x) > (y)) ? (x) : (y))
#define min(x,y) (((x) > (y)) ? (y) : (x))

#endif






[LISTING TWO]

// -------------- window.c

// A C++ window library

#include <stddef.h>
#include <string.h>
#include <ctype.h>
#include "window.h"
#include "console.h"

#define HEIGHT (bt - tp + 1)
#define WIDTH (rt - lf + 1)

// ------- constructor for a Window
Window::Window(unsigned left, unsigned top, // 0 - 79, 0 - 24
 unsigned right, unsigned bottom,
 color wfg, color wbg)
{
 savecursor();
 initconsole();
 hidecursor();
 // ----- adjust for windows beyond the screen dimensions
 if (right > SCREENWIDTH-1) {
 left -= right-(SCREENWIDTH-1);
 right = SCREENWIDTH-1;
 }
 if (bottom > SCREENHEIGHT-1) {
 top -= bottom-(SCREENHEIGHT-1);

 bottom = SCREENHEIGHT-1;
 }
 // ------- initialize window dimensions
 lf = left;
 tp = top;
 rt = right;
 bt = bottom;
 // ------- initialize window colors
 fg = wfg;
 bg = wbg;
 // ------- initialize window cursor and tab stops
 row = col = 0;
 tabs = TABS;
 // ---------- save the video rectangle under the new window
 wsave = new unsigned[HEIGHT * WIDTH];
 hsave = NULL;
 savevideo(wsave, tp, lf, bt, rt);
 // --------- draw the window frame
 box(tp, lf, bt, rt, fg, bg);
 // -------- clear the window text area
 clear_window();
 unhidecursor();
}

// ------- destructor for a Window
Window::~Window(void)
{
 // ----- restore the video RAM covered by the window
 restorevideo(wsave, tp, lf, bt, rt);
 delete wsave;
 if (hsave != NULL)
 delete hsave;
 restorecursor();
}

// ------- hide a window without destroying it
void Window::hidewindow(void)
{
 if (hsave == NULL) {
 hsave = new unsigned[HEIGHT * WIDTH];
 savevideo(hsave, tp, lf, bt, rt);
 restorevideo(wsave, tp, lf, bt, rt);
 }
}

// --------- restore a hidden window
void Window::restorewindow(void)
{
 if (hsave != NULL) {
 savevideo(wsave, tp, lf, bt, rt);
 restorevideo(hsave, tp, lf, bt, rt);
 delete hsave;
 hsave = NULL;
 colors(fg,bg);
 }
}

// -------- add a title to a window
void Window::title(char *ttl)

{
 setcursor(lf + (WIDTH - strlen(ttl) - 1) / 2, tp);
 colors(fg, bg);
 window_printf(" %s ", ttl);
 cursor(col, row);
}

// ------- write text body to a window
Window& Window::operator<<(char **btext)
{
 cursor(0, 0);
 text = btext;
 if (*btext != NULL)
 *this << *btext++;
 while (*btext != NULL && row < HEIGHT-3)
 *this << '\n' << *btext++;
}

// -------- write a line of text to a window
Window& Window::operator<<(char *ltext)
{
 while (*ltext && col < WIDTH - 2 && row < HEIGHT - 2)
 *this << *ltext++;
 return *this;
}

// -------- write a character to a window
Window& Window::operator<<(char ch)
{
 cursor(col, row);
 switch (ch) {
 case '\n':
 clreol();
 if (row == HEIGHT-3)
 scroll(1);
 else
 row++;
 case '\r':
 col = 0;
 break;
 case '\b':
 if (col)
 --col;
 break;
 case '\t':
 do
 *this << ' ';
 while (col % tabs);
 break;
 default:
 if (col == WIDTH - 2)
 *this << '\n';
 colors(fg,bg);
 window_putc(ch);
 col++;
 return *this;
 }
 cursor(col, row);
 return *this;

}

// ----- position the window cursor
void Window::cursor(unsigned x, unsigned y)
{
 if (x < WIDTH-2 && y < HEIGHT-2) {
 setcursor(lf+1+x, tp+1+y);
 row = y;
 col = x;
 }
}

// ------ clear a window to all blamks
void Window::clear_window(void)
{
 cursor(0,0);
 clreos();
}

// --- clear from current cursor position to end of window
void Window::clreos(void)
{
 unsigned rw = row, cl = col;
 clreol();
 col = 0;
 while (++row < HEIGHT-2)
 clreol();
 row = rw;
 col = cl;
}

// --- clear from current cursor position to end of line
void Window::clreol(void)
{
 unsigned cl = col;
 colors(fg,bg);
 while (col < WIDTH-2)
 *this << ' ';
 col = cl;
}

// ----- page and scroll through the text file
void Window::page(void)
{
 int c = 0, lines = 0;
 char **tx = text;

 hidecursor();
 // ------ count the lines of text
 while (*(tx + lines) != NULL)
 lines++;
 while (c != ESC) {
 c = getkey();
 char **htext = text;
 switch (c) {
 case UP:
 if (tx != text) {
 --tx;
 scroll(-1);

 unsigned x, y;
 cursor(&x, &y);
 cursor(0, 0);
 *this << *tx;
 cursor(x, y);
 }
 continue;
 case DN:
 if (tx+HEIGHT-3 < text+lines-1) {
 tx++;
 scroll(1);
 unsigned x, y;
 cursor(&x, &y);
 cursor(0, HEIGHT-3);
 *this << *(tx + HEIGHT - 3);
 cursor(x, y);
 }
 continue;
 case PGUP:
 tx -= HEIGHT-2;
 if (tx < text)
 tx = text;
 break;
 case PGDN:
 tx += HEIGHT-2;
 if (tx+HEIGHT-3 < text+lines-1)
 break;
 case END:
 tx = text+lines-(HEIGHT-2);
 if (tx > text)
 break;
 case HOME:
 tx = text;
 break;
 default:
 continue;
 }
 *this << tx;
 text = htext;
 clreos();
 }
 unhidecursor();
}

// --------- scroll a window
void Window::scroll(int d)
{
 videoscroll(d, tp+1, lf+1, bt-1, rt-1, fg, bg);
}

// ------ utility notice window
Notice::Notice(char *text)
 : ((SCREENWIDTH-(strlen(text)+2)) / 2, 11,
 ((SCREENWIDTH-(strlen(text)+2)) / 2) + strlen(text)+2,
 14, NOTICEFG, NOTICEBG)
{
 *this << text << "\n Any key ...";
 hidecursor();
 getkey();

 unhidecursor();
 hidewindow();
}

// ------ utility error window
Error::Error(char *text)
 : ( (SCREENWIDTH-(strlen(text)+2)) / 2, 11,
 ((SCREENWIDTH-(strlen(text)+2)) / 2) + strlen(text)+2,
 14, ERRORFG, ERRORBG)
{
 *this << text << "\n Any key ...";
 hidecursor();
 getkey();
 unhidecursor();
 hidewindow();
}

// ------ utility yes/no window
YesNo::YesNo(char *text)
 : ( (SCREENWIDTH-(strlen(text)+10)) / 2, 11,
 ((SCREENWIDTH-(strlen(text)+10)) / 2) + strlen(text)+10,
 13, YESNOFG, YESNOBG)
{
 *this << text << "? (Y/N) ";
 int c = 0;
 hidecursor();
 while (tolower(c) != 'y' && tolower(c) != 'n')
 c = getkey();
 unhidecursor();
 hidewindow();
 answer = tolower(c) == 'y';
}








[LISTING THREE]

/* ----------- console.h -------- */

#ifndef CONSOLE
#define CONSOLE

#include <disp.h>

// -------- cursor and keyboard functions (via BIOS)
void hidecursor(void);
void unhidecursor(void);
void savecursor(void);
void restorecursor(void);
int getkey(void);

// -------- key values returned by getkey()
#define BELL 7
#define ESC 27

#define UP 200
#define BS 203
#define FWD 205
#define DN 208
#define HOME 199
#define END 207
#define PGUP 201
#define PGDN 209

#define attr(fg,bg) ((fg)+(((bg)&7)<<4))

// --------- video functions (defined as Zortech C++ equivalents)
#define initconsole() disp_open()
#define closeconsole() disp_flush()
#define savevideo(bf,t,l,b,r) disp_peekbox(bf,t,l,b,r)
#define restorevideo(bf,t,l,b,r) disp_pokebox(bf,t,l,b,r)
#define box(t,l,b,r,fg,bg) disp_box(1,attr(fg,bg),t,l,b,r)
#define colors(fg,bg) disp_setattr(attr(fg,bg))
#define setcursor(x,y) disp_move(y,x)
#define window_printf disp_printf
#define window_putc disp_putc
#define videoscroll(d,t,l,b,r,fg,bg) \
 disp_scroll(d,t,l,b,r,attr(fg,bg));


#endif








[LISTING FOUR]

/* ----------- console.c --------- */

/* PC-specific console functions */

#include <dos.h>
#include <conio.h>
#include "console.h"

/* ------- video BIOS (0x10) functions --------- */
#define VIDEO 0x10
#define SETCURSORTYPE 1
#define SETCURSOR 2
#define READCURSOR 3
#define HIDECURSOR 0x20

#define SAVEDEPTH 20 /* depth to which cursors are saved */

static int cursorpos[SAVEDEPTH];
static int cursorshape[SAVEDEPTH];
static int sd;

union REGS rg;


/* ---- Low-level get cursor shape and position ---- */
static void getcursor(void)
{
 rg.h.ah = READCURSOR;
 rg.h.bh = 0;
 int86(VIDEO,&rg,&rg);
}

/* ---- Save the current cursor configuration ---- */
void savecursor(void)
{
 getcursor();
 if (sd < SAVEDEPTH) {
 cursorshape[sd] = rg.x.cx;
 cursorpos[sd++] = rg.x.dx;
 }
}

/* ---- Restore the saved cursor configuration ---- */
void restorecursor(void)
{
 if (sd) {
 rg.h.ah = SETCURSOR;
 rg.h.bh = 0;
 rg.x.dx = cursorpos[--sd];
 int86(VIDEO,&rg,&rg);
 rg.h.ah = SETCURSORTYPE;
 rg.x.cx = cursorshape[sd];
 int86(VIDEO,&rg,&rg);
 }
}

/* ---- Hide the cursor ---- */
void hidecursor(void)
{
 getcursor();
 rg.h.ch = HIDECURSOR;
 rg.h.ah = SETCURSORTYPE;
 int86(VIDEO,&rg,&rg);
}

/* ---- Unhide the cursor ---- */
void unhidecursor(void)
{
 getcursor();
 rg.h.ch &= ~HIDECURSOR;
 rg.h.ah = SETCURSORTYPE;
 int86(VIDEO,&rg,&rg);
}

/* ---- Read a keystroke ---- */
int getkey(void)
{
 rg.h.ah = 0;
 int86(0x16,&rg,&rg);
 if (rg.h.al == 0)
 return (rg.h.ah 0x80) & 255;
 return rg.h.al & 255;
}








[LISTING FIVE]

// ---------- look.c

// A C++ program to demonstrate the use of the window library.
// This program lets you view a text file

#include <stdio.h>
#include <string.h>
#include <stream.hpp>
#include <stdlib.h>
#include "window.h"

#define MAXLINES 200 // maximum number of text lines

static char *wtext[MAXLINES+1]; // pointers to text lines

// --- taken from BS; handles all free store (heap) exhaustions
void out_of_store(void);
typedef void (*PF)();
extern PF set_new_handler(PF);

main(int argc, char *argv[])
{
 set_new_handler(&out_of_store);
 if (argc > 1) {
 // ---- open a full-screen window
 Window wnd(0,0,79,24,CYAN,BLUE);
 char ttl[80];
 // ------ put the file name in the title
 sprintf(ttl, "Viewing %s", argv[1]);
 wnd.title(ttl);
 filebuf buf;
 if (buf.open(argv[1], input)) {
 istream infile(&buf);
 int t = 0;
 // --- read the file and load the pointer array
 char bf[120], *cp = bf;
 while (t < MAXLINES && !infile.eof()) {
 infile.get(*cp);
 if (*cp != '\r') {
 if (*cp == '\n') {
 *cp = '\0';
 wtext[t] = new char [strlen(bf)+1];
 strcpy(wtext[t++], bf);
 cp = bf;
 }
 else
 cp++;
 }
 }
 wtext[t] = NULL;

 // ---- write all the text to the window
 wnd << wtext;
 // ---- a YesNo window
 YesNo yn("Continue");
 if (yn.answer)
 wnd.page();
 // ------ a Notice window
 Notice nt("All done.");
 }
 else
 // ------ error windows
 Error err("No such file");
 }
 else
 Error err("No file name specified");
}

// ----- the BS free-store exhaustion handler
void out_of_store(void)
{
 cerr << "operator new failed: out of store\n";
 exit(1);
}







































September, 1989
STRUCTURED PROGRAMMING


Admitting Objects to Pascal




Jeff Duntemann, K16RA


I graduated from Lane Technical High School in Chicago in 1970. Three years
later they first admitted girls. I had an opportunity to go back seven or
eight years after graduation to speak to one of the student groups, and the
change stopped me in my tracks.
Girls! What a thought ...
The sense of balance, of progress, or rightness was astounding. The place was
no longer permeated by that old testosterone fog. Right there in front of me,
gawky boy-nerds were walking down the hall in animated conversation with gawky
girl-nerds. It made me wonder why this hadn't been done fifty years earlier,
and for a moment I ached to be fourteen again.
Over the past couple of months, both Borland and Microsoft have admitted
objects to the Pascal language, and the effect is much the same. It feels
right, right in a way that makes you want to turn the clock back to 1972 and
start all over again.
Nonetheless, having objects now is sufficient. And I promise you, programming
in Pascal will never be the same again.


QuickPascal


Microsoft's QuickPascal has finally shipped, and at $99 list (street price
considerably less, if you follow the mail order houses) it now ties with
Smalltalk/V as the least expensive DOS OOP product.
QuickPascal's environment is quite slick, with mouse support and a multifile
editor its main features. There are a source-level integrated debugger and a
very good on-line help system. Users of MS Pascal 5.0 should note that
QuickPascal is only very broadly compatible with MS Pascal, but instead is a
syntactic clone of Turbo Pascal 5.0. All Turbo Pascal 5.0 code I've tried to
compile under QuickPascal has compiled and run correctly, but ...
QuickPascal's .EXE file is considerably larger than Turbo Pascal's, and the
size of the largest compilable program under Quick is correspondingly smaller.
The two compilers' object-oriented features are different enough to be a
nuisance but similar enough to treat together.


Parallel Evolution


Considering that the two products were developed in isolation, Microsoft's
QuickPascal and Borland's Turbo Pascal 5.5 have a remarkably similar slant on
object orientation. Their differences can be attributed to ancestry:
QuickPascal adheres pretty closely to the Apple Object Pascal specification,
whereas Turbo Pascal 5.5 draws about equally on Object Pascal and C++. In an
interlocking directorate like this, similarities are likely to outnumber
differences.
For both products, objects are closely related to records and are defined in
just about the same way. The important thing to notice about object
definitions is that code and data are defined together, within a named
structure. This is how object-oriented Pascal implements encapsulation:
TYPE
 PrintByte =
 OBJECT
 BV: Byte; { Byte Value }
 FUNCTION
 DontPrint(ShouldI : Byte): Boolean;
 PROCEDURE PrintDecimal;
 PROCEDURE PrintHex;
 PROCEDURE PrintBinary;
 PROCEDURE PrintSymbol;
 END;
The new reserved word OBJECT makes this an object definition. Referencing code
and data is done through another record-like convention, dotting. Assuming an
object instance named MyByte, you would reference field BV as MyByte.BV, and
call the PrintHex method as MyByte. PrintHex.


Object Types with No Class


Adam gave all the animals unique names because when he said, "Hey you with the
four legs and a tail!" half of creation ambled over. The naming of names is a
critical issue in object-oriented programming, because so many of the concepts
are new. Sadly, there are major disparities in how QuickPascal and Turbo
Pascal refer to certain concepts.
The worst and most confusing is what to call objects themselves. Just as
variable types and variables are not the same things, objects and their types
are not the same either. An object type in Turbo Pascal is simply an "object
type," and an object variable (what we also call an instance of an object
type) is just an "object." QuickPascal uses the term "class" instead of
"object type," but the meaning is exactly the same. The term "class" comes
from Smalltalk, as I described last month. In this column I'll be using the
term "object type" rather than "class" in this column (regardless of which
Pascal is under discussion) because it's more descriptive and sounds less
mystical/magical.


Creating Objects


Turbo and Quick diverge most sharply in their means of creating objects. Turbo
Pascal objects are true to their record-like heritage. You can create an
object in the same ways you create a record, either as a static variable in
the data segment, or as a dynamic variable on the heap:
 VAR

 { Declaring a static object: }
 MyByte: PrintByte;
 { A pointer to a dynamic object: }
 MyBytePtr: ^PrintByte;

 { Allocating a dynamic object: }
 New(MyBytePtr);

These items could as well be records as objects. The syntax is the same.
QuickPascal implements objects as something truly different. An object
instance variable may exist only on the heap; there is no way to create an
object in the data segment. Syntactically, object variables sit on the fence
half-way between records and pointers. An object is treated like a record when
defined, treated like a pointer when created, and then treated like a record
when used, with a couple of twists:
 VAR
 { Allocates no space!! }
 QuickByte: PrintByte;

 { Allocates object on heap: }
 New(QuickByte);

 { Reference without caret: }
 QuickByte.BV := 42;
This is supposed to be a sort of short-hand, allowing programmers to dispense
with the multitude of caret symbols that would be necessary if objects on the
heap really were treated as pointer referents. And while shorthand is
convenient, it's hard to read, and I'm afraid this somewhat inconsistent
syntax is something (like irregular verbs in Spanish) that you just have to
bite the mnemonic bullet and remember.
Once a Quick object is created, it can be referenced just as a Turbo Pascal
object can be. The inconsistency returns when you want to reclaim an object's
heap space. You pass the object's identifier to Dispose, again as though the
object were a pointer:
 Dispose(QuickByte);


Overriding Methods


Creating a child object type that inherits from a parent object type is done
identically in both implementations. The name of the parent type is enclosed
in parentheses after the child type's OBJECT reserved word:
 TYPE
 PrintByteHP =
 OBJECT(PrintByte)
 PROCEDURE Print Symbol
 {; OVERRIDE;}
 END;
The differences between the two products emerge again when a method defined in
the parent object type is redefined, or overridden, by a new method with the
same name. Turbo Pascal simply allows the method to be overridden by
redefining it. QuickPascal requires that the overriding method be flagged to
the compiler with the OVERRIDE reserved word. I kind of like that: It makes
the code easier to comprehend by flagging unmistakably whether a method is
brand new in the object hierarchy, or whether it redefines a method farther up
the hierarchy tree.


In Search of Self


The one thing that drives me truly berserk about QuickPascal is its
Selfishness. Within the body of a method, the object's data fields need to be
qualified by the use of a new predefined identifier, Self. Here's a simple
QuickPascal method definition belonging to the PrintByte object type:
 PROCEDURE PrintByte.
 PrintDecimal;

 BEGIN
 Write(LST,Self.BV:3);
 END;
This is peculiar -- traditional Pascal scoping should allow the compiler to
recognize a reference to an object's fields as local variables. After all, the
method definition begins with the PrintByte. qualifier, and field BV is as
much a part of PrintByte as the PrintDecimal method. Most remarkably, using
the WITH Self DO structure does not work with methods. (It does work with
object data fields, however.) Methods must be qualified by Self itself,
directly. This may be a bug or it may be a feature, depending on how you look
at it, but every time I look, I see six legs.
Yet another interesting wrinkle here is that Turbo Pascal allows, but does not
require, the use of the Self identifier in the same context.


A Simple Object Pascal Example


This column's listings present a simple but useful example of Object Pascal in
action. I've given it for both Turbo Pascal 5.5 and QuickPascal, and comparing
the two will give you a reasonable feel for the way the two implementations go
about things. Listings One and Two are for Turbo Pascal, and Listings Three
and Four are for QuickPascal.
The program itself prints an ASCII table, including all 256 character patterns
from the smiley faces on up through the box drawing and math characters. The
catch in creating such a program is that while many PC-compatible printers can
print a smiley face or its brothers, the nature of the command to print such
special characters varies all over the map.
The PRINBYTE.PAS (and PRINTBYTQ.PAS for QuickPascal) implements a simple
object that is nothing more than a byte value with some methods to display it
to the printer in various formats, including decimal, hex, binary, and
"literal," that is, in the form of a single character. The value is stored in
the PrintByte object's sole data field, BV. The value-added in the PrintByte
object lies in the object's methods.
Most ASCII characters can be printed as literal symbols simply by sending them
to the printer port. Send a 65 value and you'll get an A on the printer. Send
the value 236 and you'll get an infinity symbol. Some characters, however, are
control characters, and exist not to be displayed so much as to dictate
commands to the printer. These control characters include carriage return
(13), line feed (10), and BEL (7). Send a control character to the printer and
you won't see any symbol, but instead the printer will index or beep or do
something else that involves its mechanism rather than its output.
As you might imagine, some characters do double duty as both control
characters and symbols. The control character guise of such characters
(typically, values 1-31) takes priority. Send a 7 to the printer, and the
printer's beeper will beep. To get a hardcopy of the small circular bullet
that is character 7's symbolic guise, you must send a control sequence to the
printer. Such a control sequence is often an ESC character (27) followed by
the control character, but each printer has its own way of responding to
control sequences, and no standard prevails.



Go Print Yourself!


Printing the ASCII chart on any arbitrary printer would mean a program that
knew about every printer on the market, which is obviously impossible. The way
out is for the PrintByte object to take the low road and refuse to print any
control character at all -- and let the user of the PrintByte object extend
PrintByte to match the user's specific printer. The example shown uses the HP
LaserJet II printer as the specific printer. You can modify (and should, for
practice) ASCII.PAS or ASCIIQ.PAS to match the needs of your own printer,
whatever it might be.
Extendability of code is one of the major benefits of object-oriented
programming. The way is to define a child object of the PrintByte object, and
override only the method that prints the symbol to the printer. This is the
PrintSymbol method.
ASCII.PAS and ASCIIQ.PAS (for QuickPascal) demonstrates this. A child object
type, PrintByteHP, is defined so that it inherits all of PrintByte's data and
methods. PrintByteHP changes only what must be changed -- no unnecessary
duplication of code is needed. The only change is to the PrintSymbol method,
which is redefined in ASCII.PAS and ASCIIQ.PAS. Note the presence of the
OVERRIDE reserved word in ASCIIQ.PAS.
The sense of the PrintByteHP object type is that it extends the PrintByte
object type by "knowing" how to print control symbols to the LaserJet II
printer. (The PrintByte type simply punts by printing a space for any control
character.) So you as a programmer don't have to fuss with control sequences.
You simply tell a particular PrintByteHP object to go print itself, and
shazam! You've got smiley faces all over the place.


Look Ma, No Source!


One of the interesting benefits of this process is that an object can be
extended by a programmer without having the source code. I provide you with
PRINBYTE.PAS, but all you really need is the object type's definition
(basically, the procedure headers and data definitions) and the linkable unit
file implementing the object. This can be done to some extent with Pascal
units right now, in that you can write a different version of a routine in a
unit, and as long as your rewritten routine is linked after the routine it
replaces, the replacement routine will be used by all code linked afterwards.
The real, real slick wrinkle (not shown in this example) is the way that the
original object can make use of its child objects that did not exist when the
parent object was compiled. (Think about that for a moment.) It's part of the
mind-warping notion of polymorphism, and I'll return with some concrete
examples in a future column.
In the meantime, I powerfully recommend that you pick up either Turbo Pascal
5.5 or QuickPascal and start practicing your objects now.
Nothing more important has happened to programming languages since procedures
parted the in-line code chaos in the seventies.


Blows Against the Empire


As my sixties Day-Glo Desiderata poster used to say, "No doubt the universe is
unfolding as it should." Those ubiquitous 370-class mainframes (as distinct
from what are now called -- gakkh! -- supercomputers) are evolving into their
appropriate ecological niche as relatively dumb but cavernous file servers
under the control of personal computers out where the real work gets done.
Not surprisingly, your average MIS/DP department doesn't see things this way.
(Just as T. Rex didn't say, "Hi, Boss!" to the shrew with egg yolk on his
whiskers.) Ask DP for a mainframe application, and the schedule will be
something like six months plus a fudge factor of three months to two years,
depending on how recently the MIS manager got a promotion. I can smile now
(having been out of Xerox MIS for six years) but the DP backlog is still a
serious problem out in pinstripe land.
And I don't normally have much truck with teenage mutant Ninja languages, but
one has recently come to light that has landed some serious blows against the
DP empire. It's called REXX, and if you're in the position of living in the
walls of T. Rex's nest, it's a great way of making omelets out of the big guy.
REXX is a structured language developed originally for 370-class mainframes by
a delightful IBM curmudgeon named Mike Cowlishaw. It was designed as a
replacement for the mainframe batch processor cancer they call JCL, and has
therefore been called a "super batch language." REXX can be used as a batch
language, but it's a real programming language in its own right with all the
body parts a proper language ought to have. (Including GOTO -- now that's
authentic!) It's most similar to Basic in that it's usually interpreted and
has that inescapable line-oriented mainframe flavor about it.
REXX is important to this column: It has a crisp definition published by
Cowlishaw that has enabled identical implementations on both mainframes and on
PCs under DOS. This means that you can build a REXX application in the safety
and comfort of your cubicle, and upload it to your mainframe users when T. Rex
isn't looking. Shazam! You've cut the development time for a simple
readafile/writearecord DP application from ten months to ten hours.
There are numerous mainframe implementations of REXX, and I'm told one can be
had for almost any of the innumerable different 370 operating environments.
The only commercial implementation for DOS is the one I tested and will
describe later, Personal REXX from The Mansfield Software Group.


With Feet in Two Worlds


I've not had the honor of sweating blood over the VM/CMS mainframe operating
system, so I can't tell you how REXX looks from its perspective. From DOS,
however, REXX is very much a language with feet in two worlds. On one hand, as
I've said, REXX is a perfectly ordinary and reasonable interpreted language.
All the familiar program structures are there and implemented quite nicely. On
the other hand, when REXX encounters a clause it does not recognize as part of
a REXX statement, it automatically passes that clause to DOS to execute. This
ability is what makes REXX a super-batch language.
Mansfield allows REXX to be loaded from disk as a transient program that
remains in memory only while the current REXX program is being interpreted, or
REXX can be loaded as a TSR that remains in memory until unloaded or until
reboot. The TSR REXX takes 150K of RAM, which prevents its use with many of
the larger business applications. My hunch is that memory management is the
real kicker in making REXX pay off under DOS. You'll have to experiment to get
a feel for what's possible and what's impossible.
REXX as a batch processor doesn't interest me nearly as much as REXX as a
mainframe bridge, and as a mainframe bridge it won't be managing DOS
applications.


Language Highlights


Space is short, so I won't rehash how REXX implements ordinary structured
elements like DO loops, IF statements, and so on. They're all there, and
resemble Pascal's strongly. REXX does contain some remarkable features that I
would like to call attention to. For the full story on REXX, get Cowlishaw's
"white book" on the language, which defines it thoroughly and is one of the
most readable such white books as well: The REXX Language, M.F. Cowlishaw,
Prentice-Hall, 1985. (Note that The Mansfield Software Group bundles
Cowlishaw's book with their Personal REXX product, so if you buy the book and
then buy Personal REXX, you'll be buying the book twice.)
The PARSE instruction provides a generalized string parser that dismantles a
string value into several separate variables according to a template. A good
example is entry of date values:
 SAY "Enter the date as mm/dd/yy" PARSE PULL mo'/'da'/'yr
Here, the user is told what string data to enter with the SAY instruction. The
PULL instruction is roughly analogous to Readln, in that it pulls data in from
an input queue. The template is mo '/' da '/' yr, indicating that the string
data accepted through PULL is to be separated out into three string variables,
with slash symbols acting as separators.
This is about as simple as a useful template gets, and REXX's parser is
capable of a lot more. Patterns to be matched may be stored in variables and
may be looked for in specific positions in the string if desired. Parsers are
elemental code machines that should be a part of every language, but in fact
are part of almost none.
Perhaps the most intriguing REXX construct is INTERPRET, which allows a REXX
program to build a mini-REXX program on the fly and then interpret it.
Interpreted Prolog implementations can do something like this, but none of the
traditional languages (Pascal, C, Modula-2, Basic, Fortran, ADA) have anything
that comes even close. In his book, Cowlishaw does little with INTERPRET but
uses it to write a simple formula calculator emulator, but in fact this way of
treating code as data is used in a lot of AI research -- if any readers have
used REXX in this fashion, I'd like to hear about it.


That Old Mainframe Goblin


While REXX the language is easy to program in and relatively easy to read,
REXX the language processor is haunted by that old mainframe goblin,
complexity. There are any number of ways to feed statements to REXX, and lots
of niggling little gotchas that must be digested and understood to make the
program operate to its best advantage. Many of these seem unnecessary, like
the separate TSR that does nothing but grab interrupt vector 60. Grabbing
vectors is not time consuming and should be done in an initialization
section/exit procedure manner inside the language processor, as with Turbo
Pascal. A much simpler Personal REXX could be created if The Mansfield
Software Group decided to go the distance.
On the other hand, if you can (or must) handle mainframes, you can handle
REXX. Give it a shot.


Sometimes a Great Notion...



... never quite seems to set the world on fire as it should. I am personally
fond of technical anthologies; that is, books to which several experts on a
subject have each contributed a chapter. Such books are rare, and publishers
for some reason don't like them.
So you'll have to forgive me for recommending one such book here, even though
I was a (relatively minor) contributor. It's from a small publisher and has
had little distribution, but get it if you can: Turbo Pascal Innovations,
edited by Judie Overbeek and Rick Gessner. If you can't find it at a bookstore
(likely) the book can be ordered directly from the publisher.
The book contains eight chapters in all, each on a different subject by a
different author. The topics include (among others) DOS time and date topics,
directory management, high-precision code timing, graphics, and procedural
types. I don't have time to describe them all, but there are two absolutely
essential chapters: One by Rick Gessner on user-interface design, and another
by Lane Ferris describing a unit allowing you to write TSR programs that
multitask with ordinary DOS programs.


Products Mentioned


Turbo Pascal 5.5 Borland International 1800 Green Hills Road Scotts Valley, CA
95066 408-438-8400 Turbo Pascal $149 Turbo Pascal Professional (Includes Turbo
Debugger & Turbo Assembler) $250
Turbo Pascal Innovations Ed. by Judie Overbeek & Rick Gessner Rockland
Publishing, Inc., 1989 1706 Bison Drive Kalispell, MT 59901 406-756-9079 ISBN
0-939621-01-0 $32.95 (includes listings diskette)
QuickPascal 1.0 Microsoft 16011 NE 36th Way Redmond, WA 98073 206-882-8088
$99.00
The REXX Language M.F. Cowlishaw Prentice-Hall, 1985 ISBN 0-13-780685-X $19.95
Personal REXX The Mansfield Software Group PO Box 532 Storrs, CT 06268
203-429-8402 DOS version $150.00 OS/2 version (which includes DOS version)
$175.00
For all that user-interface design is touted as the key to improved user
productivity, there's been little published on just how such interface code is
designed. Gessner does an excellent job of providing some guidelines for user
interface components, along with sample implementations of various line
editors and menus. The code is solid and easy to read, and the approach
demonstrates a sensitivity to human needs that rarely turns up in the
programmer press.
The real blockbuster in this book, however, is Lane Ferris's multitasking TSR
unit. Code like this doesn't happen by very often, and when it does, it is
rarely opened up and dissected with such clarity. Lane's unit uses a
round-robin time-slicing system to allow a TSR to pre-emptively execute
multiple tasks while the DOS foreground program multitasks on equal terms.
This means you can put something as simple as a real-time clock in the corner
of your screen, or something as complex as a pop-up calculator that doesn't
freeze your telecommunications session in the foreground. Very heavy stuff,
and beautifully implemented.
Best of all, it's not written in C.
The reason books like this are valuable is that no one person is equally good
at everything, and by presenting the best that several intelligent persons
have to offer, the book becomes far more diverse and useful than a book any
single person could create. In that respect it's like a very thick magazine
with no ads. Nothing quite like it ... even though there should be.


When the Good Die Young


I was at the American Booksellers Association convention in Washington, D.C.
when I got the news that Kent Porter had died in the saddle, doing what both
he and I do every day: Writing and editing. I met Kent when he submitted a
raft of articles to TURBO TECHNIX, and acted as his editor there for an
outstanding series of projects, including his popular "Mouse Mysteries"
series. Later, when TURBO TECHNIX became history, the partitions swapped a
little and Kent became my editor here at DDJ. We understood one another in the
way that only two editors -- who have edited one another's work -- can.
Kent's mission and mine were the same: To wipe the fuzz away from your windows
onto the programming craft, and in doing so spread the magic around to the
uninitiated. Magazines and books don't just happen -- behind each is a strong
mind and a strong hand; writer and editor working together in a common cause.
It takes some skill to see the work of the editor's hand sometimes -- unless
you're another editor.
Good night, good friend. As to your mission: Stet.
It will stand.


Availability


All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063; or call 800-356-2002 (from inside
Calif.) or 800-533-4372 (from outside Calif.). Please specify the issue and
format (MS-DOS, Macintosh, Kaypro).


_STRUCTURED PROGRAMMING COLUMN_
by Jeff Duntemann



[LISTING ONE]

{--------------------------------------------------------------}
{ PrinByte }
{ }
{ Byte-value print object for object extendability demo }
{ }
{ by Jeff Duntemann }
{ Turbo Pascal V5.5 }
{ Last modified 5/11/89 }
{--------------------------------------------------------------}


UNIT PrinByte;

INTERFACE

USES Printer;



{--------------------------------------------------------------}
{ The PrintByte object "knows" how to print its byte value in }
{ four different formats: As decimal quantity, hex quantity, }
{ binary quantity, and extended ASCII symbol. It takes the }
{ cautious path in printing symbols, as prints spaces for all }
{ of the "control" characters 0-31, "rubout" ($7F), and char }
{ $FF. If your printer has some means of printing smiley }
{ faces and all that other folderol in the low 32 characters, }
{ override the PrintSymbol method with a new method that uses }
{ whatever mechanism your printers offers to print those }
{ characters "spaced out" by this implementation of the }
{ PrintSymbol method. }
{--------------------------------------------------------------}

TYPE
 PrintByte = OBJECT
 BV : Byte; { Byte value }
 FUNCTION DontPrint(ShouldI : Byte) : Boolean;
 PROCEDURE PrintDecimal;
 PROCEDURE PrintHex;
 PROCEDURE PrintBinary;
 PROCEDURE PrintSymbol;
 END;


IMPLEMENTATION


{ Returns True for any code that CANNOT be printed verbatim }

FUNCTION PrintByte.DontPrint(ShouldI : Byte) : Boolean;

CONST
 NotThese = [0..31,127,255]; { Unprintable codes }

BEGIN
 IF ShouldI IN NotThese THEN DontPrint := True
 ELSE DontPrint := False;
END;



PROCEDURE PrintByte.PrintDecimal;

BEGIN
 Write(LST,BV:3);
END;



PROCEDURE PrintByte.PrintHex;

CONST
 HexDigits : ARRAY[0..15] OF Char = '0123456789ABCDEF';

BEGIN
 Write(LST,HexDigits[BV SHR 4],HexDigits[BV AND $0F]);
END;




PROCEDURE PrintByte.PrintBinary;

VAR
 Index : Integer;

BEGIN
 FOR Index := 7 DOWNTO 0 DO
 IF Odd(BV SHR Index) THEN Write(LST,'1')
 ELSE Write(LST,'0');
END;



PROCEDURE PrintByte.PrintSymbol;

BEGIN
 IF DontPrint(BV) THEN
 Write(LST,' ')
 ELSE
 Write(LST,Chr(BV));
END;


END.







[LISTING TWO]

{--------------------------------------------------------------}
{ ASCIIChart }
{ }
{ Object Extendability demo program: Prints an ASCII chart }
{ }
{ Note: This program contains control codes specific to }
{ the HP Laserjet Series II Printer. Use with }
{ other printers may be hazardous to your aesthetic }
{ sensibilities. }
{ }
{ Jeff Duntemann }
{ Turbo Pascal V5.5 }
{ Last modified 6/8/89 }
{--------------------------------------------------------------}



{--------------------------------------------------------------}
{ Run this on an HP Laserjet Series II, and you'll get an }
{ ASCII chart including all the weird characters for which the }
{ Series II has characters in its built-in fonts. If you have }
{ some other printer, you need only modify the PrintSymbol }
{ method to use whatever mechanism your printer has to print }

{ the weird characters. By modifying PrintSymbol you are }
{ extending the object PrintByte defined in unit PRINBYTE.PAS, }
{ WITHOUT needing full source code to PRINBYTE.PAS. }
{--------------------------------------------------------------}



PROGRAM ASCIIChart;


USES Printer, { Standard Borland unit }
 PrinByte; { PRINBYTE.PAS from DDJ for September 1989 }


CONST
 Title = 'THE EXTENDED ASCII CODE AND SYMBOL SET';
 Header = ' Dec Hex Binary Symbol Dec Hex Binary Symbol';
 BarChar = Chr(205);
 ColChar = Chr(177);
 FormFeed = Chr(12);


TYPE

{---------------------------------------------------------------}
{ This is a child object of PrintByte, defined in separate file }
{ PRINBYTE.PAS. Remember: PrintByteHP inherits EVERYTHING }
{ defined in PrintByte. EVERYTHING! The only difference is }
{ that the PrintSymbol method is overridden by an HP Series II }
{ specific symbol-print method. Everything else is EXACTLY as }
{ it is defined in PrintByte. }
{---------------------------------------------------------------}

 PrintByteHP = OBJECT(PrintByte)
 PROCEDURE PrintSymbol
 END;


VAR
 I,J,K : Integer;
 Char1,Char2 : PrintByteHP; { Instances of PrintByteHP object type }


{--------------------------------------------------------------}
{ This method overrides the PrintSymbol method defined in the }
{ PrintByte parent object defined in PRINCHAR.PAS: }
{--------------------------------------------------------------}

PROCEDURE PrintByteHP.PrintSymbol;

BEGIN
 IF DontPrint(BV) THEN { If DontPrint method says so, }
 Write(LST,Chr(27),'&p1X',Chr(BV)) { use "transparent data print" }
 ELSE { feature to print weird chars }
 Write(LST,Chr(BV)); { Otherwise, just print 'em... }
END;




PROCEDURE Spaces(NumberOfSpaces : Integer);

VAR
 Index : Integer;

BEGIN
 FOR Index := 1 TO NumberOfSpaces DO
 Write(LST,' ')
END;



PROCEDURE NewPage;

VAR
 Index : Integer;

BEGIN
 Write(LST,FormFeed);
END;



PROCEDURE PrintHeader;

VAR
 Index : Integer;

BEGIN
 Spaces(10);
 Writeln(LST,Header);
 Spaces(10);
 FOR Index := 1 TO 60 DO
 Write(LST,BarChar);
 Writeln(LST);
END;



{--------------------------------------------------------------}
{ Nothing more elaborate here than printing the byte out as a }
{ decimal number, a hex number, a binary number, and a symbol. }
{ The four "Target." calls are method calls to the object }
{ passed to PrintChar in the Target parameter. }
{--------------------------------------------------------------}

PROCEDURE PrintChar(Target : PrintByteHP);

BEGIN
 Spaces(3);
 Target.PrintDecimal;
 Spaces(3);
 Target.PrintHex;
 Spaces(3);
 Target.PrintBinary;
 Spaces(3);
 Target.PrintSymbol;
 Spaces(4);
END;



{--------------------------------------------------------------}
{ This simply sets the HP Series II to its PC symbol set. }
{--------------------------------------------------------------}

PROCEDURE InitHP;

BEGIN
 Write(LST,Chr(27),'(10U'); { Select symbol set 10U }
END;




BEGIN
 InitHP; { Select the PC symbol set }
 Spaces(10); { Output 10 spaces }
 Writeln(LST,Title); { Print out the title string }
 Writeln(LST); { Space down one line }
 FOR I := 0 TO 3 DO { 256 characters takes 4 pages }
 BEGIN
 FOR K := 1 TO 3 DO Writeln(LST); { Space down 3 lines }
 PrintHeader; { Print the chart header }
 FOR J := 0 TO 31 DO { Do 2 columns of 32 chars. }
 BEGIN
 Char1.BV := (I*64)+J; Char2.BV := (I*64)+J+32;
 Spaces(10);
 PrintChar(Char1); { Print the left column character }
 Write(LST,ColChar); { Print the column separator }
 PrintChar(Char2); { Print the right column character }
 Writeln(LST);
 END;
 NewPage; { Issue a form feed to printer }
 END;
END.







[LISTING THREE]

{--------------------------------------------------------------}
{ PrinByte }
{ }
{ Byte-value print object for object extendibility demo }
{ }
{ by Jeff Duntemann }
{ QuickPascal V1.0 }
{ Last modified 6/6/89 }
{--------------------------------------------------------------}


UNIT PrinByte;

INTERFACE


USES Printer;


{--------------------------------------------------------------}
{ The PrintByte object "knows" how to print its byte value in }
{ four different formats: As decimal quantity, hex quantity, }
{ binary quantity, and extended ASCII symbol. It takes the }
{ cautious path in printing symbols, as prints spaces for all }
{ of the "control" characters 0-31, "rubout" ($7F), and char }
{ $FF. If your printer has some means of printing smiley }
{ faces and all that other folderol in the low 32 characters, }
{ override the PrintSymbol method with a new method that uses }
{ whatever mechanism your printers offers to print those }
{ characters "spaced out" by this implementation of the }
{ PrintSymbol method. }
{--------------------------------------------------------------}

TYPE
 PrintByte = OBJECT
 BV : Byte; { Byte value }
 FUNCTION DontPrint(ShouldI : Byte) : Boolean;
 PROCEDURE PrintDecimal;
 PROCEDURE PrintHex;
 PROCEDURE PrintBinary;
 PROCEDURE PrintSymbol;
 END;


IMPLEMENTATION


{ Returns True for any code that CANNOT be printed verbatim }

FUNCTION PrintByte.DontPrint(ShouldI : Byte) : Boolean;

CONST
 NotThese = [0..31,127,255]; { Unprintable codes }

BEGIN
 IF ShouldI IN NotThese THEN DontPrint := True
 ELSE DontPrint := False;
END;



PROCEDURE PrintByte.PrintDecimal;

BEGIN
 Write(LST,Self.BV:3);
END;



PROCEDURE PrintByte.PrintHex;

CONST
 HexDigits : ARRAY[0..15] OF Char = '0123456789ABCDEF';


BEGIN
 WITH Self DO
 Write(LST,HexDigits[BV SHR 4],HexDigits[BV AND $0F]);
END;



PROCEDURE PrintByte.PrintBinary;

VAR
 Index : Integer;

BEGIN
 WITH Self DO
 FOR Index := 7 DOWNTO 0 DO
 IF Odd(BV SHR Index) THEN Write(LST,'1')
 ELSE Write(LST,'0');
END;



PROCEDURE PrintByte.PrintSymbol;

BEGIN
 WITH Self DO
 IF DontPrint(BV) THEN
 Write(LST,' ')
 ELSE
 Write(LST,Chr(BV));
END;


END.







[LISTING FOUR]

{--------------------------------------------------------------}
{ ASCIIChart }
{ }
{ Object Extendability demo program: Prints an ASCII chart }
{ }
{ Note: This program contains control codes specific to }
{ the HP Laserjet Series II Printer. Use with }
{ other printers may be hazardous to your aesthetic }
{ sensibilities. }
{ }
{ Jeff Duntemann }
{ QuickPascal V1.0 }
{ Last modified 6/6/89 }
{--------------------------------------------------------------}




{--------------------------------------------------------------}
{ Run this on an HP Laserjet Series II, and you'll get an }
{ ASCII chart including all the weird characters for which the }
{ Series II has characters in its built-in fonts. If you have }
{ some other printer, you need only modify the PrintSymbol }
{ method to use whatever mechanism your printer has to print }
{ the weird characters. By modifying PrintSymbol you are }
{ extending the object PrintByte defined in unit PRINBYTE.PAS, }
{ WITHOUT needing full source code to PRINBYTE.PAS. }
{--------------------------------------------------------------}



PROGRAM ASCIIChart;


USES Printer, { Standard Borland unit }
 PrinByte; { PRINBYTE.PAS from DDJ for September 1989 }


CONST
 Title = 'THE EXTENDED ASCII CODE AND SYMBOL SET';
 Header = ' Dec Hex Binary Symbol Dec Hex Binary Symbol';
 BarChar = Chr(205);
 ColChar = Chr(177);
 FormFeed = Chr(12);


TYPE

{---------------------------------------------------------------}
{ This is a child object of PrintByte, defined in separate file }
{ PRINBYTE.PAS. Remember: PrintByteHP inherits EVERYTHING }
{ defined in PrintByte. EVERYTHING! The only difference is }
{ that the PrintSymbol method is overridden by an HP Series II }
{ specific symbol-print method. Everything else is EXACTLY as }
{ it is defined in PrintByte. }
{---------------------------------------------------------------}

 PrintByteHP = OBJECT(PrintByte)
 PROCEDURE PrintSymbol; OVERRIDE
 END;


VAR
 I,J,K : Integer;
 Char1,Char2 : PrintByteHP; { Instances of PrintByteHP object type }


{--------------------------------------------------------------}
{ This method overrides the PrintSymbol method defined in the }
{ PrintByte parent object defined in PRINCHAR.PAS: }
{--------------------------------------------------------------}

PROCEDURE PrintByteHP.PrintSymbol;

BEGIN
 WITH Self DO
 IF Self.DontPrint(BV) THEN { If DontPrint method says so, }

 Write(LST,Chr(27),'&p1X',Chr(BV)) { use "transparent data print" }
 ELSE { feature to print weird chars }
 Write(LST,Chr(BV)); { Otherwise, just print 'em... }
END;



PROCEDURE Spaces(NumberOfSpaces : Integer);

VAR
 Index : Integer;

BEGIN
 FOR Index := 1 TO NumberOfSpaces DO
 Write(LST,' ')
END;



PROCEDURE NewPage;

VAR
 Index : Integer;

BEGIN
 Write(LST,FormFeed);
END;



PROCEDURE PrintHeader;

VAR
 Index : Integer;

BEGIN
 Spaces(10);
 Writeln(LST,Header);
 Spaces(10);
 FOR Index := 1 TO 60 DO
 Write(LST,BarChar);
 Writeln(LST);
END;



{--------------------------------------------------------------}
{ Nothing more elaborate here than printing the byte out as a }
{ decimal number, a hex number, a binary number, and a symbol. }
{ The four "Target." calls are method calls to the object }
{ passed to PrintChar in the Target parameter. }
{--------------------------------------------------------------}

PROCEDURE PrintChar(Target : PrintByteHP);

BEGIN
 Spaces(3);
 Target.PrintDecimal;
 Spaces(3);

 Target.PrintHex;
 Spaces(3);
 Target.PrintBinary;
 Spaces(3);
 Target.PrintSymbol;
 Spaces(4);
END;


{--------------------------------------------------------------}
{ This simply sets the HP Series II to its PC symbol set. }
{--------------------------------------------------------------}

PROCEDURE InitHP;

BEGIN
 Write(LST,Chr(27),'(10U'); { Select symbol set 10U }
END;




BEGIN
 InitHP; { Select the PC symbol set }
 New(Char1); New(Char2); { Create objects on the heap }
 Spaces(10); { Output 10 spaces }
 Writeln(LST,Title); { Print out the title string }
 Writeln(LST); { Space down one line }
 FOR I := 0 TO 3 DO { 256 characters takes 4 pages }
 BEGIN
 FOR K := 1 TO 3 DO Writeln(LST); { Space down 3 lines }
 PrintHeader; { Print the chart header }
 FOR J := 0 TO 31 DO { Do 2 columns of 32 chars. }
 BEGIN
 Char1.BV := (I*64)+J; Char2.BV := (I*64)+J+32;
 Spaces(10);
 PrintChar(Char1); { Print the left column character }
 Write(LST,ColChar); { Print the column separator }
 PrintChar(Char2); { Print the right column character }
 Writeln(LST);
 END;
 NewPage; { Issue a form feed to printer }
 END;
END.


















September, 1989
OF INTEREST





ParcPlace Systems has expanded beyond the world of Smalltalk with the
introduction of Objectworks for C++, the first software development system,
company president Adele Goldberg claims, that supports AT&T's C++ 2.0
specification. Objectworks provides incremental compiling and linking,
interactive source-level debugging, and source-code/class browsing. Included
with Objectworks is the AT&T C++ 2.0.
The ParcPlace C++ offering lets you import existing C++ source code or export
it to other systems. The built-in editor allows you to cross-reference
dynamically, although the system lets you use an editor of your choice.
Initially, Objectworks C++ (which, incidentally, was written in Smalltalk-80)
will be available only on the Sun-3 system. However, ParcPlace spokesperson
Doug Pollack did say that other platforms, including DOS and OS/2, would be
supported in the future. Pollack said that they would be "tracking" the
development of Microsoft's OS/2 for the 80386. Interestingly enough, one
decision ParcPlace must make before introducing a DOS or OS/2 version of
Objectworks C++ is which C compiler to support (Sun provides its own C
compiler) because, as Pollack indicated, the company does not want to support
every C compiler in the PC marketplace.
One area of opportunity for DDJ readers is in the realm of class libraries and
other support tools. "A good object-oriented product is not the language
alone," Goldberg told DDJ". It includes a class library and support tools." In
short, ParcPlace is eager to talk to all third-party developers who want to
built C++ support tools for Objectworks.
ParcPlace Systems
1550 Plymouth Street
Mountain View, CA 94043
415-691-6700
AT&T has recently released its long-awaited C++ 2.0 specification, which
includes all of the major features that Bjorne Stroustrup, the author of C++,
has developed to date. AT&T's Mike DeFazio, director of Unix system software,
called C++ 2.0 "an industrial strength release of C++" that has many new
features as well as several refinements. The 2.0 specification was immediately
lauded and supported by major vendors including Apple Computer, Glockenspiel,
Hewlett Packard, HCR Corp., Onotologic, ParcPlace Systems, and Sun
Microsystems.
Among the new features that C++ 2.0 now supports are multiple inheritance,
whereby a child object can inherit the properties of more than one parent;
typesafe linkage; default member-wise assignment and initialization of
classes; and the ability of each class to define its own "new" and "delete"
operators.
Refinements to the language include separation of specialized task and complex
libraries, enhancement and reengineering of the task library, reimplementation
of the stream I/O portion of libc.a, and reengineering of the C++ parsing
mechanism.
The 2.0 spec consists of new documentation, release notes, a product reference
manual, a library manual, and selected C++ readings. The 2.0 language system
itself is available in source code format on magnetic tape. To order the 2.0
documentation, phone AT&T Customer Information Center at 1-800-432-6600 and
ask for the C++ Product Reference Manual #307-146. Although the price wasn't
available at this writing, an AT&T spokesman told DDJ that it should be about
$20.
Licensing for the new 2.0 spec isn't that cheap, however. If you already have
a license for the current 1.2 spec, the 2.0 spec will cost you $10,000.
However, if you don't already have a license, the 2.0 spec will cost you
$20,000. For licensing information, call 1-800-828-UNIX. Reader Service No.
20.
AT&T
One Speedwell Ave.
Morristown, NJ 07960
800-247-1212
Prentice Hall has announced several new titles, including C Programming
Language, ANSI C Version, 2nd Edition, by Brian W. Kerninghan and Dennis M.
Ritchie of AT&T. Prentice Hall is publishing the ANSI version for this
best-seller to coincide with the final ANSI standardization on the C
programming language. Look for the red ANSI stamp on the new cover. This
reference provides the finalized ANSI C standard. The book covers functions
and program structure, pointers and arrays, input and output, and application
portability. ISBN 013-110362-8. $30.
Also released is Programming in C++, by Stephen Dewhurst of AT&T and Kathy
Stark of Sun Microsystems, two of the designers on the C++ compiler design
team at AT&T. The guide discusses data types and operations, procedural
programming, classes, data abstractions inheritance, object-oriented
programming, and storage management, and includes a section devoted to the
design and use of libraries and an appendix of solved exercises. And it offers
the first look at the 2.0 release of C++. Paper: ISBN 013-723156-3. $22.
The X/Window System: Programming and Applications with X, by Douglas A. Young
(Hewlett-Packard), is a guide to the X/Window system for software engineers,
applications programmers, and developers using X. It focuses on using the X
toolkit to create software applications within the X/Window system, and
discusses the use of widgets (prebuilt user interfaces like menus and command
buttons) and intrinsics (pieces of code that enable a programmer to build new
widgets out of existing ones) to create graphics user interfaces. The book
uses a series of progressive examples to introduce key concepts of X, and
presents 30 programs. Software in the book is available free on Arpanet.
Paper: ISBN 013-972167-3. $25.95.
Prentice Hall College Sales and Marketing
Prentice Hall Building
Englewood Cliffs, NJ 07632
201-592-2000
The Delta Logic Division of Poqet Computer Corp. announces Entryway, an
object-oriented application development product for the IBM PS/2 and PC
environment. Entryway allows both information systems professionals and
business managers to rapidly build stand-alone and coordinated business
applications.
According to John Hiles, vice president of Delta Logic, "As more organizations
integrate end-user computing with their information systems, data-processing
professionals and end-user departments will become increasingly involved in
personal computer software development. With Entryway, application development
more closely resembles the writing of a letter or memo on a word processor.
The application's background of text or diagrams, such as an office procedure
flow chart, is entered just as if it were a letter. Then the active,
computer-driven application components are created or specified in objects
that are placed on the text background. This 'first draft' represents a
working prototype that can be refined, expanded, and packaged for distribution
in a series of interactive Entryway sessions."
Entryway's development features include an object-oriented, interactive
development environment; a script language with more than 200 statements;
debugging and regression test facilities and forms generation. Included are
several user interface features -- a hyper-text facility, menu generation and
forms management systems, and text entry fields -- compatible with Common User
Access (CUA), IBM's developing standard for its Systems Application
Architecture (SAA).
The package includes a WYSIWYG full-page report writer, full support for
adding user-defined objects, script-language commands, and time-oriented
facilities. Entryway runs on the installed base of DOS personal computers.
Delta Logic Division can add user objects for the LAN or host connectivity
option. The cost is $795 for the development version and $250 for the run-time
package. Reader Service No. 25.
Delta Logic Division
550 Hartnell, Ste. B
Monterey, CA 93940
408-373-8688
The C Users' Group Directory of User-Supported C Source Code, Vol. II, has
been released by R&D Publications Inc. The Directory contains information
about code in The C Users' Group Library, a repository of public domain and
user-supported C source code. The C Users' Group focuses on source code useful
to experienced programmers, rather than on end-user applications.
Vol. II, edited by Robert Ward and Kenji Hino, includes file by file
descriptions for Library disk volumes 200 - 249, a comprehensive index, and
articles that describe some significant disk volumes. It also catalogs a 68000
C compiler, portable utilities, a file maintenance package, MS-DOS
implementations of several Unix utilities, Small C utilities, and several
graphics packages. Volume I is also available, and both sell for $10. Reader
service No. 21.
The C Users' Group
2120 W. 25th St., Ste. B
Lawrence, KS 66047
913-841-1631
















September, 1989
SWAINE'S FLAMES


More Senselessness with Jelly Beans




Michael Swaine


Nineteen ninety-two and the unificafion of the European market will soon be
here, bringing in a storm of problems whose solutions will require wetware,
because they will involve semantics. The story of the computer program that
translated: "The spirit is willing but the flesh is weak" into Russian and
back to English, ending up with "The vodka is fine, but the meat is rotten,"
is probably apocryphal but the problem it exemplifies is real.
Some semantic confusions might be acceptable. Motorola chipset fans would
probably not mind if a mention of the orthogonality of the 68OxO instruction
set was rendered with the sense "Motorola has the right angle." But think how
confusing technical documentation would become if "number" was translated as
though it was the comparative form of "numb;" i.e., "senseless." Then every
reference to number would become more senseless.
Sometimes dealing with numbers makes people more senseless. In a recent
column, Jeff Duntemann told of a chain-smoking nuclear power opponent who
wanted to see "the inventor of radiation" jailed. I'd like to offer a defense
of the smoker, to make her position a little less -- or more -- senseless.
The idea is that there is more to risk assessment than comparing
probabilities. There has been some research whose results suggest that when
people assess risks, they consider not only the magnitude of the risk but also
other factors affecting its acceptability. So even if the chain-smoker
understood the relative safety records of the nuclear power and the tobacco
industries (which I admit is questionable), she could well have regarded the
smoking risk as more acceptable on the grounds that it is taken willingly, not
imposed from without. Even that position can be questioned, but it shouldn't
be dismissed as mere innumeracy, which I'm afraid too many supporters of
nuclear power do. (I'm not talking about Jeff, though.)
And what is the probability of deliberate sabotage of a nuclear power plant? I
submit that a sufficiently determined terrorist organization could locate key
employees and put sufficient pressure on them to achieve any desired degree of
damage. That fact doesn't mean a whole lot, given that terrorists could just
as easily sabotage baby food, drinking water, airplanes, or anything else. But
it does put the smug, so-precise nuclear safety record figures in a gritty
real-world perspective. The risks of nuclear power would appear,
unfortunately, to involve the psychology of the terrorist mind.
The point is: Numbers have a semantics as well as a syntax.
And they have a pragmatics. Often we want to use numbers for a purpose.
Usually they can be made to cooperate. The latest IEEE salary survey figures
showing San Francisco-area engineers making 25 percent more than Johnson City,
Tennessee engineers, don't help a Silicon Valley engineer trying to get a
raise unless he first helps the figures a little. But with just a little help,
the numbers can be made to serve the engineer's purpose; for example, by
expressing the salaries as percentages of the mean (not median) cost of a
house. Of course, that might just convince him to move to Tennessee.
"Human interface design" is a cumbersome name for one of the most subtle and
important issues in system development. My cousin Corbett has looked for years
for a better term, but everything he came across seemed even more ungainly or
ugly than human interface design (FrontEnding?). It was only, he says, when he
read the "Real Time" column by My Pal Tyler in the June issue of ESP magazine
("For People Who Don't Need No Stinkin'Interface") that he finally found his
neologisms.
Tyler was expanding on the metaphor of integrated circuits as jelly beans.
Although (as Corbett pointed out to me) Tyler faded to mention that the
metaphor logically ought to run in the other direction today (there are now
more ICs than jelly beans in the world, so we should refer to jelly beans as
ICs), he did announce the Era of the Jelly Bean Computer. Buy a few jelly
beans and you can put together a jelly bean PC clone.
The term "jelly bean computer" combines two figures of speech: The obvious
metaphor of the jelly bean for the IC and the metonymy of the part (the ICs)
standing for the whole (the computer). The appeal of the term comes from the
closeness of the two figures of speech to actual fact: The chips are (at
least) as common as jelly beans, and the PC clone described is little more
than a few ICs.
So what kind of human interface would a jelly bean computer have? Corbett
asked himself. Applying metonymy, Corbett decided to represent the entire
human interface by a keyboard, and what kind of keyboard goes with a cheap PC?
A chiclet keyboard, of course. Hence Corbett's new term for human interface
design: Gluing chiclets to the jelly beans.
He hopes it will catch on.





































October, 1989
October, 1989
EDITORIAL


Communications Capers




Jonathan Erickson


In the four or five months since the DDJ listing service opened its on-line
doors, about 1500 of you have been using the service regularly, so report
sysops Bill Garrison and David Betz. This translates into an average of about
2000 minutes of on-line time per day just for downloading listings from DDJ
and MIPS magazine (a sister publication) and sending electronic mail to those
of us at the magazine.
We'll gradually begin providing more features, starting with a list of
upcoming or proposed articles. You can help us by taking a look at the
selections and letting us know which ones you'd most like to see. At the same
time, you can also tell us what you think about some of the articles we've
recently published: Which were you favorites, which ones you didn't like, what
we could have done differently.
We'd also like you to log on to give Bill and David some feedback on what you
think of the service itself -- the interface, the features, and so on. Keep in
mind that this on-line service is still in its formative stage, and you have
the opportunity to help design it the way you think it should be designed. All
Bill and David need are your comments and suggestions. So log on
(603-882-1599) and drop us an e-note.
One of my favorite articles in this issue is Costas Menico's "High-Speed Data
Transfers With NetBIOS." As usual, Costas shares some neat techniques that he
regularly uses at the Software Bottling Company. Good ideas come in batches,
as we found out when Tom Nolan, an associate scientist for Applied Research
Corp., sent in an article describing a very similar NetBIOS data transfer
technique he uses at the NASA/Goddard Space Flight Center. Unfortunately, we
weren't able to publish both articles, but we will be running another article
of Tom's in January, in which he provides the tools for a complete real-time
PC-based data acquisition system (his area of speciality), including a
schematic for the hardware and the source code for the software. This is one
article I can't wait to get into print.
So you think data communications is one area of computing that's getting
easier? If so, you might want to talk to Ed Dowgiallo, architect of the DBMS
Decathlon benchmark suite and head of the DBMS Labs. (DBMS is another sister
publication of DDJ.)
Ed spent the past summer (and then some) setting up the DBMS Labs test bed,
aging several years in the process. The initial configuration of the test bed,
says Ed, was an Everex 386, Step 25 server with five AT-class PCs, several
Archive tape drives for backup, several Tandon DataPacs for transporting data,
and various combinations of networking hardware and software. The project
initially rolled along like a new Volvo, but when one vendor canceled
development of some of its drivers, the wheels came off, and the task of
getting anything to talk to anything else quickly turned into a nightmare.
I hope Ed gets the time and opportunity to describe the nightmare, if not for
DBMS or DDJ, maybe for LAN Technology (yet another of DDJ's sister
publications). I'd like to think that I did my part to keep the project
running by taking Ed to Donovan's, his and my favorite local eatery, for an
occasional lunch and a requisite reality check.
Just to underscore the fact that communications have never been easy, I'm
reminded of a modem project at a company where I used to work. The goal was to
build a 300/1200-baud auto-dial, auto-answer modem. (Yep, this is back in the
days when 1200 baud was still cutting edge.) Pretty straightforward, right?
The modems were built, the manuals written, the boxes printed, and the
ready-to-ship packages were assembled at the warehouse. About that time, one
of the engineers discovered a problem. It seems that, for some reason or
another, the modem didn't work at 1200 baud after all.
I don't know who made the final decision on what to do, but it was quite
simple. The crates were unpacked, the boxes and manuals thrown away, the
labels on the modems peeled off, and the 300/1200 baud selector switch
removed. New labels (sans mention of 1200 baud) were stuck on, new manuals and
boxes printed and packed, and you guessed it, the company introduced a new,
state-of-the-art auto-dial/auto-answer 300-baud modem.
Such is life and, as Don Smith says later in this issue, the chaos of
communications.






































October, 1989
LETTERS







When Is a Standard Really a Standard?


Dear DDJ,
We write C using the medium model, which provides lots of room for code and
64K bytes for data. A lot of the data space is used up by ASCII strings,
messages, keyword lists, and so on.
Most of these strings are used only in one procedure, so it would be nice to
have the space they take up not be in the precious data space, where they are
as bad as globals.
Happily, the "draft" ANSI standard now (and has for more than two years)
specifies that arrays of class 'auto' can be initialized. So, in standard C
code, you can say
 void print_error message(int error-number){
 byte message1[] = "This is a long error message";
 byte message2[] = "Here is another message";
 /*...*/
 byte message999[] = "And so on";
 ...
Any reasonable implementation of a compiler would initialize the messages by
doing a move out of the code space into variables with space allocated on the
stack, and permanent data space would not be squandered on all the messages.
Unfortunately, of the 12 C compilers I have tried out, only one, the Corporate
Computer Systems compiler for the Hewlett Packard HP-3000, supports this
feature.
I have talked to Microsoft and someone there finally admitted that yes, it is
in the current draft ANSI standard but as the standard has not been finalized
yet they are not necessarily implementing all the aspects of the C language.
Now I see that Microsoft is claiming that their compiler is compatible with
the draft standard.
Joe Weisman
Computers for Marketing Corp.
San Francisco, Calif.


Infant Damnation Isn't Our Cup of Tea


Dear DDJ,
The more I see your publication, the more optimistic I become that I may
eventually learn something about programming. While they have me scratching my
head in bewilderment at times, the articles you publish avoid both the
superficiality of some publications and the rancidly pedantic turgidity of
others that too often approach the pudding-like quality of a 19th century
Welsh theologian expounding on predestination or infant damnation. When I
don't understand what you publish, I feel that I simply haven't yet
established a sufficient groundwork in my own mind.
In the special C issue, I found the discussion of Smalltalk interesting, I
learned that the graphics format of PC Paintbrush is in the mainstream, which
is important to me because I'm about to buy a digitizer which comes with it,
and "Building Your Own C Interpreter" will be closely studied. I'm looking
forward to the upcoming treatment of using assembly language to create one's
own minilanguage. My areas of interest do not require a large number of
functions or procedures, but the ones they do require are not provided
adequately in the languages I have examined.
Billy R. Pogue
Lake Havasu City, Ariz.


GUI Programming Guidelines


Dear DDJ,
Yesterday I got my copy of the July issue and the first article I read was the
one on the Presentation Manager. Since I do some programming under Digital
Research's GEM, I am very interested in how things are done under various
graphic user interfaces. But what I have trouble with is the misuse of the
right mouse button. In MFIT, it is used to terminate input and to start an
action. In some drawing programs, it is used as a replacement for the
shift-key. And some other systems, like the Smalltalk environment, use it as
the button that pops up a menu.
This is not easy to understand for the average user of these applications.
Just think if every automobile firm would place the brake on a different pedal
or even on a switch behind the steering wheel! To avoid such misuses I would
strongly recommend that every developer of applications that run under graphic
user interfaces read the Human Interface Guidelines published by Apple. This
book gives a philosophy that puts the user in the middle of the development
process of the external representation of an application. The only one who can
win by such a view of the end user is the end user.
The second comment I have to make is on the programming of graphic interfaces
in general. I do not understand why a relatively small program like MFIT
requires about ten files to compile. The actual code for the processing of the
points and the drawing of the workspace has only about half a page, the rest
just implements the STANDARD behaviour of dialog boxes, menus, and other
objects on the screen. This overhead makes development of user-friendly
applications this complicated and is the cause for the slow learning curve and
the long development time required for even small programs like MFIT.
It is this overhead that is responsible for the minor acceptance of graphic
user interfaces among software developers. I know this because it took me over
two months (about 100 hours, you see, I am a hobbyist) to create an invoice
printing program with an easy-to-use graphic interface with GEM. The actual
code is only about 2K bytes, but the overall size of the source file is about
70K bytes. After this experience, two friends of mine and I decided to develop
our own language and development environment. But soon we noticed that this is
too big a project for three hobby programmers. Now we have a lot of ideas for
common interface in which applications can be used on Unix, DOS, or the Mac
without changing the compiled program. In such an environment the user could
run the same application under several windowing systems. The system would
care about the differences in the user interface and OS. So if there are other
people interested in our thoughts, just drop a line to DDJ, who will surely
pass your letter on to us.
Andreas Berger
Neu-Isenburg, W Germany


It's All in the Numbers


Dear DDJ,
Your magazine continues to provide enjoyable articles and accompanying code. A
comment on Al Stevens's August 1989 column in which he restates Robert
Benchley "the world is divided into two kinds of people -- those who divide
the world into two kinds of people and those who do not." As a compuphobia
counselor for ten years, I have learned that the world is divided into three
kinds of people -- those who count and those who can't.

mickRacky
Oakland, Calif.
DDJ: 12 out of a dozen times, you're right.


AWK-Like Extensions Revisited


Dear DDJ,
I much appreciated Jim Mischel's article "Writing AWK-Like Extensions to C" in
the June 1989 issue of Dr. Dobb's Journal. I have used the pattern matching
routines to construct an alternative grep that shows matching lines within a
window of lines before and after the matching one. I had a shell version of
this tool from the March 1989 issue of UNIXWorld, but my C version obviously
goes much quicker.
However, I have found one problem, the functions will not find a match for the
empty line pattern ^$. This occurs because the gets( ) library function
returns just an empty string, the \0 character. In this circumstance the
re_match( ) function makes no attempt to find a match, as the body of the
while loop is never entered, and thus returns NULL. I have cured this by
changing the loop to be:
 _s_end = NULL;
 do
 {
 if( match_term(c - s, c, pat)!= FALSE)
 {
 RSTART = c - s;
 RLENGTH = _s_end - c;
 return(c);
 }
 if(*c)
 {
 c++;
 }
 } while(*c!= ENDSTR);
which always does at least one pass through the line to be matched.
John M. Howells
West Lancs, England
Jim responds: Thank you for your letter. You did indeed find a problem with
the code and I appreciate your response. I did some testing and found that you
can eliminate the awkward "if (*c)" in your code by modifying the while
statement: while (*c++ != ENDSTR);


Forth, an Object of Affection


Dear DDJ,
I enjoyed Jeff Duntemann's "Structured Programming" column in the July issue
of Dr. Dobb's.
A lot of what he said about object-oriented programming sounded like good
factoring in Forth. With good factoring, each word is reusable and you can
hide the details of the lower-level words.
Perhaps in a future article you could address OOP in the Forth environment,
such as the Forth object-oriented programming extension to HS/Forth.
Ramer W. Streed
Kato Engineering
Mankato, Minn.


Pascal Hints


Dear DDJ,
I have greatly enjoyed reading Jeff Duntemann's articles in DDJ; as my primary
language is Pascal, his column is inherently the easiest for me to understand.
I would like to make a few points about his June 1989 column: 1. When assembly
routines are short -- as in the case of BIOS hooks -- it is better to code
them in assembly, as Turbo adds much housekeeping code which swamps the
functional code. These calls are by definition machine dependent so there is
no reason not to optimize them. 2. Jeff's calendar program goes to too much
effort to do its work! As you are restricting yourself to dates after 1980, we
don't need to know about Julian and Gregorian calendars; all we do need to
know is on which day of the week falls a given date. The function to serve
this purpose is called Zeller's congruence, which I learned about from
Computer Language, March 1988, pp. 9-10. I hope that you find this information
useful.
Norman Newman
Israel
P.S. What is KI6RA?
DDJ: KI6RA is Jeff's ham radio call sign.


More on RLE


Dear DDJ,
I refer to your two articles, "Run-Length Encoding," by Robert Zigon, February
1989, and "RLE Revisited," by Phil Daley, May 1989. While Mr. Daley's
compression technique certainly solves the problems associated with
inefficient coding of streams of data with frequent character changes, it
causes another problem potentially much worse in terms of data integrity.

Consider the situation where the algorithms are used for the compression of
data where there is potential to lose a byte (or where a byte may be corrupted
due to transmission or storage media).
The loss of a byte using the former technique is susceptible to the worst case
loss of the whole of the rest of the file, if the incorrect byte is one which
marks the stream length. Since by definition, compression algorithms are used
to save storage or transmission time on large files, the losses may be
considerable.
I would be interested to read a description of an algorithm which combines the
'best of both worlds' by combining Daley's better compression technique, with
properly framed sequences which minimize the effects of inaccurate
reproduction.
Luke E. Murphy
French's Forest, Australia


Notes from Down Punder


Dear DDJ,
I am still shocked about your July 1989 editorial concerning escort agency
services being logged as "software" by credit card users.
I feel that I must chastise you, as editor, for not warning your readers
concerning the riscs associated with using recursive, or ill-behaved,
procedures in such an obvious multiuser environment.
I see the lack of such warnings as likely to spread bugs and viruses that may
severely limit some users' forthcoming endeavours. Such bugs and viruses may
disable their stack probes, or make them Unix.
P. Butterworth
New South Wales, Australia


Brute Force vs. Boyer-Moore


Dear DDJ,
Thanks for Costas Menico's article on faster string searches. The article was
well written, and the algorithm he presented is interesting and useful.
However, he seems to present this approach as being superior to the brute
force method in the general case, which it clearly isn't. The weakness in the
algorithm is the need for constructing a 256-byte skip array at every cell.
This is equivalent in CPU cycles to doing a brute force scan on a string of
about 80 bytes, and it occurs before the actual string search even begins.
Also, it should be pointed out that the timing benchmark program comparing his
POSBM to the brute force method tended to skew the results. First of all,
Turbo Pascal's built-in POS function is an extremely poor example of the brute
force method. I ran Mr. Menico's benchmark program to compare execution times
among POS, POSBM, and my own brute-force FIRSTPOS function, which is included
in my STRINGS.TPU package (available on the Borland Forum on CompuServe). Not
to toot my own horn, but on my IBM XT, FIRSTPOS was about 10 times faster than
POS, twice as fast as POSBM! So by this benchmark, POSBM seems to be inferior
to a well-implemented brute force approach.
Another skewing factor in the benchmark is the use of a 255-byte string, from
which the last five characters are selected as the search string. This
maximizes the distance over which the algorithm can skip. I ran the benchmark
using an 80-character string (probably closer to a real-world application)
with POS, POSBM, and FIRSTPOS, and the ratio of execution times was about
4.5:2.5:1, respectively.
So the question is, under what conditions is POSBM superior to the brute force
method? A long pattern string would work in its favor by maximizing the
average distance per skip. A long string to search in would also tend to
magnify the effects of skipping, but here we're limited by the 255-byte length
of Turbo Pascal strings. Eliminating the overhead of constructing the skip
array at every call would go a long way in speeding things up, so it seems an
ideal situation for using the Boyer-Moore method would be in searching a
series of strings for the same pattern, so that the skip array need be
constructed only once. Unfortunately, the way POSBM is written, there is no
way to save the skip array between calls, making it impossible to capitalize
on this situation.
Rich Winkel
Harrisburg, Missouri


C Multidimensional Arrays


Dear DDJ:
Your annual C Issue was quite enjoyable. I should like to comment on two
articles in particular.
The first, "C Multidimensional Arrays at Run Time" by Paul Anderson, is both
instructive (with respect to the use of pointers) and useful in application.
It should be noted that the techniques developed for two- and
three-dimensional arrays are easily extended to use the Far Heap as well as
the (near) Heap. One has merely to use farcalloc( ) and farmalloc( ) with a
cast to long for their numerical arguments and of course farfree( ) for
releasing the memory blocks.
In Listing Three Mr. Anderson has assumed that the determinants to be
evaluated have positive definite (non-zero) elements in the application of the
algorithm (which is misnamed). As written, a non-vanishing determinant with a
zero in a diagonal element would return a divide by zero error.
The method he has used to evaluate the determinant is known in the literature
as "Upper Triangularization." The determinant (matrix) is converted into an
upper triangular form (one in which all the elements below the diagonal are
zero) and evaluated using the theorem that the determinant of an upper
triangular matrix is the product of its diagonal elements. Note that any zero
elements on the diagonal of the initial determinant are transformed in the
upper triangularization process to non-zero elements if the determinant is
non-vanishing.
This technique was originally developed (and primarily used) in the solution
of systems of linear algebraic equations and the related problem of matrix
inversion. In developing that solution the determinant of the coefficients of
the equation is required. In many applications (particularly statistical) the
equations are inherently positive definite and the solutions presented in the
literature do not contain any tests for singularity. I presume that Mr.
Anderson had referred to those in obtaining his algorithm.
I've included the correct code (see Listing One), translated from the pseudo
code on page 143 of Numerical Methods for Computer Science, Engineering, and
Mathematics, by John H. Mathews, Prentice Hall 1987. The algorithm given there
is embedded in a slightly larger problem and I have extracted out only that
part relevant to evaluating the determinant.
Jeff Duntemann has, in my opinion, written the most intelligent article by far
on Smalltalk. He has really placed it in the proper perspective. It really
makes you wonder why it hasn't been written before. This article alone is well
worth the price of the issue many times over.
Reading it and noting my own reaction, reminded me of an event I witnessed
many years ago at a Physics Department Colloquium. Professor E.P. Wigner was
giving a talk on some new work he had recently done and when he had concluded
there was the usual question and answer session. A well-known physicist (not
quite as well known as Wigner) in the audience arose and said, "Well Professor
Wigner, that was all very nice but it seems rather elementary." And Professor
Wigner (after a momentary reflection) replied: "Yes -- when something has been
explained to you and you do understand it, it is indeed rather elementary."
Given that framework, any other comments on my part would be superfluous.
Morton F. Kaplon
Easton, Maryland

_LETTERS TO THE EDITOR_



[LISTING ONE]

/***********************************************************************/
/* Determinant Calculator Using Near Heap and Upper Triangular Matrix */

double det(arg,n) /* arg = array name, n = # elements */
char *arg;
int n;
{

 register int i,t,k,p;
 double **a, ret, x; /* Array, Return value of Det, Temp variable */
 char **sdim2(); /* defined in Listing Three */
 int * row; /* row pointer for pivoting */

 /* dynamically create 2 dim "array " array a from arg */
 a = (double **)sdim2(arg,n,n,sizeof(double));

 row = (int *)malloc(n*sizeof(int)); /* row pointer for pivoting allocation */
 if (row == (int *) NULL) {
 fprintf(stderr,"No heap space for Row Pointers for Pivoting ");
 exit(1);
 }

 /* creating upper triangular matrix with test for 0 values on diagonal */

 /* first initialize pointer vector */
 for (i = 0; i < n ; i++)
 row[i] = i ;

 /* find pivot elements */
 for (p = 0; p < n - 1; p++) {
 for (k = p + 1;k < n ;k++) {
 if ( fabs(a[row[k]][p]) > fabs(a[row[p]][p]) ) { /* switch index */
 t = row[p];
 row[p] = row[k];
 row[k] = t;
 }
 }
 /* In usual application this would be an error message that the
 * matrix is singular and the program would exit here */
 if ( a[row[p]][p] == 0 ) /* Determinant is 0 */
 break; /* No need to continue on */

 for (k = p+1;k < n; k++) {
 x = a[row[k]][p]/a[row[p]][p];
 for ( i = p + 1; i < n ;i++) /* do Gauss Jordan elimination */
 a[row[k]][i] = a[row[k]][i] - x * a[row[p]][i];
 }
 }

/* if ( a[row[n-1]][n-1] == 0 ) Determinant is 0 - This would in
 * normal application be a message Matrix is singular and an exit */

 /* value of determinant is product of diagonals of upper triangular matrix */
 for (ret = 1,i = 0;i < n; i++)
 ret *= a[row[i]][i]; /* if any of diagonals are 0 Det = 0 */

 free(row);
 free(a);

 return ret;
}





































































October, 1989
 IMPLEMENTING MULTIPLE COMPUTER COMMUNICATIONS LINKS


Mark Servello


Mark Servello is a software engineer for American Management Systems and can
be reached at 1455 Frazee Road, Suite 315, San Diego, CA 92108-4304.


It's no secret that PCs have put a tremendous amount of computing power
directly into the hands of end users, and that end users are becoming
increasingly sophisticated in customizing the machines for their particular
functions. Consequently, everyone is looking for methods to maximize
communication between individual workstations, each of which is tailored on an
individual basis to obtain the largest productivity improvement for its user.
This communication is necessary in order to improve the effectiveness of the
business unit, by allowing transfer and sharing of information within and
between groups.
This article discusses the general concept of PC-to-minicomputer
communication. As an example, I describe how a network can be constructed
using common telephone equipment and a surplus minicomputer, which becomes a
network server, and illustrate methods for getting computers to communicate
under programmed, rather than terminal, control.


The Environment


In our office, we tend to work in small groups on individual projects. The
members of each group share information frequently, and often one or more
groups are in the document print/revision cycle. We've had a relatively new
DEC PDP-11/73 machine sitting around in the office for some time (a circa 1985
leftover from an expired contract), and we decided that its multiuser
capabilities would enable us to use it as a network controller. As such, we
could use it for sharing information on an as-needed basis and for providing
spooled access to the office's printing resources. Because laser and
high-speed draft printers are not small expenditures, it is important to use
them effectively, which is exactly what a multiuser computing system can do
with its print spooling software.
When we moved into our current offices, we installed a second jack as a data
connection at every phone outlet wall panel and several more in common usage
areas. The extra line is an RJ-11 type modular jack wired to handle all eight
modem control signals.
Each data jack at a telephone wall panel in the offices leads back to a single
patch block in the computer room (see Figure 1). These jacks are plugged into
terminal interfaces on the PDP-11, which also has all shared devices such as
printers and modems connected through the terminal lines. The shared devices
under PDP-11's control use all eight RS-232 modem control signals for maximum
reliability at high speed. The connecting cables between the wall jacks and
office PCs (and between the corresponding patch block location and the PDP-11)
are standard six-conductor, with an RJ-11-to-DB25/DB9 connector at the PC's
serial port. For these lines, the PDP-11 provides TD, RD, DTR, and Signal
Ground, which are wired/jumpered by the connector to allow serial
communication over the port. All wiring from the connector to the PDP-11 is
straight-through, and each connector is labeled with the type of device it is
configured for (this lesson was learned the hard way).


The Software


The hardware was, of course, in place much sooner than the software.
Communication between the individual PC workstations and the PDP-11 was
established relatively quickly, using a variety of communications packages
configured on the PC as Digital Equipment Corporation VT100 terminal
emulators. Fortunately, our phone company was careful to install all jack
lines uniformly throughout the offices. Even so, some serial devices are
finicky, so an RS-232 Breakout Box came in handy. (I know, programmers don't
change light bulbs because it's a hardware problem, but what happens when
you're programming the hardware?)
We were able to use this type of communication with the PDP-11 to accomplish
simple printing to high-speed line and dot matrix printers by first using
Kermit or XModem to transfer a print file to the PDP-11, then spooling the
file to the device. We also used the Kermit program on the PDP-11 to establish
communications between the PDP-11 and all of the shared modems and printers.
What we really needed, however, was a method by which an individual PC could
behave as if the shared device (a printer for instance) was directly connected
to it, so that users could run applications programs (like word processors)
and print directly to the serial port. On the other end, the PDP-11 had to be
capable of accepting the output to the serial port and placing it into a print
file until the PC application program was finished. Once the file was
completed, it had to be spooled to the appropriate printer.
Because of the variations in possible PC connections and the way the PDP-11's
operating system (DEC RSTS/E) works, I decided that two programs would have to
be written. The first program, running on the PC, provides configuration
control and establishes sessions with the PDP-11. The second program, running
on the PDP-11, performs communication functions with the PC program by
providing status, executing commands, and catching print output for spooling.
The DEC-provided minicomputer spooling program then routes printout files to
the proper printer, as specified by the user.
I decided to use a bottom-up approach and began writing the PC program's
serial port interface using Turbo Pascal 5.0.
For the actual serial interrupt service routines, I used Ray Duncan's
marvelous book Advanced MS-DOS Programming (Microsoft Press, 1988) to
familiarize myself with the basic PC methodology (I'm a minicomputer
programmer), then employed a Turbo Pascal translation tailored for my needs.
Flow control from the PDP-11 to the shared devices is by the full eight-line
RS-232 modem signals. On the PC connection, however, only four lines (TD, RD,
Signal Ground, and DTR) are normally used, because the PDP-11 uses DTR to
determine whether a terminal is active, and logs out any process when its
terminal's DTR goes low. Flow control to/from the PCs is XON/XOFF in the data
path. The ISR routines in Listing One show the XOFF sent when the PC's receive
queue is within one second of being full (as determined by the data rate at
the current transmission speed). The XON is sent so the minicomputer can
resume transmitting when at least two seconds of space are available in the
receive queue.
I initially used a simple driver program with routines that would loop,
checking the comm port and displaying what was received. The program then
checked the keyboard, sending whatever printable characters that were typed to
the comm port. Turbo Debugger made code testing simple, because we had
established that communications with the PDP-11 were reliable by using a
communications program. No special programming was required on the PDP-11 at
this point, as its normal terminal logon and control dialogue provided data to
ensure that the PC was receiving and transmitting correctly.
After the initial communication routines were verified and the configuration
routines added, I began writing the code necessary to establish a session with
the PDP-11. RSTS/E (resource sharing, time sharing/extended), a relatively old
multiuser operating system, requires that each user identify himself with a
user identification code (UIC) that consists of a project number (range 1 -
255), a programmer number (range 0 - 255), and a password. This is fine for
technical people, who are (sometimes) willing and accustomed to cryptic logon
sequences, but analysts and word processors are a different manner. None of my
users would consent to having to log on and run a program manually each time
they needed to print a document/file. I therefore had to develop PC routines
that would first wake up the RSTS/E operating system on the configured serial
port, log the user on, and then run a matching program on the PDP-11 to
control its spooling operations. All the user would have to specify were
printer selection and queue control options. Fortunately, DEC still gives the
sources for the major commonly used system programs (CUSPs) into its multiuser
operating systems, among which is the system logon program.
While researching the most expedient method for identifying users to the
PDP-11 and logging them in automatically, I discovered that DEC has already
installed an Optional Feature Patch in the RSTS/E logon program for this
purpose. This patch allows the log-on program to recognize specific strings
from terminal ports, to bypass password verification, and to automatically
execute a captive program connected to that terminal line. The terminal line
itself would serve to identify the user uniquely. I decided that the PDP-11's
operating system security was sufficient for preventing unauthorized access to
that machine, so long as the key word that activates the logon bypass is
stored in encrypted format in the PC program.
The RSTS/E CUSP programs are written in the Basic-PLUS-2 language (a rather
Pascal-like variant of Basic). It was a simple matter to implement the source
change enabling this patch and to provide the logon program with the
recognition string and program name to be executed when that string is
detected on a terminal line. Debugging the logon sequence was quick and
straightforward using the PDP-11's system console to monitor the logon program
and Turbo Debugger to monitor the PC.
Figure 2 shows the sequence of operations necessary to establish a cooperative
communication session between a PC and the PDP-11. The first two steps, as
described earlier, control the PC's serial port and establish a valid logon
session under the PDP-11's multiuser operating system. The third step, in
which the minicomputer sends device information to the PC interface program,
is critical. As the minicomputer maintains a table of devices available on
line, which it sends to each PC when the interface program finishes the logon
process, various printers can be added, removed, or moved to new locations on
the PDP-11 without changing any of the PC software.
When the user selects a printer, the minicomputer specifies which spooler to
use for printout routing, as shown in Step 4. The bulk of two-way
communication takes place in Step 5, where the PC user can display and
manipulate the print queues for the various attached printers. I decided to
keep the queue manipulation operations as simple as possible for the users,
even though this imposed an additional burden on the programs.
The user is able to display the queue for any printer, including the file
name, user name (from the terminal connection), length, and date/time the file
was queued. The user can then examine, change, or delete queue entries. This
is one area where we rely upon professional courtesy, as there is no security
checking on queue operations, and any user can manipulate all entries in the
queue. After any necessary queue manipulation has been performed the user
initiates PDP-11 file capture mode, directing the PC's serial port output to a
file. This file capture continues until one minute has passed from the last
output over the serial link; the file on the minicomputer is then closed and
added to the spooler queue for the selected printer.
I decided to use individual command and response packets for communication
between the minicomputer and the PC control programs. Table 1 shows the PDP-11
packets used and the information contained in each, while Table 2 shows the
same information about the PC packets. Figure 3 shows the physical makeup of
each packet.
Table 1: PDP-11 packets

 Packet Packet Description
 Type Name
 ---------------------------------------------------

 100 PDP-11 OK Tells PC program that PDP-
 11 is active and awaiting
 commands

 101 Device List Tells the PC program that
 Header device description packets
 follow, and how many

 102 Device Description of printer
 Description available for use

 103 Queue List Tells the PC program that
 Header queue entry packets follow,

 and how many

 104 Queue List Contains information about
 Entry the queue entry

 105 PDP-11 Error Indicates an error in PDP-11
 packet processing

Table 2: PC packets

 Packet Packet Description
 Type Name
 -----------------------------------------------------------

 200 Micro Tells PDP-11 that the
 Acknowledge preceding packet was received
 correctly

 201 Printer Select Tells PDP-11 which printer to
 use for subsequent operations

 202 Request Asks PDP-11 to send the
 Queue List queue list for the selected
 printer

 203 Delete Entry Tells PDP-11 to delete a
 queue entry

 204 Move Entry Tells PDP-11 to move a
 queue entry to another
 position in the queue

 205 Hold Entry Tells PDP-11 to hold the
 entry in the queue, but not
 print the file

 206 Release Entry Tells PDP-11 to release a
 previously held queue entry for
 printing in turn

 207 Print Start Tells PDP-11 that print file
 output is to begin from PC

 208 Print End Tells PDP-11 that print file
 output is finished

 209 Micro Error Indicates an error in PC
 packet processing

After the PC serial port has been initialized and attached, the PDP-11
recognition string activates the minicomputer program. The minicomputer
program responds with an OK packet, followed by a device list header packet
and as many individual device packets as necessary (up to a maximum of 20) to
inform the PC program of all available print devices. This list is used to
generate a menu of printer choices for the user, and is available under
function key command.
When the available device list has been sent to the PC, it displays a function
menu and accepts/processes commands from the user until a print file output
operation begins. When a time limit of one minute has passed since any PC
output activity, the minicomputer program closes the file, spooling it to the
currently selected printer queue. After a file has been spooled to a printer,
one additional minute is allowed for the PC program to restart command
processing to the minicomputer. If no further communication is received, the
PDP-11 operating system kills the minicomputer process.
Debugging the timing and content of the send/receive/respond packets sent
between the two programs involved more than just the use of the Turbo Debugger
on the PC. The large number of packet types and the varying sizes of data
sections in each packet required a driver program to test the interchange.
The PC driver let me input packets to be sent to the PDP-11 while displaying
the contents of received packets. It also provided a status display of any
packet rejected because of a transmission error or bad checksum.
Listing Two shows the constant and type definitions for packets in the PC
program. The PACKET_REC type definition uses the Pascal variant record
structure to define each packet type, with the data portion of each packet
broken into field definitions applicable to the packet type. This listing also
shows the main processing portions of the packet send/receive procedures. A
similar test program was written on the PDP-11 to display the contents of each
packet received or sent on it, in the sequence of transmission. In addition to
responding to PC commands, the PDP-11 program has commands that cause it to
send PDP-11 OK, Device List, Printer Queue, and PDP-11 Error packets.
Once the command and response packet routines were working in the PDP-11 and
PC programs, modifying the display/command processors on each machine into
final form was fairly easy. The bottom-up implementation approach made a
complete toolbox of data types and procedures available in the form of Turbo
Pascal units on the PC and linkable subprograms on the minicomputer. I then
had only to link these units and complete the user interface.
On the PDP-11, the same components used in the program that communicates with
the PC were used to develop a system-wide communication monitor that maintains
a complete transaction log of packets sent/received on the PDP-11. This log,
incidentally, can also be used to provide project-oriented usage charge backs
if the equipment involved must be depreciated. Fortunately, our PDP-11 was
depreciated long ago, so this has not yet been necessary. Of course, there's
always the next upgrade....


Conclusion



I hope this article has provided some insight into the problem of getting
cooperative programs with potentially different languages to communicate on
different platforms. Utilizing the bottom-up implementation approach, first
establish reliable terminal communication between the devices. Don't forget
your breakout box here, even if you hate hardware, because it sure can save
you time. Second, implement serial communication configuration and processing
routines in your program. Third, create procedures to handle the dialogue of
commands and responses. Finally, ensure that the sequence and content of each
dialogue are correct before moving on to complete functionality and user
interface. If you build your modules into cooperating building blocks, this
methodology should serve you well regardless of whether your communication
medium is via a modem, twisted-pair, Ethernet, or fiber-optic LAN.


Availability


All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063, or call 800-356-2002 (from inside
Calif.) or 800-533-4372 (from outside Calif.). Please specify the issue number
and format (MS-DOS, Macintosh, Kaypro).

_Implementing Multiple Computer Communications Links_
by Mark Servello


[LISTING ONE]

 Unit Serial_IO;
 {************ Unit Interface Description ***************}
 Interface
 Type Config_rec = record { contains the configuration info }
 { for serial communication and user}
 { interface }
 IRQ : Integer;
 Port : Integer;
 Data : Integer;
 Baud : Integer;
 Rate : integer; { bytes/sec }
 Parity : Char;
 StopBits : Integer;
 DataBits : Integer;
 Snow : Boolean;
 Lines : Integer;
 Attention: String[40];
 Fore : Integer;
 Back : Integer;
 end;

 Var Current_Cfg : Config_Rec;
 Procedure Check_Receive (var ch : char);
 Procedure Check_Send;
 Procedure Configure( New_Cfg : Config_Rec );

 {******************* Unit Implementation ******************}
 Implementation
 uses dos,crt; { DOS and CRT units are utilized }
 Const queue_max = 3936; { queue can hold 48 lines X 82 char}

 { *********** Serial Port Constants ***********************}
 COM1_data = $03f8; { COM1 Data port }
 COM1_IRQ = $04; { COM1 IRQ Number}
 COM2_data = $02f8; { COM2 Data port }
 COM2_IRQ = $03; { COM2 IRQ Number}
 ier_offset = 1; { UART IER Reg }
 mcr_offset = 4; { UART Master Reg}
 sts_offset = 5; { UART Status Reg}
 IRQ3_Int = $0B; { IntVec for IRQ3}
 IRQ4_Int = $0C; { IntVec for IRQ4}
 IRQ5_Int = $0D; { IntVec for IRQ5}
 IRQ6_Int = $0E; { IntVec for IRQ6}

 IRQ7_Int = $0F; { IntVec for IRQ7}
 PIC_CTL = $20; { Cmd for 8259 }
 PIC_MASK = $21; { Mask for 8259 }
 EOI = $20; { EoI command }
 TBE = $20; { TBE bit }
 XOFF_Char = #19; { ^S }
 XON_Char = #17; { ^Q }
 CR = #13;
 LF = #10;

 Type Queue_type = record
 queue : array[1..queue_max] of byte;
 front,rear : integer;
 count : integer;
 end;
 Port_Status = (XON, XOFF);
 Var Transmit_Queue,
 Receive_Queue : Queue_Type;
 Receive_Status,
 Transmit_Status : Port_Status;
 Com_STS : Integer; { Serial Status I/O Port }
 mask_value : integer; { Control mask word }
 old_isr : pointer; { storage for com port }
 { ISR vector in place }
 {**********************************************************}
 { Serial Interrupt Service Routine - grab the char and put }
 { it in the queue }
 {**********************************************************}
 Procedure Serial_ISR; Interrupt;
 var ch : byte; { for the incoming char }
 regs : registers; { for using BIOS to beep bell }
 next_rear : integer;
 begin
 inline($FA); { Disable interrupts }
 ch := port[current_cfg.data]; { get character from port }
 with receive_queue do
 begin
 next_rear := rear + 1;
 if next_rear > queue_max then { wrap the pointer if }
 next_rear := 1; { necessary }
 if next_rear <> front then
 begin { put char in queue }
 rear := next_rear;
 queue[rear] := ch;
 end
 else
 begin { queue full,beep bell }
 regs.ax := $0E07;
 intr($10,regs);
 end;
 inc(count); { Inc # entries and }
 { Check for queue getting full. Send XOFF when one }
 { second of space left }

 if count > (queue_max - current_cfg.rate) then
 begin
 Receive_status := XOFF;
 repeat until (port[com_sts] and TBE)<>0;
 port[current_cfg.data] := ord(XOFF_Char);

 end;
 end; { END WITH }
 inline($FB); { Enable interrupts }
 port[PIC_CTL] := EOI { send end of interrupt to PIC }
 end; { END PROCEDURESERIAL_ISR }

 {**********************************************************}
 { Attach Com Port Procedure - takes over interrupt vector }
 { and initializes the UART entries in the configuration }
 { table. }
 {**********************************************************}
 Procedure Attach_Com_Port;
 var mask_value : byte;
 Int_Num : integer;
 begin
 Case Current_Cfg.IRQ of
 3 : Int_Num := IRQ3_Int;
 4 : Int_Num := IRQ4_Int;
 5 : Int_Num := IRQ5_Int;
 6 : Int_Num := IRQ6_Int;
 7 : Int_Num := IRQ7_Int;
 end;
 GetIntVec(Int_Num, old_ISR); { Save old intvec }
 SetIntVec(Int_Num, @Serial_ISR); { point to the }
 { Serial_ISR procedure }
 port[Current_Cfg.data+mcr_Offset] := $0B; { Set DSR/OUT2 }
 port[Current_Cfg.data+ier_Offset] := $01; { enable ints }
 mask_value := port[pic_mask]; { read PIC mask}
 mask_value := mask_value and { allow ints }
 (not (1 shl current_cfg.irq)); { on com port }
 port[pic_mask] := mask_value; { write it back}
 { to PIC }
 receive_status := XON; { send XON to }
 repeat until (port[com_sts] and TBE)<>0; { let other end}
 port[current_cfg.data] := ord(XON_Char); { know we're }
 { here. }
 transmit_status := XON;
 end; { END ATTACH_COM_PORT }

 {**********************************************************}
 { Release Com Port Procedure - Gives the com port interrupt}
 { back to the previous holder. }
 {**********************************************************}
 Procedure Release_Com_Port;
 Var Int_Num : Integer;
 begin
 Case Current_Cfg.IRQ of
 3 : Int_Num := IRQ3_Int;
 4 : Int_Num := IRQ4_Int;
 5 : Int_Num := IRQ5_Int;
 6 : Int_Num := IRQ6_Int;
 7 : Int_Num := IRQ7_Int;
 end;
 mask_value := port[pic_mask];
 mask_value := mask_value or (1 shl current_cfg.IRQ);
 port[pic_mask] := mask_value;
 SetIntVec(Int_Num, Old_ISR); { Restore the com port int-}
 { errupt vector }
 Receive_Status := XOFF;

 Transmit_Status:= XOFF;
 end;

 {**********************************************************}
 { Check_Receive Procedure - This procedure checks the in- }
 { coming com port queue. If any characters are waiting, }
 { they are appended to the incoming string for program }
 { processing. }
 {**********************************************************}
 Procedure Check_Receive (var ch : char);
 begin
 with receive_queue do
 if front <> rear then { Queue empty when front ptr }
 { = rear ptr }
 begin
 front := front + 1;
 if front > queue_max then
 front := 1;
 ch := chr(queue[front]);
 Case ch of
 XOFF_Char : Transmit_Status := XOFF;
 XON_Char : Transmit_Status := XON;
 end; { END CASE CH }

 { Check queue count and send XON if receiving stop- }
 { ped and queue has 2 seconds of space free }
 dec(count);
 if (count - (2 * current_cfg.rate)) > 0 then
 begin
 receive_status := XON;
 repeat until (port[com_sts] and TBE)<>0;
 port[current_cfg.data] := ord(XON_Char);
 end;
 end; { END IF FRONT <> REAR }
 end; { END PROC CHECK_RECEIVE }

 {***********************************************************}
 { Check_Send Procedure - This procedure handles sending }
 { chars out the COM port. If there are any characters wait- }
 { ing in the send queue, they are sent one at a time. }
 {***********************************************************}
 Procedure Check_Send;
 Var ch : char;
 done : boolean;
 Begin
 done := false;
 with transmit_queue do
 repeat
 if (front = rear) or { Queue empty when front ptr }
 { = rear ptr }
 (Transmit_Status = XOFF) then { Don't send }
 done := true
 else
 begin
 if front > queue_max then
 front := 1;
 ch := chr(queue[front]);
 repeat until (port[com_sts] and TBE)<>0;
 port[current_cfg.data] := ord(ch);

 end;
 until done;
 End; { END PROCEDURE CHECK_SEND }

 Procedure Configure( New_Cfg : Config_Rec );
 begin
 { Routine here reads configuration file based on location }
 { contained in environment string, then attaches the com }
 { port and sets communication parameters }
 end;
 begin { Unit Initialization }
 Configure( Current_Cfg );
 end.








[LISTING TWO]


 Unit Packet_Comms;

 Interface
 Const Pkt_PDP_OK = 100;
 Pkt_Dev_hdr = 101;
 Pkt_Dev_lst = 102;
 Pkt_Q_hdr = 103;
 Pkt_Q_lst = 104;
 Pkt_PDP_Err = 105;
 Pkt_Micro_OK = 200;
 Pkt_Print_Sel = 201;
 Pkt_Q_Req = 202;
 Pkt_Q_Del = 203;
 Pkt_Q_Move = 204;
 Pkt_Q_Hold = 205;
 Pkt_Q_Rel = 206;
 Pkt_Prt_Start = 207;
 Pkt_Prt_End = 208;
 Pkt_Micro_err = 209;
 Invalid_PDP_Packet = 01;
 Invalid_Checksum = 02;

 Type Seq_Type = array[1..2] of char;
 Fname_Type = array[1..9] of char;
 Dname_Type = array[1..20] of char;
 Packet_Rec = Record
 Data_Checksum : array[1..5] of char;
 Case Packet_Type : byte of
 Pkt_PDP_OK: (* PDP-11 OK has no fields *)();
 Pkt_Dev_Hdr: (Number_of_Devices : Seq_Type);
 Pkt_Dev_Lst: (Dev_Num : Seq_Type;
 Dev_Name : Dname_Type;
 Desc : array [1..40] of char;
 Default : char);
 Pkt_Q_Hdr: (Num_Entries : Seq_Type);

 Pkt_Q_Lst: (Q_Seq : Seq_Type;
 Q_Filename : Fname_Type;
 User : array [1..20] of char;
 Length : array [1..7] of char;
 Date : array [1..10] of char;
 Time : array [1..5] of char);
 Pkt_PDP_Err: (PDP_Error : Char);
 Pkt_Micro_Ok: (* Micro OK has no fields *)();
 Pkt_Print_Sel: (Print_Name : Dname_Type);
 Pkt_Q_Req: (* Request for queue list *)();
 Pkt_Q_Del: (D_Filename : Fname_Type;
 Del_Flag : Char);
 Pkt_Q_Move: (M_Filename : Fname_Type;
 Position : Seq_Type);
 Pkt_Q_Hold: (H_Filename : Fname_Type);
 Pkt_Q_Rel: (R_Filename : Fname_Type);
 Pkt_Prt_Start: (* Print file initialize *)();
 Pkt_Prt_End: (* Print file end *)();
 Pkt_Micro_Err: (Micro_Error : Char);
 End;

 Procedure Receive_Packet( Var Packet : Packet_Rec );
 Procedure Send_Packet ( Var Packet : Packet_Rec );

 Implementation
 Uses SerialIO;
 Procedure PDP_OK ( Var Packet : Packet_Rec;
 Var Comp_Checksum : Integer); forward;
 Procedure Dev_Header ( Var Packet : Packet_Rec;
 Var Comp_Checksum : Integer);
 Procedure Dev_Desc ( Var Packet : Packet_Rec;
 Var Comp_Checksum : Integer);
 Procedure Q_Header ( Var Packet : Packet_Rec;
 Var Comp_Checksum : Integer);
 Procedure Q_Entry ( Var Packet : Packet_Rec;
 Var Comp_Checksum : Integer);
 Procedure PDP_Err ( Var Packet : Packet_Rec;
 Var Comp_Checksum : Integer);
 Procedure Micro_Ack ( Var Packet : Packet_Rec;
 Var Comp_Checksum : Integer);
 Procedure Print_Select ( Var Packet : Packet_Rec;
 Var Comp_Checksum : Integer);
 Procedure Req_Q ( Var Packet : Packet_Rec;
 Var Comp_Checksum : Integer);
 Procedure Del_Entry ( Var Packet : Packet_Rec;
 Var Comp_Checksum : Integer);
 Procedure Move_Entry ( Var Packet : Packet_Rec;
 Var Comp_Checksum : Integer);
 Procedure Hold_Entry ( Var Packet : Packet_Rec;
 Var Comp_Checksum : Integer);
 Procedure Rel_Entry ( Var Packet : Packet_Rec;
 Var Comp_Checksum : Integer);
 Procedure Print_Start ( Var Packet : Packet_Rec;
 Var Comp_Checksum : Integer);
 Procedure Print_End ( Var Packet : Packet_Rec;
 Var Comp_Checksum : Integer);
 Procedure Micro_Err ( Var Packet : Packet_Rec;
 Var Comp_Checksum : Integer);


 Procedure Receive_Packet( Var Packet : Packet_Rec );
 Var Comp_Checksum,Comm_Checksum,Count,Val_Error : Integer;
 ch : Char;
 Err_Flag : Boolean;
 Checksum_Str : string[5];
 begin
 Err_Flag := False;
 Repeat
 comp_checksum := 0;
 with packet do
 begin
 check_receive( ch ); { See if a packet's coming }
 Val(ch, packet_type, val_error);
 Case packet_type of
 Pkt_PDP_OK: PDP_OK ( Packet, Comp_Checksum );
 Pkt_Dev_Hdr: Dev_Header ( Packet, Comp_Checksum );
 Pkt_Dev_Lst: Dev_Desc ( Packet, Comp_Checksum );
 Pkt_Q_Hdr: Q_Header ( Packet, Comp_Checksum );
 Pkt_Q_Lst: Q_Entry ( Packet, Comp_Checksum );
 PKT_PDP_Err: PDP_Err ( Packet, Comp_Checksum );
 else
 begin
 packet_type := Pkt_Micro_Err;
 Micro_Error := chr(Invalid_Checksum);
 Send_Packet( Packet );
 Err_Flag := True;
 end;
 end; { End CASE }
 If not Err_Flag then
 begin
 For Count := 1 to 5 do
 begin
 Check_receive( ch );
 checksum_str := checksum_str + ch;
 end;
 Val(Checksum_str, comm_checksum, val_error);
 If (val_error<>0) or (Comm_Checksum<>Comp_Checksum) then
 begin
 packet_type := Pkt_Micro_Err;
 Micro_Error := chr(Invalid_Checksum);
 Send_Packet( Packet );
 Err_Flag := True;
 end
 else
 begin
 packet_type := Pkt_Micro_Ack;
 Send_Packet( Packet);
 end; { End Error }
 end; { End Checksum Rcv }
 end; { End With Packet }
 Until not Err_Flag;
 end;

 Procedure Send_Packet( Var Packet : Packet_Rec );
 Var ch : Char;
 Comp_Checksum,
 Count,
 Val_Error : Integer;
 Err_Flag : Boolean;

 Checksum_Str : string[5];
 Temp_Packet : Packet_Rec;

 begin
 Err_Flag := False;
 Repeat
 comp_checksum := 0;
 with packet do
 begin
 Case packet_type of
 Pkt_Micro_OK: Micro_Ack ( Packet, Comp_Checksum );
 Pkt_Print_Sel: Print_Select ( Packet, Comp_Checksum );
 Pkt_Q_Req: Req_Q ( Packet, Comp_Checksum );
 Pkt_Q_Del: Del_Entry ( Packet, Comp_Checksum );
 Pkt_Q_Move: Move_Entry ( Packet, Comp_Checksum );
 Pkt_Q_Hold: Hold_Entry ( Packet, Comp_Checksum );
 Pkt_Q_Rel: Rel_Entry ( Packet, Comp_Checksum );
 Pkt_Prt_Start: Print_Start ( Packet, Comp_Checksum );
 Pkt_Prt_End: Print_End ( Packet, Comp_Checksum );
 Pkt_Micro_Err: Micro_Err ( Packet, Comp_Checksum );
 end;
 Str( Comp_Checksum, Checksum_Str );
 While (Length(Checksum_str) < 5) do
 Checksum_Str := '0' + checksum_str;
 For Count := 1 to 5 do
 check_send(checksum_str[count]);
 Receive_Packet( Temp_Packet );
 If Temp_Packet.Packet_Type <> Pkt_PDP_OK then
 Err_Flag := True;
 end; { End With Packet }
 Until not Err_Flag;
 end;

 {**************** Unit Initialization Main Code Block *************}
 Begin
 End.


























October, 1989
LZW DATA COMPRESSION


Here's an all-purpose data compression technique that belongs in your
programming toolbox




Mark R. Nelson


Mark is a programmer for Greenleaf Software, Inc., Dallas, Texas. Mark can be
reached through the DDJ office.


Every programmer should have at least some exposure to the concept of data
compression. Programs such as ARC by System Enhancement Associates (Wayne,
N.J.) and PKZIP by PKWARE (Glendale, Wisc.) are ubiquitous in the MS-DOS
world. ARC has also been ported to quite a few other operating systems, for
example, Unix, CP/M, and so on. CP/M users have long had SQ and USQ to squeeze
and expand programs. Unix users have the COMPRESS and COMPACT utilities. Yet
the data compression techniques used in these programs typically show up in
only two places: File transfers over phone lines and archival storage.
Data compression has an undeserved reputation for being difficult to master,
hard to implement, and tough to maintain. In fact, the techniques used in the
previously mentioned programs are relatively simple, and can be implemented
with standard utilities taking only a few lines of code. In this article, I'll
discuss Lempel-Ziv-Welch (LZW) compression, a good, all-purpose data
compression technique that belongs in every programmer's toolbox.
LZW, for example, by compressing the screens, can easily chop 50K bytes off a
program that has several dozen help screens. With LZW compression, 500K bytes
of software could be distributed to end users on a single 360K byte floppy
disk. Highly redundant database files can be compressed to ten percent of
their original size.


LZW Fundamentals


The original Lempel/Ziv approach to data compression was first published in
1977, and Terry Welch's refinements to the algorithm were published in 1984.
The algorithm is surprisingly simple. In a nutshell, LZW compression replaces
strings of characters with single codes. It does not do any analysis of the
incoming text. Instead, it just adds every new string of characters it sees to
a table of strings. Compression occurs when a single code is output replacing
the string of characters.
The code generated by the LZW algorithm can be of any length, but it must have
more bits in it than a single character. The first 256 codes (when using 8-bit
characters) are by default assigned to the standard character set. The
remaining codes are assigned to strings as the algorithm proceeds. The sample
program runs, as shown in Listing One with 12-bit codes. This means codes 0 -
255 refer to individual bytes, and codes 256 - 4095 refer to substrings.


Compression


The LZW compression algorithm in its simplest form is shown in Figure 1. Each
time a new code is generated, it means a new string has been added to the
string table. Examination of the algorithm shows that LZW always checks
whether the strings are already known and, if so, outputs existing codes
rather than generating new codes.
Figure 1: The compression algorithm

 ROUTINE LZW_COMPRESS
 STRING = get input character
 WHILE there are still input characters DO
 CHARACTER = get input character
 IF STRING+CHARACTER is in the string table THEN
 STRING = STRING+character
 ELSE
 output the code for STRING
 add STRING+CHARACTER to the string table
 STRING = CHARACTER
 END of IF
 END of WHILE
 output the code for STRING

A sample string used to demonstrate the algorithm is shown in Figure 2. The
input string is a short list of English words separated by the / character. As
you step through the start of the algorithm for this string, you can see that
in the first pass through the loop the system performs a check to see if the
string /W is in the table. When it doesn't find the string in the table, it
generates the code for /, and the string /W is added to the table. Because 256
characters have already been defined for codes 0 - 255, the first string
definition can be assigned to code 256. After the system reads in the third
letter, E, the second string code, WE, is added to the table, and the code for
letter W is output. This process continues until, in the second word, the
characters / and W are read in, matching string number 256. In this case, the
system outputs code 256, and adds a three-character string to the string
table. The process again continues until the string is exhausted and all of
the codes have been output.
Figure 2: The compression process

 Input string:/WED/WE/WEE/WEB/WET

 Character input Code output New code value and associated string
 ------------------------------------------------------------------

 /W / 256 = /W
 E W 257 = WE
 D E 258 = ED

 / D 259 = D/
 WE 256 260 = /WE
 / E 261 = E/
 WEE 260 262 = /WEE
 /W 261 263 = E/W
 EB 257 264 = WEB
 / B 265 = B/
 WET 260 266 = /WET
 <EOF> T

The sample output for the string is also shown in Figure 2, along with the
resulting string table. As you can see, the string table fills up rapidly,
because a new string is added to the table each time a code is generated. In
this highly redundant example input, five code substitutions were output,
along with seven characters. If we were using 9-bit codes for output, the
19-character input string would be reduced to a 13.5-byte output string. Of
course, this example was carefully chosen to demonstrate code substitution. In
the real world, compression usually doesn't begin until a sizable table has
been built, usually after at least 100 or so bytes have been read in.


Decompression


The companion algorithm for compression is the decompression algorithm. It
takes the stream of codes output from the compression algorithm and uses it to
exactly recreate the input stream. One reason for the efficiency of the LZW
algorithm is that it does not need to pass the string table to the
decompression code. The table can be built exactly as it occurred during
compression, using the input stream as data. This is possible because the
compression algorithm always outputs the STRING and CHARACTER components of a
code before it uses the code in the output stream. This means that the
compressed data is not burdened with carrying a large string translation
table.
The decompression algorithm is shown in Figure 3. Just like the compression
algorithm, it adds a new string to the string table each time it reads in a
new code. All it needs to do in addition is to translate each incoming code
into a string and send it to the output.
Figure 3: The decompression algorithm

 ROUTINE LZW_DECOMPRESS
 Read OLD_CODE
 output OLD_CODE
 WHILE there are still input characters DO
 Read NEW_CODE
 STRING = get translation of NEW_CODE
 output STRING
 CHARACTER = first character in STRING
 add OLD_CODE + CHARACTER to the translation table
 OLD_CODE = NEW_CODE
 END of WHILE

Figure 4 shows the output of the algorithm given the input created by the
compression discussed earlier in the article. The important thing to note is
that the decompression string table ends up looking exactly like the table
built up during compression. The output string is identical to the input
string from the compression algorithm. Note that the first 256 codes are
already defined to translate to single character strings, just like in the
compression code.
Figure 4: The decompression process

 Input codes:/ W E D 256 E 260 261 257 B 260 T

 Input OLD_CODE STRING CHARACTER New table entry
 NEW_CODE Output
 ------------------------------------------------------

 / / /
 W / W W 256 = /W
 E W E E 257 = WE
 D E D D 258 = ED
 256 D /W / 259 = D/
 E 256 E E 260 = /WE
 260 E /WE / 261 = E/
 261 260 E/ E 262 = /WEE
 257 261 WE W 263 = E/W
 B 257 B B 264 = WEB
 260 B /WE / 265 = B/
 T 260 T T 266 = /WET



The Catch



Unfortunately, the nice, simple, decompression algorithm shown in Figure 4 is
just a little too simple. There is a single exception case in the LZW
compression algorithm that causes some trouble on the decompression side. If
there is a string consisting of a (STRING, CHARACTER) pair already defined in
the table, and the input stream sees a sequence of STRING, CHARACTER, STRING,
CHARACTER, STRING, the compression algorithm outputs a code before the
decompressor gets a chance to define it.
A simple example illustrates the point. Imagine the string JOEYN is defined in
the table as code 300. When the sequence JOEYNJOEYNJOEY appears in the table,
the compression output looks like that shown in Figure 5.
Figure 5: Sample problem

 Input string:... JOEYNJOEYNJOEY ...

 Character input New code value and associated string Code output
 ------------------------------------------------------------------

 JOEYN 300 = JOEYN 288 (JOEY)
 A 301 = NA N
 . . .
 . . .
 . . .
 JOEYNJ 400 = JOEYNJ 300 (JOEYN)
 JOEYNJO 401 = JOEYNJO 400

When the decompression algorithm sees this input stream, it first decodes the
code 300, then outputs the JOEYN string and adds the definition for, lets say,
code 399 to the table, whatever that may be. It then reads the next input
code, 400, and finds that it is not in the table. This is a problem.
Fortunately, this is the only case where the decompression algorithm will
encounter an undefined code. Because it is, in fact, the only case, you can
add an exception handler to the algorithm. The modified algorithm just looks
for the special case of an undefined code and handles it. In the example in
Figure 6, the decompression routine sees code 400, which is undefined. Because
it is undefined, it translates the value of OLD_CODE, which is code 300. It
then adds the CHARACTER value, J, to the string. This results in the correct
translation of code 400 to string JOEYNJ.
Figure 6: The modified decompression algorithm

 ROUTINE LZW_DECOMPRESS
 Read OLD_CODE
 output OLD_CODE
 WHILE there are still input characters DO
 Read NEW_CODE
 IF NEW_CODE is not in the translation table THEN
 STRING = get translation of OLD_CODE
 STRING = STRING+CHARACTER
 ELSE
 STRING = get translation of NEW_CODE
 END of IF
 output STRING
 CHARACTER = first character in STRING
 add OLD_CODE + CHARACTER to the translation table
 OLD_CODE = NEW_CODE
 END of WHILE



The Implementation Blues


The concepts used in the compression algorithm are so simple that the whole
algorithm can be expressed in only a dozen lines. But because of the
management required for the string table, implementation of this algorithm is
somewhat more complicated.
In the code accompanying this article (see Listing One), I have used code
sizes of 12-, 13-, and 14-bits. In a 12-bit code program, there are
potentially 4096 strings in the string table. Each and every time a new
character is read in, the string table has to be searched for a match. If a
match is not found, a new string has to be added to the table. This causes two
problems. First, the string table can get very large very fast. Even if string
lengths average as low as 3 or 4 characters each, the overhead of storing a
variable length string and its code can easily reach 7 or 8 bytes per code. In
addition, the amount of storage needed is indeterminate, as it depends on the
total length of all the strings.
The second problem involves searching for strings. Each time a new character
is read in, the algorithm has to search for the new string formed by
STRING+CHARACTER. This means keeping a sorted list of strings. Searching for
each string takes on the order of log2 string comparisons. Using 12-bit words
potentially means doing 12-string comparisons for each code. The computational
overhead can be prohibitive.
The first problem can be solved by storing the strings as code/character
combinations. Because every string is actually a combination of an existing
code and an appended character, you can store each string as a single code
plus a character. For example, in the compression example shown, the string
/WEE is actually stored as code 260 with appended character E. This takes only
3 bytes of storage instead of 5 (counting the string terminator). By
back-tracking, you find that code 260 is stored as code 256 plus an appended
character E. Finally, code 256 is stored as a /character plus a W.
Doing the string comparisons is a little more difficult. The new method of
storage reduces the amount of time needed for a string comparison, but it
doesn't cut into the number of comparisons needed to find a match. This
problem is solved by using a hashing algorithm to store strings. What this
means is that you don't store code 256 in location 256 of an array, you store
it in a location in the array based on an address formed by the string itself.
When you are trying to locate a given string, you can use the test string to
generate a hashed address and, with luck, can find the target string in one
search.
Because the code for a given string is no longer known merely by its position
in the array, you need to store the code for a given string along with the
string data. In Listing One, there are three array elements for each string.
They are: code_value[i], prefix_code[i], and append_character[i].
When you want to add a new code to the table, use the hashing function in
routine find_match to generate the correct i. find_match generates an address,
then checks to see if the location is already in use by a different string. If
it is, find_match performs a secondary probe until an open location is found.
The hashing function in use in this program is a straightforward xor-type hash
function. The prefix code and appended character are combined to form an array
address. If the contents of the prefix code and character in the array are a
match, the correct address is returned. If that element in the array is in
use, a fixed offset probe is used to search new locations. This continues
until either an empty slot is found, or a match is found. The average number
of searches in the table usually stays below 3 if you use a table about 25
percent larger than needed. Performance can be improved by increasing the size
of the table. Note that in order for the secondary probe to work, the size of
the table needs to be a prime number. This is because the probe can be any
integer between 1 and the table size. If the probe and the table size are not
mutually prime, a search for an open slot can fail even if there are still
open slots available.
Implementing the decompression algorithm has its own set of problems. One of
the problems from the compression code goes away. When you are compressing,
you need to search the table for a given string. During decompression, you are
looking for a particular code. This means that you can store the prefix codes
and appended characters in the table indexed by their string code. This
eliminates the need for a hashing function and frees up the array used to
store code values.
Unfortunately, the method used to store string values causes the strings to be
decoded in reverse order. This means that all the characters for a given
string have to be decoded into a stack buffer, then output in reverse order.
In the program given here, this is done in the decode_string function. Once
this code is written, the rest of the algorithm turns into code easily.
A problem encountered when reading in data streams is determining when you
have reached the end of the input data stream. In this particular
implementation, I have reserved the last defined code, MAX_VALUE, as a special
end of data indicator. Though this may not be necessary when reading in data
files, it is helpful when reading compressed buffers out of memory. The
expense of losing one defined code is minimal in comparison to the convenience
gained.


Results



It is somewhat difficult to characterize the results of any data compression
technique. The level of compression achieved varies quite a bit, depending on
several factors. LZW compression excels when confronted with data streams that
have any type of repeated strings. Because of this, it does extremely well
when compressing English text. Compression levels of 50 percent or better can
be expected. Likewise, compressing saved screens and displays generally shows
great results. Trying to compress data files is a little more risky. Depending
on the data, compression may or may not yield good results. In some cases,
data files compress even more than text. A little bit of experimentation
usually gives you a feel for whether your data will compress well or not.


Your Implementation


The code accompanying this article works. It was written, however, with the
goal of being illuminating, not efficient, with some parts of the code being
relatively inefficient. The variable length input and output routines, for
example, are short and easy to understand, but require a lot of overhead. You
could experience real improvements in speed in an LZW program using
fixed-length 12-bit codes, just by recoding these two routines.
One problem with the code listed here is that it does not adapt well to
compressing files of differing sizes. Using 14- or 15-bit codes gives better
compression ratios on large files (because they have a larger string table to
work with), but poorer performance on small files. Programs such as ARC get
around this problem by using variable length codes. For example, when the
value of next_code is between 256 and 511, ARC inputs and outputs 9-bit codes.
When the value of next_code increases to the point where 10-bit codes are
needed, both the compression and decompression routines adjust the code size.
This means that the 12-bit and 15-bit versions of the program do equally well
on small files.
Another problem on long files is that frequently the compression ratio begins
to degrade as more of the file is read in. The reason for this is simple:
Because the string table is of finite size, after a certain number of strings
have been defined, no more can be added. But the string table is good only for
the portion of the file that was read in while it was built. Later sections of
the file may have different characteristics and really need a different string
table.
The conventional way to solve this problem is to monitor the compression
ratio. After the string table is full, the compressor watches to see if the
compression ratio degrades. After a certain amount of degradation, the entire
table is flushed and gets rebuilt from scratch. The expansion code is flagged
when this happens because the compression routine sends out a special code. An
alternative method is to keep track of how frequently strings are used, and to
periodically flush values that are rarely used. An adaptive technique like
this may be too difficult to implement in a reasonable-sized program.
One final technique for compressing the data is to take the LZW codes and run
them through an adaptive Huffman coding filter. This generally exploits a few
more percentage points of compression, but at the cost of considerably more
complexity in the code as well as quite a bit more run time.


Portability


The code in Listing One was written and tested on MS-DOS machines and has
successfully compiled and executed with several C compilers. It should be
portable to any machine that supports 16-bit integers and 32-bit longs in C.
MS-DOS C compilers typically have trouble dealing with arrays larger than 64K
bytes, preventing an easy implementation of 15- or 16-bit codes in this
program. On machines using different processors, such as the VAX, these
restrictions are lifted, and using larger code sizes becomes much easier.
In addition, porting this code to assembly language should be fairly easy on
any machine that supports 16- and 32-bit math, and offers significant
performance improvements. Implementations in other high-level languages should
be straightforward.


Bibliography


Terry Welch, "A Technique for High-Performance Data Compression," Computer,
June 1984.
J. Ziv and A. Lempel, "A Universal Algorithm for Sequential Data Compression,"
IEEE Transactions on Information Theory, May 1977.
Rudy Rucker, Mind Tools, Houghton Mifflin Company, Boston, Mass.: 1987.


Availability


All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063, or call 800-356-2002 (from inside
Calif.) or 800-533-4372 (from outside Calif.). Please specify the issue number
and format (MS-DOS, Macintosh, Kaypro).

_LZW Data Compression_
by Mark R. Nelson



[LISTING ONE]


/****************************************************************************
** LZW data compression/expansion demonstration program.
** Mark R. Nelson
*****************************************************************************/
#include <stdio.h>

#define BITS 12 /* Setting the number of bits to 12, 13 */
#define HASHING_SHIFT BITS-8 /* or 14 affects several constants. */
#define MAX_VALUE (1 << BITS) - 1 /* Note that MS-DOS machines need to */
#define MAX_CODE MAX_VALUE - 1 /* compile their code in large model if */
 /* 14 bits are selected. */
#if BITS == 14
 #define TABLE_SIZE 18041 /* The string table size needs to be a */
#endif /* prime number that is somwhat larger */
#if BITS == 13 /* than 2**BITS. */
 #define TABLE_SIZE 9029
#endif
#if BITS <= 12

 #define TABLE_SIZE 5021
#endif

void *malloc();

int *code_value; /* This is the code value array */
unsigned int *prefix_code; /* This array holds the prefix codes */
unsigned char *append_character; /* This array holds the appended chars */
unsigned char decode_stack[4000]; /* This array holds the decoded string */

/****************************************************************************
** This program gets a file name from the command line. It compresses the
** file, placing its output in a file named test.lzw. It then expands
** test.lzw into test.out. Test.out should then be an exact duplicate of
** the input file.
****************************************************************************/

main(int argc, char *argv[])
{
FILE *input_file;
FILE *output_file;
FILE *lzw_file;
char input_file_name[81];
/*
** The three buffers are needed for the compression phase.
*/
 code_value=malloc(TABLE_SIZE*sizeof(unsigned int));
 prefix_code=malloc(TABLE_SIZE*sizeof(unsigned int));
 append_character=malloc(TABLE_SIZE*sizeof(unsigned char));
 if (code_value==NULL prefix_code==NULL append_character==NULL)
 {
 printf("Fatal error allocating table space!\n");
 exit();
 }
/*
** Get the file name, open it up, and open up the lzw output file.
*/
 if (argc>1)
 strcpy(input_file_name,argv[1]);
 else
 {
 printf("Input file name? ");
 scanf("%s",input_file_name);
 }
 input_file=fopen(input_file_name,"rb");
 lzw_file=fopen("test.lzw","wb");
 if (input_file==NULL lzw_file==NULL)
 {
 printf("Fatal error opening files.\n");
 exit();
 };
/*
** Compress the file.
*/
 compress(input_file,lzw_file);
 fclose(input_file);
 fclose(lzw_file);
 free(code_value);
/*

** Now open the files for the expansion.
*/
 lzw_file=fopen("test.lzw","rb");
 output_file=fopen("test.out","wb");
 if (lzw_file==NULL output_file==NULL)
 {
 printf("Fatal error opening files.\n");
 exit();
 };
/*
** Expand the file.
*/
 expand(lzw_file,output_file);
 fclose(lzw_file);
 fclose(output_file);
 free(prefix_code);
 free(append_character);
}

/*
** This is the compression routine. The code should be a fairly close
** match to the algorithm accompanying the article.
**
*/
compress(FILE *input,FILE *output)
{
unsigned int next_code;
unsigned int character;
unsigned int string_code;
unsigned int index;
int i;
 next_code=256; /* Next code is the next available string code*/
 for (i=0;i<TABLE_SIZE;i++) /* Clear out the string table before starting */
 code_value[i]=-1;
 i=0;
 printf("Compressing...\n");
 string_code=getc(input); /* Get the first code*/
/*
** This is the main loop where it all happens. This loop runs util all of
** the input has been exhausted. Note that it stops adding codes to the
** table after all of the possible codes have been defined.
*/
 while ((character=getc(input)) != (unsigned)EOF)
 {
 if (++i==1000) /* Print a * every 1000 */
 { /* input characters. This */
 i=0; /* is just a pacifier. */
 printf("*");
 }
 index=find_match(string_code,character); /* See if the string is in */
 if (code_value[index] != -1) /* the table. If it is, */
 string_code=code_value[index]; /* get the code value. If */
 else /* the string is not in the*/
 { /* table, try to add it. */
 if (next_code <= MAX_CODE)
 {
 code_value[index]=next_code++;
 prefix_code[index]=string_code;
 append_character[index]=character;

 }
 output_code(output,string_code); /* When a string is found */
 string_code=character; /* that is not in the table*/
 } /* I output the last string*/
 } /* after adding the new one*/
/*
** End of the main loop.
*/
 output_code(output,string_code); /* Output the last code */
 output_code(output,MAX_VALUE); /* Output the end of buffer code */
 output_code(output,0); /* This code flushes the output buffer*/
 printf("\n");
}
/*
** This is the hashing routine. It tries to find a match for the prefix+char
** string in the string table. If it finds it, the index is returned. If
** the string is not found, the first available index in the string table is
** returned instead.
*/
find_match(int hash_prefix,unsigned int hash_character)
{
int index;
int offset;

 index = (hash_character << HASHING_SHIFT) ^ hash_prefix;
 if (index == 0)
 offset = 1;
 else
 offset = TABLE_SIZE - index;
 while (1)
 {
if (code_value[index] == -1)
 return(index);
if (prefix_code[index] == hash_prefix && append_character[index] ==
hash_character)
 return(index);
 index -= offset;
 if (index < 0)
 index += TABLE_SIZE;
 }
}
/*
** This is the expansion routine. It takes an LZW format file, and expands
** it to an output file. The code here should be a fairly close match to
** the algorithm in the accompanying article.
*/
expand(FILE *input,FILE *output)
{
unsigned int next_code;
unsigned int new_code;
unsigned int old_code;
int character;
int counter;
unsigned char *string;
char *decode_string(unsigned char *buffer,unsigned int code);
 next_code=256; /* This is the next available code to define */
 counter=0; /* Counter is used as a pacifier. */
 printf("Expanding...\n");

 old_code=input_code(input); /* Read in the first code, initialize the */

 character=old_code; /* character variable, and send the first */
 putc(old_code,output); /* code to the output file */
/*
** This is the main expansion loop. It reads in characters from the LZW file
** until it sees the special code used to inidicate the end of the data.
*/
 while ((new_code=input_code(input)) != (MAX_VALUE))
 {
 if (++counter==1000) /* This section of code prints out */
 { /* an asterisk every 1000 characters*/
 counter=0; /* It is just a pacifier. */
 printf("*");
 }
/*
** This code checks for the special STRING+CHARACTER+STRING+CHARACTER+STRING
** case which generates an undefined code. It handles it by decoding
** the last code, adding a single character to the end of the decode string.
*/
 if (new_code>=next_code)
 {
 *decode_stack=character;
 string=decode_string(decode_stack+1,old_code);
 }
/*
** Otherwise we do a straight decode of the new code.
*/
 else
 string=decode_string(decode_stack,new_code);
/*
** Now we output the decoded string in reverse order.
*/
 character=*string;
 while (string >= decode_stack)
 putc(*string--,output);
/*
** Finally, if possible, add a new code to the string table.
*/
 if (next_code <= MAX_CODE)
 {
 prefix_code[next_code]=old_code;
 append_character[next_code]=character;
 next_code++;
 }
 old_code=new_code;
 }
 printf("\n");
}
/*
** This routine simply decodes a string from the string table, storing
** it in a buffer. The buffer can then be output in reverse order by
** the expansion program.
*/
char *decode_string(unsigned char *buffer,unsigned int code)
{
int i;

 i=0;
 while (code > 255)
 {

 *buffer++ = append_character[code];
 code=prefix_code[code];
 if (i++>=4094)
 {
 printf("Fatal error during code expansion.\n");
 exit();
 }
 }
 *buffer=code;
 return(buffer);
}
/*
** The following two routines are used to output variable length
** codes. They are written strictly for clarity, and are not
** particularly efficient.
*/
input_code(FILE *input)
{
unsigned int return_value;
static int input_bit_count=0;
static unsigned long input_bit_buffer=0L;
 while (input_bit_count <= 24)
 {
 input_bit_buffer = (unsigned long) getc(input) << (24-input_bit_count);
 input_bit_count += 8;
 }
 return_value=input_bit_buffer >> (32-BITS);
 input_bit_buffer <<= BITS;
 input_bit_count -= BITS;
 return(return_value);
}
output_code(FILE *output,unsigned int code)
{
static int output_bit_count=0;
static unsigned long output_bit_buffer=0L;
 output_bit_buffer = (unsigned long) code << (32-BITS-output_bit_count);
 output_bit_count += BITS;
 while (output_bit_count >= 8)
 {
 putc(output_bit_buffer >> 24,output);
 output_bit_buffer <<= 8;
 output_bit_count -= 8;
 }
}


















October, 1989
HIGH-SPEED FILE TRANSFERS WITH NETBIOS


Don't let normal serial communication slow you down




Costas Menico


Costas is a senior software developer and part owner of The Software Bottling
Company and a regular contributor to DDJ. He can be reached at 6600 Long
Island Expressway, Maspeth, NY 11378. MCI Mail SBC. CompuServe: 72377, 1121.


If you regularly transfer large amounts of data between PCs, you're familiar
with the time-consuming constraints of 1200-, 2400-, or even 9600-baud
communications. No doubt you've wished more than once you could transfer files
at speeds that local area network (LAN) users have become used to; speeds as
(relatively) low as 2,500,000 bits/ sec. (2.5-Mbits/sec.) for ARCnet, to as
high as 10-Mbits/sec. for Ethernet.
Considering that in many environments, LANs are getting to be as prevalent as
hard disks, high-speed file exchange using LAN facilities is becoming
increasingly common. But even if you don't have access to a LAN, it's still
possible to use LAN adapter cards, which now sell for as low as $100 to $200
each, as a means of exchanging files at high speeds between two (or more) PCs.
You don't even need the network server or operating system to do so. All you
need are the cards, a cable, and a quasi-standard driver called NetBIOS, which
is usually supplied or sold with most LAN adapter cards -- and the program
I'll describe in this article.
The program, which I call XNet, is a custom file transfer program that uses
the NetBIOS interface. The nice thing about the program is you don't have to
concern yourself with the usual serial communications nuisances -- error
corrections, checksums, baud rates, and such. NetBIOS does all this (and more)
for you. The program doesn't even have to wait around to receive data because
NetBIOS can notify your routine when there is data and let you know if a data
transfer to another node failed. The beauty of all this is that data is
transferred from 2.5-Mbits/sec. to 10-Mbits/ sec., depending on the type of
adapter cards you're using.


The NetBIOS Interface


NetBIOS is usually loaded into DOS as a device driver from the CONFIG.SYS.
Follow the board manufacturer's installation guidelines. Before calling
NetBIOS, you must set the command and other parameters in the message control
block (MCB). You then load the register pair ES:BX with the segment and offset
address of the MCB. To call the NetBIOS, you execute an interrupt 5Ch. For a
list of NetBIOS commands, see Figure 1; for the structure of the MCB, refer to
Figure 2. When the interrupt completes, you check the mcb_retcode for a
successful completion. If it is non-zero, it usually means there was a problem
that your program must react to.
Figure 1: NetBIOS commands and values

 NetBIOS commands used in this program

 msg_reset=$32; Reset the node
 msg_status=$33; Determine the current state of the node
 msg_add_name=$30; Add a 16 char unique node name to NetBIOS
 msg_listen=$11; Listen for a node to establish session
 msg_call=$10; Call another node to establish a session
 msg_hang_up=$12; Hangup the session with a node
 msg_send=$14; Send a block of data to a node
 msg_receive=$15; Receive a block of data from a node

Other NetBIOS commands

 msg_add_group_name=$36; Add a group name
 msg_cancel=$35; Cancel the last MCB command
 msg_chain_send=$17; Send 2 data buffers, one after
 another
 msg_delete_name=$31; Delete a name from the adapter
 msg_find_name=$78; Find a a node name on the LAN
 msg_receive_any=$16; Receive from any session partner
 msg_receive_broadcast_datagram=$23; Datagram from any node on LAN
 msg_receive_datagram=$21; Datagram from a specific name/group
 msg_send_broadcast_datagram=$22; Datagram to anyone on LAN
 msg_send_datagram=$20; Datagram to specific name/group
 msg_session_status=$34; Status of a session

Figure 2: NetBIOS message control block

 buffer=array[1..buffsize] of byte; Buffer type declaration
 buffp=^buffer; Pointer type to the buffer
 arrname=array[1..16] of char; Array for names type


 Message control block record

 mcb=record
 mcb_command: byte; Command to execute
 mcb_retcode: byte; Return code value
 mcb_lsn: byte; Local session #
 mcb_num: byte; Number of name added
 mcb_buffer: pointer; Data buffer address
 mcb_length: word; Buffer length in bytes
 mcb_callname: arrname; Name on remote node
 mcb_name: arrname; Name of local node
 mcb_rto: byte; Receive timeout (NOT USED)
 mcb_sto: byte; Send timeout (NOT USED)
 mcb_post: pointer; Post routine address (NOT USED)
 mcb_lana_num: byte; Adapter card to use. 0 is first
 mcb_cmd_cpl: byte; Command status if NOWAIT is used
 mcb_reserve: array[1..14] of byte; Other detailed info
 end;

Although, not implemented in the XNet program, a feature that needs special
attention must be explained. Most commands allow you to call with the no-wait
bit (bit 8) set. This means that after you have executed the call to NetBIOS,
you do not have to wait for the completion. Rather, NetBIOS notifies you by
calling a designated routine called a "post routine." The post is an
interrupt-like function that you write, and whose address you set in the
mcb_post field of the MCB. When your routine is finally called, your program
then checks the MCB for errors, or any other relevant fields. This is a
powerful feature, for without it, most network software would not work so
elegantly.


MCB Explained


The MCB fields are set by your program before calling the NetBIOS and are set
again by NetBIOS after the call. You may have many independent MCBs as long as
you point ES:BX to the right one before calling NetBIOS. Not all commands use
all the fields.
The first and most important field is the mcb_command. Here, you set the code
for the command you wish to execute (see Figure 1). The next field,
mcb_retcode, is where the completion return code is set by NetBIOS. It must be
set to FFh before calling NetBIOS. The session number mcb_lsn is returned to
your program by the NetBIOS after a successful connection with another node on
the system. You use this for all interactions with that particular node. The
number mcb_num identifies a local node name in the adapter card, while
mcb_buffer is a pointer to the data buffer for receiving or sending blocks of
data. The mcb_length determines the length of the data in the mcb_buffer,
mcb_callname, and mcb_name are the names used when calling or listening for
another node. The send and receive time-out values are mcb_rto and mcb_sto. If
they are set to zero, there will never be a data transmission time-out because
of a problem or delay on the network due to other traffic. The mcb_post field
is set to point to our interrupt routine handler when we call NetBIOS with the
no-wait in the command field bit set. The LAN card number, usually set to 0,
is mcb_lana_num, and mcb_cmd_cpl is a command completion code when we call
NetBIOS with the no-wait bit in the command field set. The mcb_reserve field
is used to fill in information by some NetBIOS commands.


NetBIOS Commands


As you can see from Figure 1, the NetBIOS has quite an assortment of commands.
For now, I'll explain only the commands used in the XNet program.
The first command is the msg_reset, which resets the LAN adapter card and
clears out any node names it may have been using. You should only use this
command if no other software is using the LAN card at the time. The command
msg_status returns the LAN card's node address, a unique number between 1 and
255, depending upon the installation of your LAN adapter. No two adapters can
have the same node number. The msg_add_name command allows your node to be
known by some arbitrary, unique name on the network. Your node can have
multiple names, but no two nodes can have the same name. The msg_listen
command makes your program wait for someone to talk to you and is the opposite
of msg_call. Together they use each other's names to establish a session. The
commands msg_send and msg_receive send and receive data via mcb_buffer and
msg_hang_up, which hang up any session with another node.
Your program could be communicating with many nodes at once, provided you have
established a unique session with them. Each session has its own number that
is used for this purpose.


The XNet File Transfer Program


Even though I chose to write XNet in Turbo Pascal (see Listing One), you can
use C, or for time/space critical functions, assembler. The program is as
simple as you can get for transferring data from one node to the other. I did
not use the no-wait capability; the program must be run simultaneously on each
system (run it on node A first, and then run it on node B).
I programmed each NetBIOS function into a procedure call passing the required
parameters. All the procedures start with net_. The procedure init_mcb is
called to set the MCB to a known state. Then each one of the commands sets the
calling parameters before executing an interrupt 5Ch.
Notice that net_reset and net_status check if the NetBIOS driver is installed.
This is done by checking if the 5Ch vector value is set to a non-zero value.
Any errors during a call to NetBIOS will cause a message indicating the
command and error code (in hex) to be displayed on the screen. The program
then terminates by calling the terminate function.
Although XNet should work with any adapter board configuration that supports
NetBIOS, I tested it on PC210 ARCnet boards from Standard Microsystem Corp.
(Long Island, N.Y.) using their NetBIOS.
My CONFIG.SYS file had the following line which tells NetBIOS the port, the
IRQ, and the memory buffer for the board:
 DEVICE=SMCARC.SYS /P2E0 /I2 ME000.
To compile the program, simply compile to disk. To run the EXE file, type XNET
at the DOS prompt.
Start the program on whatever nodes will be talking to each other. You'll
first see your node's number on the screen, and then you'll be prompted for
the remote node's number (a number between 1 and 255).
You are now ready to receive on one station and send on the other. On station
A, select Receive and give the file name to save into. On station B, select
Send and give the file name to send. As soon as you give the file name to
send, the program will transmit the file, showing in bytes the size of the
file transferred. For small files (around 128K), the transfer is practically
instantaneous. In cases of a message specifying the error code, the command
code the error occurred on will appear.


Program Flow


When the program is started up, it will first call the net_reset procedure.
This initializes the board and checks that you have NetBIOS running. After the
reset, the program calls net_status to get the node's number. This is the
first byte of the data pointed to by mcb_buffer. For convenience, I use this
number as the name of the node by converting it to a string and calling
net_add_name. For the other node to communicate, it must use this name.
The program will then prompt for the remote node's number which must be
different from the local. It will then ask if you wish to Send, Receive, or
Exit. If you choose Exit, the program terminates by calling the procedure
Terminate. If you choose Send, it will call the procedure setup_call_send, and
for Receive, it will call setup_listen_receive.
The procedure setup_listen_receive asks for the file name to save to. If the
file exists, it will ask you to overwrite it. It will then proceed to listen
for a call from the remote node using the function net_listen, and wait
indefinitely.
The setup_call_send procedure asks for the file name to send, verifies its
existence, and calls the remote node using the function net_call. If the
remote node is not listening, it prints a message to Retry or Abort. If you
abort, the program terminates; otherwise, it reexecutes the net_call.
Once the two nodes establish a session via the above description, a session
number is returned by NetBIOS. This session number is then used to send the
file and receive it on the other end. The procedures send_the_file and
receive_the_file will perform this function.

The file is sent/received using net_send and net_receive, respectively. The
first send is a 4-byte (2 word) file size number that tells the receiver how
big a file to expect. The sender will send the file in chunks of 64K bytes
with the last chunk being the remainder of the 64K. As soon as the file size
is reached, they both terminate by calling terminate. If there is a disk error
(such as disk full) during the receive, the program terminates by hanging up
-- using net_hang_up and calling the terminate function.


Technical References


Standard Microsystems Corp., Long Island, N.Y. ARCNET Installation Guide.
IBM, Token-Ring Network PC Adapter, Technical Reference Manual.


Availability


All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063, or call 800-356-2002 (from inside
Calif.) or 800-533-4372 (from outside Calif.). Please specify the issue number
and format (MS-DOS, Macintosh, Kaypro).

_High-Speed File Transfers With NetBios_
by Costas Menico



[LISTING ONE]


program xnet;
{
 Program to demonstrate file transfer between PCs
 using the NETBIOS device driver. This program should work with
 any hardware and software that support the NETBIOS interface.
 Network software other than the NETBIOS is not required.
 Program was tested with the PC260 Arcnet boards from SMC. The
 CONFIG.SYS had the following line to install the NETBIOS:
 device=smcarc.sys /p2e0 /i2 /me000
 Program author: Costas Menico
}

{$I-,R-}
uses dos, crt;

const
 { Maximum # of bytes to transfer in a single send }
 buffsize = 64*1024-1;
 lancard=0; { Default network card }
 nowait = $80; { Return immediately from command.
 Call the POST routine when done. (NOT USED)}
 wait=$0; { Wait until command is done. }

 { NETBIOS Commands used in this program }
 msg_reset=$32; { Reset the node }
 msg_status=$33; { Determine the current state of the node }
 msg_add_name=$30; { Add a 16 char unique node name to NETBIOS }
 msg_listen=$11; { Listen for a node to establish session }
 msg_call=$10; { Call another node to establish a session }
 msg_hang_up=$12; { Hangup the session with a node }
 msg_send=$14; { Send a block of data to a node }
 msg_receive=$15; { Receive a block of data from a node }

type
 buffer=array[1..buffsize] of byte; { Buffer type declaration }
 buffp=^buffer; { Pointer type to the buffer }

 arrname=array[1..16] of char; { Array for names type }

 { Message control block record }
 mcb=record
 mcb_command: byte; { Command to execute }
 mcb_retcode: byte; { Return code value }
 mcb_lsn: byte; { Local session # }
 mcb_num: byte; { Number of name added }
 mcb_buffer: pointer; { Data buffer address }
 mcb_length: word; { Buffer length in bytes }
 mcb_callname: arrname; { Name on remote node }
 mcb_name: arrname; { Name of local node }
 mcb_rto: byte; { Receive timeout (NOT USED) }
 mcb_sto: byte; { Send timeout (NOT USED) }
 mcb_post: pointer; { Post routine address (NOT USED) }
 mcb_lana_num: byte; { Adapter card to use. 0 is first }
 mcb_cmd_cpl: byte; { Command status if NOWAIT is used }
 mcb_reserve: array[1..14] of byte; { Other detailed info }
 end;

{ Memory declarations }
var
 b: buffp; { Data buffer block }
 m: mcb; { Message control block }
 r: registers; { Registers used in INT $5C }

 localname, callname: arrname; { Local and remote name variables }

 netaddr: pointer; { NETBIOS $5C Interrupt address }

 fi: file; { File handle for reading or writing }
 filename: string[64]; { Filename path string }

 mode: char; { Sending or receiving }
 nodenum: word; { Our card's node number, 1-255 }

 remotenode,
 localnode: string[3]; { Remotes and local node numbers }

 lsn: byte; { Tracks our session number }

 fsize, bytecount: longint; { File size and bytes sent/received }
 count: word; { Number of bytes to send/receive }

 noerr: boolean; { General use error flag }
 ans: char; { Readkey variable }
{-------------------------------------------------------------------}
procedure init_mcb(var m:mcb);
{ Initialize a message control block to blanks and nulls }
begin
 m.mcb_command:=0;
 m.mcb_retcode:=$ff; { Must be set to $FF }
 m.mcb_lsn:=0;
 m.mcb_num:=0;
 m.mcb_buffer:=nil;
 m.mcb_length:=0;
 fillchar(m.mcb_callname,16,' ');
 fillchar(m.mcb_name,16,' ');
 m.mcb_rto:=0;

 m.mcb_sto:=0;
 m.mcb_post:=nil;
 m.mcb_lana_num:=lancard;
 m.mcb_cmd_cpl:=0;
 fillchar(m.mcb_reserve,14,0);
end;
{-------------------------------------------------------------------}
procedure net_reset(var m:mcb);
{ Reset the node card }
begin
 init_mcb(m);
 m.mcb_command:=msg_reset;
 netaddr:=ptr(memw[0:$5c*4], memw[0:$5c*4+2]);
 if netaddr<>nil then
 begin
 r.es:=seg(m);
 r.bx:=ofs(m);
 intr($5c,r);
 end;
end;
{-------------------------------------------------------------------}
procedure terminate;
{ Terminate XNET }
begin
 close(fi); { Close open file }
 if ioresult<>0 then ;{ Clear the error flag just in case }
 net_reset(m); { Reset the adapter. Deletes all activity }
 freemem(b,buffsize); { Free heap memory (Out of Habit) }
 halt; { Go have coffee and think about enhancements}
end;
{-------------------------------------------------------------------}
procedure net_error(var m: mcb);
{ Print a NETBIOS error and prompt user }
var ans: char;
 function hex(h:byte):string;
 { Convert a byte to hex notation }
 var i:byte;
 hexc:string[2];
 const
 hs:string[16]='0123456789ABCDEF';
 begin
 i:=(h shr 4);
 hexc:=hs[i+1];
 i:=(h and $0f);
 hexc:=hexc+hs[i+1];
 hex:=hexc;
 end;
begin
 if m.mcb_retcode=0 then exit;
 writeln('NETBIOS error code $',hex(m.mcb_retcode),
 ' in command code $',hex(m.mcb_command));
 ans:=readkey;
 terminate;
end;
{-------------------------------------------------------------------}
procedure net_status(var m:mcb; waitbit:byte; mcb_buffer:buffp;
 mcb_length:word; mcb_callname:arrname;
 mcb_post: pointer);
{ Get the current NETBIOS status }

begin
 init_mcb(m);
 m.mcb_command:=waitbit+msg_status;
 m.mcb_buffer:=mcb_buffer;
 m.mcb_length:=mcb_length;
 m.mcb_post:=mcb_post;
 move(mcb_callname,m.mcb_callname,16);
 netaddr:=ptr(memw[0:$5c*4], memw[0:$5c*4+2]);
 if netaddr<>nil then
 begin
 r.es:=seg(m);
 r.bx:=ofs(m);
 intr($5c,r);
 end;
end;
{-------------------------------------------------------------------}
procedure net_receive(var m:mcb; waitbit:byte; mcb_buffer:buffp;
 mcb_length:word; mcb_lsn:byte;
 mcb_post: pointer);
{ Wait to receive a data block from the node we are in session with }
begin
 init_mcb(m);
 m.mcb_command:=waitbit+msg_receive;
 m.mcb_buffer:=mcb_buffer;
 m.mcb_length:=mcb_length;
 m.mcb_lsn:=mcb_lsn;
 m.mcb_post:=mcb_post;
 r.es:=seg(m);
 r.bx:=ofs(m);
 intr($5c,r);
end;
{-------------------------------------------------------------------}
procedure net_hang_up(var m:mcb; waitbit:byte; mcb_lsn:byte;
 mcb_post: pointer);
{ Hang up on the other guy. Not polite but who's perfect. }
begin
 init_mcb(m);
 m.mcb_command:=waitbit+msg_hang_up;
 m.mcb_lsn:=mcb_lsn;
 m.mcb_post:=mcb_post;
 r.es:=seg(m);
 r.bx:=ofs(m);
 intr($5c,r);
end;
{-------------------------------------------------------------------}
procedure net_send(var m:mcb; waitbit:byte; mcb_buffer:buffp;
 mcb_length:word; mcb_lsn:byte; mcb_post: pointer);
{ Send a block of data to the node we are in session with. }
begin
 init_mcb(m);
 m.mcb_command:=waitbit+msg_send;
 m.mcb_buffer:=mcb_buffer;
 m.mcb_length:=mcb_length;
 m.mcb_lsn:=mcb_lsn;
 m.mcb_post:=mcb_post;
 r.es:=seg(m);
 r.bx:=ofs(m);
 intr($5c,r);
end;

{-------------------------------------------------------------------}
procedure net_add_name(var m:mcb; waitbit:byte; mcb_name:arrname;
 mcb_post: pointer);
{ Tell NETBIOS our name. Must be unique anywhere in the network }
begin
 init_mcb(m);
 m.mcb_command:=waitbit+msg_add_name;
 move(mcb_name,m.mcb_name,16);
 m.mcb_post:=mcb_post;
 r.es:=seg(m);
 r.bx:=ofs(m);
 intr($5c,r);
end;
{-------------------------------------------------------------------}
procedure net_call(var m:mcb; waitbit:byte; mcb_callname,
 mcb_name:arrname; mcb_post: pointer);
{ Call callname, and let him know we are ready }
begin
 init_mcb(m);
 m.mcb_command:=waitbit+msg_call;
 move(mcb_name,m.mcb_name,16);
 move(mcb_callname,m.mcb_callname,16);
 m.mcb_post:=mcb_post;
 r.es:=seg(m);
 r.bx:=ofs(m);
 intr($5c,r);
end;
{-------------------------------------------------------------------}
procedure net_listen(var m:mcb; waitbit:byte; mcb_callname,
 mcb_name:arrname; mcb_post: pointer);
{ Listen if callname is calling us }
begin
 init_mcb(m);
 m.mcb_command:=waitbit+msg_listen;
 move(mcb_name,m.mcb_name,16);
 move(mcb_callname,m.mcb_callname,16);
 m.mcb_post:=mcb_post;
 r.es:=seg(m);
 r.bx:=ofs(m);
 intr($5c,r);
end;
{-------------------------------------------------------------------}
procedure copytoarr(s: string; var name: arrname);
{ Copy a string to a 16 byte array. Blank fill to end. }
begin
 fillchar(name,16,' ');
 move(s[1], name, length(s));
end;
{-------------------------------------------------------------------}
procedure send_the_file;
{
 Start sending file. First send the file size (2 words).
 Then send the rest in block of 64K with the remainder
 as the last block.
}
begin
 { Get file size and display }
 fsize:=filesize(fi);
 gotoxy(1,23); write('File size ',fsize);


 { Send the length of the file. Must be in 2 words }
 move(fsize, b^, 4);
 net_send(m, wait, b, 4, lsn, nil);
 net_error(m);

 bytecount:=0;
 noerr:=true;

 { Loop until the file is sent. }
 while (bytecount<fsize) and (noerr) do
 begin
 { Read a block and if no error then send }
 blockread(fi, b^, buffsize, count);
 if ioresult<>0 then
 noerr:=false
 else
 begin
 net_send(m, wait, b, count, lsn, nil);
 net_error(m);
 bytecount:=bytecount+count;
 gotoxy(1,24); write('File size sent ',bytecount,' ');
 end;
 end;
end;
{-------------------------------------------------------------------}
procedure receive_the_file;
{
 Start receiving file and save to disk. First get the file size.
 Then receive in blocks of 64K with the remainder as the last block
}
begin
 { Get the file size. Block sent must be in 2 words }
 net_receive(m, wait, b, buffsize, lsn, nil);
 move(b^,fsize,4);
 { Display it }
 gotoxy(1,23); write('File size ',fsize);
 bytecount:=0; { File size sent counter }
 noerr:=true;
 { Loop, receiving block in 64K increments }
 while (bytecount<fsize) and (noerr) do
 begin
 { Receive }
 net_receive(m, wait, b, buffsize, lsn, nil);
 net_error(m);
 { Save to file }
 blockwrite(fi, b^, m.mcb_length);
 { If an error abort else show file size sent so far. }
 if ioresult<>0 then
 begin
 noerr:=false;
 writeln('Disk full error');
 net_hang_up(m, wait, lsn, nil);
 terminate;
 end else
 begin
 bytecount:=bytecount+m.mcb_length;
 gotoxy(1,24); write('File size received ',bytecount,' ');
 end;

 end;
end;
{-------------------------------------------------------------------}
procedure setup_call_send;
{ Ask for file name to send and call the remote station. Hopefully
 the remote is listening }
begin
 noerr:=true;
 { Get the file name to send }
 while noerr do
 begin
 write('Pathname of file to send (blank to exit)? ');
 readln(filename);
 if filename='' then terminate;
 assign(fi,filename);
 reset(fi,1);
 if ioresult<>0 then
 writeln('File does not exist.')
 else
 noerr:=false;
 end;
 { Get the local node and the remote node into arrays}
 copytoarr(localnode,localname);
 copytoarr(remotenode,callname);
 { Call 'callname' using our 'localname'. He should be
 expecting our call }
 noerr:=false;
 while not noerr do
 begin
 net_call(m, wait, callname, localname, nil);
 { Was the remote node available to listen? }
 if m.mcb_retcode<>0 then
 begin
 writeln('Remote Node, ',remotenode,' not ready. Retry/Abort?');
 ans:=readkey;
 if upcase(ans)='A' then net_error(m);
 end else
 noerr:=true;
 end;
 lsn:=m.mcb_lsn; { Save the session number NETBIOS blessed us with}

 send_the_file;
 close(fi);
end;
{-------------------------------------------------------------------}
procedure setup_listen_receive;
{ Ask for file name to receive into and listen for the remote
 node's call }
begin
 noerr := true;
 { Get filename to save in to. If file exists verify and overwrite. }
 while noerr do
 begin
 write('Pathname of where to save received file (blank to exit)? ');
 readln(filename);
 if filename='' then terminate;
 assign(fi,filename);
 reset(fi);
 if ioresult=0 then

 begin
 writeln('File EXISTS. Do you wish to overwrite (Y/N)? ');
 ans:=readkey;
 if upcase(ans)='Y' then noerr:=false;
 close(fi);
 end else
 noerr:=false;
 end;
 rewrite(fi,1);
 { Get the local and remote nodes into array strings }
 copytoarr(localnode,localname);
 copytoarr(remotenode,callname);
 { Listen for the remote node to call up any moment }
 net_listen(m, wait, callname, localname, nil);
 lsn:=m.mcb_lsn; { Save the session number NETBIOS blessed us with}
 net_error(m);

 receive_the_file;
 close(fi);
end;
{-------------------------------------------------------------------}
{ XNET Main program start }
{-------------------------------------------------------------------}
begin

 clrscr;
 { Get a data buffer from the heap }
 getmem(b, buffsize);
 { Initialize fi to something }
 assign(fi,'NUL');
 { Are we supposed to reset? }
 net_reset(m);
 net_error(m);
 { Check our status }
 copytoarr('*', localname); { Create our localname }
 { Check our node's NETBIOS status and
 in to the first get the node number (address) }
 net_status(m, wait, b, buffsize, localname, nil);
 net_error(m);
 { Get our node number and add it as a node name
 The node number is set by "net_status" and is
 in the first byte of the data buffer "b^"}
 nodenum:=mem[seg(b^):ofs(b^)];
 writeln('Your Station Number is: ',nodenum); writeln;

 str(nodenum,localnode); { Convert to string array }
 copytoarr(localnode,localname);
 net_add_name(m, wait, localname, nil); { Add to NETBIOS }
 net_error(m);
 { At this point the NETBIOS is aware of our presence }
 { Ask the user for the remote's node number.
 This is the node we wish to communicate with.
 It may not have the same number as our node }
 remotenode:=localnode;
 while (remotenode=localnode) do
 begin
 write('Enter remotes station #: ');
 readln(remotenode);
 end;

 { Ask for user's intentions. Send/Receive/Exit }
 writeln('[S]end-file, [R]eceive-file or [E]xit');
 mode:=readkey;

 case upcase(mode) of
 'S': setup_call_send; { Send the file. }
 'R': setup_listen_receive; { Receive the file. }
 end;
 terminate;
end.




















































October, 1989
FINITE STATE MACHINES FOR XMODEM


FSMs are one way to cope with the chaos of communications


 This article contains the following executables: SMITH.ARC


Donald W. Smith


Don is a senior course developer and instructor for Wide Area Network Systems
at Tandem Computers, Inc. He has been programming micros since CP/M days and
has been programming in C since 1985. He can be contacted at 7029 Via Del Rio,
San Jose, CA 95139 or on CompuServe: 76515,3406.


Even though my coworkers often refer to me as a "Commie," I don't take offense
because my specialty is data communications and networking, an area of
computer science that is often overlooked in favor of more stimulating topics
such as compiler technology. But what we Commies do and what compiler writers
do aren't that much different, particularly because we share an important tool
called "finite state automata" (FSA). The main difference between what they do
and what we do is where we get our events. Compiler events come from source
code files (text tokens), while most communication events come from another
computer, or from strange things in between.
This article presents a more generalized description of FSA, one in which data
structures and code are used to implement XModem, a well-understood
communications protocol, using a technique that is general enough to be
implemented with just about any other protocol as well. The goal here is to
explain how to use standard C language constructs to write dependable,
maintainable programs using finite state machines (FSM). The applications
include other communications protocols, realtime data acquisition, or just
about anything with a fairly predictable flow of events.


Lexical Analyzers and Such


The Unix environment provides a number of specialized tools, YACC (yet another
compiler compiler) and LEX (a lexical analyzer), for instance, that are
designed to maintain the sanity of compiler writers. YACC utilizes grammatical
rules that can have associated actions. YACC can be adapted to generate
FSM-type machines, but requires that you learn "yet another set of syntax."
(YASS?)
Other utilities have also appeared to generate FSM code. They are all valid
techniques, if the tool is well understood by developers as well as
maintainers. But all require an extra level of "indirection" in the coding
process to learn a specialized tool. The technique I examine in this article
does not use smoke screens or mirrors and requires only a standard C compiler.


Finite State Basics


Finite state machines are directed graphs where nodes are called states, and
arcs are actions (or transitions). Each state includes a well-defined set of
events. There must be an action and a next state defined for each event, even
if the action is a null action. Events drive the machine from state to state.
Once a good starting state is established and an initial event has come in,
the machine is off and running.


State Diagrams


State diagrams are often used to document state machines. Figure 1 is the
state diagram for XModem receive. The receive machine (Figure 2) uses a total
of four states (the circles), with a maximum of four events each. As the
number of events increases, the state diagrams sacrifice some attractiveness.
The final state (Exit) is used only to help human readers; it does not respond
to events, and is not included in the diagram. Each event directs the machine
to an action, illustrated by a box with rounded corners. All actions in this
example are passed parameters, which are usually #defines and are shown in
parentheses. Some actions are shared by multiple events, for instance,
Frame_Wait(RESEND). Unfortunately, there is no "Industry Standard" state
diagram for the XModem protocol. (Ward Christensen isn't the only one who
forgot this step. But then again, CP/M machines didn't do graphics well.) The
state diagram in Figure 1 is exceptionally short and sweet because the
protocol is simple.


Coding Technique


There are many ways to implement finite state machines. One method, using a
precompiler (similar to LEX and YACC), adds an extra layer of syntax on top of
the source code to identify states, events and actions. Remember, FSA are
normally a compiler writer's game. What could be more natural than yet another
language? The output C code is often strewn with labels and GOTOs, which are
not part of my structured programming vocabulary.
It is also possible to limit the number of global variables (thus, side
effects) when hand coding state machine logic. This provides stand-alone send
and receive modules that share a short list of external variables. A short
list of global variables in each module is declared static, to prevent them
from slipping past the guards at the file door.
The C language provides the data type of pointer to function that is as
flexible as using GOTOs and labels, but also provides a series of advantages,
such as that parameters can be passed, values can be returned, and block
structuring maintained.


Static Tables


Data structures may be initialized with anything that is a constant at compile
time. Function pointers and pointers to variable locations fully qualify, but
the contents of variables do not.
XMRECV (see Listing Three) shows the initialized state table using function
pointers to actions (A_...). The parameter field is defined as an int. This
int parameter conveniently accommodates pointers to anywhere in the small
model.
Most state machines use actions that are inherently simple, and rely on entry
points (labels) to prevent duplication of code. This technique uses standard
functions for actions which naturally take parameters. This code requires all
functions to accept a single int parameter, due to the struct used for the
state table. Other parameter types are casted to int as required.
Unfortunately, this technique is not universally portable. For example, if
pointers occupy more than the size of an int, a larger data type should be
substituted to hold them.


Code Walkthrough



The FSM presented here consists of five modules. Listing One CTERM.H, and
Listing Two, COMMN.H, define the system. Most of my discussion, however, will
concentrate on the short "main loop" of the XMRECV.C file in Listing Three.
The same technique is also used in the XMSEND.C module Listing Four) but
refers to a different state table. The action functions speak for themselves
and illustrate the XModem protocol, which has been done before. CTERM1.C,
Listing Five, is a terminal emulator that demonstrates the use of state
machine driven communications protocols using the C language. Use makect1 in
Listing Six to compile CTERM1.C.
The story normally starts in terminal mode. To receive a file, press the PgDn
key to get into the xmodem_recv() module. Note that the mode variable is set
to M_XRecv on the way out of terminal mode. The initial action of
A_Prep_Recv() returns the first event and, hopefully, a valid file name. A
while (mode == M_XRecv) loop takes control until something good (or really
bad) happens.
Within the while loop, a copy of the event is made for future reference. (Sort
of like saving yesterday's newspaper, right?) Then a pointer is set up for
easier access to the current state table entry (cur_entry). The current state
and event of the state machine is then traced, if tracing was enabled at
compile time. The predefined action to execute is determined then called,
passing the appropriate parameter. Notice that the event variable is filled
with the return value from the function call to new_action( ). Before leaving,
the system determines a new current state from the next_state field of the
state table.
In this case, the user should be presented with the opportunity to abort at
any time. The routine keyfun() provides this capability, without allowing the
dreaded ^C (Control-C) display or program termination. Programs that alter
interrupt vectors should not abort without putting things back the way they
were. One single character is defined to be the escape character (default is
ESC) that takes you out of the M_XRecv via a call to the action A_Recv_End().
This code could be tighter and run faster, but a few extra variables help
follow the action and keep the code more readable. Hopefully, smart compilers
will use registers where possible anyway.


Designing States


One of the most challenging aspects of designing good state machines is
determining how many states there should be. There are no hard and fast rules.
This is one of those areas of "fuzzy logic" that intelligent humans were built
to handle. The goal is to define states where only a limited number of events
can occur. That number is up to the designer, but normally should not exceed
ten.
It is easier to define the structure to hold the initialized state table if
all states have the same number of valid events. It could be an interesting
exercise for the reader to design a state table with a variable number of
events for each state.
The following are a few notes to keep in mind while determining how many
states to define:
Use a flowchart of tasks to help derive the state diagram
keep the main path clean (as few states/actions as possible)
decide which return value means "All's Well." Maybe O?


Actions and Reactions


Actions are functions that do work and return events. Most actions normally do
not call other actions and perform fairly simple tasks. It is often necessary
to maintain some idea of context between calls. For example, the A_Frame_Wait
action maintains internal (static) counters for retries while waiting for a
response to a control character sent (Ack, Nak, and so on).
Remember that states do no work. (The Governor of California would probably
take exception to my last statement, but then I'd have been quoted out of
context.) Actions are where the actual work is done.
The action Validate() probably does the most work in the receive logic. It
updates the screen's packet counter and waits for the interrupt handler to
finish receiving the frame. It then checks the packet header and the packet
CRC/Checksum, and finally writes the packet data to disk if all is well.
Validate() also has the authority to advance the (file) global packet counter
(pkt) before returning 0, if all is well.
Some protocol purists may notice that it is possible to tolerate more than a
one-second delay between characters. But some networks in use today cannot
always guarantee such a luxury. Validate() returns a TIMEOUT only after two
successive one-second time-outs. Remember that we already have seen the SOH,
and should not need to wait much longer.
XModem uses a fixed frame size, which differs significantly from other common
protocols. Protocols with an ETX (end of text) signifying the end of a
variable size frame, should use another state (and action) to receive the
frame before validating. It is too slow to wait for a deadline and start
looking backwards. Table 1 shows a trace of a successful receive of three
record files while Table 2 lists the three states involved in receiving a
packet.
Table 1: A trace of successful receive

 The three states involved in receiving a packet
 _________________________________________________

 State: Init_Recv, Event: 0, Note: fname O.K
 State: Incoming, Event: 2, Note: timeout
 State: Incoming, Event: 2, Note: timeout
 State: Incoming, Event: 0, Note: got one
 State: First_What, Event: 0, Note: got SOH
 State: De_Pktize, Event: 0, Note: pkt OK
 State: Incoming, Event: 0, Note: got one
 State: First_What, Event: 0, Note: got SOH
 State: De_Pktize, Event: 0, Note: pkt OK
 State: Incoming, Event: 0, Note: got one
 State: First_What, Event: 0, Note: got SOH
 State: De_Pktize, Event: 0, Note: pkt OK
 State: Incoming, Event: 0, Note: got one
 State: First_What, Event: 2, Note: got EOT

Table 2: States in motion

 The three states involved in receiving a packet are:
 ________________________________________________________________
 1. Incoming: Waiting for the first character.
 Decodes the response from Frame_Wait(INIT).
 Normally calls Which_Ctrl() action.

 2. First_What: Decodes the response from Which_Ctrl() action based on
 first character (SOH, CAN, EOT or ???)
 Normally calls Validate() action.


 3. De-Pktize: Decodes the response from Validate() action.
 Normally calls Frame_Wait(NEXT) is all is well.



Layers and Levels


The OSI Reference model seems to have been described in every publication in
the world. The goal is to provide inter-computer networking. Computers not
only can communicate, but also share resources with other types of machines.
They can be connected directly, or across a network of conforming machines.
The XModem protocol is not a good example of a layered protocol. It is a
point-to-point file transfer protocol that requires operator intervention at
both ends; crude, but functional. In OSI terms, XModem provides mostly layer 2
(link layer) services, skips layers 3 through 6, and provides a single service
of layer 7 FTAM.
A good example of layering in use today is X.25. The link layer uses HDLC
(high-level data link control) and is defined by a set of state diagrams
between neighbors. The network layer provides routing services in complex
networks, and communicates with peer network layers in distant machines.
Individual layers should be implemented as separate state machines. If the
same program handles more than one layer, a mechanism to communicate between
layers is required. The lower layers are given precedence when a queue of
events has built up. In this case, "it" rolls uphill!


Implementing a Protocol


The first step to implementing a communications protocol is to gain a good
understanding of how it works. Many are well documented, but some require the
source code to unravel nuances like time-outs and recovery techniques. Some
proprietary protocols must be "reverse engineered" using data line monitors
and specialized tools.
The next step is to break the communications protocol down into layers, and
then to define states within layers. An outliner (with hierarchies of text)
can help to develop the states, events, and next states. A few walk throughs
can really pay off as this step is being finalized.
Validate the machine against "script" files of events. Stub out the actions
and read events from a script file. A good trace facility can record the route
taken through the machine. The trace output can be compared to what was
expected from the script file. New states or events can be added here without
great difficulty, if necessary.
Then it is time to actually code the actions. It is amazing how quickly this
can flow, given a good low-level library and proven state machines. By this
time, the responsibilities of each action should be fairly well understood.
Remember, actions are simply functions that return events!
The final step takes at least two computers. Always validate the protocol
against the best implementation available. The closer to the source, the
better the copy.


Conclusion


A friend of mine who has been on the "bleeding edge" of computer science,
implementing business solutions for years, seems to thrive on the ordeal.
After years in other areas of computer science, he made the conversion to
communications. When asked why, he simply replied, "Because comm is where the
chaos is!"
Commies are no different from other programmers. Code words line Ack, Nak, and
CRC are used to scare bright, young talent away from the field. The perfect
protocol has yet to be written; or if it has, it isn't in the public domain
... yet!


Bibliography


Donald Berryman, "Implementing Standard Communications Protocols in C," The C
Users Journal, vol. 3, no. 1.
ISO, "Reference Model for Open Systems Interconnection," ISO 7498.
Joe Campbell, C Programmers Guide to Serial Communications, Howard W. Sams &
Co., Indianapolis, Ind.: 1987.
Donald Kranz, "Christensen Protocols in C," Doctor Dobb's Journal (June 1985).
Kent Williams, "State Machines in C," Computer Language magazine (February
1986).


Availability


All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063, or call 800-356-2002 (from inside
Calif.) or 800-533-4372 (from outside Calif.). Please specify the issue number
and format (MS-DOS, Macintosh, Kaypro).

_Finite State Machines for XModem_
by Donald W. Smith



[LISTING ONE]

/* CTERM.H defines for CTERMx series. */

#define BUFSIZE 128
#define DISKREAD (BUFSIZE * 40)

/* some ASCII defines */

#define SOH 0x01 /* start of header */
#define EOT 0x04 /* end of transmission */
#define ACK 0x06 /* positive acknowledgement */
#define BS 0x08 /* backspace */
#define CR 0x0D /* Carriage Return */
#define NAK 0x15 /* Negative acknowledgement */
#define CAN 0x18 /* Cancel */
#define EoF 0x1A /* End of File (used for name) */
#define ESC 0x1B /* ASCII escape key */
#define CRC 0x43 /* ASCII 'C' (CRC mode request) */
#define BADNAME 0x75 /* Received bad name checksum */
#define TIMEOUT -1 /* for state machine logic */

static char *SPEED_LIST[] =
 { "50", "75", "110", "135", "150", "300", "600", "1200",
 "1800", "2000", "2400", "3600", "4800", "7200", "9600", "19200",
 "28800", "38400", "57600" };

static int SPEED_VALS[] =
 { 50, 75, 110, 135, 150, 300, 600, 1200,
 1800, 2000, 2400, 3600, 4800, 7200, 9600, 19200,
 28800, 38400, 57600, 0 }; /* zero for anchor */

static char *PARITY_LIST[] =
 { "NONE", "NONE", "ODD", "EVEN", }; /* matches L_CTRL p_enable + p_even */

static char *STOP_LIST[] =
 { "ONE", "TWO" }; /* matches L_CTRL p_two_stops */

static char *BITS_LIST[] =
 { "FIVE", "SIX", "SEVEN", "EIGHT" };

/* Define some common values for the lctrl bit fields */
#define ate1none 0x03;
#define sev1even 0x1A;

typedef int (*action)(); /* action is a pointer to a function */

struct event_entry {
 char comment[20]; /* for commented reading and tracing capability */
 action act; /* pointer to action function */
 int param; /* parameter to pass to function */
 enum send_state next_state; /* from an enumerated list of states */
 };

/* The following enumeration is used in all modules */
enum modes { M_Cmd, M_Term, M_Config, M_XSend, M_XRecv };

/* This struct maps the data packets for the protocol */
typedef struct pkt {
 unsigned char soh;
 unsigned char pkt;
 unsigned char pkt_cmp;
 unsigned char data[BUFSIZE];
 unsigned char crc1;
 unsigned char crc2;
} XPKT;

/* Defines used for keyfun(). - Map exactly to BIOS intr 16h 0 and 1 */

#define KEYHIT 1
#define READNEXT 0
#define BIOS_KEY 0x16 /* for int86 call for keyboard interrupt */

/* The following defines are used to map scan codes for f keys and specials */
#define HOME 0x4700
#define PGUP 0x4900
#define END 0x4F00
#define PGDN 0x5100
#define INS 0x5200
#define DEL 0x5300
#define CBRK 0x0000 /* Book says 0x5400. I see 0x0000 */








[LISTING TWO]

#define PORT 2 /* currently a define 12/16/88 */
#define NAMESIZE 24 /* Used by xmsend and recv */
#define TXTRIES 5 /* Transmit retries */
#define RXTRIES 10 /* Receive retries */

/* Parameters to pass to Send Action Make_Pkt and Recv Action Frame_Wait */
#define RESEND 0
#define INIT 1
#define NEXT 2

/* The following declaration is used to pass config info to Config_Comm(). */
typedef struct { /* Map to UART Line Control bits */
 unsigned wlen : 2; /* Word length - 5 */
 unsigned two_stops : 1; /* 1: Two stops, 0: One stop */
 unsigned parity : 2; /* 00, 01: NONE, 10 ODD, 11 EVEN */
 unsigned p_stuck : 1; /* 1: Stuck, 0: Normal */
 unsigned set_break : 1; /* 1: Send break 0: Stop break */
 unsigned div_latch : 1; /* 1: See divisors 0: See data bytes */
 unsigned : 8;
 } L_CTRL;

 typedef union {
 unsigned char lctrl;
 L_CTRL lbits;
 } U_BITS;

 typedef struct
 {
 unsigned speed; /* value from atoi() of value from speed array */
 U_BITS ubits;
 } S_INIT;








[LISTING THREE]

/* XMRECV.C: Xmodem receive state machine processing */

#include <stdio.h>
#include <ctype.h>
#include <string.h>
#include "cterm.h" /* common defines and structs for cterm */
#include "commn.h" /* brings in S_INIT struct */

enum recv_state
 { S_Init_Recv, S_Incoming, S_First_What, S_DePktize, S_Exit };

#define TRACE 1 /* to turn on state machine tracing */
/* #define SMTRACE 1 */
#ifdef SMTRACE
static char *state_list[] =
 {"Init_Recv", "Incoming", "First_What", "De-Pktize", "Exit"};
#endif

#define RECV_EVENTS 4 /* # of events per RECV state */

/* Variables local to this file only */
static char r_fname[NAMESIZE+1]; /* name of file to open */
static FILE *r_fptr = NULL; /* file pointer or number to use */
static int sohor = SOH; /* location to store first char of pkt hdr */

static int pkt = 1; /* expected packet number */
static S_INIT prev_conf; /* save prev (parity) conf */
static int virgin = 1; /* 0 = beyond initial NAK stage */

/* EXTERNAL variables */
extern int comport; /* which comm port to use (from CTERM) */
extern int crc; /* flag for CRC (!0) or checksum (0) */
extern unsigned crcaccum; /* from xmutil */
extern unsigned char checksum; /* ditto */
extern S_INIT cur_config; /* from CTERM. For timeout calc */
extern enum modes mode; /* ditto term mode or... */
extern int eschar; /* ditto escape character variable */
extern int keyfun(int); /* ditto BIOS keyboard I/O */
extern unsigned int fgetsnn(FILE *, char *, int);

/* Messages posted by A_Recv_End */
/* If declared as char *user_msg, can't be used in state table.
 * No variables allowed. But this way creates constants! */
extern char user_msg[];
extern char cancel[];
extern char badwrite[];
extern char eof_msg[];
extern char giveup[];
extern char badcomm[];

/************ Receive Actions: ********************/

/* ----- A_Prep_Recv: Prompts for file to receive, attempts open.
 * Returns: 0 if O.K., 1 if open fails, 2 is user abort. */
A_Prep_Recv( char *fname )
{

 int retval;

 fputs("\n Please Input file name to receive: ",stdout);
 fgetsnn (stdin, fname, NAMESIZE );
 if ( (fname[0] == eschar) (fname[0] == '\0') )
 return(2);
 if ( (r_fptr = fopen (fname, "wb")) == NULL ) {
 printf("\n Cannot open %s. Try again.\n", fname);
 return(1);
 }

 prev_conf = cur_config; /* save entry config */
 cur_config.ubits.lctrl = ate1none; /* Force things to 8/1/N */
 Config_Comm( comport, cur_config );

 eat_noise(); /* clear out any garbage from buffer */
 return(0);
}

/* ----- A_Frame_Wait: Send a ctrl char, wait for reply. -------
 * Returns: 0: OK, 1: comm error, 2: timeout, 3: no retries. */
A_Frame_Wait(int which)
{
 char inch;
 int errval; /* returned from reads and writes */
 int numread = 1;
 static int passes; /* give up after 10 retries */
 static char last;
 int retval = 0; /* Running value to return */

 if (virgin) { /* Waiting for first answer to NAK */
 switch (which) {
 case INIT: crc = 0; /* Go for CRC first -- fallthru will flip */
 passes = RXTRIES;
 pkt = 1; /* Initialize to first expected pkt num */
 case RESEND: crc = !crc; /* flip global flag */
 last = (crc == 0) ? NAK : CRC;
 break;
 default: retval = 3; /* Should not occur... but */
 }
 }
 else { /* Not virgin. Normal Retry logic */
 switch (which) {
 case NEXT: last = ACK;
 passes = RXTRIES;
 break;
 case RESEND: if (passes-- == 0) {
 last = CAN;
 retval = 3;
 passes = RXTRIES; /* Reset to default */
 }
 else
 last = NAK;
 break;
 default: retval = 3; /* An ounce of prevention */
 }
 }

 errval = writecomm( &last, 1);

 if (errval != 0)
 return(1); /* Get out now! */

 eat_noise(); /* clear out any garbage */

 if (retval != 3) {
 errval = read_comm( &numread, &inch, 10000 );
 if (errval == TIMEOUT)
 return (2);
 else { /* Got a live one! */
 sohor = inch; /* set global */
 if ( (virgin) && (inch == SOH) ) { /* We're rolling! */
 printf("\n\nReceiving file %s using %s.\n",
 r_fname,(crc == 0) ? "CheckSum" : "CRC" );
 fputs("\nAwaiting packet # 0001",stdout);
 virgin = 0; /* flip the local flag */
 }
 }
 }
 return(retval);
}

/* ----- A_Which_Ctrl: Parses first char received. -------------
 * Returns: 0: SOH, 1: CAN, 2: EOT, 3: unexpected (junk) */
A_Which_Ctrl(char *lead)
{
 switch (*lead) {
 case SOH: return(0);
 case CAN: return(1);
 case EOT: return(2);
 default: return(3);
 }
}

/* ----- CRC_Good: Calculates the CRC/Checksum. ----------------
 * Returns: 0 if OK, 2 if error */
CRC_Good(char *buf, int crcflag, unsigned char crchi, unsigned char crclo)
{
 register int i;

 crcaccum = 0; /* zero out global crc and checksum value */
 checksum = 0;

 for (i = 0; i < BUFSIZE; i++, buf++)
 updcrc(*buf);
 updcrc(0); /* Xmodem deviant CRC calc */
 updcrc(0);

 if (crcflag == 0) {
 if (crchi != checksum)
 return(2);
 }
 else {
 if ( (crclo + ( crchi << 8)) != crcaccum )
 return(2);
 }
 return(0);
}


/* ----- Action Validate: After SOH, validates the xmodem header.
 * Returns: 0: OK, 1: bad header, 2: bad CRC, 3: char timeout. */
A_Validate(int *crcflag )
{
 int retval;
 int readnum = (*crcflag == 0) ? 131 : 132; /* pass to read_comm */
 int togo = readnum; /* if partial, running count */
 int msecs; /* how long to wait */
 XPKT r_pkt; /* packet receive buffer */
 unsigned char *diskbuf = (unsigned char *) &r_pkt.data;
 unsigned char *curptr = (char *) &r_pkt.pkt; /* Rem: got SOH already */
 long frame_bits = ( (BUFSIZE + 3) * 10 *1000L );

 printf("\b\b\b\b%4d",pkt); /* Allow up to 9999 frames */

 while (readnum != 0) {
 msecs = (int)( frame_bits / (long)cur_config.speed );
 delay(msecs); /* Let the interrupt handler work */
 retval = read_comm( &readnum, curptr, msecs );
 curptr = curptr + readnum; /* adjust curptr to next avail loc */
 readnum = (togo -= readnum); /* adjust BOTH to remainder of pkt */

 if (retval == TIMEOUT) { /* Give it one more second if short */
 togo = 1; /* prep togo for 1 char read test */
 retval = read_comm( &togo, curptr, 1000);
 if (retval == TIMEOUT) /* Bad news. Dead line */
 return(3);
 curptr++; /* recovered! adjust and try again */
 togo = --readnum;
 }
 frame_bits = togo * 10; /* Adjust by bits per character */
 }

 if (~r_pkt.pkt != r_pkt.pkt_cmp) {
 return(1);
 }
 if ( r_pkt.pkt != (pkt % 256) )
 if ( r_pkt.pkt == ( (pkt - 1) & 0xFF ) ) {
 return(0); /* duplicate packet! Ack and ignore */
 }
 else
 return(1); /* Nak and retry.. probably useless but... */

 retval = CRC_Good(diskbuf, *crcflag, r_pkt.crc1, r_pkt.crc2);
 if (retval != 0) {
 return(2);
 }

 fwrite(diskbuf, BUFSIZE, 1, r_fptr);
 pkt++;
 return (0);
}

/* ----- Action EatRest: Eats the rest of a packet. ---------- */
A_EatRest(int calories)
{
 int toeat = calories;
 int retval = 0;
 long frame_bits;

 char junkbuf[BUFSIZE + 4];

 if (calories > BUFSIZE)
 calories = BUFSIZE + 4;
 frame_bits = ( calories * 10 * 1000L);
 delay( (unsigned)(frame_bits/(long)cur_config.speed) + 500 );

 while (retval != TIMEOUT) {
 retval = read_comm( &toeat, junkbuf, 1000);
 toeat = 1;
 }
 retval = A_Frame_Wait(RESEND);
 return(retval);
}

/* ----- Action Recv_End: Only way out of Recv state machine. */
A_Recv_End ( char *reason )
{
 char eotc = ACK; /* just in case we really Receive the file */

 if (r_fptr != NULL) { /* Did we even get started??? */
 if (reason != eof_msg) { /* Started, but bad news during xfer */
 eotc = CAN;
 unlink(r_fname); /* deletes the old file */
 }
 fclose(r_fptr);
 writecomm(&eotc, 1);
 Config_Comm( comport, prev_conf ); /* Put whatever parity back in */
 }

 printf("\n *** Ending session. %s.\a\n",reason);

 virgin = 1;
 mode = M_Cmd;
 return (RECV_EVENTS - 1); /* last event always has next state S_Exit */
}

/************ R E C E I V E S T A T E T A B L E ****************/
 struct event_entry recv_machine[(int)S_Exit][RECV_EVENTS] =
 { /* S_Init_Recv */
 { { "fname O.K" , A_Frame_Wait , INIT , S_Incoming },
 { "fname bad" , A_Prep_Recv , (int)r_fname , S_Init_Recv },
 { "user abort" , A_Recv_End , (int)user_msg, S_Exit },
 { "comm error" , A_Recv_End , (int)badcomm , S_Exit } },
 /* S_Incoming */
 { { "got one" , A_Which_Ctrl , (int)&sohor , S_First_What },
 { "comm error" , A_Recv_End , (int)badcomm , S_Exit },
 { "timeout" , A_Frame_Wait , RESEND , S_Incoming },
 { "no retries" , A_Recv_End , (int)giveup , S_Exit } },
 /* S_First_What */
 { { "got SOH" , A_Validate , (int)&crc , S_DePktize },
 { "got CAN" , A_Recv_End , (int)cancel , S_Exit },
 { "got EOT" , A_Recv_End , (int)eof_msg , S_Exit },
 { "got junk!" , A_EatRest , BUFSIZE , S_Incoming } },
 /* S_DePktize */
 { { "pkt OK" , A_Frame_Wait , NEXT , S_Incoming },
 { "bad hdr" , A_EatRest , BUFSIZE , S_Incoming },
 { "bad CRC" , A_Frame_Wait , RESEND , S_Incoming },
 { "timeout" , A_Frame_Wait , RESEND , S_Incoming } }

 };


/* -------------- Xmodem Receive state machine --------------- */
xmodem_recv()
{
 char inkey; /* place for user to abort */
 int event; /* event returned from action */
 int prevent; /* previous event */
 struct event_entry *cur_entry; /* pointer to current row/col of sm */
 action new_action; /* next action to perform */
 enum send_state cur_state = S_Init_Recv;

 event = A_Prep_Recv(r_fname);

 while (mode == M_XRecv)
 {
 prevent = event; /* save the previous event for next state */
 cur_entry = &recv_machine[(int)cur_state][event];

#ifdef SMTRACE
 printf("State: %16s, Event: %2d, Note: %20s\n",
 state_list[(int)cur_state], event, cur_entry->comment );
#endif

 /* Based on the current state and event, execute action(param) */
 new_action = cur_entry->act;
 event = new_action(cur_entry->param);
 cur_state = recv_machine[(int)cur_state][prevent].next_state;

 if ( keyfun(KEYHIT) ) {
 inkey = (char) keyfun(READNEXT); /* Truncate to key only */
 if (inkey == eschar)
 A_Recv_End(user_msg);
 }
 }
 return (0);
}







[LISTING FOUR]

/* XMSEND.C Xmodem Send state machine processing. */

#include <conio.h> /* for putch call */
#include <ctype.h>
#include <io.h> /* for filelength call */
#include <stdio.h>
#include <string.h>
#include "cterm.h"
#include "commn.h" /* brings in S_INIT struct and defines */

enum send_state
 { S_Init_Send, S_Sync_Wait, S_Make_Pkt, S_Send_Pkt, S_Data_Response, S_Exit
};


#ifdef TRACE
char *state_list[] =
 {"Init_Send", "Sync_Wait", "Make_Pkt", "Send_Pkt", "Data_Response", "Exit"};
#endif

#define SEND_EVENTS 4 /* number of events handled per send state */

/* Variables local to this file only */
static char s_fname[NAMESIZE+1]; /* name of file to open */
static FILE *s_fptr = NULL; /* file pointer or number to use */
static XPKT s_pkt; /* packet to send */
static S_INIT prev_conf; /* saves previous bits, parity during xfer */

/* EXTERNAL variables and functions */
extern int comport; /* which comm port to use (from CTERM) */
extern unsigned crcaccum; /* from xmutil */
extern unsigned char checksum; /* ditto */
extern int crc; /* ditto */
extern S_INIT cur_config; /* from CTERMx. For send time calc */
extern int eschar; /* ditto escape character variable */
extern enum modes mode; /* ditto term mode or... */
extern int keyfun(int); /* ditto BIOS call to keyboard */

/* If declared as char *user_msg, can't be used in state table.
 * No variables allowed. But this way creates constants! */
extern char user_msg[];
extern char nonak[];
extern char cancel[];
extern char badread[];
extern char eof_msg[];
extern char giveup[];

/************ Send Actions: ********************/

/* ----- A_Get_Fname: Prompts for file to transmit, opens. -----
 * Returns: 0: OK, 1: open failed, 2: user abort. ------------ */
A_Get_Fname( char *fname )
{
 long fbytes;
 int frecs;
 int fsecs;

 printf("\n Please Input file name to transmit: ");
 fgetsnn (stdin, fname, NAMESIZE );
 if ( (fname[0] == eschar) (fname[0] == '\0') )
 return(2);
 if ( (s_fptr = fopen (fname, "rb")) == NULL ) {
 printf("\n Cannot open %s. Try again.\n", fname);
 return(1);
 }
 fbytes = filelength( fileno(s_fptr) );
 frecs = ( (fbytes / BUFSIZE) + ( (fbytes % BUFSIZE == 0) ? 0 : 1 ) );
 /* The following adds time for turn around (ACK/NAK), but no errors */
 fsecs = (int) ( (fbytes * 10) / (cur_config.speed / 2 ) );

 printf("\n File %s: %4d records, est. min:sec %3d:%2d at %d bps.\n",
 fname, frecs, fsecs / 60, fsecs % 60, cur_config.speed );


 prev_conf = cur_config; /* save entry config */
 cur_config.ubits.lctrl = ate1none; /* Force things to 8/1/N */
 Config_Comm( comport, cur_config );

 eat_noise(); /* Clear out any garbage in the input queue */
 return(0);
}

/* ----- A_Init_Wait: Waits for initial sync character. ---------
 * Returns: The value returned from A_Wait(). */
A_Init_Wait(int expected)
{
 static int tries = 2; /* try initial CRC, then once more */
 static int passes = 10; /* give up after 10 junk reads */
 static int last;
 int retval;

 switch(expected) {
 case CRC: last = CRC; /* If we really want CRC... */
 break;
 case NAK: last = NAK; /* or if we only want Checksum */
 break;
 case NEXT: if (--tries == 0) { /* want to switch? */
 last = (last == CRC) ? NAK : CRC;
 tries = 2;
 }
 }
 printf("\rAwaiting %s...",(last == CRC) ? "CRC" : "NAK");
 retval = A_Wait(last);
 if (retval != 0) {
 if (passes-- == 0)
 return(3); /* cancelled */
 else
 return(retval);
 }

 passes = 10; /* reset passes counter */
 crc = (last == CRC) ? 1 : 0;
 return(retval);
}

/* ------ A_Wait: Waits for appropriate time for a character.
 * Returns: 0: match, 1: if other, 2: timeout, 3: cancel. */
A_Wait( int expected)
{
 char inch;
 int errval;
 int numread = 1;
 int retval = 0;

 errval = read_comm( &numread, &inch, (expected == SOH) ? 1000 : 10000 );
 if ( numread > 0 ) {
 if (inch == (char) expected)
 retval = 0;
 else retval = (inch == CAN) ? 3 : 1 ;
 }
 else
 if (errval == TIMEOUT)
 retval = 2;


 return (retval);
}

/* ----- Action Make_Pkt: Reads from disk, formats packet. -----
 * Returns: 0: OK, 1: disk trouble, 2: EOF found. */
A_Make_Pkt(int which )
{
 register int i;
 int errval;
 unsigned int lo_crc;
 static int pkt;
 static unsigned char *diskbuf = (unsigned char *) &s_pkt.data;
 static unsigned char *curptr; /* where are we now? */

 crcaccum = 0; /* zero out global crc and checksum value */
 checksum = 0;

 for (curptr = diskbuf, i = 0; i < BUFSIZE; i++, curptr++) {
 if ( (errval = getc(s_fptr)) == EOF )
 break;
 *curptr = errval;
 updcrc(errval);
 }
 if (i == 0)
 return(2); /* That's all folks! */

 for ( ; i < BUFSIZE; i++, curptr++) { /* Zero fill the rest of packet */
 *curptr = 0;
 updcrc(0);
 }

 if (which == INIT) {
 printf("\n\nSending file %s using %s.\n",
 s_fname,(crc == 0) ? "CheckSum" : "CRC");
 pkt = 1;
 }
 else pkt = (++pkt % 256);

 s_pkt.soh = SOH;
 s_pkt.pkt = pkt;
 s_pkt.pkt_cmp = ~pkt;
 updcrc(0); /* finish off xmodem variation */
 updcrc(0);
 lo_crc = crcaccum;
 if (crc != 0) {
 s_pkt.crc1 = (crcaccum >> 8); /* high byte first */
 s_pkt.crc2 = lo_crc;
 }
 else
 s_pkt.crc1 = checksum;

 return (0);
}

/* ----- Action Send_Pkt: Send a packet out the comm port. ------
 * Returns: 0: OK, 1: write err, 2: no retries, 3: cancelled */
A_Send_Pkt( int why )
{

 static int retries = TXTRIES; /* If not general, make a global table */
 int errval;

 switch (why) {
 case NEXT: retries = TXTRIES;
 putch('.'); /* show we are making progress */
 break;
 case NAK:
 case TIMEOUT:
 case RESEND: --retries;
 putch('R');
 }

 if (!retries) {
 retries = TXTRIES;
 return (2);
 }

 errval = writecomm( (char *) &s_pkt, (crc != 0) ? 133 : 132 );
 if (errval)
 return(1);

 eat_noise(); /* clear out any garbage */
 return (0);
}

/* ----- Action Send_End: Only way out of the Send state. --- */
A_Send_End ( char *reason )
{
 char eotc = EOT; /* just in case we really transmit the file */
 int notdone = 1; /* have we received an ACK to our EOT? */
 int eotries = 10; /* Should be enough for most */

 if (s_fptr != NULL) { /* Did we even get started??? */
 if (reason != eof_msg) { /* Started, but bad news */
 eotc = CAN;
 writecomm(&eotc, 1);
 }
 else /* eof = We did it! Send our EOT and get out */
 while ( (notdone != 0) && (eotries--) ) {
 writecomm(&eotc, 1);
 notdone = A_Wait(ACK);
 }
 fclose(s_fptr);
 Config_Comm( comport, prev_conf ); /* Put whatever parity back in */
 }

 printf("\n *** Ending session. %s.\a\n",reason);

 mode = M_Cmd;
 return (SEND_EVENTS - 1); /* last event always has next state S_Exit */
}

/*************** S E N D S T A T E T A B L E *******************/
 struct event_entry send_machine[(int)S_Exit][SEND_EVENTS] =
 { /* S_Init_Send */
 { { "fname O.K" , A_Init_Wait , CRC , S_Sync_Wait },
 { "fname bad" , A_Get_Fname , (int)s_fname , S_Init_Send },
 { "user abort" , A_Send_End , (int)user_msg, S_Exit },

 { "no retries" , A_Send_End , (int)nonak , S_Exit } },
 /* S_Sync_Wait */
 { { "in sync" , A_Make_Pkt , INIT , S_Make_Pkt },
 { "unexpected" , A_Init_Wait , NEXT , S_Sync_Wait },
 { "timeout" , A_Init_Wait , CRC , S_Sync_Wait },
 { "cancelled" , A_Send_End , (int)cancel , S_Exit } },
 /* S_Make_Pkt */
 { { "pkt ready" , A_Send_Pkt , NEXT , S_Send_Pkt },
 { "bad disk?" , A_Send_End , (int)badread , S_Exit },
 { "done!" , A_Send_End , (int)eof_msg , S_Exit },
 { "trouble!" , A_Send_End , (int)giveup , S_Exit } },
 /* S_Send_Pkt */
 { { "sent O.K." , A_Wait , ACK , S_Data_Response },
 { "comm error" , A_Send_Pkt , RESEND , S_Send_Pkt },
 { "no retries" , A_Send_End , (int)giveup , S_Exit },
 { "cancelled" , A_Send_End , (int)cancel , S_Exit } },
 /* S_Data_Response */
 { { "ack rcvd." , A_Make_Pkt , NEXT , S_Make_Pkt },
 { "not ack" , A_Send_Pkt , NAK , S_Send_Pkt },
 { "timeout" , A_Send_Pkt , TIMEOUT , S_Send_Pkt },
 { "cancelled" , A_Send_End , (int)cancel , S_Exit } }
 };


/* -------------------- Send state machine ------------------ */
xmodem_send()
{
 char inkey; /* In case the user wants to abort */
 int event; /* event returned from action */
 int prevent; /* previous event */
 struct event_entry *cur_entry; /* pointer to current row/col of sm */
 action new_action; /* next action to perform */
 enum send_state cur_state = S_Init_Send;

 event = A_Get_Fname(s_fname);

 while (mode == M_XSend) {
 prevent = event; /* save the previous event for next state */
 cur_entry = &send_machine[(int)cur_state][event];

#ifdef TRACE
 printf("State: %16s, Event: %2d, Note: %20s\n",
 state_list[(int)cur_state], event, cur_entry->comment );
#endif

 /* Based on the current state and event, execute action(param) */
 new_action = cur_entry->act;
 event = new_action(cur_entry->param);
 cur_state = send_machine[(int)cur_state][prevent].next_state;

 if ( keyfun(KEYHIT) ) { /* from CTERM */
 inkey = (char) keyfun(READNEXT); /* Truncate high order */
 if (inkey == eschar)
 A_Send_End(user_msg);
 }
 }
 return (0);
}









[LISTING FIVE]

/* CTERM1.C by Donald W. Smith. CIS 76515,3406.
 * A minimal terminal emulator to demonstrate the use of state
 * machine driven communications protocols using the C language.
 * Use makect1. to compile. */

#define VERSION fputs("\n\t CTERM 1.11: 4/26/89 DWS\n\n",stdout)
#define BUFLEN 200
#define LINELEN 80 /* Max user input length. Lots of slack */

#include <conio.h>
#include <ctype.h> /* for Turbo C is... functions */
#include <dos.h>
#include <process.h> /* For system() call */
#include <signal.h> /* Ctrl-C and Ctrl-Break handling */
#include <stdio.h>
#include <stdlib.h> /* For system() call */
#include "cterm.h" /* defines for cterm series */
#include "commn.h" /* defines shared by myint and cterm */
#include "getargs.h" /* for getargs access */

#define CMDDIM(x) (sizeof(x)/sizeof(x[0]))

/* --------- GLOBALS ------------------ */
enum modes mode = M_Cmd; /* Term, Config, etc. */
char inbuf[BUFLEN+1];
char outbuf[LINELEN+1]; /* Used for short output */
S_INIT cur_config = {1200}; /* Current port config */
int comport = 1; /* Current comm port number */
int bbsmode = 0; /* BBS (8,1,N) or T16 (7,1,E) */
int eschar = ESC; /* keyboard escape character */
FILE *cap_fptr; /* file ptr for capture file */
static union REGS rg; /* used for keyfun() */

/* ---------- External variables -------------- */
extern unsigned _heaplen = 4096; /* TurboC 1.5 and above */

/* ---------- External routines ----------- */
/* -- From myint -- */
extern void Config_Comm( int port, S_INIT conf );
extern S_INIT Get_Conf( int port );
extern int incomm();
extern int Inst_IH(void interrupt (*faddr)(), int comnum);
extern int Remove_IH();
extern int writecomm(unsigned char *buf, int len);
extern int xmit_break();
/* -- from xmutil -- */
extern int read_comm(int *num, char *buf, int wait);
/* -- from object only files -- */
extern int getargs( int, char**, ARG *, int, int (*usage)() );


ARG Argtab[] = {
 { 'b', BOOLEAN, &bbsmode, "BBS mode (8,1,N) vs. T16)" },
 { 'c', INTVAR, &comport, "1 = COM1, 2 = COM2" },
 { 'e', INTVAR, &eschar, "Escape char (0x..)" },
 { 's', INTVAR, &cur_config.speed, "speed in bps" } };

/* ----- fgetsnn: Gets a line from file, replacing \n with NULL.
 * Return # chars, or EOF */
int fgetsnn(FILE *fp, char *s, int size)
{
 register int i;
 int c;

 for (i = 0; (i < size-1) &&
 (c = fgetc(fp)) != EOF &&
 (c != '\n') ; ++i)
 s[i] = c;
 s[i] = '\0';
 if (c == EOF)
 return(EOF);

 return(i);
}

/* ----- capture_sw: Enables saving sessions to (PC) disc. -----
 * Returns: 0 O.K. 1 if open fails, 2 if ESC hit. */
int capture_sw()
{
 static char cap_fname[NAMESIZE + 1] = "capture.fil";
 char cap_temp[NAMESIZE + 1];
 static int cap_sw = 0; /* capture on/off switch */

 if (cap_sw == 0) { /* Open the file for capture */
 fprintf(stdout,"\n Capture to file <%s> or : ",cap_fname);
 fgetsnn (stdin, cap_temp, NAMESIZE );
 if (cap_temp[0] == eschar)
 return(2);
 if (cap_temp[0] != '\0')
 strncpy(cap_fname, cap_temp, NAMESIZE);
 if ( (cap_fptr = fopen (cap_fname, "a+t")) == NULL ) {
 printf("\n Cannot open %s. Try again.\n", cap_fname);
 return(1);
 }
 cap_sw = 1;
 }
 else { /* we are already capturing. Close and get out */
 fclose( cap_fptr );
 cap_sw = 0;
 }
 return(0);
}

/* ----- keyfun: Use to call BIOS keyboard input services ------
 * Use instead of keypressed and bioskey to prevent DOS ^C's. */
int keyfun(int serv)
{
 rg.h.ah = serv;
 int86(BIOS_KEY, &rg, &rg);
 if (serv == KEYHIT)

 return ((rg.x.flags & 0x40) == 0);
 else
 return (rg.x.ax);
}

/* ----- term: Emulates a dumb terminal. -------------------- */
void term()
{
 register int i;
 int keyin; /* Key = scan code + ASCII val */
 char gochar;
 int redd;
 int ret_code,
 wait_ret;
 char *tail = inbuf; /* for tail of input buffer */
 static int cap_flag = 0; /* Is capture turned on now? */

 while (mode == M_Term) {
 redd = BUFLEN / 2; /* Go for half at a time. */
 ret_code = read_comm (&redd, inbuf, 10); /* 10 msecs */
 if ( (ret_code != 0) && (ret_code != TIMEOUT) )
 fprintf (stderr, "read_comm error %x\n", ret_code );

 if ( redd > 0 ) { /* Reading was productive */
 tail[redd] = 0; /* plant a null */

 for ( i = 0; i < redd; i++) { /* check for specials */
 if ( isprint( gochar = inbuf[i] & 0x7F) /* zero hi */
 ( isspace(gochar) ) /* CR,LF.. */
 gochar == BS ) {
 putch(gochar);
 if (cap_flag)
 fputc(gochar, cap_fptr);
 } /* printable test */
 } /* for loop */
 } /* end if reading was productive */

 if ( keyfun(KEYHIT) ) {
 keyin = keyfun(READNEXT); /* Retrieve Scan Code + Key */
 gochar = keyin & 0xFF; /* truncates */
 if (gochar == 0) { /* Function key or a special */
 switch (keyin) {
 case PGUP: mode = M_XSend;
 xmodem_send();
 mode = M_Term;
 break;
 case PGDN: mode = M_XRecv;
 xmodem_recv();
 mode = M_Term;
 break;
 case CBRK: xmit_break();
 break;
 case INS: if (capture_sw() == 0)
 cap_flag = !cap_flag;
 default: break;
 }
 }
 else { /* Some plain old ascii character came in */
 if ( gochar != eschar ) {

 outbuf[0] = gochar;
 writecomm(outbuf, 1);
 }
 else /* ESCAPE entered */
 mode = M_Cmd; /* leave terminal mode */
 }
 } /* end if keypressed */
 } /* while mode = M_Term */
 return;
}

/* ----- off_to_dos: Prompts for command to pass to dos. ----- */
void off_to_dos()
{
 char buf[LINELEN];

 fputs("\nInput DOS Command (CR returns to menu)\nDOS>",stdout);

 while ( fgetsnn(stdin, buf, LINELEN) ) { /* > 1 means got 1 */
 system(buf);
 fputs("\nDOS>",stdout);
 }
}

/* ----- config: Prompts for new config (speed or default).-- */
void config()
{
 S_INIT work;
 char *cptr;
 char buf[LINELEN];
 unsigned inval;
 int inlen;
 int i = 0;

 work = Get_Conf(comport); /* a struct assignment */

 fputs("\n Current config shows:\n", stdout);

 printf("%5u, parity %s, %s stops, %d bits/char.\n",
 work.speed,
 PARITY_LIST[work.ubits.lbits.parity],
 STOP_LIST[work.ubits.lbits.two_stops],
 work.ubits.lbits.wlen + 5);

 fputs("0 = T16 ( 7, 1, Even )\n", stdout);
 fputs("1 = BBS ( 8, 1, None )\n", stdout);
 fputs("other = new speed value\n", stdout);

 inlen = fgetsnn(stdin, buf, LINELEN); /* Got one parameter */
 if (inlen > 0) {
 inval = atoi(buf);
 if (inval == 0) {
 work.ubits.lctrl = sev1even; /* 7,1,EVEN */
 bbsmode = 0;
 }
 if (inval == 1) {
 work.ubits.lctrl = ate1none; /* 8,1,NONE */
 bbsmode = 1;
 }

 if (inval > 1) {
 while ( SPEED_VALS[i] && SPEED_VALS[i] != inval )
 i++;
 if (SPEED_VALS[i] == 0)
 printf("\n Speed %d unavailable.\n",inval);
 else { /* found a valid new speed */
 work.speed = inval;
 }
 }
 Config_Comm(comport, work);
 cur_config = work; /* Publish the results */
 }
 else
 fputs("\n Exiting Config mode.\n",stdout);

 if ( inlen == (LINELEN - 1) )
 while (fgetc(stdin) != EOF) /* Purge garbage in buffer */
 ;

 mode = M_Cmd;
}


/* ----- prompt_wait: Prompts with string, parses command. ------
 * Returns: The (int) index number of the command entered. */
prompt_wait ( char *prompt, char *cmds[], int n, int help )
{
 char buffer[LINELEN]; /* used by fgetsnn */
 int i = 0;

 while (i == 0) { /* Don't bite on CR only input */
 printf("\n%s",prompt);
 i = fgetsnn(stdin, buffer, LINELEN );
 }

 if ( i == (LINELEN - 1) )
 while (fgetc(stdin) != EOF) /* Purge garbage in buffer */
 ;

 for ( i = 0 ; i < n ; i++ )
 if ( *cmds[i] == toupper(buffer[0]) ) /* Match first char */
 return(i);
 return (help); /* not found... return help default */
}

/* ----- main_help: Shows canned help message --------------- */
void main_help()
{
static char *details[] =
 { "Config : Set up comm parameters.\n",
 "Dos, : Calls DOS with commands.\n",
 "Help, : See this help info.\n",
 "Quit : Exit program.\n",
 "Rcvx, : Receive file using Xmodem.\n",
 "Sendx, : Send file using Xmodem.\n",
 "Term, : Dumb terminal mode.\n" };

 register int i;


 fputs("\n Valid commands are:\n",stdout);
 for (i = 0 ; i < CMDDIM(details) ; i++)
 printf("%s",details[i]);
}

/* ----- main_menu: Prompts for input, dispatches if valid -- */
void main_menu()
{
 static char *prompt = "CTERM>";
 static char *maincmds[] =
 { "CONFIG",
 "DOS",
 "HELP", /* used for default below (index 2 ) */
 "QUIT",
 "RCVX",
 "SENDX",
 "TERM" };

 while (mode == M_Cmd) {
 switch ( prompt_wait(prompt, maincmds, CMDDIM(maincmds), 2 ))
 {
 case 0: mode = M_Config;
 config();
 break;
 case 1: off_to_dos();
 break;
 case 2: main_help();
 break;
 case 3: printf("\n *** Closing Comm port %d.", comport);
 close_comm();
 exit(1);
 case 4: mode = M_XRecv;
 xmodem_recv();
 fputs("\nReturned from xmodem recv!\n",stdout);
 break;
 case 5: mode = M_XSend;
 xmodem_send();
 fputs("\nReturned from xmodem send!\n",stdout);
 break;
 case 6: mode = M_Term;
 eat_noise();
 term();
 break;
 default: main_help();
 } /* end switch */
 } /* end while */
}

/* ----- Catch_23: Traps ^C + ^Break during user I/O. ------- */
void Catch_23()
{
 signal(SIGINT, Catch_23); /* Re-install self each time */
 return;
}

/* ----- Catch_24: Traps Disk (Abort, Retry, Fail?) errors -- */
int Catch_24(int errval, int ax, int bp, int si)
{
 char msg[25];

 int drive;

 if (ax < 0) { /* device error */
 bdosptr( 0x09, "device error$", 0);
 hardretn(-1);
 }

 drive = (ax & 0x00FF);
 sprintf(msg, "disc error on drive %c $", 'A' + drive);
 bdosptr( 0x09, msg, 0);
 hardretn(2);
}

/* ----- usage: Give user quick help before exit. ----------- */
usage()
{
 printf("\n Defaults: T16, 1200 bps, 8,1,NONE, COM1, ESC.\n");
}

/* ----- main: Gets the ball rolling! ---------------------- */
main( int argc, char *argv[] )
{
 int error;

 VERSION;
 signal(SIGINT, Catch_23);
 harderr(Catch_24);

 argc = getargs( argc, argv, Argtab, CMDDIM(Argtab), usage );

 error = init_comm(comport, bbsmode);
 if (error != 0) {
 fprintf(stderr,"\n *** Comm Port %d Init FAILED!",comport);
 return(2);
 }

 fprintf(stderr,"\n *** Comm Port %d Init O.K. *** ", comport);
 main_menu();
 return(1);
}







[LISTING SIX]

#Make file for cterm series Turbo C. 1/23/89 DWS
#3/20/89: Added getargs and stoi (.obj only) from Holub
#use Make -fmakect1
#Small memory model
MDL = s
LIB = c:\turboc\lib

#implicit rules
# To add debug info: TCC (-v), TLINK (/v)
.c.obj:

 tcc -c -m$(MDL) $<

#explicit rules
cterm1.exe: commint.obj xmsend.obj xmrecv.obj xmutil.obj cterm1.obj
 tlink ..\lib\c0s cterm1 xmsend xmrecv xmutil commint getargs stoi, \
 cterm1, , ..\lib\cs

xmsend.obj: xmsend.c cterm.h commn.h

xmrecv.obj: xmrecv.c cterm.h commn.h

commint.obj: commint.c commn.h commint.h

xmutil.obj: xmutil.c cterm.h commn.h commint.h

cterm1.obj: cterm1.c cterm.h commn.h

#end makefile












































October, 1989
HAMMING-CODE DECODING


You don't have to sacrifice efficiency to achieve reliability




Ben White


Ben is a C programmer and employee-owner at Lockheed Missiles & Space Co.,
0/6215, B/551, PO Box 3504, Sunnyvale, CA 94088-3504.


Since the earliest days of electronic computation and data communication,
equipment designers have had to grapple with data transmission errors. For
example, data being transmitted over a noisy channel, such as a telephone
line, may have some bits garbled by external phenomena to the extent that they
no longer have their original value. In addition, data stored in certain types
of computer memory occasionally lose the value in a single bit. In either
case, if the data are not somehow corrected or retransmitted, they are
worthless. If an error is undetected, the results can be catastrophic --
imagine that the data represent a withdrawal transaction from a bank's
automatic teller machine.
Because of the need for reliable, accurate data transmission and storage,
various methods have been developed over the years to deal with this problem.
By introducing a certain amount of redundancy into the transmitted data, an
error can be detected, and even corrected, at the receiving end. A very
simple-minded usage of redundancy is shown in the following example. Suppose
that we wish to transmit data in 8-bit bytes. By transmitting each byte twice,
the receiver can compare corresponding bits and flag any discrepancy as an
error. For example, compare the two transmitted bytes:
 11010011
 11011011
These two bytes differ at the fifth bit from the left, indicating an error.
There is no way to know which value (0 or 1) is the correct value for that
bit, so the receiver must request retransmission. By using triple redundancy,
this problem can be eliminated, as shown:
 11010011
 11010011
 11011011
Under this scheme, each byte is transmitted three times, and we again note
that there is a discrepancy in the fifth bit. The bit appears as a 0 in two of
the transmitted bytes, and as a 1 in only one byte. We can assume with a high
degree of confidence that the correct value for that bit is 0, and
retransmission is not necessary. This self-correcting capability is enormously
useful in situations such as satellite communications, where signal
propagation takes a long time.
The problem with the above approach is that it is inefficient. With triple
redundancy, only 33 percent of the channel capacity is used for actual data.
The use of Hamming codes is a widely used method of achieving the same, or
better, results. A subset of the bits in each data word is assigned to
(overlapping) groups, and a "check bit" is given to each group. This allows
the detection/correction of errors in received code words with much more
efficient use of the channel than the method previously described.


Hamming-Code Theory


To illustrate this procedure, let's assume that we wish to transmit 4-bit
message (data) words across a noisy channel. We also want to be able to detect
and correct any single bit error occurring during transmission. It will be
necessary to create three check groups, and assign three of the message bits
to each group. Number the message and check bits as follows:
 P1 P2 M3 P4 M5 M6 M7
P1, P2, and P4 are check bits (P stands for parity). M3, M5, M6, and M7 are
the message bits. Before transmitting the 7-bit code word, values must be
assigned to the check bits. This is done as follows: P1 is assigned odd parity
over M3, M5, and M7. P2 is assigned odd parity over M3, M6, and M7. Finally,
P4 is assigned odd parity over M5, M6, and M7. For example, take the code
word:
 M3 M5 M6 M7 1 0 1 1
Because the exclusive-OR of M3, M5, and M7 is 0, P1 will be 1. The 4-bit
group, including the check bit itself, must have odd parity. By the same
reasoning, P2 is 0, and P4 is 1. The complete code word to be transmitted is:
 M3 M5 M6 M7
 1 0 1 1
You may notice that the P1 check bit represents the parity over all message
bit positions with the 2{1} bit present, as in positions 3, 5, and 7 (011,
101, 111 in binary). P2 is parity for the 2{2} bit positions, and P4 is parity
for the 2{4} bit positions. The placement of check bits in the completed code
word is done to reinforce this structure conceptually. In actual practice, any
ordering of the bits will do, as long as sender and receiver agree on the same
order.
To see that the added check bits provide enough redundancy to detect and
correct any single bit error, look at the receiving and decoding process. When
the receiver has all 7 bits of the code word, the grouping of bits is
repeated, and parity of each 4-bit group is checked. P1, M3, M5, and M7 are
exclusive-ORed together and checked for odd parity; the same process occurs
for the P2 and P4 groups.
If one or more of the three groups do not have odd parity, a transmission
error has occurred. Assume that the received code word is 1010011, which
differs from the transmitted code word. The bits of the P1 group are 1101, the
P2 group is 0111, and the P4 group is 0011. The exclusive-OR of the 4 bits in
each group is 0, 0, and 1, respectively. As one of the groups, P4, does not
have odd parity, an error has been detected.
We have all of the information needed to locate the bit position of the error
and we can correct it. The three parity bits computed by the receiver are
arranged in descending order, left to right, by group number, P4, then P2,
then P1. Arranged this way, the bits are called the "syndrome." In the above
example, the syndrome is 100, which is the binary number four (4). It just so
happens that bit 4 (counting from the left) in the received code word is the
bit in error. Simply invert the bit to correct it. Bit 4 also happens to be
one of the check bits, and nicely illustrates that the Hamming method of error
detection works for any bit in the code word, whether it is a message or a
check bit.
The Hamming method can be adapted to work for longer message words. Efficiency
is gained in terms of the check bits to message bits ratio, but the
probability of an error occurring in a given code word is increased. On the
other hand, if more check bits are used, greater numbers of bit errors in a
word can be detected and corrected.


Encoding and Decoding


Methods used in logic designs to add check bits to message words include
parity-generation circuits (Figure 1) and address-table lookups (as through a
ROM). In the first case, exclusive-OR gates are used to generate odd parity
from 2-bit pairs. The outputs are exclusive-ORed again to compute parity for a
4-bit group. The parity (check) bits are then transmitted with the message
bits.
For lookup-based check-bit generation, the message bits are presented as the
address to a memory. The data at each memory location is the appropriate
combination of check bits for that message word. Again, assume a 4-bit
message, and three check bits for single-error correction. The resulting
memory lookup table would require four address bits and 3-bit data words, or a
16-word x 3-bit memory. As with the exclusive-OR network, the check bits
produced from the lookup table are transmitted with the message bits.
The traditional method for decoding received data is with an exclusive-OR
circuit, as shown in Figure 2, for the 4-bit message and 3-bit check scheme.
The 7 bits of the code word are presented in parallel to the exclusive-OR
gates. These are divided into three groups, corresponding to the three check
bits. The odd parity of each of the 4 bits in the group is calculated, and the
resulting syndrome is presented to a decoder. The decoder can be considered to
be the error detector. If an error is indicated by the syndrome, one of the
outputs Y1 to Y7 will be asserted. If the syndrome is 0, Y0 will be asserted,
meaning no error was detected.
The decoder outputs are used to drive the correction logic, consisting of one
exclusive-OR gate for each message bit. The check bits do not need to be
explicitly corrected, as they are used only internally by the receiver. This
is the reason nothing is connected to the Y1, Y2, and Y4 outputs of the
decoder.
Normally, the exclusive-OR gates will pass the message bit through unchanged.
In some instances, the decoder asserts one of the outputs Y3, Y5, Y6, or Y7.
This will cause the corresponding gate to invert its received message bit and
correct the error automatically.
An alternate approach to decoding, which is similar to encoding by ROM table
lookup, boasts the same simplicity of design. For the decoding case, the ROM
needs a 7-bit address and 4-bit words, or a 128-word x 4-bit ROM (see Figure
3). The trick is in deciding what to store in each word of the ROM.
In those cases where the incoming code word is correct, the answer is clear.
There are 16 possible message words (2{4}) and 16 different correct code
words. The message word is stored at each of these 16 ROM addresses. When the
7-bit code word is presented at the inputs, the embedded message word will be
delivered at the outputs. It can then be passed on by the receiver as correct
message word.
For example, if the (valid) code word 1011011 is received, it is applied to
the address inputs of the ROM. The data stored at that location is 1011, the
four embedded message bits from the code word. The data at address X = X, for
the 16 correct code words.
The remaining 112 (2{7} - 2{4}) ROM locations correspond to erroneous code
words. Given a set of 16 possible correct code words and assuming that one
single-bit error occurs, each original code word can change in seven different
ways. This means there are 16*7 = 112 possible erroneous code words, but
because of the redundancy built in, a received code word containing a single
error has one unambiguous correct code word. The reason this is so has to do
with the "Hamming distance" of the code scheme, a concept that would,
unfortunately, require a rather lengthy explanation.
We must store something at these 112 locations that will automatically correct
the message portion of the code word. If each erroneous code word is a ROM
address, then the data stored at that address should be the corrected code
word. In other words, the data stored at address Y = X, where X is the
corresponding correct code word.
Assume that the code word 1010011 is received and presented at the address
inputs of the ROM. As it happens, this is an erroneous code word. The data
stored at location 1010011 would be the corrected code word, 1011, without the
check bits.



Hamming-Code Benefits


By considering each permutation in advance (an easily computable task for any
realistic code word length), it is possible to program a ROM to do the
decoding at the receiving end. The ROM-based design is simpler and cleaner
than the exclusive-OR circuit and, for small word sizes, approximately as
fast.
Furthermore, ROM-based designs can easily be extended to handle longer message
and code word lengths. With longer code words, the efficiency ratio of message
bits to check bits increases, although the chances of an error occurring in a
given code word are increased. The ROM-based design is also able to
accommodate different code schemes that can detect and/or correct more errors
in a code word by using more check bits and assigning the groups differently.
In these cases, a different size ROM is substituted and programmed. With the
exclusive-OR circuit, a complete redesign is often necessary.


References


Hamming, R.W. "Error Detecting and Error Correcting Codes," Bell System
Technical Journal, vol. 29, (April, 1950), 147 - 160.
Kohavi, Zvi. Switching and Finite Automata Theory, Second edition. New York:
McGraw-Hill, 1978. 14 - 19.
















































October, 1989
EXECUTABLE SPECIFICATIONS WITH PROLOG


Prolog's dual nature takes you from design to implementation




Gregory L. Lazarev


Gregory Lazarev is the president of Applied Logic Programming, Inc. and can be
reached at 262 Tomkenn Rd., Philadelphia, PA 19151.


The Prolog language is the best known example of a Logic Programming
implementation, with the uniqueness that it can be viewed declaratively as
well as procedurally. Therefore, Prolog may provide an attractive tool in
software engineering, particularly in the areas of "executable" specifications
and rapid prototyping (Davis, 1982; Kowalski, 1982; Leibrandt and Schnupp,
1984). Lazarev (1989) described a mapping mechanism for converting a
structured specification in the form of data flow diagrams (DFDs) into an
executable Prolog program. In this article, the mapping is extended by
including a case of partial matching. Several examples of mapping are
presented, and the advantages of using this methodology to improve the quality
of DFDs are discussed.


Introduction


The standard software development cycle of analysis, design, and
implementation provides many benefits, but it also has serious drawbacks. One
problem is that the mapping from a declarative specification to its procedural
implementation is complex. There is no automated procedure to transform
analysis into design, and verifying program correctness is an unresolved
problem.
The nonexplicit representation of requirements reveals itself in maintenance
problems. The lack of executable specifications (due to informal syntax and
semantics) leads to long delays in evaluating whether the direction chosen for
development is right.
Practical experiences using methodologies based on the
analysis-design-implementation model clearly demonstrate the critical nature
of these difficulties (Richter, 1986). It has been recognized that "an
analysis specification, once finished, proved to be of little value." In other
words, it neither can be verified nor transformed automatically into a design.
Further, "good analysis may require some design and implementation -- some
REAL TESTING of feasibility -- along the way."
Structured Analysis (De Marco, 1979) is often used to perform the software
analysis phase. Structured Analysis is a data-oriented methodology with three
main elements. First, there are data flow diagrams consisting of
transformations and data flows. Transformations (or "bubbles") represent
distinct logical functions, and data flows stand for the data input and output
associated with transformations.
The second element is a data dictionary, which defines elements of structured
analysis (that is, data flows, transformations, files, and so forth). Finally,
there is a structured English specification used to describe functional
primitives. Functional primitives are the nonpartitioned, lowest-level
bubbles.
In structured analysis, we decompose a transformation into new transformations
on a lower level. This procedure is continued until no more decompositions are
possible, and then structured English is used to describe the lowest-level
nondecomposable elements.
It can be argued that the core of the described problems is the existence of
two orthogonal semantics involved; the specification is declarative and the
implementation is procedural (Lazarev, 1987). Procedural information (such as
correlations between data flows) is provided on the level of functional
primitives but is totally separated from DFDs. Therefore, the usefulness of
structured analysis as a software engineering tool is limited. Prolog provides
a direction that narrows the gap between analysis and implementation. Prolog's
declarative nature is ideally suited for the analysis phase, while Prolog's
procedural capabilities are well suited for software implementation by making
a specification "executable." The specification becomes a program that can be
run and debugged. This approach is conceptually similar to rapid prototyping:
A user can see whether his requirements as stated are what he actually wants,
and the programmer can verify the correspondence of the program to the user's
requirements.


Mapping DFDs to an Executable Specification


The mechanism of automatic mapping between the DFDs (for all levels above the
level of functional primitives) and an executable Prolog program was described
by Lazarev (1989). The most important feature of this mapping is the
one-to-one correspondence between the DFDs and the resulting Prolog program.
Each DFD is represented as a Prolog predicate with two arguments -- input and
output structures. Each structure's components are elements of the Data
Dictionary. DFDs by themselves are not enough to generate Prolog code
automatically. The missing ingredient is a correlation among the input and
output data flows for each transformation or, alternatively, a correlation
among transformations themselves. We'll assume that these correlations (AND
and OR correlations) are known.
The following steps describe a mapping algorithm. The case of partially
matched inputs and outputs represented by (d.4) and (d.5) is an extension of
the original algorithm. Steps (a), (b), and (c) are performed on every DFD
level. Step (d) deals with interaction among levels:
a) Describe each transformation as a predicate with the same name. All
predicates have two arguments; an input argument represented as structure
i(I1, I2, ..., In), where I1, .., In are input data flows; and an output
argument represented as structure o(O1, O2, ..., On), where O1, .., On are
output data flows.
b) Represent OR-correlations explicitly by replacing predicates with several
predicates of the same name, one for each mutually exclusive case. For
example, a predicate bb( i(B), o(B1, B2, B3)) with OR-correlations among data
flows B1, B2, and B3 can be replaced with the following:
 bb( i(B), o(B1))
 bb( i(B), o(B2))
 bb( i(B), o(B3))
c) Review the resulting set of predicates and mark all groups that form
mutually exclusive sets. Such cases may arise either from OR-correlations
among data flows or from more complicated OR-correlations among
transformations.
d) Build a set of Horn clauses. Each clause has an upper-level predicate as
its head, and predicates from one level below as its body. There may be
multiple clauses for any given upper-level predicate. Clauses must satisfy the
following requirements for a goal (that is, a head's predicate) and subgoals
(body's predicates):
d.1) Each group of mutually exclusive predicates marked in (c) can provide, at
most, one subgoal in a clause.
d.2) Each input of each subgoal must be matched with some input of the goal or
with an output of some previous subgoal (a balanced input).
d.3) Each output of each subgoal must be matched with some output of the goal
or with an input of some later subgoal (a balanced output).
d.4) If (d.2) or (d.3) fail then unmatched inputs (outputs) of a subgoal are
replaced by an atom nil. At least one input and one output of each subgoal
must be non-nil.
d.5) Unmatched arguments of the goal (inputs and outputs) are replaced by an
atom nil.
Let's illustrate this procedure using a model project life cycle taken from De
Marco (1979) (see Figure 1a, Figure 1b, and Figure 1c). Minor changes have
been made in decomposing the structured analysis "bubble" (Figure 1c).
These DFDs do not include any OR-correlations, so steps (b) and (c) in the
mapping algorithm do not apply. The corresponding Prolog program is shown in
Listing One. In order to be executable, it includes facts (like those shown in
Example 1) that provide a representation for the lowest decomposition level.
Example 1: Sample Prolog facts from Listing One

 str_design( i( struct_spec, config_data),
 o( test_plan, pack_design) ).
 derive_log_equiv( i( user_req, curr_phys_DFD),
 o( curr_log_DFD) ).

Despite the fact that the decomposition process can be stopped at any level,
the value of such a mapping program is maximized by going to the level of
nondecomposable functional primitives.
As Listing One demonstrates, the mapping preserves key features of DFDs such
as a top-down design with capabilities for further refinement, and modularity.
Developing an executable specification substitutes -- to a large degree -- for
the design and programming phases of the traditional structured methodology.
The resulting program is strictly logical, therefore there are no predefined
input/output arguments (a property known as invertability). Three query types
are supported:

1. Find all possible input/output combinations. This corresponds to the case
of totally unbound parameters. The query has the form:
 ?- predicate( X, Y)
2. For some given input/output parameters find unbound parameters (that is,
conditions under which the DFD predicate is true). This corresponds to the
case of partially instantiated variables, and includes two important
subclasses:
a) For a totally defined input, find an output
b) For a totally defined output, find an input
3. Check whether the DFD predicate with all variables instantiated is true.
The queries, shown in Figure 2, for Listing One provide an illustration.
Figure 2: Queries used for Listing One

 ?- project(X, Y).
 X = i(user_req0)
 Y = o(system0, budg_sched0)

 ?- project(i( user_req0), Out).
 Out = o(system0, budg_sched0)

 ?- project(i( user_req0), o(system0, budg_sched0)).
 yes

 ?- str_anal(X, Y).
 X = i(user_req0, feas_doct0)
 Y = o(struct_spec0, budg_sched0, phys_req0)

 ?- str_anal((user_req0, 12), Out).
 12 = feas_doct0
 Out = o(struct_spec0, budg_sched0, phys_req0)



Extension to Partial Matching


Data flow diagrams may have serious deficiencies, and because of the
complexity of the total picture, problems often remain hidden. The mapping
procedure based on Prolog may be helpful in revealing these problems. Examples
in this and the following sections illustrate this point.
Consider the diagrams in Figure 3.
Suppose that transformations bb and cc are declared to be OR-correlated. Step
(c) of the mapping algorithm produces the following set of mutually exclusive
predicates:
 bb(i(A,B), o(O1))
 cc(i(C), o(O2))
Step (d) cannot be achieved using full matching, but partial matching using
nil atoms is possible. The output is shown in Figure 4.
Figure 4: Resulting program generated from the DFD in Figure 3

 top(i(l), o(O1, nil)):- aa(i(I), o(A, B, nil)),
 bb(i(A, B), o(O1)).
 top(i(l), o(nil, O2)):- aa(i(I), o(nil, nil, C)),
 cc(i(C), o(O2)).

The program in Figure 4 demonstrates that, because of OR-correlations between
transformations bb and cc; the output of transformation aa is either (A and
B), or C (but not both); and the output of the transformation top is either O1
or O2 (but not both). In other words, the appearance of "nils" shows that DFDs
can be further decomposed ("normalized"). In our case, the diagram on each
level can be replaced by two, as shown in Figure 5.
The mapping of this diagram results in a "nil-free" Prolog program. The same
program can also be obtained from Figure 3 by declaring two OR-correlations
among data flows:
 (1) O1, O2
 (2) (A, B), C
One role of nil-arguments in programs, similar to Figure 4, is to perform a
unification of distinct concepts. Under some circumstances it makes sense to
use such programs rather than to perform further decomposition.
"Disconnected networks" are a special case of partial matching. Consider, for
example, the two-level diagrams in Figure 6.
Suppose that level 2 diagrams are mutually exclusive (that is, OR-correlated).
As can be seen from Listing Two the nil arguments in the heads of both clauses
fully complement each other. This is an indicator of disconnected networks.
Such a case must be transformed into a simpler case (in this case, by deleting
a level 1 diagram and moving current level 2 diagrams to level 1).


Lost Causality


I've shown how the quality of DFDs can be evaluated by analyzing the
corresponding executable specifications. But, an inability to create an
executable specification from DFDs when using the described algorithm may be a
sign of problems in the diagrams: They may be "non-normalized" or they may
have sequencing ambiguities (which are resolved in structured English). For
instance, consider the two-level diagram in Figure 7.
Suppose that there are no OR-correlations and no further decomposition
provided. Step (a) of the mapping algorithm produces the following set of
predicates:

 top( i(K), o( L) ).
 aa_ee( i(K, D), o( A, L) ).
 bb_dd( i(A, C), o( B, D) ).
 cc( i(B), o(C)).
But step (d) fails because there is no way to express the predicate top in
terms of three lower-level predicates (aa_ee, bb_dd, cc). The reason is
straightforward: The predicates are involved in a circular relation. For
example, the predicate bb_dd must precede cc according to the data flow B, but
the same predicate must follow cc according to data flow C. In a more complex
diagram, circular structures may not be seen as easily, but the mapping
algorithm provides a good indication of such problems. One solution may be
further decomposition. Suppose that predicates aa_ee and bb_dd may be
decomposed into two independent predicates (aa, ee and bb, dd). In this case,
step (a) results in:
 top( i( K), o( L) ).
 aa( i( K), o( A) ).
 ee( i( D), o( L) ).
 bb( i( A), o( B) ).
 dd( i( C), o( D) ).
 cc( i( B), o( C) ).
The resulting clause is straightforward:
 top( i(K), o(L) ) :-
 aa( i( K), o(A) ),
 bb( i( A), o( B) ),
 cc( i( B), o( C) ),
 dd( i( C), o( D) ),
 ee( i( D), o( L) ).
The sequence of processes in this program (aa, bb, cc, dd, ee, in this order)
shows that any attempt to express top in terms of composite processes (like
aa_ee or bb_dd) with nontime-sequential elements will fail. "Causality," or
time-dependence, is lost and a program cannot be constructed.


Conclusion


By incorporating 1. a set of DFDs with data flows defined by a data
dictionary, 2. procedural information taken from functional primitives, and 3.
the procedural semantics of Prolog, it is possible to create executable
specifications as software programs. Such a program would contain basic
control structures that is, sequence, decision, and repetition (through
recursion). The output programs are in Prolog. Although Prolog is base on a
restricted form of logic, it is adequate for our purposes. These programs are
"pure" (free from side effects). As such, they are invertible and consequently
provide a broader scope of experimentation than corresponding DFDs.
Not only can the specification be made executable, but the suggested approach
is also helpful in verifying and improving specifications. This is done by
incorporating a partial matching extension into an original algorithm.
The approach presented is evolutionary. It extends the DFDs' methodology by
making it executable. As a result, the main advantage of the standard DFD
methodology -- a graphical language used to solve a communication problem --
remains intact. All of the above makes logic programming and Prolog very
promising in software engineering.


References


Davis, R. Runnable Specification as a Design Tool, "Logic Programming" (eds.
Clark and Tarnlund), Academic Press, London: 1982, pp. 141-149.
Kowalski, R.A. "AI and Software Engineering," Datamation, Nov. 1984, pp.
92-102.
Lazarev, G.L. "Solving Problems with Prolog," AI EXPERT, July 1987, pp. 59-68.
Why Prolog? Prentice-Hall, 1989.
Leibrandt, U. and Schnupp, P. "An Evaluation of Prolog as a Prototyping
System," in Rapid Prototyping, R. Budde (ed.), Springer-Verlag, Berlin: 1984,
pp. 424-433.
De Marco, T. (1979) Structured Analysis and System Specification, Yourdon
Press, New York, N.Y.: 1979.
Richter, C. "An Assessment of Structured Analysis and Structured Design," ACM
Sigsoft Software Engineering Notes, 1986, v.11, no. 4, pp. 75-83.

_Executable Specifications With Prolog_
by Gregory Lazarev



[LISTING ONE]


project( i( User_req), o( System, Budg_sched) ):-
 survey( i( User_req), o( Feas_doct) ),
 str_anal( i( User_req, Feas_doct),
 o( Struct_spec, Budg_sched, Phys_req) ),
 hw_study( i( Phys_req), o( Config_data, Hardw) ),
 str_design( i( Struct_spec, Config_data),
 o( Test_plan, Pack_design) ),
 implem( i( Hardw, Test_plan, Pack_design), o( System) ).

str_anal( i( User_req, Feas_doct),
 o( Struct_spec, Budg_sched, Phys_req) ):-
 study_curr_environ( i( User_req), o( Curr_phys_DFD) ),

 derive_log_equiv( i( User_req, Curr_phys_DFD),
 o( Curr_log_DFD) ),
 model_new_log_sys( i( User_req, Curr_log_DFD, Feas_doct),
 o( New_log_DFD, DD, Trans_desc) ),
 model_new_phys_sys( i( New_log_DFD),
 o( New_phys_DFD, Budg_sched, Phys_req) ),
 produce_struct_spec( i( New_phys_DFD, DD, Trans_desc),
 o( Struct_spec) ).

survey( i( user_req0), o( feas_doct0) ).
hw_study( i( phys_req0), o( config_data0, hardw0) ).
str_design( i( struct_spec0, config_data0),
 o( test_plan0, pack_design0) ).
implem( i( hardw0, test_plan0, pack_design0), o( system0) ).
study_curr_environ( i( user_req0), o( curr_phys_DFD0) ).
derive_log_equiv( i( user_req0, curr_phys_DFD0),
 o( curr_log_DFD0) ).
model_new_log_sys( i( user_req0, curr_log_DFD0, feas_doct0),
 o( new_log_DFD0, dd0, trans_desc0) ).
model_new_phys_sys( i( new_log_DFD0),
 o( new_phys_DFD0, budg_sched0, phys_req0) ).
produce_struct_spec( i( new_phys_DFD0, dd0, trans_desc0),
 o( struct_spec0) ).






[LISTING TWO]

bb( i( B, B1, nil), o( nil, E) ) :-
 bb'( i( B, B1), o(E) ).
bb( i( nil, nil, C), o( D, nil) ) :-
 bb''( i( C), o( D) ).



























October, 1989
A GLOBAL VARIABLE DEVICE DRIVER FOR MS-DOS


GLOVAR lets you modify an ancestor's environment




Jim Mischel


Jim Mischel is a former financial systems programmer and database consultant.
He can be reached at 20 Stewart St., Durango, CO 81301 or on CompuServe:
73717,1355.


While the MS-DOS environment string setup is a handy way to define system-wide
information, it suffers from one major drawback: The inability of a child
process to make permanent modifications to an ancestor's environment. Why, one
might ask, would you want to do such a thing? I asked the same question the
first time somebody brought it up, but as luck would have it, I soon found
myself answering it.
I support an application that makes extensive use of environment variables to
store configuration information; things like screen colors, data drive
assignments, and so on -- many of which are session-dependent. The users have
a bad habit of using the "DOS Shell" command and then executing the
application again, then EXITing back to the previous instance of the program.
This doesn't cause a problem until some bright-minded individual exits the
application, EXITs DOS, and winds up back in the first instance of the
program. Any changes made in the second instance of the program have now been
lost and the first instance may contain invalid configuration information.
To complicate matters, the application runs on a diskless network workstation,
making it difficult to create a session configuration file. Add to this the
TSR that will be communicating with and getting its configuration information
from the application, and the need for system-wide global variables becomes
evident.
My first reaction, of course, was to use the DOS environment variables; an
approach I soon discarded when I found that child processes can't change the
parent's environment. Next I tried back-chaining -- passing environment
addresses from one program to the other, thereby always changing the
ancestor's environment. This worked fine but for two drawbacks. First, if a
change to an ancestor's environment would required more memory, DOS would
allocate another block of memory above the currently active program -- thus
forever trapping any programs between the ancestor and the newly allocated
block. The other problem is with COMMAND.COM. Whenever a process shells to
DOS, COMMAND.COM is loaded and assumes that it is Lord of the Universe inside
your computer. Any back-chaining stops at the latest invocation of
COMMAND.COM. I've heard that it's possible to back-chain beyond this point,
but I'm not too sure the method is stable.
Several months ago, somebody put forth this problem on the CompuServe DDJ
Forum. At that time, someone else suggested that a device driver be written
that would support true global variables. I waited a while for the driver to
appear but got impatient and decided to tackle it on my own.
The global variable device driver (GLOVAR.ASM, Listing One) is installed in
your system at boot time. Simply place the line
 device=glovar.sys
in your CONFIG.SYS file and reboot the machine. Access to the global variables
is provided through the use of the DOS read command and several functions that
are provided in the device driver. Example programs in both Turbo C (GVAR.C,
Listing Two and GVAR.H, Listing Three), and Turbo Pascal (GVAR.PAS, Listing
Four) demonstrate the use of the driver. The Turbo Pascal program requires an
assembly language interface to the driver (TPGVAR.ASM, Listing Five). Compile
and link instructions are included as comments in the listings.


How to Use It


In order to access the global variables, an application program must first
open the device (using the standard DOS open file function) and then read the
address of the interface structure from the device. The read routine in the
device driver is rather simple minded in that it assumes the application is
requesting 4 bytes of information (the size of a far pointer). The device can
be closed after this read, as all other functions are provided through direct
calls to the driver's code. init and read are the only two DOS functions
supported by the driver.
The interface structure contains four far pointers; a pointer to the memory
block and a pointer to each of the three interface functions. This structure
table is shown in the C header file (GVAR.H) and in the Turbo Pascal interface
routines (TPGVAR.ASM).
The three functions provided by the device driver, their C and Pascal
declarations, and a short description are shown shortly. The Pascal functions
use a type VARSTR, that is defined as
 type varstr = string[255];

function set_gvar (varname, vardef: varstr): boolean; external; int far pascal
set_gvar (const
char far * varname, const char far * vardef);
set_gvar assigns a value to a global variable. It returns TRUE (-1) if the
variable was set successfully, or FALSE (0) if the global variable buffer in
the driver is full. A variable can be removed by assigning it a null value (as
in set_gvar ("VARNAME","");).
 Function get_gvar (varname: varstr; var vardef: varstr): boolean;
external;int far pascal get_gvar (const char far *varname, char far * vardef);
get_gvar returns the value of a global variable previously defined using the
set_gvar function. If the global variable exists, the function returns TRUE
(-1) and the vardef parameter holds the value of the variable. If the global
variable does not exist, the function returns FALSE (0), and the vardef
parameter is a null string.
 procedure flush_gvars; external; void far pascal flush_gvars (void);
flush_gvars removes all variable definitions from the buffer.
Using the routines is really quite simple. In the C version, the three
functions set_gvar(), get_gvar(), and flush_ gvars() are macros that expand to
indirect function calls through pointers in the GVAR_DEF structure. The Pascal
version calls assembly language subroutines that convert the passed arguments
to ASCIIZ strings, and then perform an indirect function call in much the same
manner as the C version.
There are some cautions and a few restrictions regarding use of the driver.
First, because Turbo Pascal's strings have a maximum length of 255 bytes,
there is a possibility of string overrun when using a Turbo Pascal program to
retrieve variables set by C or assembly language programs. Neither the driver
nor the Turbo Pascal interface routines address this issue. An overrun string
will most likely cause your program to crash. The same goes for C programs --
if your buffer is smaller than the global variable, unpleasant things are
bound to happen.
Global variable names may contain any characters other than the equal sign (=)
and null (0). Variable values may contain any character other than null. All
variable names are mapped to uppercase.
You may have noticed that the address of the memory block is provided in the
GVAR_DEF structure. This pointer has been provided as a means to do other
things with the block of reserved memory (such as passing data between
programs). Note, however, that using the memory in this manner will interfere
with any programs that are using the global variable routines.


How It Works


Upon initialization, the driver outputs a sign-on message, places the proper
addresses in the interface structure (global_table), sets the required
top-of-memory address for DOS, and flushes the global variable table. Because
the initialization code is executed only once, the global variable table
overlays it, thus reducing the amount of memory taken up by the driver.
The read call is equally simple. It merely takes the address of the interface
structure and writes it as a long pointer into the buffer provided by DOS in
the request header. It then sets the number of bytes read to 4 and exits. The
set_gvar function works in two steps. First, it finds and removes the
specified variable and its definition from the table, moving the rest of the
entries to fill the space previously occupied by the now deleted variable.
Then it installs the variable name (mapped to uppercase) with the new
definition at the end of the table. This process is a bit slower than the
"use-until-full-then-collect-garbage" method of managing string space, but it
makes the driver code much simpler (and thus smaller). If I was dealing with a
much larger block of memory (in the order of 10K or more), I'd worry more
about the time involved in moving things around.
The get_gvar function uses the find_var internal function to locate the
variable (if it exists), finds the beginning of the definition, and copies it
into the buffer provided in the 'vardef' parameter.
The flush_gvars function is the ultimate in simplicity. It simply places two
null bytes (ASCII 0) at the beginning of the table, thereby signifying the end
of all defined strings.
I was a little apprehensive at first about attempting to write a DOS device
driver. Happily, it turned out to be quite a bit easier than I had expected.
All of the information required was gleaned from Ray Duncan's excellent book
Advanced MS-DOS, Microsoft Press, 1986. In fact, the device driver template
presented in that book formed the basis of my driver.
The most difficult part of this entire exercise was getting the Turbo Pascal
interface functions to work. For some reason I couldn't get Turbo Debugger to
work in source debugging mode with the assembly language functions. Most
likely, I was doing something wrong. Testing the device driver was really
quite simple. I originally wrote it to be linked into my Turbo C test program
and used Turbo Debugger to test all the routines. When I was convinced it was
stable, I made the necessary modifications, booted my system with the device
driver installed, and was truly amazed when it worked the first time.
In its present state the driver will not support concurrent update by multiple
processes. Adding this functionality would be a matter of returning a 'busy'
status in response to a get, set, or flush call. This modification is left as
an exercise to the reader.
While this driver does not provide permanent modification to an ancestor's
environment, it does provide a safe, fast, documented way to maintain true
system-wide global variables, thus achieving the desired effect without
unnecessarily complicating the application program.



Availability


All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063; or call 800-356-2002 (from inside
Calif.) or 800-533-4372 (from outside Calif.). Please specify the issue number
and format (MS-DOS, Macintosh, Kaypro).

_A Global Variable Device Driver for MS-DOS_
by Jim Mischel



[LISTING ONE]

 name globals
 page ,132
 title 'GLOVAR - global variable device driver
;
; GLOVAR.ASM
;
; Copyright 1989, Jim Mischel
;
; This MS-DOS device driver provides true global variables to application
; programs.
;
; Sample make file:
; #
; # glovar.mak - build global variable device driver glovar.sys.
; #
; glovar.sys: glovar.asm
; tasm glovar
; link glovar;
; exe2bin glovar.exe glovar.sys
;
 ideal
 locals

 model tiny,pascal
 CodeSeg

 org 0 ;device drivers start at offset 0

max_cmd equ 12 ;only 12 functions recognized by the driver
cr equ 0dh
lf equ 0ah
buff_size equ 1024 ;buffer size in bytes
TRUE equ -1
FALSE equ 0

;
; Device Driver Header
;
header dd -1 ;link to next device, -1 = end of list
 dw 8000h ;simple character device driver
 dw strat ;device Strategy entry point
 dw intr ;device Interrupt entry point
 db '$$GVAR$$' ;Device name

rh_ptr dd ? ;saved request header pointer


;
; Command codes dispatch table. Only functions 0 (initialize) and 4 (read)
; are supported. Other functions go to a stub routine that simple returns
; a non-error status.
;
dispatch:
 dw init ; 0 = initialize driver
 dw not_imp ; not implemented
 dw not_imp ; not implemented
 dw not_imp ; not implemented
 dw read ; 4 = read from device
 dw not_imp ; not implemented
 dw not_imp ; not implemented
 dw not_imp ; not implemented
 dw not_imp ; not implemented
 dw not_imp ; not implemented
 dw not_imp ; not implemented
 dw not_imp ; not implemented
 dw not_imp ; not implemented
 dw not_imp ; not implemented
;
; MS-DOS Request Header structure definition
;
struc request
rlength db ? ;length of request header
unit db ? ;unit number for this request (block devices)
command db ? ;request header's command code
status dw ? ;driver's return status word
reserve db 8 dup (?) ;reserved area
media db ? ;media descriptor byte (block devices)
address dd ? ;memory address for transfer
count dw ? ;byte/sector count value
sector dw ? ;starting sector value (block devices)
ends request ;end of request header template

; Status codes returned by routines in AX
error equ 8000h
busy equ 0200h
done equ 0100h
unknown_command equ 3

;
; Device strategy routine -- save address of request header.
;
strat: mov [word ptr cs:rh_ptr],bx
 mov [word ptr cs:rh_ptr+2],es
 retf

;
; Interrupt routine. Dispatch to the proper routine.
;
proc intr far uses ax bx cx dx ds es di si
 pushf
 push cs
 pop ds ;point to local data
 les di,[rh_ptr] ;ES:DI = Request Header
 mov bl,[es:di+request.command] ;bx = command code
 xor bh,bh
 cmp bx,max_cmd ;make sure it's legal

 jle intr1
 mov ax,error+unknown_command ;Error: unknown command
 jmp short intr2
intr1: shl bx,1 ;form index to dispatch table
 call [word ptr bx+dispatch] ;and branch to driver routine
 les di,[rh_ptr] ;ES:DI = request header
intr2: or ax,done ;set Done bit
 mov [es:di+request.status],ax ;store status in request header
 popf
 ret
endp intr

;
; 'stub' routine for un-implemented functions.
;
not_imp:
 xor ax,ax ;set good status
 retn

;
; Read -- return address of 'global_table'
;
; This routine assumes that the application program asked for 4 (no less!)
; bytes (the size of a far pointer).
;
; This function is called with ES:DI pointing to the MS-DOS request header.
;
read: lds si,[dword ptr es:di+request.address] ;DS:SI is buffer address
 mov [word ptr ds:si],offset global_table ;store offset
 mov [word ptr ds:si+2],cs ;and segment
 mov [word ptr es:di+request.count],4 ;4 bytes transferred
 xor ax,ax ;return good status
 ret

;
; The following three routines (set_var, get_var, flush_vars) are accessed
; by applications programs through FAR calls. The addresses of these
; functions are provided in the 'global_table' structure, the address of
; which is passed to the application when it does a read on the $$GVAR$$
; device.
;

;
; set_var -- assign a value to the global variable pointed to by the
; 'varname' parameter. If the 'vardef' parameter points to the null string,
; the variable is removed from the table.
;
; Returns TRUE (-1) if successful, FALSE (0) if variable table overflows.
;
proc set_var far uses si di ds, varname:far ptr, vardef:far ptr
 lds si,[varname]
 call near ptr remove_var ;try to remove variable
 les di,[vardef]
 cmp [byte ptr es:di],0 ;check for NULL string
 jnz do_set
 mov ax,-1 ;and if so, just call it quits
 jmp short set_done
do_set: lds si,[varname] ;otherwise,
 call near ptr install_var ;install the new def

set_done:
 ret
endp set_var

;
; Return the definition of the variable pointed to by 'varname' in the
; space pointed to by 'vardef'. Returns -1 if the variable exists, 0 if
; not. If the variable does not exist the null string is returned in 'vardef'.
;
proc get_var far uses si di ds, varname:far ptr, vardef:far ptr
 lds si,[varname]
 call find_var
 jnc get1 ;if found, continue
 les di,[vardef] ;otherwise,
 mov [byte ptr es:di],0 ;store NULL string
 xor ax,ax ;set failure status
 jmp short @@done ;and exit
get1: cld ;variable found, now find '='
 mov al,'='
 mov cx,-1
 repnz scasb
 mov si,di
 push es
 pop ds ;DS:SI now points to first character of definition
 les di,[vardef] ;ES:DI points to return vardef area
@@loop: lodsb ;copy the definition to
 stosb ;the 'vardef' string
 or al,al
 jnz @@loop
 mov ax,TRUE ;set status to TRUE
@@done: ret
endp get_var

;
; flush all variables from the global variable table. This is done by placing
; a NULL byte at the start of the table and updating the 'next_mem' variable
; so that it also points to the head of the table.
;
flush_vars:
 mov [word ptr cs:glovars],0
 mov ax,offset glovars
 mov [word ptr cs:next_mem],ax
 retf

;
; Support routines.
;
; Remove the variable (name pointed to by ds:si) and its definition from
; the global variables table.
;
remove_var:
 call find_var ;find the variable
 jc @@done
 cld
 mov si,di ;SI is start of variable name
 mov al,0
 mov cx,-1
 repnz scasb ;move past definition
 add [word ptr cs:next_mem],cx ;update next_mem pointer

 inc [word ptr cs:next_mem]
 xchg si,di ;si=next variable, di=old var
 push cs
 pop ds
 mov cx,offset end_buff
 sub cx,si ;cx is byte count
 rep movsb ;variable has been erased
@@done: ret

;
; Called with ds:si pointing to variable name, es:di pointing to definition.
;
install_var:
 push es
 push di ;save 'vardef' address
 cld
 mov al,0
 mov cx,-1
 repnz scasb ;get length of definition
 not cx
 mov bx,cx ;and save
 push bx ;will be used to copy the definition
 push ds
 pop es
 mov di,si
 mov cx,-1
 repnz scasb ;get length of variable name
 not cx
 push cx ;will be used to copy the variable name
 add bx,cx ;bx=strlen(varname)+strlen(vardef)+2
 mov ax,buff_size + offset glovars
 sub ax,[word ptr cs:next_mem]
 cmp ax,bx
 jnc inst1 ;if not enough room
 xor ax,ax ;then fail
 add sp,8 ;clean up stack
 ret
inst1: push cs
 pop es
 mov di,[word ptr cs:next_mem] ;ES:DI points to buffer
 pop cx ;restore varname length
 dec cx ;don't want null terminator
@@loop1:
 lodsb
 call toupper ;convert char to upper_case
 stosb ;and store
 loop @@loop1
inst2: mov al,'='
 stosb ;store the '=' separator
 pop cx ;restore definition length
 pop si ;restore 'vardef' address
 pop ds
 rep movsb ;and copy definition to buffer
 mov [word ptr cs:next_mem],di
 stosb ;store extra null byte for safe keeping
 mov ax,TRUE
 ret

;

; find_var -- attempt to find the variable pointed to by DS:SI in the
; global variables buffer.
; On success, AX = TRUE (-1), carry is cleared, and ES:DI points to the first
; character of the variable name in the buffer.
; If the variable is not found, AX = FALSE (0), carry flag is set, and ES:DI
; is undefined.
;
find_var:
 cld
 push cs
 pop es
 mov di,offset glovars
 mov dx,si ;save name starting address
find0: mov bx,di ;and variable starting address
 mov si,dx
 dec di
find1: lodsb
 call toupper
 inc di
 cmp al,[byte ptr es:di]
 jz find1
 or al,al ;didn't match, at end of string?
 jnz get_next_var ;if not, go for next variable
 cmp [byte ptr es:di],'=' ;at end of varname in buffer?
 jnz get_next_var ;if not, do next variable
 mov di,bx ;variable found
 mov ax,TRUE
 ret
get_next_var:
 mov al,0
 mov cx,-1
 repnz scasb ;get start of next variable
 cmp [byte ptr es:di],0 ;if not at end of buffer
 jnz find0 ;then do next variable
 xor ax,ax
 stc ;the variable wasn't found
 ret
;
; toupper - convert the character in AL to upper-case.
;
toupper:
 cmp al,'a'
 jc tod
 cmp al,'z'+1
 jnc tod
 sub al,32
tod: ret

 even ;might as well

;
; The address of this table is passed back to the application whenever
; a 'read' call is made on the device.
; The application uses this table to access the memory buffer and the
; driver functions.
;
global_table:
 dd ? ;address of memory buffer
 dd ? ;address of set_var routine

 dd ? ;address of get_var routine
 dd ? ;address of flush_vars routine

next_mem dw offset glovars ;pointer to end of defined variables

glovars: ;start of glovar buffer

;
; Initialization code.
;
; Initialize the function pointers table and the global data buffer, and
; return the address of the first byte of free memory after all the device
; driver code. The global variables buffer overlays this initialization code.
;
; Upon entry, ES:DI points to the MS-DOS request header. DS contains the
; local data segment (same as CS). This routine makes no attempt to save
; any registers.
;
init: mov ax,cs
 mov es,ax
 mov di,offset dhaddr
 call hexasc ;convert load segment address to ASCII
 mov ax,cs
 mov es,ax
 mov di,offset mbaddr
 call hexasc
 push cs
 pop es
 mov ax,offset glovars
 mov di,offset mbaddr+5
 call hexasc
 mov ah,9 ;print sign-on message and
 mov dx,offset ident ;driver load address
 int 21h
 les di,[rh_ptr] ;request header address in ES:DI
;store first usable memory address in request header
 mov [word ptr es:di+request.address],offset end_buff
 mov [word ptr es:di+2+request.address],cs
;
; Now set up the 'global_table' structure with proper memory addresses.
;
 mov [word ptr global_table],offset glovars
 mov [word ptr global_table+2],cs
 mov [word ptr global_table+4],offset set_var
 mov [word ptr global_table+6],cs
 mov [word ptr global_table+8],offset get_var
 mov [word ptr global_table+10],cs
 mov [word ptr global_table+12],offset flush_vars
 mov [word ptr global_table+14],cs
 xor ax,ax ;return good status
 mov [word ptr glovars],ax ;and clear glovars buffer
 ret

ident db cr,lf,lf
 db 'Global variable device driver version 1.0',cr,lf
 db 'Copyright 1989, Jim Mischel',cr,lf
 db 'Device Header at '
dhaddr db 'XXXX:0000. '
 db 'Memory buffer at '

mbaddr db 'SSSS:OOOO',cr,lf,lf,'$'

;
; hexasc - converts a binary 16-bit number into a hex ASCII string
;
; call with AX = value to convert, ES:DI = address to store 4-character
; string.
;
hexasc: mov bx,ax ;save value here
 mov cx,4 ;initialize character counter
 mov dx,cx
hexasc1:
 xchg cx,dx
 rol bx,cl ;isolate next 4 bits
 mov al,bl
 and al,0fh
 add al,'0' ;convert to ASCII
 cmp al,'9' ;if 0-9
 jbe hexasc2 ;then jump
 add al,'A'-'9'-1 ;otherwise add offset for A-F
hexasc2:
 stosb
 xchg cx,dx
 loop hexasc1
 ret

; reserve space for rest of global variable buffer.
 db buff_size - (offset $ - offset glovars) + 1 dup (0)

 even
end_buff:
 end






[LISTING TWO]


/*
 * GVAR.C - program to test global device driver interface.
 * Required structure and function definitions are in the file GVAR.H
 *
 * Sample make file.
 *
 * #
 * # tcgvar.mak - make C gvar test program
 * #
 * gvar.exe: gvar.c gvar.h
 * tcc -ms gvar
 *
 */
#include <stdio.h>
#include <conio.h>
#include <fcntl.h>
#include <io.h>
#include "gvar.h"


void main (void) {
 char buff[1024];
 int fhandle;

 if ((fhandle = _open ("$$GVAR$$", O_RDONLY)) == -1) {
 puts ("Error: can't open global variables file");
 return;
 }
 read (fhandle, &gvars, sizeof (char far *));
 printf ("global table at %Fp\n", gvars);
 _close (fhandle);
 if (!set_gvar ("JIMSVAR", "Hi jim.!"))
 puts ("Error setting variable JIMSVAR");
 if (!get_gvar ("jimsvar", buff))
 puts ("JIMSVAR not defined");
 else
 printf ("JIMSVAR=%s\n", buff);
 set_gvar ("jimsvar","");
 if (!get_gvar ("JIMSVAR", buff))
 puts ("JIMSVAR not defined");
 else
 printf ("JIMSVAR=%s\n", buff);
}







[LISTING THREE]

/*
 * gvar.h -- required definitions for C interface to global variable
 * device driver.
 *
 * Copyright 1989, Jim Mischel
 */

/*
 * structure of table returned by glovar read call.
 */
typedef struct {
 void far *mem_buff; /* pointer to memory buffer */
/* function pointers */
 int far pascal (far *set_var) (char far *, char far *);
 int far pascal (far *get_var) (char far *, char far *);
 void far pascal (far *flush_vars) ();
} GVAR_DEF;

GVAR_DEF far *gvars; /* global pointer to global variables structure */

/*
 * function macros
 */
#define set_gvar (*gvars->set_var)
#define get_gvar (*gvars->get_var)
#define flush_gvars (*gvars->flush_vars)








[LISTING FOUR]

{*
 * GVAR.PAS - Test global variable device driver.
 *
 * This program depends on the routines in TPGVAR.ASM to provide the interface
 * to the device driver.
 *
 * Make sure the file tpgvar.obj is in the current directory
 * (or Turbo's /Object directory) before compiling this program.
 *
 * Sample make file for compiling this program:
 * #
 * # tpgvar.mak - make pascal gvar test program
 * #
 * gvar.exe: gvar.pas tpgvar.obj
 * tpc gvar
 *
 * tpgvar.obj: tpgvar.asm
 * tasm tpgvar
 *
 *}

{$F+} { asm routines require far calls }
{$L tpgvar}
program test_globals;
type
 varstr = string[255];

var
 buff : varstr;

function gvar_init : boolean; external;
function set_gvar (varname : varstr; vardef : varstr) : boolean; external;
function get_gvar (varname : varstr; var vardef : varstr) : boolean; external;
procedure flush_gvars; external;

begin
 if (not gvar_init) then begin
 writeln ('Can''t open gvars file. Program aborted.');
 halt;
 end;
 if (not set_gvar ('JIMSVAR', 'HELLO THERE'))
 writeln ('Error setting variable JIMSVAR');
 if (not get_gvar ('jimsvar', buff))
 writeln ('JIMSVAR not defined')
 else
 writeln ('JIMSVAR=', buff);
 trash := set_gvar ('jimsvar','');
 if (not get_gvar ('jimsvar', buff))
 writen ('JIMSVAR not defined')
 else

 writeln ('JIMSVAR=', buff);
end.






[LISTING FIVE]


 page ,132
 name tpgvar
 title 'TPGVAR -- Turbo Pascal global variable interface routines'
;
; TPGVAR.ASM
; These routines provide the Turbo Pascal interface to the global variable
; device driver.
;
; The routines provided are:
;
; GVAR_INIT
; function gvar_init : boolean; external;
; Returns TRUE if successful, FALSE otherwise. MUST be called before using
; any of the other functions.
;
; SET_GVAR
; function set_gvar (varname : varstr; vardef : varstr) : boolean; external;
; Defines the variable 'varname' with the value 'varstr'. Returns FALSE
; if inserting this variable will over-run the global variables buffer.
;
; GET_GVAR
; function get_gvar (varname : varstr; var vardef : varstr) : boolean;
external;
; Return the definition of the variable 'varname' in the string 'vardef'.
; Returns FALSE if 'varname' doesn't exist.
;
; FLUSH_GVARS procedure flush_gvars; external;
; Removes all global variables from the buffer.
;
; Copyright 1989, Jim Mischel
;
 model tpascal
 ideal
 locals

;
; global variable interface structure
; A pointer to this type of structure is returned by the $$GVAR$$ device
; in response to a read call.
;
struc gvars
membuff_ptr dd ?
set_var_ptr dd ?
get_var_ptr dd ?
flush_vars_ptr dd ?
ends

 CodeSeg
;

; This has to go into the code segment because 'TPascal' model won't allow
; initialization in the data segment.
;
gvar_filename db '$$GVAR$$',0 ;global variable device name
gvar_ptr dd ? ;pointer to global variables structure
glovar_name db 256 dup (?) ;passed to driver routines
glovar_def db 256 dup (?)

public GVAR_INIT
public SET_GVAR
public GET_GVAR
public FLUSH_GVARS

;
; open the $$GVAR$$ file and read the address of the header into gvar_ptr
; returns TRUE if successful, FALSE if not.
;
proc gvar_init uses ds
 push cs
 pop ds
 mov dx,offset gvar_filename
 mov ax,3d00h
 int 21h ;open the file
 jc @@error ;error opening file
 push ax ;save file handle
 mov bx,ax ;file handle in BX
 mov dx,offset gvar_ptr
 mov ah,3fh
 mov cx,4
 int 21h ;read address of gvar structure
 pop bx ;file handle in BX
 mov ah,3eh
 int 21h ;close the file
 mov ax,-1
 jmp short @@done
@@error:
 xor ax,ax
@@done: ret
endp gvar_init

;
; This macro converts the Turbo Pascal string in the 'from' parameter to an
; ASCIIZ string in the 'to' parameter.
;
macro @make_asciiz from, to
 lds si,[&from&]
 push cs
 pop es
 mov di,offset &to&
 push cs ;these values are pushed now, but not
 push di ;used until the gvar call below
 call near ptr make_asciiz
endm

;
; Macro calls the specified routine.
;
macro @gvar_call routine
 les bx,[cs:gvar_ptr]

 call [dword ptr es:bx+gvars.&routine&]
endm

;
; Define 'varname' with value 'vardef'. Returns TRUE if successful, FALSE
; if the global variables buffer overflows.
;
proc set_gvar uses ds, varname:far ptr, vardef:far ptr
 @make_asciiz varname, glovar_name
 @make_asciiz vardef, glovar_def
 @gvar_call set_var_ptr
 ret
endp set_gvar

;
; Return definition of 'varname' in 'vardef'. Returns TRUE if successful,
; FALSE if 'varname' does not exist. If 'varname' does not exist, 'vardef'
; is set to the null string.
;
; CAUTION: if the returned variable definition is longer than 255 characters,
; many strange and not-so-wonderful things will happen.
;
proc get_gvar uses ds, varname:far ptr, vardef:far ptr
 @make_asciiz varname, glovar_name
 push cs
 mov ax,offset glovar_def
 push ax
 @gvar_call get_var_ptr ;get the variable
 push ax ;save return status
 push cs ;now convert definition to TP string
 pop ds
 mov si,offset glovar_def
 les di,[vardef]
 push di ;save to set length
 inc di ;start string at second byte
 xor cx,cx ;cx is byte count
 cld ;better safe than sorry
@@loop: lodsb
 or al,al
 jz @@done
 stosb
 inc cx ;bump length
 jmp short @@loop
@@done: pop di ;restore pointer to length byte
 mov [byte ptr es:di],cl ;and store length there
 pop ax ;restore return status
 ret
endp get_gvar

;
; flush all global variables
proc flush_gvars
 @gvar_call flush_vars_ptr
 ret
endp flush_gvars

;
; convert Turbo Pascal string into ASCIIZ string.
; Call with ds:si pointing to source string (TP format)

; es:di pointing to destination buffer
; Make sure this is accessed through a near call.
;
make_asciiz:
 lodsb ;get length
 mov cl,al
 xor ch,ch ;in CX
 cld
 rep movsb ;move the string
 mov [byte ptr es:di],0 ;and place the null terminator
 retn
 end


















































October, 1989
FIRST LOOK AT COMMONVIEW


OOP for Windows and PM




Noel J. Bergman


Noel is a software development consultant specializing in leading-edge
technologies, including object-oriented programming, MS-Windows, and OS/2. He
can be reached on CompuServe at: 76704,34.


Object-oriented programming (OOP) promises to provide developers with a wide
range of benefits. Programs will be easier to write, easier to extend, and
easier to make portable. CommonView is an object-oriented C++ library
(currently bundled with Glockenspiel's first-rate C++ implementation for DOS
and OS/2). It is among the first such packages designed to bring OOP to the
masses of C programmers who work with (or are interested in working with)
Microsoft Windows and OS/2 Presentation Manager.
As we will see, CommonView has the potential to achieve this goal, but the
current incarnation is immature.


What is CommonView?


You say that you're all in favor of easier and more portable programming, but
what is Common-View? CommonView is an object-oriented library that maps the
functionality of graphical user interfaces (GUIs) onto a set of object
classes.
Glockenspiel bundles CommonView with their enhanced implementation of C++ for
OS/2, DOS, and Windows. The total package includes the C++ translators,
standard C++ libraries, the CommonView libraries, and tutorial and sample
programs. A "driver" program, which is similar to Microsoft's CL.EXE driver,
makes the process of using the C++ package almost like that of using Microsoft
C.
CommonView differs from most object-oriented programming languages (such as
Smalltalk, ACTOR, and C_Talk) that derive all of the classes in a tree with a
single-base class (object) at the root. CommonView's classes are divided into
a few separate categories: windows and events, controls, and miscellaneous
classes.


Installing CommonView


For most programmers, installing CommonView should be smooth. The CommonView
installation program allows you to specify a separate directory for each type
of file that the program installs. Unfortunately for me, however, I place
headers, libraries, and executables on a different partition from the
partition that contains sample code. The install program doesn't allow you to
specify separate drives, so the installation of CommonView on my personal
system was more involved. I had to install the different components into the
directory paths where I wanted the components to end up. I used XCOPY to move
directories from the installed disk to the disk where I wanted them to be, and
then deleted the original directories.
I encountered only a few minor glitches (such as leaving a header for one of
the sample applications in the wrong directory) during the process of
installing CommonView.
There is, however, one major problem for DOS users. The current version of C++
for DOS is very demanding with respect to memory, and some of the sample
applications will not compile on most DOS machines. To solve this problem, you
can spend an extra $100 to buy an extended memory version of C++. As another
solution, you can compile under OS/2 and then link under DOS, which is the
route I took.
A small problem also arises if you use separate names for the different types
of libraries. For example, I use SLIBCER, SLIBCEP, and SLIBCEW for the names
of the small model libraries for DOS, OS/2, and Windows. The driver program
isn't quite smart enough to use the libraries directly.
The modified .CMD and .BAT files that I use to build CardFile are listed in
Figure 1. Most of the options should be familiar because they are the same
options used by Microsoft C's CL driver. The -lY option tells CCXX to add the
library XlibY to the linker's list. (X is the model size, and Y is the
extension.) CCXX calls the resource compiler for you, so when you link under
DOS, the Windows resource compiler is called, rather than the PM's resource
compiler.
Figure 1: The modified CMD and .BAT files used to build CardFile

 MK.BAT: -- The original batch file from Glockenspiel

 ccxx -c -Gw -Zp cardfile.cxx
 ccxx -c -Gw -Zp edit.cxx
 ccxx -c -Gw -Zp expose.cxx
 ccxx -c -Gw -Zp cardapp.cxx
 ccxx -c -Gw -Zp cardstor.cxx
 ccxx -c -Gw -Zp dlg.cxx
 ccxx -c -Gw -Zp menu.cxx
 ccxx -c -Gw -Zp misc.cxx
 ccxx -Gw -Zp /NOE *.OBJ -oCardFile CardFile.RC slibw.LIB
 CardFile.DEF

 COMPIT.CMD: A batch file to compile under OS/2

 set cl=
 set
 INCLUDE=c:\pmsdk\include;c:\windows\include;c:\commonvu\include
 set ccxx=Lr -FPi -Gsw -Zp
 ccxx -c -Zp cardfile.cxx
 ccxx -c -Zp edit.cxx

 ccxx -c -Zp expose.cxx
 ccxx -c -Zp cardapp.cxx
 ccxx -c -Zp cardstor.cxx
 ccxx -c -Zp dlg.cxx
 ccxx -c -Zp menu.cxx
 ccxx -c -Zp misc.cxx

 LINKIT.BAT: A batch file to link under DOS with the Windows library

 set cl=
 set LIB=c:\pmsdk\lib;f:\winsdk\lib;c:\njb\lib
 set ccxx=Lr -Icew -FPi -Gsw -Zp
 ccxx /NOE *.OBJ -oCardFile CardFile.RC slibw.LIB /NOD:slibce
 CardFile.DEF



Windows and Events


The classes that make up the Window hierarchy are listed in Figure 2 . The
indentation style used in Figure 2 shows inheritance relationships.
Figure 2: Classes that make up the Window hierarchy

 EventContext
 Window
 AppWindow
 TopAppWindow
 ChildAppWindow
 DialogWindow

This category contains the classes responsible for implementing event
dispatching and windows. The ancestor class at the top of this hierarchy is
EventContext, from which a single descendant, Window, is derived.
CommonView's EventContext class virtualizes event dispatching. The inheritable
virtual functions are Dispatch and Default. Dispatch is an event router.
Default is the default event handler, and is invoked when a default event
handler isn't overridden, or when the dispatcher receives messages for which
it doesn't have a handler.
Technically, other types of event-driven systems could be derived from
EventContext, but none are provided in the current CommonView release. In
fact, the CommonView documentation says, "Although it is possible, in theory,
to derive from EventContext, it is not possible for the CommonView programmer
to overload effectively its member functions. To do this would require an
indepth knowledge of both CommonView's implementation and the underlying Event
system."
One of the drawbacks of this limitation is that you cannot implement serial
I/O with CommonView, unless you resort to the use of timer messages. The
simple fact is that in MS-Windows there are no communications events to
dispatch. Serial I/O is performed by using a modified event loop that polls
both the comm driver and the message queue.
Furthermore, CommonView's implementation of look and feel differs from the
implementation of look and feel that is represented by the MS-Windows and PM
Application Style Guides. In order to implement MDI under Windows 2.x, you
must play some poorly documented tricks with Windows, and work with messages
not directly supported in CommonView. Compounding the problem is the fact that
if you go to all the trouble to implement MDI under CommonView, you will code
look and feel directly into your application. If you port your application to
other environments you will be required to change look and feel, and thus
losing much of the promised portability. Also, if Microsoft changes either the
MDI specification or the way in which the specification is implemented (highly
likely), you will have to change the application code. This should be the
responsibility of the library.
If you decide to implement the Microsoft Style Guide suggestions in your
application, you will want to create your own "appropriate look and feel"
object subclasses based upon the CommonView classes, and then use them
throughout your coding. This approach makes the process of moving your
CommonView application to a new environment less problematic. A sample MDI
application available from Microsoft upon request or for downloading from
their CompuServe Forum. The sample application illustrates most of the nastier
tricks that are necessary.
The Window class provides two things. First, it implements an Event system for
windows. Second, it defines the methods for manipulating Window objects.
The first point is important to understand. Normal MS-Windows or PM
programming involves the use of the "Window Procedure" or WndProc. In
CommonView, WndProc is virtualized with the Window classes' event handlers.
The event handlers contain the list of private entry points to which Event
objects are sent in response to messages being received from the environment.
This process is performed by translating MS-Windows messages into CommonView
Event objects, and then dispatching them to the correct handler in the
relevant window.
The different subclasses of Event can be received by the different types of
event handlers. The subclass represent mouse events, key events, scroll
events, menu events, control events, and window status events.
The Window class implements approximately 30 event handlers that receive and
act on these Event objects. Some representative event handlers are WindowInit,
MouseButtonDn, Activate, Expose, VerticalScroll, and MenuSelect. Most
CommonView programmers will spend the majority of their time writing event
handlers. New types of Window subclasses are created by deriving a subclass
and overriding the inherited event handlers with new event handlers, which
implement the behaviors unique to the new class.
ChildAppWindow represents child windows. ChildAppWindows are owned by
instances of TopAppWindow, the class that represents pop-up windows in
MS-Windows. DialogWindow instances are modal dialogs. If you want to create
modeless dialogs, build them by using the AppWindow subclasses and Controls.


Control


All of the classes in the Control hierarchy (Figure 3) that come with
CommonView should also be familiar to Windows and PM programmers. This
hierarchy consists of many of the common controls that most programmers
already use, and can easily be extended. It is important to understand that
Controls do not do anything in the CommonView structure. Controls cause
instances of ControlEvt, ScrollEvt, EditEvt, and so on to be generated and
dispatched to event handlers that belong to windows. All of the work is
performed in the event handlers. The constructors used to build the Control
instances attach the Controls to DialogWindow or AppWindow subclass instances.
Text controls do handle the text editing that is part of the representation.
Figure 3: Classes in the Control hierarchy

 Control
 ScrollBar
 HorizScrollBar
 VerticalScrollBar
 FixedIcon
 TextControl
 Edit
 SingleLineEdit
 MultiLineEdit

 FixedText
 ListBox
 Button
 PushButton
 ClickBox
 RadioButton



Miscellaneous Classes


CommonView provides other classes such as cursors and bitmaps, although not in
any particular hierarchy. The collection of classes is provided in Figure 4.
Most of these classes should be familiar to Windows programmers and PM
programmers because they represent things within those environments with which
we are already acquainted. For example, Accel is a class for building and
working with keyboard accelerators. BitMap is a class for working with
bitmaps. ResString, on the other hand, may not be familiar. This class is
CommonView's way to handle constant strings in Resource files. ResStrings can
be useful for allowing localization of your program (that is, customize
messages or switch languages). Pair simply provides an ordered pair of
integers, but is still a useful class. For example, Range can specify the
limits of a scrollbar, a Point might contain a screen location.
Figure 4: The collection of miscellaneous classes provided in CommonView

 Accel
 Bitmap
 Brush
 Caret
 Color
 Cursor
 Font
 Icon
 Menu
 SysMenu
 MessageBox
 ErrorBox
 Pair
 Dimension
 Point
 Range
 Selection
 Pen
 Rectangle
 ResString



Containers


CommonView comes with, and makes extensive use of, a set of extremely useful
classes that Glockenspiel designed and released to the public domain. These
classes are Container and FreeStore, which make up CommonView's
memory-management system.
Containers are a virtualization of various logical access methods. For
example, CommonView provides Heaps, Rings, Stacks, and Tables. Containers
represent the way that we think about locating and accessing our data. For
example, Tables provide keyed access to data. They are implemented as Rings
with key access, but could also be implemented with a B-tree. Containers do
not handle the actual storage of the items -- that is where FreeStores come
into play.
FreeStores are a virtualization of physical (rather than logical) access to
data. They provide a virtual Heap construct, on top of which the Container
classes are built. CommonView comes with three FreeStore classes for
MS-Windows: LocalHeap, GlobalHeap, and disk-based Heap. The disk-based
FreeStore has not been implemented for OS/2, although that will undoubtedly be
corrected in a future release. The source for all of the Container and
FreeStore classes is provided, so you can implement the disk-based FreeStore
yourself if you need it.
Containers and FreeStores take data and provide you with handles. Each
Container is typed, which means that you only store a single kind of data
within the Container. Actually, however, this is not entirely true. As long as
each item's class is derived from the class whose type the Container holds, it
is OK to store instances of that class in the Container. Thus, if you create a
Heap of Windows, you can store instances of any Window subclass within that
Heap.
A third class, Lock, is related to Containers and FreeStores. Lock is used to
access data contained within that Heap. A Lock is created with the handle of
the data item, and ensures that the data is both available in memory and
locked in place. When you are finished with the Lock, you destroy it, either
explicitly or by letting it go out of scope.
The Lock class is primarily an OOP convenience. The actual locking and
unlocking process is performed by the FreeStore.


DOODLE: A Sample CommonView Application


Let's take a look at a simple WinApp from the CommonView package that simply
allows us to freehand sketch with the mouse. DOODLE (Listing One) is about as
basic a CommonView program as one might want to write.
DOODLE defines a new subclass of TopAppWindow. This new type of window, the
DoodleWind, only overrides two of the inherited default event handlers:
MouseButtonDn and MouseDrag. Notice that no constructor is defined for
DoodleWind. The protected constructor inherited from TopAppWindow is
sufficient.
When the mouse button is depressed, DoodleWind remembers the starting location
for the freehand sketch. When the mouse is moved, DoodleWind moves to the last
remembered location of the mouse, draws a line from that location to the
current mouse location, and remembers the current mouse location for use the
next time MouseDrag is invoked by the Event dispatcher.
That's it. In a few lines of code this WinApp creates a movable, resizable
window with a system menu, and is capable of freehand drawing. We could extend
it fairly easily. Let's say that we maintain a list of the mouse locations
used in the drawing, perhaps by using a subclass of Ring -- one of
CommonView's Container classes. We would override the default Expose handler
to repaint the display, using the stored Points in our PointRing. We also
override the default MenuCommand handler while adding a menu item, in order to
erase the list of Points and start over. When MenuCommand handles the erase
request, it should also use the RePaint method provided by Window in order to
force a redisplay of the whole window -- MenuCommand should not erase the
window itself.
MoveTo and LineTo are methods inherited from Window. LineTo, PaintRectangle,
and TextPrint are essentially the only output methods that belong to Window,
although there is control over the fonts, line types, and so on that the
output methods would display.
Glockenspiel is developing a DrawObject hierarchy that would contain all of
the things that could be displayed within windows. In the meantime, you can
"kick down." This term means that you ask a CommonView object to give you its
Window or PM handle, so that you can write code directly to the environment.
DrawObjects will respond to a Draw message and will be owned by Windows.
Glockenspiel had intended to provide some samples, but pulled them from the
disk at the last minute. A late summer release of CommonView is expected to
include DrawObject, so by the time you read this article, some examples should
be available to use as models for your own classes.

DOODLE also shows us the basic structure of a CommonView program. It contains
a class, App. This class has a method, Start( ), that we need to provide.
App::Start( ) is the CommonView equivalent of main( ).
App::Start( ) first creates a window by declaring an instance of DoodleWind.
The system menu, a border, and a caption are added to the window. Finally, the
window is made visible, and CommonView's dispatcher [Exec( )] is turned on.
This is the basic structure to the CommonView initialization and startup
sequence. You will see this structure repeated in each CommonView application.


Conclusion


CommonView does make simple Windows and PM applications easier to use. The
compiled applications are very fast and small due to the fact that CommonView
is contained in a DDL (and, in fact, encourages the creation of other DDLS).
Unfortunately, with CommonView's current limitations -- including the lack of
printer support, EventContexts documentation, or graphics -- the initial
release of the product may prove unusable for many programmers. The major
upgrade expected in late summer 1989 is alleged to correct the vast majority
of these problems, and is anxiously awaited by many CommonView pioneers.


Availability


All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063, or call 800-356-2002 (from inside
Calif.) or 800-533-4372 (from outside Calif.). Please specify the issue number
and format (MS-DOS, Macintosh, Kaypro).


Product Information


CommonView, Version 1.2, ImageSoft Inc., 2 Haven Ave., Port Washington, NY
11050, 800-245-8840. System requirements: IBM PC AT or compatibles with at
least 640K of RAM, Microsoft Windows 2.03 Software Developer's Kit (SDK) or
Presentation Manager 1.0 SDK, Microsoft C 5.1 compiler. CommonView is bundled
with Glockenspiel's C++ for $495.

_A First Look At CommonView_
by Noel J. Bergman


[LISTING ONE]

#include <CommonVu.hxx>

class DoodleWind : public TopAppWindow
{
 Point LastPt;
protected:
 long far MouseDrag ( MouseEvt );
 long far MouseButtonDn ( MouseEvt );
};

void App::far Start()
{
 DoodleWind Doodle;
 Doodle.EnableSysMenu ();
 Doodle.EnableBorder ();
 Doodle.SetCaption ( "Doodle" );
 Doodle.Show ();
 Exec ();
}

long DoodleWind::far MouseButtonDn ( MouseEvt Evt )
{
 LastPt = Evt.Where ();
}

long DoodleWind::far MouseDrag ( MouseEvt Evt )
{
 MoveTo ( LastPt );
 LineTo ( LastPt = Evt.Where ());
}
































































October, 1989
PROGRAMMING PARADIGMS


Parker's Perceptions




Michael Swaine


I pulled up in front of Dave Parker's house with a trunkful of preconceptions,
the consequence of knowing too much history.
There are several reasons why research in neural networks went quiescent for
over a decade, but the most effective single cause for the big chill was
surely the book Perceptrons. In that book, Marvin Minsky and Seymour Papert
proved that the single-level Perceptron model could not compute certain basic
functions of their inputs. Then M&P went on to speculate, erroneously, that
research in multi-level Perceptrons and similar models would be equally
sterile. Anyone looking for funding for neural net research after Perceptrons
came out met a cold response.
There are also several reasons why neural network research has been heating up
of late, and why it has begun to produce some success stories, but the most
effective, single cause may have been the discovery -- by several people at
different times -- of an algorithm called "back propagation." "Backprop" shows
that Minsky and Papert were wrong about the sterility of multi-level nets. One
of the independent discoverers of this algorithm is Dave Parker. A lot of
money is now flowing into neural net research and development, and Parker is
one of the people who turned the faucet back on.
None of that money is flowing in Parker's direction. Today, he works out of
his modest Silicon Valley home, running a small graphics software company. His
company, Acrobits, hasn't made much of a splash yet, although it's a young
company and it seems to have a good product. My preconceptions said that
Parker wasn't profiting from the current interest in neural networks -- either
financially or in prestige -- as much as others were who had made less
significant contributions to the field. My preconceptions said that a
discoverer of the back-propagation algorithm, who was now writing graphics
software for PCs, had reason to be, if not bitter, at least a little cynical.
Parker threw a bucket of clarity on my preconceptions. When I gently prodded
him about why he wasn't working in neural nets, he explained patiently that,
fundamentally, this hot new research area involved nothing more earth-shaking
than some parallel implementations of familiar minimization algorithms. As for
the value of his own back-propagation algorithm, the day is past, he
explained, when you could make a living selling an algorithm. Of course, some
people in neural nets are making a very good living. But Parker doesn't seem
to care one way or the other. He enjoys what he's doing -- programming,
running his own business, satisfying customers rather than grantfunders -- and
likes the feeling that he's in control of where his money is coming from. He
continues to track research and developments in neural networks, but his own
present involvement in the area is limited to occasional lectures and classes,
where his emphasis is on demystifying the complex of algorithms,
architectures, and ideas that are collectively called neural networks.
After I met Dave's wife, who is building a laser in the garage, and their two
children, one walking, one crawling, we went into the office, which the
contractor no doubt thought was a spare bedroom. Here Parker started pulling
out neural net materials for me: articles, transparencies, a listing of a
Turbo Pascal program demonstrating back propagation. Since Parker hasn't
worked in the area in two years, the neural nets shelf of his bookcase was a
disorganized heap of papers and journals. It reminded me of the confusing
state of the literature on neural nets, and I mentioned this as we put cups of
instant coffee into the microwave.
Parker's response was to give me a dose of demystification.
Parker: It is confusing. People may make up new names and use all kinds of
jargon. There are whole strains of neural networks with their own jargon. Like
Brain State in a Box: This is a kind of parallel steepest descent, but it has
its own jargon because it was developed at one university and everybody there
talks the same. You read the literature and you think: How does this relate to
anything else?
Swaine: Is there some way to keep various models straight?
Parker: Yes. It's very simple if you view everything as a parallel
implementation of a minimization algorithm. Learning can be viewed as a
minimization problem, so you just ask, What is the minimization algorithm
being implemented?
Swaine: I see. The whole point of neural nets is that they learn, so you can
sort out the different models by asking how they learn. Let's try it. Hopfield
Net.
Parker: OK. You ask, What minimization algorithm does it implement? The answer
is, none. A Hopfield Net doesn't learn. Hopfield nets are useful; they can
tell us some things about the capabilities of neural nets, but they don't
learn anything.
Swaine: That was easy. Boltzmann Machine.
Parker: A Boltzmann Machine can learn. Since it learns, a Boltzmann Machine is
a (parallel) implementation of some sort of minimization algorithm. The
question is, what algorithm? [He picks up a pile of paper from a desk, drafts
for an Acrobits ad, turns them over, and begins drawing on the back of the top
sheet.] The simplest way to see it is to look at the one-dimensional case.
Plot all the weights on one axis, and performance on the other. [He draws two
axes and a sinusoidal curve, with two dips, one shallow and one deep.] The
weights are just all of the adjustable parameters in the network; in the
brain, these correspond to permeabilities and whatnot. You're plotting
everything you can adjust versus how well your brain -- or your neural network
-- does.
Swaine: So the algorithm is trying to find the set of weights that minimizes
the performance measure -- which is probably something like number of errors,
or distance from a goal, since it's something we want to minimize.
Parker: Right. One very crude algorithm is to pick a random set of weights,
see how you do, remember the set, then pick another random set of weights. If
this set is better, remember it; if it's worse, forget it. And one way of
implementing the randomization is to add noise to the weights you've chosen.
Swaine: I've read that the adding of noise is called simulated annealing, a
reference to a process in metallurgy. Is this a good algorithm?
Parker: Well, it's been proven that if the distribution of the noise slowly
gets smaller, the probability will go to one that you will find the global
minimum. [He points to the deep dip in the curve.] It takes a long time,
though. This is just a random search, so a Boltzmann Machine, if you look at
it as a minimization algorithm, is a parallel implementation of a random
search.
Swaine: Which is about as crude as you can get.
Parker: But it has the nice property that if the noise you add gets less and
less noisy, the probability goes to one that you'll find the global minimum.
The idea is that when you first start, you search all over the surface, and
then when you narrow down to a certain area you gradually add less and less
noise because the probability is higher that you're [near the global minimum].
Swaine: Here's an easy one: Back propagation.
Parker: Back propagation is "parallel steepest descent." Steepest descent is a
well-known algorithm. Rather than picking a random place, you just keep going
downhill. That won't necessarily get you out of a local minimum. [He points to
the shallow dip in the curve.] But there are ways around that.
Swaine: I'd like to hear about the ways around.
Parker: Oh, that's a neat one. [He pulls out another piece of paper and
redraws the graph, this time with a diagonal line representing a third
dimension at right angles to the performance and weight dimensions.] I should
give credit for this: This picture was first drawn for me by mathematician Dan
Asimov. This third dimension represents the inputs to the system, the things
you have no control over. For instance, in your brain you supposedly have
control over protein levels and that kind of stuff, but you have no control
over the fact that a car is bearing down on you. The inputs are things that
you have no control over during the learning process, the data that you're
getting from the external world; the weights are the stuff that's inside you;
and the performance is [the output] that we measure. In any particular task
you're working on you're only in a small area of all the possible inputs the
system could be getting. There are entirely different inputs for, say, scuba
diving, versus learning physics. So if you're studying physics, as my wife is,
you can look at the performance surface in that narrow band [of inputs]. Now
if you're studying physics and using a steepest descent algorithm and you're
here [he points to the shallow dip in the curve], you're in big trouble,
because you're always going to be trying to go downhill, which will always
take you to the local minimum, and you're never going to get over the hump.
You're stuck in a rut. There's a human analogy: People are often said to be
stuck in a rut, doing a lot of work but not getting anywhere. Now, what a
human will often find helpful in that situation is to go and do something
different for a while.
Swaine: Get up and go for a walk.
Parker: Yeah, go for a walk, play tennis, go hiking.... And what they're doing
is going to a different subspace of their inputs. And there the performance
surface may look radically different: You don't expect that a tennis pro, just
by playing tennis all the time, will become a great physicist, or vice versa.
So what happens when the physicist goes off and plays tennis, since the
optimum tennis weights are not the optimum physics weights, is that the
weights get moved in some direction. And if the tasks are uncorrelated, that
direction will be random. So you go off and do something different and when
you come back to your original task, if you're lucky, you'll be working on it
for a while and you'll say, Hey, why didn't I think of that before?
Swaine: So you adjust the inputs, and the weights are adjusted as a
consequence, rather than directly tweaking the weights randomly, as the
Boltzmann Machine does.
Parker: By exploring different parts of your input space you can give your
weights random pushes in different directions and hopefully be able to find
the global minimum.
Swaine: Why is that approach better than a Boltzmann Machine?
Parker: They're good at different things. If you have access to the weights,
and all you're doing is learning physics or doing circuit board layout, it is
much quicker to just kick all the weights randomly, stay in the task subspace,
do a Boltzmann Machine followed by steepest descent, and repeat. But people
don't have access directly to their weights, so rather than kicking the
weights directly, you can go to an uncorrelated subspace of your inputs, and
then come back to work on your task again. It's a slower process, but you
don't need access to your weights.
Swaine: We've all had that experience of getting up from the keyboard when
we're stuck on a problem and going off to do something seemingly completely
unrelated, then finding when we come back to the problem that the solution is
suddenly obvious. Some people would claim, I think, that the intervening
activity is not really uncorrelated, that there is some deep connection
between this particular physics problem and, say, tennis.
Parker: And when it works, they'd be right. [It would be a great aid to
teaching physics] if you knew that, when you were in the physics subspace and
were at a certain local minimum, a good thing to do would be to go play the
harpsichord. But we don't generally know that. [But that's not the only
advantage of this approach over the Boltzmann Machine.] To do the Boltzmann
Machine you need to keep around two sets of weights: Your best and the
current. Using this approach you only need your current ones, but it does take
longer.
Swaine: One begins to get the message from all these algorithms that learning
simply takes a long time.
Parker: Teaching people takes years and years. So anyway, even if you use
parallel steepest descent -- which isn't guaranteed to find the global minimum
-- you can still find it, by doing this kind of thing.
Swaine: So what's the most powerful neural net model? What's the best we can
do today?
Parker: Is there a more powerful minimization algorithm than steepest descent?
Yes, there is. It's been around for a long time, and it's called "Newton's
Method." Steepest descent is based on the first derivative of the performance
surface: It looks at the first derivative and tries to follow it. Newton's
Method uses the second derivative, so it's based on more information.
Swaine: How does that work?
Parker: A handy analogy is from skiing. Imagine that you're on a steep
overhang and the obvious way to get back to the lodge is to go straight down
the steep slope to the valley and then down the gentle slope of the valley to
the lodge. Steepest descent is like equivalent to doing nothing but sitting on
your skis. You go very quickly down the slope to the valley, but then you get
to the valley and you slow down, and it gets very inefficient. What expert
skiers will do is cut across the slope, by putting pressure on one of their
skis, and that's exactly what Newton's Method does. The parameters that
control Newton's Method are equivalent to the pressure you put on your ski. If
you just sit on your skis, your speed is proportional to the steepness of the
slope, which is true of steepest descent. But with Newton's Method, you move
at constant velocity.
Swaine: But at some cost. What does using Newton's Method cost you?
Parker: Newton's Method is trickier to implement and more computationally
intensive.
Swaine: It sounds like there is no one best algorithm for all possible
situations, which I suppose is not surprising.
Parker: Newton's Method is a superior algorithm, so if you can afford the
computational overhead, it pays off for you.
Swaine: Where exactly does it pay off?
Parker: One way in which it pays off is this. Steepest descent takes a curved
path, and Newton's Method takes a straight path. The best path is the straight
path.
Swaine: Obviously. But you mean something more than the fact that a straight
line is the shortest distance between two points.
Parker: Right. The disadvantage of the curved path is, if you follow a curved
path and then stop learning halfway through, you may be no closer to where you
want to go, and you've moved away from where you used to be. So you've
forgotten old but useful information without having learned very much new
information. Whereas if you follow a straight line and stop, you're closer to
where you want to go and as close to where you used to be as you can be.

Swaine: So if you can afford it, Newton's Method is best. Sort of the
high-priced algorithm. It occurs to me that, even if we can't pick an absolute
best algorithm, nature has already made a choice. Too bad we don't know -- yet
-- what algorithm nature picked for human learning.
Parker: Yes; you can think of almost any example of human learning as
minimization. If you accept the hypothesis that learning is exactly
minimization, then people must be implementing some minimization algorithm,
and since Newton's Method is the more powerful of the two, one would hope that
it was a form of Newton's Method.
Swaine: Really? I was finding your description of modified backprop pretty
convincing in terms of my experience in learning things.
Parker: I would think it was a back-propagation form of Newton's Method.
Swaine: Nature has probably had time to try out many algorithms. But if
Newton's Method is the most powerful minimization algorithm we know, and if
it's actually feasible to implement in a human brain....
Parker: The performance difference is great enough that the other guys would
have died out.
The listing that accompanies this month's column is Parker's demonstration of
back propagation, written in Turbo Pascal. It's not intended to be an
efficient implementation, and it won't teach you much if you just run it and
examine the output. The code, though, is well documented and is intended to be
read. Here are some aids to reading the listing.
The program simulates a neural net (simulates because it is purely software
and is not a parallel implementation) of four cells; four neurons. Each neuron
is a simple computational unit. It has two input channels, a channel for
receiving the error signal and one output channel. The inputs are modified by
a weight associated with each input. The neuron computes its output based on
the inputs and the weights; in this implementation the computation is a
sigmoid function, but other computations are possible.
The neurons are connected in layers, with the outputs of neurons in one layer
feeding the inputs of neurons in the next. The original inputs are treated as
the outputs of a nonexistent 0-th level, and the outputs of the top level are
compared with the desired outputs to generate error signals. The error signals
propagate back through the net, causing the weights in the neurons to be
adjusted. The idea is that "good" paths through the net will get weighted more
and more heavily and "bad" paths will get lower and lower weights, so that
eventually only the "good" paths will be taken.
The demo neural net (Listing One) learns a simple task. Given one of four
input patterns, (0, 0), (1, 0), (0, 1), (1, 1), the net is to produce the
appropriate one of the possible output patterns, (0, 0), (0, 1), (0, 1), (1,
1). Described in this way, it's a pretty meaningless task, but it makes a
little more sense when expressed in functional terms: The two values in the
output pattern are respectively the logical AND and the logical OR of the two
input values.
So the operation of the program is:
Compute the inputs and corresponding outputs
Propagate the inputs forward through the net
Compare the actual output to the desired output and compute the error
Propagate the error signals back through the net and update the
Weights, and check to see if the classification has been learned.
The last step involves examining the sum of squares of the error signals and
testing for convergence. The input and output values are not really 0s and 1s
as stated above, but numbers close to 0 and 1, and the program quits when the
classification is learned to within a preset degree of tolerance -- when it
gets close enough to 0s and 1s.
Training a neural net of this sort requires that desired output for each pass
be known, so that the error signals can propagate back through the net to
cause the weights to be adjusted. That's the back propagation, and what the
net is learning to do is to minimize these error signals. In this demo
program, the computation of error is handled automatically by having an array
of the desired outputs for each set of inputs. That might make the whole
enterprise seem pointless, and in the case of the demo, it is: If you examine
the routine that computes the inputs and desired outputs, you'll see that we
have already calculated the right answers before the network even started
running.
In a real neural net implementation, computing the error could be more
complicated, and in some cases might require that a person evaluate the
response. Or, more interestingly, we might be able to write a routine that
recognizes and characterizes errors when it sees them, even though we couldn't
write an algorithm to produce correct output. In chess, for example, we can't
yet write a program that plays perfectly, but we can write programs that do a
perfect job of recognizing checkmate, and we can implement various
position-evaluation heuristics. These are cases in which we can generate error
signals to train the net even though we don't know how to solve the problem.
And if our error signals are good, and if our algorithm is one that doesn't
get stuck in a rut, our network, left to run uninterruptedly and to learn from
its mistakes, will eventually learn to play perfect chess.
But with current hardware, probably not in our lifetimes.


Availability


All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063, or call 800-356-2002 (from inside
Calif.) or 800-533-4372 (from outside Calif.). Please specify the issue number
and format (MS-DOS, Macintosh, Kaypro).

_Programming Paradigms_
by Michael Swaine



[LISTING ONE]

Program BackPropagationDemo;

Const NumOfRows = 2; (* Number of rows of cells. *)
 NumOfCols = 2; (* Number of columns of cells. *)
 LearningRate = 0.25; (* Learning rate. *)
 Criteria = 0.005; (* Convergence criteria. *)
 Zero = 0.05; (* Anything below 0.05 counts as zero. *)
 One = 0.95; (* Anything above 0.95 counts as one. *)

Type CellRecord = Record
 Output : Real; (* Output of the current cell. *)
 Error : Real; (* Error signal for the current cell. *)
 Weights: Array[0..NumOfCols] Of Real; (* Weights in cell. *)
 End;

Var CellArray : Array[0..NumOfRows,0..NumOfCols] Of CellRecord; (* Cells. *)
 Inputs : Array[1..NumOfCols] Of Real; (* Input signals. *)
 DesiredOutputs: Array[1..NumOfCols] Of Real; (* Desired output signals. *)

Procedure CalculateInputsAndOutputs( Iteration: Integer );
Var I: Integer;
Begin (* Calculate the inputs and desired outputs for the current iteration.
*)
 (* The inputs cycle through the 4 patterns (0.05,0.05), (0.95,0.05), *)
 (* (0.05,0.95), (0.95,0.95). The corresponding desired outputs are *)
 (* (0.05,0.05), (0.05,0.95), (0.05,0.95), (0.95,0.05). The first *)

 (* desired output is the logical AND of the inputs, and the second *)
 (* desired output is the logical XOR. *)
If (Iteration Mod 2) = 1 Then Inputs[1] := One Else Inputs[1] := Zero;
If (Iteration Mod 4) > 1 Then Inputs[2] := One Else Inputs[2] := Zero;
If (Inputs[1] > 0.5) And (Inputs[2] > 0.5) Then DesiredOutputs[1] := One
Else DesiredOutputs[1] := Zero;
If (Inputs[1] > 0.5) Xor (Inputs[2] > 0.5) Then DesiredOutputs[2] := One
Else DesiredOutputs[2] := Zero;
End;

Procedure UpdateCellOnForwardPass( Row, Column: Integer );
Var J : Integer;
 Sum: Real;
Begin (* Calculate the output of the cell at the specified row and column. *)
With CellArray[Row,Column] Do
 Begin
 Sum := 0.0; (* Clear weighted sum of inputs. *)
 For J := 0 To NumOfCols Do (* Form weighted sum of inputs. *)
 Sum := Sum + Weights[J]*CellArray[Row-1,J].Output;
 Output := 1.0/(1.0+Exp(-Sum)); (* Calculate output of cell. This *)
 (* is called a sigmoid function. *)
 Error := 0.0; (* Clear error for backward pass. *)
 End;
End;

Procedure UpdateCellOnBackwardPass( Row, Column: Integer );
Var J: Integer;
Begin (* Calculate error signals and update weights on the backward pass. *)
With CellArray[Row,Column] Do
 Begin
 For J := 1 To NumOfCols Do (* Back propagate the error to the cells *)
 CellArray[Row-1,J].Error := (* below the current cell. *)
 CellArray[Row-1,J].Error+Error*Output*(1.0-Output)*Weights[J];
 For J := 0 To NumOfCols Do (* Update the weights in the current cell. *)
 Weights[J] :=
 Weights[J] +
 LearningRate*Error*Output*(1.0-Output)*CellArray[Row-1,J].Output;
 End;
End;

Var I, J, K : Integer; (* I loops over rows, J loops over columns,*)
 (* and K loops over weights. *)
 ConvergedIterations: Integer; (* Network must remain converged for four *)
 (* iterations (one for each input pattern).*)
 Iteration : Integer; (* Total number of iterations so far. *)
 ErrorSquared : Real; (* Error squared for current iteration. *)

Begin
ClrScr; (* Initialize the screen. *)
Writeln('Iteration Inputs Desired Outputs Actual Outputs');
Iteration := 0; (* Start at iteration 0. *)
ConvergedIterations := 0; (* The network hasn't converged yet. *)
For I := 1 To NumOfRows Do (* Initialize the weights to small random
numbers.*)
 For J := 1 To NumOfCols Do
 For K := 0 To NumOfCols Do
 CellArray[I,J].Weights[K] := 0.2*Random-0.1;
For I := 0 To NumOfRows Do (* Initialize outputs of dummy constant cells. *)
 CellArray[I,0].Output := One;
Repeat

 CalculateInputsAndOutputs(Iteration);
 For J := 1 To NumOfCols Do (* Copy inputs to dummy input cells. *)
 CellArray[0,J].Output := Inputs[J];
 For I := 1 To NumOfRows Do (* Propagate inputs forward through network. *)
 For J := 1 To NumOfCols Do
 UpdateCellOnForwardPass(I,J);
 For J := 1 To NumOfCols Do (* Calculate error signals. *)
 CellArray[NumOfRows,J].Error :=
 DesiredOutputs[J]-CellArray[NumOfRows,J].Output;
 For I := NumOfRows Downto 1 Do (* Propagate errors backward through *)
 For J := 1 To NumOfCols Do (* network, and update weights. *)
 UpdateCellOnBackwardPass(I,J);
 ErrorSquared := 0.0; (* Clear error squared. *)
 For J := 1 To NumOfCols Do (* Calculate error squared. *)
 ErrorSquared := ErrorSquared + Sqr(CellArray[NumOfRows,J].Error);
 If ErrorSquared < Criteria Then (* If network has converged, increment *)
 ConvergedIterations := ConvergedIterations + 1 (* convergence *)
 Else ConvergedIterations := 0; (* count, else clear convergence count. *)
 If (Iteration Mod 100) < 4 Then (* Every 100 iterations, write out *)
 Begin (* information on the 4 patterns. *)
 If (Iteration Mod 100) = 0 Then GotoXY(1,2);
 Write(' ',Iteration:5,' '); (* Write iteration number. *)
 For J := 1 To NumOfCols Do (* Write out input pattern. *)
 Write(Inputs[J]:4:2,' ');
 Write(' ');
 For J := 1 To NumOfCols Do (* Write out desired outputs. *)
 Write(DesiredOutputs[J]:4:2,' ');
 Write(' ');
 For J := 1 To NumOfCols Do (* Write out actual outputs. *)
 Write(CellArray[NumOfRows,J].Output:4:2,' ');
 Writeln;
 End;
 Iteration := Iteration + 1; (* Increment iteration count *)
Until (ConvergedIterations = 4) Or (Iteration = 32767);
 (* Stop when the network has converged on all 4 input patterns, or when*)
 (* we are about to get integer overflow. *)
If ConvergedIterations <> 4 (* Write a final message. *)
Then Writeln('Network didn''t converge')
Else Writeln('Network has converged to within criteria');
End.






















October, 1989
C PROGRAMMING


More C++ and a Step Up to ANSI C




Al Stevens


Our collection of C++ tools began last month with a simple window manager. The
first class we built was a Window that pops up when it is declared from within
a program and pops down when it goes out of scope. The Window methods include
the following:
Overloaded << operators that add a character, a line, or a block of text to
the Window
Page and scroll through the Window
Hide the Window and restore it to the screen
Change the Window's display colors and default tab stops
Position and read the Window's cursor
Clear text from the entire Window, from the current cursor position to the end
of the current line, and to the end of the window
The Window class became the base class for three simple derived classes, the
YesNo window, the Notice window, and the Error window. Those classes
illustrated class inheritance as supported by C++. This month we will look
more closely at inheritance with a popdown menu class that is derived from the
Window. Then we will introduce a string data type that resembles the strings
of Basic.


Menu Classes


To continue our exploration of C++, we will build classes that implement two
kinds of menus: The sliding bar menu similar to those used by programs such as
Lotus 1-2-3, and the popdown menu similar to the ones that SideKick uses. The
sliding bar menu class, named "SlideBar," is its own independent class. The
popdown menu class, called "PopdownMenu," is derived from the Window class.
These menus are similar to ones we developed in traditional C last year for
the TWRP and SMALLCOM projects.
A class is a new data type, described by the programmer to extend the
language. When we built the Window and its derived classes, we effectively
added a data type. with that addition, our C++ programs can have chars, ints,
longs, floats, doubles, structs, unions, Windows, Notices, Errors, and YesNos.
Next we will add SlideBars and PopdownMenus. A derived class inherits all the
characteristics of the base class from which it is derived and adds its own
private and public parts.
Our SlideBar and PopdownMenu classes perform operations similar to one another
but with different menu formats. When you declare either class, a menu is
created and displayed, and the user is prompted to make a selection. Depending
on the user's selection, an applications function is selected from ones that
you associate with the selections when you declare the class. Let's look first
at the SlideBar class.
Sliding Bar Menus -- When you declare a SlideBar variable, you specify the
screen line where the menu will appear, the text of the selections, the first
selection to be highlighted by a selection cursor, and pointers to the
functions associated with the selections. As soon as you declare the SlideBar,
its constructor function displays the menu and prompts the user to make a
selection. The user can move the selection cursor back and forth by pressing
the right and left arrow keys and can make a selection by pressing the Enter
key when the cursor is on the desired selection. The user can also make a
selection by pressing the first letter of the selection's name. This
convention requires that you assign selection names with unique first letters
when you design the menu. When the user makes a selection, the associated
function executes.
When you declare one of these menu classes, your program remains in the member
functions and in the applications functions associated with selections until
the constructor function returns. The constructor function calls the private
dispatch member function to manage user selections and dispatch your
applications functions. The SlideBar class has a terminate member function
that the applications programs call to tell the dispatch function to terminate
menu processing. The statement in your program that follows the SlideBar
declaration will then execute, but the menu will remain visible until the
SlideBar goes out of scope.
A dispatched applications function can use the current_selection member
function to determine which vertical selection on the menu caused its
dispatch.
Pop Down Menus -- Next let's consider the PopdownMenu class. It is similar in
operation to the SlideBar class, but the menu takes the form of a window and
is, therefore, derived from the Window class. When you declare the
PopdownMenu, you specify the column and row coordinates where the upper left
corner of the PopdownMenu window displays. The other initialization parameters
are the same as those of the SlideBar menu, and the operation is similar. With
the PopdownMenu, however, the user moves the selection cursor up and down
rather than to the right and left.
Besides the terminate and current_selection member functions, which work like
those of the SlideBar class, the PopdownMenu class includes additional
features. A PopdownMenu can have selections that are selectively disabled.
This means they are displayed but not available for selection. They have a
unique color scheme, and the selection cursor passes over them when the user
moves it up and down. The disable_selection and enable_selection public member
functions allow the applications code to disable and enable a specified menu
selection.
PopdownMenus also support toggle selections, ones that change a binary state
switch but do not have associated applications member functions to dispatch.
They display a check mark next to their name if the toggle is on and no check
mark if the toggle is off. The test_toggle public member function allows an
application to test the current value of a selection's toggle.
The definitions of the SlideBar and PopdownMenu classes appear in Listing One
menus.h. A program that will use either of these classes will include this
file. Along with the class definitions are definitions of the color schemes
for menus. The Window class from last month provides for the initialization
and changing of a Window's colors. The menu classes, however, assume that a
program uses a consistent color scheme for all menus and does not require you
to identify the menu colors every time you declare a menu. Listing Two is
menus.c, the code that implements the classes. To use them, you must link your
program with the object files that you compile from these files. To use the
PopdownMenu class, you will need the window.h and window.c files from last
month. All the programs in this and last month's columns compile with the
Zortech C++ compiler, Version 1.07.
Listings Three and Four are demoslid.c and demopop.c. These programs
demonstrate the use of the two menu classes. Demoslid.c defines a SlideBar
menu with four selections. Each of the selections executes a function in
demoslid.c. The first three of these simulated applications functions declares
a Window that the function uses to identify itself. After the user presses a
key, the function returns. The fourth function is the Quit selection. It uses
a YesNo class (defined in window.h and discussed last month) to ask the user
to verify the quit command. If the user says "yes," the function calls the
terminate member function to tell the menu to quit.
Demopop.c is similar to demoslid.c, but it demonstrates the additional
features available in the PopdownMenu class. It declares a PopdownMenu with
five selections. These selections behave in ways that suggest the File menu of
an editor program. There are selections to load a file, save a file, and
declare a new file. There is a toggle selection named Option. The last
selection is the Quit selection.
The Save selection is initially set to be a disabled selection. The minus sign
in the text name "-Save" identifies it as such. Whenever you use the Load or
New selections, their dispatched functions call the enable_selection public
member function to enable the Save selection. When you then use the Save
selection, its dispatched function calls the disable_selection function to
disable itself.
The Option selection is a toggle. It is implicitly defined as such by the NULL
function pointer that the demopop.c program specified for the Option
selection's associated applications function. When the user chooses the Option
selection, the class automatically inverts the toggle setting. A toggle
setting is represented by the appearance or absence of the check mark symbol
('\xfb') as the last character of the selection's name. We use the Option
toggle in the Quit function to see if we need to use a YesNo class to verify
the Quit request. If the toggle is on, we do not ask the user for
verification. This usage serves to illustrate the mechanics of toggle
selections.
By combining the SlideBar class with a series of PopdownMenu classes, you
could build a menu system where the sliding bar is at the top of the screen
with popdown menus under each sliding bar selection. This is the kind of menu
system used by many programs. In my development of the two menu classes, I
attempted to go the next logical step and develop those traditional sliding
bar/popdown menus. The wall I ran into was either the limit of Zortech C++ or
my own inexperience with the C++ language. A two-dimensional menu driver needs
pointers to arrays of function pointers, or something similar. I was not able
to get these constructs working within the realm of the new operator or as
parameters to overloaded constructor functions. It is also not clear to me how
the variable argument list feature of C++ and ANSI C fit into the overloaded
function construct. These issues are typical of the ones you and I will
encounter as we try to use C++ in ways that our imaginations move us, and,
when and if I solve them, I will share the solutions.


The String Class


When I migrated from Basic to C, I mourned the loss of the string variable.
K&R reassured me that character arrays and standard functions using character
pointers would, with care, serve the same purpose, but I missed the old way,
wanting ever since to be able to say this in a C program:
if (username == "Wynton")
 username = username + " Marsalis";
or better yet:
username += " Marsalis";
With C, we must use the strcat function to perform such a concatenation, and
the receiving string must be long enough to receive the added value. There are
other useful string operations in Basic, and I've been wanting them in C for a
long time.
Well, want no more. C++ brings that capability to the C language. It does,
that is, if you roll your own string class, and that is just what we are about
to do.
Listing Five is strings.h, the header file that describes the new class, named
"string." Listing Six is strings.c, the code that implements the string class.
The string has one private part, a character pointer. When you declare the
string, the constructor function initializes the pointer. All operations on
the string use this pointer. There are four constructor functions for the
string class, supporting the four different ways you can declare a string and
illustrating the C++ technique for overloading constructor functions.
C++ lets you overload functions. This means that several functions can have
the same name but different parameter types. The language translator decides
which function you are calling based on the parameters you are passing. (If C
had this feature, we would not need a strcpy function and a strncpy function,
for example. One function could handle both operations.) Function overloading
does, however, mandate the use of prototypes, and that is a good requirement.
The four ways of declaring a string are shown here:
string name 1; // null string
string name2("George"); // character pointer

string name3(name2); // another string
string name4(80); // length
Each of these declarations will establish a string with a pointer to the
appropriate character array. The name 1 string will contain a pointer to a
null string. The other three strings will contain pointers to character
arrays. All four constructors use string space taken from the free store (the
C++ heap) with the C++ new operator. Each constructor makes a new copy of the
string value rather than simply pointing to the initializing value. The name4
string points to an array of 81 characters all with a zero value.
String Assignments -- Once you declare a string, there are a number of
operations you can perform on it. First let's consider the assignment
operator. Once you establish a string variable, you can assign either a
character array or another string to it as shown here.
string oldstring; // declare two strings
string newstring;
oldstring = "Hello, Dolly", // assign an array
newstring = oldstring; // assign a string
These two assignments are achieved with the operator= overloaded functions
that have a character pointer and a string as their parameters. In C++ you can
overload the unary and binary C operators to work with classes in ways you
design. You cannot change the way the operators work with standard C data
types, you cannot create operators that do not exist in C, you cannot use
unary operators as binary ones, and so forth. Your use of operator overloading
must be done in ways that make sense to the language translator's lexical scan
and parser. We'll look at more operator over-loading later.
The stradr Function -- The stradr public member function returns the address
of the string. It is defined as a const char * function, which means that it
returns a pointer to something that cannot be modified. The intention here is
to prevent an applications program from changing the value of a string through
the stradr function. All changes to strings should be made with the string
operations described soon. This intention is not always realized. Zortech C++
does not disallow you from assigning the value returned from the function to a
regular character pointer. It does prevent you from using the function call in
an expression that would change the string. For example, the following code is
OK according to Zortech:
 string myname("Joe")
 char *cp = myname.stradr();
 *cp = 'M';
The compiler should warn you that the second statement is assigning a
pointer-to-const to a regular pointer, but it does not. Experiments with Turbo
C and Microsoft C reveal that they do issue such warnings (in a C context, of
course). See the ANSI Corner later for more discussion on this circumstance as
it relates to C, not C++.
Zortech will issue an error if you try this code:
 string myname("Joe");
 *myname.stradr() = 'M';
The right, left, and mid Functions -- The string class includes three public
member functions designed to emulate the RIGHT$, LEFT$, and MID$ substring
functions of Basic. These functions return pieces of strings as new strings.
For example:
 string name("George Kingfish Stevens");
 string firstname;
 string middlename;
 string lastname;
 firstname = name.left(6);
 middlename = name.mid(8, 8);
 lastname = name.right(7);
This code assigns to the three null strings the three parts of the originally
initialized name.
String Concatenation -- The string class includes several ways that you can
concatenate strings. Assume that you have strings named newname, name and
lastname. These are the ways that you can concatenate strings:
 newname = name + "Smith";
 newname = name + lastname;
 name = name + "Smith";
 name = name + lastname;
 name += "Smith";
 name += lastname;
String concatenation does not require you to assure enough space for the added
value. String sizes grow according to their needs.


Relational Operators


Our new string class allows us to make relational tests between strings and
between strings and character arrays. You can make the following tests:
 if (name == "Sam")
 if (name < othername)
and any other combination of two strings or one string and a character array
where the relational operator is one of these:
==, !=, <, >, <=, >=
The only restriction is that the left side of the test must be a string class
rather than an array.
String Subscripts -- You can use the [] subscript operators to read the
individual character values of a string in ways similar to how you work with
regular C character arrays. For example:
 string name("Otis");
 char ch = name[2];
The ch character variable will receive the 'i'from the string. You cannot,
however, do this:
 name[2] = 'e'; // invalid statement
because the value returned by the overloaded [] operator is not a C lvalue.
You can, however, use the overloaded + operator to form an lvalue. The
following operations are valid:
 ch = *(name+2);
 *(name+2) = 'e';
The string class represents what I believe to be the real potential for C++,
its ability to extend the C language with reusable, generic data types.
Perhaps you have no desire to make C look more like Basic with our string
class, but the exercise reveals the possibilities that class definition adds
to the C language.
The C++ language is still without the large user base that would drive us
toward standard conventions. The ANSI C committee has decided not to undertake
the standardization of C++. Therefore, not until Borland and Microsoft
introduce C++ compilers, complete with integrated development environments and
hot-shot debuggers, can PC developers get serious about it. We can only hope
that it happens in the not-too-distant future, but there have been no
announcements. Until that time, C++ is still a wonderful study in what
programming ought to be, and I encourage you to get into it.


The ANSI Corner: const and volatile



The ANSI X3J11 committee submitted their draft proposed standard for the C
language to ANSI for approval last spring, but some snags developed, mostly
procedural or bureaucratic. Those should be cleared up soon, and a true ANSI C
standard should exist, perhaps by the time you read this column.
The draft standard document is not exactly fireside reading. Eventually it
will be the final authority on how C should work, but it is neither a tutorial
text nor an easily understood reference document. Compiler developers will
study it in great detail trying to conform. We, the consumers, must trust our
compiler writers to have correctly interpreted and implemented the standard
language.
This small section of the DDJ "C Programming" column is a new monthly addition
that will address some of the features that the ANSI C standard adds to the C
language. Most of these features are already implemented in the C compilers
you use now because the compiler writers have been closely following the
development of the proposed standard. Each month we will look at another part
of the ANSI standard.
The const Type Qualifier -- A const variable is one that your program cannot
modify with an assignment or by incrementing or decrementing. You can declare
a variable to be a const in one of these ways:
1. const int i1;
2. int const i2;
3. const int *ip1;
4. int const *ip2;
5. int *const ip3;
6. const int *const ip4;
7. int const *const ip5;
The first two forms declare integers that cannot be changed. The only correct
way to put a value into this integer or any other const-qualified variable is
with initialization as shown here.
 const int i = 123;
The third and fourth forms are pointers to integers where the integers cannot
be changed. The fifth form is a pointer to an integer that can be changed, but
the pointer itself cannot be changed. The sixth and seventh forms are pointers
that cannot be changed and that point to integers that also cannot be changed.
The const type qualifier is not perfect. There are exceptions to the
protection it will provide, and the ANSI document abdicates responsibility for
most of them, saying, "If an attempt is made to modify an object defined with
a const-qualified type through use of an lvalue with non-const-qualified type,
the behavior is undefined." There must have been a good reason for that. The
original K&R does not include the const keyword, so the reason cannot be to
protect existing, pre-ANSI code.
What does all this mean? What, for example, should happen if you were to
initialize a non-const pointer with the address of a const variable as shown
here?
 const int i;
 int *ip = &i;
Some compilers courageously give a warning for this code. It should probably
be an error, but the wimpy language of the ANSI spec does not provide for
that. Call it an error and you do not conform, I think.
Suppose we pass a pointer of the third form above to a function that is
expecting (by virtue of its prototype and declaration) a non-const pointer.
Consider this:
 const char *ch = "123";
 strcpy(ch, "456");
Once again, some compilers issue warnings. This code should, however, be an
error, because the strcpy function is defined to expect a normal pointer as
the first parameter, and will, in fact change wherever that pointer points. If
you ignore the warning, or use a compiler that does not issue one, the const
keyword is worthless in this context.
ANSI specifies "undefined" behavior when const is used along with a function
declaration, meaning that the following code may or may not give the desired
results:
 const char *myaddr(void);
The ANSI position (or nonposition) would seem to leave open what should happen
when one source code file declares an external variable as const and another
declares the same variable as non-const. Turbo C, for example, offers no
protection for a const variable if another source file leaves out the
const-type qualifier. A more appropriate behavior would depend on a linker
that knows the type qualification of external variables.
According to ANSI, if a struct declaration includes the const-type qualifier,
the structure members are const. Microsoft C conforms to this rule, but Turbo
C does not. ANSI's expression of this rule is vague and implicit and probably
subject to interpretation.
If an array is const-qualified, that means its elements are const.
The volatile Type Qualifier -- A volatile variable is one that might be
modified from an external, asynchronous source, such as an interrupt service
routine. Its purpose is to allow compilers to bypass some optimization when
handling the variable. For example, you might have a global variable that your
program is working with. A hardware interrupt occurs, and the interrupt
service routine modifies that variable. If your compiler is unaware that such
modifications could occur, the compiled code might be saving the variable in a
register or on the stack rather than in its designated memory location. If you
qualify the variable as a volatile, the compiler then knows to always keep the
value in a location available to external influence. This could have tricky
consequences. It might be necessary for the compiled code to disable
interrupts whenever it is working with the variable, for example, which could
introduce timing problems.
Obviously, the volatile type qualifier has no meaning when applied to an
automatic variable. Static variables declared inside functions could be
modified by interrupts if the interrupted function is called recursively from
the interrupt service routine, so they are subject to the benefits of the
volatile qualifier, but automatic variables are usually on the stack or in
registers and each recursive call to a function has its own copies of the
automatic variables.
A const, volatile Variable -- ANSI provides for a variable that has both the
const and the volatile type qualifiers. If you code this statement:
 extern const volatile int x;
the variable exists somewhere else in a way that it can be modified by an
interrupt, but the local function cannot modify it.


Availability


All source code for articles in this issue is available on a single disk. To
order, send $14.95 (Calif. residents add sales tax) to Dr. Dobb's Journal, 501
Galveston Dr., Redwood City, CA 94063, or call 800-356-2002 (from inside
Calif.) or 800-533-4372 (from outside Calif.). Please specify the issue number
and format (MS-DOS, Macintosh, Kaypro).

_C Programming_
by Al Stevens


[LISTING ONE]


// ------------ menus.h

#ifndef MENUS
#define MENUS

#include "window.h"

#define MAXSELECTIONS 12
#define CHECK '\xfb' // IBM Graphics character set check mark


#define MENUFG CYAN
#define MENUBG BLUE
#define SELECTFG BLACK
#define SELECTBG WHITE
#define DISABLEFG LIGHTGRAY
#define DISABLEBG BLUE

//
// SLIDING BAR MENUS
//
class SlideBar {
 int row; // menu screen row
 void (**mfunc)(SlideBar&); // selection functions
 char **mtext; // selection titles
 int selections; // number of selections
 int quitflag; // flag for appl to say quit
 unsigned *msave; // save area for menu bar
 int selection; // selection position
 int dispatch(int sel, int titlewidth);
public:
 SlideBar(int line, char **text, int sel,
 void (**func)(SlideBar&));
 ~SlideBar();
 void terminate(void)
 { quitflag = 1; }
 int current_selection(void)
 { return selection; }
};

//
// POPDOWN MENUS
//
class PopdownMenu : Window {
 int selections; // number of selections
 void (**mfunc)(PopdownMenu&); // selection functions
 char **text; // address of menu text
 int quitflag; // flag for appl to say quit
 int selection; // current selection position
 // --- private methods
 int get_selection(int sel);
 int dispatch(int sel);
 int menuheight(char **text);
 int menuwidth(char **text);
 void operator<<(char **text);
public:
 PopdownMenu(unsigned left, unsigned top,
 char **text, int sel, void (**func)(PopdownMenu&));
 ~PopdownMenu(){};
 void terminate(void)
 { quitflag = 1; }
 void disable_selection(int sel)
 { *text[sel-1] = '-'; }
 void enable_selection(int sel)
 { *text[sel-1] = ' '; }
 int test_toggle(int selection);
 int current_selection(void)
 { return selection; }
};


#endif







[LISTING TWO]

// ----------- menus.c

#include <stddef.h>
#include <string.h>
#include <stdio.h>
#include <ctype.h>
#include <conio.h>
#include "menus.h"
#include "console.h"

static void select(int row,int sel, char *ttl,int set,int wd);
#define ON 1
#define OFF 0

//
// SLIDING BAR MENUS
//

// ----------- constructor for a sliding menu bar
SlideBar::SlideBar(int line, char **text, int sel,
 void (**func)(SlideBar&))
{
 savecursor();
 hidecursor();
 initconsole();
 // ------ menu variables
 quitflag = 0;
 mfunc = func;
 mtext = text;
 // -------- save video memory
 msave = new unsigned[SCREENWIDTH];
 row = min(line-1, 24);
 savevideo(msave, row, 0, row, SCREENWIDTH-1);
 // --------- display the menu bar
 colors(MENUFG, MENUBG);
 setcursor(0, row);
 int cols = SCREENWIDTH;
 while (cols--)
 window_putc(' ');
 // ---- compute the width of the selection texts
 int titlewidth = 0;
 for (int i = 0; mtext[i] && i < MAXSELECTIONS; i++)
 titlewidth = max(titlewidth, strlen(mtext[i]));
 // ----- save the selection count
 selections = i;
 // ------ display the selection texts
 for (i = 0; i < selections; i++)
 select(row, i+1, mtext[i], OFF, titlewidth);
 // ------- dispatch the menu's selections

 dispatch(sel, titlewidth);
}

// ----------- destructor for a menu bar
SlideBar::~SlideBar(void)
{
 restorevideo(msave, row, 0, row, SCREENWIDTH-1);
 delete msave;
 restorecursor();
 unhidecursor();
}

// ------ navigate the menu and dispatch a chosen function
int SlideBar::dispatch(int sel, int titlewidth)
{
 savecursor();
 int sliding = 1;
 if (sel)
 selection = sel;
 while (sliding) {
 // ---- highlight the menu bar selection
 select(row, selection, mtext[selection-1],
 ON,titlewidth);
 // ----- read a selection key
 int c = getkey();
 switch (c) {
 case ESC:
 // ----- ESC key quits
 sliding = 0;
 break;
 case FWD:
 // ------ right-arrow cursor key
 select(row, selection, mtext[selection-1],OFF,
 titlewidth);
 if (selection++ == selections)
 selection = 1;
 break;
 case BS:
 // ------ left-arrow cursor key
 select(row, selection, mtext[selection-1],OFF,
 titlewidth);
 if (--selection == 0)
 selection = selections;
 break;
 default:
 // ---- test for 1st letter match
 for (int i = 0; i < selections; i++)
 if (tolower(c) == tolower(mtext[i][1])) {
 // -- match, turn off current selection
 select(row, selection,
 mtext[selection-1],
 OFF,titlewidth);
 // --- turn on new selection
 selection = i+1;
 select(row, selection,
 mtext[selection-1],
 ON,titlewidth);
 break;
 }

 if (i == selections)
 break;
 case '\r':
 // ------- ENTER key = user selection
 if (mfunc[selection-1])
 (*mfunc[selection-1])(*this);
 sliding = !(quitflag == 1);
 break;
 }
 }
 restorecursor();
 select(row,selection,mtext[selection-1],OFF,titlewidth);
 return quitflag ? 0 : selection;
}

// --- set or clear the highlight on a menu bar selection
static void select(int row,int sel,char *ttl,int set,int wd)
{
 setcursor(5+(sel-1)*wd, row);
 if (set == OFF)
 colors(MENUFG, MENUBG);
 else
 colors(SELECTFG, SELECTBG);
 window_printf(ttl);
}

//
// POPDOWN MENUS
//

// -------- constructor for the PopdownMenu
PopdownMenu::PopdownMenu(unsigned left, unsigned top,
 char **mtext, int sel, void (**func)(PopdownMenu&))
 : (left, top, left+1+menuwidth(mtext),
 top+1+menuheight(mtext), MENUFG, MENUBG)
{
 *this << mtext;
 mfunc = func;
 text = mtext;
 selection = sel;
 selections = menuheight(text);
 // ------ dispatch the menu selection
 dispatch(sel);
}

// ------- write text selections into the popdown menu
void PopdownMenu::operator<<(char **mtext)
{
 hidecursor();
 int y = 0;
 // ----- a NULL-terminated array of character pointers
 text = mtext;
 while (*mtext != NULL) {
 cursor(0, y++);
 char hold = **mtext;
 if (**mtext == '-') {
 set_colors(DISABLEFG, DISABLEBG);
 **mtext = ' ';
 }

 *this << *mtext;
 **mtext++ = hold;
 set_colors(MENUFG, MENUBG);
 }
 unhidecursor();
}

// ------------ get a popdown menu selection
int PopdownMenu::get_selection(int sel)
{
 // ---- set the initial selection
 if (sel)
 selection = sel;
 int selecting = 1;
 int c;
 while (selecting) {
 // ------- display the menu's selection texts
 *this << text;
 // ------ watch for disabled selections
 if (**(text+selection-1)=='-')
 c = DN; // force a key to
 // bypass a disabled selection
 else {
 // ---- highlight the current selection
 cursor(0, selection-1);
 set_colors(SELECTFG, SELECTBG);
 *this << *(text + selection - 1);
 set_colors(MENUFG, MENUBG);
 hidecursor();
 c = getkey(); // --- read the next keystroke
 }
 switch (c) {
 case ESC:
 case FWD:
 case BS:
 // ---- ESC,FWD, or BS will terminate selection
 selecting = 0;
 break;
 case UP:
 // ------- up-arrow cursor key
 do
 if (--selection == 0)
 selection = selections;
 while (**(text+selection-1) == '-');
 break;
 case DN:
 // ------- down-arrow cursor key
 do
 if (selection++ == selections)
 selection = 1;
 while (**(text+selection-1) == '-');
 break;
 default:
 // ----- other key, test first letter match
 for (int i = 0; i < selections; i++) {
 if (tolower(c) == tolower(text[i][1]) &&
 *(text[i]) != '-') {
 selection = i+1;
 selecting = 0;

 }
 }
 break;
 case '\r':
 // ---- ENTER key is a selection
 selecting = 0;
 break;
 }
 }
 return c == '\r' ? selection : c;
}

// ------------ get and dispatch a popdown menu selection
int PopdownMenu::dispatch(int sel)
{
 int upanddown = 1;
 while (upanddown) {
 // ---------- read a user selection
 sel = get_selection(sel);
 switch (sel) {
 // -------- these keys exit the menu
 case FWD:
 case BS:
 case ESC:
 upanddown = 0;
 break;
 default:
 // -------- user has made a menu selection
 if (mfunc[selection-1]) {
 // ----- execute a menu selection function
 hidewindow();
 (*mfunc[selection-1])(*this);
 upanddown = !(quitflag == 1);
 restorewindow();
 }
 else {
 // ----- no function, must be a toggle
 char *cp = text[selection-1];
 cp += strlen(cp)-1;
 if (*cp == ' ')
 *cp = CHECK;
 else if ((*cp & 255) == CHECK)
 *cp = ' ';
 }
 break;
 }
 }
 return sel == ESC ? ESC : 0;
}

// --------- compute the height of a popdown menu
int PopdownMenu::menuheight(char **text)
{
 int height = 0;
 while (text[height])
 height++;
 return height;
}


// --------- compute the width of a popdown menu
int PopdownMenu::menuwidth(char **text)
{
 int width = 0;
 while (*text) {
 width = max(width, strlen(*text));
 text++;
 }
 return width;
}

// ----- test the setting of a toggle selection
int PopdownMenu::test_toggle(int sel)
{
 char *cp = text[sel-1];
 cp += strlen(cp)-1;
 return (*cp & 255) == CHECK;
}






[LISTING THREE]

// ---------- demoslid.c

#include <stddef.h>
#include <conio.h>
#include <stdio.h>
#include "menus.h"
#include "console.h"

// ----------- File menu
static char *fmenu[] = {
 " Load ",
 " Save ",
 " New ",
 " Quit ",
 NULL
};

static void load(SlideBar&);
static void save(SlideBar&);
static void newfile(SlideBar&);
static void quit(SlideBar&);

static void (*ffuncs[])(SlideBar&)={load,save,newfile,quit};

void main(void)
{
 SlideBar menu(1, fmenu, 1, ffuncs);
}

static void load(SlideBar& menu)
{
 Window wnd(20,10,40,20,BLACK,CYAN);
 wnd.title("(Stub Function)");

 wnd << "\n\n\n\n LOAD A FILE";
 getkey();
}

static void save(SlideBar &menu)
{
 Window wnd(20,10,40,20,YELLOW,RED);
 wnd.title("(Stub Function)");
 wnd << "\n\n\n\n SAVE A FILE";
 getkey();
}

static void newfile(SlideBar &menu)
{
 Window wnd(20,10,40,20,YELLOW,RED);
 wnd.title("(Stub Function)");
 wnd << "\n\n\n\n NEW FILE";
 getkey();
}

static void quit(SlideBar& menu)
{
 YesNo yn("Quit");
 if (yn.answer)
 menu.terminate();
}






[LISTING FOUR]

// ---------- demopop.c

#include <stddef.h>
#include <conio.h>
#include <stdio.h>
#include "menus.h"
#include "console.h"

// ----------- File menu
static char *fmenu[] = {
 " Load ",
 "-Save ",
 " New ",
 " Option ",
 " Quit ",
 NULL
};

static void load(PopdownMenu&);
static void save(PopdownMenu&);
static void newfile(PopdownMenu&);
static void quit(PopdownMenu&);

static void (*ffuncs[])(PopdownMenu&) =
 { load, save, newfile, NULL, quit };


void main(void)
{
 PopdownMenu menu(20, 10, fmenu, 1, ffuncs);
}

static void load(PopdownMenu& menu)
{
 Window wnd(20,10,40,20,BLACK,CYAN);
 wnd.title("(Stub Function)");
 wnd << "\n\n\n\n LOAD A FILE";
 menu.enable_selection(2); // enable the save command
 getkey();
}

static void save(PopdownMenu &menu)
{
 Window wnd(20,10,40,20,YELLOW,RED);
 wnd.title("(Stub Function)");
 wnd << "\n\n\n\n SAVE A FILE";
 menu.disable_selection(2); // disable the save command
 getkey();
}

static void newfile(PopdownMenu &menu)
{
 Window wnd(20,10,40,20,YELLOW,RED);
 wnd.title("(Stub Function)");
 wnd << "\n\n\n\n NEW FILE";
 menu.enable_selection(2); // enable the save command
 getkey();
}

static void quit(PopdownMenu& menu)
{
 if (menu.test_toggle(4))
 menu.terminate();
 else {
 YesNo yn("Quit");
 if (yn.answer)
 menu.terminate();
 }
}






[LISTING FIVE]


// -------- strings.h

#ifndef STRINGS
#define STRINGS

#include <string.h>


class string {
 char *sptr;
public:
 // CONSTRUCTORS
 // -------- construct a null string
 string(void);
 // ------- construct with a char * initializer
 string(char *s);
 // ------- construct with another string as initializer
 string(string& s);
 // -------- construct with just a size
 string(int len);

 // DESTRUCTOR
 ~string(void) { delete sptr; }

 // MEMBER FUNCTIONS
 // ------ return the address of the string
 const char *stradr(void) { return sptr; }

 // SUBSTRINGS
 // ------ substring: right len chars
 string right(int len);
 // ------ substring: left len chars
 string left(int len);
 // ------ substring: middle len chars starting from where
 string mid(int len, int where);

 // ASSIGNMENTS
 // -------- assign a char array to a string
 void operator=(char *s);
 // ---------- assign a string to a string
 void operator=(string& s) { *this = s.sptr; }

 // CONCATENATORS
 // ------- 1st concatenation operator (str1 += char *)
 void operator+=(char *s);
 // ------- 2nd concatenation operator (str1 += str2;)
 void operator+=(string& s) { *this += s.sptr; }
 // ------- 3rd concatenation operator (str1 = str2+char*;)
 string operator+(char *s);
 // ------- 4th concatenation operator (str1 = str2 + str3;)
 string operator+(string& s) { return *this + s.sptr; }

 // RELATIONAL OPERATORS
 int operator==(string& s) { return strcmp(sptr,s.sptr)==0;}
 int operator!=(string& s) { return strcmp(sptr,s.sptr)!=0;}
 int operator<(string& s) { return strcmp(sptr,s.sptr)< 0;}
 int operator>(string& s) { return strcmp(sptr,s.sptr)> 0;}
 int operator<=(string& s) { return strcmp(sptr,s.sptr)<=0;}
 int operator>=(string& s) { return strcmp(sptr,s.sptr)>=0;}
 int operator==(char *s) { return strcmp(sptr,s)==0; }
 int operator!=(char *s) { return strcmp(sptr,s)!=0; }
 int operator<(char *s) { return strcmp(sptr,s)< 0; }
 int operator>(char *s) { return strcmp(sptr,s)> 0; }
 int operator<=(char *s) { return strcmp(sptr,s)<=0; }
 int operator>=(char *s) { return strcmp(sptr,s)>=0; }

 // SUBSCRIPTORS

 char operator[](int n) { return *(sptr + n); }
 char* operator+(int n) { return sptr + n; }
};

#endif







[LISTING SIX]

// -------- strings.c

#include <stddef.h>
#include <stream.hpp>
#include "strings.h"

// -------- construct a null string
string::string(void)
{
 sptr = new char;
 *sptr = '\0';
}
// ------- construct with a char * initializer
string::string(char *s)
{
 sptr = new char[strlen(s)+1];
 strcpy(sptr, s);
}
// ------- construct with another string as initializer
string::string(string& s)
{
 sptr = new char[strlen(s.sptr)+1];
 strcpy(sptr, s.sptr);
}
// -------- construct with just a size
string::string(int len)
{
 sptr = new char[len+1];
 memset(sptr, 0, len+1);
}
// -------- assign a char array to a string
void string::operator=(char *s)
{
 delete sptr;
 sptr = new char[strlen(s)+1];
 strcpy(sptr, s);
}
// ------- 1st concatenation operator (str1 += char *;)
void string::operator+=(char *s)
{
 char *sp = new char[strlen(sptr) + strlen(s) + 1];
 strcpy(sp, sptr);
 strcat(sp, s);
 delete sptr;
 sptr = sp;

}
// ------- 3rd concatenation operator (str1 = str2 + char*;)
string string::operator+(char *s)
{
 string tmp(*this);
 tmp += s;
 return tmp;
}
// ------ substring: right len chars
string string::right(int len)
{
 string tmp(sptr + strlen(sptr) - len);
 return tmp;
}
// ------ substring: left len chars
string string::left(int len)
{
 string tmp(len+1);
 strncpy(tmp.stradr(), sptr, len);
 return tmp;
}
// ------ substring: middle len chars starting from where
string string::mid(int len, int where)
{
 string tmp(len+1);
 strncpy(tmp.stradr(),sptr+where-1,len);
 return tmp;
}


































October, 1989
STRUCTURED PROGRAMMING


Humpty-Duntemann's Handy Object-Oriented Glossary




Jeff Duntemann, K16RA


To update the old egg a little: When I use a word, it means just what I choose
it to mean, no more and no less -- unless, of course, you're the one holding
the AK- 47.
We've come pretty close to bloodshed lately on CompuServe's Borland
Programming Forum A, Section 1, over words as they apply to OOP. There's been
an explosion in programmer jargon in the past year or two, and shrapnel is
flying every which way. The word "static" alone seems to have five different
interpretations, depending not only on what language you're using, but on how
you're using the term within the language.
This way lies madness. Most of the rest of this month's column, therefore,
will be a glossary of OOP jargon as it applies to Smalltalk, Actor,
QuickPascal, and Turbo Pascal 5.5. I won't be going into jargon specific to
C++ unless that jargon has been borrowed for one of the other languages.
(Turbo Pascal 5.5 borrows heavily from the C++ lexicon.) Let's not all kill
one another over two-bit words like "virtual."
OOP is the future. It will certainly help if by the time we reach that future
we're all speaking the same language.


Abstract Object Type


This Borlandism is synonymous with Smalltalk's term abstract superclass. An
abstract object type is one designed only to be inherited from. Actual
instances (see also) of abstract object types or abstract superclasses are
never created.
Because none of the OOP languages in common use under DOS support multiple
inheritance (see also) abstract object types allow two otherwise unrelated
object types to be tied together into a single class hierarchy (see also).
For example, in a windowing system there might be a spreadsheet window and a
telecommunications window. It doesn't make sense to make the spread-sheet
window a child of the telecomm window, nor the telecomm window a child of the
spreadsheet window. The two are pretty thoroughly independent. However, both
have a border and a position and both may be dragged around on the screen.
Therefore, a Window abstract object type is created so that the spreadsheet
window and telecomm window can inherit those common characteristics from a
common ancestor type. A Window object by itself would not be useful, so no
instances of object type Window are ever actually created. Class Window may,
however, act as a "mask" over one of its child object types, as described
under polymorphism.


Ancestor Class


(In Turbo Pascal 5.5, ancestor object type.) Any class or object type further
up in the class hierarchy (see also) from a given class is an ancestor class
to that class. The immediate ancestor class of any class is its parent class
(see also.)


Binding


The process through which the caller of a routine gets the address of that
routine is called binding. In non-OOP languages, this is straightforward: The
compiler binds caller and call-ee at compile time by building a CALL
instruction to the call-ee's address. The decision is made by the compiler,
and cannot be changed at run time.
OOP allows late binding (see also), which defers the binding of caller and
call-ee until run time through some behind-the-scenes magic involving tables
of procedure addresses. Something approximating late binding can be done with
procedural types, as implemented in Turbo Pascal 5.0 and QuickPascal 1.0, but
OOP brings late binding to maturity. Late binding makes polymorphism possible.


Browser


There's nothing specific to object-oriented programming about browsers, but
the word is most used in OOP environments such as Smalltalk. A Smalltalk
browser is a lot like an inspector in Turbo Debugger: A special-purpose editor
that allows you to look at and modify the various parts of an object. A
browser typically has several separate interior regions called "subviews" or
"panes," each of which shows a different aspect of the object being browsed.
One pane may summarize the object's data fields, while another indicates where
it falls in the class hierarchy, and yet another shows the actual code making
up the object's methods.
Browsers are a good idea, and I predict that they will become a lot more
common as our data abstraction tools like OOP become more sophisticated and
the structures we use larger and more prolix.


Child Class


(In Turbo Pascal 5.5, this term would be child object type.) Any class that
inherits from another class is a child class of that class. In other words, if
class B inherits from class A, B is a child class of A.


Class


Classes are to objects what types are to records -- the complete description
of the item's innards. So close is this relationship that for Turbo Pascal 5.5
Borland dropped the term class altogether in favor of object type, which it
considered more self-explanatory and less jargon-like. (Object type and class
are synonyms, but only Borland uses object type.) Class is the correct term,
however, in nearly all other OOP languages, including Smalltalk, C++, Actor,
and QuickPascal.
Like a type, a class is a template rather than an entity allocated at some
address in memory. You create an instance (see also) of a class, and that
instance is an object. Again, it's much like creating an instance of a record
type, which is the record itself. Be warned that in most OOP languages,
creating an instance of a class is done differently than creating an instance
of an ordinary (non-object) type.
A good capsule definition of class is a type that exhibits the characteristics
of object orientation: Encapsulation and inheritance.



Class Hierarchy


This Smalltalk term is synonymous with object hierarchy (Turbo Pascal 5.5) and
class tree (Actor). Inheritance (see also) allows classes to be defined as
"children" of existing classes, such that the children have access to all data
and code defined in the parent class. A sophisticated OOP application might
have an elaborate structure of related classes that resembles a tree:
 Window
 
 
 ---Form
 
 
 ---Field
 
 
 ---IntergerField
 
 
 ---BooLeanField
 
 
 ---StringField
A class hierarchy is created by defining abstract objects (see also) at the
top of the hierarchy, and giving those abstract objects all of the most
generally applicable code and data for the whole hierarchy. The abstract
objects thus exist as "broadcast stations" from which the most general code
and data may be inherited. As you move toward the leaves, each child class
adds code and data that is more and more specific in nature, until the leaves
of the hierarchy tree are objects that do fully useful tasks.
In the mini-hierarchy shown, the Window class might be nothing more than a
rectangular subset of the screen with a border. It contains only X,Y position
values and flags indicating whether or not it is currently visible or active.
Its methods would allow it to be dragged around the screen and made visible or
invisible, but nothing more. The Form class might add a border and mechanisms
for vertical/horizontal scrolling. The Field class adds generalized methods
for setting and returning a pointer to a value, but does not yet commit to any
particular type of value. Only at the leaves of the hierarchy do classes like
BooleanField provide a completely useful object -- in this case, a field for
the entry and editing of Boolean values. Remember that BooleanField retains
everything its parents provided: Windows, drag methods, scroll bars, and so
on. It only adds the last and most specific parts of the object: Those parts
catering to the Boolean data type.
A class hierarchy is an extremely powerful tool for managing the complexity of
an OOP application. It distributes data and functionality along a line from
general to specific, and allows the programmer to zero in on only the portion
of the functionality that is being worked on at any given time.


Class Tree


The Actor term for class hierarchy (see also).


Dynamic Objects


When objects are allocated on the heap, they are dynamic objects, just as
ordinary variables that are allocated on the heap are dynamic variables.
You're unlikely to run into this term unless you're using or reading about
Turbo Pascal 5.5. In QuickPascal and Macintosh Object Pascal, all objects must
be allocated on the heap, so like it or not they're all dynamic, and thus the
term loses its purpose and isn't used. Turbo Pascal 5.5, however, allows
objects to be allocated statically in the data segment exactly as ordinary
variables are. So just as in all Pascals you can have static variables and
dynamic variables, in Turbo Pascal you can have static objects and dynamic
objects.


Early Binding


Binding (see also) is the process by which the caller of a routine is given
the address of that routine. Traditional Pascal procedure calls are good
examples of early binding, because the compiler plugs the destination address
into the machine-code CALL instruction when the instruction is generated, at
compile-time. Special mechanisms exist in OOP languages to defer binding until
run time; this is called late binding (see also).


Encapsulation


At the heart of the notion of OOP is the practice of defining code (in the
form of methods, see also) and data together as a single-named entity called
an object. This rolling up of methods and data into a unified bundle is part
and parcel of the term encapsulation. The problem is, some people (notably,
the Smalltalk partisans) take it considerably further, and hold that
encapsulation means that users of an object cannot directly know of or
reference the object's data. The data lives at the center of the capsule, as
it were, and can only be accessed through one of the methods that surround it.
In other words, if a Boolean field within an object is named Visible, users of
the object cannot perform a test such as
 IF MyObject.Visible THEN
 MyObject.Drag ELSE
 MyObject.Relocate;
Instead, a method named Is Visible would have to be defined to return the
value of the data field Visible, and two additional methods (perhaps Show and
Hide) would be needed to flip the Boolean state of Visible inside the object.
This is literally true in Smalltalk. Smalltalk's encapsulation of data is
absolute. In other OOP languages, this sort of absolute encapsulation is
allowed, but not enforced. C++, QuickPascal, and Turbo Pascal 5.5 allow
ordinary references to an object's data from anywhere within the object's
scope. In other words, if the object itself is "visible," its data fields are
also visible. The programmer can choose the degree to which data fields are
encapsulated within objects.
Absolute encapsulation places an unavoidable performance burden on some types
of programs, hence the more lenient encapsulation of C++ and Object Pascal. In
most cases, it's a good idea to access object data only through the object's
methods, but C++ and Object Pascal people can break that rule according to
their own judgment -- and must accept the consequences if they judge badly.


Extendibility



Through the property of inheritance (see also), objects may be extended
without access to the objects' source code. The mechanism used is overriding
methods (see override.) A child class of an existing class defines a new
method with the same name as an inherited method. This new method adds some
new functionality to the inherited method and replaces it within the child
class.


Inheritance


One of the fundamental characteristics of OOP is that a class (see also) may
be defined in terms of an existing class. The new class inherits all
definitions made within the existing class (called its "parent class") and any
definitions its parent class may have inherited from classes further up the
class hierarchy (see also.)
Inherited data and methods may be used as freely as data and methods defined
within the class itself.


Instance


The term instance is not limited to OOP situations, but it's used more
frequently in connection with objects than with earlier data structures like
arrays and records. An instance is a realization in memory of the description
we call a class or an object type. Classes or object types are defined after
the TYPE reserved word, whereas instances are declared after VAR. Instances
are thus object variables, which we normally just call objects.


Late Binding


Binding (see also) is the process by which the caller of a routine is given
the address of that routine. This can be done at compile time (early binding;
see also) or it can be done at the time the actual call is made. Binding the
caller to the routine at the time the call is made is called late binding.
Many different mechanisms are used to implement late binding, but most involve
a behind-the-scenes table of method addresses specific to a given class or
object type. To execute a late-bound method DoIt belonging to object MyGadget
(in Object Pascal terms the statement MyGadget.DoIt;) the code must first
determine the class of MyGadget, locate the method table for that class, look
up the table entry for method DoIt, and then pass control to the address at
DoIt's entry in the table.
Late binding makes polymorphism (see also) possible.


Method


Through encapsulation (see also) code and data are combined into a
single-named entity called an object. The code portion of an object consists
of some number of routines called methods. The headers of an object's methods
are defined within the object's definition, and the bodies are defined later
in this glossary.
In the Location object definition shown later in the entry for virtual, there
are four methods: MoveTo, Show, Hide, and IsVisible. In Turbo Pascal 5.5 and
QuickPascal, methods may be either procedures or functions and are defined in
almost exactly the same way. In Turbo Pascal 5.5, methods may also be either
static methods or virtual methods (see also.) All of QuickPascal's methods are
the equivalent of Turbo Pascal's virtual methods; static methods are not
present in QuickPascal.


Message


This term is used only by Smalltalk and Actor. A message is a command to an
object. When received by the object, the message selects which method is to be
executed in response to that message.
This scheme (called "message-passing") is the functional paradigm for
polymorphism (see also) in Smalltalk and Actor. The message selects its method
at run time through late binding (see also.)
In Object Pascal and C++, the name of a virtual method is analogous to the
message, and late binding resolves the name of a virtual method to the address
of the code implementing the correct method for the true class of the object
making the method call.


Multiple Inheritance


In most OOP languages, each class or object type has only one parent, from
which it inherits everything defined by that parent or inherited by that
parent from classes higher in the class hierarchy. Some newer OOP languages
not yet available for DOS (including C++ 2.0 and Eiffel) allow a class to
inherit from two or more parent classes. A class hierarchy thus becomes a sort
of class web, and there are significant questions as to what happens when more
than one entity with the same name is inherited by a single class.
Note that Smalltalk, Actor, Turbo Pascal 5.5 and QuickPascal do not implement
multiple inheritance, so I won't go into much more detail here.


Object


An object is an instance (see also) of a class or object type. By an instance
I only mean that it's a variable, with an allocation of memory somewhere, and
has some special properties that make it an object and not a record. In
Smalltalk and (to a slightly lesser extent) Actor, everything is an object. In
C++, Turbo Pascal 5.5, and QuickPascal, objects coexist peaceably with older,
simpler types such as integers, Booleans, characters, and records.


Object type


Borland International redefined some of the jargon when they released Turbo
Pascal as their implementation of Object Pascal. Their term object type is
synonymous with class (see also) used in nearly all other OOP languages. An
object type or class can be seen as a set of instructions by which the
compiler builds objects in memory. In this it is no different from an ordinary
record or array type definition that specifies how many bytes in size a
variable of that type will be, and how some of those bytes in combination
represent different flavors of data.
Again, think of object types or classes as templates by which the compiler
whacks out actual objects as needed. These objects are called instances (see
also) of the object type. An instance is just a variable of an object type.



Override


Object classes ordinarily inherit all methods from their parent class. The
child class has the option, however, of redefining a method it inherits from
its parent class or (through the parent class) some more distant ancestor
class. This process of redefining an inherited method is called overriding the
ancestor's method. This is the primary way that objects are extendible. A
child class overrides an existing method and adds new or more specific
behavior to the overriding method.
In Turbo Pascal 5.5, methods are overridden simply by redefining them under
the same name. In QuickPascal, however, the reserved word "OVERRIDE" must be
included after the method header of the overriding method:
 TYPE
 Point : OBJECT(Location)
 Color : Integer;
 PROCEDURE Show; OVERRIDE;
 FUNCTION GetColor : Integer;
 PROCEDURE SetColor-(NewColor : Integer); END;



Parent Class


(In Turbo Pascal 5.5, parent object type.) Through the property of inheritance
(see also), an object class may be defined as the child of an existing class.
This existing class is called the parent class of the new class. Smalltalk
calls this a superclass (see also). QuickPascal (but not Turbo Pascal 5.5)
objects can reference data and methods defined within their parent classes by
qualifying a reference with the reserved word "INHERITED."
Parent classes are sometimes called "ancestor classes."


Polymorphism


From the Greek for "many shapes," polymorphism allows a single method name to
act as a doorway to numerous separate methods, with the actual method called
chosen by the language's late binding mechanisms at the time the call is made.
This is most tersely explained by example. In an object hierarchy, an abstract
class (see also) called "Field" implements a generic data field not committed
to any given type of data. Field defines numerous generic methods for
manipulating data, including GetValue, PutValue, and Edit. Field has numerous
child classes, one for each specific type of data: IntField, BooleanField,
CharField, StringField, and so on. Each of these child classes overrides
Field's generic methods with methods specific to the child classes' own data
types. That is, IntField.Edit edits an integer value, StringField.Edit edits a
string value, and so on.
One of the subtler rules of OOP is that assignment compatibility and pointer
compatibility are extended down an object hierarchy. In other words, in our
example an object of class Field may be assigned an object of class IntField,
StringField, BooleanField, and so forth. (But not the other way around!) A
pointer defined as pointing to the Field class may also point to any of
Field's child classes. Such an assignment is called a polymorphic assignment.
This means that an object of class Field may in fact be a mask over an object
of class StringField or any of Field's other child classes. Precisely which
class wears the Field mask doesn't matter. Given a Field object named MyField,
when the method call MyField.Edit is made, late binding selects the correct
Edit method at the time the call is made. If a StringField object had been
assigned to MyField, then the StringField.Edit method would be called. On the
other hand, if a BooleanField object had been assigned to MyField, then the
BooleanField.Edit method would be called -- and the decision is not made until
the call itself is made.
In a sense, what polymorphism allows us to do is say to an arbitrary Field
object: "Go edit yourself!" and the Field object will select the correct Edit
method to use. One command -- Edit -- has a different shape for each different
child class of Field. Polymorphism!
This is difficult business, but it is enormously powerful. I'll deal with
polymorphism at length in a future column.


Static


The word static is currently the bad boy of OOP programming, and
single-handedly causes more confusion than all other OOP terms combined,
including that old devil polymorphism.
The problem mainly involves Turbo Pascal 5.5. In a Turbo Pascal context, the
adjective "static" has two uses (and two opposites) with wildly different
contexts. One use involves the two kinds of methods: Static methods are
early-bound methods, as opposed to virtual methods, which are late-bound
methods. The other use involves the two ways objects can be created in memory:
A static object is an object allocated in the data segment, whereas a dynamic
object is an object allocated on the heap. Thus in one context "static" is the
opposite of "virtual," while in the other context "static" is the opposite of
"dynamic."
The confusion is compounded by the fact that the term "static" is not used at
all in QuickPascal, so you Microsofters are probably wondering what all the
hoohah is about.
(I've provided more detailed definitions of static methods, static objects,
dynamic objects, and virtual methods elsewhere. To pin down your complete
understanding of the very slipdpery term static be sure to read them all!)


Static Methods


Only C++ and Turbo Pascal 5.5 allow the definition of static methods. A static
method is a method subject to early binding (see also) only. Methods subject
to late binding are virtual methods (see also). Note that the term virtual
methods is not used in QuickPascal because late binding is applied to all
methods, making all methods virtual and thus making the distinction between
virtual and static methods unnecessary.
Methods are static by default in Turbo Pascal 5.5. To make a method virtual
you must add the reserved word "VIRTUAL" immediately after the method header.
In the "Location" object definition shown in the entry for virtual, the MoveTo
and IsVisible methods are static, whereas the Show and Hide methods are
virtual.
Because static methods are early-bound, a static method call is identical to
an ordinary procedure or function call. There is an additional level of
indirection involved in making a Turbo Pascal 5.5 virtual method call, so
static methods are slightly faster than virtual methods. (But only slightly.)
Static methods may be overridden, but are not subject to polymorphism (see
also).


Static Objects


A static object is an object allocated in a program's data segment rather than
on the heap. The term is another one that Borland International borrowed from
C++ in implementing Turbo Pascal 5.5. QuickPascal, Smalltalk, and Actor make
no use of the term, because in those languages all objects are dynamically
allocated.
A static object is very much like a super-record having the special properties
characteristic of object-oriented programming. Like an ordinary record, you
create static objects in the VAR section of your program:
 VAR
 MyRecord : RecordType;

 MyObject : ObjectType;
Nothing magical about it, no pointers, no need to mess with New or Dispose.
Static objects are a good place to start learning about objects, since you can
concentrate on using objects without worrying so much about creating them
correctly.


Subclass


Like the term superclass, subclass is commonly used only with regard to
Smalltalk. A subclass is a child class. In other words, if class B inherits
from class A, class B is a subclass of class A.


Superclass


Within an object hierarchy (see also) every class has only one immediate
ancestor, called its "parent" or superclass. This term is commonly used only
in Smalltalk.


Virtual


The term virtual was one of several borrowed from C++ by Borland International
and applied to their implementation of Object Pascal released as Turbo Pascal
5.5. It is not used in QuickPascal nor in Smalltalk nor Actor.
"VIRTUAL" is one of four new reserved words in Turbo Pascal 5.5. It is a
qualifier placed after a method header in the object type definition:
 TYPE Location =
 OBJECT
 X,Y : Integer;
 Visible : Boolean;
 PROCEDURE MoveTo (NewX,NewY : Integer);
 PROCEDURE Show; VIRTUAL;
 PROCEDURE Hide; VIRTUAL;
 FUNCTION IsVisible : Boolean;
 END;
The use of the VIRTUAL reserved word makes the method that precedes it a
virtual method, which is a method that may be late bound. (See late binding.)
Keep in mind that the word "virtual" in this context has nothing whatsoever to
do with virtual memory or other storage issues. Also keep in mind that
functionally, all methods in QuickPascal are virtual methods. In Turbo Pascal
there is the option of defining methods as static methods (see also) and the
reserved word VIRTUAL was chosen to differentiate between early and late bound
methods.


Virtual Methods


Except in Turbo Pascal 5.5 and C++, all methods are virtual methods, so the
term is not used much outside of Turbo Pascal 5.5. Virtual methods are defined
as methods that take part in late binding (see also).
In early binding (see also) the address of a method is baked into the "CALL"
machine instruction that performs the method call, by the compiler at
compile-time. In late binding, the actual address of the method being called
is not determined until the time that the call takes place. How this is done
varies from language to language (and is done differently, in fact, by Turbo
Pascal 5.5 and QuickPascal) but almost always involves a table of method
addresses hidden inside the object that owns the methods. To make a late-bound
method call, the address of the desired method is looked up in the table of
method addresses, and then control is passed to that address.
Note that all virtual methods of the same name must have identical method
headers, including the identical type and order of parameters.
The real power of virtual methods involves polymorphism, and is not an easy
thing to describe in a paragraph or two. I'll take up the subject of late
binding and polymorphism in detail in a future column.


Algorithms Over Easy


The difference between a sort library and a book of sort algorithms is the
difference between giving a guy a fish and teaching him how to fish. If you
don't know how your tools work, you're at their mercy -- which is akin to the
feeling of downing your last sardine with no more in the can.
The best algorithms book I've ever seen crossed my desk last week: Turbo
Algorithms, by Keith Weiskamp, Namir Shammas, and Ron Pronk. Space is short,
so I can't describe it in detail. But the (rather remarkable) idea is this:
The authors present a fairly large number of useful algorithms and then
implement each one in all four Turbo languages -- Pascal, C, Basic, and
Prolog. As you might imagine, the book is by needs terse, but it is nicely
written and has some of the best technical figures I've seen in a long time.
The algorithms cover sorting, searching, stacks, queues, binary and AVL
(balanced) trees; singly-linked, doubly-linked, and circular lists; and word
and token string processing. That's a lot of ground to cover with real code in
four languages, but somehow, it works. I found the math section pretty tough
going, but that could be my own aversion to the subject, and the rest of it
was both graspable and immediately useful.
As it happens, Turbo Libraries is one in a series of three solid books by the
same authors, all of them spanning the four Turbo languages; but as Wiley's
book distribution system is as brain-dead as their books are good, you may
have to order them through your local bookstore. They are, however, well worth
the wait.


Those Old Release-Level Blues


In my September column I raised some eyebrows by reporting that "Users of MS
Pascal 5.0 should note that QuickPascal is only broadly compatible with MS
Pascal...." Microsoft's current major release level of their command-line
Pascal compiler is 4, not 5, a mistake ascribable to late-night, bleary-eye
syndrome. Although, the next version of Microsoft Pascal has yet to be
announced, Microsoft has confirmed that any future release will be upwardly
compatible with QuickPascal 1.0. The aboriginals who can only count 1 .. 2 ..
many had their finger on something -- how are we ever going to keep these
things straight in the year 2000, when we're dealing with MS Basic 21.0 and
Turbo Pascal 17.5?


Products Mentioned



Turbo Algorithms: A Programmer's Reference Keith Weiskamp, Namir Shammas, and
Ron Pronk John Wiley & Sons, Inc., 1989 ISBN 0-471-61009-7 Softcover, 444
pages $26.95 Listings diskette $24.95
Turbo Language Essentials: A Programmer's Reference Keith Weiskamp, Namir
Shammas, and Ron Pronk John Wiley & Sons, Inc., 1989 ISBN 0-471-60907-2
Softcover, 500 pages $24.95
Turbo Libraries: A Programmer's Reference Keith Weiskamp, Namir Shammas, and
Ron Pronk John Wiley & Sons, Inc., 1989 ISBN 0-471-61005-4 Softcover, 478
pages $26.95 Listings diskette $24.95
QuickC with Quick Assembler Microsoft, Inc. 16011 NE 36th Way Redmond WA 98073
206-882-8088 $199

























































October, 1989
OF INTEREST
Erik Labs has released BiModem, a file transfer protocol for the IBM PC and
compatibles that can send and receive files simultaneously, and which allows
operators to chat during file transfer. The protocol is capable of attaining
transfer speeds over 98 percent efficient in both directions at the same time.
While many bulletin board interfaces are available and can be used with any
communications program that permits execution of an external program, the
protocol has been implemented in GT POWER (P&M Software).
A modified 32-bit CRC and a 32-bit Filesum insure file accuracy, and the
feature "refresh transfers" compares files and sends only the information that
has changed. Download requests can be located through multiple directories;
files may be recovered from either an aborted files directory or the receiving
directory. Additional file requests may be added and transfers in progress
canceled during the process of transferring files. Doubledos and DesqView are
automatically recognized, giving time to other tasks.
The protocol has the ability to tag file requests from disk or a files list,
and a Hot key program is included to tag files displayed on your screen.
Transfer requests can be addressed to specific users to take place
automatically the next time they connect. The price is $25 ($32.50 with laser
printed documentation).
Erik Labs
3431 W Thunderbird Rd., Ste. 13-311
Phoenix, AZ 85023
602-942-5403
602-979-5720 (bulletin board)
Matrix Software Technology has introduced a telecommunications black box
called "BlackBox" that contains a series of objects that support
communications between computers via modem or direct connection. This enables
users to add independent telecommunications sessions and sophisticated
scripting to their Layout programs. "Inside every BlackBox," according to
Vincent Garofalo, president of Matrix, "our software engineers have built a
set of the most wanted capabilities. Rather than reinventing the wheel, our
users can concentrate on building better applications."
BlackBoxes are software objects that users can manipulate in Layout. It is
unnecessary for users to know what is happening inside the object -- they can
concentrate instead on connecting BlackBoxes with other objects on their
cards.
Inside the BlackBox, Layout employs objects that perform complex functions,
which allows users to add a range of capabilities in a particular area when
they create their applications. These include functions for electronically
exchanging text, documents, programs, and graphics between two different
computers. Additional objects are phone book management, VT100 terminal
emulation, and a graphical scripting capability (which facilitates the design
of programs that allow users to outline a series of steps based on the
telecommunications dialog). The individual, modular nature of the BlackBox
makes it easy for users to incorporate one or more of these capabilities as
they build their programs.
At the same time, Matrix is introducing a dBase BlackBox to create standard
user interfaces. Both are priced at $69.95, and run with Matrix Layout,
Version 2.0, now available for $199.95. Layout and the BlackBoxes both run on
all IBM PCs, PS/2s, and compatibles, with support for all major graphics
cards. A mouse is optional.
Matrix Software Technology Corporation
1 Massachusetts Technology Center
Harborside Drive
Boston, MA 02128
800-533-5644
617-567-0037 (in Mass.)
The Santa Cruz Operation Inc. (SCO) has begun shipping SCO Unix System V/386,
Release 3.2, a full-featured addition to the SCO System V family of products
that addresses the need for standards and security in a commercial Unix
system. SCO Unix System V/386 also runs on 486-based systems, and is the
foundation of Open Desktop, SCO's integrated operating environment for 386-
and 486-based PCs.
SCO Unix System V/386 features easy-to-use, menu-based setup and
administration compatible with SCO Portfolio, FIPS 151.1 conformance, X/Open
conformance, ANSI 8-bit internationalization, enhanced security, file system
improvements, SCO device drivers that support dozens of standard peripherals,
and the highest level of compatibility with SCO Xenix applications.
SCO Unix System V/386 includes improved versions of the custom utility,
on-line documentation, and an integrated system administration shell. It also
offers users a new, streamlined installation procedure, a mail system that
allows communication with a variety of electronic mail systems, a badtrk
utility so users can enter the manufacturer's hard disk defect list, and an
interactive configure program that makes it easy to tune the system's
configuration and add or delete device drivers. And more tutorials, expanded
reference sections, extensive examples, as well as a road map and
quick-reference guide are included.
The SCO Unix System V/386 development system contains MS-DOS and OS/2
cross-development features; full COFF and OMF support; IEEE Posix, ANSI C,
X-Open, and C2-security libraries; the Microsoft 5.0 C compiler; the AT&T C
compiler; Microsoft Code View; and SCO CGI graphics development tools. The
package also includes a guide to device drivers, which annotates examples of
all classes of device drivers, line disciplines, streams, and file systems.
It is available for 386 IBM AT-compatible systems equipped with an
AT-compatible ESDI or SCSI disk controller. It will soon be available for IBM
PS/2 models 55SX, 70, and 80, and for Micro Channel compatibles. The list
price for the two-user version is $595; the unlimited multiuser version is
$895. Upgrades are also available.
The Santa Cruz Operation, Inc.
400 Encinal St., P.O. Box 1900
Santa Cruz, CA 95061
408-425-7222
Jensen & Partners International Inc. are shipping a new TopSpeed Modula-2
Communications Toolkit that is designed to help programmers write applications
that use IBM PC serial port hardware. The toolkit was written entirely in
Modula-2, and gives access to a wide range of capabilities, from utilizing
hardware registers to terminal emulation.
The Communications Toolkit provides for low-level device driving. Procedures
in the RS-232 module allow you to select and initialize any of four hardware
ports, test for carrier, install/uninstall the serial interrupt handler,
select XON/XOFF or RTS/CTS flow control, perform serial read and write, and
assert the break signal to the modem interface.
Procedures in the file transfer support module include status window support
to monitor the transfer status, and sending and receiving data via XModem,
YModem, windowed XModem, and Kermit protocols.
Applications support includes routines, macro transmission procedures, timer
routines, and VT1OO, VT52, and BBS ANSI terminal emulation. There are also
procedures for testing and setting logging modes, reading, writing, and
printing files, file selections, and parsing and building file names.
Scripting for automatic logon to remote services includes a compiled script
language with commands for loading new scripts, condition testing, screen
clearing, downloading/uploading files, manipulating log files, transmitting/
receiving strings to the modem interface or keyboard, and more.
The Communications Toolkit includes over 8000 lines (260K) of source code for
21 modules. There is documentation on the 8250 UART, cabling information for
PC XT and AT, and a demonstration program.
TopSpeed is also shipping a Modula-2 B-tree Toolkit with tools for writing
powerful database applications. Both toolkits sell for $79.95 each.
Jensen & Partners International, Inc.
1101 San Antonio Rd., Ste. 301
Mountain View, CA 94043
800-543-5202 (inside U.S.)
800-543-8452 (inside Canada)
FORTH Inc. has released pF/x, a real-time multiuser, multitasking operating
system in a version designed to run coresident with PC/MS-DOS on 80386-based
systems. "pF/x will support as many concurrent, asynchronous tasks and users
as the 80386 will allow," according to spokesperson Mary Hawkins, and "is
ideal for developing applications such as instrumentation, process control,
industrial sensors, and anywhere else there is a need for real-time data
acquisition, analysis, and control."
pF/x is compatible with PC/MS-DOS, Versions 3.3 and higher. It normally runs
in protected mode, but transparently switches to compatibility mode when
accessing system resources. It can increase processing speed by linking
time-critical interrupt routines directly to the hardware interrupt vectors
(for zero interrupt latency), and takes only 9.28 microseconds for a
multi-tasker loop of two tasks on a 25 MHz 80386.
pF/x is part of the polyForth development environment, which features the
Forth language, macro-assemblers for the 80386 and 80387, editor, target and
turnkey compilers, graphics, database support, and utilities. pF/x costs
$3850.
FORTH, Inc.
111 N. Sepulveda Blvd.
Manhattan Beach, CA 90266
800-55-FORTH; 213-372-8493
Lattice Inc. has released the new Lattice Communications Library, which they
claim is a comprehensive set of high-level and low-level C functions for
creating applications that perform asynchronous communications.
"The library supports DOS, OS/2, and AmigaDOS environments with a full set of
functions for XModem, YModem, Kermit, and ASCII protocols," according to
Robert Hansen, Lattice vice president. Functions included in the library give
programmers the ability to select, open, and close a communications port; set
baud rate, data bits, parity, stop bits, and buffer size; and send and receive
characters from the communications port. Other functions dial and hang up the
telephone; reset the modem; set automatic answering; send and receive files
using ASCII, XModem, YModem, or Kermit error-checking protocols; and display
the progress of file transfers.
The price is $250, with source code available for an additional $250. It
supports Lattice C, Version 3.4. Versions for Microsoft and Turbo C compilers,
as well as an AmigaDOS version, are also available. Lattice will include the
library in Version 6.0 of the Lattice C Compiler for DOS and OS/2.
Lattice, Inc.
2500 South Highland Ave.
Lombard, IL 60148
312-916-1600
Addison-Wesley has published the Apple Communications Library, a new series by
Apple's networking and communications publications department that addresses
computer networking, and includes books for newcomers to networking as well as
for programmers and developers.
AppleTalk Network System Overview, $14.95 in paperback, is part of the Library
Reference series, and describes different Apple networking products and how
they fit together in a system. This book is for those new to Apple's
networking product line, and is designed to assist developers, network
administrators, and managers in decision-making for product development and
purchases. Subjects include cabling methods, the sharing of files and
printers, and components of the AppleTalk network system.
Inside AppleTalk, $34.95 in hardcover, is in the Library Technical series, and
is a current technical guide to the AppleTalk network system for programmers
and developers. The book describes AppleTalk's protocol architecture,
providing programmers with the information they need to keep their software
and hardware compatible with the AppleTalk network system. It covers topics
such as physical and data link alternatives, transmission between nodes,
handling addressing discrepancies, facilitating end-to-end transmission of
data, and end-user services implementation.

These books and others in the series are available wherever computer books are
sold.
Addison-Wesley Publishing Company
Reading, MA 01867
617-944-3700
American Megatrends Inc. (AMI) announces AMIDIAG, a menu-driven diagnostic
utility for 286- and 386-based IBM PCs and compatibles. According to AMI,
AMIDIAG exhaustively tests all hardware subsystems, including motherboard,
memory, hard disk drives, floppy disk drives, keyboard, video adapter,
monitor, printer port, and serial port.
AMIDIAG features easy-to-use menus, English-language diagnostic messages,
user-configurable test routines, error logging, graphic display of the
location of defective DIP and SIMM memory chips, and a choice of continuous,
time-bound, or count-bound testing in batch or interactive modes.
Hard disk tests for hard disk controllers using MFM and ESDI data encoding
include media analysis, force bad tracks, data transfer rate, track-to-track
seek time, seek test, read/verify test, and check cylinder test. Users can
either select from the first 46 drive types defined in the machine's ROM BIOS
or define their own. AMIDIAG can also perform partial or full hard disk
formatting, with optional optimized interleave for faster performance. Floppy
drive tests include random and sequential read/write, disk change line test,
and speed test.
AMI president S. Shankar says that "AMIDIAG can diagnose defective memory
chips, bad hard disk sectors, and other critical subsystems before they become
a problem." AMIDIAG retails for $99. A monitor ROM and diagnostics card is
also available, for $995.
AMI 1346 Oakbrook Dr., Ste. 120
Norcross, GA 30093
404-263-8181



















































October, 1989
SWAINE'S FLAMES


Desultory Philippic #41




Michael Swaine


In July I did some work for a computer-industry veteran who had an idea for a
new business: Selling a product based on recent research in an exciting and
active new area of science. You would recognize his name, but he doesn't want
any premature publicity for his business -- whose prospects now look pretty
grim, anyway -- so I'll just call him Will.
Will had done his homework: He had lined up a respected professor doing
significant work in this area of science at a prestigious school, who would
act as technical consultant and lend his name to the product; he had
researched the legal implications of the business; he had analyzed production
issues, including how to maintain laboratory-level quality control; he had
talked to potential suppliers; and he had written a business plan. When he was
confident that he had his act together, he called researchers at the leading
research institute in this area of science, and arranged to meet them to
discuss his plans. The support of this institute was critical to Will's
business plan, because it was the only source of certain components of the
product he wanted to sell. He explained to them, rather eloquently I think,
how his product would raise public consciousness regarding their work, begin
the vital process of educating the public about this important area of science
and the vast potential it has for improving the quality of life, even for
extending the human life span.
Cold is the word for the reaction he got: The researchers, including the Nobel
laureate who directs the lab, are afraid of adverse public reaction to their
work. Paranoia springs eternal in the human breast -- and is usually
justified, too. The researchers harbor the conviction that the public has
little interest in science other than a blind, unreasoning fear of it; and
they back up their conviction with empirical data.
I am inclined to think that Will's product would have had a healthy effect,
raising public awareness of important scientific issues. I can't say that the
researchers were wrong, though; ignorance and fear feed one another, and the
general public is ignorant and fearful of modern science. But isn't it obvious
that the place to break the cycle is with kids? Where are the chemistry sets
and science kits that past generations of American kids grew up with?
Computers are technology, not science, but they are also a delivery medium for
getting science into the home. That's what Will wanted to do. I hope some of
you want the same thing and have better luck.
I'm launching a newsletter devoted to HyperCard issues, a sort of Writer's
Digest for the stackware market, with scripting tutorials and advanced user
information. (Anyone interested can get more information from me at Card
Tricks, 31 Patrick Road, Santa Cruz, CA 95060.) It was while considering how
best to review stacks for the newsletter that I confronted the dilemma (for a
reviewer, anyway) that software is becoming less like cameras and more like
movies. The appropriate response for computer magazines, at least for computer
software magazines, is to become less like camera magazines and more like film
magazines.
It's still of some value to tick off the features and run the benchmarks, of
course. PC Magazine has made a fortune for Ziff-Davis with those huge tables
of checkmarks detailing every tallyable fact about every laser printer paper
tray.
But the more subjective aspects of software evaluation are growing in
importance. Depth, general appropriateness to the task, ease of use and
learning, responsiveness, and intuitiveness have always been able to make or
break a product, regardless of the number of checkmarks it got in the feature
tables. For some games, responsiveness and realism are everything. But for
many products it is becoming increasingly important to deliver a well-crafted
work rather than a bag of nifty features.
It might not be stretching definitions too far to say that it's the difference
between reviews and criticism. Henry James, who was both author and critic,
said, "The practice of 'reviewing' ... has nothing in common with criticism
.... To criticize is to appreciate, to appropriate, to take intellectual
possession, to establish ... a relation with the criticized thing and to make
it one's own." The software authors of the 1990s will deserve criticism, and
there is no such thing in computer publishing today. There could be. There
should be.









































November, 1989
November, 1989
EDITORIAL


A C Special Issue and A Quick Look at History




Jonathan Erickson


For the last couple of months, it's been busy around here, real busy in fact.
Between getting out the regular issue and the Macintosh special issue (which
you should have in your hands by now if you ordered it) and attending any
number of important conferences (such as OOPSLA), it generally seems that
there aren't enough hours in a day, days in a week, or bits in a byte.
On top of everything, we've been working on a second special issue that I
think really is special -- Dr. Dobb's C Sourcebook for the 1990's. In this C
sourcebook, we'll be looking at the direction C will take over the coming
years while -- in the same issue -- providing the kind of practical C tools
and utilities that you can use today.
What's in the C special issue? For starters, Al Stevens, our C columnist, made
the pilgrimage to AT&T Bell Labs where he talked first with Dennis Ritchie,
then with Bjarne Stroustrup about C and C++, respectively. They discussed how
and why C has evolved, and where the languages will be going in the future.
The interviews aren't Al's only contribution to the issue, however. He also
contributed "A C Programmer's Guide to C++," one of several C++ related
articles, all of which cover the recently released C++ 2.0 specification. In
yet another article, Narain Gehani and Bill Roome, architects of Concurrent C
(and authors of an article in the issue of DDJ you're reading right now),
discuss discrete event simulation and Concurrent C.
Even though the articles alluded to above are generally forward-looking, we
aren't neglecting the tools C programmers need now. To begin with, we
revisited one of the more popular C articles we've run in recent years:
Stewart Nutter's "Automatic Module Control In C" (August 1988). In this latest
incarnation, Stewart's program is a launching pad for Ron Winter, who uses his
version to maintain nearly 100,000 lines of code. Other bread-and-butter
articles in the issue include Paul Anderson's piece on customized memory
allocators, a feature on debugging the stack in C, the source code for a
general-purpose list manager, and more. In total, over 4000 lines of C source
code are listed in the magazine, all of which are available from the DDJ Forum
on CompuServe, the DDJ listing service, or on disk.
When will the C special issue be available? At the same time as our regular
January issue, which translates to around the first or second week in
December.
How do you get your copy of the C Sourcebook? Here's the best part: We're
bundling it with the January issue, so if you're a subscriber, you'll get it
free. Two magazines -- the regular and the special -- for the price of one.
For those who usually get DDJ off the newsstand, the special issue will also
be available on newsstands or it can be ordered directly.


And Now for the History Lesson


In his interviews with Dennis and Bjarne, Al records some of the history of
why C and C++ have evolved the way they have. It's easy to forget that a
technology as young as computing has a history, but it does, and that history
should be documented. Although our mission isn't to provide a narrative of the
computing industry, doing so is a natural outgrowth of our goal (that is, to
serve as a communication medium for the exchange of ideas and information
among serious programmers).
Stop and think about it. Fortran has been around for 30 or so years, Cobol
about the same. This year is the 25th anniversary of Basic. Gary Kildall wrote
PL/1 nearly 20 years ago. C was designed about the same time (and B, C's
predecessor, was created 10 years before that). Pascal, Modula-2, Forth, and a
host of other languages have come on the scene since then. 1990 will be DDJ's
15th year. Tiny Basic, Small C, Dr. Dobb's Journal of Computer Calisthenics &
Orthodontia. Fifteen years is a long time, and a lot of history has passed
between our covers in the meantime. Dr. Dobb's C Sourcebook continues this
tradition.
I'll close by mentioning that Mike Swaine recently told me he and Paul
Freiberger, his coauthor for Fire In the Valley, the wonderful history of PCs
published in 1984, have approached their publisher about updating the book.
Here's my two cents worth to the publisher: Do it. A lot has happened over the
past four or five years that needs to be chronicled. I'll buy a copy of the
second edition, and I'll bet that a lot of others will to .

































November, 1989
LETTERS







It's Binary Trees, Not B-trees


Dear DDJ,
The article, "Setting Precedence," September 1989, misused the term "B-tree"
in place of binary tree. A naive reader might understand these references to
be to binary search trees instead of what they actually are. B-trees and their
variants (B+ trees) are balanced multiway trees used to form indexes for large
data files. The term B-tree was first used in 1972 by its developers, Bayer
and McCreight, to describe a way of forming a multiway tree from the bottom up
so that it was always balanced. Bayer and McCreight have never revealed the
origin of the name; the B could stand for balanced, broad, Boeing (their
employer at the time), Bayer, or could be from some inside joke. By the way,
an exceptionally clear presentation of the B-tree concept is in Folk and
Zoellick's File Structures: A Conceptual Toolkit. This presentation also
includes a large amount of C code.
Terry Johnson
Stillwater, Oklahoma
Dear DDJ,
In "Setting Precedence" by Mark Peterson (September 1989), the author uses the
term "B-tree" when he means "binary tree." Real B-trees are rarely binary and
are always balanced. A definition of B-tree can be found in Donald Knuth's The
Art of Computer Programming, Volume 3/Sorting and Searching, p. 473: A B-tree
of order m is a tree which satisfies the following properties:
i) Every node has <= m sons.
ii) Every node, except for the root and leaves, has >=m/2 sons.
iii) The root has at least 2 sons (unless it is a leaf).
iv) All leaves appear on the same level, and carry no information.
v) A nonleaf node with k sons contains k-1 keys.
Further on in the article are the description and figure of a full 4-bit
binary tree. The author says there are only two such trees, when it seems to
me there are 16. This stems from the fact that there must be one leaf at a
level lower than all others, which is shown in Figure 3 as the node numbered
0. However, that lower leaf could be any value from 0 to 15.
While maintaining balanced trees can improve efficiency, one should not jump
at the chance in every case. With random input, ordered trees tend to stay
remarkably balanced by themselves. In an application such as compiler symbol
tables, additional work for balancing would normally be wasted. Each
application must be evaluated individually to determine if maintaining balance
is worthwhile.
Tim Paterson
Renton, Washington


CP/M Lives!


Dear DDJ,
In his letter printed in your September issue, Arpad Elo Jr. said that he had
decided to write a TECO editor for CP/M. Please allow me to save him the
trouble: I wrote such a beast about 12 years ago. Called TED, this program
could be considered a superset of TECO editors found on TOPS-10 and RT-11
systems (except for the screen-oriented features). Among other things, it has
extended Q-register functions, including the ability to use any Q-register as
the editing buffer.
TED occupies about 13K of code space. Although it was at first a commercial
offering, it has long been freely available through bulletin board systems and
on conferencing systems such as BIX. One caveat: it only runs on Z-80 based
CP/M systems.
Mark E. Mallett
Litchfield, New Hampshire
PS: Though I remember TECO fondly, I'm a longtime EMACS convert.


A* Heuristics


Dear DDJ,
I have just finished reading Randy Nevin's article "Autorouting with the A*
Algorithm" in the September 1989 issue of DDJ. I am very pleased to see
heuristic search and its applications receiving coverage in your fine journal.
I would, however, like to add some comments on A*, and additionally, its
heuristic function, H(x). First, readers may be interested in knowing that the
BFS (Breadth-First Search) algorithm is actually a special case of A*,
obtained by setting H(x) = 0 and distance(pred(x),x) = 1 for all nodes x (the
notation is as defined in Figures 1 and 3 of Nevin's article). Second, the A*
algorithm always terminates with a solution whenever one exists (such an
algorithm is said to be "complete"). This property holds regardless of whether
A* is applied to finite or infinite graphs. Finally, if the heuristic
function, H(x), used by A* is optimistic, that is, if H(x) always
underestimates (or exactly estimates) the cheapest cost of a path going from
node x to a goal node (such a heuristic is typically labeled "admissible"),
then A* is guaranteed to find the optimal solution path whenever a solution
path exists. Further constraints on H(x) yield even stronger results
pertaining to the number of nodes expanded by (and hence the run-time of) A*.
The interested reader is referred to Heuristics: Intelligent Search Strategies
for Computer Problem Solving, by Judea Pearl (Addison-Wesley: Reading, Mass.,
1984).
Andrew R. Spillane
University of Virginia
Charlottesville, Virginia


CA for the Rest of Us


Dear DDJ,
This is just a note to update your readers on the current state of cellular
automation simulators (as mentioned in the sidebar to the article on
"Simulated Annealing" by Michael McLaughlin in your September 1989 issue).
Autodesk Inc. is now shipping CA Lab, also known as Rudy Rucker's Cellular
Automata Laboratory. CA Lab runs on a PC clone with CGA, and updates 320 x 200
pixels several times a second. CA Labs supports either bits per pixel, and new
rules can be programmed in C, Pascal, or Basic.
Rudy Rucker

Mathenaut, Autodesk
Sausalito, California


APL Deserves Some Respect...


Dear DDJ,
In general, I agree with most of the technical articles in journals such as
yours. However, I believe the column by Jeff Duntemann in the August 1989
issue contains at least one misleading statement.
APL is my favorite language, and I don't think it's weird. In fact, it is more
powerful and concise than any language mentioned in your journal. At one of
our SIG/APL meetings some time ago, we were told by an IBM employee that APL
is used there internally for almost everything, then converted to Cobol,
Fortran, etc. as needed for the outside.
W. S. (Bill) Cook
Marina Del Rey, California


.... And So Does Oberon


Dear DDJ,
In his July 1989 "Structured Programming" column, Jeff Duntemann mentioned
Niklaus Wirth's new language, Oberon. Considering the quality of Wirth's
previous languages, I think that any new language he develops should be
examined. I believe the chances are that it will be found to be an excellent
language.
I would like to see a discussion of it in DDJ, or at least a mention of how
more information about it can be obtained.
Brian Jedrick
Nutley, New Jersey


Another Country Heard From


Dear DDJ,
I am writing from distant country, from Poland. In my country there are
subscribers of Dr. Dobb's Journal too. We take pleasure in informing you that
in our opinion DDJ is a journal on very high programming level and full of
tips and tricks.
I am a student and a member of a student computer circle. We have AT and XT
compatibles. We are very interested in C language and, for this reason, in Al
Steven's C column. (Among other programs, we have the incredible Turbo C 2.0).
Al has developed the communication program Smallcom. In our country, private
communication via modems is rising now (there are some mailboxes like Fido
net). His program is ideal for us (the cheapest, but powerful!). We have tried
it to run. But here the troubles have arisen. Unfortunately we don't have some
of the source code for libraries TWRP, context sensitive help, and windows
(DDJ September to December 1988 issues). Could you do us a favor? Can you send
us copies (reprints) of that source code from DDJ Sept. - Dec. 1988 with Al
Stevens's articles? It is the only way for us to obtain needed source code and
his explanation to it (our DDJ is a gift subscription).
That's all for now. I sincerely hope you will be able to help us in these
matters. We wish you to keep the DDJ as it is! Best wishes to you all the
editors of DDJ! I am looking forward to hearing from you.
Artur Terech
Krakow, Poland
DDJ: The source code and copies of the articles you need are in the mail.


Finite State Machines Are Fine by Me


Dear DDJ,
I found the article on procedure tables by Tim Berens in DDJ #154 (August
1989) quite interesting. I have to confess that I have not used pointers to
functions in C as much as I probably should because I found the syntax of
pointers a bit messy and confusing, but they are a powerful tool that I should
have in my programming toolbox. One application for an array of pointers to
functions is to implement a finite state machine.
For the benefit of those not familiar with the finite state machine I should
explain that the concept is that the program has a limited (finite) number of
states or modes and behaves differently in each state or mode. For example I
am using the vi text editor to write this letter. At the moment it is in text
entry mode and if I were to type a capital M, as I have just done, it enters
it in the buffer as part of the document that I am preparing. If I press the
escape key it changes to the command mode and then if I type a capital M it
moves the cursor to the beginning of the line at the middle of the screen.
Whether or not the author of vi thought of it as a finite state machine I do
not know. Each state or mode does its thing but it also must watch for
indications that it should turn matters over to some other state and initiate
the transfer when it is indicated.
I know of at least three ways of programming a finite state machine and there
are probably others that I never heard of. A finite state machine can be
created using gotos, a switch, or a procedure table.
It is said that the goto method can be the most efficient, but it is usually
difficult to understand the code and therefore difficult to maintain unless it
is done in a very disciplined manner. The code for each state is labeled and
whenever a transition to a different state is needed a go to the label for
that state's code is executed.
I have usually used the switch method for my finite state machines. This
requires an integer variable called state or mode or whatever is appropriate
for that application, and a loop containing a switch controlled by this
variable. There is a case for each state and either the code for that state is
put directly into the switch or it is written as a function and called within
that case. When it is necessary to transfer to a different state, the code
within the case can assign a different value to the state variable. Putting
the code directly in the switch cases avoids the overhead of a function call
and is therefore slightly more efficient; however, it can result in an
undesirably long switch. It is possible to put the code directly in the switch
only for the simpler states and use function calls for the states that require
larger amounts of code. Unless efficiency is of great importance, it is
probably better to use function calls and have a shorter, easier to understand
switch.
The procedure table method produces what is probably the most compact and
easiest code to understand. To set up a finite state machine using a procedure
table, a function is needed for each state and a state variable to index the
procedure table in addition to the procedure table itself. The functions
should return an integer type and, besides carrying out the work of that
state, the function must watch for indications that a different state is
required and then return the number for the next state. The procedure table is
an array of function pointers, and by pointing them at the state functions a
number is assigned to each function. The functions can then be called by
number. The computer is perfectly happy with numbers but we humans find it
easier to use names, and of course the functions themselves must have names.
The plan that I use is to name each state, use the name of the state in lower
case for the function name, and assign the same name in uppercase to the
number for that state by using a preprocessor define.
If you have one of the newer compilers this might be a good place to use the
new enumeration feature of the C language. A loop is established containing a
call to one of the state functions by means of the procedure table indexed by
the state variable and returning a new value of the state variable. This one
statement is really all that is necessary in the loop, although in some cases
it may be desirable to add some error detecting code. Unless it is desired to
have the finite state machine run until somebody turns off the computer of
pulls the plug (and this is sometimes done for a dedicated process control
computer), there has to be some way to shut down the finite state machine.
There are at least two ways to do this that I know of and probably others that
I am not familiar with. There could be a terminal state whose function tidies
things up (closes files, removes temporary files, etc.), and then calls exit(
) and so never returns, or there could be a value of the state variable not
corresponding to any function, which would cause the loop to end. Even if the
terminal state method is used, it is unwise to use a forever loop. If the
state variable ever gets out of range, then memory outside the range of the
pointer array would be used as a function pointer and almost anything could
happen when code was entered at some random point or data was interpreted as
code! The only sure thing about such a situation would be that it would be a
disaster. The easiest way to guard against this is to use a while or for loop
with the condition for continuing being that the state variable is in range.
Don't forget that the state variable must be initialized to one of the states
before the loop is entered. Usually the finite state machine always starts
with the same state, or it can be made to start in the same state by having an
initialization state which, often among other duties, has the task of
selecting the starting state and transferring to it.
If such a state is used, it is never entered from any of the other states so
that it executes only at the startup of the finite state machine. It is even
possible to do it all in one for statement. It might go something like:
for(state=0;state>=0&&state < NSTATES;state=(*proctbl[state])( )fp));
While that is the most compact form, it might be clearer as:
state = 0 while(state .= 0 && state < NSTATES) state = (*proctbl[state]) (fp);
Both of these assume that the starting state is in the zero slot of the
procedure table. It would probably be clearer and less restrictive to use the
name defined as the starting state in place of zero for the initialization of
the state variable. The zero in the comparison for the while condition should
remain zero, however. NSTATES is, of course, the number of states assigned
slots in the procedure table, and therefore the number of slots in the
procedure table. Since these slots are numbered zero through NSTATES-1 the
check on the upper limits is state < NSTATES.
When the functions are written, there are some special considerations and
problems pertaining to finite state machines that should be kept in mind. Care
should be exercised in writing the code that determines whether the state
should continue or transfer to another state lest a situation should arise
that would be rejected by all of the states. This could result in an endless
loop of transfers from state to state with none of the states doing anything
about it. Perhaps a special state named "CONFUSION" would be needed to deal
with these unresolvable situations.
Another problem that often comes up when programming finite state machines is
the situation where a state starts to process an item before it discovers, and
possibly before it can discover, that the item requires a different state.
Avoid having the state take any irreversible action before it is certain that
the item being processed is its business. Frequently this problem occurs in
connection with input. A state will read in an item and then find that the
item should have been read and processed by a different state. If the input is
being read in one character at a time then the answer is to use the ungetc( )
function. When larger chunks are being read, such as a line or a record, there
is more of a problem. If it is certain that input is coming from a random
access file then it is possible to use ftell( ) to record the position in the
file before each read and when necessary to use fseek( ) to "push back" what
was read. Unfortunately this does not work for input from most devices or from
pipes.
A more general method is to preread. This works where the unit read is always
the same, one line or one record or whatever. A global buffer, accessible to
all the functions, is provided and the first item is read into it either
before the finite state machine is started, or by the initialization state if
that is used. The processing loop within each state would be a do loop that
would process the item in the buffer, read a new item into the buffer, and
decide whether to loop to process the new item or transfer to another state.
There is always the question of whether or not a finite state machine is the
right program structure to use for a given application. This depends on what
is meant by "right." At one extreme are those that program for efficiency no
matter if that trick that gives some tiny advantage in efficiency makes the
code hard to understand and maintain. At the opposite extreme are those that
seem to have contempt for any considerations of efficiency and for the sake of
understandability and maintainability they follow a rigid set of rules that
permit only a few structures that have been blessed by the high priests of
structured programming. Neither of these are likely to use the finite state
machine concept. One rejects it because any formal structure is nonsense to
him, and the other because it is not one of the sacred structures. Most
programmers are somewhere in between. They realize that some structure is
needed and take the pragmatic approach of if the structure fits the
application use it and if it does not then find one that will fit.
Some say that a finite state machine, even when implemented using the switch
or the procedure table, is really goto programming in disguise. To some extent
this may be true but if the assembly language used to implement the favorites
of the structured programmers, such as the while statement, is examined you
will find gotos. Yet structured programming with its hidden gotos does have
definite advantages. Therefore there must be merit in hiding the gotos. The
merit in hiding the gotos lies in concealing confusing details so that the
overall picture can be seen and comprehended more easily. But hiding the
details does not do much good if what does show is confusing and hard to
understand.
The finite state machine concept is a powerful tool, but like any tool, the
more powerful it is the better the results if used skillfully, and the worse
the results if misused. The finite state machine structure is quite general.
It can be used to program almost anything, but it is the best choice for only
a small fraction of what it can be used for programming. Earlier I described
how a finite state machine could be implemented with gotos. Unfortunately this
process can be reversed and any bit of goto programming can be translated to a
finite state machine. Simply write each labeled hunk of code as a state
function with gotos and places where the program would simply fall through to
another hunk of code replaced by return statements returning the number for
the function containing the code that would be reached. The resulting finite
state machine could then be implemented by either the switch or the procedure
table method. The result should superficially resemble structured programming,
but bowl of spaghetti programming is bad programming no matter how it is
implemented.
This letter is much too long. You probably will not find room to publish my
ramblings. Anyway, writing out my ideas helps me get clear in my own mind so
it is not a total waste of paper.

David S. Tilton
Manchester, New Hampshire
DDJ: Thanks for your explanation of finite state machines, David. Be sure to
check our October 1989 issue for more info on FSMs.



























































November, 1989
DATA-FLOW MULTITASKING


A bridge between single and multiple processor systems




Rabindra P. Kar


Robin is a senior engineer with the Intel Systems Group and author of the
Rhealstone real-time benchmark proposal. He can be reached at 5200 N.E. Elam
Young Parkway, Hillsboro, OR 97124-6497.


In recent years, it has become apparent that computer designs based on the
traditional Von Neumann computer architecture (a single processor stepping
sequentially through a linear memory) are reaching their performance limits.
Performance continues to rise as hardware clock rates get faster and features
such as pipelining and cache memory improve. However, the realization is
dawning that to achieve quantum performance leaps with existing technology,
some form of multiple processor architecture must be used.
In the 1980s, we have seen the appearance of many multiple processor computer
designs, but the number of these computers in actual use is tiny compared with
traditional, single CPU systems. There are several reasons for this.
Multiprocessor systems are expensive. They cost more because they use more
hardware, and do not enjoy the economies of scale that come with high-volume
production and sales. Furthermore, applications and/or systems programmers
must deal with a vastly more complex hardware environment, making the entire
development cycle difficult. Programmers must identify the sections of an
application that can be tackled in parallel, and then devise programs that
will take advantage of the hardware. Little performance gain can be achieved
for applications that are sequential in nature, unless data-flow concepts are
used in software design.
While multiprocessing systems are expensive and relatively uncommon,
multitasking has been with us for decades. With the notable exception of DOS,
most operating systems in common use today (including Unix, IBM's OS/ MVS,
OS/2, iRMX and VRTX in the real-time world) offer multitasking. That brings up
an interesting question. Can applications programs be written on multitasking
systems today that will be easy to port to the multiprocessing systems of
tomorrow? Is this possible even for applications that are sequential in
nature?
This article is about data-flow multitasking, a software concept that may help
bridge the gap between multitasking and multiprocessing capability.


Data-Flow Concepts


Data-flow architectures have been studied for at least three decades, but they
have been overshadowed by the now dominant centralized processing,
control-flow type architectures identified with Von Neumann. In a data-flow
machine, storage (memory) is not viewed as a separate, passive area, but as a
link between data processing operations (instructions). Data processing
operations are enabled (executable) when input data is available to them, and
when their previous output has been read and disposed of. This concept is
similar to "just-in-time" assembly lines pioneered by major Japanese
manufacturers.
The easiest explanation of data-flow concepts is by a simple example -- a
program that computes the roots of a quadratic equation [ax + bx + c = 0].
Figure 1 shows a flowchart for this program in classic sequential style. Each
step in the program cannot be executed until the result(s) of the previous
operation are known. The traditional approach to executing this program is to
have the computer's CPU step through each operation sequentially (read,
compute DET, compute NUM+/-, compute RES) and then loop back to process the
next set of data. The input data, and all intermediate results, are stored in
a part of the computer's memory. This is referred to as the "control-flow"
approach.
Now, let's modify the flowchart in Figure 1. Instead of thinking of each
operation as a subroutine or function that a single CPU must execute
sequentially, imagine a CPU (or hard-wired logic) dedicated to each operation.
Each CPU (or logic module) can be called an operation processor (OP). Think of
memory as a set of mailboxes connecting the OPs as shown in Figure 2.
Each OP is either executing its specific operation, or it is in "wait" mode.
In wait mode, it constantly looks for new data in the mailbox preceding the
operation (its "input mailbox"). The OP also checks that whatever result it
placed in the mailbox following the operation (its "output mailbox") has been
read by the next OP in line. If these two conditions are met, it reads the
input mailbox data, executes the operation, puts the result into the output
mailbox, and goes back to wait mode. This is called the "data-flow" approach
-- each CPU is dedicated to one operation, and data "flows" from one CPU to
another.
In a true data-flow machine, each operation is a basic machine instruction,
such as multiplication or addition. Because we are trying to superimpose a
macroscopic data-flow strategy on a control-flow machine, however, the
computations within each operation (calculating DET or NUM, for example) are
still executed with a control-flow approach. It is only the relationship
between the major operations in our flowchart that simulate a data-flow
architecture.


Multiple Tasks Simulate Multiple CPUs


The data-flow approach is promising because it allows the power of multiple
processors to be harnessed to process a stream of data in a sequential
algorithm. The problem today with this approach is that most programmers only
have access to single CPU (Von Neumann) architectures, whether they are VAX
terminals, workstations, or humble PC/AT clones. Yet much of today's software
may need to be ported to powerful, multiprocessing machines within the next
decade.
Can application programs using data-flow concepts be written today? Yes, if
the multitasking abilities of today's operating systems are used. Each OP in
the above example can be effectively simulated by a separate task that reads
data from a "mailbox," executes the operation, and writes the result to the
next mailbox. Most multitasking operating systems offer features such as
pipes, queues, and others for intertask communication. The programmer can set
these up to perform the mailbox function. If the OS allows tasks to share
memory/data segments, simply assigning some global variables as "mailboxes" is
an effective, high-performance way of passing data between tasks. This
technique is shown in the following section through a programming example. The
programs are based on the flowcharts in Figure 1 and Figure 2.


A Programming Example


Listing One, page 84, shows quad_eq.c, a C program that calculates roots of a
quadratic equation using the "normal" control-flow approach. Input data (100
sets of coefficients) was "created" in this example program simply for
convenience. In actuality the data would be read from some I/O device.
In a single CPU environment, quad_eq.c is a simple and efficient program.
Because this is the environment that 99 percent of today's computer science
majors learned about programming, it also seems like the natural way. However,
if a multi-processor system became available, there is no obvious way to port
this program using parallel computation to speed it up.
Listing Two, page 84, shows dflow.c, a multitasking version of quad_eq.c,
which runs under Intel's real-time multitasking operating system, iRMX II.
Each computational function in quad_eq.c has been turned into a computational
task in dflow.c. For example, the numerator( ) function is replaced by
num_task( ). Global variables act as mailboxes to pass data between tasks.
Essentially, dflow.c implements the flowchart in Figure 2, with computational
tasks substituting for OPs.
In quad_eq.c, main( ) controls the flow of the program -- the sequence in
which each function (subroutine) is called. In dflow.c, main( ) tells the OS
which tasks should be created. After that, it gets out of the way (lowers its
own priority) while the computational tasks (det_task( ), num_task( ), and
res_task( )) process all the input data. The computational tasks delete
themselves when the data is exhausted (when the "a" coefficient of the next
set of data is 0.0), returning control to main( ), which promptly exits.
Global variables det, num, index1, index2, and array y[] constitute the
mailbox information passed from one task to the next. In a data-flow
environment, each operation is performed only when there is new data in a
function's input mailbox, and only if the result of that function's previous
operation has been read from its output mailbox. The variables det and num are
used for this synchronization (num_task( ) will do a computation only when it
sees a non-zero value in det, and zero in num). After the variable det is read
it is set to zero, so det_task( ) will know that its output mailbox was read.
Each task calls rq$sleep (0, &status) if the time is not right to process new
data. The first parameter of rq$sleep is the sleep-time. A sleep-time of zero
tells the operating system that the running task is willing to give up the CPU
in favor of some other equal priority task that needs it.
The program dflow.c is a data-flow implementation of a typical control-flow
program. It is a multitasking solution to a straightforward computation. It is
conceptually simple to port a multitasking solution to a multiprocessor
environment, and reap the enormous performance benefits of concurrent
computation.


Multitasking Versus Single-Task Performance


While data-flow multitasking is a bridge to high performance in a
multi-processor environment, it may lower performance slightly in a single-CPU
system. There are two main reasons for this. The first is that CPU time spent
calling and returning from a function or subroutine is usually less than the
time required for a task switch. For this reason, quad_eq.c will execute
faster than dflow.c on most single CPU multitasking systems. Secondly,
intertask communication mailboxes using global variables may not be feasible
because of OS restrictions, or the necessity to keep the software more modular
and easier to maintain. Using other communication features, such as pipes, may
impose a performance penalty.
In some situations, though, data-flow multitasking can be used to speed up a
program, even on a single CPU system. A good example is when the program must
talk to relatively slow I/O devices. Multitasking can be used to overlap
inherent I/O delays. While one task is waiting for a disk write to complete,
another task may write to the console or read from a keyboard.
Listing Three, page 84, shows res_task2( ), code for a task that could be
added to dflow.c. This code calculates the root of a quadratic. It is
identical to res_task( ) code. Except that it prints its output to a file
rather than the console. Note that det_task( ) and num_task( ) involve no
interaction with any slow I/O device, the CPU probably spends most of its time
waiting for the print operation to complete. With two or more tasks printing
the results on different output devices, I/O delays can be overlapped, and the
program is speeded up.



Multiple Processors


In general, if a computation has been divided into n number of operations, at
least n tasks must be spawned to implement data-flow multitasking. When ported
to a multiprocessor system, each task will be concurrently executed by a
separate processor, thus utilizing n processors. The mailboxes can be global
variables in memory on a common bus (tightly coupled multiprocessing), or
implemented by message passing using a wide variety of interprocessor
connection schemes.
If there are more processors available, you can utilize more than two
processors per operation and share common input and output mailboxes. Assuming
that interprocessor communication overhead is much smaller than the execution
time for any operation, the effects of using multiple processors varies. If
the processor subsystems execute each of the n operations in the same amount
of time, no performance advantage can be gained by using multiple processors
per operation.
In most cases, though, the execution time for each of the n operations will be
unequal. A rule of thumb is that the optimum number of processors per
operation is proportional to the execution time of each operation. If the
program involves three operations and operation 1 takes t milliseconds,
operation 2 takes 2t and operation 3 takes 3t, then optimal performance is
achieved by using one processor to execute operation 1, two processors for
operation 2, and three for operation 3.
When multiple processors are executing the same operation concurrently, data
integrity in the mailboxes becomes a tricky problem. Test and set operations
on mailbox data must be made atomic (uninterruptible) by hardware or software
means. For example, if there are two processors dedicated to calculating the
numerator in Listing Two, each processor must be able to read the value of det
or num and set the value in complete privacy. The other processor must not be
able to access the data between the read and set actions.


Relevance of Real-Time Operating Systems


There are several reasons why a real-time operating system (iRMX II) was used
as a foundation for the data-flow multitasking example in this article. 1.
Many real-time data acquisition and analysis applications are pushing the
limits of a single-CPU solution. Application designers are looking at
real-time multiprocessing to provide high performance at a reasonable cost.
Data-flow multitasking is the first software step towards that goal.
2. iRMX II is part of the family of real-time OSs. The newer members of this
family are oriented towards distributed multiprocessing. This is the very
environment that data-flow multitasking programs should be ported to in order
to use their built-in concurrency.
3. A critical technical advantage of a real-time OS is determinism -- the fact
that task switching happens on predictable events or system calls, and not
according to some application-invisible time-slicing algorithm. This is
extremely important when there is more than one task accessing the same input
and/or output mailbox.
Both res_task( ) and res_task2( ) in Listings Two and Three, read their input
data from the global variable num (their input mailbox). Each one sets num to
0.0 after reading it, to tell num_task( ) that the mailbox has been read. The
C code that does this is:
 res = num / (2 * y[index2].a);
 num = 0.0;
These C statements translate into several machine instructions each. If the OS
implements multitasking by a round-robin time-slicing algorithm, a task switch
could occur anytime between the machine instruction that reads num from memory
and the one that sets it to 0.0. If res_task( ) were so interrupted, and
res_task2( ) was activated before res_task( ) resumed, both tasks would read
the same data and compute the same result twice. A similar problem was
described in the previous section regarding the use of two processors per
operation.
In iRMX II, task switching between equal priority tasks will not occur until a
system call is made. Thus, res_task( ) and res_task2() will not be switched in
or out except at the point where they call rq$sleep. Most real-time kernels
and OSs behave similarly. Some non-real-time OSs do allow you to stop
task-switching between certain boundaries ("critical sections" in OS/2, for
instance), but you must use this feature explicitly. If an OS offers
multitasking without this capability, only one task should be spawned per
operation.


Summary


Data-flow multitasking is a promising solution to the challenges software
faces over the next decade. It involves looking at a sequential program
through new eyes. Each major operation, function, or subroutine is viewed as a
separate task. The activation of tasks is synchronized by the flow of
processed data from one task to another. When ported to a multi-processor
environment, each task can be concurrently executed by one or more processors.
The use of data-flow multitasking may impose some performance penalty in a
computation-intensive single-CPU environment. This can be more than offset if
multitasking is used to overlap the slow I/O operations executed by most
programs. Porting a program to a multi-CPU environment almost always brings a
huge performance boost.
Data-flow multitasking is particularly relevant for real-time applications.
The need for high performance at moderate cost is much more pressing in the
real-time world than it is in batch or interactive computing. Real-time
multitasking operating systems and/or kernels also provide a high level of
determinism, which precludes race conditions associated with multiple task
access data mailboxes.

_DATE-FLOW MULTITASKING_
by Rabindra Kar


[LISTING ONE]

/* quad_eq.c -
 * Calculate roots of quadratic equation: ax + bx + c = 0
 */

#include "stdio.h"
#include "rmx.h"
#include "math.h"

#define NUM_VALS 100

unsigned status;
struct coeffs {double a, b, c;} y[NUM_VALS+1];
double det;
double num;
double res;

/* Calculate determinant */
determinant(i)
unsigned i;
{
 det = y[i].b * y[i].b - 4 * y[i].a * y[i].c;
}

/* Calculate numerator */

numerator(i)
unsigned i;
{
 num = pow(det,0.5) - y[i].b;
}

/* Calculate, print result */
result(i)
unsigned i;
{
 res = num / (2 * y[i].a);
 printf("Coeffs = %.1f,%.1f,%.1f Root = %7.2f\n",
 y[i].a,y[i].b,y[i].c,res);
}

main()
{
 unsigned i;
 /* Create input data. Done for convenience. Could have read the data
 * from any input device.
 */
 for (i = 1; i < NUM_VALS; i++)
 {y[i].a = (double)i; y[i].c = (double)i; y[i].b = 3*(double)i;}

 /* Compute the roots */

 for (i = 1; i < NUM_VALS; i++)
 {
 determinant(i);
 numerator(i);
 result(i);
 }
 printf("\n");
}





[LISTING TWO]

/****************************************************************************\
 * dflow.c -
 * Compute the roots of a quadratic equation using multitasking
 * and data flow concepts.
 * Operating System: iRMX II. Compiler: iC-286 V3.2
\****************************************************************************/

#include "stdio.h"
#include "rmx.h"
#include "math.h"

#define NUM_VALS 100

unsigned status, task_t;
FILE *fp;

/* "union" used to decompose a pointer into segment:offset */
typedef struct {unsigned offset; unsigned sel;} ptr_s;

union { unsigned *pointer; ptr_s ptr; } ptr_u;

/* Input data area */
struct coeffs {double a, b, c;} y[NUM_VALS+1];

/* "DET" mailbox -
 * Output from main task; input to num_task
 */
double det = 0.0;
unsigned index1 = 0;

/* "NUM" mailbox -
 * Output from num_task; input to res_task
 */
double num = 0.0;
unsigned index2 = 0;

/************************ Computational tasks *************************/

det_task()
{ /* Calculate determinant */
 unsigned i;
 for (i = 1; y[i].a != 0.0; i++)
 {
 while (det != 0.0) rq$sleep(0,&status);
 index1++;
 det = y[i].b * y[i].b - 4 * y[i].a * y[i].c;
 }
}

num_task()
{ /* Calculate numerator */
 while (y[index1].a != 0.0)
 {
 while ((det == 0.0) (num != 0.0)) rq$sleep(0,&status);
 index2++;
 num = pow(det, 0.5) - y[index1].b;
 det = 0.0;
 }
 rq$delete$task(0,&status); /* Delete self */
}

res_task()
{ /* Calculate result, print on console */
 double res;
 while (y[index2].a != 0.0)
 {
 while (num == 0.0) rq$sleep(0,&status);
 res = num / (2 * y[index2].a);
 num = 0.0;
 printf("Coeffs = %.1f,%.1f,%.1f Root = %7.2f\n",
 y[index2].a, y[index2].b, y[index2].c, res);
 }
 printf("\n");
 rq$delete$task(0,&status); /* Delete self */
}

/****************************** Main Program ******************************/


main()
{
 unsigned i;
 unsigned short pri;

 /* Create input data. Done for convenience. Could have read the data
 * from any input device.
 */
 for (i = 1; i < NUM_VALS; i++)
 {y[i].a = (double)i; y[i].c = (double)i; y[i].b = 3*(double)i;}

 y[NUM_VALS].a = 0.0; /* 0.0 indicates end of input data */

 /* Place a pointer to any variable in union "ptr_u", so the data segment
 of this program becomes known.
 */
 ptr_u.pointer = &status;

 /* Find the priority level that iRMX accords the main task */
 pri = rq$get$priority (NULL, &status);

 /*
 * Spawn the operation tasks. All have the same priority as main.
 */
 task_t = rq$create$task (pri, (long)det_task, ptr_u.ptr.sel,
 0L, 1024, 1, &status);
 if (status != 0) printf("det_task create error = %u\n",status);

 task_t = rq$create$task (pri, (long)num_task, ptr_u.ptr.sel,
 0L, 1024, 1, &status);
 if (status != 0) printf("num_task create error = %u\n",status);

 task_t = rq$create$task (pri, (long)res_task, ptr_u.ptr.sel,
 0L, 1024, 1, &status);
 if (status != 0) printf("res_task create error = %u\n",status);

 /*
 * Lower main task's priority so main() will be idled by OS while other
 * tasks do computations. Will return here when input data is exhausted.
 */
 rq$set$priority (NULL, (pri+1), &status);

 /*********************************************************************/

 rq$sleep(10, &status); /* Let other tasks complete */
 printf("\n Done! \n");

}





[LISTING THREE]


/*
 * This code is similar to "res_task()" in dflow.c, except
 * that the computed result is output to a file instead of the (default)

 * console screen.
 * This code could be added to dflow.c along with two statements in
 * main() to: 1. create a "res_task2()" task, and 2. open a file for
 * output with fp as its file pointer. By so doing, the I/O delay involved
 * in waiting for console output could be overlapped with file output delays,
 * thus speeding up the program. Of course, the results would then be
 * distributed between the console and the output file.
 */

res_task2()
{ /* Calculate result, print to file */
 double res;

 while (y[index2].a != 0.0)
 {
 while (num == 0.0) rq$sleep(0,&status);
 res = num / (2 * y[index2].a);
 num = 0.0;
 fprintf(fp, "Coeffs = %.1f,%.1f,%.1f root = %7.2f\n",
 y[index2].a, y[index2].b, y[index2].c, res);
 }
 rq$delete$task(0,&status); /* Delete self */
}







































November, 1989
A PARALLEL MAKE WITH DESQVIEW


dvmake can easily run four tasks at once




Mark Streich


Mark is currently a graduate student at the University of Colorado in Boulder,
specializing in compilers and parallel computers. He may be contacted through
the DDJ office.


The make utility started out as a Unix tool for creating executable
applications from a large number of program files. Its main purpose is to
determine which files of an application need to be recompiled and issue
whatever commands are necessary to do so.
As computers with multiple processors became available, some developers
created make utilities capable of executing the various make commands on
separate processors. This allowed each processor to make a separate target,
which resulted in faster make times, although the speedup was not as great as
expected.
Unfortunately, parallel computers have not yet reached the masses, although
multitasking environments and operating systems for single-CPU computers have;
such operating systems, Unix and OS/2 among them, are great -- if you have
several megabytes of memory, a fast hard disk, a 386, and the money to buy the
toolkits.
With this in mind, I decided to develop a parallel make that DDJ readers can
both use and afford. After surveying the available development environments, I
turned to Quarterdeck Office System's (QOS, Santa Monica, Calif.) DESQview,
which can juggle programs running on everything from the original to fast
386-based PCs, to create a parallel make I call dvmake (short for "DESQview
make"). Listing One, page 86, lists the C code for the program.
dvmake requires DESQview in order to run. DESQview-specific applications like
dvmake make extensive use of DESQview's API and usually take advantage of
DESQview's ability to spawn new tasks, display menus, and communicate with
other applications. (Other types of DESQview applications include those that
act differently running under DESQview than they do running under DOS alone,
as well as those that don't take advantage of any of DESQview's capabilities.)


What Is There to Gain?


The idea behind parallel computers, and supposedly behind multitasking
software, is that a computer can get more done in the same amount of time if
it's doing two or more things simultaneously.
make often needs to do several things, and until now it had to do them one
after another. For example, make may have to compile three C source files one
after another, then link the object files together to create an executable
program. Let's assume that it takes 1 minute for each compilation and 1 for
the linking step for a total of 4 minutes.
dvmake, on the other hand, can easily run four tasks at once, so
theoretically, you can remake the entire program in 1 minute. Wrong. The best
that you could hope for is 2 minutes because the linking step can't even start
until all of the object files have been created (meaning the source files have
been completely compiled).
Let's assume it takes 2 minutes to compile a program called farm.c, and that
you only have enough memory to run two tasks at a time. If you compile both a
program called pig.c and another called cow.c in parallel (taking 1 minute),
and then compile farm.c, you spend 3 minutes compiling. If you rearrange the
scheduling, you could compile farm.c and pig.c in parallel, and after pig.c
completes, then compile cow.c. This ordering would take only 2 minutes.
Now that you know how to schedule parallel tasks, you should know that dvmake
does nothing to optimize the scheduling because no matter what you do, a
single-CPU computer requires (at least) the same amount of time to run several
tasks in parallel as it does to run the same tasks sequentially. After all, it
can only execute one instruction at a time.


Who's on First?


Although dvmake does not optimize the scheduling, it does have to guarantee
that a target file's dependents are available before the target can be made.
This means: All object files must be created before they can be linked
together to create the executable program.
dvmake uses a three-step process to determine what needs to be made and then,
using a "greedy" algorithm, schedules those items to be made. (For a
description of the greedy algorithm, see "Simulated Annealing" by Michael
McLaughlin, DDJ, September 1989.)
The first step is similar to the original mk program written by Allen Holub
(DDJ, August 1985 ). A binary tree, sorted on the target file name
(being_made), is created by reading the mkfile. Each node in the tree contains
the target, its dependents, its time and date stamp, and the commands to
remake the target. The dependencies( ) function creates this binary tree.
The binary tree is traversed recursively in make_queue( ), starting at First,
which is either the first target listed in the mkfile or at the target name
you add to the command line when running dvmake. Rather than traversing the
binary tree depth-first, left-to-right, the recursion is based on the
dependents of the target. For a particular target file to be made, it must be
either a direct or indirect dependent of First.
If a target file has dependents that have time stamps more recent than its
own, the target file is added to a queue of items to be made (MkQueue). The
target's time stamp (in the binary tree, not on disk) is changed to fool any
parents of the target file into thinking that they too must be made.
You may wonder why I put the item to make in a queue, when I could just as
easily make the item once I know it has to be made. (In fact, this is exactly
how the mk program works.) The answer is scheduling.
By placing the targets to make in a queue, we can easily find any targets
whose dependents have all been made and start making them in parallel. We're
not stuck with the order provided to us by recursively searching the binary
tree. If some intrepid reader decides to add a smarter scheduling algorithm,
it is much easier to do so with a queue.
dvmake is a sequential program up through the execution of make_queue( ). By
keeping most of the program sequential, it can be debugged using available
tools. Once multiple tasks are created, neither CodeView nor Turbo Debugger
can help, and the QOS API Debugger only tracks the API calls our program
makes, not where our program is executing.
Once make_queue( ) has completed, MkQueue is either empty or contains a list
of target files to make. If there is something to make, we start the third
part of the make process. To help you understand what is about to be
explained, Figure 1 shows when parallel tasks are started and stopped.
The startup( ) function creates two new, concurrently executing tasks. As
Figure 2 illustrates, one task executes the dvmenu( ) function that displays a
small menu that allows you to stop the program. The other task runs the make(
) function that determines which target files can be made.
Both of the new tasks run in parallel with the "main line" program that calls
the output( ) function. output( ) manages the output from the various programs
that are specified in the command blocks of the mkfile.
Figure 2: dvmake being executed

# Make farm.exe using the turbo c
 compiler

farm.exe: cow.obj pig.obj farm.obj
 tlink /x farm cow pig,farm,,cc

pig.obj: pig.c animals.h stdio.h
 tcc -mc -c pig

farm.obj: farm.c stdio.h

 tcc -mc -c farm

cow.obj: cow.c animals.h stdio.h
 tcc -mc -c cow

# Null targets not needed in a "real"
 makefile
# but required for dvmake

animals.h:

stdio.h:

pig.c:

cow.c:

farm.c:

The make( ) function constantly checks to see if there is a target file in
MkQueue that has all of its dependents available, as noted by the made flags
in their nodes. If it finds one, it attempts to start an application window
capable of running the commands listed below the target in the mkfile.
(Starting an application window is identical to using DESQview's Open Window
command.)
Once the application window has been created, another task is started that
executes the dispatch( ) function. dispatch( ) sends commands to the
application window to create the target file, and when all of the commands
have been sent, it sends an Exit command for the window to close itself.
Although I've glossed over many of the details so far, you probably have
realized that there are always three tasks executing concurrently, plus two
tasks for each target file being made. It may seem like overkill, but the
division of labor among the tasks keeps intertask communication (and
interruptions) to a minimum. And there is little housekeeping on our part.
Each task does its work and stops. DESQview gets rid of completed tasks.
More Details.


The DESQview API


Before we get into the details of the program, you should have a basic
understanding of the DESQview API. The API allows you to access numerous
"objects" such as windows, keyboards, and mouse pointers. As I've already
mentioned, you can start or stop separate threads of control known as "tasks."
The API also provides mailboxes for intertask communication or, as we use
them, for semaphores.
Semaphores are extremely important in parallel programs because they guarantee
that only one task can access a device or data structure at a time. dvmake
uses them to ensure that only one task accesses a given queue at any time. If
several tasks attempted to get an item from a queue at the same time, they
might all get the same item, rather than separate items that they really want.
Mailboxes are treated as semaphores by using two API mailbox functions,
mal_lock( ) and mal_unlock( ). If a mailbox is currently locked, any tasks
that attempt to lock it are suspended until it becomes unlocked, at which time
the first task that tried to lock it succeeds.
All API objects are known by their handle, an unsigned long integer. Each
object is accessed using that object's functions. Window function names start
with win_, task functions with tsk_, mailbox functions with mal_, application
functions with app_, and so on. API functions that do not fit one of the
object categories start with api_.


Meet Your Maker


The make( ) function is only concerned with finding the target files in
MkQueue that have all of their dependents available. It removes a target from
the queue, runs through the target's dependent list (depends_on), and either
puts the target back in MkQueue (if the target has unmade dependents) or gets
ready to start making the target.
Making a target requires an application window capable of executing the
commands listed in the mkfile and another task to issue all of the commands to
the application window. The application window is simply a DOS shell that
waits for commands to be entered through the keyboard.
The API function, app_start( ), takes as an argument a pointer to a program
information file (PIF) that has been either loaded into memory from disk or
created in memory, which pif_init( ) does. The PIF structure has the same
layout as the DVP files that DESQview creates for applications in the Open
Window menu.
The app_start( ) function has several problems that need to be addressed.
DESQview can only start as many programs as can fit in RAM. When it attempts
to start a program when there is not enough RAM, DESQview swaps other programs
to disk to make room, causing those programs to stop whatever they were doing.
You should set System Swapping in the Rearrange-Tune Performance menu to No to
avoid the performance penalty of swapping. Or you can simply set the Swapping
option to No for the window in which you run dvmake. If you don't, dvmake may
get swapped to disk, leaving the open application windows idle.
Another problem with app_start( ) is that whenever it is called, any DESQview
menus that are being displayed are removed from the screen as if you had
escaped out of them. Because make( ) sits in a loop while attempting to start
applications, you are essentially forbidden access to any of DESQview's menus,
including resizing or moving windows.
To access DESQview menus, you must first enter DESQview with "Exdev" via the
menu that is put on the screen by dvmenu( ). When you select the menu option,
the make( ) function stops calling app_start( ) until you either quit or
deselect the option. This does not stop the other tasks from running, it
merely keeps any new tasks from starting.
The final problem I had with app_start( ) is that the application window it
creates always becomes the foreground window. This can be partially overcome
by forcing the application to the background with the app_go-back( ) API call.
Even with the small time delay between the two calls, the new application
window can "eat" some keystrokes, including dvmake, that you may have typed
into the foreground window.
After the application window is open, another task is created that runs the
dispatch( ) function. dispatch( ) sends commands to the application by placing
the keystrokes directly into the application's keyboard buffer. It
accomplishes this by using the key_write() API call.
More Details.
dispatch( ) also plays a key role in how you see the output from the various
commands that are running concurrently. Rather than forcing you to make sense
of output being displayed in each of the application windows, dispatch( )
attempts to redirect each command's output to a temporary file. The output( )
function is responsible for displaying the temporary files in the main window.
The output is displayed in a reasonable order, and you can then redirect it to
any file you choose.


Making Something


Running dvmake is extremely simple, although there are many cautionary notes.
You can run dvmake in a DOS window with 64K or more RAM available to it. The
window must be nonswappable, have about 15K of system memory (setable on the
Advanced Options screen from Change a Program), be able to run in the
background, not write directly to the screen, and share the CPU when in the
foreground.
Because dispatch( ) sends commands to application windows through the
keyboard, any programs you use cannot intercept keyboard input. If they do,
the commands that dispatch( ) sends may be received by your program rather
than the application window, and will therefore not be executed as DOS
commands.
Most development tools allow you to specify all needed parameters on the
command line, which you must do. There is little reason to run several
programs at the same time if all of them are expecting input from you, because
you can only type into one of them at a time.
The command to run the program is: dvmake [-w] [-k nnn] [target], and all
three parameters are optional. The -w switch allows you to see the application
windows rather than the default of hiding them. The -k nnn switch allows you
to specify the amount of memory to give to each application window if the
default of 256K is either not enough or too much. You can specify which target
file to make by adding its name to the command line.
You must have the DESQview API C Library to compile the program, although I
have included a runnable version on this month's source code disk. Be sure to
compile the program with byte-alignment ON, or else the PIF structure will not
be correct.



The Bad News


I forewarned that running multiple tasks on a single-CPU computer was
inefficient; I must slightly modify Jerry Pournelle's Law from "One user per
CPU" to "One task per CPU."
To test dvmake, I had it make the Small C compiler using the Small C compiler
and Borland International's Turbo Assembler and linker. I ran the program
using DESQview, Version 2.24, on my Dell Model 310 (20MHz 386) with 2 Mbytes
of memory and Quarterdeck's QEMM memory manager. I also compared dvmake's time
with Borland's make program running under DOS alone and as the only task under
DESQview. Table 1 shows the results. Comparing the times for Borland's make
shows how much overhead is involved to simply have DESQview available. There
was not much additional overhead to run multiple tasks under dvmake. For those
of you who are interested, I was able to get all four parts of Small C
compiling at once.
Table 1: dvmake comparison with make

 Program Time
 (seconds)
----------------------------

 dvmake 332
 make (DESQview) 278
 make (DOS) 131



Extending the Program


You can improve dvmake in any number of ways, many of which I considered
doing, but rejected because the program would have ended up all make and
little DESQview.
You may consider adding the aforementioned optimized scheduling algorithm,
although as I mentioned, you will get little more than the satisfaction of
knowing that the program is not wasting its time.
A more useful addition would be some type of parameter for each target file
that specifies the amount of RAM to allocate for its application window. You
could then tailor each window to the programs that make the target, and get
more tasks running concurrently.


Final Impressions


dvmake should satisfy everyone's desire to put their computer through its
paces. Those of you with 33MHz 386s and 8 Mbytes of RAM should be able to slow
it down to about half the speed of the original PC, creating more leisurely
development cycles.
Overall, I was impressed with how the DESQview API performed. As the source
code shows, it's fairly simple to create multitasking programs. This is a far
cry from the two-page "Hello, World" programs using OS/2, but then again, you
still have to shoehorn your programs into 640K.


References


Baalbergen, Erik H. "Design and Implementation of Parallel Make," Computing
Systems, Vol. 1, No. 2, Spring 1988.
Holub, Allen. "C Chest," Dr. Dobb's Journal, August 1985.
Davis, Stephen R. DESQview, A Guide to Programming the DESQview Multitasking
Environment, (Redwood City, CA: M&T Books, 1989).
Almasi, George S. and Gottlieb, Allan. Highly Parallel Computing, (Redwood
City, CA: Benjamin/Cummings, 1989).


The DESQview Toolset


To create DESQview-aware or DESQview-specific programs, you need some of the
API tools Quarterdeck provides. Although dvmake only uses the API C Library,
QOS also provides an API Reference, Turbo Pascal Library, API Debugger, and
Panel Designer.
The API reference is the basis for the libraries, and includes the assembly
language hooks required to use the API, and a manual that describes the
concepts embodied in the API. You could get by with only this reference, but
you can save a lot of time by also purchasing one of the libraries.
The C and Turbo Pascal Libraries provide the same functions as the reference,
but uses C or Pascal rather than assembly language. The C Library supports
compilers available from Borland, Microsoft, Lattice, Metaware, and Watcom,
and use either the ANSI C definition or the more traditional K&R definition.
The Turbo Pascal Library supports version 4.0 and later.
The API Debugger is neither a source-level nor an assembly-level debugger. It
allows you to debug multiple tasks, trace and break on various types of API
calls, and display the contents of memory or the registers used in the calls.
The source debuggers provided with the supported languages are unable to debug
multiple tasks running under DESQview, so debugging can be difficult.
The API Debugger was unable to let me find out why a window running dvmake
would disappear when the program completed. It would also make itself the
foreground application whenever dvmake started a new application window, which
made it impossible to debug some keyboard-related problems.
The Panel Designer is handy if you need to design help or message windows,
selection boxes, or menus. dvmake only has one menu, and you can get the same
results with library calls, so I decided to ignore the Panel Designer for this
project. -- M.S.



What Makes make Make


The concept of what make does is simple, but is best explained with an
example. For the sake of continuity, I repeat the example provided by Allen
Holub in the August 1985 DDJ. Allen originally wrote mk ("a make with half of
it missing"), upon which dvmake is based.
Suppose you want to create an executable program called farm.exe. farm.exe is
based upon three object files, farm.obj, cow.obj, and pig.obj, all of which
are, in turn, based upon C source files. All three source files have an
#include <stdio.h> statement, and both cow.c and pig.c have an #include
<animals.h> statement.

If you change cow.c, you need to recompile it to create a new cow.obj, and
then link this with the other object files to create farm.exe. If you change
something in animals.h, you need to repeat the same process for both cow.c and
pig.c because they "depend" upon animals.h.
You describe the dependencies in a makefile, or in dvmake's case, a mkfile.
You also include in the mkfile the commands to be executed to either recompile
or relink the appropriate files. dvmake reads the mkfile, figures out what has
changed, decides what needs to be updated, and executes the commands. A mkfile
for the above program is shown in Figure 3. mkfiles have four parts:
You designate a line as a comment by placing a # sign in column 1 of the line.
'Targets" are files that dvmake has to decide whether or not to make. You
specify targets by typing their names followed immediately by a colon.
On the same line as the target, you list the "dependencies." Dependencies are
any files that the target depends upon; that is, if the dependency file
changes, you want the target file updated. A target is not required to have
any dependencies, and if it has none, it is always made.
On the line or lines immediately after the target line, you list the commands
required to make the target. There may be as few as zero, and up to MAXBLOCK
commands listed. An empty line is required to terminate the command block and
separate it from further target lines.
All lines may be continued to the next line by ending with a \ character, just
as in C. For example

 This is\
 one line\
 of text.


For those of you who are make aficionados, note that I do not mention macros.
Macros allow you to simplify many of the command lines. Don't fret over this
loss, however, because as you will see in Table 1, dvmake won't be your
primary make utility. -- M.S.

_A PARALLEL MAKE WITH DESQVIEW_
by Mark Steich


[LISTING ONE]
/*---------------------------------------------------------------------
 * DVMAKE, adapted by Mark Streich
 * Original Mk by Allen Holub, Doctor Dobb's Journal, August 1985
 *
 * Some functions may be specific to Borland Int'l. Turbo C 2.0.
 * DESQview 2.01 (or later), and API C Library required.
 *
 * Compile with byte alignment ON.
 *
 * Run from within a non-swappable DOS window with at least 64K
 * of memory, 15K of system memory, runs in the background and
 * shares the cpu when in the foreground. On non-386 systems,
 * you will have to say that it does not write directly to the screen.
 */

#include <stdio.h> /* for printf(), sprintf(), etc. */
#include <io.h> /* for open(), close(), getftime() */
#include <fcntl.h> /* for O_RDONLY used in open() */
#include <string.h> /* for various string functions */
#include <stdarg.h> /* for variable argument err() */
#include "dvapi.h" /* provided by DESQview API */

/*----------------------------------------------- DEFINES -----------------*/

#define MAXLINE 127 /* Maximum DOS command line length */
#define MAXBLOCK 16 /* Max number of lines in an action */
#define MAXDEP 32 /* Max number of dependencies */
#define MAXFNM 64 /* Max length of file name/dir */
#define COMMENT '#' /* Delimits a comment */
#define MAKEFILE "mkfile" /* Name of makefile */
#define OLDTIME 0x0 /* the Beginning of Time (very old) */
#define NEWTIME 0xFFFFFFFFL /* the End of Time (very young) */

#define DV_VER 0x201 /* DESQview version required: 2.01 */
#define STKSIZE 512 /* size of tasks' local stack */
#define NORMAL 0 /* normal status state */
#define ACCESSDV 1 /* access DESQview, so no new tasks */
#define ABORT 2 /* kill of all processes */


/*----------------------------------------------------------------------
 * iswhite(c) evaluates true if c is white space.
 * skipwhite(s) skips the character pointer s past any white space.
 * skipnonwhite(s) skips s past any non-white characters.
 * waitwhile(event) gives up task's time slice while event is true.
 */

#define iswhite(c) ((c)==' ' (c)=='\t')
#define skipwhite(s) while( iswhite(*s) ) ++s;
#define skipnonwhite(s) while( *s && !iswhite(*s) ) ++s;
#define waitwhile(event) while( event ) api_pause();

/*----------------------------------------------- TYPEDEFS ----------------












 * The entire mkfile is read into memory before it's processed. It's
 * stored in a binary tree composed of the following structures:
 * depends_on and do_this are argv-like arrays of pointers to character
 * pointers. The arrays are null terminated so no count is required.
 * The time field is a 32 bit ulong consisting of the date and time
 * fields returned from DOS. The date and time are concatanated with
 * the date in the most significant bits and the time in the least
 * significant. This way they can be compared as a single number.
 */

typedef struct _tn /* node for dependencies */
{
 struct _tn *lnode; /* pointer to left sub-tree */
 struct _tn *rnode; /* pointer to right sub-tree */
 char *being_made; /* name of file being made */
 char **depends_on; /* names of dependent files */
 char **do_this; /* Actions to be done to make file */
 ulong time; /* time & date last modified */
 ulong apphan; /* what app is making this item */
 char made; /* flag indicating made or not */
 int tsknum; /* what task number was assigned */
}
TNODE;

typedef struct _qn /* queue of items */
{
 void *item; /* item in queue */
 struct _qn *next; /* next item in the queue */
}
QNODE;

typedef struct /* definition of Program Information File (PIF) */
{
 char reserved1[2];

 char prog_title[30]; /* blank filled */
 uint maxmem; /* max memory size in k-bytes */
 uint minmem;
 char program[64]; /* command to start program, 0-terminated */
 char def_drive; /* 'A', 'B', ..., or blank */
 char def_dir[64]; /* default directory, 0-terminated */
 char params[64]; /* parameters, 0-terminated */
 byte init_screen; /* screen mode (0-7) */
 byte text_pages; /* # of text pages used */
 byte first_intr; /* # of first interrupt vector to save */
 byte last_intr; /* # of last interrupt */
 byte logical_rows; /* logical size of window buffer */
 byte logical_cols;
 byte init_row; /* initial row to display window */
 byte init_col;
 uint system_mem; /* system memory in k-bytes */
 char shared_prog[64]; /* shared program file name, 0-terminated */
 char shared_data[64]; /* shared program data, 0-terminated */
 byte control_byte1; /* control byte 1, encoded as follows:
 80H - writes direct to screen












 40H - foreground only
 20H - uses math coprocessor
 10H - accesses system keyboard buffer
 01H - swappable */
 byte control_byte2; /* control byte 2, encoded as follows:
 40H - uses command line parameters
 20H - swaps interrupt vectors */
 char open_keys[2]; /* keys to use for Open Window menu */
 uint script_size; /* size of script buffer in bytes */
 uint auto_pause; /* pause after this many tests for input
 during one clock tick (normally 0) */
 byte color_mapping; /* non-zero to disable auto color mapping */
 byte swappable; /* non-zero if application is swappable */
 char reserved2[3]; /* should be zero */
 byte auto_close; /* non-zero to automatically close on exit */
 byte disk_reqd; /* non-zero if diskette required */
 byte reserved3; /* MUST HAVE VALUE OF 1 */
 byte shared_mem; /* non-zero if program uses shared system mem
*/
 byte physical_rows; /* initial size of physical window */
 byte physical_cols;
 uint max_expanded_mem; /* max amount of expanded mem avail to app */
 byte control_byte3; /* control byte 3, encoded as follows:
 80H - automatically assigns position
 20H - honor maximum memory value
 10H - disallow Close command
 08H - foreground-only when in graphics

 04H - don't virtualize */
 byte key_conflict; /* keyboard conflict (0-4, usually 0) */
 byte graphics_pages; /* # graphics pages used */
 uint system_mem2; /* system memory - overrides system_mem */
 byte init_mode; /* initial screen mode, normally 0FFH */
 char reserved4[22];

} PIF; /* note that the sizeof(PIF) MUST be 416, or else we've made a typo
*/???

/*----------------------------------------------- GLOBAL VARIABLES --------*/

static TNODE *Root = NULL; /* Root of file-name tree */
static FILE *Makefile; /* Pointer to opened makefile */
static int Inputline = 1; /* current input line number */
static char *First = NULL; /* Default file to make */

static char ShowWin = 0; /* Display tasks? */
static char Parallel = 0; /* Are we multitasking yet? */
static char Status = NORMAL; /* processing status */

static char CurDir[MAXFNM]; /* Directory called from */
static char Error[MAXLINE] = "";/* Error saved for later printing */

static int RunCnt = 0; /* how many tasks running */
static int MemSize = 256; /* default task memory size */
static int ReDirLen = 0; /* length of redirection file */













static QNODE *MkQueue = NULL; /* queue of items to make */
static QNODE *TskQueue = NULL; /* queue of tasks to run */
static QNODE *OutQueue = NULL; /* queue of files to output */

static ulong TskQueueLock; /* semaphore for TskQueue */
static ulong OutQueueLock; /* semaphore for OutQueue */
static ulong AllocLock; /* semaphore for malloc/free */
static ulong MainWin; /* handle of Main Window */
static ulong MenuTsk; /* handle of Menu Task */
static ulong MakeTsk; /* handle of Make Task */

static PIF Pif; /* default Prog Info File */
static int Lpif; /* length of the PIF */

/*----------------------------------------------- ERROR ROUTINES ----------*/

void err( char *msg, ... )
{
 /* print the message and optional parameter and either
 * stop immediately if we haven't started up parallel tasks,

 * or just set our status to ABORT and stop later
 */

 static char temp[MAXLINE];
 va_list argptr;

 /* Print the error message, if we haven't already, and abort */
 if (!strlen(Error))
 {
 /* print the location of the error in the mkfile */
 sprintf(Error,"Mk (%s line %d): ", MAKEFILE, Inputline );

 va_start(argptr,msg);
 vsprintf(temp,msg,argptr); /* print the error message */
 va_end(argptr);

 strcat(Error,temp); /* and concatenate the two */
 }

 if (Parallel) /* are we multitasking? */
 {
 /* notify everyone that there are problems, but don't stop yet */
 api_beginc();
 Status = ABORT;
 api_endc();
 }
 else /* not multitasking, so we can STOP */
 {
 fprintf(stderr,"%s",Error); /* print the error message */

 mal_free(TskQueueLock); /* get rid of our semaphores */
 mal_free(OutQueueLock);
 mal_free(AllocLock);













 api_exit(); /* tell DESQview we're done */
 exit(1); /* and STOP */
 }
}

/*----------------------------------------------- MEMORY ROUTINES ---------*/

void *gmem( int numbytes )
{
 /* Get numbytes from malloc. Print an error message and
 * abort if malloc fails, otherwise return a pointer to
 * the memory. Uses semaphores because malloc() not re-entrant.
 */


 void *p;
 extern void *malloc();

 mal_lock(AllocLock); /* get access to heap */
 p = malloc(numbytes); /* grab some memory */
 mal_unlock(AllocLock); /* free access to heap */

 if (p == NULL) /* were we successful */
 err("Out of memory");

 return( p );
}

void fmem( void *ptr )
{
 /* Frees memory pointed to by ptr. */

 extern void free();

 mal_lock(AllocLock); /* get access to heap */
 if (ptr) free(ptr); /* free the memory if ptr not NULL */
 mal_unlock(AllocLock); /* free access to heap */
}

/*----------------------------------------------- QUEUE ROUTINES ----------*/

void enqueue(QNODE **queue,void *item) /* add item to the queue */
{
 QNODE *qptr;

 /* get memory for new node */
 if ( (qptr = (QNODE *) gmem(sizeof(QNODE))) == NULL )
 err("Out of memory");
 else
 {
 qptr->item = item; /* add the item */
 qptr->next = *queue; /* point to the next item in the queue, if any
*/
 *queue = qptr; /* point to the new front of the queue */
 }












}

void *dequeue(QNODE **queue) /* return an item from the queue */
{
 QNODE *qptr,*qptr2 = NULL;
 void *iptr;


 if (*queue == NULL) /* is the queue empty? */
 return( NULL );
 else
 {
 /* find the end of the queue */
 for (qptr = *queue; qptr->next != NULL; qptr = qptr->next)
 qptr2 = qptr; /* qptr2 points to qptr's predecessor */

 iptr = qptr->item; /* get the item to return */

 fmem( qptr ); /* remove the last item from the queue */
 if (qptr2 != NULL)
 qptr2->next = NULL;
 else
 *queue = NULL; /* nothing left in queue */

 return( iptr ); /* return a pointer to the item removed */
 }
}

int inMkQueue(char *being_made)
{
 /* see if "being_made" item is already in the MkQueue */
 QNODE *qptr;

 if (MkQueue != NULL) /* is the queue empty? */
 for (qptr = MkQueue; qptr != NULL; qptr = qptr->next)
 if (strcmp(((TNODE *)qptr->item)->being_made,being_made) == 0)
 return(1); /* being_made is in the queue */

 return( 0 ); /* MkQueue is empty or being_made not in it */
}

/*----------------------------------------------- INITIALIZE PIF ----------*/

void init_pif(PIF *pif,int *lenpif,char *title,int rows,int cols)

 /* Initialize the Program Information File to start the new application.
 * By default it is just a non-swappable dos window with 256K.
 */
{
 *lenpif = sizeof(PIF);

 /* set defaults now, and particulars later */
 memset(pif->reserved1,0,2);
 memset(pif->prog_title,' ',30); /* blank filled */
 strncpy(pif->prog_title,title,strlen(title));













 pif->maxmem = MemSize; /* memory required for app in k-bytes */
 pif->minmem = MemSize;
 strcpy(pif->program,""); /* command to start program, 0-terminated
*/
 pif->def_drive = CurDir[0]; /* ' ', 'A', 'B', ... */
 strcpy(pif->def_dir,CurDir+2); /* default directory, 0-terminated */
 strcpy(pif->params,""); /* parameters, 0-terminated */
 pif->init_screen = 0x7F; /* screen mode (0-7) (??) */
 pif->text_pages = 1; /* # of text pages used */
 pif->first_intr = 0; /* # of first interrupt vector to save */
 pif->last_intr = 255; /* # of last interrupt */
 pif->logical_rows = rows; /* logical size of window buffer */
 pif->logical_cols = cols;
 pif->init_row = 0; /* initial row to display window */
 pif->init_col = 0;
 pif->system_mem = 0; /* system memory in k-bytes */
 pif->shared_prog[0] = 0; /* shared program file name, 0-terminated
*/
 pif->shared_data[0] = 0; /* shared program data, 0-terminated */
 pif->control_byte1 = 0x20; /* control byte 1, encoded as follows:
 80H - writes direct to screen
 40H - foreground onlay
 20H - uses math coprocessor
 10H - accesses system keyboard buffer
 01H - swappable */
 pif->control_byte2 = 0x400x20; /* control byte 2, encoded as follows:
 40H - uses command line parameters
 20H - swaps interrupt vectors */
 memset(pif->open_keys,' ',2); /* keys to use for Open Window menu */
 pif->script_size = 256; /* size of script buffer in bytes */
 pif->auto_pause = 0; /* pause after this many tests for input
 during one clock tick (normally 0) */
 pif->color_mapping = 0; /* non-zero to disable color mapping */
 pif->swappable = 0; /* non-zero if application is swappable */
 memset(pif->reserved2,0,3); /* should be zero */
 pif->auto_close = 0; /* non-zero to close on program exit */
 pif->disk_reqd = 0; /* non-zero if diskette required */
 pif->reserved3 = 1; /* MUST HAVE VALUE OF 1 */
 pif->shared_mem = 0; /* non-zero if prog uses shared memory */
 pif->physical_rows = 0; /* initial size of physical window */
 pif->physical_cols = 0; /* 0's allow DV to set */
 pif->max_expanded_mem = 65535u; /* max amt of expanded mem avail to app */
 pif->control_byte3 = 0x800x10; /* control byte 3, encoded as follows:
 80H - automatically assigns position
 20H - honor maximum memory value
 10H - disallow Close command
 08H - foreground-only when in
graphics
 04H - don't virtualize */
 pif->key_conflict = 0; /* keyboard conflict (0-4, usually 0) */
 pif->graphics_pages = 0; /* # graphics pages used */
 pif->system_mem2 = 0; /* system memory - overrides system_mem */
 pif->init_mode = 0xFF; /* initial screen mode, normally 0FFH */
 memset(pif->reserved4,0,22);













}

/*----------------------------------------------- GENERATE TEMP FILE NAME --*/

char *gen_name(int tsknum,int cmdnum)
{
 /* Generate a new output file name, d:\dir\DVMKxxyy.$$$, where
 d:\dir\ is the directory in which the "dvmake" was started,
 xx = task number, and yy = command number (both in hex).
 Returns a pointer to the newly allocated file name.
 */

 char *new_name; /* generated name */
 char tsk_cmd[5]; /* task/command number string */

 if ( (new_name = (char *) gmem(MAXFNM)) == NULL )
 err("Out of memory");
 else
 {
 strcpy(new_name,CurDir); /* directory name */

 if (CurDir[strlen(CurDir)-1] != '\\')
 strcat(new_name,"\\"); /* add backslash to dir */

 strcat(new_name,"DVMK"); /* first 4 chars of name */

 sprintf(tsk_cmd,"%02x%02x",tsknum,cmdnum);
 strcat(new_name,tsk_cmd); /* last 4 chars of name */

 strcat(new_name,".$$$"); /* add an extension */
 }
 return( new_name );
}

/*----------------------------------------------- MENU TASK ---------------*/

int dvmenu( void )
{
 /* display a menu to control the status of the make, and don't
 * quit until someone wakes me up with tsk_post(MenuTsk)
 */

 ulong kbd,win; /* handles for keyboard, window */
 ulong whichobj; /* handle of object that has input */
 char *kbuf; /* message buffer */
 int klen, /* message length */
 state; /* state of selected item */

/* this string defines the contents of the menu */
static char mkmenu[] = "\
 Access DV A \

 Quit Q ";

static char mkmenutbl[] =












{
 ftab(2,FTH_KEYSELECT+FTH_MODIFIED+FTH_AUTORESET,0,0,9,2),
 0,0,0,13,FTE_SELECT,'A',1,0,
 1,0,1,13,FTE_SELECT,'Q',1,0,
};

 win = win_new("DVMAKE",6,2,14); /* get a new window */
 win_logattr(win,1); /* and set its logical attributes */
 win_attr(win,1);
 win_disallow(win,ALW_HSIZE); /* do not allow resizing menu */
 win_disallow(win,ALW_VSIZE);
 win_swrite(win,mkmenu); /* write the contents and */
 win_stream(win,mkmenutbl); /* field table to the menu window */
 fld_marker(win,175); /* set the selected field marker */

 kbd = key_new(); /* get a keyboard for the menu */
 key_open(kbd,win);
 key_addto(kbd,KBF_FIELD); /* and put it into field mode */

 /* put and display the menu in the top right corner of the main window */
 win_poswin(win,MainWin,PSW_LEFT,0,PSW_RIGHT,0,0);
 win_unhide(win);
 win_top (win); /* make sure it's the one on top */

 /* go until someone wakes me up with tsk_post(MenuTsk) */
 for (whichobj = 0; whichobj != tsk_me(); )
 {
 /* wait for something to show up in our object queue */
 if ((whichobj = obq_read()) == kbd)
 {
 key_read(kbd,&kbuf,&klen);/* see what field was selected */
 state = qry_type(win,(int) *kbuf); /* is it ON or OFF */

 if ((int) *kbuf == 1 && /* was "Access DV" toggled? */
 Status != ABORT)
 {
 api_beginc(); /* make sure err() hasn't aborted */
 if (Status != ABORT) /* in the interim */
 Status = (state == FLT_SELECT ? ACCESSDV : NORMAL);
 api_endc();
 }
 else /* selected "Quit" */
 {
 Status = ABORT;

 fld_reset(win); /* show only Quit as selected */
 fld_type(win,2,FLT_SELECT);
 }
 }
 }

 /* get rid of menu window, keyboard */
 key_free(kbd);
 win_free(win);
}













/*----------------------------------------------- GET TIME ROUTINE --------*/

ulong gtime( char *file )
{
 /* Return the time and date for file, or if the file
 * does not exist, assume it is very old
 *
 * The DOS time and date are concatanated to form one
 * large number.
 * THIS ROUTINE IS NOT PORTABLE (because it assumes a 32
 * bit ulong to provide for the time functions).
 */

 short handle; /* Place to remember file handle */
 struct ftime time; /* date/time structure */
 ulong utime = 0; /* use to convert time to ulong */
 char xtern *searchpath(); /* search PATH for the file, */
 /* defined in TURBO C's dir.h */

 if ((handle = open(searchpath(file),O_RDONLY)) == -1)
 {
 /* File doesn't exist. Return a very old date & time */
 return( OLDTIME );
 }
 else
 {
 /* File exists, so get the time */
 if ( getftime(handle,&time) )
 err("DOS returned error from date/time request");

 if ( close(handle) )
 err("DOS returned error from file close request");

 /* pack the time into an unsigned long for comparisons */
 utime = (ulong) time.ft_year << 25;
 utime = (ulong) time.ft_month << 21;

 utime = (ulong) time.ft_day << 16;
 utime = (ulong) time.ft_hour << 11;
 utime = (ulong) time.ft_min << 5;
 utime = (ulong) time.ft_tsec;

 return( utime );
 }
}

/*----------------------------------------------- CHAR STORAGE ------------*/

char **stov( char *str, int maxvect )
{
 /* "str" is a string of words separated from each other by
 * white space. Stov returns an argv-like array of pointers
 * to character pointers, one to each word in the original
 * string. The white space in the original string is replaced












 * with nulls. The array of pointers is null-terminated.
 * "Maxvect" is the number of vectors in the returned
 * array. The program is aborted if it can't get memory.
 */

 char **vect, **vp;

 vp = vect = (char **) gmem( (maxvect + 1) * sizeof(str) );
 while ( *str && --maxvect >= 0 )
 {
 skipwhite(str);
 *vp++ = str;
 skipnonwhite(str);
 if ( *str )
 *str++ = 0;
 }

 *vp = 0;
 return( vect );
}

/*----------------------------------------------------------------------*/

char *getline( int maxline, FILE *fp )
{
 /* Get a line from the stream pointed to by fp.
 * "Maxline" is the maximum input line size (including the
 * terminating null. A \ at the end of line is
 * recognized as a line continuation, (the lines
 * are concatanated). Buffer space is gotten from gmem().

 * If a line is longer than maxline it is truncated (i.e.
 * all characters from the maxlineth until a \n or EOF is
 * encountered are discarded.
 *
 * Returns: NULL on a malloc failure or end of file.
 * A pointer to the malloced buffer on success.
 */

 static char *buf;
 char *bp;
 int c, lastc;

 /* Two buffers are used. Here, we are getting a worst-case buffer
 * that will hold the longest possible line. Later on we'll copy
 * the string into a buffer that's the correct size.
 */

 if ( (bp = buf = (char *) gmem(maxline)) == NULL )
 return( NULL );

 while(1)
 {
 /* Get the line from fp. Terminate after maxline
 * characters and ignore \n following a \.












 */

 Inputline++; /* Update input line number */

 for ( lastc=0; (c = fgetc(fp)) != EOF && c!='\n'; lastc=c)
 if ( --maxline > 0 )
 *bp++ = c;

 if ( !( c == '\n' && lastc == '\\') )
 break;

 else if ( maxline > 0 ) /* erase the \ */
 --bp;
 }
 *bp = 0;

 if ( (c == EOF && bp == buf) 
 (bp = (char *) gmem((int) (bp-buf)+1)) == NULL )
 {
 /* If EOF was the first character on the line or
 * malloc fails when we try to get a buffer, quit/
 */


 fmem(buf);
 return( NULL );
 }

 strcpy( bp, buf ); /* Copy the worst-case buffer to the one */
 /* that is the correct size and ... */
 fmem( buf ); /* free the original, worst-case buffer, */
 return( bp ); /* returning a pointer to the copy. */
}

/*----------------------------------------------------------------------*/

char **getblock( FILE *fp )
{
 /* Get a block from standard input. A block is a sequence of
 * lines terminated by a blank line. The block is returned as
 * an array of pointers to strings. At most MAXBLOCK lines can
 * be in a block. Leading white space is stripped.
 */

 char *p, *lines[MAXBLOCK], **blockv = lines;
 int blockc = 0;

 do {
 if ( (p = getline(MAXLINE,fp)) == NULL)
 break;

 skipwhite(p);

 if ( ++blockc <= MAXBLOCK )
 *blockv++ = p;












 else
 err("action too long (max = %d lines)",MAXBLOCK);

 } while ( *p );

 /* Copy the blockv array into a safe place. Since the array
 * returned by getblock is NULL terminated, we need to
 * increment blockc first.
 */

 blockv = (char **) gmem( (blockc + 1) * sizeof(blockv[0]) );
 movmem( lines, blockv, blockc * sizeof(blockv[0]) );
 blockv[blockc] = NULL;

 return( blockv );
}


/*----------------------------------------------------------------------*/

TNODE *makenode( void )
{
 /* Create a TNODE, filling it from the mkfile, and return a
 * pointer to it. Return NULL if there are no more objects
 * in the makefile.
 */

 char *line, *lp;
 TNODE *nodep;

 /* First, skip past any blank lines or comment lines.
 * Return NULL if we reach end of file.
 */

 do {
 if ( (line = getline(MAXLINE,Makefile)) == NULL )
 return( NULL );

 } while ( *line == 0 *line == COMMENT );

 /* At this point we've gotten what should be the dependency
 * line. Position lp to point at the colon.
 */

 for ( lp = line; *lp && *lp != ':'; lp++ )
 ;

 /* If we find the colon position, lp to point at the first
 * non-white character following the colon.
 */

 if ( *lp != ':' )
 err( "missing ':'"); /* This will abort the program */
 else
 for ( *lp++ = 0; iswhite(*lp); lp++ )












 ;

 /* Allocate and initialize the TNODE */

 nodep = (TNODE *) gmem( sizeof(TNODE) );
 nodep->lnode = NULL;
 nodep->rnode = NULL;
 nodep->being_made = line;
 nodep->time = gtime( line );

 nodep->depends_on = stov( lp, MAXDEP );
 nodep->do_this = getblock( Makefile );
 nodep->made = 1; /* assume has already been made, but later change
*/
 nodep->apphan = 0;
 nodep->tsknum = 0;

 return( nodep );
}

/*----------------------------------------------- TREE ROUTINES -----------*/

TNODE *find( char *key, TNODE *root )
{
 /* If key is in the tree pointed to by root, return a pointer
 * to it, else return 0.
 */

 int notequal;

 if ( !root )
 return( 0 );

 if ( (notequal = strcmp(root->being_made,key)) == 0 )
 return( root );

 return( find( key, (notequal > 0) ? root->lnode : root->rnode) );
}

/*----------------------------------------------------------------------*/

int tree( TNODE *node, TNODE **rootp )
{
 /* If node's key is in the tree pointed to by rootp, return 0
 * else put it into the tree and return 1.
 */

 int notequal;

 if ( *rootp == NULL )
 {
 *rootp = node;
 return( 1 );
 }













 if ( (notequal = strcmp( (*rootp)->being_made, node->being_made)) == 0 )
 return( 0 );


 return( tree( node, notequal > 0 ? &(*rootp)->lnode : &(*rootp)->rnode)
);
}

/*----------------------------------------------------------------------*/

int dependencies( void )
{
 /* Manufacture the binary tree of objects to make. First
 * is a pointer to the first target file listed in the
 * makefile (ie. the one to make if one isn't explicitly
 * given on the command line. Root is the tree's root pointer.
 */

 TNODE *node;

 if ( (node = makenode()) != NULL )
 {
 /* has First been assigned a value yet? */
 if (First == NULL)
 First = node->being_made;

 if ( !tree(node, &Root) )
 err("Can't insert first node into tree !!!\n");

 while ( (node = makenode()) != NULL )
 if ( !tree( node, &Root ) )
 fmem( node );
 return( 1 );
 }

 return( 0 );
}

/*----------------------------------------------- CREATE MAKE QUEUE -------*/

void make_queue( char *what )
{
 /* Simulate a sequential make, building up a queue of items to make.
 * The dependency tree is descended recursively.
 */

 TNODE *snode; /* Source file node pointer */
 TNODE *dnode; /* dependent file node pointer */
 int doaction = 0; /* If true do the action */
 static char *zero = (char *)0;
 char **linev = &zero;

 if ( (snode = find(what, Root)) == NULL )
 err("Don't know how to make <%s>\n", what );














 if ( !*(linev = snode->depends_on)) /* If no dependencies */
 ++doaction; /* always do the action */

 for ( ; *linev; linev++ ) /* Process each dependency */
 {
 make_queue( *linev );

 if ( (dnode = find(*linev, Root)) == NULL )
 err("Don't know how to make <%s>\n", *linev );

 if ( snode->time <= dnode->time )
 {
 /* If dependent node is more recent (time is greater)
 * than the source node, do something. If the times
 * are equal, assume that neither file exists but that
 * the action will create them, and do the action.
 */
 ++doaction;
 }
 }

 if ( doaction ) /* are we going to do anything */
 {
 /* are there any commands, and is this node not in MkQueue */
 if ( snode->do_this && *snode->do_this && **snode->do_this &&
 !inMkQueue(snode->being_made) )
 {
 snode->time = NEWTIME; /* Assume the time will change */
 snode->made = 0; /* This item has not been made yet */
 enqueue(&MkQueue,snode); /* Add to list of things to make */
 }
 }
}

/*----------------------------------------------- DISPATCH TASK -----------*/

int dispatch( void )
{
 /* Grab a job from the TskQueue, send all of the commands to that
 * application through its keyboard, close the app, and return.
 */

 TNODE *snode; /* Source file node pointer */
 char **linev; /* Command to execute */
 char *outfnm; /* file to redirect output to */
 int cmdcnt; /* how many commands were redirected */
 ulong keyhan; /* handle for keyboard */
 static int taskcnt = 0; /* give each task a unique identifier */
 void send_keys(ulong keyhan, char *keys);

 mal_lock(TskQueueLock); /* grab the task queue */
 snode = (TNODE *) dequeue(&TskQueue); /* get a task from the queue */
 mal_unlock(TskQueueLock); /* let go of the task queue */














 keyhan = key_of(snode->apphan); /* get keyboard handle */

 api_beginc();
 snode->tsknum = taskcnt++; /* assign a new task number */
 api_endc();

 /* get window to "escape out of" any commands that creeped in at startup
*/
 send_keys(keyhan,"\x1B"); /* "\x1B" is the Escape key character */

 for (linev = snode->do_this, cmdcnt = 0;
 *linev && **linev && Status != ABORT; linev++)
 {
 send_keys(keyhan,*linev); /* send the command */

 /* if the command doesn't already redirect output, do so */
 if (strchr(*linev,'>') == NULL && strlen(*linev)+ReDirLen < MAXLINE)
 {
 /* get a new output file */
 if ((outfnm = gen_name((int) snode->tsknum,cmdcnt++)) != NULL)
 {
 send_keys(keyhan," > "); /* send a redirect command */
 send_keys(keyhan,outfnm);
 fmem(outfnm); /* free up the temp file name */
 }
 else
 --cmdcnt; /* we couldn't redirect anything */
 }

 send_keys(keyhan,"\r"); /* send return key to run the command */
 }

 send_keys(keyhan,"exit\r"); /* make window exit itself when done */
 waitwhile(api_isobj(snode->apphan)); /* and wait for task to go away */

 snode->made = 1; /* show that this item was made */

 if (cmdcnt > 0) /* were any commands redirected? */
 {
 mal_lock(OutQueueLock); /* get access to output queue */
 enqueue(&OutQueue,snode); /* add to output list */
 mal_unlock(OutQueueLock); /* free access to queue */
 }

 api_beginc(); /* this task is done, so */
 --RunCnt; /* update # of running tasks */
 api_endc();

}

void send_keys(ulong keyhan, char *keys)
{
 /* send string to keyboard, pausing every 5 characters to avoid
 * overflowing any buffers. You could try different values, but
 * 5 seems to be a reasonable compromise between speed and making












 * sure no keys are lost.
 */

 int keyctr;

 if (keys != NULL)
 while (*keys)
 {
 waitwhile(key_sizeof(keyhan)); /* wait for keyboard space */
 for (keyctr = 0; *keys && keyctr < 5; keyctr++)
 key_write(keyhan,keys++,1,0); /* send a single keystroke */
 }
}

/*----------------------------------------------- MAKE TASK ---------------*/

int make( void )
{
 /* Actually execute the commands. Items are removed from the
 * MkQueue queue, and placed back on the queue if its dependents
 * haven't been made yet. This task runs in parallel with others.
 */

 TNODE *snode; /* Source file node pointer */
 char **linev; /* Command to execute */
 char doaction; /* Should we do anything? */
 char *stack; /* stack for dispatch() tasks */

 /* while there are still items to make */
 while ( (snode = (TNODE *) dequeue(&MkQueue)) != NULL && Status != ABORT )
 {
 /* make sure all dependents have been made before acting */
 for (linev = snode->depends_on, doaction = 1; *linev; linev++ )
 if (!(find(*linev,Root)->made))
 doaction = 0;

 if (!doaction)
 {
 enqueue(&MkQueue,snode); /* put the item back on the queue */
 api_pause(); /* and give up our time slice */

 }
 else
 {
 /* put the item on the task queue, and start up its dispatch */
 mal_lock(TskQueueLock); /* grab the task queue */
 enqueue(&TskQueue,snode); /* put a task on the queue */
 mal_unlock(TskQueueLock); /* let go of the task queue */

 /* get stack space for the command dispatcher */
 if ( (stack = (char *) gmem(STKSIZE)) == NULL )
 err("Out of memory");

 /* keep trying to start a new application
 * unless user aborts or wants to access DESQview menu












 */
 while ( Status != NORMAL 
 (snode->apphan = app_start((char *) &Pif,Lpif)) == 0 )
 {
 if (Status == ABORT) /* get out now! */
 break;
 else if (RunCnt == 0 && /* can't we even get 1 running? */
 Status == NORMAL && snode->apphan == 0)
 err("Cannot start a single process");
 else
 api_pause(); /* just give up our time slice */
 }

 if (snode->apphan != 0)
 {
 /* either hide or put the application in the background */
 if (ShowWin)
 app_goback(snode->apphan);
 else
 app_hide(snode->apphan);

 /* start up another command dispatcher, with no window */
 tsk_new(dispatch,stack,STKSIZE,"",0,0,0);

 api_beginc();
 ++RunCnt; /* we've started another task */
 api_endc();
 }
 }
 }

 waitwhile(RunCnt > 0); /* wait for all tasks to finish */
}


/*----------------------------------------------- OUTPUT TASK -------------*/

void output( void )
{
 /* Send files (created by redirecting output to DVMKxxyy.$$$) to
 * standard output in same order they were created, keeping all
 * output for a given dependency together.
 */
 TNODE *onode; /* Output node pointer */
 FILE *infile; /* file to read input from */
 char *infnm; /* file name of input */
 char **linev; /* pointer to commands */
 int ch,counter;

 /* loop until everything is finished */
 while ((api_isobj(MakeTsk) OutQueue != NULL) && Status != ABORT)
 {
 mal_lock(OutQueueLock); /* get access to output queue */

 if ( (onode = (TNODE *) dequeue(&OutQueue)) == NULL )












 {
 mal_unlock(OutQueueLock); /* free access to queue */
 api_pause(); /* and give up our time slice */
 } /* because there's nothing to output */
 else
 {
 mal_unlock(OutQueueLock); /* free access to queue */

 for ( linev = onode->do_this, counter = 0;
 *linev && **linev; linev++ )
 {
 printf("\n%s\n",*linev); /* print the command executed */

 /* make sure dvmake was able to redirect output */
 if (strchr(*linev,'>') == NULL &&
 strlen(*linev)+ReDirLen < MAXLINE)
 {
 /* get the file name and open the file */
 infnm = gen_name((int) onode->tsknum,counter++);

 /* lock access to memory (fopen() gets memory) */
 mal_lock(AllocLock);

 if (infnm == NULL (infile = fopen(infnm,"r")) == NULL)
 {
 mal_unlock(AllocLock); /* free access to memory */


 /* not a drastic error, but tell user */
 printf("Can't open %s\n",infnm);
 }
 else /* we successfully opened the temporary file */
 {
 mal_unlock(AllocLock); /* free access to memory */

 /* copy the file to stdout */
 while ((ch = fgetc(infile)) != EOF)
 putchar(ch);

 /* get access to memory (fclose() releases memory) */
 mal_lock(AllocLock);

 fclose(infile); /* close, and */
 remove(infnm); /* erase the temporary file */

 mal_unlock(AllocLock); /* free access to memory */
 }

 fmem(infnm);
 }
 }
 }
 }

 /* if user aborts, get rid of remaining temporary files */












 if (Status == ABORT)
 {
 waitwhile(api_isobj(MakeTsk));/* wait for make() to stop first */

 mal_lock(OutQueueLock); /* gain access to queue */

 while ( (onode = (TNODE *) dequeue(&OutQueue)) != NULL )
 for ( linev = onode->do_this, counter = 0;
 *linev && **linev; linev++ )
 if (strchr(*linev,'>') == NULL && /* could we redirect? */
 strlen(*linev)+ReDirLen < MAXLINE)
 if ((infnm = gen_name((int) onode->tsknum,counter++)) !=
NULL)
 {
 remove(infnm); /* erase the temporary file */
 fmem(infnm); /* and free up the memory */
 }

 mal_unlock(OutQueueLock); /* free access to queue */

 }
}

/*----------------------------------------------- INITIALIZE ROUTINES -----*/

int controlbrk( void ) /* handles control-break interrupts */
{
 return( 1 ); /* return non-zero to continue running */
}

/*-------------------------------------------------------------------------*/

void getoptions(int argc,char *argv[])
{
 int i;
 char *getcwd(); /* defined in dir.h */
 void ctrlbrk(); /* defined in dos.h */

 TskQueueLock = mal_new(); /* semaphore for TskQueue */
 OutQueueLock = mal_new(); /* semaphore for OutQueue */
 AllocLock = mal_new(); /* semaphore for malloc/free */

 /* get the current directory, used to save output files */
 if (getcwd(CurDir,MAXFNM) == NULL)
 err("Cannot get current directory");

 ReDirLen = strlen(CurDir)+16; /* length of redirection file name */

 /* initialize the control-break handler to call controlbrk() */
 ctrlbrk(controlbrk);

 for (i = 1; i < argc; i++) /* get the command line switches */
 {
 /* windows switch */
 if (strcmp(argv[i],"-w") == 0 strcmp(argv[i],"-W") == 0)












 ShowWin = 1;

 /* memory size switch */
 else if (strcmp(argv[i],"-k") == 0 strcmp(argv[i],"-K") == 0)
 {
 if (i < argc - 1)
 {
 if ((MemSize = atoi(argv[++i])) <= 0)
 err("Invalid memory size parameter for -k switch");
 }
 else
 err("Missing memory size parameter for -k switch");

 }

 /* help switch */
 else if (strcmp(argv[i],"-h") == 0 strcmp(argv[i],"-H") == 0)
 {
 printf("dvmake [-w] [-k nnn] [-h] [target]\n");
 printf(" -w switch displays windows \n");
 printf(" -k switch sets task memory to nnn K-bytes \n");
 printf(" -h switch displays help message \n");
 }

 /* anything else with - or / must be mistake */
 else if (argv[i][0] == '-' argv[i][0] == '/')
 err("Invalid command switch: %s",argv[i]);

 /* anything else must be the first item to make */
 else
 {
 if ( (First = (char *) gmem(strlen(argv[i])+1)) == NULL)
 err("Out of memory");
 strcpy(First,argv[i]);
 }
 }
}

/*----------------------------------------------- START PARALLEL TASKS ----*/

void startup( void )
{
 char *mkstack, /* stacks for various tasks */
 *menustack;

 MainWin = win_me(); /* get handle of main window */

 /* initialize a PIF buffer with appropriate window size */
 if (ShowWin)
 init_pif(&Pif,&Lpif," DVMAKE TASK ",25,80);
 else
 init_pif(&Pif,&Lpif," DVMAKE TASK ",1,1);

 /* get stack space for the parallel tasks */
 if ( (mkstack = (char *) gmem(STKSIZE)) == NULL 












 (menustack = (char *) gmem(STKSIZE)) == NULL )
 err("Out of memory");

 /* we are now in Parallel mode */
 Parallel = 1;


 /* start a task to track a menu, with no title or window */
 MenuTsk = tsk_new(dvmenu,menustack,STKSIZE,"",0,0,0);

 /* start a task to make the items, with no title or window */
 MakeTsk = tsk_new(make,mkstack,STKSIZE,"",0,0,0);
}

/*----------------------------------------------- STOP PARALLEL TASKS -----*/

void finishup( void )
{
 /* get dvmenu() to stop by posting its Object Queue */
 tsk_post(MenuTsk);

 /* wait for everything to really finish */
 waitwhile(api_isobj(MakeTsk) api_isobj(MenuTsk));
}

/*----------------------------------------------- MAIN PROGRAM ------------*/

main( int argc, char *argv[] )
{
 /* if DESQview is not running or version is too low, display a message */
 if (api_init() < DV_VER)
 {
 printf ("dvmake requires DESQview version %d.%02d or later\n",
 DV_VER/256,DV_VER%256);
 return( 1 );
 }
 else
 {
 api_level(DV_VER); /* tell DV what extensions to enable */

 getoptions( argc, argv ); /* get command line arguments */

 if ( (Makefile = fopen(MAKEFILE, "r")) == NULL )
 err("can't open %s\n", MAKEFILE );

 if ( !dependencies() ) /* is there anything in the mkfile */
 {
 fclose(Makefile);
 err("Nothing to make");
 }
 else
 {
 fclose(Makefile);
 make_queue( First ); /* simulate the sequential make */














 if (MkQueue != NULL) /* does anything need to be made? */
 {
 startup(); /* start parallel tasks */
 output(); /* display output from tasks */
 finishup(); /* stop parallel tasks */
 }

 mal_free(TskQueueLock); /* get rid of our semaphores */
 mal_free(OutQueueLock);
 mal_free(AllocLock);

 if (Status == ABORT) /* print the error message */
 fprintf(stderr,"%s",Error);

 api_exit(); /* tell DESQview we're done */
 return( Status == ABORT ? 1 : 0 );
 }
 }
}









































November, 1989
CONCURRENT C FOR REAL-TIME PROGRAMMING


C for parallel processing




N.H. Gehani and W.D. Roome


Narain and William are the architects of Concurrent C and authors of The
Concurrent C Programming Language (Silicon Press). They can be reached at AT&T
Bell Labs, 600 Mountain Ave., Murray Hill, NJ 07974.


Concurrent C is an upward-compatible extension of C that provides parallel
programming facilities with implementations available for several versions of
the Unix system (that is, System V and BSD 4.2) on a variety of computers.
Concurrent C has also been implemented on different kinds of multiprocessors
such as a number of independent computers connected by a local area network
(Ethernet).
A Concurrent C program is structured as a set of processes that interact with
each other by sending messages. Messages are sent to and replies received from
another process by calling transactions associated with the process. To avoid
confusion with "database transactions," we'll use the term "transaction" to
mean a Concurrent C process interaction. Transactions are like remote
procedure calls with one important difference -- the receiving process can
schedule acceptance of the calls. Transactions can be synchronous or
asynchronous. Synchronous transactions implement the extended rendezvous
concept (as in the Ada language): Two processes interact by first
synchronizing, then exchanging information (bidirectional information
transfer), and finally by continuing their individual activities. A process
calling a synchronous transaction is forced to wait (unless it times out)
until the called process accepts the transaction and performs the requested
service. With asynchronous transactions, the caller does not wait for the
called process to accept the transaction; instead the caller continues with
other activities after issuing the transaction call. Information transfer in
asynchronous transactions is unidirectional, from the calling process to the
called process. Concurrent C, as a compile-time option, also works with C++.
Our objectives in enhancing C with concurrent programming facilities were to
provide a tool for writing programs that run on genuinely parallel hardware
and to provide a test bed for experimenting with distributed programming
facilities. In this article, we'll describe Concurrent C and then use it to
write a nontrivial real-time program (a tty controller). We assume that you
are familiar with the C language, so we only describe the Concurrent C
extensions to C.


Concurrent C Characteristics


Concurrent C extends C for parallel programming by providing facilities for
specifying process types; creating processes; specifying the processor on
which a process is to run; specifying, querying, and changing process
priorities; synchronous transactions; asynchronous transactions; delays and
time-outs; interrupt handling; waiting for a set of events such as
transactions; accepting transactions in a user-specified order; process
abortion; and collective termination.
To give you a flavor of Concurrent C, we'll show you the definition of a
process that buffers messages (characters in this example). Producer processes
(p[i]) call the buffer process to put messages in the buffer, and consumer
processes (c[i]) call it to get messages from the buffer, as shown in Figure
1. A process definition in Concurrent C consists of a specification (or type)
and a body. Example 1 shows the specification of the buffer process.
Example 1: Specification of the buffer process

 process spec buffer(int max)
 {
 trans void put (int c);
 trans int get ();
 };

The buffer process has two synchronous transactions: put and get. Characters
are put into the buffer by calling put; if the buffer is full, a call to put
will not be accepted. Transaction get is used to take characters from the
buffer; if the buffer is empty, a call to get will not be accepted until a
character is available.
The direction of information transfer is not dependent on which process makes
the transaction call. In this example, the process calling transaction put
sends characters to the buffer process while the process calling get gets
characters from the buffer process.
Transactions put and get of the buffer process are synchronous in that the
calling processes are blocked until the transaction call is accepted and
executed. Transaction calls can be declared to be asynchronous (by using the
keyword async instead of the transaction result type), in which case the
calling process is not blocked. Asynchronous transaction calls do not return a
result.
Synchronous transaction calls can be used for bidirectional information
transfer: Arguments are used to send information to the called process and the
transaction result is used to return information to the calling process.
Synchronous transaction calls can also be used to synchronize execution.
The buffer process type can be used just like the predefined types to declare
variables, function types, and so on; for instance, process buffer buf, b[10].
Example 2 shows the body of the buffer process that implements a circular
buffer.
Example 2: The body of the buffer process which implements a circular buffer

 process body buffer (max)
 {
 int *buf; /*circular buffer of size max*/
 int n = 0; /*number of chars in buffer*/
 int in = 0; /*index of next empty slot*/
 int out = 0; /*index of next character*/
 char *malloc();

 buf = (int*) malloc (max*sizeof (int));
 for (;;)
 select {
 (n < max):
 accept put (c)
 buf [in] = c;
 in = (in +1) % max;
 n++;
 or
 (n > 0):

 accept get()
 treturn buf[out];
 out = (out + 1) % max;
 n--;
 }
 }

The buffer process allocates space for the buffer and then repeatedly executes
a select statement that is used to wait for a set of alternative events. This
statement has two alternatives, separated by the keyword or. Each alternative
starts with a Boolean test and is followed by an accept statement. The Boolean
test is called a "guard" and gives the conditions under which that alternative
can be chosen. For example, the guard for the first alternative is true
whenever the buffer process has space to store a character, and the guard for
the second alternative is "true" whenever the buffer process has characters to
give out. Transaction calls are accepted using the accept statement.
When the select statement is executed, it selects one alternative, executes
all statements in that alternative, and skips the other alternative. For an
accept alternative to be selected, its guard must be true and there must be a
pending transaction call that satisfies the leading accept statement. If both
alternatives are eligible, the select statement picks one randomly. If both
guards are true but no transaction calls are waiting, the select statement
waits for the first call of either type.
Initially the buffer is empty and only the first guard is true. Consequently,
the select statement waits for a put transaction call to arrive and then
executes the first alternative. The buffer process then loops back and
executes the select statement again. Now both guards are true, so it waits for
the next put or get transaction call to arrive.
Processes are created (instantiated) using the create operator; for example,
buf = create buffer(1024). The create operator returns the process id of the
newly created process. When creating a process, the processor on which the
process is to run and the process priority can also be specified.
Processes communicate by means of transaction calls that have the form
process-id.transaction-name(arguments). For example, character 'a' is sent to
the buffer process with the transaction call buf.put('a') and is retrieved
from the buffer process with the call c = buf.get( ). Transaction calls behave
like function calls. For example, get returns the character taken from the
buffer as a value of type char. In general, a transaction call is allowed
wherever a function call is allowed.
Transaction calls can also be made using transaction pointers that encapsulate
the process id and the transaction name as illustrated by the program segment
in Example 3.
Example 3: Transaction calls can be made using transaction pointers

 char c;
 trans void (*tp) (int);
 ...
 tp = buf.put;
 ...
 (*tp) ('a');

Transaction pointers are similar to function pointers except that their
declarations are prefixed by the keyword trans. Because a transaction pointer
does not include the process type, a transaction pointer can refer to
transactions of different process types, provided these transactions have the
same parameter and return-value types.
Concurrent C also supports timed (synchronous) transaction calls that allow
the calling p ocess to withdraw the transaction call if it is not accepted
within the specified period.
As discussed earlier, transaction calls are accepted with the accept
statement. By default, these calls are accepted in FIFO order. The order in
which these calls are accepted can be changed with the by clause. The suchthat
clause allows the selection of the transaction call to be based on the
arguments of the transaction and on the state of the called process.
In general, a select statement can have an arbitrary number of alternatives,
not just two alternatives as shown in the buffer example. The alternatives of
the select statement can be used for accepting transaction calls, timing out,
collective termination of all the processes leading to program termination, or
execution of arbitrary Concurrent C statements. For any execution of the
select statement, Boolean expressions can be used to mask out one or more of
the alternatives.


Real-Time Programming Facilities


Concurrent C allows transactions to be accepted in a user-specified order;
thus the user can accept urgent transaction calls first. Specifically,
transaction call acceptance can be based on the transaction name, the order in
which transaction calls are received, transaction arguments, and the state of
the called process.
Asynchronous message passing provides maximum flexibility because processes
can compute and perform message sends and receives in parallel in any way they
want. A process sending a message is not blocked until the receiver gets
around to accepting the message; this allows the sender to attend to other,
possibly critical, events. Asynchronous message passing is especially
important in situations where the inter-process communication time is high (as
in case of multiprocessors). Asynchronous message passing also allows
pipelining of multiple messages from the same process and allows the receiving
process to accept the messages in the most appropriate order.
Concurrent C provides facilities for specifying, querying, and changing
process priorities. Process priorities are specified when creating a process,
but the priority can be changed at any time. Processes that need to respond
quickly to events such as interrupts, which need immediate attention, are
given high priorities.
Concurrent C provides facilities for timed synchronous transaction calls. This
allows a process to withdraw a transaction call if it has not been accepted
within the specified period. A process can also time-out if a transaction call
does not arrive within a specified period. Time-outs prevent a process from
being blocked for an unduly long period.
An important aspect of real-time programming is interrupt handling. The
implementation-dependent function c_associate is used to indicate that the
specified interrupt should be converted to a call to the specified
transaction. To avoid losing interrupts, it is important that they (the
associated transactions) are handled quickly. Delay in handling interrupts can
occur because of the overhead in converting the interrupt to a transaction
call, the delay in scheduling the driver process, and the delay in accepting
the call.
These items are implementation- and application-dependent. In many cases, by
giving the interrupt handling process a high priority and designing this
process to give preference to accepting interrupt transactions, interrupts can
be handled within real-time constraints.


A Display-Terminal Driver Example


Suppose that you are writing the software for an embedded system -- one that
runs on a dedicated microprocessor. Examples of embedded systems are an
industrial process controller, a robot controller, or the control program for
a VCR. Let's suppose a display terminal is connected to your system, and
occasionally you want to write messages to that display and wait for the user
to type a reply. Your program runs on a bare machine -- there is no operating
system, so you have to write all the software for controlling the display.
First we'll give a simplified description of the device (alas, real devices
are not this simple, but if you follow this example, you should have no
trouble with real hardware). The device interface consists of two 8-bit
registers, the input and output character buffers, and two flag bits,
input-ready and output-ready. These are in the processor's memory and can be
accessed like any other memory locations. Whenever the output-ready bit is on,
the program can write a character into the output character register. The
hardware then turns the output-ready bit off and sends the character to the
display. When the character has been sent, the hardware turns the output-ready
bit back on, and the software can write another character. At 9600 baud, it
takes about 1 millisecond to output a character; at 1200 baud, it takes about
8.3 milliseconds. When the user types a character on the keyboard, the
hardware places that character in the input character register and turns on
the input-ready bit. The program can then read the character from the
register. When that happens, the hardware turns off the input-ready bit until
the user types the next character. Once the user has typed a character, our
program must read that character from the input register before the user types
another character; if not, the character will be lost.
The hardware also generates an input-ready interrupt when it turns the
input-ready bit on, and an output-ready interrupt when it turns the
output-ready bit on.
Note that the terminal is really two independent devices: A display for output
and a keyboard for input. When the user types a character, the hardware does
not automatically display that character on the screen. Instead, our software
must "echo" that character to the display. Furthermore, most display devices
have separate carriage return and line feed operations; when the user types
Return, our software should send a carriage return and a line feed to the
display. We would also like to give the user a few creature comforts:
Backspace should erase the previous character, Ctrl-C (for example) should
kill the line that the user has typed, Ctrl-S should stop output so the user
can read the display, and Ctrl-Q should restart output.
The simple way to output characters to the display is to repeatedly test the
output-ready bit until it's on, write the character to the output register,
and repeat until all the characters are written. This is called "wait-loop"
I/O. The disadvantage is that the system cannot do anything else until the
output is complete. Sometimes this is acceptable, but usually it isn't. And
even if wait-loop I/O is acceptable for output, it probably will not be for
input -- the user types too slowly!
We need a solution that allows input from the keyboard and output to the
display to overlap with other processing -- we need concurrent programming.
The solution is to use several processes that form a subsystem of our program
and provide display input and output services to the other processes in our
program. For historical reasons, we'll call these processes the "TTY driver"
subsystem.
Figure 2 illustrates the interaction between the processes. Characters typed
by the user flow down on the left from the keyboard hardware to the ttyInput
process, where they are taken by other processes in our program. Characters to
be displayed flow up on the right from other processes to the ttyOutput
process and then to the display hardware. Process ttyOutput handles output to
the display, and process ttyInput is the input buffer process. These processes
form the "client interface" to the TTY subsystem, while other processes
("clients") in the program call put of ttyOutput to send characters to the
display, and call get of ttyInput to read characters typed on the keyboard.
Processes ttyLine and ttyReader are internal processes (they interact only
with other processes of the TTY subsystem) and will be described later. The
specification of the TTY subsystem processes is shown in Listing One (page
100).
Characters are output by calling transaction put of ttyOutput; its arguments
are the number of characters and a pointer to the start of the string.
Transaction put queues the characters for output; the transaction returns
before the characters are sent to the display. Transaction ready is called
when an output-ready interrupt occurs, stop temporarily stops the output to
the display, and start restarts the output. Process ttyOutput accepts put
transaction calls even when the output is stopped; the characters are queued
until output is resumed.
We require the processes that call transaction put to end a line with a
carriage return and line-feed characters; this simplifies the ttyOutput
process.
Process ttyInput is the input buffer process. The characters in this buffer
have been "cooked:" They have been echoed to the display, backspace and
line-kill processing have been performed, and so on. If characters are
available in the buffer, transaction get returns the next character. If the
buffer is empty and the argument is 1, then get waits for a character to
arrive. If the argument is 0, then get returns -1 immediately. Thus get(0) is
non-blocking. Transaction put inserts a character into the input buffer.
Any number of processes can output characters at the same time. Process
ttyOutput guarantees that the characters written by one put transaction call
will be printed together on the display. So if several processes output data
at the same time, as long as each put transaction sends one line, the lines
will be interleaved in an arbitrary fashion, but the characters in each line
will stay together. If several processes try to get characters from ttyInput
at the same time, they will get characters on a first-come, first-served
basis. This is probably not what you want. We assume that the processes
cooperate among themselves to ensure that only one process tries to read
characters at a time -- usually this occurs naturally.
Process ttyReader is a simple buffer manager; it takes characters from the
hardware input register as soon as the user types them, and saves them in a
buffer. Transaction ready of this process is called when an input-ready
interrupt occurs, and process ttyLine calls get to get characters from the
buffer. The process ttyLine has no transactions; it gets characters from
ttyReader, echoes them to ttyOutput, handles backspace and line-kills, and so
on. When ttyLine gets a full line, it sends the characters in that line to the
ttyInput process.
Function ttyReply (see Listing Two, page 100) illustrates how to use these
processes. It first prints the msg argument on the display and then waits for
the user to type a reply line, which the function saves in the reply argument.
If the user does not type a reply within 30 seconds, the function prints a
nasty message and starts over. This function uses the timed-transaction call
within 30 ? ttyIn.get(1) : -1
If the ttyIn process accepts the get transaction call within 30 seconds, the
value of this expression is the value returned by the process. If the process
does not accept the get transaction call, then the call is automatically
withdrawn, the second expression (-1) is evaluated, and its value becomes that
of the expression. So if the user types a character within 30 seconds, the
function breaks out of the loop and reads the rest of the line. If not, the
function prints a nasty message and repeats.



Implementation


Function ttyInit (Listing Three, page 100) creates the TTY subsystem processes
and saves their ids in global variables. Normally, main will call ttyInit when
the program starts. Notice that we have specified priorities for some of the
processes. If no priority is specified, a process is created with priority 0.
Process ttyOutput is created with priority 1, so that when it is ready -- that
is, when the display is ready for a character -- the ttyOutput process will be
scheduled before the other processes. Process ttyReader is given priority 2
because it is even more important that we take a character from the input
register as fast as possible.
Function c_associate arranges to call a transaction when an interrupt occurs.
The first argument is a transaction pointer specifying the process and
transaction to be called. The second argument specifies the interrupt. Listing
Four, page 100, gives some definitions that are common to all of the processes
in the TTY subsystem.
Process ttyReader (Listing Five, page 100) is a variant of the buffer manager
in Example 2. It loops forever, waiting for either a ready or a get
transaction. Transaction get takes a character from the buffer and returns it,
just as in the buffer manager. Transaction ready is automatically called when
the hardware generates an input-ready interrupt. ttyReader then calls the
uartGetChar function (the hardware interface to the device is often called a
Universal Asynchronous Receiver/Transmitter or UART) to get a character from
the input register. If it is a start or stop character, the process calls the
appropriate transaction of the ttyOutput process. Otherwise, if there is space
in the buffer, the process saves the character; if not, it discards the
character.
Note that when the buffer is full, we discard characters rather than not
accepting ready transactions. We do this so that we can always accept a start
character. Suppose that the user types a stop character and then continues
typing. The ttyOutput process will eventually fill up its buffer and will not
accept put transactions until output is restarted. Because ttyLine calls put
to print the characters entered at the keyboard, ttyLine will eventually stop
until output is restarted. Because ttyLine is the only process that takes
characters from the ttyReader process' buffer, the ttyReader buffer will
eventually fill up. At that point the system will be frozen until the user
types a "start" character. If ttyReader stops accepting characters when its
buffer is full, then the user could never type a start character, and the
system would be deadlocked. That is why ttyReader always accepts characters
typed by the user, even if it has to discard them.
Process ttyLine (Listing Six, page 100) maintains a buffer containing the line
that the user is currently typing. ttyLine repeatedly gets a character from
the ttyReader process, handles that character, and if the line is complete,
sends the line to the ttyInput process and clears the buffer. For example,
when given an "erase" character (backspace), the process outputs a backspace,
blank, and backspace to erase the last character on the display, and then
erases the last character in the line buffer.
The input processing is not sophisticated. For example, in many display
drivers, a special character, such as backslash (\), indicates that the next
character is to be treated as a regular character. ttyLine does not do that,
but it would be relatively easy to add it. Adding this feature is a sequential
programming problem, not a concurrent programming one. A concurrent
programming language such as Concurrent C lets you turn a concurrent program
problem into a set of sequential programming problems -- namely the processes
-- that you can then solve independently.
Process ttyInput (Listing Seven, page 100) manages the input buffer. It is
almost identical to the buffer process in Example 2. The only addition is a
second accept statement for the get transaction. The guards on those two
accept statements are mutually exclusive: The first guard is true only if the
buffer has characters, the second is true only if the buffer is empty. The
second accept statement has a suchthat clause that Concurrent C evaluates for
each pending transaction call. If the expression is true (non zero),
Concurrent C accepts that call; otherwise, Concurrent C holds the transaction
call so that it can be accepted later. Thus if the buffer has characters, the
process accepts a get transaction regardless of the value of the argument.
When the buffer is empty, the process only accepts get transactions whose
arguments are 0 -- that is, the non-locking get requests. Therefore ttyInput
immediately accepts any get(0) transaction call; if the buffer has a
character, the process returns it; otherwise, ttyInput returns -1. On the
other hand, a get(1) call will only be accepted when the buffer has a
character.
Process ttyOutput (Listing Eight, page 100) manages the output buffer. It
repeatedly waits for one of four possible transaction calls and sends a
character to the display if the display is ready, output has not been stopped,
and there is a character in the buffer. Transaction ready is called when the
device generates an output-ready interrupt. Function uartPutChar writes a
character to the output register. Note how we use a suchthat clause with the
put transaction to accept the call only if there is enough space in the
buffer. Also note that the process accepts put calls even when output is
stopped, as long as there is space in the buffer.
At press time, AT&T is planning to make Concurrent C a product that will be
available in the near future. The reader interested in Concurrent C can call
1-800-828-UNIX to check on the availability of Concurrent C.


References


ANSI C. Draft Proposed American National Standard for Information Systems --
Programming Language C. 1988.
Cox, I.J. and N.H. Gehani. "Concurrent C and Robotics." 1987 IEEE Conference
on Robotics and Automation, Raleigh, NC, 1987.
Dijkstra, E. W. "Cooperating Sequential Processes." In Programming Languages,
edited by F. Genuys. Academic Press, San Diego, Calif., 1968.
Gehani, N.H. C: An Advanced Introduction (ANSI C Edition). Computer Science
Press, Rockville, Maryland, 1988.
Gehani, N.H. 1988. "Message Passing: Synchronous vs. Asynchronous." Submitted
for Publication.
Gehani, N.H. and W.D. Roome. "Rendezvous Facilities: Concurrent C and the Ada
Language," IEEE Transactions on Software Engineering, vol. 14, no. 11,
(November), pp. 1546-1553, 1988.
Gehani, N.H. and W.D. Roome. "Concurrent C++: Concurrent Programming With
Class(es)." Software -- Practice & Experience, vol. 18, no. 12, pp. 1157-1177,
1988.
Gehani, N.H. and W.D. Roome. Concurrent C. Silicon Press, Summit, N.J., 1989.
Hoare, C.A.R. "Communicating Sequential Processes." CACM, vol. 21, no. 8
(August), pp. 666-677, 1978.
Kernighan, B.W. and D.M. Ritchie. The C Programming Language. Prentice-Hall,
Englewood Cliffs, N.J., 1978.
Roome, W.D. The CTK: An Efficient Multi-Processor Kernel. AT&T Bell
Laboratories, 1986.
Stroustrup, B. The C++ Programming Language. Addison Wesley, Reading, Mass.,
1986.

_CONCURRENT C FOR REAL-TIME PROGRAMMING_
by N.H. Gehani and W.D. Roome


[LISTING ONE]


process spec ttyOutput()
{
 trans void put(int nc, char *pc);
 trans async ready(), start(), stop();
};

process spec ttyInput()
{
 trans void put(char c);
 trans int get(int wait);
};

process spec ttyReader()
{
 trans async ready();
 trans char get();
};

process spec ttyLine();

process ttyInput ttyIn;
process ttyOutput ttyOut;

process ttyReader ttyRdr;





[LISTING TWO]

#include "concurrentc.h"
#include "tty.h"

void ttyReply(msg, reply)
 char *msg, *reply;
{
 int c;
 char *nasty = "\r\nWAKE UP!\r\n";

 while (1) {
 ttyOut.put(strlen(msg), msg);
 c = within 30 ? ttyIn.get(1) : -1;
 if (c != -1)
 break;
 ttyOut.put(strlen(nasty), nasty);
 }
 while (c != '\n') {
 *reply++ = c;
 c = ttyIn.get(1);
 }
 *reply = '\0';
}





[LISTING THREE]

#include "concurrentc.h"
#include "tty.h"

void ttyInit()
{
 ttyOut = create ttyOutput() priority(1);
 c_associate(ttyOut.ready, 1/*DUM*//*PSoutput-ready-interruptPS*/);
 ttyRdr = create ttyReader() priority(2);
 c_associate(ttyRdr.ready, 2/*DUM*//*PSinput-ready-interruptPS*/);
 ttyIn = create ttyInput();
 create ttyLine();
}





[LISTING FOUR]

#define M_RdrBuff 2048
#define M_OutBuff 2048
#define M_InBuff 1024

#define M_LineLen 1024

#define StopChar 0x13 /* ctl-S */
#define StartChar 0x11 /* ctl-Q */
#define EraseChar '\b' /* backspace */
#define KillChar 0x03 /* ctl-C */





[LISTING FIVE]

#include "concurrentc.h"
#include "tty.h"
#include "defs.h"

process body ttyReader()
{
 int n = 0, in = 0, out = 0;
 char c, buff[M_RdrBuff];

 while (1) {
 select {
 accept ready();
 c = uartGetChar();
 if (c == StartChar) {
 ttyOut.start();
 } else if (c == StopChar) {
 ttyOut.stop();
 } else if (n < M_RdrBuff) {
 buff[in] = c;
 in = (in + 1) % M_RdrBuff;
 n++;
 }
 or (n > 0):
 accept get()
 treturn buff[out];
 out = (out + 1) % M_RdrBuff;
 n--;
 }
 }
}





[LISTING SIX]

#include "concurrentc.h"
#include "tty.h"
#include "defs.h"

process body ttyLine()
{
 int n = 0, putline = 0, i;
 char c, buff[M_LineLen];


 while (1) {
 switch (c = ttyRdr.get()) {
 case EraseChar:
 ttyOut.put(3, "\b \b");
 if (n > 0)
 --n;
 break;
 case KillChar:
 if (n > 0)
 ttyOut.put(2, "\r\n");
 n = 0;
 break;
 case '\r':
 case '\n':
 ttyOut.put(2, "\r\n");
 buff[n++] = '\n';
 putline = 1;
 break;
 default:
 ttyOut.put(1, &c);
 buff[n++] = c;
 if (n >= M_LineLen)
 putline = 1;
 break;
 }
 if (putline) {
 for (i = 0; i < n; i++)
 ttyIn.put(buff[i]);
 putline = n = 0;
 }
 }
}





[LISTING SEVEN]

#include "concurrentc.h"
#include "tty.h"
#include "defs.h"

process body ttyInput()
{
 int n = 0, in = 0, out = 0;
 char buff[M_InBuff];

 while (1) {
 select {
 (n < M_InBuff):
 accept put(c)
 buff[in] = c;
 in = (in + 1) % M_RdrBuff;
 n++;
 or (n > 0):
 accept get(wait)
 treturn buff[out];
 out = (out + 1) % M_RdrBuff;

 n--;
 or (n == 0):
 accept get(wait) suchthat (!wait)
 treturn -1;
 }
 }
}





[LISTING EIGHT]

#include "concurrentc.h"
#include "tty.h"
#include "defs.h"

process body ttyOutput()
{
 int n = 0, in = 0, out = 0;
 int running = 1, ttyrdy = 1, i;
 char buff[M_OutBuff];

 while (1) {
 select {
 accept start()
 running = 1;
 or accept stop()
 running = 0;
 or accept ready()
 ttyrdy = 1;
 or accept put(nc, pc) suchthat (n+nc <= M_OutBuff) {
 for (i = 0; i < nc; i++) {
 buff[in] = pc[i];
 in = (in + 1) % M_OutBuff;
 }
 n += nc;
 }
 }
 if (ttyrdy && running && n > 0) {
 uartPutChar(buff[out]);
 out = (out + 1) % M_OutBuff;
 n--;
 ttyrdy = 0;
 }
 }
}














November, 1989
LINKING WHILE THE PROGRAM IS RUNNING


Run-time dynamic linking in OS/2




Andrew Schulman


Andrew is a software engineer doing CD-ROM and network programming at a large
software firm in Cambridge, Mass. He can be reached at 32 Andrew St.,
Cambridge, MA 02139.


"In theory anyway linking to a target <t> can be achieved at the earliest
whenever it becomes feasible to make <t> known. It is, in fact, even more
interesting to consider until how late linking can be postponed."
-- Elliott I. Organick The Multics System: An Examination of its Structure
(1972)
"The evolution of loaders is interesting because it is an example of a trend
common to many areas of both software and hardware, the trend to delay binding
as long as possible."
-- Robert M. Graham
Principles of Systems Programming (1975)
Once upon a time, there was no such thing as linking, or at least no such
thing as a "linkage editor." As long as there was no separate or independent
compilation, there was no need for a program that combines fragments of
programs. But as soon as there were "linkers," it became interesting to see
how long linking could be deferred. ("Never do today what you can put off
until tomorrow.")
Every professional programmer has heard that OS/2 is built on "dynamic
linking," a standardized mechanism for attaching add-ins to the operating
system. OS/2, itself, consists largely of add-ins; for example, the
Presentation Manager's (PM) graphical windowed environment is a collection of
OS/2 dynamic-link libraries (DLLs), as is the OS/2 kernel application program
interface (API).
The operation of dynamic linking is largely transparent to the programmer. As
David Cortesi explained in his December 1987 DDJ article "Dynamic Linking in
OS/2: Built-in facilities for the third-party extension of OS/2," a programmer
using a procedure in a DLL declares it just as he would declare any other
external reference; calls to such a procedure look the same as calls to a
routine in a "normal" static-link .LIB library. The only immediately visible
difference is a smaller executable file. The actual code for the procedure is
kept in the DLL, not copied into your executable file as in normal linking.
OS/2 does all the work of dynamically linking to the DLL code.
But there is another form of dynamic-linking that is not transparent. Called
"run-time dynamic linking," this method of linking requires you carry out "by
hand" the work normally done by LINK.EXE and OS/2. In exchange, you get to
defer linking until very late -- in fact, linking takes place while the
program is running!


Why Would You Want to Do That?


Because computing consists largely of "solutions in search of a problem," and
of building features that no one has yet asked for, you frequently encounter
facilities that are nifty but not useful. So, before we get into how they
work, we'd better determine what run-time dynlinks are for in the first place.
The ability to defer linking so that it occurs during run-time can be used in
any application in which users can do programming while the application is
running. Examples include any interpreted language, an Emacs text editor with
an embedded language, a database manager or spreadsheet with a LOAD/CALL
add-in facility, or a debugger with an "execute statement" menu option. A
straight object-code compiler is not an example because the programming is
done before the compiler is run.
There is a close parallel between "delayed linking" and what in
object-oriented programming (OOP) is called "late binding." (The two are not
synonymous, however.) With OOPs, late (delayed) binding turns a selector into
the actual method to handle a message. In OS/2, delayed linking takes commands
a user issues at run-time and turns them into actual function calls.
In both OOPs and OS/2, it is definitely not the program's responsibility to
maintain the table that associates symbolic names with actual code, because
that would constitute early binding. Your program's responsibility is to pass
the symbolic names through, without interpreting them. This means that with
both OOPs and OS/2 run-time dynamic linking, a program can use facilities that
didn't even exist when the program was compiled.
There is more than an analogy between OS/2 delayed linking and OOPs delayed
binding. Some object-oriented systems, such as the Andrew Toolkit and Class C
preprocessor developed at Carnegie-Mellon University, use delayed linking in
order to implement dynamic demand loading and linking of classes in running
applications. (For a discussion of the connection between dynamic binding and
dynamic linking, see Philippe Gautron and Marc Shapiro, "Two Extensions to
C++: A Dynamic Link Editor and Inner data," Usenix Proceedings C++ Workshop,
Santa Fe, New Mexico 1987). Because of its support for run-time dynlinks, OS/2
is a perfect platform for building OOPs environments.


The Importance of Being ASCIIZ


"Our future plans include: ... Programmatic interface: Some programs,
particularly based on interpretive languages such as Lisp, can dynamically
generate dynamic references. We would like to support the handling of such
references through a common mechanism, and thus wish to provide a
program-accessible interface to the services now provided invisibly."
-- Robert A. Gingell, et al., Shared Libraries in SunOS (1987).
OS/2's programmatic interface for dynamic linking under program control is
built upon the functions listed in Table 1 that comprise the OS/2 module
manager. Of these, DosLoadModule and DosGetProcAddr are the crucial functions.
In fact, with these two functions alone we can access the others, for with
these two functions we can in fact access any function in an OS/2 DLL.
Table 1: The OS/2 module manager

 DosLoadModule
 DosGetProcAddr
 DosFreeModule
 DosGetModName
 DosGetModHandle

Those of you familiar with Microsoft Windows will detect the similarity to
LoadLibrary( ) and GetProcAddress( ), and those familiar with the Macintosh
can consider OS/2's DosGetProcAddr combination of the toolbox a Named
Resouvce( ) Get and GetTrapAddress( ). And programmatic access to shared
libraries is planned for the forthcoming release 4.0 of Unix System V (Unix
Review, August 1989).
Run-time dynlinks illustrate the importance of ASCII strings in OS/2. Nearly
any object in OS/2 can have an ASCIIZ (zero-terminated ASCII string) name
associated with it. The well-known "named pipes" of the LAN Manager are one
example. Just as most systems make it easy to access a file if you know its
name, OS/2 also makes it easy to access pipes, semaphores, queues, or shared
memory blocks if you know their names (see Ray Duncan, "Interprocess
Communications in OS/2," June 1989 DDJ). Likewise, with run-time dynlinks, if
you have the ASCIIZ name of a routine, you can load the code for the routine
and derive its equivalent function pointer.
Given the ASCIIZ name of a DLL, DosLoadModule loads the DLL and returns a
handle to it. This module handle can then be passed, along with the ASCIIZ
name of a procedure exported from the DLL, to DosGetProcAddr, which returns
the function pointer corresponding to the procedure name.
A function which, given an ASCIIZ string, returns a function pointer, is
typical of interpreters and debuggers. Here we find it at the core of an
operating system.
Furthermore, OS/2 is providing more than just a functional interface to a
symbol table. Ralph Griswold, inventor of the string-oriented programming
languages SNOBOL and Icon, makes the point that run-time dynamic-linking
"takes two forms: 1. Connecting the string name of a function with resident
code for it -- that is, doing at run-time what usually is done at
compile-time, and; 2. Actually loading non-resident code for a named function.
The latter is, of course, more difficult to implement than the former." OS/2
provides the second variety of delayed linking, which is a superset of the
first.
While OS/2 provides a method for turning ASCIIZ names into function pointers,
possibly by loading the code for the function, it does not provide any method
for actually calling the function. It provides "directory assistance," but
won't dial the number for you. Instead, the function pointer returned from
DosGetProcAddr should be passed, together with any arguments the procedure
expects, to whatever facility your language provides for indirect far calls.



Using the OS/2 Module Manager


CALLDLL1.C (Listing One, page 102) is a short example of how this works. When
this rather contrived example runs, it links to the file ALIAS.DLL. (ALIAS is
a handy command-line editor for OS/2, written by Andrew Estes, and modelled
after Chris Dunford's CED for MS-DOS.) CALLDLL1.C calls two procedures
provided by ALIAS: First, a command-line synonym is added to ALIAS's synonym
table, then we ask ALIAS to display the table.
If DosLoadModule returns anything other than 0, it was unable to load the
requested module. It will also fill a buffer with the name of the offending
module if you provide such a buffer (in CALLDLL1.C, I didn't). This sounds
silly, because the offending module is the same as the module you asked to be
loaded -- but that's not actually the case; DLLs can call other DLLs. Note
also that DosLoadModule should be passed the name of the module, (that is,
ALIAS), not the name of the file (for example, ALIAS.DLL), except when
accessing a DLL not located along your LIBPATH. In that case use the full
pathname. (For example,
 "C:\\OS2\\ALIAS\\ALIAS.DLL"
In Listing One, I'm using hard-wired strings instead of hard-wired function
pointers (LIST_SYN instead of List_Syn( )). This helps to illustrate how
run-time dynamic linking works, but generally it is pointless to trade one
form of inflexibility for another. Instead of embedding string literals like
ALIAS and LIST_SYN directly in the code, usually we'll use a string-valued
variable or an expression that yields a string.
This example isn't quite as foolish as it first appears. Had this program
accessed List_Syn( ) using load-time dynamic linking, OS/2 would refuse to
even load the program on a machine that didn't have ALIAS: "SYS1804: The
system cannot find the file ALIAS." But by using run-time dynamic linking, the
program itself detects the absence of the DLL and can respond in some
appropriate way. This is an additional benefit of run-time dynamic linking
over load-time ("eager") dynamic linking. Load-time dynamic links are
hard-wired; they're actually not all that dynamic.
There is one "gotcha" involved with calling DosGetProcAddr, which trips
everybody up at least once: It is case-sensitive. In Listing One, I passed the
function name ADDSYN to DosGetProcAddr. DosGetProcAddr would have failed on
"AddSyn" or "addsyn," returning ERROR_PROC_NOT_FOUND. The name you use must
match exactly the name exported from the DLL. To get a listing of these names,
you can use Microsoft's EXEHDR utility. Unfortunately there is no OS/2 API
call to enumerate the procedures a DLL exports (though you can write enumproc
( ) yourself).
Generally, routines that use the Pascal calling convention will be exported in
ALL CAPS, though Modula-2 compilers for OS/2 produce export names that look
like Module$ProcName or Module_ProcName. DLL routines using the cdecl calling
convention will generally be exported in_lowercase, with a leading underscore.
There's one other trick to using DosGetProcAddr. Unfortunately, while the OS/2
kernel masquerades as DOSCALLS.DLL, this pseudomodule does not export ASCIIZ
names. Every other OS/2 module (including KBDCALLS and VIOCALLS) exports them,
but DOSCALLS provides only "ordinal numbers."
Once you know the ordinal number for a DOSCALL function, you can pass it to
DosGetProcAddr in one of two forms: Take the example of DosGetProcAddr itself,
whose ordinal number is 45
 DosGetProcAddr(module, "#45",
 &dosgetprocaddr);
 dosgetprocaddr(module,
 MAKEP(0,45), &dosgetprocaddr);
Aside from the annoying special case of DOSCALLS, DosGetProcAddr is, in
effect, a "named" equivalent to the MS-DOS GetVect operation. But while DOS
GetVect is complemented by SetVect, and likewise GetTrapAddress on the
Macintosh is complemented by SetTrapAddress, in OS/2 there is no
DosSetProcAddr. Instead, individual modules are responsible for supplying
their own mechanism for plug-in replacements. The KBD-, MOU-, and VIO-based
subsystems in OS/2 can be replaced using KbdRegister, MouRegister, and
VioRegister. There is no such thing as DosRegister.
When finished with a module a program can call DosFreeModule. This is
unnecessary if the program is about to terminate anyway. DosFreeModule
decrements a reference count that OS/2 keeps for DLLs: When the count goes
down to zero, the DLL is released from memory (remember that multiple
applications may be using a DLL at the same time). After calling
DosFreeModule, a program still has the function pointers returned from
DosGetProcAddr, but they are no longer valid.
While C is generally used to illustrate run-time dynlinks, similar programs
can be written in any other language for which an OS/2 version is available.
CALLDLL.MOD (Listing Two, page 102) is a Modula-2 equivalent to Listing One. I
used JPI TopSpeed Modula-2 here. Because of the non-standardization of
Modula-2 equivalent code for Stony Brook Modula-2 or Logitech Modula-2 would
look slightly different. CALLDLL.LSP (Listing Three, page 102) is the same
program again, this time written in OS2XLISP. This version is so short not
only because of the expressive power of Lisp, but because OS2XLISP itself is
built around run-time dynamic linking.
DosLoadModule and DosGetProcAddr are equally important. Usually the function
name is viewed as more of a variable than the module name, but in fact it
often works the other way around; the function name can stay the same while
the module name varies. For instance, you might have the same function BitBlt(
) implemented in a VGA.DLL, an EGA.DLL, and so on. There is an analogy here to
polymorphism in object-oriented programming, where the same generic operation
can be applied to objects of different classes in an inheritance chain: The
OS/2 module is like an OOPs class, and the functions within a module are like
the selectors implemented by that class. Just as in OOPs, the run-time system
transforms a class/selector pair into a single output, the method, so OS/2
run-time dynlinks transform a module/function name pair into a single output,
the function pointer.


A Higher Level


It's convenient to say that DosGetProcAddr takes an ASCIIZ name and returns
the corresponding function pointer but, as Listings One and Two show, that's
not exactly how it looks. Instead, as with all OS/2 kernel functions,
DosGetProcAddr returns only an error code. Because most high-level languages
provide only one retval, any other information, such as the function pointer
we are actually interested in, must be passed back in VAR parameters.
We think of DosGetProcAddr in this way:
FUNCPTR DosGetProcAddr
(HANDLE module, ASCIIZ procname);
But the way OS/2 provides DosGetProcAddr actually looks like this:
 ERRCODE DosGetProcAddr
 (ASCIIZ failbuf, WORD size, ASCIIZ procname, FUNCPTR *p_procname);
Compared with MS-DOS functions, OS/2 functions look high-level. Whereas DOS
functions are invoked by stuffing registers and doing an INT 21, OS/2
functions are invoked by putting arguments on the stack and doing a FARCALL.
But this should not delude us into thinking that the OS/2 API actually is
high-level, only the parameter passing mechanism is high level.
There is no rule saying we have to use the OS/2 facilities in the "raw" form
in which Microsoft and IBM provide them, however. In fact, aside from demo
programs like those in Listings One and Two, OS/2 facilities should never be
used directly. For reasons of portability, readability, maintainability, and
various other "bilities," which I've forgotten, code that directly depends on
OS/2 should be isolated from application-level code. Present company excluded,
distrust any programmer who puts #include "os2.h" in the main module of their
program. OS/2 may have that HLL look, but putting DosGetProcAddr( ) in the
middle of main( ) is the same as putting int86(0x21, &r, &r) in the middle of
main( ).
PROC1.C (Listing Four, page 102) provides a level of abstraction on top of the
OS/2 Module Manager. This file #includes "os2.h," but a program that uses
run-time dynlinks doesn't. Instead, it #includes "procaddr.h" (see Listing
Five, page 102).
While the functions in Listing Four are only two lines each, they add
considerable expressive power over the native OS/2 versions. Now, we can say
things as shown in Example 1.
Example 1: Expressive functions

 WORD alias = loadmodule("ALIAS");
 PFN listsyn = (PFN) getprocaddr (alias, "LIST_SYN");
 if (listsyn)
 (*listsyn) ();
 freemodule (alias);

or:

 PFN listsyn;
 if (listsyn = (PFN) procaddr ("ALIAS", "LIST_SYN"))
 (*listsyn) ();

PFN is the typedef that OS/2 provides for pascal function pointers. Why does
getprocaddr() in Listing Four instead return a ULONG (4-byte unsigned number)?
The answer is: Partially as a reminder that a far function pointer is just a
4-byte number, but mainly because the Microsoft C compiler dislikes casting
pascal function pointers to cdecl or vice versa, though it accepts casting
between a function pointer and a ULONG (it correctly warns about "different
levels of indirection").
The C program in Listing Six, page 102 (CALLDLL2.C) uses the higher-level
routines in Listing Four; it dynamically links to _printf in CRTLIB.DLL, the
"C run-time library in a DLL" provided with Microsoft C 5.1. The file size is
a sure sign that something strange is going on here: We're calling printf()
but CALLDLL2.EXE is only 1817 bytes! One odd property of CRTLIB.DLL is that
you can run-time dynamically link to it only if you are also load-time
dynamically linked to it; this is because an undocumented function in
CRTLIB.DLL (_CRT_INIT) is used to initialize the C run-time library.
Note that CALLDLL2.C does not #include <stdio.h>, even though it calls printf(
). printf( ) is coming to us only at run-time. There's no point in telling the
linker about it, much less the compiler and preprocessor! This is a difficult
point to sink in: the only reason CALLDLL2.C is able to use printf( ) is
because it passed the string CRTLIB to loadmodule( ) and the string _printf to
getprocaddr( ).
Some of the lines in CALLDLL2.C are numbered to make discussion easier. Line 1
shows the simple call (*printf)( ). Line 2 uses the newer ANSI C style for
calling through a function pointer, in which pfn( ) is the same as (*pfn)( ).
This matches Modula-2, in which the syntax for indirect calling through a
procedure pointer is indistinguishable from a "normal" call. In Line 3, printf
displays its own address, using %Fp. printf( ) returns the number of
characters printed. Line 4 makes sure that we really can get this retval.


String Invocation



"The general format of a procedure call is
 expression(parameters)
Usually we use the declared procedure name as the expression, but there is
nothing to prevent us from writing an arbitrarily complicated expression."
-- Martin Richards and Colin Whitby-Strevens BCPL: The Language and its
Compiler (1980)
Line 5 of Listing Six is a little odd, but it best represents what we're
actually doing:
 ((CFN) getprocaddr(loadmodule ("CRTLIB"),"_printf"))("Goodbye");
The pointer to printf is just the retval from getprocaddr; we call printf
through this function retval. The assembly equivalent is shown in Example 2.
What's going on here is "string invocation." Essentially, we're not calling
printf through a pointer (which is just a temporary unnamed value on the
stack), we're calling printf through its ASCIIZ name. The string function
names that appear in "string invocation" are part of the larger class of
objects, "executable strings" (the DDE_EXECUTE strings used in OS/2 and
Windows Dynamic Data Exchange are executable strings). Syntactic sugar for
this might be: "CRTLIB._printf' ("Goodbye");
Example 2: Assembly language equivalent

 PUSH "Goodbye"
 PUSH "_printf"
 PUSH "CRTLIB"
 CALL loadmodule
 ; loadmodule consumed "CRTLIB"
 ; and produced handle to crtlib
 CALL getprocaddr
 ; getprocaddr consumed crtlib-handle and "_printf"
 ; and produced pointer to printf on top of stack
 ; "Goodbye" is still on stack
 CALL [top of stack]
 POP retval from _printf

Because () is a binary operator between a function and the set of its
arguments, this weird syntax could be implemented in C++ by overloading the (
) function-invocation operator. Actually, the Icon programming language
(successor to SNOBOL) offers just this syntax, in which "write" ("Goodbye") is
equivalent to write("Goodbye").
At this point, we had better ask again, "Of what possible use is that?"
Because run-time dynamic linking means that an ASCIIZ string can be used in
place of the name of a procedure, any expression that yields such a string can
also be used. The procedure to be invoked is just as much a variable as the
arguments it takes.
The module name and function name are just strings, why hard-wire them into
the executable? Why not pass them on the command line? Let argv[1] be passed
to loadmodule( ) and argv[2] be passed to getprocaddr( )! Let argv[3..argc]
hold any arguments expected by the function whose name we've put in argv[2].
The result is a general-purpose program in which the function to be invoked
and the arguments to be passed to it are completely up in the air until the
program runs. This is exactly what we're going to do: Write a mini C
interpreter, in very few lines of code, using OS/2 run-time dynlinks.


Mini Tiny Small C


The mini-interpreter's syntax is shown in Figure 1where [args . . .] are zero
or more arguments to the function, and each one is either a string, a
character, an unsigned word, a long or a float. CALLDLL.C (Listing Seven, page
102) uses some dumb, but generally effective, rules to figure out the type of
an argument. [%mask] is an optional printf( ) mask that both designates the
type of the function's return value, and displays that return value. For
instance, %s tells CALLDLL that a function returns a 4-byte value, which
should be printed out as a string. The default retval mask is %u, which gets a
two - byte retval and displays it as an unsigned word. Example 3 lists some
legal calls to the interpreter.
Figure 1: Syntax for the mini-interpreter

 calldll <module name> <function name or ordinal number> [args...] [%mask]

Example 3: Legal calls to the interpreter

 calldll viocalls VIOWRTTTY "hello world" 11 0
 calldll doscalls DosBeep 2000 300
 calldll doscalls 50 2000 300 ; DOSBEEP
 calldll doscalls DosMkDir \foobar 0L
 calldll doscalls DosRmDir \foobar 0L
 calldll pmwin WINQUERYACTIVEWINDOW 1L 0 %lu
 calldll crtlib _printf "goodbye world: %lu" 666L
 calldll crtlib SQRT -1.0 %f
 calldll crtlib _toupper 'b' %c
 calldll jpilib FIO$Exists 12 CALLDLL.EXE

This is truly a general-purpose program: It can write to the screen, beep the
speaker, get the square root of -1, get the HWND of the active window in PM,
or perform pretty much any action put in a DLL.
For the most part, CALLDLL is an ignorant "pass through." The intelligence for
associating the strings VIOCALLS and VIOWRTTTY with the function VioWrtTty( ),
for example, is located entirely within OS/2's module manager. CALLDLL blindly
passes the first two arguments on the command-line to DosLoadModule and
DosGetProcAddr.
Ignorance is bliss: The more ignorant the pass-through is, the more powerful.
Because CALLDLL does no interpretation on module and function names, it can be
used to call functions that didn't even exist when CALLDLL was compiled. This
is the same flexibility as provided by "late binding" in OOPs: Instead of
using a switch( ) statement associating specific actions with various run-time
symbols, a program simply passes the symbol through and the system determines
what action to call.
Now, if CALLDLL represents the ideal dumb pass-through, what are those switch(
) statements in Listing Seven?
While the module and function names are passed through to OS/2, the functions
args and retval are a different matter. Once OS/2 has given us back a function
pointer, we're on our own. Almost all the code in CALLDLL.C is devoted to
pushing arguments on the stack before the function is called, and getting back
a retval afterwards.
Here we run into a problem with all high-level languages: There is no
completely general procedure pointer. C (actually, pre-ANSI C without
prototypes) can easily handle functions with different numbers and types of
arguments gated through the same function pointer, but even C won't multiplex
functions with different types of return values through the same function
pointer. Pushing a function's arguments, calling the function and getting its
return value are too tightly coupled.
In the underlying assembly language there is a completely general function
pointer, because the various components of function calling are clearly
separated:
 PUSH param(s)
 PUSH func
 CALL [top of stack)
 RETRIEVE retval(s)

Forth is probably the only HLL that naturally splits function calls up in this
way. In order to write the mini-interpreter, I needed to simulate this in C.


Jiggling the Stack


The first step is to write a function to push its argument on the stack and to
leave it there. In his book, The Programmer's Essential OS/2 Handbook, David
Cortesi shows how to write such a function for Pascal: "The gimmick is simple:
return without clearing the stack!" To do this in C, the function uses the
Pascal calling convention. Here is the entire function:
 VOID NEAR PASCAL push( ) {}
Given C's relaxed type checking, this can be called with any type of argument.
The compiler dutifully pushes the argument onto the stack and there it stays.
To handle multiple arguments, CALLDLL.C uses a "push loop:" push( ) is called
for each command-line argument, working upwards for the Pascal calling
convention and downwards for cdecl. Note that push() must be called from
within the same function that's going to make the indirect call to consume the
arguments on the stack. For this reason, CALLDLL.C calls push( ) from within a
PUSH_ARG( ) macro that gets expanded inline.
Now the arguments are on the stack. To get the correct return value, our
generic function pointer f must be cast to the appropriate type of function
pointer. It is called with no arguments because its arguments are already on
the stack as shown in Example 4.
Example 4: Arguments already on the stack

 switch (retval_typ)
 {
 case typ_string: printf(mask, ((STRFN) f) ()); break;
 case typ_word: printf(mask, f()); break;
 ...
 }

CALLDLL does nothing with f( )'s return value except print it (using whatever
printf mask the user supplied). The invocation of f( ) takes place as an
argument to printf.
Finally, it is the responsibility of the caller of a cdecl function to pop the
arguments off the stack, so we need a pop( ) function. I couldn't write this
in C, so it is supplied by POP.ASM (Listing Eight, page 104).


Naming DOSCALLS


I said earlier that the DOSCALLS pseudomodule (the API exported by the OS/2
kernel) is an anomaly in that its routines (such as DosGetProcAddr,
DosAllocSeg, or DosOpen) do not have ASCIIZ names present at run-time.
You can easily correct this deficiency. Instead of: calldll DOSCALLS 50 2000
300, you can say: calldll DOSCALLS DosBeep 2000 300.
PROC2.C (Listing Nine, page 104) shows a new version of the higher-level
run-time dynamic-linking routines. Solving the DOSCALLS problem is another
reason to have our own level on top of OS/2 -- we can use it to iron out such
inconsistencies. PROC2.H is the external interface to PROC2.C, and is shown in
Listing Two (page 106).
In the new version of getprocaddr( ), if an ASCIIZ name is passed in for a
function in DOSCALLS, the function getdoscall( ) does a binary search of a
table that associates ASCIIZ names with function pointers.
Where does this table come from? (Warning! Extremely boring material ahead!)
Listing Eleven (page 106) is an AWK script that massages the file BSEDOS.H
into a table of DOSCALLS. The AWK output, DOSCALLS.C, is not reproduced here,
because it is boring. If you don't have AWK but want to produce your own
DOSCALLS.C, you can use any decent text editor on a copy of BSEDOS.H. The
thing is to turn lines like:
 USHORT APIENTRY DosGetProc-Addr(HMODULE, PSZ, PPFN);
into lines like:
"DosGetProcAddr", DosGetProcAddr,
Unfortunately this table introduces a touch of "early binding" into an
otherwisewait-until-the-last-possible-moment-to-do-it program.


A DLL for Handling DLLs


There is one last step in the development of the routines in PROC2.C: Put them
in a DLL. Listing Twelve (page 106) shows PROCADDR.DEF, which defines the DLL
for the OS/2 linker. All the data in PROCADDR.DLL is shared -- multiple
clients of PROCADDR.DLL use the same version of the DOSCALLS table.
Once PROC2.C is in a DLL, we can even call these routines from the CALLDLL
command line. For those who like this sort of thing, this is the sort of thing
that they like:
C > calldll PROCADDR PROCADDR
 PROCADDR PROCADDR %
 Fp044F:00A0
This calls the function procaddr( ) in PROCADDR.DLL, passing it the parameters
PROCADDR and PROCADDR, so that the address of procaddr( ) itself is printed
out.
Because it is extremely stupid, CALLDLL is general-purpose and powerful. On
the other hand, because CALLDLL knows nothing about the routines that it
calls, it can't check the arguments passed on the command-line. For example,
CALLDLL DOSCALLS DOSGETINFOSEG 1L 2L is wrong. But because OS/2 runs under
protected mode this isn't too bad. Instead of crashing your machine, OS/2 just
terminates CALLDLL and displays the ever-popular GP Fault dump.
If this were a genuine interpreter that tried to execute more than one line of
user code, this would be a big problem. Even though the error lies in the
user's code, OS/2 terminates the interpreter because, as far as it's
concerned, that's what caused the general-protection violation. It would seem
the only choice for OS/2 interpreters is either to verify all user input, or
to let OS/2 close you down because of bad user input. In fact, there is a way
to catch GP faults in OS/2, using DosPTrace( ), but this is a subject for
another article.
Other improvements could be made to CALLDLL: Interpreting multiple lines of
code from a file, putting function results into variables, taking the address
of variables (for example, how would one properly call DosGetInfoSeg?). These
are left as an exercise for the reader.


Extensible OS/2


Richard Stallman's original article on Emacs ("EMACS: The Extensible,
Customizable, Self-Documenting Display Editor," Proceedings of the ACM SIGPLAN
SIGOA Symposium on Text Manipulation, June, 1981) is sort of the manifesto of
extensible systems. One of the requirements he sets forth is that "an on-line
extensible system must be able to accept, and then execute, new code while it
is running." Stallman goes on to argue that what one needs is a good language
(Lisp, for example) rather than a good operating system with dynamic linking
(such as Multics). In fact, there is no conflict here, because OS/2 is as
extensible an operating system as Lisp is a language.
Compiling and linking are processes of throwing away information. But dynamic
linking relies on the presence at run time of names usually discarded in
compiling or linking. Keeping the ASCIIZ names of functions around in DLLs
means that executables under OS/2 have moved a little closer to the type of
"object code" found in interpreted environments and debuggers.
Should OS/2 succeed, run-time dynlinks will play an important role in the
construction of extensible languages and of products with embedded languages
and add-in facilities. For a system built around add-in DLLs, run-time dynamic
linking is the ultimate Add-In Manager.


_LINKING WHILE THE PROGRAM IS RUNNING_
by Andrew Schulman


[LISTING ONE]

/*
calldll1.c -- run-time dynamic linking to Estes's ALIAS.DLL
cl -Lp calldll1.c
*/

#include <stdlib.h>
#include <stdio.h>
#define INCL_DOSMODULEMGR
#include "os2.h"

#define NIL ((void far *) 0)

void fail(char *msg) { puts(msg); exit(1); }

void main()
{
 unsigned (far pascal *addsyn)(char far *msg);
 unsigned (far pascal *listsyn)(void);
 unsigned alias;

 if (DosLoadModule(NIL, 0, "ALIAS", &alias) != 0)
 fail("can't find ALIAS");
 DosGetProcAddr(alias, "ADDSYN", &addsyn);
 DosGetProcAddr(alias, "LIST_SYN", &listsyn);
 (*addsyn)("ep \\os2\\eps\\epsilon");
 (*listsyn)();
 DosFreeModule(alias);
}







[LISTING TWO]

MODULE calldll;
(* JPI TopSpeed Modula-2 for OS/2 *)
(* run-time dynamic linking to Estes's ALIAS.DLL *)

FROM InOut IMPORT WriteString, WriteLn;
IMPORT Dos;

PROCEDURE fail (msg : ARRAY OF CHAR);
BEGIN
 WriteString(msg); WriteLn; HALT;
END fail;

VAR
 addsyn : PROCEDURE (ADDRESS) : CARDINAL;
 listsyn : PROCEDURE () : CARDINAL;
 alias : CARDINAL;

 ret : CARDINAL; (* ignored retval *)

BEGIN
 IF (Dos.LoadModule(NIL, 0, "ALIAS", alias) # 0) THEN
 fail("can't find ALIAS");
 END;
 ret := Dos.GetProcAddr(alias, "ADDSYN", PROC(addsyn));
 ret := Dos.GetProcAddr(alias, "LIST_SYN", PROC(listsyn));

 (* In the next line, the string _must_ be passed as an
 ADDRESS, not as an ARRAY OF CHAR: Modula-2 passes open
 arrays as _six_ bytes on the stack -- two bytes for the
 length, followed by the address of the array itself --
 but OS/2 DLL's generally expect only the string itself
 (zero-terminated of course). *)

 ret := addsyn(ADR("ep \os2\eps\epsilon"));
 ret := listsyn();
 ret := Dos.FreeModule(alias);
END calldll.








[LISTING THREE]

; calldll.lsp
; OS2XLISP run-time dynamic linking to Estes's ALIAS.DLL

(define alias (loadmodule "ALIAS"))
(if (zerop alias)
 (error "can't find ALIAS"))
(call (getprocaddr alias "ADDSYN") "ep \os2\eps\epsilon")
(call (getprocaddr alias "LIST_SYN"))
(freemodule alias)







[LISTING FOUR]

/*
proc1.c -- implements higher-level access to OS/2 run-time dynlinks
cl -c -Lp proc1.c
*/

#define INCL_DOSMODULEMGR
#include "os2.h"
#include "procaddr.h"

#define NIL ((void far *) 0)


WORD loadmodule(ASCIIZ name)
{
 WORD h;
 return DosLoadModule(NIL, 0, name, (PHMODULE) &h) ? 0 : h;
}

ULONG getprocaddr(WORD module, ASCIIZ name)
{
 ULONG pf;
 return DosGetProcAddr(module, name, (PPFN) &pf) ? 0 : pf;
}

ULONG procaddr(ASCIIZ module, ASCIIZ name)
{
 return getprocaddr(loadmodule(module), name);
}

BOOL freemodule(WORD h)
{
 return (! DosFreeModule(h));
}







[LISTING FIVE]

/*
procaddr.h -- higher-level access to OS/2 run-time dynlinks
*/

typedef unsigned WORD;
typedef unsigned short BOOL;
typedef unsigned long ULONG;
typedef char *ASCIIZ;

WORD loadmodule(ASCIIZ name);
ULONG getprocaddr(WORD module, ASCIIZ name);
ULONG procaddr(ASCIIZ module, ASCIIZ name);
BOOL freemodule(WORD handle);









[LISTING SIX]

/*
calldll2.c -- run-time dynamic linking to CRTLIB.DLL, using PROC1.C
requires MSC 5.1 CRTEXE.OBJ

cl -AL -c calldll2.c proc1.c

link /nod/noi crtexe.obj calldll2 proc1,calldll2,,crtlib.lib os2;

output:
 Hello from calldll2
 Hello again, using new ANSI C style
 _printf lives at 03EF:1098
 printf returned 27
 Goodbye
*/

#include "procaddr.h"

typedef ULONG (far cdecl *CFN)();

main(int argc, char *argv[])
{
 WORD (far cdecl *printf)();
 WORD crtlib;
 WORD ret;

 crtlib = loadmodule("CRTLIB");
 printf = (CFN) getprocaddr(crtlib, "_printf");
 (*printf)("Hello from %s\n", argv[0]); /* 1 */
 printf("Hello again, using new ANSI C style\n"); /* 2 */
 ret = printf("_printf lives at %Fp\n", printf); /* 3 */
 printf("printf returned %d\n", ret); /* 4 */
 ((CFN) getprocaddr(loadmodule("CRTLIB"),"_printf"))("Goodbye"); /* 5 */
 freemodule(crtlib);
}








[LISTING SEVEN]

/*
calldll3.c -- run-time dynamic linking from the command-line
requires MSC 5.1 CRTEXE.OBJ, uses proc2.obj or procaddr.dll
doesn't include "os.h"

to use proc2.obj:
cl -AL -c -Gs2 -Ox -W2 calldll3.c proc2.c
link /nod/noi crtexe.obj calldll3 proc2,calldll3.exe,,crtlib.lib os2.lib;

to use procaddr.dll (IMPLIB procaddr.lib):
cl -AL -c -Gs2 -Ox -W2 calldll3.c
link /nod/noi crtexe.obj calldll3,calldll3,,procaddr.lib crtlib.lib os2.lib;

to run:
calldll3 <module name> <function name or ordinal number> [args...] [%mask]
examples:
calldll3 VIOCALLS VIOWRTTTY "hello world" 5 0
calldll3 doscalls DosMkDir \foobar 0L
calldll3 doscalls DosRmDir \foobar 0L
calldll3 DOSCALLS DosBeep 2000 300

calldll3 DOSCALLS 50 2000 300 ; DosBeep
calldll3 CRTLIB _printf "goodbye world: %lu" 666L " [%d]"
calldll3 CRTLIB ACOS -1.0 %.15f
calldll3 CRTLIB SQRT -1.0 %f
calldll3 CRTLIB _toupper 'b' %c
calldll3 PROCADDR LOADMODULE PROCADDR %X
*/

#include <mt\stdlib.h>
#include <mt\stdio.h>
#include <mt\string.h>

#include "local.h"
#include "proc2.h"

typedef enum { typ_string, typ_byte, typ_word, typ_long, typ_float } TYPE;

TYPE NEAR type(char *arg);
TYPE NEAR retval_type(char *s);

VOID fail(char *msg) { puts(msg); exit(1); }

/*
 push() : see Cortesi, Programmer's Essential OS/2 Handbook, pp.136-137
*/
VOID NEAR PASCAL push() { }
extern WORD pop(void);

#define PUSH_ARG(arg) \
{ \
 switch (type(arg)) \
 { \
 case typ_string: push(arg); c += 2; break; \
 case typ_byte: push(arg[1]); c += 1; break; \
 case typ_word: push(atoi(arg)); c += 1; break; \
 case typ_long: push(atol(arg)); c += 2; break; \
 case typ_float: push(atof(arg)); c += 4; break; \
 } \
}

#define SYNTAX_MSG \
 "syntax: calldll3 <module name> <func name or ord#> [args...] [%mask]"

main(int argc, char *argv[])
{
 FN f;
 TYPE retval_typ = typ_word;
 char *mask = "%u";
 WORD module;
 BOOL is_cdecl;
 int i, c;

 if (argc < 3)
 fail(SYNTAX_MSG);

 /* handle optional printf mask */
 if (strchr(argv[argc-1], '%'))
 retval_typ = retval_type(mask = argv[--argc]);


 if ((module = loadmodule(argv[1])) == 0)
 fail("can't load module");

 /* pass ASCIIZ string or ordinal number */
 f = getprocaddr(module, isdigit(argv[2][0]) ? atol(argv[2]) : argv[2]);
 if (! f)
 fail("can't get function");

 is_cdecl = ! (strcmp(strupr(argv[1]), "CRTLIB"));

 /* push in reverse order for cdecl */
 if (is_cdecl)
 {
 for (i=argc-1, c=0; i>=3; i--)
 PUSH_ARG(argv[i]);
 }
 else
 {
 for (i=3; i<argc; i++)
 PUSH_ARG(argv[i]);
 }

 /* args are on the stack : call (*f)() and print retval */
 switch (retval_typ)
 {
 case typ_string: printf(mask, ((STRFN) f)()); break;
 case typ_byte: printf(mask, ((BYTEFN) f)()); break;
 case typ_word: printf(mask, f()); break;
 case typ_long: printf(mask, ((LONGFN) f)()); break;
 case typ_float: printf(mask, ((FLOATFN) f)()); break;
 }

 if (is_cdecl)
 for (i=0; i<c; i++)
 pop();

 freemodule(module);
 return 0;
}

/*
 type() uses some dumb rules to determine the type of an argument:
 if first character of arg is a digit or '-'
 and if arg contains '.' then it's a floating-point number
 else if last character is an 'L' then it's a long
 else it's a unsigned word
 else if first character is an apostrophe
 it's a single-byte character
 otherwise
 it's a string
*/
TYPE NEAR type(char *arg)
{
 if (isdigit(arg[0]) (arg[0] == '-' && isdigit(arg[1])))
 {
 char *p = arg;
 while (*p)
 if (*p++ == '.')
 return typ_float;

 return (*--p == 'L') ? typ_long : typ_word;
 }
 else
 return (arg[0] == '\'') ? typ_byte : typ_string;
}

/*
 retval_type() uses a printf() mask (e.g., %s or %lX) to determine
 type of return value
*/
TYPE NEAR retval_type(char *s)
{
 while (*s)
 {
 switch (*s)
 {
 case 's' : return typ_string; break;
 case 'c' : return typ_byte; break;
 case 'p' : case 'l' : case 'I' : case 'O' : case 'U' :
 return typ_long; break;
 case 'e' : case 'E' : case 'f' : case 'g' : case 'G' :
 return typ_float; break;
 }
 s++;
 }

 /* still here */
 return typ_word;
}







[LISTING EIGHT]

; pop.asm

DOSSEG
.MODEL large
.CODE pop_text
PUBLIC _pop, _sp

_pop proc far
 ; save away far return address
 pop cx
 pop bx
 ; pop word off stack and return it in AX
 pop ax
 ; push far return address back on stack
 push bx
 push cx
 ret
_pop endp

; useful for testing
_sp proc far

 mov ax,sp
 ret
_sp endp

end






[LISTING NINE]

/*
proc2.c

to make procaddr.dll:
cl -Alfu -c -Gs2 -Ox -W2 -DDLL proc2.c
link /nod/noi proc2,procaddr.dll,,llibcdll.lib os2,procaddr.def;
implib procaddr.lib procaddr.def
copy procaddr.dll \os2\dll
*/

#include <string.h>

#ifdef DLL
int _acrtused = 0;
#endif

#define INCL_DOS
#include "os2.h"

#include "local.h"
#include "proc2.h"

typedef struct {
 char *name;
 USHORT (APIENTRY *f)();
 } DOSCALLS;

/*
 include table generated from BSEDOS.H with AWK script DOSCALLS.AWK
 table looks like:
 LOCAL DOSCALLS NEAR dos[] = {
 "", 0,
 ...
 "DosGetHugeShift", DosGetHugeShift,
 "DosGetInfoSeg", DosGetInfoSeg,
 ...
 } ;
 DOSCALLS.C also contains #define NUM_DOSCALLS
*/
#include "doscalls.c"

LOCAL FN NEAR getdoscall(ASCIIZ name);
LOCAL USHORT NEAR doscalls = 0;

WORD pascal loadmodule(ASCIIZ name)
{

 WORD h;
 return DosLoadModule((void far *) 0, 0, name, (PHMODULE) &h) ? 0 : h;
}

/*
 if name is actually a four-byte ordinal number, use it as is
 otherwise if module is not DOSCALLS, use it as is
 otherwise if module is DOSCALLS, get ordinal number and use it instead
*/
FN pascal getprocaddr(WORD module, ASCIIZ proc)
{
 FN f;

 if (! doscalls) doscalls = loadmodule("DOSCALLS");

 if ((module == doscalls) && FP_SEG(proc))
 return getdoscall(proc);
 else
 return DosGetProcAddr(module, proc, (PPFN) &f) ? 0 : f;
}

FN pascal procaddr(ASCIIZ module, ASCIIZ proc)
{
 return getprocaddr(loadmodule(module), proc);
}

BOOL pascal freemodule(WORD h)
{
 return (! DosFreeModule(h));
}

/*
 do binary search of table, looking for name, returning function ptr
 */
LOCAL FN NEAR getdoscall(ASCIIZ name)
{
 signed cmp, mid;
 signed base = 1, top = NUM_DOSCALLS+1;

 name = strupr(name);

 for (;;)
 {
 mid = (base + top) / 2;
 cmp = strcmp(name, strupr(dos[mid].name));

 if (cmp == 0) return (FN) dos[mid].f;
 else if (mid == base) return 0;
 else if (cmp < 0) top = mid;
 else if (cmp > 0) base = mid;
 }
}






[LISTING TEN]


/*
proc2.h
*/

extern WORD pascal loadmodule(ASCIIZ name);
extern FN pascal getprocaddr(WORD module, ASCIIZ proc);
extern FN pascal procaddr(ASCIIZ module, ASCIIZ proc);
extern BOOL pascal freemodule(WORD handle);







[LISTING ELEVEN]

# doscalls.awk
# creates doscalls.c from bsedos.h
# doscalls.c is #included by proc2.c
# C>sort -b +2 \os2\inc\bsedos.h awk -f doscalls.awk > doscalls.c

# bsedos.h contains prototypes such as:
# USHORT APIENTRY DosCreateThread(PFNTHREAD, PTID, PBYTE);
# doscalls.awk turns these into string name/function ptr pairs:
# "DosCreateThread", DosCreateThread,

BEGIN { init() }

END { fini() }

$2 ~ /APIENTRY/ && $3 ~ /Dos/ { doscall($3) }

function init() {
 print "/* doscalls.c */"
 print "LOCAL DOSCALLS NEAR dos[] = {"
 print "\"\",\t0,"
 }

function fini() {
 print "} ; "
 print "#define NUM_DOSCALLS\t", num_doscalls
 }

function doscall(s) {
 gsub(/\(/, " ", s) # replace open paren with space
 split(s, arr) # tokenize
 print "\"" arr[1] "\", " arr[1] "," # print with and without quotes
 num_doscalls++
 }









[LISTING TWELVE]

; procaddr.def

LIBRARY PROCADDR
DESCRIPTION 'Run-Time Dynamic Linking'
DATA SINGLE SHARED
PROTMODE
EXPORTS
 LOADMODULE
 GETPROCADDR
 PROCADDR
 FREEMODULE





[LISTING THIRTEEN]

/*
local.h -- miscellaneous definitions
*/

typedef unsigned short WORD;
typedef unsigned short BOOL;
typedef char far *ASCIIZ;
typedef unsigned long ULONG;
typedef double FLOAT;
typedef WORD (far *FN)();
typedef ASCIIZ (far *STRFN)();
typedef char (far *BYTEFN)();
typedef WORD (far *WORDFN)();
typedef ULONG (far *LONGFN)();
typedef FLOAT (far pascal *FLOATFN)();

#define FP_SEG(p) ((WORD) ((ULONG) (p) >> 16))
#define FP_OFF(p) ((WORD) (p))

#define isdigit(c) ((c) >= '0' && (c) <= '9')

#ifndef NEAR
#define NEAR near
#define PASCAL pascal
#define VOID void
#endif

#define LOCAL static


Example 1: Expressive functions

 WORD alias = loadmodule("ALIAS");
 PFN listsyn = (PFN) getprocaddr(module, "LIST_SYN");
 if (listsyn)
 (*listsyn)();
 freemodule(alias);

or:


 PFN listsyn;
 if (listsyn = (PFN) procaddr("ALIAS", "LIST_SYN"))
 (*listsyn)();



Example 2: Assembly language equivalent

 PUSH "Goodbye"
 PUSH "_printf"
 PUSH "CRTLIB"
 CALL loadmodule
 ; loadmodule consumed "CRTLIB"
 ; and produced handle to crtlib
 CALL getprocaddr
 ; getprocaddr consumed crtlib-handle and "_printf"
 ; and produced pointer to printf on top of stack
 ; "Goodbye" is still on stack
 CALL [top of stack]
 POP retval from _printf


Example 3: Legal calls to the interpreter

 calldll viocalls VIOWRTTTY "hello world" 11 0
 calldll doscalls DosBeep 2000 300
 calldll doscalls 50 2000 300 ; DOSBEEP
 calldll doscalls DosMkDir \foobar 0L
 calldll doscalls DosRmDir \foobar 0L
 calldll pmwin WINQUERYACTIVEWINDOW 1L 0 %lu
 calldll crtlib _printf "goodbye world: %lu" 666L
 calldll crtlib SQRT -1.0 %f
 calldll crtlib _toupper 'b' %c
 calldll jpilib FIO$Exists 12 CALLDLL.EXE


Example 4: Arguments already on the stack

 switch (retval_typ)
 {
 case typ_string: printf(mask, ((STRFN) f)()); break;
 case typ_word: printf(mask, f()); break;
 ...
 }


Example 5: Associating ASCIIZ names with function pointers

 LOCAL DOSCALLS NEAR dos[] = {
 "", 0,
 ...
 "DosGetProcAddr", DosGetProcAddr,
 "DosGetPrty", DosGetPrty,
 ...
 } ;


































































November, 1989
CONTAINER OBJECT TYPES IN TURBO PASCAL


Extend the power of Turbo Pascal by "containerizing" data structures




Anders Hejlsberg


Anders is the chief architect of Turbo Pascal, now in its fifth generation.
Anders is currently working on Turbo Pascal and other core technology products
at Borland Int., Scotts Valley, Calif.


In modern programming, most applications depend heavily on data structuring
methods that lie far beyond the built-in functionality of a programming
language. In fact, large portions of typical applications are devoted to the
maintenance of data structures, such as linked lists, dynamic arrays, binary
trees, and so on. But even so, few, if any, programming languages provide
built-in support for data structures other than simple records and arrays. As
a result, scores of programmers are forced to "invent the wheel" over and over
again as they write and debug the necessary data structure management code for
each new application. Among the key benefits of object oriented programming
(OOP) is the ability to create Container Object Types which, when packaged in
library modules, essentially extend the underlying programming language with
new ways of structuring data. Using Container Object Types, a programmer
needn't worry about insertion and deletion on linked lists, balancing of
binary trees, sorting of arrays, and so on; all such algorithms can be
implemented once and for all in a library of Container Object Types. In this
article, I'll demonstrate how to implement and use a number of Container
Object Types in Turbo Pascal 5.5. In particular, I'll implement Container
Object Types for linked lists and binary trees, and show how these structures
can be used in a Pascal cross-reference generator program.


Container Object Types


You may think you know nothing about Container Object Types, when in fact you
probably use them in every program you write. The Pascal array type, for
instance, is an example of a container type (although not an object-oriented
one). An array contains a number of elements of a specific type, and allows
access to those elements through indices. And a Container Object Type is just
that: A type that contains a number of elements and allows access to those
elements in some way.
Consider, for example, a linked list. It consists of a number of nodes, each
of which contains a link that points to the next node in the list. To access
the list, one follows links down the chain of nodes. Both the linked list, and
the array contain a number of elements and allow access to those elements in
some way.
Many other data structures are natural candidates for implementation as
container types. Stacks (last-in, first-out), queues (first-in, first-out),
trees, dynamic arrays, and hash tables, like linked lists and arrays, all
share the ability to store and access data elements and could be profitably
"containerized."
While some applications may be able to get along with fixed-sized arrays, most
programs require structures that dynamically resize themselves, that have the
ability to sort themselves, and so on. When the number of elements in a
structure isn't known beforehand, and when access to the elements is strictly
sequential, a linked list is probably more appropriate than a fixed-size
array. Likewise, when elements have to be sorted, a binary tree may be a
better choice. It seems strange, then, that most programming languages only
provide a couple of container types (such as array and record), when indeed
there are many others to consider. Wouldn't it be nice, for example, to be
able to declare a "list of Windows," a "queue of Transactions," or a "tree of
Identifiers." With Container Object Types, you can do just that, although the
syntax may be a little different.
Container Object Types allow lists, stacks, queues, trees, and other data
structures to be made truly generic, almost as if they were language
extensions. Just as procedures and functions in units (such as Dos, Overlay,
and Graph) are extensions to the set of built-in routines (such as WriteLn,
ReadLn, Length, and Sqrt) of Turbo Pascal, units with Container Object Types
can act as extensions to the built-in array and record types.
The benefits are numerous. First of all, Container Object Types save time,
because you don't have to write and debug code to manage data structures over
and over again. Such code can be written and debugged once, and then reused in
any number of applications. Second, Container Object Types can save space and
can reduce errors. Imagine, for example, an application that uses linked lists
for a number of different purposes. There might be a linked list of windows
(the desktop), a linked list of menus (the menu bar) with linked lists of menu
items, and so on. Instead of having separate linked-list managers for windows,
menus, and menu items, there can be one generic Container Object Type that
manages them all, with a resulting reduction in code space and potential bugs.
Finally, if all linked lists in an application are derived from one Container
Object Type, then any optimizations made to that container will improve
performance throughout the entire application.


Implementing and Using Container Object Types


When it comes to implementation, Container Object Types are no different than
ordinary object types. A container type has data fields and methods, and it
can inherit from other object types. The methods of a Container Object Type
provide access to the data stored in the container. Where brackets ([]) are
used to access an array, method calls are used to access a container. A linked
list container, for example, will have methods to insert, append, and remove
elements, and methods to get at the first, last, next, and previous elements
in the list.
Implementing a Container Object Type is a true exercise in data abstraction.
When you write code for the methods of a Container Object Type, you deal only
with the structure of the data in the container, not with the data itself.
Just as you can index an array without knowing exactly what the array
contains, you can insert, append, and remove elements on a linked list without
knowing exactly what the elements are. But you do need to know something about
the elements. In the case of an array, you need to know the size of each
element, and in the case of a linked list, you need to know where to find the
link to the next element. The key is to know just enough about the elements to
access them, but not more.
Such partial knowledge of container elements is achieved through abstract
element types. Each element on a linked list needs a pointer to the next
element in the list:
 type
 ListNodePtr = ^ListNode;
 ListNode = object
 Next: ListNodePtr;
 end;
If each element on the linked list is an object type derived from ListNode,
then each element is guaranteed to have a Next link. For example, a linked
list element that contains an integer value could be defined as:
 type
 IntNodePtr = ^IntNode;
 IntNode = object(ListNode)
 Value: Longint;
 end;
The ListNode type is known as an abstract type. It exists only so that other
types can be derived from it, but is otherwise useless, because it contains no
data. IntNode, on the other hand, contains useful data, and because it is
derived from ListNode, it inherits the Next field, and can act as a ListNode
in a linked list. Of course, you can derive other types from ListNode, such as
StringNode, WindowNode, and MenuNode, all of which can be put on a linked
list.
Given the ListNode type, now implement a Container Object Type that "contains"
it. In this case, I'll show a simple last-in, first-out stack of ListNodes,
which has methods to Push and Pop elements:
 type
 StackList = object
 First: ListNodePtr;
 procedure Push(N: ListNodePtr);
 function Pop: ListNodePtr;
 ...
 end;
To "push" an element onto the stack, the new element is inserted at the
beginning of the list:
procedure StackList.Push(N:ListNodePtr);
begin

 N^.Next := First;
 First := N;
end;
To "pop" an element, return the first element on the list and remove that
element from list:
function StackList.Pop: ListNodePtr;
begin
 Pop := First;
 if First <> nil then First := First^.Next;
end;
Here's a simple program fragment that uses the StackList container type:
 var
 S: StackList;
 P: IntNodePtr;
 begin
 ...
 S.Push(P);
 P := IntNodePtr(S.Pop);
 ...
 end;

Notice in particular that a typecast is required to assign the result of Pop
to P, but that it is not required to pass P to Push. From the compiler's point
of view, Push and Pop operate only on elements of type ListNode -- in fact,
Push and Pop know nothing about the existence of an IntNode type. When passing
P to Push, the compiler can guarantee that P is a ListNode, because IntNode is
derived from ListNode. But going the other way and assigning the result of Pop
to P, the compiler can make no such assumption. The pointer to a ListNode
returned by Pop is not known at compile time to be also a pointer to an
IntNode. Because I have only Pushed pointers to IntNodes, however, I can
safely convert the return value by using a typecast.
Such typecasts are quite common in applications that use Container Object
Types. The compiler doesn't know what is in the container, but the application
does, and it can safely promote the elements.


The Contain.PAS Unit


To demonstrate the implementation of Container Object Types in Turbo Pascal
5.5, I have provided Contain.PAS, which is shown in Listing One (see page
108). The Contain unit implements a linked list container and a binary tree
container.
The Base type is an abstract object type that serves as the ultimate ancestor
of all these object types. The only thing it declares is a destructor called
"Done3," and in itself, the Base type is quite useless, though objects derived
from Base are guaranteed to always have destructors.
The List type implements a linked list container, which contains elements that
are derived from the ListNode type. The List's Last field points to the last
ListNode in the list, and the Next field of the last ListNode points to the
first node, which then points to the next node, and so on. Storing the list in
a circular fashion like this allows the efficient implementation of both an
Insert and an Append method; if a pointer to the first element of the list had
been stored instead, the Append method would become very inefficient, because
it would have to traverse the entire list before appending the new element.
Also because of the circular structure, each ListNode can itself provide a
Prev method, which returns the previous node by traversing the entire circle.
The Init constructor initializes the list by setting the Last field to nil,
and the Done destructor disposes the list by disposing each node. The Insert,
Append, and Remove methods are used to insert, append, and remove nodes.
Notice how Remove doesn't dispose the node, but rather just removes it from
the list. First, Last, Next, and Prev are used to traverse the list; they all
return nil to indicate the end of the iteration. First and Next are used to
step forward through the list, and Last and Prev are used to step backwards.
The latter is a rather slow process, though, because finding the previous node
requires traversing the entire list.
The ForEach method applies an action to each node in the list. The action is
specified in the form of a procedure parameter, which must be compatible with
the ListAction procedure type. Starting with the first element in the list,
the Action procedure is called for each element. The ForEach method is an
alternative to the First/Next methods.
Writing:
 L.ForEach(SomeAction);
corresponds to writing:
 P := L.First;
 while P <> nil do
 begin
 SomeAction(P);
 P := L.Next(P);
 end;
The action may be to dispose of the element (which is why the actual source
code in ForEach calls the Next method before calling the Action procedure). An
example of the use of ForEach is the Deletemethod, which uses ForEach to
dispose of the entire list of elements.
The Tree type implements a sorted binary tree, which contains elements that
are derived from the TreeNode type. Each node in the tree has a key value that
is used to automatically sort nodes as they are inserted into the tree. The
Tree's Root field points to the root of the tree, and the Left and Right
fields of each TreeNode point to nodes whose key values are less than and
greater than the key value of the node itself. The Tree does not allow
duplicate key values.
Of particular interest is the Tree's Compare and GetKey methods. These methods
are virtual, and are meant to be overridden by users of the Treetype. The
GetKey method is passed a pointer to a TreeNode and must return a pointer to
the key value field in the node. The Compare method is passed two such key
value pointers, and must return -1, 0, or 1, indicating Key1 < Key2, Key1 =
Key2, and Key1 > Key2, respectively. The default GetKey method simply returns
a pointer to the node itself, which in some cases is sufficient. The default
Compare method, however, always returns 0 (indicating equivalence), which of
course is useless because every node will appear to be identical to every
other node. The Compare method therefore must be overridden with an
implementation that actually compares the keys.
The Insert method inserts a new node into the tree unless a node with the same
key value already exists. The Find method returns a pointer to a node with a
given key value, or nil if no such node exists. Both Insert and Find are
implemented through the more general Search method. Search scans the tree for
a node with a particular key value, and returns a pointer to the node. If the
node doesn't exist, a user-specified function is called. The user function may
create a new node and return a pointer to it, in which case that new node is
entered into the tree. Alternatively, the user function may return nil, in
which case no new node is created. In either case, the result of the user
function also becomes the result of the Search method. The user function is
specified as a function parameter to Search, and the type of the parameter
must be compatible with the TreeCreate function type.
Like the List container type, the Tree container type has a ForEach method
that applies an action to all nodes in the tree, in the order in which the
nodes are sorted. The action is specified in the form of a procedure
parameter, which must be compatible with the TreeAction procedure type. The
action may be to dispose of the node (which is why the Traverse procedure in
ForEach copies the Right pointer into a temporary before calling the Action
procedure). An example of the use of ForEach is the Delete method, which uses
ForEach to dispose of the entire tree.
As an example of the use of the Tree type, consider implementing a binary tree
with nodes that contain a string and are ordered according to the value of the
string:
 StrPtr = ^String;

 StrNodePtr = ^StrNode;
 StrNode = object(TreeNode)
 Value: StrPtr;
 end;
To create a tree of such StrNode elements, a StrTree type is derived from the
generic Treetype, and the Compare and GetKey methods are overridden:
StrTree = object(Tree)
 function Compare(Key1, Key2: Pointer): Integer; virtual;
 function GetKey(N: TreeNodePtr): Pointer; virtual;
end;
The GetKey method returns a pointer to the string value in the node. Notice
how a typecast is required to promote the node from a TreeNode to a StrNode.

function StrTree.GetKey(N: TreeNodePtr): Pointer;
begin
 GetKey := StrNodePtr(N)^.Value;
end;
The Compare method likewise typecasts the key pointers into string pointers,
and compares the string values:
function StrTree.Compare(Key1,Key2: Pointer): Integer;
begin
 if StrPtr(Key1)^<StrPtr(Key2)^ then
 Compare := -1 else
 if StrPtr(Key1)^>StrPtr(Key2)^
 then Compare := 1 else
 Compare := 0;
end;


The Crossref.PAS Program


Now that a Container Object Type module has been implemented, I'll show a
program that actually uses those container types. The Crossref.PAS program
(see Listing Two, page 109) uses linked lists and binary trees to generate a
cross-reference listing of a Pascal source file. A listing of the source file
with line numbers and a list of all identifiers in the source file is
produced, and each identifier is followed by a list of numbers of the lines
that reference the identifier.
To run the CrossRef program, use a command line such as this:
 CROSSREF MYPROG.PAS
This generates a cross-reference listing of MYPROG.PAS on the screen. Because
CrossRef writes to the standard Output file, the cross-reference can be
redirected to a file. For example:
 CROSSREF MYPROG.PAS >MYPROG .CRF
Within the CrossRef program, the LineRef object type is used to track line
numbers of references to a particular identifier. It is derived from the
ListNode type, so that reference line numbers can be kept on a linked list.
An IdentRef object represents an identifier and the line numbers of all the
lines that reference it. The identifier is stored in the Name field as a
pointer to a string, and the line numbers are kept on a linked list using the
Lines field. The IdentRef type is derived from the TreeNode type, so that all
identifiers can be kept in a sorted binary tree.
The IdentTree type implements a tree of IdentRef objects. It is derived from
the generic Treetype, and the Compare and GetKey methods are overridden to
extract and compare key values from IdentRef objects. In particular, GetKey
returns the Name string pointer stored in each IdentRef, and Compare compares
the strings.
The general flow of the CrossRef program is as follows: First the identifier
tree is initialized using the Idents.Init constructor. Then the Input and
Output files are prepared and are assigned buffers. Next, all Turbo Pascal
reserved words are inserted into the tree, so that they can be ignored in the
cross-reference listing. The ProcessFile procedure processes the input file
and produces a source listing with line numbers, and the PrintIdents procedure
prints the identifier cross reference. Finally, the Idents.Done destructor is
called to dispose of the entire cross-reference tree.
The InsertKeyWord procedure uses a recursive binary iteration to insert the
reserved words into the tree. This ensures optimal distribution of the keyword
entries in the tree; if a straight for loop had been used to insert the
keywords, the tree would become a worstcase unbalanced tree, because the
keywords are sorted alphabetically in the KeyWord table.
The ProcessFile procedure "tokenizes" the input stream to isolate identifier
references. It ignores strings, hex-numbers, and comments, and calls GetIdent
to process each identifier. GetIdent, after reading the identifier, uses the
Search method to find or create a corresponding IdentRef node. The NewIdent
procedure is called if Search cannot locate the identifier in the tree, and
NewIdent then creates a new IdentRef with one LineRef in the Lines list.
Because the tree contains only objects of type IdentRef, the result of the
Search method call in GetIdent can be typecast to the IdentRefPtr type. If the
resulting IdentRef has an empty Lines list, it is a reserved word, and such
references are ignored. Otherwise, if the line number of the last reference on
the Lines list is not the current line number, then a new LineRef object is
added to the list.
The PrintIdents procedure calls PrintRef for each IdentRef object in the
Idents tree. PrintRef ignores the IdentRef if the Lines list is empty,
indicating a reserved word; otherwise, it prints the identifier, and calls
PrintLine for each LineRef in the Lines list. PrintLine prints the reference
line number with a maximum of RefPerLine references per line. Again, notice
the typecasts in both PrintRef and PrintLine, which promote the generic
TreeNode and ListNode types into the specific IdentRef and LineRef types.


Conclusion


The ability to add new data types that behave such as built-in data types
makes it possible to extend Pascal itself to behave like a more abstract
language. One only needs to provide a generic object type for stacks, linked
lists, or queues. Then anyone can use the object's operations simply by
creating a type definition for the objects to be manipulated. Objects that are
to be managed can be as simple as a stack of integers or as complex as a list
of windows open on a desktop complete with text buffers, scroll bars, and
mouse support.
Regardless of the complexity of the objects managed, the basic object type
never needs to be modified at the source level. Because the behavior of object
types can be extended and modified through inheritance, users of object
libraries only need the interface specification to modify the behavior of
derived objects. Object libraries differ substantially from the conventional
libraries provided by third-party vendors, where access to source code is a
critical issue. As programmers become less concerned with source code
availability, I think there will be two different types of programming
projects -- projects that provide object libraries and projects that use them.
Builders of object libraries are assured that they can provide truly reusable
code. Users of object libraries will experience tremendous productivity boosts
by inheriting, rather than reinventing the wheel every time they need data
structure management code for a new application.

_CONTAINER OBJECT TYPES IN TURBO PASCAL_
by Anders Hejlsberg


[LISTING ONE]

unit Contain;

{$S-}

interface

type

{ Base object type }

 Base = object
 destructor Done; virtual;
 end;


{ Abstract linked list node type }

 ListNodePtr = ^ListNode;
 ListNode = object(Base)
 Next: ListNodePtr;
 function Prev: ListNodePtr;
 end;

{ Linked list iteration procedure type }

 ListAction = procedure(N: ListNodePtr);

{ Linked list type }

 ListPtr = ^List;

 List = object(Base)
 Last: ListNodePtr;
 constructor Init;
 destructor Done; virtual;
 procedure Append(N: ListNodePtr);
 procedure Delete;
 function Empty: Boolean;
 procedure ForEach(Action: ListAction);
 function First: ListNodePtr;
 procedure Insert(N: ListNodePtr);
 function Next(N: ListNodePtr): ListNodePtr;
 function Prev(N: ListNodePtr): ListNodePtr;
 procedure Remove(N: ListNodePtr);
 end;

{ Abstract binary node type }

 TreeNodePtr = ^TreeNode;
 TreeNode = object(Base)
 Left, Right: TreeNodePtr;
 end;

{ Binary tree iteration procedure type }

 TreeAction = procedure(N: TreeNodePtr);

{ Binary tree node creation procedure type }

 TreeCreate = function(Key: Pointer): TreeNodePtr;

{ Binary tree type }

 TreePtr = ^Tree;
 Tree = object(Base)
 Root: TreeNodePtr;
 constructor Init;
 destructor Done; virtual;
 function Compare(Key1, Key2: Pointer): Integer; virtual;
 procedure Delete;
 function Empty: Boolean;
 function Find(Key: Pointer): TreeNodePtr;
 procedure ForEach(Action: TreeAction);
 function GetKey(N: TreeNodePtr): Pointer; virtual;

 procedure Insert(N: TreeNodePtr);
 function Search(Key: Pointer; Create: TreeCreate):
TreeNodePtr;
 end;

implementation

{ Base methods }

destructor Base.Done;
begin
end;

{ ListNode methods }

function ListNode.Prev: ListNodePtr;
var
 P: ListNodePtr;
begin
 P := Self;
 while P^.Next <> Self do P := P^.Next;

 Prev := P;
end;

{ List methods }

{$F+}

procedure DelListNode(N: ListNodePtr);
begin
 Dispose(N, Done);
end;

{$F-}

constructor List.Init;
begin
 Last := nil;
end;

destructor List.Done;
begin

 Delete;
end;

procedure List.Append(N: ListNodePtr);
begin
 Insert(N);
 Last := N;
end;

procedure List.Delete;
begin
 ForEach(DelListNode);
 Last := nil;
end;


function List.Empty: Boolean;
begin
 Empty := Last = nil;
end;

procedure List.ForEach(Action: ListAction);

var
 P, Q: ListNodePtr;
begin
 P := First;
 while P <> nil do
 begin
 Q := P;
 P := Next(P);
 Action(Q);
 end;
end;

function List.First: ListNodePtr;
begin
 if Last = nil then First := nil else First := Last^.Next;
end;

procedure List.Insert(N: ListNodePtr);
begin
 if Last = nil then Last := N else N^.Next := Last^.Next;
 Last^.Next := N;

end;

function List.Next(N: ListNodePtr): ListNodePtr;
begin
 if N = Last then Next := nil else Next := N^.Next;
end;

function List.Prev(N: ListNodePtr): ListNodePtr;
begin
 if N = First then Prev := nil else Prev := N^.Prev;
end;

procedure List.Remove(N: ListNodePtr);
var
 P: ListNodePtr;
begin
 if Last <> nil then
 begin
 P := Last;
 while (P^.Next <> N) and (P^.Next <> Last) do P := P^.Next;
 if P^.Next = N then

 begin
 P^.Next := N^.Next;
 if Last = N then if P = N then Last := nil else Last := P;
 end;
 end;
end;

{ Tree methods }


var
 NewTreeNode: TreeNodePtr;

{$F+}

function GetTreeNode(Key: Pointer): TreeNodePtr;
begin
 GetTreeNode := NewTreeNode;
end;

procedure DelTreeNode(N: TreeNodePtr);
begin

 Dispose(N, Done);
end;

{$F-}

constructor Tree.Init;
begin
 Root := nil;
end;

destructor Tree.Done;
begin
 Delete;
end;

function Tree.Compare(Key1, Key2: Pointer): Integer;
begin
 Compare := 0;
end;

procedure Tree.Delete;

begin
 ForEach(DelTreeNode);
 Root := nil;
end;

function Tree.Empty: Boolean;
begin
 Empty := Root = nil;
end;

function Tree.Find(Key: Pointer): TreeNodePtr;
begin
 NewTreeNode := nil;
 Find := Search(Key, GetTreeNode);
end;

procedure Tree.ForEach(Action: TreeAction);

 procedure Traverse(P: TreeNodePtr);
 var
 R: TreeNodePtr;

 begin

 if P <> nil then
 begin
 R := P^.Right;
 Traverse(P^.Left);
 Action(P);
 Traverse(R);
 end;
 end;

begin
 Traverse(Root);
end;

function Tree.GetKey(N: TreeNodePtr): Pointer;
begin
 GetKey := N;
end;

procedure Tree.Insert(N: TreeNodePtr);
begin

 NewTreeNode := N;
 N := Search(GetKey(N), GetTreeNode);
end;

function Tree.Search(Key: Pointer; Create: TreeCreate):
TreeNodePtr;

 procedure Traverse(var P: TreeNodePtr);
 var
 C: Integer;
 begin
 if P = nil then
 begin
 P := Create(Key);
 P^.Left := nil;
 P^.Right := nil;
 Search := P;
 end else
 begin
 C := Compare(Key, GetKey(P));
 if C < 0 then Traverse(P^.Left) else
 if C > 0 then Traverse(P^.Right) else

 Search := P;
 end;
 end;

begin
 Traverse(Root);
end;

end.





[LISTING TWO]


program CrossRef;

{$S-}
{$M 8192,8192,655360}

uses Contain;


const

 MaxIdentLen = 20; { Maximum identifier length }
 LineNoWidth = 6; { Width of line numbers in listing }
 RefPerLine = 8; { Line numbers per line in
cross-reference }
 IOBufSize = 4096; { Input/Output buffer size }

 FormFeed = #12;
 EndOfLine = #13;
 EndOfFile = #26;

type

{ Input/Output buffer }

 IOBuffer = array[1..IOBufSize] of Char;

{ Identifier string }

 IdentPtr = ^Ident;
 Ident = string[MaxIdentLen];

{ Line reference object }

 LineRefPtr = ^LineRef;
 LineRef = object(ListNode)
 LineNo: Integer;
 constructor Init(Line: Integer);
 end;

{ Identifier reference object }

 IdentRefPtr = ^IdentRef;
 IdentRef = object(TreeNode)
 Lines: List;
 Name: IdentPtr;
 constructor Init(S: Ident);
 destructor Done; virtual;
 end;

{ Identifier tree }

 IdentTreePtr = ^IdentTree;
 IdentTree = object(Tree)
 function Compare(Key1, Key2: Pointer): Integer; virtual;
 function GetKey(N: TreeNodePtr): Pointer; virtual;
 end;

const


{ Turbo Pascal reserved words }

 KeyWordCount = 52;
 KeyWord: array[1..KeyWordCount] of string[15] = (
 'ABSOLUTE', 'AND', 'ARRAY', 'BEGIN', 'CASE', 'CONST',
 'CONSTRUCTOR', 'DESTRUCTOR', 'DIV', 'DO', 'DOWNTO', 'ELSE',
 'END', 'EXTERNAL', 'FILE', 'FOR', 'FORWARD', 'FUNCTION',
 'GOTO', 'IF', 'IMPLEMENTATION', 'IN', 'INLINE', 'INTERFACE',
 'INTERRUPT', 'LABEL', 'MOD', 'NIL', 'NOT', 'OBJECT', 'OF',
 'OR', 'PACKED', 'PROCEDURE', 'PROGRAM', 'RECORD', 'REPEAT',
 'SET', 'SHL', 'SHR', 'STRING', 'THEN', 'TO', 'TYPE', 'UNIT',
 'UNTIL', 'USES', 'VAR', 'VIRTUAL', 'WHILE', 'WITH', 'XOR');


var

 Idents: IdentTree; { Tree of IdentRef objects }
 LineCount: Integer; { Current line number }
 RefCount: Integer; { Counter used by PrintLine }
 InputBuffer: IOBuffer; { Standard input buffer }
 OutputBuffer: IOBuffer; { Standard output buffer }

{ LineRef constructor }

constructor LineRef.Init(Line: Integer);
begin
 LineNo := Line;
end;

{ IdentRef constructor }

constructor IdentRef.Init(S: Ident);
begin
 Lines.Init;
 GetMem(Name, Length(S) + 1);

 Name^ := S;
end;

{ IdentRef destructor }

destructor IdentRef.Done;
begin
 FreeMem(Name, Length(Name^) + 1);
 Lines.Done;
end;

{ Compare keys of two IdentRef objects in an IdentTree }

function IdentTree.Compare(Key1, Key2: Pointer): Integer;
begin
 if IdentPtr(Key1)^ < IdentPtr(Key2)^ then Compare := -1 else
 if IdentPtr(Key1)^ > IdentPtr(Key2)^ then Compare := 1 else
 Compare := 0;
end;

{ Return the key of an IdentRef object in an IdentTree }


function IdentTree.GetKey(N: TreeNodePtr): Pointer;
begin
 GetKey := IdentRefPtr(N)^.Name;
end;

{ Insert keywords in identifier tree }

procedure InsertKeyWord(L, R: Integer);
var
 I: Integer;
begin
 I := (L + R) div 2;
 Idents.Insert(New(IdentRefPtr, Init(KeyWord[I])));
 if L < I then InsertKeyWord(L, I - 1);
 if I < R then InsertKeyWord(I + 1, R);
end;

{$F+}

{ Create and return a new IdentRef object }

function NewIdent(Key: Pointer): TreeNodePtr;
var
 P: IdentRefPtr;
begin
 New(P, Init(IdentPtr(Key)^));
 P^.Lines.Append(New(LineRefPtr, Init(LineCount)));
 NewIdent := P;
end;

{$F-}

{ Process input file and print listing }

procedure ProcessFile;
var
 Ch: Char;

{ Get next character from input file }

procedure GetChar;

begin
 if Eof then Ch := EndOfFile else
 begin
 if Ch = EndOfLine then
 begin
 Inc(LineCount);
 Write(LineCount: LineNoWidth, ': ');
 end;
 if not Eoln then
 begin
 Read(Ch);
 Write(Ch);
 if (Ch >= 'a') and (Ch <= 'z') then Dec(Ch, 32);
 end else
 begin
 ReadLn;
 WriteLn;

 Ch := EndOfLine;
 end;
 end;
end;

{ Get next token from input file }

procedure GetToken;

{ Get identifier from input file and enter into tree }

procedure GetIdent;
var
 Name: Ident;
 P: LineRefPtr;
begin
 Name := '';
 repeat
 if Length(Name) < MaxIdentLen then
 begin
 Inc(Name[0]);
 Name[Length(Name)] := Ch;
 end;
 GetChar;
 until ((Ch < '0') or (Ch > '9')) and

 ((Ch < 'A') or (Ch > 'Z')) and (Ch <> '_');
 with IdentRefPtr(Idents.Search( Name, NewIdent))^ do
 if not Lines.Empty then
 if LineRefPtr(Lines.Last)^.LineNo <> LineCount then
 Lines.Append(New(LineRefPtr, Init(LineCount)));
end;

begin { GetToken }
 case Ch of
 'A'..'Z', '_':
 GetIdent;
 '''':
 repeat
 repeat
 GetChar;
 until (Ch = '''') or (Ch = EndOfFile);
 GetChar;
 until (Ch <> '''');
 '$':
 repeat
 GetChar;

 until ((Ch < '0') or (Ch > '9')) and
 ((Ch < 'A') or (Ch > 'F'));
 '{':
 begin
 repeat
 GetChar;
 until (Ch = '}') or (Ch = EndOfFile);
 GetChar;
 end;
 '(':
 begin

 GetChar;
 if Ch = '*' then
 begin
 GetChar;
 repeat
 while (Ch <> '*') and (Ch <> EndOfFile) do GetChar;
 GetChar;
 until (Ch = ')') or (Ch = EndOfFile);
 GetChar;
 end;

 end;
 else
 GetChar;
 end;
end;

begin { ProcessFile }
 Ch := EndOfLine;
 GetChar;
 while (Ch <> EndOfFile) do GetToken;
 Write(FormFeed, EndOfLine);
end;

{$F+}

{ Print a LineRef object }

procedure PrintLine(N: ListNodePtr);
begin
 if RefCount = RefPerLine then
 begin

 WriteLn;
 Write(' ': MaxIdentLen + 1);
 RefCount := 0;
 end;
 Inc(RefCount);
 Write(LineRefPtr(N)^.LineNo: LineNoWidth);
end;

{ Print an IdentRef object }

procedure PrintRef(N: TreeNodePtr);
begin
 with IdentRefPtr(N)^ do if not Lines.Empty then
 begin
 Write(Name^, ' ': MaxIdentLen + 1 - Length(Name^));
 RefCount := 0;
 Lines.ForEach(PrintLine);
 WriteLn;
 end;
end;


{$F-}

{ Print identifier tree }


procedure PrintIdents;
begin
 Idents.ForEach(PrintRef);
 Write(FormFeed, EndOfLine);
end;

begin { CrossRef }
 Idents.Init;
 LineCount := 0;
 if Pos('.', ParamStr(1)) = 0 then
 Assign(Input, ParamStr(1) + '.PAS')
 else
 Assign(Input, ParamStr(1));
 Reset(Input);
 SetTextBuf(Input, InputBuffer);
 SetTextBuf(Output, OutputBuffer);
 InsertKeyWord(1, KeyWordCount);
 ProcessFile;
 PrintIdents;
 Idents.Done;
end.









































November, 1989
EXTENSIBLE HASHING


A keyed random access method for disk files




Steve Heller


Steve Heller has been programming for more than 18 years and is the president
of Chrysalis Software Corp., P.O. Box 0335, Baldwin, NY 11510. He can be
reached through CompuServe: 71101, 1702.


Next to every DOS programmer's terminal must be a fortune cookie with a
fortune that reads "640K is the mother of invention." And if you've ever had
to manage large files of records, you've probably cracked this cookie more
than once.
The goal of this article is to show how you can retrieve any record in a
multimegabyte file with one disk access, and any record in any size file with
a maximum of two accesses. KRAM.PAS, Listing One, page 116, (written using
Turbo Pascal 5.0) allows any record in a file of up to approximately 1.4
Mbytes of user data to be retrieved with one disk access. But before we look
at the code, let's examine the basic algorithm.


Extensible Hashing


Extensible hashing uses a record's key to compute a hash code. The first n
bits of the hash code (n = 10 in KRAM.PAS) are used as an index into a table
of block numbers. All records whose hash codes are the same in their first n
bits are stored in the same block. For faster access, a possible optimization
is to use the full hash code again to look up the record within the block.
This is especially valuable when using large data blocks, as a significant
amount of time can be consumed in the search for empty slots when adding
records.
When a block is full, it must be split into two new blocks. The distribution
of records between the two new blocks is based on one more bit of the full
hash code than was used before the split. For example, if the hash code used
to look up the record in the old block was 4 bits long, with the value 1011,
then the first new block would contain records with hash codes of 10110, and
the second new block would contain records with hash codes of 10111.


Index-in-Memory


There are two ways of extending the capacity of a KRAM file. For maximum speed
and simplicity of programming, you can keep the entire index table in memory,
writing it out to the KRAM file only when it is modified as blocks are split,
as is done in KRAM.PAS. This "index-in-memory" approach, however, limits the
maximum size of the files that you can use: For example, if you are willing to
set aside about 140K bytes for the index and the data buffers, you could
retrieve any record from a file of about 128 Mbytes in one access.
The other possibility is to keep the index table mostly on disk in the KRAM
file, and read portions of it into memory as needed. This requires two disk
accesses, one for the index and one to retrieve the data, but only 16K bytes
of memory would suffice for any size file. Of course, as a compromise, you
could keep a few of the most recently used index records in memory. This would
reduce the number of accesses to the index if your references tend to be
repetitious.
While discussing the code, I will mention other possible extensions and
optimizations that were not included in the code, primarily because the
listings would have become too large.


Data Structures


First, let's examine the data structures used to keep track of the current
status of a KRAM file. The sizes of these structures are parameterized so that
you can adapt them to your particular application. This is important primarily
when you use the "index-in-memory" approach, as the maximum file size is then
limited by the number of index entries and the size of each data block. The
maximum data storage available in this case can be approximated by the
formula:
 DATASIZE*INDEXCOUNT*.67;
DATASIZE refers to the number of bytes in a data block, and INDEXCOUNT refers
to the number of entries in the index. (Note that INDEXCOUNT must be a power
of two because it must have one entry for each possible hash code of the
maximum length allowed.)
The constant .67 is an approximation of the packing factor or the proportion
of the file that is occupied by data records assuming random keys. The exact
value of the packing factor depends on the distribution of keys, but shouldn't
vary much from this value.
For example, if you needed to store 100,000 100-byte records for a total data
storage requirement of 10 Mbytes, you could set INDEXCOUNT to 8192 and
DATASIZE to 2048, giving a maximum accessible file size of 16 Mbytes. This
provides an approximate storage capacity of 11.2 Mbytes.
The memory requirement for the data and index buffers in the "index-in-memory"
approach is:
 (2*INDEXCOUNT) + (3*DATASIZE);
as three data buffers are required when splitting a block. The example of
100,000 100-byte records would require 2*8192 + 3*2048, or 22K bytes of memory
for the index and data buffers.


Initialization


KramInit is used to create a new KRAM file. Once created, KramInit initializes
the first data block to all zeros, and sets all pointers in the index block to
point to that first data block. KramInit also initializes the parameter block,
which contains the data length, the key length, and the current high block
number.
Once the file has been initialized, call KramOpen to open the file, read the
parameter block and the index block into memory, and allocate space for the
temporary data block. The declaration for the FileRec record type shows how
the file information is associated with the file pointer.
Turbo Pascal 5.0 represents an untyped file as a record type, called
"FileRec." The UserData field, however, is never accessed by Turbo Pascal, and
is free for user-written routines to store data in. Therefore, I have
redefined that field as:
 UserData: array [1 . . 4] OF pointer;
Currently, only the first element is used to store the address of the
KramParam block for the file. Therefore, any number of KRAM files may be open
at once, subject to system limitations on file handles, with only the file
handle itself being passed to the KRAM routines.


Adding Records to the File



KramAdd is used to write records to the KRAM file. KramAdd returns TRUE if the
record was added successfully (the key was not a duplicate of one in the
file), and FALSE otherwise.
KramAdd first calls HashCode with the key value to calculate which index entry
corresponds to the record to be stored. It then retrieves the block number
pointed to by that index entry. If that block is not the one currently in
memory, it must be retrieved from disk. A possible optimization here is to
keep more than one data block in memory, discarding the least recently used
block when its buffer is needed for another block.
The next task is to discover whether the key is a duplicate of one already in
the block, and if not, whether there is an empty slot in the block to store
the new record. If an empty slot is found (and there is no duplication), then
the data is moved from the function arguments KeyValue and DataValue to that
slot.
If no empty slot is available, and there is no duplication of keys, we must
split this block to make room for the new record. The first subtask here is to
scan the index table to determine which index entries are affected by the
split (that is, which index entries point to this block). Note that if only
one index entry points to this block, the block cannot be split because one
entry cannot point to more than one block. Splitting a block under these
conditions would cause one of the new blocks to be inaccessible.
To prevent this from occurring, page the index from disk, allowing it to be
increased in size as needed. When using a paged index, you should also keep in
each data block a record of the number of bits of the key that are used to
access that block. This way, you won't have to page through a large part of
the index table to find out which entries are affected by a split. For
example, if a data block contains all records with a hash code starting with
the 5 bits 11101, splitting it would affect all index entries that start with
11101. After determining that at least two entries point to this block,
allocate another block to accommodate the overflow, and update the higher
numbered half of the affected entries so they point to the new block. Next,
allocate two temporary block buffers, LowDataPtr^ and HighDataPtr^ (so called
because they will receive the records with lower- and higher-hash codes,
respectively). Next, initialize them to zeros so they are ready to receive the
records from the full block.
The distribution of records from the full block to the two temporary buffers
depends on the updated entries in the index table. The key of each record is
used to calculate the hash code for that record and the block number is looked
up in the index table. If the block number in the table is the same as the
block number of the full block, the record will end up in the old block when
we are through because the old block number was kept for the first (lower)
half of the updated index table entries.
For example, suppose that before block 12 became full, the relevant section of
the index table looked like that shown in Figure 1.
Figure 1: The index table before a split

 Element Value
 Number
----------------
 . . .
 100 12
 101 12
 102 12
 103 12
 104 12
 105 12
 106 12
 107 12
 . .

If, after the split, the highest block number in use was 22, the table would
look like that shown in Figure 2.
Figure 2: The index table after a split

 Element Value
 Number
----------------
 . . .
 100 12
 101 12
 102 12
 103 12
 104 23
 105 23
 106 23
 107 23
 . .

All records with hash codes from 100 to 103 would remain in block 12, and
records with hash codes from 104 to 107 would be moved to block 23.
After the records have been moved to their new places in LowDataPtr^ and
HighDataPtr^, the file is updated. First, the old block (now approximately
half-empty) is written back to its old position. Next, the new block is added
to the end of the file. The parameter and index buffers are then written back
to the file to keep the high block number and index tables up-to-date.
Finally, we return to the top of the WHILE loop to handle the insertion of the
record, now that there is room in the block it belongs in.


The Rest of the Code


KramRead is basically the same as KramAdd (except that it doesn't have to deal
with the splitting of blocks) and requires little explanation. KramClose does
what its name implies; it closes the file, after making sure that anything
that has been changed is written out to the file. Be sure to call this routine
if you have added any records to the file. KramInfo is a procedure that you
can call to find out the key and data lengths of a KRAM file.
HashCode takes the key of a record and returns a 32-bit hash code, from which
the calling program extracts the number of bits it needs to access the index
table. SeekBlock takes a data block number and positions the file to enable
that block to be read or written.
The main program is a driver that illustrates how to call these routines. It
will read an ASCII file of lines, each consisting of a string key and numeric
data, optionally initializing and loading the KRAM file. Then it will retrieve
all the records, checking that they all exist and have the same data in them
as when they were created. The data I used for testing is the same as the
record numbers of the keys; the test records were created by a small test data
generating program, KRAMDATA.PAS (see Listing Two, page 121).
In many applications, it is necessary to be able to delete records from a KRAM
file. The simplest way to do this is to zero out the key and data of the
record you want to delete, so that KramAdd can reuse its slot. This would not
work properly, however, if you decided to use hashing to speed up the lookup
within a block. In that case, you could change the key to something that would
not occur in your application, such as FFh. You would then change KramAdd to
reuse a slot that had that value, and change the block-splitting function to
ignore records that had that key. Thus, whenever a block was split, any
deleted records that had not already been reused would disappear.


Conclusion


Lest you get the impression that KRAM files are the best possible organization
for keyed access, I should mention two limitations of KRAM files, which make
them unsuitable for certain applications. First, this is not an indexed
sequential access method, but truly a keyed random access method. There is no
convenient way of retrieving the records other than by supplying the exact key
for the record you need.

The second limitation is that KRAM files are not very efficient in their use
of disk storage. You can expect that a file containing 1 Mbyte of your data
will occupy approximately 1.5 Mbytes on the disk. This is an unavoidable side
effect of the dynamic storage allocation method that gives KRAM files their
tremendous speed. If these limitations do not adversely affect your
application, KRAM files are probably the best way of organizing data that
requires random access by key.
By the way, a shareware version of KRAM is also available. For more
information, contact Chrysalis Software Corporation, at the address at the
beginning of this article. Registered users will receive an updated version of
the program, incorporating all of the optimizations and extensions mentioned
in this article.

_EXTENSIBLE HASHING_
by Steve Heller


[LISTING ONE]

{KRAM.PAS - A Keyed Random Access Method for Turbo Pascal 5.0.}
{Copyright (c) 1989 by Chrysalis Software Corp. All rights reserved.}
{Written by Steve Heller.}

{$V-}

program KramTest;

uses Crt;

const
 PARAMSIZE = 128; {the size of the parameter block, in bytes}

 DATASIZE = 2048; {the size of a data block, in bytes}

 INDEXCOUNT = 1024; {the number of index entries. Must be a power of 2.}
 INDEXSIZE = INDEXCOUNT * Sizeof(integer); {the size of the index block}

type
 KramDataType = array[1..DATASIZE] of byte;
 KramDataPtr = ^KramDataType;
 KramIndexType = array[1..INDEXCOUNT] of integer;
 KramIndexPtr = ^KramIndexType;

{The variant record type below is used so that we can add new}
{parameters if needed, without changing the size of the parameter}
{block in the data file.}
 KramParamType = record
 case integer of
 0: (

{the items below are saved from one run to another}
 KeyLength:integer;
 DataLength:integer;
 HighBlock:integer;
{the items above are saved from one run to another}

{the ones below are valid only during the current run}
 CurrentBlock:integer;
 BlockModified:boolean;
 DataPtr:KramDataPtr;
 IndexPtr:KramIndexPtr;
 );
 1: (Dummy:array[1..PARAMSIZE] of byte);
 end;
 KramParamPtr = ^KramParamType;

 FileRec = RECORD
 Handle : word;
 Mode : word;

 RecSize: word;
 Private: array [1..26] OF byte;

{note: this is a nonstandard declaration}
 UserData: array [1..4] OF pointer;

 Name : array [0..79] OF char;
 END;




FUNCTION HashCode(Key : string):longint;

{Use this function to calculate a pseudo-random number based on the}
{value of its argument. The result is used to determine which index}
{entry will be used to access the record with the given key. The}
{algorithm used here is appropriate for ASCII key values, as it uses}
{the low five bits of each byte of the key.}

VAR
 i : integer;
 result : longint;
 temp1 : longint;
 temp2 : longint;
 bytetemp : byte;

BEGIN

 result := 0;

 FOR i := 1 TO length(Key) DO
 BEGIN
 temp1 := result shl 5;
 temp2 := result shr 27;
 result := temp1 or temp2;
 bytetemp := ord(Key[i]);
 result := result xor bytetemp;
 END;

 HashCode := result;

END;




PROCEDURE SeekBlock(VAR KramFile : file; BlockNum : integer);

{Use this procedure to position the file pointer to a particular data}
{block. In order to do this, we must skip the parameter block and }
{the index block. Also note that the first data block is #1.}

VAR
 BlockPosition : longint;

BEGIN

 BlockPosition := PARAMSIZE + INDEXSIZE + (DATASIZE * (BlockNum-1));

 Seek(KramFile,BlockPosition);

END;




PROCEDURE KramInit(FileName : string; KeyLength, DataLength: integer);

{Use this procedure to initialize a new KRAM file. It sets up the}
{key length and the data length according to the input arguments.}
{The high data block number is set to 1 and data block #1 is}
{initialized to zeroes (an empty block). The index block is set to}
{all 1's, so that all accesses will go to the empty data block.}

VAR
 KramFile:file;
 Index : KramIndexType;
 Data : KramDataType;
 Params : KramParamType;
 i : integer;


BEGIN

 Assign(KramFile,FileName);
 Rewrite(KramFile,1);

 FillChar(Params.Dummy,SizeOf(Params.Dummy),0);

 Params.KeyLength := KeyLength;
 Params.DataLength := DataLength;

{the highest data block number in use is #1}
 Params.HighBlock := 1;

 BlockWrite(KramFile,Params,SizeOf(Params));

{Initialize the index block to all 1's, as only data block #1 exists}

 FOR i := 1 TO INDEXCOUNT DO
 Index[i] := 1;

 BlockWrite(KramFile,Index,SizeOf(Index));

{Initialize the first data block to all zeroes}

 FOR i := 1 TO DATASIZE DO
 Data[i] := 0;

 BlockWrite(KramFile,Data,SizeOf(Data));

 Close(KramFile);

END;





PROCEDURE KramOpen(VAR KramFile:file;KramFileName:string);

{Use this procedure to open the file. It reads the parameter block}
{and the index block from the beginning of the file and allocates}
{space for the data block.}

VAR
 RecsRead : integer;
 ParamPtr : KramParamPtr;

BEGIN

 Assign(KramFile,KramFileName);
 Reset(KramFile,1);

 New(ParamPtr);

 KramParamPtr(FileRec(KramFile).UserData[1]) := ParamPtr;

 BlockRead(KramFile,ParamPtr^,SizeOf(KramParamType),RecsRead);
 IF RecsRead <> SizeOf(KramParamType) THEN
 BEGIN
 WriteLn('Invalid KRAM file: ',KramFileName);
 Halt(1);
 END;

 New(ParamPtr^.IndexPtr);
 New(ParamPtr^.DataPtr);
 ParamPtr^.CurrentBlock := 0;

 BlockRead(KramFile,ParamPtr^.IndexPtr^,
 SizeOf(KramIndexType),RecsRead);

 IF RecsRead <> SizeOf(KramIndexType) THEN
 BEGIN
 WriteLn('Invalid KRAM file: ',KramFileName);
 Halt(1);
 END;

END;




PROCEDURE KramClose(var KramFile:file);

{Use this procedure to close the file after use. It also deallocates}
{the dynamic storage used for the parameter, index, and data blocks.}

{IMPORTANT NOTE: If you have modified the file, by adding records,}
{for example, you MUST close the file to ensure that all the records}
{have been written to the file.}

VAR
 ParamPtr : KramParamPtr;

BEGIN

 ParamPtr := KramParamPtr(FileRec(KramFile).UserData[1]);

 SeekBlock(KramFile,ParamPtr^.CurrentBlock);
 BlockWrite(KramFile,ParamPtr^.DataPtr^,DATASIZE);

 Dispose(ParamPtr^.DataPtr);
 Dispose(ParamPtr^.IndexPtr);
 Dispose(ParamPtr);

 Close(KramFile);

END;



FUNCTION KramAdd(VAR KramFile : file; KeyValue : string;
 DataValue : string):boolean;

{Use this procedure to add records to a KRAM file that has been opened}
{by KramOpen. You must supply the file pointer that was returned by}
{KramOpen in "KramFile",the key in "KeyValue", and the data in}
{"DataValue". The algorithm is known as "extensible hashing".}

VAR
 ParamPtr : KramParamPtr;
 LowDataPtr : KramDataPtr;
 HighDataPtr : KramDataPtr;
 HashVal : longint;
 BlockNumber : integer;
 TempBlockNumber : integer;
 RecordLength : integer;
 i,j,k : integer;
 SlotFound: boolean;
 FoundFirst : boolean;
 FoundLast : boolean;
 FirstBlock : integer;
 LastBlock : integer;
 MidBlock : integer;
 KramFileName : string;
 TempKeyValue : string;
 OldOffset : integer;
 LowOffset : integer;
 HighOffset : integer;
 Duplicate : boolean;
 KeepLooking : boolean;

BEGIN

 ParamPtr := KramParamPtr(FileRec(KramFile).UserData[1]);

 KramFileName := FileRec(KramFile).Name;

 RecordLength := ParamPtr^.KeyLength + ParamPtr^.DataLength;

 REPEAT

 HashVal := HashCode(KeyValue) and (INDEXCOUNT - 1);

 BlockNumber := ParamPtr^.IndexPtr^[HashVal+1];

 IF ParamPtr^.CurrentBlock <> BlockNumber THEN

 BEGIN
 IF (ParamPtr^.BlockModified)
 and (ParamPtr^.CurrentBlock <> 0) THEN
 BEGIN
 ParamPtr^.BlockModified := FALSE;
 SeekBlock(KramFile,ParamPtr^.CurrentBlock);
 BlockWrite(KramFile,ParamPtr^.DataPtr^,DATASIZE);
 END;
 SeekBlock(KramFile,BlockNumber);
 BlockRead(KramFile,ParamPtr^.DataPtr^,DATASIZE);
 ParamPtr^.CurrentBlock := BlockNumber;
 END;


 Duplicate := FALSE;
 KeepLooking := TRUE;
 SlotFound := FALSE;
 i := 1;
 OldOffset := 1;

 {initialize length of temporary string used for comparison}
 TempKeyValue[0] := char(ParamPtr^.KeyLength);

 WHILE KeepLooking AND (i <= DATASIZE DIV RecordLength) DO
 BEGIN
 IF ParamPtr^.DataPtr^[OldOffset] = 0 THEN
 BEGIN
 KeepLooking := FALSE;
 SlotFound := TRUE;
 END
 ELSE
 BEGIN
 Move(ParamPtr^.DataPtr^[OldOffset],TempKeyValue[1],ParamPtr^.KeyLength);
 IF TempKeyValue = KeyValue THEN
 BEGIN
 Duplicate := TRUE;
 KeepLooking := FALSE;
 END
 ELSE
 BEGIN
 OldOffset := OldOffset + RecordLength;
 i := i + 1;
 END;
 END;
 END;

 IF SlotFound THEN
 BEGIN
 Move(KeyValue[1],ParamPtr^.DataPtr^[OldOffset],
 ParamPtr^.KeyLength);

 Move(DataValue[1],
 ParamPtr^.DataPtr^[OldOffset+ParamPtr^.KeyLength],
 ParamPtr^.DataLength);

 ParamPtr^.BlockModified := TRUE;
 END
 ELSE IF Duplicate = FALSE THEN
 BEGIN

{First we must determine how many index entries are affected}
{by the split}
 FoundFirst := FALSE;
 FoundLast := FALSE;
 LastBlock := INDEXCOUNT;
 FOR i := 1 TO INDEXCOUNT DO
 BEGIN
 IF (ParamPtr^.IndexPtr^[i] = BlockNumber)
 and (not FoundFirst) THEN
 BEGIN
 FirstBlock := i;
 FoundFirst := TRUE;
 END;
 IF (ParamPtr^.IndexPtr^[i] <> BlockNumber)
 and FoundFirst and (not FoundLast) THEN
 BEGIN
 LastBlock := i - 1;
 FoundLast := TRUE;
 END
 END;
 IF FirstBlock >= LastBlock THEN
{we are out of room}
 BEGIN
 WriteLn('KRAM file: ',KramFileName,' is full.');
 Halt(1);
 END;

{Now we have to allocate another block for the split.}
 ParamPtr^.HighBlock := ParamPtr^.HighBlock + 1;
 MidBlock := (FirstBlock + LastBlock - 1) DIV 2;

{We will assign the items that have the higher hash code}
{to the new block}
 FOR i := MidBlock+1 TO LastBlock DO
 ParamPtr^.IndexPtr^[i] := ParamPtr^.HighBlock;

{Now we have to go through all the items in the block to be split}
{and assign them to the old block or the new one according to their}
{new hash codes, 1 bit longer than the previous ones. An extra}
{temporary block makes this easier.}
 New(LowDataPtr);
 New(HighDataPtr);

 FillChar(LowDataPtr^,SizeOf(LowDataPtr^),0);
 FillChar(HighDataPtr^,SizeOf(HighDataPtr^),0);

 OldOffset := 1;
 LowOffset := 1;
 HighOffset := 1;
 FOR i := 1 TO DATASIZE DIV RecordLength DO
 BEGIN
 Move(ParamPtr^.DataPtr^[OldOffset],TempKeyValue[1],
 ParamPtr^.KeyLength);

 TempKeyValue[0] := char(ParamPtr^.KeyLength);
 HashVal := HashCode(TempKeyValue) and (INDEXCOUNT - 1);
 TempBlockNumber := ParamPtr^.IndexPtr^[HashVal+1];
 IF TempBlockNumber = BlockNumber THEN
{send to the lower one}

 BEGIN
 Move(ParamPtr^.DataPtr^[OldOffset],
 LowDataPtr^[LowOffset],RecordLength);

 LowOffset := LowOffset + RecordLength;
 END
 ELSE
 BEGIN
 Move(ParamPtr^.DataPtr^[OldOffset],
 HighDataPtr^[HighOffset],RecordLength);

 HighOffset := HighOffset + RecordLength;
 END;
 OldOffset := OldOffset + RecordLength;
 END;

 SeekBlock(KramFile,BlockNumber);
 BlockWrite(KramFile,LowDataPtr^,DATASIZE);

 SeekBlock(KramFile,ParamPtr^.HighBlock);
 BlockWrite(KramFile,HighDataPtr^,DATASIZE);

 Dispose(LowDataPtr);
 Dispose(HighDataPtr);

{Make sure the same block isn't used the the next time we have to}
{expand the file. Also note that the data block is out of date.}
 ParamPtr^.CurrentBlock := 0;
 Seek(KramFile,0);
 BlockWrite(KramFile,ParamPtr^,SizeOf(KramParamType));
 BlockWrite(KramFile,ParamPtr^.IndexPtr^,
 SizeOf(KramIndexType));

 END;

 UNTIL SlotFound or Duplicate;

 IF Duplicate THEN
 KramAdd := FALSE
 ELSE
 KramAdd := TRUE;

END;




FUNCTION KramRead(var KramFile:file; KeyValue: string;
 var DataValue: string):boolean;

{Use this function to read records from a KRAM file that has been}
{opened by KramOpen. You must supply the key in "KeyValue" and the}
{file pointer that was returned by KramOpen in "KramFile". The data}
{stored under that key will be returned in "DataValue". The return}
{value is TRUE if the record was found, and FALSE if it wasn't.}

VAR
 ParamPtr : KramParamPtr;
 HashVal : longint;

 BlockNumber : integer;
 RecordLength : integer;
 i,j,k : integer;
 SlotFound: boolean;
 KeepLooking : boolean;
 KramFileName : string;
 TempKeyValue : string;
 DataOffset : integer;

BEGIN

 ParamPtr := KramParamPtr(FileRec(KramFile).UserData[1]);

 KramFileName := FileRec(KramFile).Name;

 RecordLength := ParamPtr^.KeyLength + ParamPtr^.DataLength;

 HashVal := HashCode(KeyValue) and (INDEXCOUNT - 1);

 BlockNumber := ParamPtr^.IndexPtr^[HashVal+1];

 IF ParamPtr^.CurrentBlock <> BlockNumber THEN
 BEGIN
 IF (ParamPtr^.BlockModified)
 and (ParamPtr^.CurrentBlock <> 0) THEN
 BEGIN
 ParamPtr^.BlockModified := FALSE;
 SeekBlock(KramFile,ParamPtr^.CurrentBlock);
 BlockWrite(KramFile,ParamPtr^.DataPtr^,DATASIZE);
 END;
 SeekBlock(KramFile,BlockNumber);
 BlockRead(KramFile,ParamPtr^.DataPtr^,DATASIZE);
 ParamPtr^.CurrentBlock := BlockNumber;
 END;


 KeepLooking := TRUE;
 SlotFound := FALSE;
 i := 1;
 DataOffset := 1;

 {initialize length of temporary string used for comparison}
 TempKeyValue[0] := char(ParamPtr^.KeyLength);

 WHILE KeepLooking AND (i <= DATASIZE DIV RecordLength) DO
 BEGIN
 IF ParamPtr^.DataPtr^[DataOffset] = 0 THEN
 KeepLooking := FALSE
 ELSE
 BEGIN
 Move(ParamPtr^.DataPtr^[DataOffset],TempKeyValue[1],ParamPtr^.KeyLength);
 IF TempKeyValue = KeyValue THEN
 BEGIN
 SlotFound := TRUE;
 KeepLooking := FALSE;
 END
 ELSE
 BEGIN
 DataOffset := DataOffset + RecordLength;

 i := i + 1;
 END;
 END;
 END;


 IF SlotFound THEN
 BEGIN
 Move(ParamPtr^.DataPtr^[DataOffset+ParamPtr^.KeyLength],
 DataValue[1],ParamPtr^.DataLength);

 DataValue[0] := char(ParamPtr^.DataLength);
 KramRead := TRUE;
 END
 ELSE
 KramRead:= FALSE;

END;




PROCEDURE KramInfo(var KramFile:file; var KeyLength: integer;
 var DataLength: integer);

{Use this procedure to get the key and data lengths from a KRAM file that has}
{been opened by KramOpen. You must supply the file pointer that was returned}
{by KramOpen in "KramFile".}

VAR
 ParamPtr : KramParamPtr;

BEGIN

 ParamPtr := KramParamPtr(FileRec(KramFile).UserData[1]);

 KeyLength := ParamPtr^.KeyLength;

 DataLength := ParamPtr^.DataLength;

END;




VAR
 KramTestFile : file;
 FileName : string;
 DataFileName : string;
 KeyLength : integer;
 DataLength : integer;
 KramFile : file;
 Ok : boolean;
 InputFile : text;
 Key : string;
 RecNum : integer;
 TestRecNum : integer;
 RecNumStr : string;
 InputFileBuf : array [1..10240] of char;

 Create : string;
 Status : integer;
 ReadStatus : boolean;
 AddStatus : boolean;
 Temp : string;
 Data : string;
 TempKey : string;
 TempData : string;

BEGIN

 ClrScr;
 WriteLn('KRAM.PAS - A Keyed Random Access Method for Turbo Pascal 5.0.');
 WriteLn('Copyright (c) 1989 by Chrysalis Software Corp. All rights
reserved.');
 WriteLn('Written by Steve Heller.');
 WriteLn;
 WriteLn('This program is shareware: if you find it useful, you should');
 WriteLn('register it by sending a check in the amount of $39.95, ');
 WriteLn('payable to Chrysalis Software Corporation, to the following
address');
 WriteLn;
 WriteLn('Chrysalis Software Corporation');
 WriteLn('P. O. Box 0335');
 WriteLn('Baldwin, NY 11510');
 WriteLn;
 WriteLn('Registered users will receive an updated version of the program,');
 WriteLn('incorporating all of the optimizations and extensions mentioned');
 WriteLn('in this article.');
 WriteLn;
 Write('Please hit ENTER to continue.');
 ReadLn(FileName);
 ClrScr;

 Write('Please enter the name of the data file to be used for input: ');
 ReadLn(DataFileName);

 Write('Please enter the name of the KRAM file: ');
 ReadLn(FileName);

 Write('Create the KRAM file (Y/N): ');
 ReadLn(Create);
 IF (Create[1] = 'Y') or (Create[1] = 'y') THEN
 BEGIN

 Write('Please enter the key length: ');
 ReadLn(KeyLength);
 Write('Please enter the data length: ');
 ReadLn(DataLength);

 Assign(InputFile,DataFileName);
 Reset(InputFile);
 SetTextBuf(InputFile,InputFileBuf);

 KramInit(FileName,KeyLength,DataLength);
 KramOpen(KramTestFile,FileName);

 RecNum := 0;
 AddStatus := TRUE;
 WHILE NOT EOF(InputFile) DO
 BEGIN

 IF RecNum Mod 10 = 0 THEN
 Write(RecNum:5);
 RecNum := RecNum + 1;
 ReadLn(InputFile,Temp);
 Key := Copy(Temp,1,KeyLength);
 Data := Copy(Temp,KeyLength+1,length(Temp));
 AddStatus := KramAdd(KramTestFile,Key,Data);
 IF AddStatus = FALSE THEN
 BEGIN
 WriteLn;
 WriteLn('Unable to add key: ',Key);
 END;
 END;

 WriteLn;
 Close(InputFile);
 KramClose(KramTestFile);
 END;

 Assign(InputFile,DataFileName);
 Reset(InputFile);
 SetTextBuf(InputFile,InputFileBuf);

 KramOpen(KramTestFile,FileName);

 KramInfo(KramTestFile,KeyLength,DataLength);

 RecNum := 0;
 WHILE NOT EOF(InputFile) DO
 BEGIN
 IF RecNum Mod 10 = 0 THEN
 Write(RecNum:5);
 RecNum := RecNum + 1;
 ReadLn(InputFile,Temp);

 Key := copy(Temp,1,KeyLength);
 TempData := copy(Temp,KeyLength+1,DataLength);

 ReadStatus := KramRead(KramTestFile,Key,Data);
 IF Not ReadStatus THEN
 BEGIN
 WriteLn;
 WriteLn('Key ',TempKey,' not found.');
 END;
 IF TempData <> Data THEN
 BEGIN
 WriteLn;
 WriteLn('Record number ',RecNum,' does not match');
 END;
 END;

 WriteLn;

 Close(InputFile);
 KramClose(KramTestFile);

END.






[LISTING TWO]

{KRAMDATA.PAS - generates data for KRAM testing}
{890226:1245}

VAR
 s : String[255];
 t : Text;
 n : LongInt;
 i : LongInt;
 j : Integer;
 KeyLength : Integer;
 DataLength : Integer;
 FName : String[80];

BEGIN

 Randomize;

 WriteLn;

 Write('Name of data file to be generated: ');
 ReadLn(FName);

 Write('Number of items to generate: ');
 ReadLn(n);

 Write('Length of Keys: ');
 ReadLn(KeyLength);

 Write('Length of Data: ');
 ReadLn(DataLength);

 Assign(t,Fname);
 Rewrite(t);

 FOR i := 1 TO N DO
 BEGIN
 IF i = 1000*int(i/1000) THEN Write(i:10);
 s := '';
 FOR j := 1 TO KeyLength DO
 s := s + chr(random(26)+65);
 Write(t,s);
 WriteLn(t,i:DataLength);
 END;

 Close(t);

END.










November, 1989
OPTIMIZING IN A PARALLEL ENVIRONMENT


Data independence is the key to performance




Barr E. Bauer


Barr works for Schering-Plough Research, a pharmaceutical company. He can be
reached at 60 Orange Street, Bloomfield, NJ 07003.


The emerging class of small, relatively inexpensive multiple-processor
workstation computers provides a degree of raw computing power that seriously
challenges the computing power offered by minisupercomputers. The availability
of the appropriate software tools -- language extensions, parallelizing
compilers, and parallel environment debuggers and profilers -- makes it
possible to exploit this multiprocessor architecture in a straightforward
manner.
This article explores the SGI parallelization scheme that Silicon Graphics has
implemented in its Power Series two-processor and four-processor work-stations
and their accompanying software development tools. I use a test program to
take you through this exploration process. First, I'll introduce the syntax,
highlighting coding problems and solutions. Next, I'll describe the actual
implementation of the parallelization scheme. Finally, I'll show the bottom
line for high-performance computing, and confirm that performance gains are
possible with the SGI scheme if one is mindful of its limitations.


SGI Parallelization Scheme


My exploration strategy centers around the DO loop. If the iterations of the
loop are independent from each other, then the loop can be divided among n
available processors, and can run roughly n times faster. These separate
executing processes are known as "threads." This parallelization scheme is not
intended for use with complex, independently executing routines (although a
separate set of Unix tools exists to accomplish that).
The goal here is to accelerate the speed of more common forms of code. For
instance, consider the two DO loops in Figure 1. The scalar DO loop is
transformed into a parallel loop by the inclusion of a parallelization
directive immediately before the DO statement.
Figure 1: A scalar and a parallel DO loop

 Scalar:

 do i = 1, n
 a(i) = b(i) * c(i)
 enddo

 Parallel:

 c$doacross local(i), share(a, b, c, n)
 do i = 1, n
 a(i) = b(i) * c(i)
 enddo

The program is parallelized after compilation with the -mp option. The
parallelization directive affects only the DO loop that immediately follows
that directive. The directive resembles a Fortran comment, and is ignored by
the compiler when the -mp option is not used. More importantly, the directive
is ignored by other platforms that do not have multiple processors.
The syntax allows for LOCAL, SHARE, and LASTLOCAL declarations to restrict the
scope of variables and improve program performance (see sidebar 1, page 74).
LOCAL variables are localized to each thread. SHARE variables are in common
between executing threads. LASTLOCAL variables are treated like LOCAL except
that the value of the variable in the thread, which finishes last, is made
available to code past the parallelized loop.
A variety of Fortran statements, including subroutines, functions, and
intrinsics, are allowed inside of parallelized loops. The only restriction is
that the statements must have no side effects -- the statements must depend
upon input in the passed arguments, they cannot modify common data, and they
cannot use static data. All of the intrinsic functions that are supplied in
the library meet this requirement and can be used safely.
The one construct that is forbidden is branching outside the DO loop.
Typically, this construct is a logical test that causes premature termination
of the loop. It doesn't make sense to use a conditional exit, which may behave
differently for each thread in a parallel environment.
Additionally, a number of functions that are unique to the parallel
environment are available to help the executing program work more efficiently
within the multiprocessor environment. The straightforward syntax of these
functions, (which retains the algorithm in its original form), along with
transparency to other platforms, facilitate implementation. (This is
especially useful for programmers who do not wish to master the details of
systems-level programming.) The syntax is conservative in the sense that the
loop code itself needs no restructuring; the addition of the directive (and
the optional addition of the scope declarations) are the only requirements.
The retention of the basic algorithm maintains clarity and aids in the
transformation of scalar code.
The parallel implementation restructures the code during compilation into
separate slaved processes for the parallelized loop (see sidebar 2, page 76).
The parallelized program is under software control during execution, and can
create as many processes as are appropriate for the machine on which the code
is executing. (The number of processes can range from 1, for a single
processor system, to 4.) The sensing and adjustment is automatic so that
separate versions of a program need not be created for a multiplatform
environment. This feature offers a great deal of flexibility to programmers
who use networked NFS systems, in which a single executable can go to any
target machine. In practice, this works well, although parallelized programs
that run on single processor systems execute more slowly due to the system
overhead.


Data Dependency


The primary limitation of a parallelized program is "data dependency," in
which one of several currently executing threads alters data that can be read
by any other executing thread, producing nonsensical results. When this method
of parallelization is used, many common coding situations become unusable
unless they are recognized, and then solutions are applied.
Consider, for example, the DO loop in Example 1. The process of successful
parallelization requires each iteration of the loop to be data-independent of
any other iteration of the loop. In this way, the loop can be broken into n
separate parallel processes, or can even be left scalar, without affecting the
results. As long as the code in the loop for all values of the index is run,
regardless of the order or grouping of the code, the outcome is the same.
Example 1: A typical DO loop

 do i = 1, n
 a(i) = x * b(i)
 enddo


Data dependency occurs when a variable is modified (or could be modified) by
one executing thread, while at the same time, other executing threads may be
reading or modifying the same variable. The value of the variable becomes
unpredictable, causing the code in the loop to produce results that vary from
the scalar and also vary from run to run.
More Details.
In the DO loop in Example 2, the array variable arr references a value that is
not current with the index. The variable is manipulated and then assigned to
arr(i). Other threads can write to arr(i-1), and arr(i) can be referenced by
other threads. The iterations are not truly independent. When the DO loop is
parallelized, the parallel version cannot guarantee the same result that is
guaranteed by use of the scalar version of the loop. This type of data
dependence is known as "recurrence."
Example 2: A DO loop in which the array variable references a value that is
not current with the index

 do i = 2, n
 arr(i) = b(i) - arr(i-1)
 enddo

Data dependency inhibits parallelization and does not yield the performance
inherent in the machine. The solution to the problem of data dependency is
recognition of its existence, followed by modification of the algorithm. A
simple modification of the algorithm isn't always possible to do, and not all
algorithms can be parallelized. In short, the ability to break data dependency
is algorithm-specific. Most algorithms must be approached as special cases.
Many different types of dependency can prevent parallelization. A number of
common situations have data dependencies that can be generalized. In addition,
other situations have data dependencies that are not obvious. Once again, the
key is recognition of the existence of data dependency, especially when
existing scalar code is adapted to the parallel environment. When
parallelization is not possible, the solution usually involves major
restructuring of the algorithm.


Recurrence


The data dependency case discussed in the previous section is an example of
recurrence. Modified values recur during the execution cycle of the threads
that are associated with a parallelized loop, causing unpredictable results.
There is no general way to parallelize loops in which recurrence occurs. One
possible solution is to rewrite the algorithm. If the recurrence occurs in the
inner loop of nested loops, a solution is to parallelize the outer loop.
To see how this works, consider the parallelization shown in Figure 2. If x is
used only inside of the loop and the value of x is assigned before x is used,
then the loop can be declared a local variable, and can be parallelized. If
the value of x is not assigned first, then x becomes data dependent upon the
value of x from other iterations. The loop cannot be parallelized by declaring
x to be LOCAL.
Figure 2: The effect of data dependency with LOCAL variables

Local Variables:

 do i = 1, n
 x = a(i)**3
 b(i) = x * b(i)
 enddo

The variable x is dependent. Without an explicit declaration as LOCAL, this
loop cannot be parallelized.

c$doaccross local(i, x)
 do i = 1, n
 x = a(i)**3
 b(i) = x*b(i)
 enddo

Figure 3 presents another example of data dependence, where the value of indx
is passed between iterations, causing data dependency to occur. Although indx
is used inside of the loop, the value of indx is read before indx is assigned,
so indx cannot be a local variable. Dependency can also be created by the
manner in which indx is determined, but the determination of indx in this
particular case is independent.
Figure 3: Example of loop-carried values (complicated indexing)

 Scalar:

 index = 0
 do i = 1, n
 indx = indx +i
 a(i) = b(indx)
 enddo

 Parallel:

 c$doacross local(i, indx)
 do i = 1, n
 indx = (i* (i + 1))/2
 a(i) = b(indx)
 enddo

If the variable can be determined uniquely for each iteration, so that a value
for the variable is not passed from iteration to iteration, then the loop can
be parallelized. The value of indx in Figure 3 is determined only from the
loop index i. In order to arrive at this solution, the original algorithm was
rewritten and is no longer straightforward. This solution, then, is highly
algorithm-dependent.
More Details.
The next example of indirect indexing is shown in Figure 4. The problem is
seen in the last line of the loop. The value of iix is dependent upon the
values stored in the ixoffset and indexx arrays, and cannot be guaranteed to
be independent from iteration to iteration. This problem is different from the
loop carried value problem because the value of iix is looked up, rather than
calculated. Unless the contents of indexx and ixoffset are known to not
introduce dependency, iix must be considered dependent.
Figure 4: Example of indirect indexing


 Scalar:
 do i = 1, n
 ix = indexx(i)
 iix = ixoffset(ix)
 total(iix) = total(iix) + delta
 enddo

 Parallel:
 c$doacross local(ix, i)
 do i = 1, n
 ix = indexx(i)
 iix(i) = ixoffset(ix)
 enddo

 do i = 1, n
 total(iix(i)) = total(iix(i)) + delta
 enddo

Indirect indexing is also a great source of run-time bugs that are nearly
impossible to find. The solution is to remove the data-dependent element and
place it into a nonparallelized loop. The array index for the second loop is
now the only task of the first loop, and is stored in array elements indexed
by i, which is common to both loops. Half of the original loop is now
parallelized, but performance may be an issue if n is small. Rewriting the
loop, or even leaving the original loop as a nonparallelized loop, should be
considered if performance suffers in the two-loop case. Note that data
dependency is still present in the second loop.
"Sum reduction" (Figure 5) is a common procedure that is highly
data-dependent. The implementation of sum reduction is similar to the
implementation of the loop-carried variable case. In the case of sum
reduction, however, the value of total is presumably used outside of the loop
(while indx was only used inside of the loop).
Figure 5: Example of sum reduction

 Scalar:

 total = 0.0
 do i = 1, n
 total = total + a(i)
 enddo

 Parallel:

 c$doacross local(i, k)
 do k = 1, 4 ! assume no more than 4
 processors
 sub_total(k) = 0.0
 do i = k, n, 4
 sub_total(k) = sub_total(k) +a (i)
 enddo
 enddo

 total = 0.0
 do i = 1, 4
 total = total + sub_total(i)
 enddo

The solution to the data dependency of sum reduction is similar to the
solution used for the indirect indexing case: Subtotals are determined,
permitting the loop to be parallelized, and the loop is followed by a grand
summation loop. The original loop is converted into the inner loop of a nested
pair of loops. Parallelization then takes place on the outer loop, which then
divides the inner loop into the maximum number of parallel processes.
Unlike the example of indirect indexing, which contains two loops of roughly
equal size, the bulk of the original loop in this case is parallelized and
followed by a small grand summation loop. In order to prevent cache and
performance problems, the threads interleave the array and are not handled as
a set of contiguous blocks. The solution to sum reduction can also be made
more efficient and more general. An example of this approach is shown in
TEST.F (Listing One, page 122).


Performance Issues


A number of issues can affect the performance of parallel schemes. Foremost
among these issues are overhead, caching, and load balancing.
The CPU overhead is approximately 100 cycles for loop parallelization. The
loop must iterate more than about 50 times in order to recover that overhead.
Otherwise, small parallelized loops will run slower than the speed of scalar
loops.
Two solutions can be employed: Enclosing the maximum amount of work in the
parallelized loop, and using multiple versions of a loop. The parallelization
process can accommodate a wide range of normal Fortran functionality,
including IF ... THEN ... ELSE ... ENDIF blocks, other loops, function calls,
subroutine calls, and even G0T0s. If data dependency does not introduce
restrictions, the inclusion of as much functionality in the loop as possible
cuts down on overhead and keeps the highest number of threads in action to
improve performance. This solution also means that outer loops should be
examined preferentially to inner loops so that the greatest amount of code is
parallelized.
The other solution to the problem of CPU overhead is to write both parallel
and scalar versions of a loop, and place them in an IF ... (index gt.
threshold) ... THEN ... (do parallel) ... ELSE ... (do scalar) ... ENDIF
block. This is a good approach for maximizing program performance when the
value of the index varies across the threshold value of roughly 50 iterations.
These multi-processor machines have exceptionally large data caches. If
programs are not written with the cache in mind, performance can suffer
significantly. In practical terms, when a multidimensional array is referenced
inside nested loops, the leftmost index of the array must be varied fastest by
the innermost loop so that memory is addressed as contiguously as possible.
(Fortran stores arrays in column order, so the arrays must be referenced in
what always seems to me to be a backwards order. I will say more about cache
effects later.)
The parallelization process handles load balancing by dividing the target DO
loop into equal pieces, based upon the number of available threads (usually
the same number as the number of available processors). If the loop contains
IF blocks, or variable-sized inner loops (as is the case in the next example),
then the execution time for each thread can be random. As a consequence, if
the work of the parallelized loop is not performed equally, time is wasted as
completed threads wait for the slower threads to finish. The execution of the
code that follows the loop will not occur until all of the threads are
completed. The load is not balanced, and the penalty is increased execution
time due to the idle processors.
The code in Example 3 is a particularly vicious example of load imbalance.
Assume that the outer loop is parallelized. The duration of the execution of
the inner loop varies according to the value of the index of the outer loop.
Although the outer loop's index is divided into ranges of equal size, the
threads associated with the ranges that have larger values of the index will
take longer to complete.

Example 3: An example of load imbalance

 do i = 1, n
 do j = 1, i
 a(j, i) = a(j, i) * xmult
 enddo
 enddo

As Example 4 shows, a solution is to use index i to break the outer loop into
interleaving blocks, such that each thread has a range of values of i. No
thread has only the large values of i and the resulting longer execution time.
The result is that all threads execute in about the same amount of time. The
load is balanced. The only concern is that the process of interleaving can
disrupt contiguous memory blocks, which can potentially degrade cache
performance.
Example 4: Load balancing

 num_threads = mp_numthreads ()
 c$doacross local (i, j, k)
 do k = 1, num_threads
 do i = k, n, num_threads
 do j = 1, i
 a (j, i) = a(j, i) * xmult
 enddo
 enddo
 enddo



The Bottom Line


The real test of the efficiency of the SGI parallelization scheme is with the
clock. The goal of the scheme is faster execution, which allows larger and
more complex problems to be addressed.
A short test program was developed that reflects the types of tasks I would
like to speed up in my own work. It fills and manipulates large,
multidimensional arrays using nested DO loops, and generates a grand total via
sum reduction. The nested loops allow me to examine alternative
parallelization strategies. The grand total number serves as a behavior
reference for different cases.
Three cases were tested: A purely scalar case (SCA), an inner loops
parallelized case (IN), and an outer loop parallelized case (OUT). To test
each of the cases, the appropriate * comments were removed in order to unmask
the desired compiler directives.
Each program was compiled at optimization level 2. This level is primarily
loop optimization, and currently is the maximum level of optimization that is
permitted for the -mp option. I have found that programs compiled at
optimization level 1 typically perform about 40 percent slower than programs
that are compiled at level 2, making it essential to compile at the higher
optimization level.
Both a two-headed 4D/120s system and a four-headed 4D/240s system were
evaluated. The various configurations are listed in Table 1. Instead of
listing aggregate power, I prefer to list the power per processor. This gives
an honest view of these machines and reflects nominal scalar performance. The
processing power is relative to the old standard of scientific computing --
the VAX 11/780 -- as being 1 mips by definition. These 13 and 20 mips numbers
are real!
Table 1: Platforms

 model 4D/120s 4D/240s
 processors 2 4
 processors MIPS R2000 MIPS R3000
 clock speed (MHz) 16 25
 processing power (mips) 2x13 4x20
 FPU 2xR2010 4xR3010
 Mflops (double precision) 2x1.5 4x3.0
 data cache (kbytes/processor) 64 64
 ram (Mbytes, nonstandard) 32 64

I have broken down the timing results into the categories of User, System, and
Total (wall clock). The times were determined by using the Unix time utility,
and were determined on machines free of competing tasks. The speedup is
relative to scalar, and reflects the number of processors that contribute to
the job. The times between the two-processor and four-processor systems are
not comparable because the CPU and clock rates are very different for each
system.
All versions of TEST.F are well-behaved. Each version returns a value of
0.518e+8 for the grand total, which indicates correct operation under both
scalar and parallel modes. The value of ITEST that is typically returned is
random, and ranges between 0 and 5 (except in the IN case, when it returns an
enormous integer value that reflects the increased cycling time being
measured, as it should). The random nature of the value of ITEST, which is
declared as type LASTLOCAL, suggests that caution must be used in the
interpretation and use of LASTLOCAL values.
The speedup, as shown in Tables 2 and 3, is impressive. In both processor
tests, the difference between the IN and OUT tests likely reflects load
balancing, but is small, regardless. This suggests that parallelization of the
most time-dependent code, rather than of the largest quantity of most code, is
the real issue. Outer loops that enclose much-varied code can be far more
difficult to parallelize than inner loops, due to the occurrence of data
dependency. The difference between IN and OUT suggests that no significant
performance penalty may exist if time-dependent inner loops are preferentially
parallelized. The results also suggest that no generalization is safe, and
that a variety of coding possibilities should be examined.
Table 2: Two-processor results (sec.)

 case user sys total speedup
---------------------------------

 SCA 40.4 2.1 42.5 1.00
 IN 21.0 0.8 21.8 1.95
 OUT 21.8 1.0 22.8 1.86

Table 3: Four-processor results (sec.)

 case user sys total speedup
---------------------------------


 SCA 25.6 1.0 26.6 1.00
 IN 7.1 0.2 7.3 3.64
 OUT 6.4 0.5 6.9 3.86



Cache Tests


While experimenting with TEST.F, I uncovered a performance issue that is
apparently uncommon to programming on PC-class machines: Data cache
management. Although the issue of data cache management is common to both
scalar and parallel programming environments, the observed effects of
mismanagement of the cache, and their cure, are important enough to highlight
separately because they affect performance. The cure is the proper use of
array indexing.
Intentional cache mismanagement leads to significant performance degradation.
Management in this context consists of the maximization of the use of existing
data in the cache, and the minimization of data movement into the cache from
regular memory. The amount of the reuse of existing cache data is commonly
referred to as the "hit rate". Better performance is obtained from optimizing
the hit rate.
In practical terms, the hit rate can be increased by addressing memory
contiguously. Memory references of this sort typically consist of array
variables in which contiguous blocks of values can be moved into the cache
with less overhead than is required in order to move single values into the
cache multiple times. Keep in mind that in Fortran, multidimensional arrays
are stored in column order; the leftmost index represents contiguous memory
addresses. Therefore, nested DO loops must vary the leftmost index of a
two-dimensional array fastest, so the references are to contiguous memory
locations.
Cache mismanagement was tested by reversing the array index variables, and by
reversing the array size declarations. The array sizes guarantee that the
arrays cannot be wholly contained in the cache and that the movement of data
into the cache is handled via the movement of individual array elements.
The reason why cache management is a concern is shown in Table 4. Note that
the speedup for parallelization is less, due to increased system overhead,
while the slowdown relative to the speed observed for proper array indexing is
substantial. This translates to an additional day added to the time required
to perform a three-day simulation, which is a significant penalty. The effects
of cache mismanagement may be inconsequential for tasks that have small memory
requirements. In the case of large memory requirements, the penalty for
improper use of the cache can be considerable.
Table 4: Two-processor reversed-array indexing (sec.)

 case user sys total speedup slowdown
-----------------------------------------

 SCA 45.7 6.9 52.6 1.00 1.24
 OUT 22.8 7.0 29.7 1.77 1.31



Conclusion


Will the application of these tools to real programs translate to significant
performance gains, as shown by TEST.F? This will very likely depend upon the
individual program. In the case of one program that is important to my own
work, I have created significant speed enhancements by the parallelization of
key routines found by profiling. An additional increase in speed is also
possible by paying attention to load balancing and cache management.
Number crunching has never been cheap. Not everyone has access to a
supercomputer, and not every problem can justify the cost. Al Cameron, who
writes the "Engineering/Science" column for MIPS magazine, advocates the use
of workstations dedicated to running one problem around the clock for long
periods of time as a low-cost alternative to supercomputing. I share his
opinion, but this solution is clearly acceptable only if you can afford to
wait. Machines such as the Power Series from Silicon Graphics provide the raw
power needed for numerically intensive computing, plus the software tools
needed to extract that power. With properly parallelized programs, the wait is
considerably shorter.


Acknowledgments


I would like to thank Nancy Marx and Josh Blumert, both of Silicon Graphics,
for their helpful discussions. I would especially like to thank the authors of
the "SGI Parallelization Documentation" for an exceptionally clear and useful
manual. All documentation should be written this way. Many of the examples
(somewhat simplified) were taken directly from the documentation, with
permission from Silicon Graphics.


Parellelization Directives


Each directive begins in column 1 and behaves as a comment unless
multiprocessing is enabled during compilation (-mp option). Code containing
directives is generally portable except to platforms, which use a similar
scheme such as the CRAY.
Variables referenced in the scope arguments are by name only; declaration is
done in the usual manner at the top of the program. Default scope declaration
is SHARE for all undeclared variables and for variables appearing in COMMON
blocks, except the DO loop index, which is LASTLOCAL. Variables can only
appear in one scope argument.

C$DOACROSS [LOCAL(list...),
SHARE(list...), LASTLOCAL(list...)]

This is the principal directive used to select a loop for parallelization. It
immediately proceeds the selected loop and any of three optional scope
arguments. C$DOACROSS directives do not nest and subsequent nested directives
are ignored.
LOCAL(list ... ) are all variables local to each separately executing thread.
All LOCAL variables must be explicitly declared, and all data types, including
arrays, can be of type LOCAL.
SHARE(list ... ) are all variables in common to all executing threads and the
remainder of the program. SHARE variables are typically arrays, while nonarray
variables can be used if read only, to avoid data dependency.
LASTLOCAL(list ... ) are local variables in which the last executing thread
returns its value for use past the parallelized block. The value returned is
not always the assumed value and great caution must be used especially if load
balancing between threads is not even.
C$& -- This is treated as a continuation line for lenghty parallelization
directives.
C$ -- This is used to selectively include lines or blocks of code during
parallelization. Typically, this is used to mask lines that are parallel
specific or call routines from the parallel library. -- B.B.



Parallel Implementation



Upon encountering a C$DOACROSS statement, the compiler creates a new
subroutine that represents the loop and its contained functionality, and then
replaces the loop with a call to the library routine MP_MULTI. The new
subroutine is named by prepending an underscore to the name of the routine
that originally called the loop and then appending an underscore and a number
to that name. The number that is appended starts at 1, and is incremented for
each C$DOACROSS contained in the original routine. The new subroutine handles
variables declared as LOCAL as local variables. Variables declared as SHARE
are resolved by reference back to the original routine. Any required
additional variables are local.
The new routine is called with four arguments: The starting value of the
index, the maximum number of executions of the loop, the increment value of
the index, and a unique process ID number. The remainder of the routine
consists of the parallelized DO loop. The DO loop is restructured (see Listing
Two, page 123) in order to permit multiple copies of the new routine running
in parallel to work with independent sections of the DO loop. The code of the
new subroutine is written once and the portion of the DO loop that is
manipulated is controlled by the calling routine, MP_MULTI. This permits a
great deal more run-time flexibility than is available when hard-coded
routines are used.
At run time, the parallelization is effected by a master routine, which
handles all of the scalar code. At the point in the code when the DO loop
would have been executed, the master routine passes the name of the new
process to MP_MULTI, along with the starting value of the DO loop index, the
number of cycles, and the index increment, MP_MULTI divides the loop according
to the number of available processors, and then invokes the new process that
number of times, with each new process as a slave routine. Additional routines
manage interprocess signalling and synchronization. The slave subprocesses do
not have to run in synchronization with each other. When the last slave
process is completed, control returns to the master routine at the point past
the loop.
This parallelization scheme is very flexible. Parallelized code can run on a
single processor system without change, while dynamically adapting to the
number of available heads. At the other extreme, the number of threads can be
limited dynamically by setting the environment variable NUM_THREADS to a
number less than the maximum number of available heads.
-- B.B.



_OPTIMIZING IN A PARALLEL ENVIRONMENT_
by Barr E. Bauer


[LISTING ONE]

 program test 1
* 2
* purpose is to test SGI parallelization scheme for loop selection, 3
* numerically-intensive calculations, and total reduction. See text 4
* for details. 5
* 6
 parameter (MAXFIRST=250, MAXSECOND=250, MAXTHIRD=10) 7
 real*8 a(MAXTHIRD,MAXSECOND,MAXFIRST) 8
 real*8 b(MAXTHIRD,MAXSECOND,MAXFIRST) 9
 real*8 sub_total(MAXFIRST), partial_total(4) 10
 real*8 d(MAXTHIRD), c, tmp ! local variables 11
 real*8 dist(MAXSECOND,MAXFIRST), grand_total 12
 real*8 grand_total ! test for proper operation 13
 logical parallel ! selects 2-version loops 14
 integer*4 iflag ! used to show LASTLOCAL value 15
 16
 data parallel /.false./ 17
 data sub_total, iflag /MAXFIRST*0.0, 0/ 18
* 19
* outer loop: contains both interior loops 20
* 21
 22
* C$doacross local(k,j,i,tmp,d,c), share(a,b,sub_total,dist), 23
* C$& lastlocal(iflag) 24
 25
 do i = 1, MAXFIRST 26
* 27
* first inner loop: fills arrays a and b 28
* 29
 30
* C$doacross local(j,k,c), share(i,a,b) 31
 32
 do j = 1, MAXSECOND 33
 do k = 1, MAXTHIRD 34
 a(k,j,i) = dsqrt(dfloat(i*j*k)) 35
 c = 1.0 - a(k,j,i) 36
 if (c .le. 0.0 .and. i .lt. j*k) then 37
 c = -c 38
 else 39
 c = c**2 40
 endif 41
 b(k,j,i) = 32*(dcos(c)**5)*dsin(c)- 42
 1 32*(dcos(c)**3)*dsin(c)+ 43
 2 6*dcos(c)*dsin(c) 44
 enddo 45

 enddo 46
* 47
* seond inner loop: determines distance and starts summation 48
* 49
 50
* c$doacross local(j,k,d,tmp), share(i,a,b,dist,sub_total), 51
* c$& lastlocal(iflag) 52
 53
 do j=1, MAXSECOND 54
 tmp = 0.0 55
 do k = 1, MAXTHIRD 56
 d(k) = a(k,j,i) - b(k,j,i) 57
 enddo 58
 do k = 1, MAXTHIRD 59
 tmp = tmp + d(k)**2 60
 enddo 61
 dist(j,i) = dsqrt(tmp) 62
 if (dist(j,i) .le. 0.1) iflag = iflag + 1 63
 sub_total(j) = sub_total(j) + dist(j,i) 64
 enddo 65
 enddo 66
* 67
* the next section is an example of sum reduction optimized to the 68
* parallel environment and the use of a more efficient 2 loop summation 69
* 70
* if -mp option is active, parallel is set to .true. which then 71
* selects the parallel version 72
* 73
 74
C$ parallel = .true. 75
 grand_total = 0.0 76
 if (parallel) then ! parallel version 77
C$ num_threads = mp_numthreads() 78
 ichunk = (MAXFIRST + (num_threads - 1))/num_threads 79
 80
C$doacross local(k,j), 81
C$& share(num_threads,partial_total,sub_total,ichunk) 82
 83
 do k = 1, num_threads ! this loop is parallelized 84
 partial_total(k) = 0.0 85
 do j = k*ichunk - ichunk + 1, min(k*ichunk,MAXFIRST) 86
 partial_total(k) = partial_total(k) + sub_total(j) 87
 enddo 88
 enddo 89
 do j = 1, num_threads ! smaller loop handled as scalar 90
 grand_total = grand_total + partial_total(j) 91
 enddo 92
 else ! the scalar version 93
 do j = 1, MAXFIRST 94
 grand_total = grand_total + sub_total(j) 95
 enddo 96
 endif 97
 98
 if (parallel) then 99
C$ write (*,10) grand_total, num_threads 100
C$ write (*,20) iflag 101
 else 102
 write (*,30) grand_total 103
 write (*,40) iflag 104

 endif 105
 stop 106
C$10 format(1x,'grand total = ',g10.3,'threads = ',i4) 107
C$20 format(1x,'parallel iflag = ',i10) 108
30 format(1x,'grand total = ',g10.3) 109
40 format(1x,'scalar iflag = ',i10) 110
 end 111





[LISTING TWO]


(source code)

 subroutine example(a, b, c, n)
 integer*4 n
 real*4 a(n), b(n), c(n)

 (additional code)

c$doacross local(i, x)
 do i=1, n
 x = a(n) * b(n)
 c(n) = x**2
 enddo

 (additional code)

 return
 end

(the loop is transformed to)

 subroutine _example_1(
 1 _local_start, ! index starting value
 2 _local_ntrip, ! number of loop executions
 3 _incr, ! index increment
 4 _my_threadno) ! unique process ID number

 integer*4 _local_start, _local_ntrip, _incr, _my_threadno

 integer*4 i ! declared local
 real*4 x ! declared local

 integer*4 _tmp ! created local

 i = _local_start
 do _tmp = 1, _local_ntrip
 x = a(i) * b(i)
 c(i) = x**2
 i = i + _incr
 enddo
 return
 end



Example 1: A typical DO loop


do i = 1, n
 a(i) = x * b(i)
enddo


Example 2: A DO loop in which the array variable references a
value that is not current with the index

do i = 2, n
 arr(i) = b(i) - arr(i-1)
enddo

Example 3: An example of load imbalance

 do i = 1, n
 do j = 1, i
 a(j, i) = a(j, i) * xmult
 enddo
 enddo


Example 4: Load balancing

 num_threads = mp_numthreads()
c$doacross local(i, j, k)
 do k = 1, num_threads
 do i = k, n, num_threads
 do j = 1, i
 a(j, i) = a(j, i) * xmult
 enddo
 enddo
 enddo



























November, 1989
PROGRAMMING PARADIGMS


Two Early Neural Net Implementations




Michael Swaine


There is exciting work being done in neural networks, such as the artificial
retina project that Francis Crick is involved with. Unfortunately, most of the
readily accessible implementations of neural network algorithms are
unimpressive. The reason is that, for most of us, a "readily accessible"
implementation of an algorithm is one that runs on a sequential machine, and
neural networks are inherently nonsequential. They are, as Dave Parker defined
them here last month, parallel implementations of minimization algorithms. The
algorithms themselves can be interesting to examine, and last month we looked
at some of these algorithms, especially Parker's own PC implementation of the
back propagation algorithm. But we did not address the other half of the
issue: The parallel implementation.
This month we'll look at two early implementations of neural nets. Both were
remarkable successes in their target domains. Both, interestingly, used analog
devices to implement parallel algorithms in largely discrete systems. Both
addressed the canonical neural net problem of visual pattern classification,
but not in the same way. And both pose some challenges for current neural net
implementations.


MINOS II


MINOS I and its successors, MINOS II and III, were built at SRI under contract
from the U.S. Army over a period from 1958 to 1967. Little was ever published
in the general press regarding these machines, but they were described in
contemporary documents, all unclassified and available to the curious. That
point is worth making because of the controversy over Minsky and Papert's
attack on neural net research in their 1969 book Perceptrons. Information on
the MINOS project was at least accessible to Minsky and Papert before they
wrote Perceptrons, and there is evidence that Minsky had visited the SRI labs
and was aware of the project's objectives and results in the 1960s.
The stated objective of the MINOS work was "to conduct a research study and
experimental investigation of techniques and equipment characteristics
suitable for practical application to graphical data processing for military
requirements." I take that to mean "to find workable algorithms and
architectures for military image processing." Or: "See if you can build us a
machine to spot tanks in aerial recon photos." The SRI team approached the
problem by building artificial neurons out of multi-aperture magnetic cores
and linking them in a network under the control of a learning algorithm. Or:
They implemented a minimization algorithm in a parallel architecture. Or: They
built a neural net.
As illustrated in Figure 1, MINOS II most clearly shows the organization of
the machines. MINOS II consists of four units: An optical preprocessor, an
adaptive unit, a training/comparator unit, and an output unit. The
preprocessor (see Figure 2) takes the data in the form of a series of static
image frames from slides, film, or a TV camera, and compresses each frame to a
100-bit word, which it passes to the adaptive unit. The adaptive unit, a
neural net, performs the classification, and is called adaptive because it
learns.
There are two phases in the operation of the machine: Training and
classifying. The training/comparator unit is only active during the training
phase, when it accepts correct responses as input, compares them to the
responses generated by the adaptive unit, and adjusts parameters called
weights in the adaptive unit, based on the comparison. The output unit
displays the results.
The preprocessor contains a 32 x 32 array of lenses in front of a photographic
plate, each lens reading in parallel from a storage tube. The tube gets its
data from a television camera (there are also provisions for reading from
slides and film), so the preprocessor is essentially reading a digitized
real-world image off a "dumb" retina. As Figure 3 shows, the photographic
plate has a mask for each of the 1024 images, and a photocell associated with
each image/ mask generates a binary signal based on the amount of light
transmitted. The preprocessor then employs some algorithm, usually a
task-specific algorithm, to reduce the 1024 bits to a single 100-bit code for
input to the adaptive unit. The algorithm is generally not very sophisticated,
because the goal is not true pattern classification, which comes later, but
simply gross data reduction without losing features of the data that will be
needed later for the pattern classification.
The adaptive unit is made up of threshold logic units (TLUs) in two layers.
Each TLU computes a weighted sum of its binary inputs and generates a binary
output, which is 1 if the weighted sum exceeds some threshold, and -1
otherwise. Physically, these TLUs are magnetic cores. The inputs to the 63
TLUs in the first layer are the 100 bits of the code word supplied by the
preprocessor; that is, the preprocessed representation of an input image
frame. These 63 TLUs feed the nine TLUs in the second layer, each second-layer
TLU taking input from seven first-layer TLUs and computing a majority-rule
function, outputting a 1 only if a majority of its inputs are 1s. The use of
these second-level "committee" TLUs facilitates having the machine respond
correctly without requiring that every TLU do so.
The nine second-level TLUs provide for 2{9}, or 512 different outputs, so the
machine is capable of classifying its input patterns into any of 512
categories. During the training phase, the correct 9-bit classification is
input for each input pattern, and the training/comparator unit compares the
output of the adaptive unit with this correct answer. If they are not equal
(as they usually will not be early in training) the training algorithm traces
back through the adaptive unit's TLU outputs to adjust weights so that the
correct response will result the next time this input pattern is presented. It
first looks at second-level TLUs, then at the first-level TLUs feeding the
incorrect second-level TLUs. The algorithm works so as to change as few
weights as possible, and to change those weights that have to be changed by
the smallest amount. (The weights, it should be noted, are analog values.)
Testing of the complex MINOS machines must have been trying. The reports on
the work mention experiments terminated because of things like malfunctioning
slide projectors. The hard-earned results are interesting.
In a test that explored its ability to pick out objects against a noisy
background, MINOS II was given aerial photos, some of which showed tanks, and
was trained to pick out the photos with tanks. It learned to classify a set of
50 photos after 28 iterations of the set. It was then presented with a new set
of photos, and classified 32 out of 34 of them correctly, at least one of its
two errors being "reasonable;" -- one a human classifier might have made as
well.
Another test employed more categories, requiring MINOS II to classify standard
military map symbols, presented in a variety of orientations, into 15
categories. With a total of 30 symbols, MINOS II was trained to infallible
performance on this set in 40 iterations. With 75 symbols, which meant five
different orientations for each symbol, it had not yet learned the
classifications in 75 iterations, at which time a slide projector problem
terminated the experiment. How good was MINOS II? It's hard to say. William
Huber, who was the project monitor for the Army for the duration of the
project, cited one complicating factor in his 1967 paper on the work: The
system's tendency to "train around" defective operations, much as the human
brain relearns over new pathways after cells are damaged. One day a power plug
fell out of the wall and the machine was short one power supply; the only
evidence of this was that training took a couple more iterations than usual.
Measures of performance are also fairly meaningless when the exact task cannot
easily be reconstructed on a conventional computer.
One thing is certain: MINOS I knew, in 1960, how to solve the XOR problem that
typified Minsky and Papert's devastating critique of neural networks in
Perceptrons. In fact, the MINOS team used a generalization of the XOR problem
as a routine test for the machine.
The generalization of the XOR problem was to differentiate a horizontal bar
from a vertical one anywhere in the pixel figure, and the retina was taken to
be an 8 x 8, toroidally connected field, with the right edge contiguous with
the left, the top with the bottom. The toroidal extension allowed eight,
rather than five, positions in each orientation.
There's one challenge for any current neural net: Solve the toroidal XOR
problem.


ADAM I


The history of the ADAM project is so poorly documented as to be lost in
legend; I couldn't find the date on which the project was begun, there are no
reliable contemporary descriptions from that period, and there have been
acrimonious debates over the genesis of the device. The ADAM I has always been
a favorite topic in the popular press, though, and it has been reexamined
recently in the computer science literature in the light of the resurgence of
interest in neural networks. ADAM I was one of the most successful neural net
implementations ever. It was also extremely complex; current neural net
implementations tend to be much simpler, and this is probably appropriate, but
there is much that can be learned from a study of the ADAM I architecture.
Visual images are generated in ADAM I by a lens that projects scenes onto a
grid of photoreceptive cells. These cells, along with two layers of other
cells connected in a network, form the receptor net, which in turn connects to
a highly parallel central processing system that in a sequential machine would
be called the CPU, but in ADAM I is called the CNS (central neural system?).
There are actually two separate and redundant receptor nets to provide two
images for further processing. This system of organization provides the raw
data for the perception of depth.
There is a high degree of organization and differentiation at the level of the
receptor net. There are two kinds of photoreceptive cells, responding to
different wave lengths of light, and the receptors have many-to-one and
one-to-many connections with cells in the next level. These intermediate cells
usually have many-to-one but sometimes have one-to-many connections with the
last layer of cells in the receptor net. Then there are cells that carry
signals between two photoreceptive cells, allowing interaction and inhibition,
and there are also cells that carry signals back to the photoreceptors from
lower cells.
This complex organization permits a lot of processing and transformation of
the image in the receptor net itself, yet it does not alter the overall
topography of the initial image. The main purpose of the receptor net, as with
MINOS II's preprocessor, is data reduction. By the time the data gets through
the receptor net, the number of bits transmitted has been reduced by a factor
of 1000. The receptors are extremely sensitive, responding to one quantum of
light energy, and despite the 1000-fold data reduction, the entire system is
also very responsive: Six quanta striking six different receptors can be
enough to trigger an output response.
ADAM I was not the first device to employ this general kind of receptor-net
organization: The earlier-still FROG I showed a high degree of specialization
in cells in its receptor net, with four or five kinds, each extracting certain
local features from its bitmap input. One such feature was the center-surround
pattern, which responds to a pattern consisting of a small dark patch on the
image, surrounded by a lighter area. Other specialized cells responded to
changes from one image to another; that is, movements of objects in the
imaging field. In FROG I the CNS reflected the four or five cell types of the
receptor net and allowed the system to recognize four or five classes of
visual phenomena. In a sense, that's all the discriminating it could do.
Some such specialized cells were retained at the lower levels in later models,
including ADAM I, but in the evolution of the architecture, many such
functions migrated higher in the system, to the CNS. In ADAM I, there is a
faithful reconstruction at the CNS level of the topology of the received
image, but only a few of the cells in the CNS use the information in this
form. Most CNS cells, in several levels deep, do more complex processing of
features of the image. They do edge detection and center-surround
identification, for example.
ADAM I, which has not yet gone out of production, could solve some remarkably
difficult classification problems. One problem that the ADAM I architecture
solves is to count the number of ADAMs in a complex scene displayed in low
resolution. I'll let you solve the problem for yourself.
You might try that recognition task on your favorite neural net.
Then again, maybe you just did.














November, 1989
C PROGRAMMING


C++ and Linked Lists




Al Stevens


This month I'm venturing further into C++ with a new class, the LinkedList.
Programmers have been using the linked list data structure for years. It
allows you to store an unknown number of data items in a table where the
entries do not need to be adjacent in memory. It also solves the problems
encountered with standard arrays when the entries have different sizes. Here's
how the linked list works.
A linked list data structure links a list of data items together with a set of
pointers. The list itself has a pointer to the first and last entries in the
list. These pointers are called the "list head." A linked list consists of the
list head and the data entries. Each data entry in the list has a pointer to
the one just previous to it and a pointer to the one just past it. The first
and last pointers in the list head are initially NULL. They are initialized
with non-NULL values when the first item is added to the list. The first item
in the list has a NULL previous-item pointer, and the last item in the list
has a NULL next-item pointer.
To traverse a linked list in a forward sequence, you start with the value in
the first-item pointer from the list head. If that pointer is NULL, the list
is empty. Otherwise, it points to the first data item. That item has a pointer
to the next data item. You move forward by using the next-item pointer in each
successive item until you find an item with a NULL next-item pointer, meaning
you are at the end of the list. A reverse-sequence route through the list
works the same way but starts with the last-item pointer in the list head and
uses the previous-item pointer in the items.
To add an item to the end of the list, you make its previous-item pointer
point to where the last-item pointer points, and you make the last-item
pointer point to the new item. You must also make the next-item pointer in
what was the last item point, with its next-item pointer, to the new item.
Deleting an entry from a linked list is a matter of breaking and repairing the
next-last chain. The deleted item's previous item must now point to the
deleted item's next item and vice versa. Of course, you must test if the
deleted item is the first or last (or both) and repair the first and last
pointers in the list head accordingly.
That's what a linked list is. They all work pretty much the same. Where they
differ is in the format of the data items they manage. As such, the linked
list is a good candidate to be a generic C++ class, and I'm sure that's not a
new idea.
Listing One, page 152 and Listing Two, page 152, are linklist.h and
linklist.cpp, the source files that implement the LinkedList class.
To use the LinkedList class, you include linklist.h in your program.
Surrounding a data structure with the properties of a linked list is simply a
matter of declaring a LinkedList object and adding those structures as entries
to the list. The data structure itself can be as simple as a character and as
complex as a new class. When you pass its address and size to the linked list
member function (method) that adds entries, you control the data in a linked
list. Here is how you declare a list:
 LinkList mylist;
That's all there is to it. Until the class goes out of scope, the list, empty
at first, exists. Here is how you add data entries to the list:
 mylist.addentry(&data, sizeof(data));
If you declare a list and then begin to add entries, they will be added in
turn at the end of the list. Each new entry follows the one just before it,
and is at the end until another one comes along.
If you traverse a list and then add an entry while positioned in the middle of
the list, the entry will be added just past the one you most recently
retrieved. If you are positioned at the start of the list, the entry will be
added as the first one.
To retrieve an entry from a linked list, you meander through the list forwards
or backwards and look at the data values within each entry. Each of the member
functions that pass through the list returns a pointer to the list entry that
it found. Here are the methods for traversing the list:
 void *getfirst(void);
 void *getnext(void);
 void *getprev(void);
 void *getlast(void);
 void *getcurr(void);
You would call one of them in this manner:
 char *cp = mylist.getfirst();
Each of these member functions returns a pointer to the data value of the
found entry. You can assign these returned values to a pointer to whatever you
have in the list. If the list is empty or a list scan reaches the end or
beginning, the function returns a NULL pointer.
To delete an entry, you navigate to it and call the member function that
deletes the current entry. Here is the call for that operation:
 mylist.delete_entry();
Listing Three, page 152 is demolist.cpp, a C++ program that uses the
LinkedList class to collect and manipulate a list of names. The program uses
the string class from last month, and the first thing I realized was that the
string class needs a member function that returns the size of the string. Now
this is the time when a rigorously controlled object-oriented project would
likely build a derived class just to add that feature. But, because we are not
so rigorous, and not controlled at all, we'll just add the function to our
original string class. Refer to strings.h from last month, and insert these
lines with the member functions:
// -------- return the length of a string
int length(void)
 { return strlen(sptr)+1;}
The demolist program prompts you to type some names into a string. It adds
each name you type to a LinkedList. When you are done, you type "end," and the
program displays a menu. You can display the names, insert a new name at the
current location, move forward and backward through the list, and delete the
current name.


C++ Compilers


The C++ code for the past few months works with Zortech C++. I compiled and
ran the string code from last month with two other PC compilers, Intek C++ and
Guidelines C++. These compilers are ports of the AT&T C++ 1.2 release, and, as
such, are really language translators. You need a compiler as well as to
compile the C code that the C++ translator builds from your C++ source code.
Intek C++ has versions that work with Watcom C, Turbo C, Microsoft C, and
MetaWare High C. The Intek compiler works only on a 386 machine with extended
memory. I had to disable my Expanded Memory Manager to use Intek C++, and I
found one serious bug: The compiler driver program only works every other time
you run it. It needs a control file, which it builds if it does not find it.
After the compile is done, if the compile goes to completion, the compiler
process deletes the file. The problem is that if the file is not there, the
program builds it but aborts. If the file is there, the program uses it and
deletes it. Therefore, the program works only every other time. The technical
support people at Intek were not aware of this problem and seemed to think
there was something wrong with my setup. Only after I told them the exact
sequence of steps to reproduce the problem did they acknowledge and promise to
fix it. I got around it with a batch file that saves and restores the critical
control file. Because of the hardware requirements and the price ($495 plus
the price of your compiler), I don't see much of a future for this product in
the PC world.
Guidelines C++ is cheaper ($295) than Intek and does not require a 386 or
extended memory. You do need to add the price of a compiler, however.
Guidelines C++ is available to work with Microsoft C only, although they tell
me a Turbo C version will be included when they release their port of the AT&T
C++ 2.0 release, which should have occurred by the time you read this. I found
no problems with Guidelines C++ other than those imposed by C++ 1.2 itself.
The AT&T translator is not aware of the ANSI C treatment of void pointers
against real pointers. ANSI C allows a function to have a prototype that
specifies a void pointer. The calling function can pass any kind of pointer,
and the called function deals with it as though it were a character pointer.
Similarly, a function that is prototyped to return a void pointer can be
called to assign its value to any kind of data pointer. This convention allows
you to use functions such as malloc and memcpy without needing casts. It also
allows the NULL global to be #defined as a void pointer with the value zero.
The AT&T C++ 1.2 translator gags on such doings, and so the code in this
month's column works with neither Intek nor Guidelines C. I gagged at putting
all those casts in, being spoiled by my use of ANSI-conforming compilers of
late, and so I decided to stick with Zortech C++ until the others come out
with C++ 2.0.
The Zortech C++ compiler is a real bargain when compared to the others. It has
a list price of $150, and you do not need another compiler to use it. It is
built to work with the new-style ANSI conventions. (A C compiler comes with
the package.) Its disadvantage is that it is not a pure port of the AT&T code;
it does not share the same bugs and features with the rest of the C++ world,
and so your code will not be as portable as you might want it to be.


The ANSI Corner: Protests, Trigraphs, Escape Sequences, and Strings


Do you wonder why it takes so long to get a language standard approved? The
ANSI X3J11 committee is nearing six years defining the standard for the C
language. Perhaps this story will help to explain why.
Many years ago, I worked for a small consulting firm. We bid on a government
contract that disqualified as bidders any manufacturers of computing
equipment. The government awarded the job to a large firm, and my boss wrote a
letter of protest saying the winning bidder should have been disqualified
because they manufactured teletype machines, then a common console terminal
device. Such a protest automatically invokes a bureaucratic procedure of
response and usually delays the work until the matter is closed. Every time we
got an official reply, we simply wrote another letter, keeping the wheels of
inquiry turning while the wheels of progress were stuck. I asked him why he
bothered. He said that the letters cost him only his time, while the rewards
were in knowing that he had thrown a monkey wrench into the works. And, as he
said, "... just because they p***ed me off."
The ANSI X3J11 committee has similarly irritated a C programmer, a Mr. Russell
Hansberry, and he has responded in turn with an appeal that challenges the
validity of X3J11's work. Like the nuisance letters of my whimsical boss of
yore, this appeal has further delayed approval of the proposed C standard.

The first problem was that Hansberry's original comments were overlooked by
the committee, probably lost in the shuffle. The committee, having mislaid the
letter, failed to prepare the required formal response before submitting the
draft standard to X3, and so the letter's author could and did demand that the
committee back up and address his concerns.
X3J11 considered each of the points in the letter and voted to disapprove them
all. There was much ado and flapdoodle, but the disapproval prevailed by
democratic action. Most people who embrace the concepts of a self-governing
free society would have accepted majority rule and retreated. Hansberry,
apparently deciding that democracy was not working in his favor, chose instead
to attack on other fronts. Justice is never swift where people are free.
Hansberry exercised his rights and filed an appeal that raised 40 technical
and procedural issues in an apparent attempt to invalidate all of the
committee's work. X3 voted in August in a reconsideration ballot, and no
negative votes were cast, so the technical issues of Hansberry's appeal were
effectively defeated. The procedural issues, however, still remain, and they
must be addressed before approval is final. If X3 votes down these issues,
Hansberry can still appeal to ANSI. Some of his concerns have merit, but
others are aimed at changing the language in ways that would endanger existing
code. In the meantime, the rest of us are waiting for an approved standard for
the C language.
Maybe next year....


ANSI Additions


Although the original charter for the ANSI C standard specification was to
document a standard for existing practice of the C language, the proposed
draft does add some features to the C language. The most notable of these is
the new-style function definitions and declarations, called "prototypes,"
which were adopted from C++. Other new features are useful, too, but could
possibly go unnoticed unless you took the time to learn them. Among these
additions are trigraphs, hexadecimal escape sequences, a few new character
constants, and adjacent string literals.


Trigraphs


Trigraphs aren't as useful as the other additions, but you'll need to know
about them because you might trip over them some day. A trigraph is a way to
express those characters that C uses extensively but that do not exist in some
non-ASCII character sets, most notably a set defined by the International
Standards Organization as ISO 646. There are nine such characters that ISO 646
does not include. These are #, [, ], {, }, \, , ~, and ^. Try writing a C
program without them. To allow users of ISO 646 to program in C, ANSI has
introduced the use of trigraphs as a way to express these characters. A
trigraph is two question marks followed by a character that does exist in ISO
646 and that resembles the missing character. These are the trigraphs for the
missing characters:
 Trigraph Replaces

 ??= #
 ??( [
 ??/ \
 ??) ]
 ??' ^
 ??< {
 ??! 
 ??> }
 ??- ~
Write a program with these trigraphs, and it would probably win one of those
stupid obfuscated C contests (or a beautiful APL contest). Nonetheless, the
users of those other character sets should not be denied the language, and so
some accommodation was needed. The trigraphs are an inelegant but workable
solution. Most of us will never need to know about trigraphs except to deal
with the rare occasion when we want to put "??" into a string while the
compiler wants to turn it into a trigraph. (Use "? \ ?".)


Escape Sequences


In the C lexicon, an escape sequence is a value in a constant or string
literal that begins with the backslash character and that translates into a
character value. You already know about octal escape sequences. They have been
in traditional K&R C since the beginning. You can code a character constant
with an octal value like this:
 char c = '\ 101';
The backslash character followed by an octal digit (O to 7) tells the compiler
that the digits following the backslash form a character value from an octal
expression. With ANSI-conforming compilers you can now also express character
constants by using hexadecimal digits as shown here:
 char c = '\ x 41';
The backslash-x escape sequence tells the compiler that the digits that follow
will be a hexadecimal character constant. Both examples just given form the
character value for the ASCII 'A' and would be better expressed in this
manner:
 char c = 'A';
With an octal escape sequence, you are allowed from one to three octal digits
(O to 7) to form the value. This gives you a theoretical maximum value of
'\777' or 511. ANSI specifies, however, that a character constant cannot
exceed the range of an unsigned char, which, on a PC is 8 bits wide or a
maximum of '\377' or 255. Machines that have wide characters can have
character constants that extend to that limit, but the three-digit limit for
octal constants still applies, so the maximum for wide characters is 511
regardless of the wide character's width.
While an octal escape sequence can have no more than three digits, a
hexadecimal character constant can have any number of hex digits following the
backslash-x escape sequence. It, too, is restricted to the maximum value of
the unsigned char, so its limit on a computer with an 8-bit character is '\ xf
f' or wider if wide characters are implemented. Other character sets might
have much wider character sizes, so ANSI does not define a theoretical limit
for the hexadecimal character constant, thus allowing for the expression of
all character constant values in those cultures (Japan, for example) where
there are far more than 256 characters in the character set.
Several common character values cannot be expressed with single alphabetic,
numeric, or graphic characters. The Escape and Carriage Return characters are
two examples. You could express these characters with octal or hexadecimal
character constants, but such expressions would not be portable to machines
with incompatible character sets. ANSI provides for escape sequences for some
but not all of the non-displayable characters. The values shown here are in
the standard for some of the character constants that have, in the ANSI view,
universal application:
 '\a' audible alarm
 '\b' backspace
 '\f' form feed
 '\n' newline
 '\r' carriage return
 '\t' horizontal tab
 '\v' vertical tab
The '\ a' and '\ v' were added by X3J11.
They were not a part of the K&R definition, but the committee added them to
the standard because they have universal application. Most console devices
have a bell (BEL in ASCII) character that sounds an audible alarm, and many
printers will respond to a vertical tab character. ANSI decided not to include
'\ e' for the Escape character because some character sets -- EBCDIC, for
example -- have no equivalent character.
Some displayable ASCII characters have meaning in the context of a character
constant or string literal. To allow you to represent these characters, ANSI
provides the following escape sequences to provide portable expressions of
their values:
 '\" single quote
 '\"' double quote
 '\?' question mark
 '\\' backslash
The single quote needs the backslash in a character constant but will work
either way in a string literal. Conversely, the double quote needs the
backslash in a string literal but works either way in a character constant.
The '\?' is specified in the ANSI draft to accommodate its expression in
character sets that require the trigraphs discussed earlier. To get a double
question mark in a string, you code "?\?".
Although the draft is not specific at this point, if a backslash is followed
by anything other than an octal digit or one of the characters ?, '< ", a, b,
f, n, r, t, v, x, ', or \, the lonely backslash is ignored (unless nothing
follows it on the line, in which case the string is assumed to continue in the
first position of the next line). At least that is a convention followed by
most so-called ANSI-conforming compilers.



Strings


You can put octal and hexadecimal escape sequences into character string
literals as well, making these usages possible, all of which deliver the same
string value.
 char *cp = "\ 101phids";
 char *cp = "\ x41phids";
 char *cp = "Aphids";
An octal escape sequence in a string continues until the compiler finds a
non-octal digit or hits the third octal digit. A hexadecimal escape sequence
continues for as long as the compiler finds hexadecimal digits. Compilers
should warn you if the character values formed exceed the upper limit of a
character on the computer.
Obviously, the third format is what you would use for the value given in the
example above, but there are occasions where you will want string values that
contain characters not represented by displayable characters. The control
strings that command an ANSI video terminal use an Escape sequence to let the
terminal know that a command is coming. To clear the screen on an ANSI
terminal (or a PC with ANSI.SYS installed), you can use this statement:
 printf(" \ 33[2J");
The " \ 33" is an octal escape sequence that forms the single character value
for the Escape character. Suppose your command string needed to have the
Escape character followed by "345". This string would not work:
 printf(" \ 333 45");
The compiler would treat the third '3' as part of the octal character constant
and would build a string that began with the character '\ 333' followed by
"45". Because octal character constants are limited to three digits, you can
code the string this way:
 printf(" \ 0 333 45");
The leading zero does the trick by forcing the \ 033 into the maximum three
digits. (Users of Turbo C 2.0 should be aware that a bug in the compiler makes
it think that all the digits in the string are part of the octal constant.)
Here is the clear-screen statement with a hexadecimal escape sequence:
 printf(" \ x1b [2J");
The " \ x1b" is the hexadecimal escape sequence for the Escape character.
Suppose now that you wanted the Escape, "345" string used above. This would
not work:
 printf(" \ x 1b 3 45");
The compiler would assume that all the digits following the " \x" are part of
the character constant because hexadecimal character constants can be wider
than two digits. A compiler for an 8-bit character machine should issue a
warning for this example, because the value exceeds the range of the unsigned
character. It should not, however, assume that you meant it to be Escape,
"345" just because you have 8-bit characters. Such a convention would promote
the development of non-portable code. So, how do you get it to work? You could
write it this way:
 printf(" % d 3 4 5", '\ x1b');
That would work, but only where you are using the printf-style formatted
output. Suppose you wanted to write the string with the puts function instead
of printf. For that purpose ANSI introduced the adjacent string literal.


Adjacent String Literals


If you code two string literals side-by-side, they are concatenated as one
null-terminated string. Here is a familiar example:
 printf("Hello," " world");
This feature has several advantages. You can express long string literals more
legibly by breaking them into several adjacent strings and putting each part
on its own line. But, even better, the problem of the hexadecimal character
escape sequence is solved. Now we can do this:
 printf(" \ x1b" "345");
And, finally, a more descriptive way of coding the string is this:
 #define ESCAPE " \ x1b"

 printf(ESCAPE "345");


Nobody Says the S Word....


One other thing X3J11 added to our culture was the word, "stringizing," which
they coined to mean something new for the C preprocessor. It's an awful word
but a useful feature. I'll use the feature, but I'll never use the word again
except maybe to tell you how awful it is. Next month I'll explain the feature
with the awful S word.

_C PROGRAMMING COLUMN_
by Al Stevens


[LISTING ONE]

// -------- linklist.h

#ifndef LINKLIST
#define LINKLIST

#include <stdio.h>

class LinkedList {
 typedef struct list_entry {
 struct list_entry *NextEntry;

 struct list_entry *PrevEntry;
 void *entrydata;
 } ListEntry;
 ListEntry *FirstEntry;
 ListEntry *LastEntry;
 ListEntry *CurrEntry;
public:
 // ---- constructor
 LinkedList(void)
 { FirstEntry = LastEntry = CurrEntry = NULL; }
 // ---- destructor
 ~LinkedList(void);
 // ---- add an entry
 void addentry(void *newentry, int size);
 // ---- delete the current entry
 void delete_entry(void);
 // ---- get the first entry in the list
 void *getfirst(void);
 // ---- get the next entry in the list
 void *getnext(void);
 // ---- get the previous entry in the list
 void *getprev(void);
 // ---- get the last entry in the list
 void *getlast(void);
 // ---- get the current entry in the list
 void *getcurr(void)
 {return CurrEntry==NULL ? NULL : CurrEntry->entrydata;}
};

#endif








[LISTING TWO]

// -------------- linklist.cpp

#include <string.h>
#include "linklist.h"

// ------- linked list destructor
LinkedList::~LinkedList(void)
{
 ListEntry *thisentry = FirstEntry;

 while (thisentry != NULL) {
 delete thisentry->entrydata;
 ListEntry *hold = thisentry;
 thisentry = thisentry->NextEntry;
 delete hold;
 }
}

// --------- add an entry to the list

void LinkedList::addentry(void *newentry, int size)
{
 /* ------- build the new entry ------- */
 ListEntry *thisentry = new ListEntry;
 thisentry->entrydata = new char[size];
 memcpy(thisentry->entrydata, newentry, size);

 if (CurrEntry == NULL) {
 thisentry->PrevEntry = NULL;
 // ---- adding to the beginning of the list
 if (FirstEntry != NULL) {
 /* ---- already entries in this list ---- */
 thisentry->NextEntry = FirstEntry;
 FirstEntry->PrevEntry = thisentry;
 }
 else {
 // ----- adding to an empty list
 thisentry->NextEntry = NULL;
 LastEntry = thisentry;
 }
 FirstEntry = thisentry;
 }
 else {
 // ------- inserting into the list
 thisentry->NextEntry = CurrEntry->NextEntry;
 thisentry->PrevEntry = CurrEntry;
 if (CurrEntry == LastEntry)
 // ---- adding to the end of the list
 LastEntry = thisentry;
 else
 // ---- inserting between existing entries
 CurrEntry->NextEntry->PrevEntry = thisentry;
 CurrEntry->NextEntry = thisentry;
 }
 CurrEntry = thisentry;
}

// ---------- delete the current entry from the list
void LinkedList::delete_entry(void)
{
 if (CurrEntry != NULL) {
 if (CurrEntry->NextEntry != NULL)
 CurrEntry->NextEntry->PrevEntry = CurrEntry->PrevEntry;
 else
 LastEntry = CurrEntry->PrevEntry;
 if (CurrEntry->PrevEntry != NULL)
 CurrEntry->PrevEntry->NextEntry = CurrEntry->NextEntry;
 else
 FirstEntry = CurrEntry->NextEntry;
 delete CurrEntry->entrydata;
 ListEntry *hold = CurrEntry->NextEntry;
 delete CurrEntry;
 CurrEntry = hold;
 }
}

// ---- get the first entry in the list
void *LinkedList::getfirst(void)
{

 CurrEntry = FirstEntry;
 return CurrEntry == NULL ? NULL : CurrEntry->entrydata;
}

// ---- get the next entry in the list
void *LinkedList::getnext(void)
{
 if (CurrEntry == NULL)
 CurrEntry = FirstEntry;
 else
 CurrEntry = CurrEntry->NextEntry;
 return CurrEntry == NULL ? NULL : CurrEntry->entrydata;
}

// ---- get the previous entry in the list
void *LinkedList::getprev(void)
{
 if (CurrEntry == NULL)
 CurrEntry = LastEntry;
 else
 CurrEntry = CurrEntry->PrevEntry;
 return CurrEntry == NULL ? NULL : CurrEntry->entrydata;
}

// ---- get the last entry in the list
void *LinkedList::getlast(void)
{
 CurrEntry = LastEntry;
 return CurrEntry == NULL ? NULL : CurrEntry->entrydata;
}





[LISTING THREE]

// -------- demolist.cpp

#include <stream.hpp>
#include "strings.h"
#include "linklist.h"

void collectnames(LinkedList& namelist);
int menu(void);
void displaynames(LinkedList& namelist);
void stepforward(LinkedList& namelist);
void stepbackward(LinkedList& namelist);
string insertname(LinkedList& namelist);

void main(void)
{
 cout << "Enter some names followed by \"end\"\n";
 // ------ a linked list of names
 LinkedList namelist;
 collectnames(namelist);
 int key = 0;
 while (key != 6) {
 switch (key = menu()) {

 case 1:
 displaynames(namelist);
 break;
 case 2:
 stepforward(namelist);
 break;
 case 3:
 stepbackward(namelist);
 break;
 case 4:
 insertname(namelist);
 break;
 case 5:
 namelist.delete_entry();
 break;
 case 6:
 cout << "Quitting...";
 break;
 default:
 break;
 }
 }
}

void collectnames(LinkedList& namelist)
{
 // ------- until the user types "end"
 while (insertname(namelist) != "end")
 ;
}

int menu(void)
{
 cout << "\n1 = display the names";
 cout << "\n2 = step forward through the names";
 cout << "\n3 = step backward through the names";
 cout << "\n4 = insert a name";
 cout << "\n5 = delete the current name";
 cout << "\n6 = quit";
 cout << "\nEnter selection: ";
 int key;
 cin >> key;
 return key;
}

// ------ read the names in a list and display them
void displaynames(LinkedList& namelist)
{
 cout << "------ NAME LIST ------\n";
 char *name = namelist.getfirst();
 while (name != NULL) {
 cout << name << "\n";
 name = namelist.getnext();
 }
 cout << "-----------------------\n";
}

// ------- step forward through the list of names
void stepforward(LinkedList& namelist)

{
 char *name = namelist.getnext();
 cout << (name ? name : "-- End of list --") << "\n";
}

// ------- step backwardward through the list of names
void stepbackward(LinkedList& namelist)
{
 char *name = namelist.getprev();
 cout << (name ? name : "-- Beginning of list --") << "\n";
}

// ------- insert a name into the list
string insertname(LinkedList& namelist)
{
 cout << "Enter a name: ";
 // ----- a string to hold one name
 string name(80);
 // ------- read a name
 cin >> name.stradr();
 // ------- add it to the list
 if (name != "end")
 namelist.addentry(name.stradr(), name.length());
 return name;
}





































November, 1989
STRUCTURED PROGRAMMING


Poly Want An Object?




Jeff Duntemann, K16RA


The toughest nut to crack in OOP's very full bowl of new concepts is certainly
polymorphism, which sounds more like a developmental defect in parrots than a
programming technique. Its very mystical - magical (dare I say legendary?)
sound makes it irresistible to marketing types and other hypesters, few of
whom can even pronounce it, much less tell you what it means.
Tersely put, polymorphism is the ability of different objects to respond in
their own way to a single directive. I'm reminded of that point in Cole
Porter's Kiss Me, Kate where Lois sings, "I'm always true to you, Darling, in
my fashion . . ." As Cole Porter fans know, being true means one thing to most
people and quite something else again to Lois.
But that's polymorphism in a nutshell. If each object in a group of five
objects has a method that edits the object's internal data you, the programmer
(as the user of the objects) don't have to know the specifics of editing each
type of data within each of the five objects. One command -- Edit -- can be
given to any of the five objects. Each object will respond to that command by
executing an editing routine custom written to serve its own data. An object
containing string data might execute a simple line editor. An object
containing Boolean data might allow its data to be flipped between its two
alternate states on presses of the space bar. An object containing a color
value might place a color chart on the screen and allow the user to edit
colors by bouncing a pointer along the edges of the chart.
It's all editing -- but each object allows the editing to be done in its own
fashion.


Reading the Mail


OK -- so what? You can always write up an editing routine for each type of
data you deal with and then CASE your way to the correct editor for each type
of data you encounter:
CASE DataType OF
 DBoolean :EditBooleanData;
 DString :EditString;
 DInteger :EditIntegerData;
 ColorValue :EditColorValue;

 etc...
You don't need object-oriented programming to do that.
True. But let's consider an advanced network-based electronic mail system.
People can send you letters, drawings, images, even digitized voice mail over
the cable. When something comes in over the net for you, an icon appears in
your in-basket area. To read the mail, you have to (in effect) edit each of
the different items of mail. Letters can be edited in a text editor; images in
an image editor. Voice mail has to be sent to a digital analog converter. So
far, so good -- you've got tools for each of those kinds of mail. But ... what
happens when you receive a technical drawing in a vector format that you
haven't seen before? If you don't have a tool for it, you can't read it. In
other words, you must be able to anticipate every different kind of mail that
comes down the cable.
Now, if you received complete objects from the net instead of just data, you
could simply point at an incoming object of uncertain type and say, "Mr.
Oddball Object, start your editor." The object obeys. It brought its own
editor along via encapsulation, and it knows how to invoke its editor. Up pops
the window and away you go, reading a type of mail you've never encountered
before.
(Side note: Executing objects received over a net like this would require
runtime dynamic linking and may not be possible using Turbo Pascal in its
current form. Such a system, however, is certainly within the realm of
object-oriented programming as we will know it within a few years.)
That's what polymorphism is good for: It decreases coupling to the point where
a system can accommodate the unexpected. If all elements of a system conform
to certain high-level standard directives, the details of implementing those
directives can be left to the individual objects without having to "inform"
the rest of the system how to deal with those objects in a detailed way.
In other words, if every element of the system knows what it means to "edit,"
none of the elements need to know how to edit anything except itself.
Each object is thus true to the system specs in its fashion.


Bound for Glory


That, however, is simply what polymorphism is. How to do it is another
question. All object-oriented languages are capable of polymorphism (in fact,
if a language can't polymorph it's not object-oriented) and each one
implements polymorphism in a slightly different way. In this column I'll be
describing Turbo Pascal's mechanisms because I understand them. Once I figure
out what QuickPascal is doing beneath the surface (which is not covered in
their rather thin little manual) I'll try to explain its innards as well.
Polymorphism depends utterly on another new concept called "late binding."
Binding has always existed in Pascal, but like one hand clapping no one took
much notice until there was another way to go about it.
Binding is simply the process by which the caller of a routine is given that
routine's address. In traditional Pascal and C, binding happens at compile
time, or at link time, if a link step is present. (Turbo and QuickPascal both
compile and link in one pass, so compilation and linking are actually the same
process.) The name of the called routine in the code is replaced by that
routine's address. Thus a link between the two is created early in the
program's life, and we call it "early binding."
Until object-oriented extensions to Pascal and C were developed, early binding
was all the binding there was. Early binding requires that all possible
procedure calls be known at compile time, precluding anything like the
electronic mail system we described earlier. In an early-bound system, the
unexpected is anathema -- all data and all code must be understood before the
program is ever run.


Virtual Method Tables


Turbo Pascal's system for achieving late binding is elegant and about as fast
as such a system is likely to be. It works like this:
All normal procedure calls in Turbo Pascal are early bound. So are calls to
all object methods that are not declared as virtual. Virtual methods in Turbo
Pascal are methods that are late bound rather than early bound. The new
reserved word VIRTUAL is present immediately after the header of all virtual
methods.
Not all objects contain virtual methods. Methods default to being static, and
are treated the same way as ordinary procedure calls. Unless you hang the
reserved word VIRTUAL after a method declaration, that method remains static
and early bound.
All objects that do contain virtual methods also contain a little something
extra: An invisible data structure called a "virtual method table" (VMT). Each
object type with virtual methods has one virtual method table, stored at the
beginning of the data segment. The commonest mistake in understanding Turbo
Pascal's late binding machinery is to believe that each object instance has
its own virtual method table. Not true -- if you have 50 objects of
object-type Circle, all 50 use the same VMT. Remember: The type owns the VMT
-- not the instance!
The VMT is nothing more than a table of 32-bit addresses. There is one "slot"
in the VMT for each virtual method -- defined or inherited -- in the VMT's
object type. (Actually, the first 32 bits of the VMT contain non-address
information, including the size of the object that owns the VMT. We'll get
back to that later on, in connection with constructors and destructors.)
Figure 1 summarizes this admittedly complicated and difficult
behind-the-scenes machinery.
The VMT structure in the data segment is created at compile time, when the
definition of the object that owns it is encountered by the compiler. The
addresses in the VMT (Bullet 1 on Figure 1) are also written at compile time,
and never change. This is another source of misunderstanding, the feeling that
somehow the addresses in the VMT are swapped around at runtime. Again, not so:
The VMT itself is fully created and filled at compile time and does not change
in any way after that!
The VMT is created in the lowest portion of the data segment, but it is
present in the .EXE image file that Turbo Pascal writes to disk. (More than
code is written to disk as part of the .EXE file -- typed constants are placed
there as well.)



"How Much, But Not What of ..."


Remember Bullwinkle's famous recipe for hush-a-boom? (Jay Wardian's cartoon
explosion that made no noise while blowing up.) He had in fact only half the
piece of paper containing the recipe, leading to his memorable line, "I know
how much, but not what of." Late binding is a lot like bringing the two halves
of Bullwinkle's recipe together. The VMT is one half, (the "what of") fully
created and ready to go in the data segment at compile time. The other half of
the recipe (the "how much") resides in an instance of an object type; in other
words, in a variable of that type.
There is a link to the VMT inside every object instance. The link is actually
the 16-bit offset of the VMT within the data segment. This link is shown in
Figure 1 as the arrow marked with Bullet 2. Because Turbo Pascal has only one
data segment, we can use a 16-bit offset rather than a full 32-bit segment:
offset address.
This link does not exist at compile time, but must be set up after the program
begins running. The setup is accomplished by a special kind of method called a
"constructor." Constructors were originally invented for C++, and are one of
numerous concepts borrowed from C++ by Borland for its object extension to
Turbo Pascal. Every object type that has virtual methods must also have a
constructor. And, even more important, every object instance's constructor
must be called before any virtual method in that instance can be called.
This is serious business. What the constructor does is connect an object
instance with its VMT, which contains the addresses of all the object's
virtual methods. Without that essential link between the instance and its VMT,
Turbo Pascal's late binding machinery will pull a nonsense address out of thin
air instead of from the VMT. If you try to call an object instance's virtual
method before calling that instance's constructor, execution will launch off
into nowhere, and your system will crash hard. At compile time, a call to an
object instance's virtual method is not given the address of the virtual
method. Instead, it is given the number of the VMT slot that contains the
address of the code implementing the virtual method. When the call is made,
Turbo Pascal finds the VMT through the link in the instance, and fetches the
address from the VMT slot specified in the code generated to implement the
call. Then an 8086 CALL is made to that address.
Voila! The two parts of the recipe are brought together, and the making of
silent explosives can begin. This time for sure, Rocky!


Family Resemblances


Now that's just how late binding works; we haven't quite gotten to applying it
to polymorphism yet. Polymorphism is the what, remember, and late binding the
how.
Polymorphism depends on a property of inheritance: That a pointer to a parent
type may also legally point to any child type of that parent type. Consider,
for example, the following object hierarchy:
 Mail
 _Reply
 _Image
 _Secret
If we define a pointer to type Mail, that pointer could legally point to any
of the others beneath Mail in the hierarchy. Similarly, a pointer defined as
pointing to Reply could also point to either Image or Secret, but not to Mail.
This extended assignment compatibility, like inheritance, moves only down the
object hierarchy, never up.
I've sketched out the VMTs of these four hypothetical object types in Figure
2. Keep Figure 2 handy during the following discussion.
Inheritance passes all data and methods defined within Mail down to all the
types descended from Mail. If Mail defines four methods, those four methods
are also present in Mail's child types. Now here's the critical fact: Any
method held in common by all four object types may be called through a pointer
to type Mail. That's polymorphism, whether the light has gone on in your head
yet or not. But it will, it will.
Suppose you have a linked list of nodes containing pointers to type Mail:
 TYPE
 MailPtr = ^Mail;
 NodePtr = ^Node;
 {Node in the linked list:}
 Node = RECORD
 Message: MailPtr;
 Next : NodePtr;
 END;
If any kind of electronic mail message is defined as a child object of type
Mail, then any message could be added to that linked list. Remember, a MailPtr
can point to type Mail or to any child type of Mail. To edit your mail, you
would simply traverse the linked list, and call the Edit method of each object
pointed to by the MailPtrpointer named Message:
 VAR
 Current, ListRoot : NodePtr;

 . . .

 Current := ListRoot;
 WHILE Current < > NIL DO
 BEGIN
 Current^.Message^.Edit;
 {Virtual method call:}
 Current := Current^.Next
 END;
For pointers pointing to Mail objects, Mail.Edit is executed. For pointers
pointing to Reply objects, Reply.Edit is executed. For pointers pointing to
Image objects, Image.Edit is executed, and so on. However, nowhere in the code
shown here is there any admittance at all that there is any type of object in
the list other than type Mail. Once a pointer to, say, a Reply object is
assigned to a MailPtr pointer, the source code treats the Reply object as
though it were a Mail object. Only through late binding does a call to the
Edit method get routed to the correct edit method -- that is, the Edit method
belonging to type Reply.
So there you have it: Polymorphism allows you to make one virtual method call
to several different types of object, and each object then responds as it
should, by using the method implementation written to serve its own needs.


Message Passing, Turbo-Style


It's often been said that Turbo Pascal 5.5 does not support message passing,
and in the strictest terms that's true. In true Smalltalk-style message
passing a message is sent to an object, and the object takes total
responsibility for calling the correct method in response to the message. In
Turbo Pascal, by contrast, we call the methods directly.
Well ... yes and no. Certainly we call static methods directly. However, when
we call a virtual method, we might think of the call as a message sent to
Turbo Pascal's late binding machinery: "I want to call the method accessed
through slot 4 of the virtual method table." The late binding machinery then
fetches the address from VMT slot 4 and makes the call. It's certainly message
passing stripped to the bone, but it's just indirect enough to qualify in my
book, and thinking of the VMT mechanism as a sort of message passing can be
helpful in understanding just what it is that the VMT does for a living.


Think of It as Polymorphism in Action



Coming up with a short, working, realistic example of polymorphism in action
is difficult. Demonstrating polymorphism requires multiple object types and
multiple method implementations, which can add up to a lot of code. Listing
One, page 154, is about as brief a demo of polymorphism as I could concoct
without moving to totally artificial situations; you know, one-line methods
that display "Hi boss! This is the Edit method!" on the screen when you
execute them. The FIELDS.PAS unit is conceptually similar to the FORMS.PAS
unit provided as a demo program with Turbo Pascal 5.5, but simpler and
shorter. FIELDTST.PAS (Listing Two, page 159) is a short program that
exercises the FIELDS.PAS unit.
FIELDS.PAS defines the short object hierarchy shown below:
 TextPosition
 _Field
 _BooleanField
 _TextField
 _IntField
Of the object types shown here, TextPosition and Field are abstract object
types, that is, types that exist only to be inherited from, or to act as heads
of polymorphic families. You never actually create instances of abstract
object types. Notice in Listing One that most of type Field's methods are
stubs that do nothing. The stubs are there so that Field's children can
override them with working substitutes. Stubs or not, slots exist in Field's
VMT for each of its virtual methods, and those slots are inherited by all of
Field's children.
A field is an object containing data and the means to display and edit that
data on the screen. Type Field is abstract, and contains no data of its own.
Each of its children represents a different kind of data: string data, Boolean
data, and integer data. Type Field and all its children have three virtual
methods: Show, Hide, and Edit. Show displays the field's data at its current
screen position, and Hide erases it from that position. Edit allows the user
to modify the data in an appropriate way.
Listing One was written to be blatantly obvious rather than clever. Notice
these things:
The Show, Hide, and Edit virtual methods for Field are stubs. They are there
only to reserve places in the VMT, so that the "real" versions belonging to
Field's child types can be invoked through a pointer to type Field.
Note the absence of method Intfield.Hide. Integers are edited on-screen as
strings, and erasing a string from the screen (which is all that it takes to
hide an IntField's data) is done the same way for any child type of TextField.
Therefore, IntField simply uses its parent's Hide method. No special
qualification is called for in Turbo Pascal. When IntField invokes a method
that it doesn't itself define, Turbo Pascal goes "up the tree" and uses the
first method it finds with the correct identifier, in this case
TextField.Hide.
Note that every constructor calls the constructor of its parent type before
doing anything else. This is a critical rule of "good style" in OOP: Let each
object type perform its own initialization. This isolates initialization code
where it has to happen, and doesn't force you to go searching for
initialization code outside of the object type when it comes time to change it
somehow.
Note that constructors are never virtual. A constructor call, because it sets
up that all-important link to the VMT, must be static because it cannot depend
on the VMT to get the constructor code's address!
With all of that in mind, I'll leave the rest of digesting Listing One up to
you. Listing Two is where the polymorphism happens, and it deserves a closer
look.


Objects Under Construction


Listing Two defines an array of four FieldPtr pointers, which point to type
Field. These pointers are given objects on the heap through calls to New, as
with any dynamically-allocated data item. The Turbo Pascal 5.5 twist is the
New syntax. New can now be used as a function, much the same as malloc in C:
<pointer var>:=New(<its pointer type>);
Note that this applies to all uses of New, not only those with OOP
connections. No, the OOP twist-within-a-twist is the invocation of
constructors from within the New call: <pointer to object> := New(<compatible
object type>, <constructor call>);
This syntax allows you to allocate and initialize an object with one
statement, and do it without having to use a temporary pointer variable to
hold the allocated object.
Note well that the type of the pointer on the left side of the assignment
statement only has to be compatible with (not identical to!) the pointer type
given in the New call:
 FieldArray[1]:= New(TextFieldPtr...
Here, FieldArray[I] is type FieldPtr, whereas New calls out a pointer of type
TextFieldPtr. Because TextField is a child of Field, their pointers are
compatible. New allocates an object of type TextField on the heap, while
returning a pointer of type Field to FieldArray[I]. Thus the New call is
setting up the pointers to allow polymorphism as well. That's a lot of mileage
from a single New call!
A reminder: You must call the constructor of each object instance before using
that instance. Turbo Pascal 5.5 includes an extension of range-checking to
help you during development. With the $R+compiler directive in force, a call
to a virtual method made from an uninitialized instance of an object generates
a range error rather than DOS McNuggets. As with range checking in general,
use it during development, then turn it off once you're satisfied things are
correct. (And, of course, when things blow up, the first thing to do is turn
the damned thing back on again and see how wrong you were!)


Freedom of Expression


Once you understand the amazing sophistication of the underlying mechanism,
the polymorphic calls themselves almost seem an afterthought:
 FOR I := 1 TO 4 DO
 FieldArray^.Show;

 FOR I := 1 TO 4 DO
 FieldArray^.Edit;
The first statement steps through the objects attached to the array and
displays their initial data on the screen through the Show virtual method. The
second steps through the array again, this time editing each object in turn
through the Edit virtual method.
The major advantage to polymorphism is that it encourages a unified high-level
way of looking at an application, or at least at families of objects. Dealing
polymorphically with objects emphasizes their similarities (that is, all
data-intensive objects need editing) without letting the details of their
differences get in the way of understanding and using them.
I'll be talking about polymorphism in many different contexts in future
columns. Digitalk has just released its Smalltalk/V PM for Presentation
Manager, and I'll (with some luck) get a chance to describe it in detail in my
next column. Conquering the complexity of PM's API is probably the most
important single job for polymorphism that I can think of right now, and at
first glance Smalltalk/V PM does it beautifully. Stay tuned.


Get-TUG-Gether '89


The Turbo User Group has quietly and with less notice than it deserves been
supporting users of Borland languages for almost five years. This year, Don
Taylor and the crew have raised their profile considerably by instituting
Get-TUG-Gether, a programmer's conference and party to equal anything I have
ever attended in that vein. The narrowness of the focus helps. I was not beset
by Unix evangelists speaking in their peculiar form of tongues ("awk! grep!
yacc!" a la Bill the Cat) nor pinstriped salesmen pushing mainframe computers
the size of small counties in Arkansas. Instead, we were a group of 100 or so
Turbo hackers trading insights and having a good time. Half a dozen Turbo
product vendors displayed and demonstrated their current releases, and I
signed books for Judy Overbeek's Rockland Press. Solid technical sessions and
small-group discussion sections rounded out the program. Remarkable (for
Seattle) weather, a great pool, and the spectacularly dedicated TUG staff made
it the best hacker gathering in recent memory.
There will be another Get-TUG-Gether next summer, from June 29 - July 1 and
Don has promised more technical tutorials, particularly for nonexperts. (I
have already committed to presenting "Pascal Pointers for Beginners" in
conjunction with a book I'll be writing on that subject.) It's cheap, it's
educational, and it feels good. Forget the invitation-only Elitist Hackers
conference in Silicon Valley; this is the future.


Products Mentioned


Turbo Pascal 5.5
Borland International
1800 Green Hills Road
Scotts Valley, CA 95066

408-438-8400
Standard package: $149.95
Professional package: $250

Smalltalk/V PM
Digitalk Inc.
9841 Airport Blvd.
Los Angeles, CA 90045
213-645-1082
$495

Turbo User Group (TUG)
TUG Lines newsletter
P.O. Box 1510
Poulsbo, WA 98370
BBS: 206-697-1151
Membership: $27 US;
$32 Canada/Mexico;
$39 elsewhere

_STRUCTURED PROGRAMMING_
by Jeff Duntemann


[LISTING ONE]

Unit Fields;

INTERFACE


USES Crt;

CONST
 IntChars = ['0'..'9','-'];
 TextChars = [#0..#255];
 Visible = True;
 Invisible = False;

TYPE
 String10 = String[10];
 String80 = String[80];
 CharSet = SET OF Char;

 PositionPtr = ^TextPosition;
 TextPosition = { ABSTRACT! }
 OBJECT
 X,Y : Integer; { Coordinates of location on the screen }
 CONSTRUCTOR Init(InitX,InitY : Integer);
 FUNCTION XPos : Integer;
 FUNCTION YPos : Integer;
 END;

 FieldPtr = ^Field;
 Field = { ABSTRACT! }
 OBJECT(TextPosition)
 VisibleState : Boolean; { True = Field is displayed }
 CONSTRUCTOR Init(InitX,InitY : Integer;
 InitVisible : Boolean);

 FUNCTION IsVisible : Boolean;
 PROCEDURE MoveTo(NewX,NewY : Integer);
 PROCEDURE Show; VIRTUAL;
 PROCEDURE Hide; VIRTUAL;
 PROCEDURE Edit; VIRTUAL;
 END;

 TextFieldPtr = ^TextField;
 TextField = { For ordinary text strings }
 OBJECT(Field)
 StringData : String80;
 FieldLength : Integer;
 CONSTRUCTOR Init(InitX,InitY : Integer;
 InitVisible : Boolean;
 InitText : String80;
 InitLength : Integer);
 FUNCTION GetData : String80;
 PROCEDURE Show; VIRTUAL;
 PROCEDURE Hide; VIRTUAL;
 PROCEDURE Edit; VIRTUAL;
 END;

 BooleanFieldPtr = ^BooleanField;
 BooleanField =
 OBJECT(Field)
 Toggle : Boolean;
 TrueString,FalseString : String80;
 CONSTRUCTOR Init(InitX,InitY : Integer;
 InitVisible : Boolean;
 InitToggle : Boolean;
 InitTrueStr,
 InitFalseStr : String80);
 FUNCTION Getdata : Boolean;
 PROCEDURE Show; VIRTUAL;
 PROCEDURE Hide; VIRTUAL;
 PROCEDURE Edit; VIRTUAL;
 END;

 IntFieldPtr = ^IntField;
 IntField =
 OBJECT(TextField)
 IntVal : Integer;
 CONSTRUCTOR Init(InitX,InitY : Integer;
 InitVisible : Boolean;
 InitIntVal : Integer);
 FUNCTION GetData : Integer;
 PROCEDURE Show; VIRTUAL;
 PROCEDURE Edit; VIRTUAL;
 END;


IMPLEMENTATION


VAR
 Blanker : String80;




FUNCTION MaxLength(String1,String2 : String) : Integer;

BEGIN
 IF Length(String1) > Length(String2) THEN
 MaxLength := Length(String1)
 ELSE
 MaxLength := Length(String2);
END;


PROCEDURE ShowBlanks(NumberOfBlanks : Integer);

BEGIN
 Write(Copy(Blanker,1,NumberOfBlanks));
END;


PROCEDURE HighLight(X,Y,TargetLength : Integer; TargetText : String);

BEGIN
 GotoXY(X,Y); ShowBlanks(TargetLength);
 GotoXY(X,Y); Write(TargetText);
END;


PROCEDURE UhUh;

BEGIN
 Sound(35); { Make first grunt }
 Delay(100);
 NoSound;
 Delay(50); { Delay between grunts }
 Sound(35); { Make second grunt }
 Delay(100);
 NoSound;
 Delay(50); { Delay after second grunt }
END;




PROCEDURE GetLine(X,Y : Integer;
 VAR MyLine : String80;
 MaxWidth : Integer;
 LegalChars : CharSet);

VAR
 Ch : Char;
 Quit,Done : Boolean;
 TempLine : String;
 WorkPoint : Integer;


 PROCEDURE DisplayLine;

 BEGIN
 GotoXY(X,Y);
 Write(TempLine);
 END;



BEGIN
 Quit := False; Done := False;
 TempLine := MyLine;
 DisplayLine;
 REPEAT
 IF KeyPressed THEN
 BEGIN
 WorkPoint := (WhereX-X) + 1;
 Ch := ReadKey;
 CASE Ord(Ch) OF
 0 : BEGIN { If the first char is 0, there's more... }
 Ch := ReadKey; { Get the second portion }
 CASE Ord(Ch) OF
 71 : GotoXY(X,Y); { Home }
 79 : GotoXY(X + Length(TempLine),Y);

 75 : IF WorkPoint <= 1 THEN Uhuh { Left Arrow }
 ELSE
 BEGIN
 Dec(WorkPoint);
 GotoXY(X+WorkPoint-1,Y);
 END;

 83 : BEGIN { Del }
 Delete(TempLine,WorkPoint,1);
 DisplayLine;
 Write(' ');
 GotoXY(X+WorkPoint-1,Y);
 END;

 END { case }
 END;
 8 : IF WorkPoint <= 1 THEN Uhuh
 ELSE
 BEGIN
 Dec(WorkPoint); { Move left one position }
 Delete(TempLine,WorkPoint,1); { Delete a char in string }
 DisplayLine; { Re-display the string }
 Write(' '); { Erase the last char }
 GotoXY(X+WorkPoint-1,Y); { And put the cursor back }
 END; { to the correct position }
 13 : Done := True; { Enter }
 27 : Quit := True; { Esc }

 32..254 : IF Ch IN LegalChars THEN
 IF Length(TempLine) >= MaxWidth THEN UhUh
 ELSE
 BEGIN
 Insert(Ch,TempLine,WorkPoint);
 DisplayLine;
 GotoXY(X+WorkPoint,Y);
 END
 ELSE Uhuh;
 END { case }
 END;
 UNTIL Done OR Quit;
 IF Done THEN MyLine := TempLine;

END;




{------------------------------------------------------------------}
{ All of the following routines are method implementations }
{------------------------------------------------------------------}


CONSTRUCTOR TextPosition.Init(InitX,InitY : Integer);

BEGIN
 X := InitX; Y := InitY;
END;


FUNCTION TextPosition.XPos : Integer;

BEGIN
 XPos := X;
END;


FUNCTION TextPosition.YPos : Integer;

BEGIN
 YPos := Y;
END;


CONSTRUCTOR Field.Init(InitX,InitY : Integer;
 InitVisible : Boolean);

BEGIN
 TextPosition.Init(InitX,InitY);
 VisibleState := InitVisible;
END;


FUNCTION Field.IsVisible : Boolean;

BEGIN
 IsVisible := VisibleState;
END;


PROCEDURE Field.MoveTo(NewX,NewY : Integer);

BEGIN
 IF IsVisible THEN Hide;
 X := NewX;
 Y := NewY;
 IF IsVisible THEN Show;
END;


PROCEDURE Field.Show;


BEGIN
END;


PROCEDURE Field.Hide;

BEGIN
END;


PROCEDURE Field.Edit;

BEGIN
END;


CONSTRUCTOR TextField.Init(InitX,InitY : Integer;
 InitVisible : Boolean;
 InitText : String80;
 InitLength : Integer);

BEGIN
 Field.Init(InitX,InitY,InitVisible);
 StringData := InitText;
 FieldLength := InitLength;
 IF InitVisible THEN Show;
END;


FUNCTION TextField.Getdata : String80;

BEGIN
 Getdata := StringData;
END;


PROCEDURE TextField.Show;

BEGIN
 GotoXY(XPos,YPos);
 Write(StringData);
 VisibleState := True;
END;


PROCEDURE TextField.Hide;

BEGIN
 GotoXY(XPos,YPos);
 ShowBlanks(FieldLength);
 VisibleState := False;
END;


PROCEDURE TextField.Edit;

VAR
 AttributeStash : Byte;


BEGIN
 IF IsVisible THEN
 BEGIN
 AttributeStash := TextAttr;
 TextAttr := $70;
 HighLight(XPos,YPos,FieldLength,StringData);
 GetLine(XPos,YPos,StringData,FieldLength,TextChars);
 TextAttr := AttributeStash;
 HighLight(XPos,YPos,FieldLength,StringData);
 END;
END;


CONSTRUCTOR BooleanField.Init(InitX,InitY : Integer;
 InitVisible : Boolean;
 InitToggle : Boolean;
 InitTrueStr,
 InitFalseStr : String80);

BEGIN
 Field.Init(InitX,InitY,InitVisible);
 Toggle := InitToggle;
 TrueString := InitTrueStr;
 FalseString := InitFalseStr;
 IF InitVisible THEN Show;
END;


FUNCTION BooleanField.Getdata : Boolean;

BEGIN
 Getdata := Toggle;
END;


PROCEDURE BooleanField.Show;

BEGIN
 GotoXY(XPos,YPos);
 IF Toggle THEN Write(TrueString)
 ELSE Write(FalseString);
 VisibleState := True;
END;


PROCEDURE BooleanField.Hide;

BEGIN
 GotoXY(XPos,YPos);
 IF Toggle THEN ShowBlanks(Length(TrueString))
 ELSE ShowBlanks(Length(FalseString));
 VisibleState := False;
END;


PROCEDURE BooleanField.Edit;

VAR
 Ch : Char;

 Done,Quit : Boolean;
 SaveState : Boolean;
 AttributeStash : Byte;

BEGIN
 IF IsVisible THEN { Only edit if it's visible... }
 BEGIN
 SaveState := Toggle; Done := False; Quit := False;
 AttributeStash := TextAttr; TextAttr := $70;
 HighLight(XPos,YPos,MaxLength(TrueString,FalseString),'');
 Show;
 REPEAT
 IF KeyPressed THEN { If there's a keystroke waiting }
 BEGIN
 Ch := ReadKey; { go get it... }
 CASE Ord(Ch) OF { and parse it. }
 0 : Ch := ReadKey; { Get second half of extended char; ignore it }
 13 : Done := True; { Enter means accept current state of Toggle }
 27 : Quit := True; { Esc means restore Toggle as it was on entry }
 ELSE BEGIN { Another other ASCII key: Flip Toggle }
 Hide; { Erase the current state string }
 Toggle := NOT Toggle; { Flip Toggle to its opposite state }
 Show; { Display the alternate state string }
 END;
 END; { CASE }
 END;
 UNTIL Done OR Quit;
 IF Quit THEN
 BEGIN
 Hide; { Erase current display of state string }
 Toggle := SaveState; { Restore original state of Toggle }
 Show; { And re-display it }
 END;
 TextAttr := AttributeStash;
 HighLight(XPos,YPos,MaxLength(TrueString,FalseString),'');
 Show;
 END;
END;



CONSTRUCTOR IntField.Init(InitX,InitY : Integer;
 InitVisible : Boolean;
 InitIntVal : Integer);

VAR
 WorkString : String10;

BEGIN
 Str(InitIntVal : 6,WorkString);
 TextField.Init(InitX,InitY,InitVisible,WorkString,6);
 IntVal := InitIntVal;
 IF InitVisible THEN Show;
END;


FUNCTION IntField.Getdata : Integer;

BEGIN

 Getdata := IntVal;
END;


PROCEDURE IntField.Show;

BEGIN
 Str(IntVal : 6,Stringdata);
 TextField.Show;
END;

{-------------------------------------------------------------------}
{ Notice that there is NO IntField.Hide! The mechanism for erasing }
{ an integer field is no different from erasing any string field, }
{ so objects of type IntField use the Hide method inherited from }
{ TextField. }
{-------------------------------------------------------------------}


PROCEDURE IntField.Edit;

VAR
 WorkValue,ErrorPos : Integer;
 AttributeStash : Byte;

BEGIN
 IF IsVisible THEN { Only edit an object if it's visible... }
 BEGIN
 AttributeStash := TextAttr;
 TextAttr := $70;
 Str(IntVal : 6,StringData); { Convert the integer value to a string }
 HighLight(XPos,YPos,Length(StringData),Stringdata);
 REPEAT { And edit the string until it's right }
 GetLine(XPos,YPos,StringData,FieldLength,IntChars);
 Val(Stringdata,WorkValue,ErrorPos);
 IF ErrorPos <> 0 THEN Uhuh;
 UNTIL ErrorPos = 0;
 IntVal := WorkValue;
 TextAttr := AttributeStash;
 HighLight(XPos,YPos,Length(StringData),StringData);
 END;
END;



BEGIN
 FillChar(Blanker,SizeOf(Blanker),' ');
 Blanker[0] := Chr(80);
END.





[LISTING TWO]

PROGRAM FieldTest;

USES Crt,

 Fields; { Published in DDJ November 1989 }

CONST
 Female = True;
 Male = NOT Female;

VAR
 FieldArray : ARRAY[1..4] OF FieldPtr;
 I : Integer;

BEGIN
 ClrScr;
 Writeln('Patient name: ');
 Writeln(' sex: ');
 Writeln(' age: ');
 Writeln(' Physician: ');

 { Initialize the objects on the heap & provide initial values: }
 FieldArray[1] := New(TextFieldPtr,Init(15,1,Invisible,'Jones,Tom',40));
 FieldArray[2] := New(BooleanFieldPtr,Init(15,2,Invisible,
 Female,'Female','Male'));
 FieldArray[3] := New(IntFieldPtr,Init(15,3,Invisible,42));
 FieldArray[4] := New(TextFieldPtr,Init(15,4,Invisible,'Dr. Asimov',40));

 { First display initial values through polymorphic calls to Show: }
 FOR I := 1 TO 4 DO FieldArray[I]^.Show;

 { Now edit each one through a polymorphic call to the Edit method: }
 FOR I := 1 TO 4 DO FieldArray[I]^.Edit;
END.
































November, 1989
OF INTEREST





IEEE Software magazine has announced a December 31 deadline for the third
annual Gordon Bell Prize for outstanding achievements in the application of
parallel processing to scientific and engineering problems.
The prize is sponsored by Gordon Bell, vice president of engineering at Ardent
Computer Systems; the judging is administered by IEEE Software, which is
published by the IEEE Computer Society, the international professional
organization of the Institute of Electrical and Electronics Engineers Inc.
Two $1000 prizes will be awarded to the two best entries of three categories.
In the performance category, the submitted program must run faster than any
other comparable engineering or scientific application. Judges will consider
the megaflop rate based on actual operation counts or the solution of the same
problem with properly tuned code on a machine of known performance as suitable
evidence.
In the price/performance category, the entrant must show that the application
divided by the cost of the system is better than any other entry. Performance
measurement is the same for the performance category. Price will be the list
price of the computational engine (CPUs, including adequate memory to run the
program). Peripherals and software need not be included in the price. Entrants
must submit the output from a Linpack 100 x 100 benchmark showing a speedup of
at least 5 megaflops.
The compiler parallelization category is for the compiler/application that
generates the most speedup, which will be measured by the execution time
without automatic parallelization divided by the execution with automatic
parallelization. A third run that uses compiler directives to improve
parallelization may also be submitted. For entry information, contact:
1989 Gordon Bell Prize
c/o IEEE Software
10662 Los Vaqueros Cir.
Los Alamitos, CA 90720-2578
TurboPower has released Object Professional 1.0, a library of object-oriented
routines that enhances the productivity of OOP in Turbo Pascal 5.5 by
providing more than 30 high-level object types with more than 1000 methods and
routines in the categories of user-interface design, data manipulation, and
low-level system access. Object Professional takes advantage of all of Turbo
Pascal 5.5's advanced object-oriented features, including constructors,
destructors, and static methods.
Object Professional allows interfaces ranging from text-mode Presentation
Manager look-alikes, to Lotus-style menu bars, to your own look and feel
design. The window object is the root of the hierarchy, and supports
overlapping, resizeable text windows with support for scroll bars, and other
mouse operations. Additional object types inherit the window's functionality,
and offer data entry fields, full-screen forms, pull-down and horizontal-bar
menu systems, pick lists, file selection boxes, and more. Generalized
capabilities for stacks, linked lists, and virtual arrays are also offered.
Object Professional's system-oriented routines provide capabilities like TSR
management, interrupt service routines, EMS and extended memory management,
enhanced keyboard support, BCD arithmetic, and others. The TSR manager is new,
and swaps to disk or EMS, squeezing complex pop-up programs into as little as
5K of normal RAM space. Although we haven't had a chance to fiddle with Object
Professional yet, TurboPower has a reputation for good products, good prices,
and good people. We'll be reviewing Object Professional before long to see if
this trend continues with this latest product.
Object Professional 1.0 is based on the Turbo Professional 5.0 toolkit, which
TurboPower will continue to sell and support, and which compiles and runs
without modification under Turbo Pascal 5.5. Object Professional sells for
$150, and includes complete source code, documentation, and a royalty-free
license to develop commercial applications.
TurboPower Software P.O.
Box 66747
Scotts Valley, CA 95066-0747
408-438-8608
Quarterdeck Office System has announced QRAM, a tool for organizing and
allocating memory resources for 8088/86-based PCs and AT-style 286 PCs. QRAM
gives XT- and AT-class PCs with EMS 4.0 or EEMS memory the capacity to load
programs into high memory (between 640K and 1024K), previously unavailable to
DOS. The ability to load LAN drivers, TSRs, and mouse drivers into high memory
allows conventional DOS memory to be used by applications. QRAM is the only
product available that adds load-high capability on top of any EMS 4.0 or EEMS
driver. It costs $59.95. Reader service no. 21.
Quarterdeck Office Systems 150 Pico Blvd. Santa Monica, CA 90405 213-392-9851
A set of productivity tools for Turbo C and Turbo Pascal programmers has been
released by Mostly Mice Software Inc. The menuing and mouse driver tools,
which are patches to the Turbo programming environment, make all Turbo C 2.0
and Turbo Pascal 5.5 commands and functions available through pull-down menus.
Programmers can then point-and-shoot at commands, then drop them into a
program.
Additionally, the tools allow you to highlight blocks of text from within an
Edit window by dragging the mouse, to switch between Edit and Watch windows by
pointing-and-shooting, and to work in either 25 or 43/50 line mode. The
drivers work within DESQview and can be used with all Microsoft-compatible
mice.
The Turbo drivers, like other Mostly Mice products (menuing and mouse drivers
for Lotus, Wordstar, WordPerfect, etc.), are developed by independent
programmers around the country who work on a royalty basis with Mostly Mice.
Company president Simeon Berman told DDJ, "I'm interested in evaluating other
people's work for our family of products, not just mouse drivers."
The Turbo C and Turbo Pascal Menuing and Mouse Drivers (both include a KEY.COM
keyboard analyzer) sell for $29.95 each. Reader service no. 22.
Mostly Mice Software, Inc.
125 Gates Ave.
Montclair, NJ 07042
201-746-9256 for technical information
800-283-4080 for orders
Digitalk Inc. is now shipping Smalltalk/V PM, the first programming
environment for OS/2 Presentation Manager. An object-oriented environment that
is fully integrated with PM, Smalltalk/V PM can be used to prototype and
deliver user-interface intensive applications, such as database front ends,
CASE tools, and financial models.
Jim Anderson, president of Digitalk, claims that the technology behind this
product "prevents PM from becoming a massive knot of complications."
The first and only fully compiled Smalltalk, this product gives developers a
responsive environment and access to PM features. Smalltalk/V PM allows the
programmer to take advantage of the unique benefits of PM without sacrificing
performance.
The source code is compatible with Smalltalk/V 286 and Smalltalk/V Mac
products, which allows portability of applications between different
environments. The developer can use browsers to navigate through code and
explore PM. Digitalk claims that the system is crash proof, which encourages
an experimental style of programming. It sells for $499.95, and includes a
user manual and tutorial for learning object-oriented programming. Jeff
Duntemann plans on looking further into this product in his December 1989
"Structured Programming" column. Reader service no. 23.
Digitalk Inc.
9841 Airport Blvd.
Los Angeles, CA 90045
213-645-1082
The MasPar Computer Corporation's family of computers has, in addition to the
standard Unix environment, a graphical, interactive programming environment
called the MasPar Programming Environment (MPPE). A versatile set of tools
that enables users to apply data parallel programming algorithm techniques to
large numbers of processors, the MPPE helps overcome design barriers that
hamper application development in other multiple processor systems. The MPPE
is designed to operate over network-based systems using the X-Window system
protocol layer, and permits programmers to develop and debug applications
using the windowing capability of a workstation.
The key tool is an "on-demand" source-level debugger. Others include browsers,
animators, and visualizers, and are accessed by a graphical user interface.
Designed for programming, adaptation, and debugging of
computationally-intensive applications, these tools supposedly bring a new
level of programmer productivity to high-end computing.
The MPPE provides program development, check-out, debug, and animation
facilities that are geared to the task of data parallel programming. An
integrated editor, data display facility, and symbolic debugger allow a
programmer to move freely between execution, check out, what-if analysis, and
code modification. A browse capability immediately answers questions such as
"where is this subroutine called?" and "where is this block COMMON
referenced?"
With the MPPE a programmer can see how code actually executes in a variety of
situations, including hardware utilization for optimization and tuning, and
aggregation of individual data cells into high-level abstractions. MPPE is
available for all MasPar-supported languages. Reader service no. 34.
MasPar Computer Corporation
749 North Mary Avenue
Sunnyvale, CA 94086
408-736-3300
The Whitewater Group has released the Whitewater Resource Toolkit for
Microsoft Windows, which allows programmers to manage a Windows application's
look and feel, and to create Windows resources such as dialog boxes, bitmaps,
cursors, icons, and pull-down menus. Programmers can also create keyboard
accelerators and string resources.
According to Whitewater's Hope Gillespie, "One of the major difficulties
facing the Windows programmer using conventional tools is the time-consuming
edit-compile-link development cycle. The Whitewater Resource Toolkit's
interactivity allows programmers to immediately see the results of their
programming from within Windows, greatly increasing their productivity."
The toolkit is a stand-alone application that complements, but does not
require, the Microsoft Windows Software Development Kit (SDK) and its resource
compiler (RC). It is compatible with all Windows resource file formats, and
can generate RC-compatible script and bitmap files. It has bitmap, cursor,
icon, menu, dialog, string table, and accelerator tables editors.
Completely written in Actor, the toolkit is geared toward C programmers,
Microsoft SDK users, and Actor programmers. It requires Microsoft Windows and
will run on the IBM PC/AT, PS/2, and compatibles with 1 Mbyte of RAM, a hard
disk, graphics display of EGA or better and adapter, mouse, and DOS 2.0 or
higher. It includes full support for the Lotus/Intel/Microsoft Expanded Memory
Specification EMS 3.2, and sells for $195.
At the same time, The Whitewater Group announced WinTrieve, a Microsoft
windows-based file indexed manager, which is a comprehensive set of tools for
building custom data management procedures into Microsoft Windows
applications. Zack Urlocker, manager of developer relations, claims it "allows
programmers to create Windows applications that access database information
faster and with less programming than ever before."
WinTrieve has an ISAM (indexed sequential access method) server, a C
application programming interface (API) library, and an Actor class library.
The ISAM server provides a method for storing and retrieving data based on
indexes, and includes random and sequential access, concurrent multiple file
access, automatic updating of indexes, file and record level locking, multiple
indexes, journaling across multiple files, and transaction commit and
rollback.
The C API library has C functions that can access the ISAM server, and
effectively hides the ISAM server while minimizing responsibility for session
initiation and termination.

WinTrieve requires Microsoft Windows and the IBM XT, AT, PS/2, and
compatibles, a hard disk, graphics display and adapter, and DOS 2.0 or higher.
It uses 90K of memory running as a separate application, and includes full
support for the Lotus/Intel/Microsoft Expanded Memory Specification EMS 3.2.
When used with C, it requires 640K, and with Actor, 1 Mbyte of RAM. It sells
for $395. Reader service no. 24.
The Whitewater Group
600 Davis St.
Evanston, IL 60201
312-328-3800
800-869-1144
Oasys Inc. has announced a Green Hills C++ compiler which, claims the company,
is the only C++ compiler that supports cross- and native-mode development. The
compiler supports K&R C and is ANSI C compliant.
In addition to providing OOP features such as data abstraction, type checking,
and overloading of function names and operators, the Green Hills C++ compiler
also provides classes with scope, and overloading new and arrow operators.
Green Hills has also built its standard optimizing techniques, such as
inlining, loop unrolling, and register caching, into the compiler.
Initially, the system will run on Sun-3 systems, though Oasys says that it
will be ported to UNIX workstation and minicomputer platforms in the near
future. Reader service no. 25.
Oasys Inc.
230 Second Ave.
Waltham, MA 02154
617-890-7889

















































November, 1989
SWAINE'S FLAMES


More on the Numbers Game




Michael Swaine


When HyperCard first appeared and its supporters proclaimed that user
programming had come to the Mac, I was among 'em. We weren't wrong: There are
a lot of ordinary folks using HyperCard and its scripting language, HyperTalk;
but HyperCard's lack of integration with anything else (it's not part of the
system, it doesn't interact with other applications in any interesting way,
and it can't run under MultiFinder effectively on mere mortals' machines) all
limit its value as a user programming tool.
As it turns out, HyperCard's greatest contribution to user programming could
turn out to have nothing to do with HyperTalk or stacks. At MacWorld Expo in
Boston I spent some time talking with developers from Software Ventures,
publishers of Microphone, and Informix, publishers of Wingz. Both companies
are making extensive use of XCMDs or XFCNs, the external commands or functions
first developed to allow developers to extend HyperTalk.
Microphone III supports, in its scripting language, virtually all the external
command functionality of HyperCard. XCMDs written for HyperCard stacks can be
plugged into Microphone directly, without modification. Wingz takes a
different approach, using only XFCNs to extend its HyperScript language (I saw
an SQL interface made up of XFCNs). This represents not so much a user
programming approach as evidence that XFCNs are becoming a kind of lingua
franca for Macintosh programming. Perhaps the packaging for the version of
HyperCard, when it finally appears, will proclaim that HyperCard also supports
XCMDs and XFCNs.
"Statistical thinking," H.G. Wells once wrote, "will one day be as necessary
for efficient citizenship as the ability to read and write." That day has
dawned on an American citizenry -- both illiterate and innumerate.
I have mentioned here the important book Innumeracy, by John Allen Paulos.
Paulos describes the innumeracy problem more clearly than anyone else I've
encountered, with many appropriate and entertaining examples of innumeracy in
the fourth estate.
I recently came across a couple of instances of innumeracy that show how
important it is to question the numbers that are presented to you every day,
and how often you can reject them on the basis of simple plausibility checks.
In a New York Times magazine piece on Steve Jobs last August, the writer said
that Jobs often overprojected sales of the Macintosh during its development
phase, sometimes by as much as 90 percent. The implication was that Jobs could
be wildly optimistic about the Mac, and independent evidence certainly
supports that implication. I haven't checked the 90 percent figure with Apple
because I'm only interested here in plausibility, but is it plausible? Is a 90
percent sales overprojection on a product as innovative as the Mac evidence of
wild enthusiasm? I don't think so, especially because the 90 percent figure is
given as the worst overprojection Jobs made. Don't you suspect that actual
sales for the first year were x thousand units, and that Jobs had once
projected sales of 10 x thousand units? That would be an overprojection of 900
percent, a figure consistent with Jobs' evangelical enthusiasm for the Mac. I
may be misjudging the Times writer, but this is exactly the sort of confusion
about percentages that often appears in newspapers.
A second implausible use of numbers appeared in Paulos's own book. Paulos
describes a safety index, a measure of risk that he would like to see attached
to risky activities when they are discussed in the press. His safety index is
the base-10 logarithm of the number of deaths (or other appropriate risky
outcome) per year attributed to the activity. Inasmuch as one person in 5300
dies each year in the U.S. from a car crash, Paulos derives a safety index for
driving (actually, for being in a moving vehicle) of log 5300, or 3.7. He
similarly derives a safety index of 2.9 for smoking, so smoking has a lower
safety index, and is less safe than riding in a car. Whether your smoking is a
greater risk to your health than your driving would depend on other factors,
such as how much you do of each, and under what conditions, but the index does
seem to provide some useful information. (In practice, you'd probably want to
use log(1 + incidence) to avoid negative indices.)
In his discussion of the index, Paulos makes the interesting assertion:
"[M]alaria's index is orders of magnitude lower in most of the world than it
is in the United States." Let's examine the plausibility of that claim. It's
clear from reading the rest of the book that when Paulos uses the expression
"order of magnitude," he means a power of 10. "Orders of magnitude," then,
means at least a power of 100. For two safety indices to differ by a power of
100, the associated incidences of malaria deaths must differ by a factor of
10{100}. This (literally astronomical) number does not look to me like a
figure that could arise from any conceivable empirical data.








































December, 1989
December, 1989
EDITORIAL


Hindsight and The Crystal Ball




Michael Floyd


Over the past few months, Mike Floyd has immersed himself in object-oriented
programming in its various guises, becoming our in-house expert on the subject
in the process. Because this month's theme is object-oriented programming, it
seems natural that Mike shares some of what he sees as the promise and the
pitfalls of OOP. -- J.E.
Earlier this year, just prior to the release of Turbo Pascal 5.5, we asked
Anders Hejlsberg, chief architect of Turbo Pascal, if Borland was going to
follow the path of C++ and distinguish Object Pascal from structured Pascal:
Was Borland adding to the product line with TOP (Turbo Object Pascal)? Anders'
reply made perfect sense. "Adding objects to Turbo Pascal is simply the next
step in the evolution of the language," he said. "From this moment on, Turbo
Pascal is an object-oriented language."
So, is this what we can expect to see in terms of programming and programming
languages over the coming decade? If not, what does the future of programming
-- object-oriented or otherwise -- hold? (Trying to answer questions such as
these is much like driving down the highway with your eyes fixed on the
rearview mirror -- you spend more time worrying about where you've been
without having a clue as to where you're going.)
Other questions come to mind too. Will C merge with C++ and become D? Although
this seems a natural next step for C, there are many pressures that may keep
it from happening, particularly the challenge of getting a standard through
ANSI.
One big question is,"Where will the next set of development tools come from?"
Programmers who have grown used to having sophisticated tools aren't willing
to return to the dark ages that existed before development environments,
debugging facilities, and library support. But for a number of reasons, OOP
tools have been relatively slow forthcoming. Until these tools begin to
emerge, many programmers will be reluctant to make the move to OOP.
Traditionally, third-party vendors have filled the gap, but the lack of a
standard or dominant environment and the past experience of putting out tools
too early has made third-party developers gun shy. Indeed, market pressure
from users and competing vendors may force some of the big guys to continually
pack their class libraries with more bang for the buck, thereby limiting
market opportunities for smaller tool developers even more.
Some of the burden falls on the shoulders of the individual programmer. After
all, an object-oriented system is extensible. In Smalltalk, you don't write
source code, you add components to the system. Likewise, languages such as C++
and Object Pascal encourage the programmer to weld homegrown components with
canned objects to create new tools for the workbench. For the optimistic
programmer, this is a time of Renaissance; for others, a time of chaos.
For those of us at DDJ, this month's issue is not just the last one of the
year, but the last of the decade. It may be appropriate, then, that
object-oriented programming is the theme that takes the next decade into the
next century.
But, lest you think DDJ is adding to the object-oriented hype (OOH?), think
again. One size fits all? We're not so sure. You may want to pay particular
attention to Scott Guthery as he tries on the "Emperor's New Clothes." And OOP
may not be the only bully on the block either, at least according to Ronald
Fischer, who believes that Functional Programming may influence the way we
program over the next decade. Ah, well. If nothing else, it'll be interesting
to see what unravels in the rearview mirror over the next ten years.








































December, 1989
LETTERS







Striding Forth With Mini-Interpreters


Dear DDJ,
"Roll your own Mini-Interpreters," by Michael Abrash and Dan Illowsky in the
September DDJ was fun to read. What they have done, essentially, is show how
to make a Forth-like program without even Forth's overhead. Forth is legendary
for producing extremely compact code within the Forth environment. The authors
went them one better by eliminating even that.
What the article doesn't mention, is that there is nothing to prevent
high-level language compilers from emitting this type of threaded code. All
that is required is to forego separate compilation and process the entire
application, including all subprograms and perhaps even the run-time library,
via a globally optimizing compiler (GOC).
A GOC knows the list of all subprograms and the entire calling tree. It can
automatically build one or more jump vector tables, and compose the
interpreted data streams. Because of the low overhead of procedure calls, even
tiny sequences such as A=B+C could be generated as threaded calls rather than
as direct machine code. Thus, subprogram granularity much finer than the
application code is possible, and the compiler would be able to trade off
code-size versus execution-speed over an extraordinary range.
Recent press reports show that even the giants like Lotus and Ashton-Tate are
killing themselves trying to make their code fit the 640K limits of DOS.
Assuming that others have similar problems, there could be quite a market for
GOCs. Attention compiler vendors! Here's a chance to make more money.
Of course, global optimization has its cost; namely glacial compilation speed.
The right way to use it would be to develop the application on a 386 in
protected mode, using fast compilers and whatever memory is needed. It should
remain in this environment even through beta testing, until the code can be
frozen. Then, assuming a bulletproof and bug free GOC, it can be compiled once
more with global optimization. The compiler must be instructed about just how
compact the program must be. For example, "take this program, which needs 843K
for code using MSC, and compile it to fit in no more than 403K while
minimizing the impact on speed." Who cares if it takes a week or so to finish?
The GOC could even be run as a service bureau, compiling on a Cray, and
charging as much as one dollar per line for a compile. If it works, it would
be well worth it.
Dick Mills
Burlington, Vermont


Where There's Smoke, There's Ire


Dear DDJ,
The article in the September 1989 DDJ entitled "Roll Your Own Minilanguages
with Mini-Interpreters" by Abrash and Illowsky offended me. No, not by the
content; by the title. No, "roll your own" is not just an expression.
I hold cigarette smokers (and smokers of anything else) in very low regard.
Next time, try some synonyms like "create," "devise," or "construct" that have
far fewer connotations.
Bob Bryla
Barrington, Illinois


More Kudos for Abrash and Illowsky


Dear DDJ,
A few remarks on a couple of subjects. One: Since the word polymorphism is
being bandied about a lot these days, the following, "loosely typed"
definition may prove interesting.
Polymorphism is a polysyllabic noun used to "encapsulate" the idea that: 1) to
treat "loosely coupled" items, you use "loosely typed" variables; and 2)
"loosely typed" variables can't be bound until run-time, an activity often
referred to by way of another buzzword, "late-binding."
Two: Several hips and half as many hoorays for the Abrash and Illowsky article
in the September issue. The most exciting thing about assembly language is not
the control it gives one over the machine, nor the reduction in memory
requirements and execution time. As Abrash and Illowsky point out, the most
exciting thing about assembly language is that it gives you as much control
over the design of your program logic and data layout as you're ever likely to
get.
To which I would like to add: If you want even more control, you need your own
assembler, perhaps your own editor, and even your own operating system -- only
for your own private use, of course (that way there's no effective,
market-oriented argument against actually tackling the job). As a fantastic
learning experience, working on any of them would be hard to beat. Let's hope
DDJ has more .ASM goodies in the queue.
Mark Rhyner
Chicago, Illinois


LZW Patent Issues


Dear DDJ,
"LZW Data Compression," by Mark Nelson (DDJ, October 1989) is a nice
exposition on the LZW algorithm. But before your readers decide to use this
method in any application (except perhaps for purely personal use), they
should know that the algorithm is patented.
Terry Welch is listed as the inventor of U.S. Patent 4,558,302, "High Speed
Data Compression and Decompression Apparatus and Method," December 1985,
assigned to Sperry Corporation (now Unisys). The Unix compress utility and
several commercial and shareware programs are apparently infringing on this
patent (unless they have licensed it from Unisys).
If you wish to use this method in a commercial setting, you should contact
Unisys for a license, or at least consult your legal counsel first.
Ray Gardner
Englewood, Colorado
Mark responds: When I wrote the LZW article I was unaware of any patent on the
algorithm. The issue has just surfaced in the press because of concern in
CCITT Group 7 over approval of the BTLZ algorithm for data compression in the
V.42bis modem standard. Unisys, British Telecom, and IBM apparently all have
some claim on the algorithm. Robert Bramson, a patent attorney for Unisys, has
been quoted as saying they will license the algorithm for a one-time fee of
$20,000.
I have not seen the Unisys patent, so I don't know what their specific claims
are. However, I am not aware of any attempt by Unisys to show infringement by
software developers. The BTLZ algorithm seems to be concerned with hardware
implementations. In the event that they do pursue their claim with software
developers, they will be very busy, as there are literally hundreds of
potentially infringing programs in the commercial marketplace alone. And they
certainly cannot claim a comprehensive patent on basic LZ compression, as
Terry Welch, the patent holder, was not the inventor.
I agree with Mr. Gardner that anyone who intends to use LZW compression in a
commercial product would be wise to consult legal counsel first.
Finally, I would like to suggest that DDJ readers begin a letter-writing
campaign directed to members of Congress, the ACM, and the IEEE. The current
confusion over copyright and patent issues in the software development world
only serves to stifle both creativity and productivity. At present the only
way questions regarding the validity of copyrights and patents are being
answered is through random decisions from legal proceedings. Copyright and
patent laws both need to be updated to work properly in the 1990s.



Small Is Better


Dear DDJ,
Jeff Duntemann is the second person I have encountered in print this month who
characterizes the evolution of S/370 mainframes as moving toward the role of a
gigantic file server. The other guy is the CEO of my present work situation.
A recent discussion (warm, not heated) with a fellow systems grunt who
specializes in network supports Jeff's observation that the mainframe "empire"
is resisting the enhancement and distribution of processing that micros bring
to the 4 techno-brews we build and support. His positions are 1) most
databases cannot be distributed, 2) the lack of standardized protocols and
network architectures eliminate most advantages of micro local processing, and
3) ancient business practices, banking for example, do not mesh well with
modern data processing technologies. Well, some of that is true yet anyone who
has jumped into micro coding from a mainframe environment knows the euphoria
of megalomaniacal control, has been amazed by the low cost of quality software
tools, and has embraced the heady vision of a computer-literate society where
programming arts will be as second nature as reading and writing.
IBM and compatible vendors continue to eliminate the need for systems support
through automated operations, packaged operating system
installation/maintenance, 4GL database administration, system managed storage,
and of course function suction into microcode, as Jeff has observed. Systems
programming at the operating system level has been reduced to configuration
management to a great degree. S/370 mainframes are becoming turnkey and
selfconfiguring, a welcomed release from drudgery and business risk. As a
result, we systems programmers who were first attracted to the science, art,
and technology of S/370 become increasingly bored with the whole mainframe
world. Some organizations have tried to cut this boredom by inventing projects
based on ALC for their programmers to play with, usually having artificial and
redundant purposes. What a waste of talent.
My AT-class system is booted, the coffee is fresh, Turbo C faithfully awaits
my attention as the word processor gets stroked. It is Sunday morning, the sky
is gray with rain, my desk is strewn with reference books and product
catalogs. An application that has never existed before occupies my background
wetware as these lines are written; plans for system expansion pop up like
menus. I am a happy programmer doing happy things: I am learning, creating,
imagining.
Come Monday morning I will return to my small, crowded office and turn my
concerns to DASD management. There will be performance statistics to analyze
and a time sheet to keep current. A product analysis report will be written, a
meeting attended. Decisions will be made slowly and safely, actions will be
delayed to an ever-shrinking outage window. I will think occasionally about
the 20 MIPS processor, the four-volume database, the worldwide network.
Problems will occur and solutions will be defined. I will cover my behind and
not rock the boat too violently because this is how you survive in the
business world.
Meanwhile there is a whole population of micro programmers out there who are
thinking at a level not known in the mainframe world in recent years, whose
mainframe concerns amount to a hill of beans. They want access and nothing
more. At their fingertips are megabytes of main storage, gigabytes of disk
storage, CASE tools, multitasking, hundreds of colors, thousands of bauds. Me,
I will hit ENTER at my graphics tube and wait for GDDM to get a few cycles. I
will print a document and receive it an hour later. I will scurry to the
machine room to check a main console; scurry to a meeting and struggle to stay
interested, write a memo, update a time sheet. I will return home to boot my
personal system and again become a happy programmer.
Business in general wants the turnkey mainframe. There is nothing wrong with
this other than it puts me and many other systems folks out on the street with
unmarketable knowledge. Notice I did not write "skills" -- we all have
tremendously marketable skills. S/370 knowledge is a good base for OS/2;
assembler knowledge a good base for micro assembler; SAS and other high-level
language knowledge a good base for Pascal, C, and others. Our skills are
inherent: Love of science, art, and technology; the ability to make a machine
do what we want; enough business savvy to survive these many years in data
processing. So bring on the turnkey mainframe; bring on the local area
network; bring on distributed processing; and give me back my machine! Let me
build the better system, create the never before seen application, make this
puppy run like it never ran before. Get me out of the erector-set mentality of
canned software, black box hardware, and Big Blue strategy.
For my colleagues who find all this ranting quite unbecoming of a
professional, I would advise opening up those purse strings and getting a home
micro system. Get some development software, build a reference/tutorial
library. Cook up some application interesting to you and develop it. It does
not matter if the program ever sells. It matters that it is your program, does
what you want, and that you learn the amazing cost, function, and performance
characteristics of the micro computer. Then and only then will we ever have
meaningful discussions on the viability of distributed, local, and personal
computing.
Two really smart people in one month comparing mainframes to file servers.
Doesn't that tell you something?
Ray A. Kampa
Chantilly, Virginia


Unix Help Wanted


Dear DDJ,
I would like to describe a problem I recently encountered with the Unix system
Bourne shell. It seems that there is a subtle interaction between path
searching, file permission flags, and file hashing. A friend asked me to do a
regression test for him after he made some last minute changes to his program.
Using FTP, I imported a copy of the last minute version. (An official beta
test version was already installed on my system.) A quick test showed that no
errors had been introduced by the changes. In fact, I could detect no
difference at all between the beta and last minute versions. I reported to my
friend that everything looked fine. Hours later, he dropped by to demonstrate
some exciting feature. Having difficulty finding this feature in my copy of
his program, he concluded that I had, in fact, been testing the official beta
version. What had happened?
My friend and I reasoned that two things had occurred First, FTP had stripped
the execute permission from my copy. Second, and heretofore unknown to me, the
path search function only finds files that have execute permission set. It
encounters the named file without execute permission, it continues its hunt
for an executable version. (If the official beta test version had not been
installed, I would have received the error message: "name: execute permission
denied.")
Being resourceful, my friend and I proceeded to reset the execute permission
flag using the chmod command. And, when we tried to execute the program again,
the path search function still yielded the official beta test version. Now
what had happened?
Hashing! The Bourne shell of System V has a thing called file hashing that
speeds up path searches. The shell remembers where in your search path it was
that invoked commands were last found. So even though path searching would
have worked properly now, it was not being used. The hash memory can be erased
with the "hash - r" command.
Now, at last, my copy of the program was being tested. Unfortunately, it was
time to go home, and the actual program test had to wait until the next day.
(Yes! Non-trivial errors were discovered and I had to retract my earlier
thumbs up.)
Perhaps the Unix developers would consider a revision to the System V
specification and redesign the path searching algorithm. Would it cause any
problem, I wonder, if, instead of looking for EXECUTABLE FILES, the path
search function looks for ANY FILE whose name is identical to that entered on
the command line. This might generate more error messages, but users could be
more confident that path searching has resolved to the expected version of a
file.
Steve Haffner
Issaquah, Washington


Graphics Programming Fix


Dear DDJ,
As you probably have been made aware, there was a typo in one of the listings
for Kent Porter's "Graphics Programming" column in the July issue. The typo
caused a program to loop in a recursive call, which would either cause a
system bang or exhausted stack space.
The typo was in the program RESIZE, in the subroutine drawstar. In this
subroutine, the first call to draw_rect should have its dy argument be (30)
not (-30). This causes the intended rectangle to be lower than the arguments
to the following floodfill anticipated. Instead of filling an object, the
floodfill routine tries to fill the viewport.
This first error revealed two other errors in earlier listings that had not
been noticed because filling a viewport had never been tested.
The first is in the EGAPIXEL routine. It checks incorrectly if a pixel is
within the vuport. It checks if the pixel is "greater than or equal" to the
vuport instead of just "greater than". If a pixel is in the vuport, it can be
equal to the limit. This caused the floodfill to reject the last fill line as
outside the vuport and make a recursive call in the reverse direction,
starting an endless loop.
The EGAPIXEL routine does not check if the pixel is less than the vuport. This
is why the endless loop doesn't happen until the fill hits the bottom.
Depending on someone's needs, the test can be either fixed for both cases or
dropped completely.
The second error is in FLOODFILL. The variable dir is not needed at all. It
can be fixed as -1 in the first loop and +1 in the second. The third loop can
be dropped completely. With these changes, there is no need to add a check in
floodfill for the error return condition from EGAPIXEL. Also this change
speeds up the fill routine by about 20 percent.
John Horvath
Acton, Massachusetts


Subscrib-er to Subscrib-ee


Dear DDJ,
I have been a fan of Jeff Duntemann since the late Borland Turbo Technix. His
October column comparing OOP word definitions in various languages was a
beauty. However, I would like to play turnabout.
In defining binding he uses the terms caller and call-ee. I quibble with
call-ee.
There is a parent-child sequence that goes something like telegraphy,
telephony, radio, electronics, computers. We have inherited much terminology
from our ancestor technologies. Since time immemorial (or at least, since the
beginnings of telephone and telegraph switchboards) the terms caller and
called have been used. These are pronounced call-lerr and call-ledd. Each word
has two equally stressed syllables.
John P. Reid
Bear, Delaware







December, 1989
NETWORK GRAPHS IN OBJECT PASCAL


Linked Lists as reusable objects




Steve Kienle


Steve has been a professional programmer since 1983 and has written several
programs that are available on the CompuServe Macintosh forums, including the
Turing Machine Editor and Face Manager programs. He can be reached at 2314
Waverly, Kalamazoo, MI 49007 or though CompuServe at 72330, 111.


The origins of OOP and the emergence of OOP features in a number of
programming languages are well known. The proponents of Ada and Modula-2 claim
that OOP is part of the design of these languages. C, the language of the
systems developer, has also been caught up in the OOP wave, spawning C++ and
Objective C. Most recently, the Apple-developed OOP dialect of Pascal has
gained momentum now that Microsoft and Borland have both added support of
objects into their respective Pascal offerings.
In this article, I will discuss how Object Pascal implements objects and
methods, and I'll describe a program that creates a network graph. I'll also
look back at the objects that were created in this program and give examples
of how they can be reused in other programs. This article assumes that you are
familiar with OOP in general, and that you have a knowledge of standard
Macintosh data types and toolbox calls such as Point, Rect, LineTo, and
FrameRect.


The Network Graph Program


The network graph program creates and manipulates a simple network graph. This
program allows the user to create and remove vertices and edges, and to move
vertices around. The graph is created in a window that allows scrolling in two
directions, up to the current size of the page. The graph is created by using
tools available in a pallet window. For simplicity, the program doesn't
support such features as saving the graph to disk, and it doesn't support the
Edit menu items.
The code for the section of the program that handles object types and method
definitions is provided in Listing One (see page 104). The executable program,
complete code, and associated MPW (Macintosh Programming Workbench) support
files are available through DDJ or from the author. (See the author biography
for more information -- Ed.)
OOP is really programming from the data up, rather than from the functions
down. OOP strives to encapsulate the data of the problem and to define methods
for manipulating that data. In the case of the network graph program, the
first step in using OOP is to define the data object that you wish to create,
which is a network graph. The mathematical definition for a graph is "an
object created by two sets. The first set consists of vertices, and the second
set contains edges." From this definition, we quickly derive three different
objects: the graph, the vertices, and the edges.
Digging a bit deeper into the definition reveals that the vertices and edges
are collected into sets. Pascal does not allow the use of nonscalars in sets.
As a result, I used a linked list to implement a set, and added two more
objects, GraphList and GraphNode, to support the linked list.
GraphNode is a fairly simple object -- it only has a Next link. GraphNode's
methods include Draw, DrawAll, Erase, EraseAll, Free, and FreeAll. Free, which
is similar to the example presented in the sidebar, erases the node and then
frees it from memory. Draw and Erase are null and serve as place holders for
the methods in the descendant objects.
DrawAll, EraseAll, and FreeAll deserve a bit more attention. They use the Next
link to pass a message to the next GraphNode in the list, and then call the
appropriate method. These methods use inherited code and allow only one method
call to cause a complete re-draw, erasure, or freeing of a GraphList. Note
that because the Next instance variable is a GraphNode, you can use Next in a
reference to another method call. This approach is shown in the SELF.Next
.EraseAll type statements, which read as: "Look at myself, get the Next
instance variable's value, and send that object instance the EraseAll
message."
More Details.
GraphList holds the head pointer of the GraphNode link list. This object
handles the special case of removing a GraphNode from the list, and also
demonstrates code reusability. Because we need a list of Vertices and Edges,
the creation of a GraphList object class creates one common location for the
list-manipulation routines. The GraphList object only contains a FirstNode
instance variable. The methods include AddNode, RemoveNode, Draw, Erase, and
Free.
Draw and Erase make the process of redrawing a complete graph easier. These
methods call DrawAll and EraseAll for FirstNode. Free also calls FreeAll for
FirstNode, but then calls the inherited Free method in order to destroy the
list object instance. AddNode and RemoveNode perform the manipulations on the
list. AddNode takes as an argument a GraphNode object instance and makes this
object the new FirstNode. RemoveNode also takes a GraphNode object instance as
an argument. RemoveNode finds and removes the appropriate GraphNode from the
list.
Note that the Vertex object type, which is maintained in a list, is based upon
the GraphNode object type. Therefore, Vertex inherits the Next instance
variable and the Draw, DrawAll, Erase, EraseAll, Free, and FreeAll methods.
Draw and Erase must then be overridden. Examine these overridden methods and
note that information about where the Vertex is centered and how large the
Vertex should be made is required. For the network graph program, the Vertex
is a circle with a radius of 10. This creates one new instance variable called
Center. Draw and Erase now create or erase the 10-point radius circle around
Center. You could set the Center instance variable by accessing the instance
variable itself. But in order to support the OOP methodology as best as
possible, I've added a SetCenter method.
Edge is directly related to the Vertex object type. Edge connects two
Vertices. As a result, I've added two new instance variables, FromVertex and
ToVertex. As with Vertex, Edge is maintained in a list and GraphNode is the
base object. Again, Draw and Erase have been overridden. Draw draws a line
from the center of FromVertex to the center of ToVertex. To keep the Vertex
objects looking clean, I modified the Vertex Draw method to erase the circle
and then frame it; the edges now start and end at the edge of the vertex
frame. Erase sets the drawing pattern to white, and then calls Draw. This
approach allows Draw to be modified in descendant objects, while Erase
continues to work unchanged.
Edge also uses the SetFrom and SetTo methods to erase the old Edge, and to
redraw the FromVertex or ToVertex to which it is attached. Edge is dependent
upon two Vertex instances, so if either of those instances move, Edge must be
redrawn. An apparently simple solution would be to modify the Vertex SetCenter
method. But Vertex has no knowledge of which Edge instances are connected to
it, so trying to locate, erase, and redraw those Edge instances would force
external knowledge into the object. Therefore, the Graph object, which has
knowledge of Edges and Vertices, handles this problem.
The Graph object is based upon TObject. Graph consists of two sets, so it
needs instance variables that point to its list of vertices and edges. We can
start the method list with some obvious methods: Draw, Erase, Free (which is
overridden), AddVertex, RemoveVertex, AddEdge, and RemoveEdge. Draw and Erase
call the EdgeList and VertexList instances of both the GraphList object and
the Draw and Erase messages. The order is important here, because I have
designed the Draw methods for the Edges instances to be drawn first.
Similarly, Free frees the EdgeList and then the VertexList.
With each event, the Macintosh provides the location at which the event
occurred. This means that in the case of the AddVertex, AddEdge, RemoveEdge,
and RemoveVertex, only the location of the mouse is available. Thus, these
methods take a point as their argument. AddVertex adds a vertex at the
specified location.
Note that because the AddNode method of the GraphList object takes a GraphNode
as an argument, the Vertex object instance cannot be used directly in the
call. This is a result of the strict type constraints of Pascal, rather than a
result of how Object Pascal is implemented. This limitation can be overcome by
typecasting the object instance. This concept, which is borrowed from C,
allows the compiler to accept the technique of sending a Vertex object
instance to AddNode. This trick is required because I defined a common
GraphList object, rather than defining VertexList and EdgeList, which would be
identical except for what they are linking. The trick is required for the
compiler only; the object instance is not modified in any way.
AddEdge is very similar to AddVertex. Instead of taking a location, AddEdge
takes two Vertex object instances and creates an Edge instance that links them
together.
RemoveEdge removes an Edge object instance. This means that we need a way to
determine the Edge instance at a given point. Look ahead a bit and note that
RemoveVertex also needs to locate an object instance based upon a location. To
do this, we add PtInNode and FindNode to the GraphNode object type and add
FindNode to the GraphList object type. The methods belong here because both
descendant object types require it. PtInNode returns True if the point is
considered to be in the object instance, and returns False otherwise.
GraphNode. FindNode checks if the point is in the present object instance. If
the point is in the present object instance, GraphNode.FindNode returns
itself; otherwise, the checking down the list. The GraphList.FindNode method
initiates the search at the FirstNode of the list and returns the result.
In order to support PtInNode, we need to define the region covered by the
object instance. I use the term "region" on purpose, because the Mac Toolbox
has a routine, PtInRgn, that does the processing for us. To create this
region, I added a new instance variable to the GraphNode object type, called
HotRegion. The SetRegion method creates the appropriate region. In the case of
the Vertex object, this region is the circle that the Vertex creates on the
screen. In the case of the Edge object, a region is created that follows the
line but is eight pixels wide.
The addition of HotRegion, which is location dependent, forces a change in
some of the other methods. As the center of a Vertex instance or an end point
of an Edge instance is set, the HotRegion must be recreated. I modified
SetCenter, SetFrom, and SetTo to reflect this requirement. Note that the Free
method of GraphNode frees the region.
After the support methods are added to GraphNode, Vertex, and Edge, we can
return to RemoveEdge. This method first finds the Edge instance in which the
point is located and removes that Edge from EdgeList.
The Remove Vertex method is similar to RemoveEdge, but requires some
additional support in the GraphNode and Edge object types. RemoveVertex first
finds the Vertex instance in which the point is located. The method then
removes all Edge instances that are connected to that Vertex instance.
Finding all such instances can be done in this method or through a method on
the Edge object. Because the list traversal logic has been isolated in the
GraphNode object, I chose to add the finding logic into the GraphNode object
but leave the instance removal in RemoveVertex. This approach involves the
creation of Connected and FindConnected methods in the GraphNode object type.
The FindConnected method also must be included in the GraphList object type.
In GraphNode, Connected always returns False; but the Edge object overrides
this and returns True if Edge is connected to the Vertex instance.
FindConnected finds the first connected instance in the list.
These routines enable you to find and remove the Edge instances connected to a
given Vertex instance. Once all the appropriate Edge instances are removed,
the Vertex object instance itself can be removed.
At this point, the special processing required for SetCenter can also be
implemented. To do this, a SetVertexCenter method was added to the Graph
object type. The purpose of the additional processing beyond the Vertex.
SetCenter call is to find all of the connected edges, and to both erase these
edges before and then redraw them after the SetCenter call. FindConnected
allows you to move through EdgeList and find all of the connected Edge
instances. The complexity of adding the special processing to the Graph object
type, rather than to the Vertex type, makes the program much cleaner and more
self contained.
Two methods have been added to make the program conform more closely to the
Macintosh interface guidelines. These methods allow a Vertex instance to be
moved with the mouse and allow an Edge instance to be added with the mouse.
The step of implementing MoveVertex in the Graph object is fairly easy. Given
a point from which the move is starting, find the Vertex at the point. Use the
Macintosh Toolbox routine DragGrayRgn to let the user pull the HotRegion
around the screen. I have separated the Mac-specific code into the DragRegion
function, which is not included in the listings with this article (but is
available online. If the drag is completed with a move, the new center of the
Vertexinstance is determined, and the Graph.SetVertexCenter method is called
to move the Vertex instance to the new location.
The code for LinkVertices is also straightforward. First, check the location
to see if it contains a Vertex instance, and then call a DragGrayLine
procedure to find where the mouse is released. If the mouse is released in a
Vertex instance that is different from the first Vertex instance, then the
Graph.Add-Edge routine is called to add the new Edge instance. DragGrayLine
follows the mouse around the window, trailing a gray line between the current
location and the starting location until the mouse button is released.
DragGrayLine then returns the location of the mouse.


Reusable Objects


GraphNode implements a linked list with some graphing methods associated with
it. GraphList maintains a list of GraphNode instances. The Vertex and Edge
object types are based upon the GraphNode type, and implement the vertices and
edges of the network graph. The Graph object is the actual network graph and
provides methods to support and modify the graph instance.
As written, the Graph object is somewhat restrictive. At the least, it allows
several instances of network graphs to be handled by the same program. Graph
can also be used at the base for additional types of graphs, such as directed
graphs, weighted graphs, and so on.
One type of network graph that I want to mention is the Cartesian graph, which
is what most people think about when you say "graph." By placing vertices
carefully, and adding edges that connect one vertex to the next one down the X
axis, we can create a program that graphs a set of data points fairly quickly.
By using several Graph-based objects, several sets of data can be plotted in
the same window. At the same time a separation of the sets can be maintained,
which is useful when data sets are edited independently.
Additional methods can be added to the Graph-based object to provide for
greater functionality. A method of traversing the graph, based upon some
algorithm, is an example of this approach. In fact, I've written a Turing
Machine Editor that is based upon these objects.

Note that objects such as the Vertex object type can be used in several ways.
Many of these uses are listed indirectly earlier this article during the
discussion of the Graph object type. Vertex can also be used as a base object
type. Naturally, you can change the way in which the object is drawn. This
change can give rise to different ways of showing each vertex in the window.
The drawing method can even be selected by an instance variable in order to
allow a graph to contain several different types of vertices. If you create a
Vertex-based class that doesn't draw at all, then you have a way to generate
line art. Note that many of these uses for Vertex type objects can be
implemented within or without the Graph object itself.
Edge is intimately tied to the Vertex class, so Edge can only be used in
conjunction with that type. Still, variations on this class can be useful in
themselves. Such variations could include different ways of drawing the edge,
such as drawing the edge as an arc or a gray bar.
Finally, the GraphList and GraphNode classes actually implement a list
structure for objects that are drawn, connected, and selected, so these
classes provide many opportunities for reuse. Any program that creates or
manipulates lists of graphical objects, such as an object-oriented drawing
program, is a candidate for use of the GraphList and GraphNode object types.


Object Pascal


In order to add OOP to Pascal, Apple modified Pascal to create Object Pascal
(or Clascal -- Class Pascal -- as it was first called). These changes were
made on the Pascal that was designed for the graphical-interface-based
computer Apple was developing at the time: The Lisa. Even though the Lisa no
longer exists, Object Pascal continues to gain momentum.
The major change to Pascal in Object Pascal is the addition of the Object
type. In Object Pascal, the classes of OOP are defined in the Type declaration
section as belonging to the Object type. The format for the Object statement
is similar to the format for the Record statement, but the declaration of the
Object statement contains two additional parts: the Base Object and the Method
declarations. The format is shown in the railroad diagram in Figure 1.
The Base Object enables the use of class inheritance in Object Pascal. To make
a new object type a descendant of another object type, place the parent's
object type name into the parentheses.
The method declarations in Object Pascal are similar to the forward procedure
and function declarations in Pascal. List the procedure or function
declarations in the block, which is where new methods on this object must be
declared. Optionally, an Override statement may follow a method declaration in
order to replace an inherited method.
Most implementations of Object Pascal include a base object type that provides
some simple methods. For example, my Pascal compiler calls its base object
TObject. TObject provides base methods for Clone and Free. Clone makes a copy
of an object instance, and Free frees an object instance from memory.
The code for each Object's methods must be included somewhere in the code
section of the program. This is handled in a similar fashion to other
procedure and function definitions, except that the procedure or function name
is prefixed with the Object Type name. Whenever instance variables are used in
method definitions, the variables are prefixed with the special SELF
reference. This prefix tells the compiler to use the values for those
variables in the object instance that called this method. Listing Two (page
108) provides a sample Circle class's type declaration and procedure
definitions. Note that Circle inherits Clone and Free from TObject.
You declare a variable that is to be used as an instance of the Object type in
the same way that you declare a Record variable. The catch is that until a new
instance is created, that variable doesn't reference a valid object instance.
The standard Pascal NEWO procedure is used for creating an object instance.
Once an object instance is created, the instance variables are referenced in
the same way that Record variables are referenced. To call an object's method,
prefix the method name with the instance variable. An example of declaring and
using a Circle object instance is shown in Listing Three, page 108.
As the example program stands now, the circle will be drawn in a window even
after the instance is freed. If this result is not desired two options are
available: Add a call to Circle.Erase before the call to Circle.Free, or
override the Free inherited from the TObject class. To implement the second
choice, use the Override statement in the method declaration section of the
Circle Object Type definition.
Override provides a way to change an inherited method. Object Pascal also
allows the new method to call the inherited method through the Inherited
statement. The use of the Inherited statement allows small changes to be made
to a method, while leaving the inherited code available and contained in the
parent object. Listing Four, page 108, adds a Free method to the Circle Object
Type and shows how to use the Override and Inherited statements. Note the use
of the SELF reference here to call another method for the same object
instance. These changes cause the example program to erase the circle when the
instance is freed.
Through the use of inheritance and overriding, several related object types
can have the same procedure name, such as Draw, and implement the procedure
differently. This factor alone is a benefit, but it's not the only strength of
Object Pascal. Object Pascal also links each object instance to the object
type methods so that it's possible to create methods in the ancestor of
several types that can be implemented without making changes in the children.
Listing Five (see page 108) shows an example of this.
You may already be noticing that Object Pascal does not support data hiding.
This failure follows from the way that Pascal itself was designed. While the
newer versions of Pascal provide some indirect data hiding through the use of
Units, the instance variables are still available to the calling program.
While I admit the importance of data hiding, I don't feel an impact from its
loss in Object Pascal. A lot of the benefits of data hiding can be realized
through careful programming practices. -- S.K.


_NETWORK GRAPHS IN OBJECT PASCAL_
by Steven Kienle


[LISTING ONE]

TYPE
 GraphNode =
 OBJECT(TObject)
 Next: GraphNode;
 HotRegion: RgnHandle;

 PROCEDURE Initialize;
 { Drawing Methods }
 PROCEDURE Draw;
 PROCEDURE DrawAll;
 PROCEDURE Erase;
 PROCEDURE EraseAll;
 { Location Methods }
 PROCEDURE SetRegion;
 FUNCTION PtInNode(Where: Point): Boolean;
 FUNCTION FindNode(Where: Point): GraphNode;
 FUNCTION Connected(Which: GraphNode): Boolean;
 FUNCTION FindConnected(Which: GraphNode): GraphNode;
 { Freeing Methods }
 PROCEDURE Free; OVERRIDE;
 PROCEDURE FreeAll;
 END;

 GraphList =
 OBJECT(TObject)
 FirstNode: GraphNode;
 PROCEDURE Initialize;
 { Drawing Methods }
 PROCEDURE Erase;
 PROCEDURE Draw;
 { GraphList Manipulation Methods }
 PROCEDURE AddNode(Which: GraphNode);

 PROCEDURE RemoveNode(Which: GraphNode);
 { Location Methods }
 FUNCTION FindNode(Where: Point): GraphNode;
 FUNCTION FindConnected(Which: GraphNode): GraphNode;
 { Freeing Methods }
 PROCEDURE Free; OVERRIDE;
 END;

 Vertex =
 OBJECT(GraphNode)
 Center: Point; { Location of Vertex }
 PROCEDURE Initialize; OVERRIDE;
 { Drawing Methods }
 PROCEDURE Draw; OVERRIDE;
 PROCEDURE Erase; OVERRIDE;
 { Location Methods }
 PROCEDURE SetRegion; OVERRIDE;
 PROCEDURE SetCenter(thePoint: Point);
 END;

 Edge =
 OBJECT(GraphNode)
 FromVertex: Vertex; { End Points of Edge }
 ToVertex: Vertex;
 PROCEDURE Initialize; OVERRIDE;
 { Drawing Methods }
 PROCEDURE Draw; OVERRIDE;
 PROCEDURE Erase; OVERRIDE;
 { Location Methods }
 PROCEDURE SetRegion; OVERRIDE;
 FUNCTION Connected(Which: GraphNode): Boolean;
OVERRIDE;
 PROCEDURE Edge.SetFrom(Which: Vertex);
 PROCEDURE Edge.SetTo(Which: Vertex);
 END;

 Graph =
 OBJECT(TObject)
 VertexList: GraphList;
 EdgeList: GraphList;
 PROCEDURE Initialize;
 { Drawing Methods }
 PROCEDURE Draw;
 PROCEDURE Erase;
 { Manipulation Routines }
 PROCEDURE AddVertex(Where: Point);
 PROCEDURE AddEdge(FromWhich, ToWhich: Vertex);
 PROCEDURE RemoveVertex(Where: Point);
 PROCEDURE RemoveEdge(Where: Point);
 PROCEDURE SetVertexCenter
 (Which: Vertex; Where: Point);
 { Macintosh Support Routines }
 PROCEDURE MoveVertex(Start: Point);
 PROCEDURE LinkVertices(Start: Point);
 { Freeing Method }
 PROCEDURE Free; OVERRIDE;
 END;

{ -------------- GraphNode Methods -------------- }

PROCEDURE GraphNode.Initialize;
 BEGIN
 SELF.Next := NIL;
 SELF.HotRegion := NIL;
 END;

PROCEDURE GraphNode.Draw;
 BEGIN
 END;

PROCEDURE GraphNode.DrawAll;
 BEGIN
 IF SELF.Next <> NIL THEN
 SELF.Next.DrawAll; { Draw next GraphNode }
 SELF.Draw; { Draw this GraphNode }
 END;

PROCEDURE GraphNode.Erase;
 BEGIN
 END;

PROCEDURE GraphNode.EraseAll;
 BEGIN
 IF SELF.Next <> NIL THEN
 SELF.Next.EraseAll; { Erase next GraphNode }
 SELF.Erase; { Erase this GraphNode }
 END;

PROCEDURE GraphNode.SetRegion;
 BEGIN
 IF SELF.HotRegion <> NIL THEN { Drop old Region }
 DisposeRgn(SELF.HotRegion);
 SELF.HotRegion := NewRgn; { Allocate a new one }
 END;

FUNCTION GraphNode.PtInNode(Where: Point): Boolean;
 BEGIN
 IF PtInRgn(Where, SELF.HotRegion) THEN
 PtInNode := True
 ELSE
 PtInNode := False;
 END;

FUNCTION GraphNode.FindNode(Where: Point): GraphNode;
 BEGIN
 IF SELF.PtInNode(Where) THEN { It's here }
 FindNode := SELF
 ELSE IF SELF.Next = NIL THEN { There are none }
 FindNode := NIL
 ELSE { Try the Next }
 FindNode := SELF.Next.FindNode(Where);
 END;

FUNCTION GraphNode.Connected(Which: GraphNode): Boolean;
 BEGIN
 Connected := False;
 END;

FUNCTION GraphNode.FindConnected(Which: GraphNode): GraphNode;

 BEGIN
 IF SELF.Connected(Which) THEN { Is this connected }
 FindConnected := SELF
 ELSE IF SELF.Next = NIL THEN { There are none }
 FindConnected := NIL
 ELSE { Try the Next }
 FindConnected := SELF.Next.FindConnected(Which);
 END;

PROCEDURE GraphNode.Free;
 BEGIN
 IF SELF.HotRegion <> NIL THEN { Free Region Space }
 DisposeRgn(SELF.HotRegion);
 SELF.Erase; { Erase then Free }
 INHERITED Free;
 END;

PROCEDURE GraphNode.FreeAll;
 BEGIN
 IF SELF.Next <> NIL THEN { Free the next GraphNode }
 SELF.Next.FreeAll;
 SELF.Free; { Then Free this GraphNode }
 END;

{ -------------- GraphList Methods -------------- }
PROCEDURE GraphList.Initialize;
 BEGIN
 SELF.FirstNode := NIL;
 END;

PROCEDURE GraphList.Erase;
 BEGIN
 IF SELF.FirstNode <> NIL THEN
 SELF.FirstNode.EraseAll; { Erase the GraphList }
 END;

PROCEDURE GraphList.Draw;
 BEGIN
 IF SELF.FirstNode <> NIL THEN
 SELF.FirstNode.DrawAll; { Draw the GraphList }
 END;

PROCEDURE GraphList.AddNode(Which: GraphNode);
 BEGIN
 Which.Next := SELF.FirstNode; { Link Which in GraphList }
 SELF.FirstNode := Which;
 END;

PROCEDURE GraphList.RemoveNode(Which: GraphNode);
 VAR
 Check: GraphNode;
 BEGIN
 { If it is the head GraphNode, relink the Head }
 IF SELF.FirstNode = Which THEN
 SELF.FirstNode := Which.Next
 ELSE BEGIN
 { Otherwise look for Which GraphNode }
 Check := SELF.FirstNode;
 WHILE (Check <> NIL) DO BEGIN

 { If Which is found, remove it from GraphList }
 IF Check.Next = Which THEN
 Check.Next := Which.Next;
 Check := Check.Next;
 END;
 END;

 Which.Free; { Free this node }
 END;

FUNCTION GraphList.FindNode(Where: Point): GraphNode;
 BEGIN { Find the Node at this location }
 IF SELF.FirstNode <> NIL THEN
 FindNode := SELF.FirstNode.FindNode(Where)
 ELSE
 FindNode := NIL;
 END;

FUNCTION GraphList.FindConnected(Which: GraphNode): GraphNode;
 BEGIN { Find the Node connected to this one }
 IF SELF.FirstNode <> NIL THEN
 FindConnected := SELF.FirstNode.FindConnected(Which)
 ELSE
 FindConnected := NIL;
 END;

PROCEDURE GraphList.Free;
 BEGIN
 IF SELF.FirstNode <> NIL THEN
 SELF.FirstNode.FreeAll; { Free the Nodes }
 INHERITED Free; { Free GraphList }
 END;

{ -------------- Vertex Methods -------------- }
PROCEDURE Vertex.Initialize;
 BEGIN
 INHERITED Initialize;
 SELF.Center.h := 0;
 SELF.Center.v := 0;
 END;

PROCEDURE Vertex.Draw;
 VAR
 theRect: Rect;
 BEGIN
 SELF.Erase; { Erase Vertex Area }
 { Set up Rectangle }
 theRect.top := SELF.Center.v - 10;
 theRect.left := SELF.Center.h - 10;
 theRect.bottom := SELF.Center.v + 10;
 theRect.right := SELF.Center.h + 10;
 { Draw Vertex }
 FrameOval(theRect);
 END;

PROCEDURE Vertex.Erase;
 VAR
 theRect: Rect;
 BEGIN

 { Set up Rectangle }
 theRect.top := SELF.Center.v - 10;
 theRect.left := SELF.Center.h - 10;
 theRect.bottom := SELF.Center.v + 10;
 theRect.right := SELF.Center.h + 10;
 { Erase Vertex }
 EraseOval(theRect);
 END;

PROCEDURE Vertex.SetRegion;
 BEGIN
 INHERITED SetRegion; { Do default processing }
 OpenRgn; { Create new region area }
 SELF.Draw;
 CloseRgn(SELF.HotRegion);
 END;

PROCEDURE Vertex.SetCenter(thePoint: Point);
 BEGIN
 SELF.Erase; { Erase Vertex at old Center }
 SELF.Center := thePoint; { Set the Center }
 SELF.Draw; { Draw Vertex at new Center }
 SELF.SetRegion; { Reset HotRegion }
 END;

{ -------------- Edge Methods -------------- }
PROCEDURE Edge.Initialize;
 BEGIN
 INHERITED Initialize;
 FromVertex := NIL;
 ToVertex := NIL;
 END;

PROCEDURE Edge.Draw;
 VAR
 Where: Point;
 BEGIN
 IF (SELF.FromVertex <> NIL) AND (SELF.ToVertex <> NIL) THEN
BEGIN
 { Start in center of FromVertex }
 Where := SELF.FromVertex.Center;
 MoveTo(Where.h, Where.v);
 { Draw line to center of ToVertex }
 Where := SELF.ToVertex.Center;
 LineTo(Where.h, Where.v);
 END;
 END;

PROCEDURE Edge.Erase;
 VAR
 pnState: PenState;
 BEGIN
 GetPenState(pnState); { Save current settings }
 PenPat(white); { Set color & Draw to erase }
 SELF.Draw;
 SetPenState(pnState); { Reset settings }
 SELF.FromVertex.Draw; { Redraw affected Vertices }
 SELF.ToVertex.Draw;
 END;


PROCEDURE Edge.SetRegion;
 BEGIN
 INHERITED SetRegion; { Do default processing }
 OpenRgn; { Create new region area }
 MoveTo(SELF.FromVertex.Center.h + 4,
SELF.FromVertex.Center.v + 4);
 LineTo(SELF.ToVertex.Center.h + 4, SELF.ToVertex.Center.v + 4);
 LineTo(SELF.ToVertex.Center.h - 4, SELF.ToVertex.Center.v - 4);
 LineTo(SELF.FromVertex.Center.h - 4,
SELF.FromVertex.Center.v - 4);
 LineTo(SELF.FromVertex.Center.h + 4,
SELF.FromVertex.Center.v + 4);
 CloseRgn(SELF.HotRegion);
 END;

FUNCTION Edge.Connected(Which: GraphNode): Boolean;
 BEGIN
 IF (SELF.FromVertex = Which) OR (SELF.ToVertex = Which) THEN
 Connected := True
 ELSE
 Connected := False;
 END;

PROCEDURE Edge.SetFrom(Which: Vertex);
 BEGIN
 IF (SELF.FromVertex <> NIL) AND (SELF.ToVertex <> NIL) THEN
BEGIN
 { Erase old edge and redraw unlinked Vertex }
 SELF.Erase;
 SELF.FromVertex.Draw;
 END;
 SELF.FromVertex := Which;
 IF (SELF.FromVertex <> NIL) AND (SELF.ToVertex <> NIL) THEN
BEGIN
 { Draw new edge and redraw linked Vertices }
 SELF.Draw;
 SELF.FromVertex.Draw;
 SELF.ToVertex.Draw;
 SELF.SetRegion; { Reset HotRegion }
 END;
 END;

PROCEDURE Edge.SetTo(Which: Vertex);
 BEGIN
 IF (SELF.FromVertex <> NIL) AND (SELF.ToVertex <> NIL) THEN
BEGIN
 { Erase old edge and redraw unlinked Vertex }
 SELF.Erase;
 SELF.ToVertex.Draw;
 END;
 SELF.ToVertex := Which;
 IF (SELF.FromVertex <> NIL) AND (SELF.ToVertex <> NIL) THEN
BEGIN
 { Draw new edge and redraw linked Vertices }
 SELF.Draw;
 SELF.FromVertex.Draw;
 SELF.ToVertex.Draw;
 SELF.SetRegion; { Reset HotRegion }

 END;
 END;

{ -------------- Graph Methods -------------- }
PROCEDURE Graph.Initialize;
 BEGIN
 New(SELF.VertexList);
 SELF.VertexList.Initialize;
 New(SELF.EdgeList);
 SELF.EdgeList.Initialize;
 END;

PROCEDURE Graph.Draw;
 BEGIN
 IF SELF.EdgeList <> NIL THEN
 SELF.EdgeList.Draw;
 IF SELF.VertexList <> NIL THEN
 SELF.VertexList.Draw;
 END;

PROCEDURE Graph.Erase;
 BEGIN
 SELF.EdgeList.Erase;
 SELF.VertexList.Erase;
 END;

PROCEDURE Graph.AddVertex(Where: Point);
 VAR
 NewVertex: Vertex;
 BEGIN
 { Create and initialize a new Vertex at Where }
 New(NewVertex);
 NewVertex.Initialize;
 NewVertex.SetCenter(Where);
 { Add new vertex to list, typecasting is required }
 SELF.VertexList.AddNode(GraphNode(NewVertex));
 END;

PROCEDURE Graph.RemoveVertex(Where: Point);
 VAR
 WhichEdge: GraphNode;
 WhichVertex: GraphNode;
 BEGIN
 { Find the appropriate Node }
 WhichVertex := SELF.VertexList.FindNode(Where);
 { If it exists... }
 IF WhichVertex <> NIL THEN BEGIN
 REPEAT
 { Find Edges Connected to the Vertex }
 WhichEdge :=
SELF.EdgeList.FindConnected(WhichVertex);
 { If an Edge exists, remove it }
 IF WhichEdge <> NIL THEN
 SELF.EdgeList.RemoveNode(WhichEdge);
 UNTIL (WhichEdge = NIL);
 { Finally, remove the Vertex }
 SELF.VertexList.RemoveNode(WhichVertex);
 END;
 END;


PROCEDURE Graph.AddEdge(FromWhich, ToWhich: Vertex);
 VAR
 NewEdge: Edge;
 BEGIN
 { Create and initialize a new Vertex at Where }
 New(NewEdge);
 NewEdge.Initialize;
 NewEdge.SetFrom(FromWhich);
 NewEdge.SetTo(ToWhich);
 { Add new vertex to list, typecasting is required }
 SELF.EdgeList.AddNode(GraphNode(NewEdge));
 END;

PROCEDURE Graph.RemoveEdge(Where: Point);
 VAR
 WhichEdge: GraphNode;
 BEGIN
 { Find the appropriate Node }
 WhichEdge := SELF.EdgeList.FindNode(Where);
 { If it exists, remove it }
 IF WhichEdge <> NIL THEN
 SELF.EdgeList.RemoveNode(WhichEdge);
 END;

PROCEDURE Graph.SetVertexCenter(Which: Vertex; Where: Point);
 VAR
 anEdge: Edge;
 BEGIN
 { Move through the EdgeList finding Connected Instances}
 anEdge := Edge(SELF.EdgeList.FindConnected(GraphNode(Which)));
 WHILE (anEdge <> NIL) DO BEGIN
 anEdge.Erase; { Erase them and move on }
 IF anEdge.Next <> NIL THEN
 anEdge :=
Edge(anEdge.Next.FindConnected(GraphNode(Which)))
 ELSE
 anEdge := NIL;
 END;
 Which.SetCenter(Where); { Set the Vertex instance's center }
 { Move through the EdgeList finding Connected Instances}
 anEdge := Edge(SELF.EdgeList.FindConnected(GraphNode(Which)));
 WHILE (anEdge <> NIL) DO BEGIN
 anEdge.Draw; { Draw them and their vertices; move on }
 anEdge.FromVertex.Draw;
 anEdge.ToVertex.Draw;
 IF anEdge.Next <> NIL THEN
 anEdge :=
Edge(anEdge.Next.FindConnected(GraphNode(Which)))
 ELSE
 anEdge := NIL;
 END;
 END;

PROCEDURE Graph.MoveVertex(Start: Point);
 VAR
 Displacement: Point;
 NewCenter: Point;
 WhichVertex: Vertex;

 BEGIN
 WhichVertex := Vertex(SELF.VertexList.FindNode(Start));
 { If the vertex is moved, find the new center and
 place the Vertex and redraw affected Edges }
 IF WhichVertex <> NIL THEN
 IF DragRegion(WhichVertex.HotRegion, Start, Displacement.h,
 Displacement.v) THEN BEGIN
 NewCenter := WhichVertex.Center;
 AddPt(Displacement, NewCenter);
 SELF.SetVertexCenter(WhichVertex, NewCenter);
 END;
 END;

PROCEDURE Graph.LinkVertices(Start: Point);
 VAR
 FirstVertex: Vertex;
 LastVertex: Vertex;
 Stop: Point;
 BEGIN
 { Find the FromVertex }
 FirstVertex := Vertex(SELF.VertexList.FindNode(Start));
 IF FirstVertex <> NIL THEN BEGIN
 DragGrayLine(Start, Stop); { Drag a line around }
 { Find the ToVertex }
 LastVertex := Vertex(SELF.VertexList.FindNode(Stop));
 IF (LastVertex <> NIL) AND (FirstVertex <> LastVertex) THEN
 SELF.AddEdge(FirstVertex, LastVertex);
 END;
 END;

PROCEDURE Graph.Free;
 BEGIN
 SELF.EdgeList.Free;
 SELF.VertexList.Free;
 INHERITED Free;
 END;






[LISTING TWO]


Type
 Circle = Object (TObject) { The Circle class declaration }
 { Instance Variables }
 Center : Point ; { The Center of the Circle }
 Radius : Integer ; { The Radius of the Circle }
 { Methods }
 Procedure Draw ; { Draw the Circle }
 Procedure Erase ; { Erase the Circle }
 end ;
Procedure Circle.Draw ;
 Var
 theRect : Rect ; { Rectangular area of the Circle }
 Begin
 { Set up the Rectangle }

 theRect.top := SELF.Center.v - SELF.Radius ;
 theRect.left := SELF.Center.h - SELF.Radius ;
 theRect.bottom := SELF.Center.v + SELF.Radius ;
 theRect.right := SELF.Cener.h + SELF.Radius ;
 FrameOval (theRect) ; { Draw it }
 End ;
Procedure Circle.Erase ;
 Var
 theRect : Rect ; { Rectangular area of the Circle }
 Begin
 { Set up the Rectangle }
 theRect.top := SELF.Center.v - SELF.Radius ;
 theRect.left := SELF.Center.h - SELF.Radius ;
 theRect.bottom := SELF.Center.v + SELF.Radius ;
 theRect.right := SELF.Cener.h + SELF.Radius ;

 EraseOval (theRect) ; { Erase it }
 End ;






[LISTING THREE]

Program DrawtheCircle ;

< Circle's type declaration >
Var
 aCircle : Circle ;

< Circle's method definitions >
Begin
 new(aCircle) ; { Get a new instance }

 aCircle.Center.h := 50 ; { Set up instance variables }
 aCircle.Center.v := 50 ;
 aCirlce.Radius := 50 ;

 aCircle.Draw ; {Draw it and free it }
 aCircle.Free ;
End.





[LISTING FOUR]

Type
 Circle = Object (TObject) { The Circle class declaration }
 { Instance Variables }
 Center : Point ; { The Center of the Circle }
 Radius : Integer ; { The Radius of the Circle }
 { Methods }
 Procedure Draw ; { Draw the Circle }
 Procedure Erase ; { Erase the Circle }
 Procedure Free ; Override ; { The Free method needs changes }

 end ;

Procedure Circle.Draw ;
 as before
Procedure Circle.Erase ;
 as before
Procedure Cirlce.Free ;
 Begin
 SELF.Erase ;
 Inherited Free ;
 End ;





[LISTING FIVE]
[Listing Four]

Type
 DrawObject = Object (TObject) { The DrawObject class declaration }
 { Instance Variables }
 Location : Point ; { The location of the Object }
 { Methods }
 Procedure Draw ; { Draw the Object }
 Procedure Erase ; { Erase the Object }
 Procedure Offset (dh, dv : Integer) ; { Offset Object by dh, dv }
 Procedure Free ; Override ; { The Free method needs changes }
 end ;
 Circle = Object (DrawObject) { The Circle class declaration }
 { Instance Variables }
 Radius : Integer ; { The Radius of the Circle }
 { Methods }
 Procedure Draw ; Override ; { Draw the Circle }
 Procedure Erase ; Override ; { Erase the Circle }
 end ;
 Rectangle = Object (DrawObject) { The Rectangle class declaration }
 { Instance Variables }
 horSize : Integer ; { The Horizontal Size of the Rectangle }
 verSize : Integer ; { The Verical Size of the Rectangle }
 { Methods }
 Procedure Draw ; Override ; { Draw the Rectangle }
 Procedure Erase ; Override ; { Erase the Rectangle }
 end ;

{ --------------- The DrawObject Methods --------------- }
Procedure DrawObject .Draw ;
 Begin
 End ;
Procedure DrawObject .Erase ;
 Begin
 End ;
Procedure DrawObject .Offset (dh, dv : Integer) ; { Offset Object by dh, dv }
 Begin
 SELF.Erase ; { Erase Object at its present location }
 { Change the location of the Object }
 SELF.Location.h := SELF.Location.h + dh ;
 SELF.Location.v := SELF.Location.v + dv ;
 SELF.Draw ; { Draw Object at its new location }

 End ;
Procedure DrawObject.Free ;
 Begin
 SELF.Erase ;
 Inherited Free ;
 End ;

{ --------------- The Circle Methods --------------- }
Procedure Circle.Draw ;
 as before
Procedure Circle.Erase ;
 as before

{ --------------- The Rectangle Methods --------------- }
Procedure Rectangle .Draw ;
 Var
 theRect : Rect ; { Rectangular area of the Circle }
 Begin
 { Set up the Rectangle }
 theRect.top := SELF.Location.v ;
 theRect.left := SELF.Location.h ;
 theRect.bottom := SELF.Location.v + SELF.verSize ;
 theRect.right := SELF.Location.h + SELF.horSize ;
 FrameRect (theRect) ; { Draw it }
 End ;
Procedure Circle.Erase ;
 Var
 theRect : Rect ; { Rectangular area of the Circle }
 Begin
 { Set up the Rectangle }
 theRect.top := SELF.Location.v ;
 theRect.left := SELF.Location.h ;
 theRect.bottom := SELF.Location.v + SELF.verSize ;
 theRect.right := SELF.Location.h + SELF.horSize ;
 EraseRect (theRect) ; { Draw it }
 End ;


























December, 1989
WRITING FILTERS IN AN OBJECT-ORIENTED LANGUAGE


Object-oriented programming with Actor makes writing filter-type programs easy




Marty Franz


Marty Franz is a software specialist for Allen Testproducts in Kalamazoo
Michigan. He also is a free-lance author whose articles have appeared in
numerous computer magazines. Marty is the author of Object-Oriented
Programming featuring Actor published by Scott Foresman & Co., from which this
article is excerpted.


A filter is a program that copies one file into another file while processing
it in some way. Filters are a common type of program. For example, you can
think of a C language compiler as a filter, because it reads an input file (in
this case, a C source program), processes it (translates it), and creates an
output file (the object program). A listing generator is another example of a
filter: it reads an input file, processes it by paginating it, and outputs it
to a printer. Because filters are a common type of program, they have a common
structure. This article discusses how the facilities in an object-oriented
language, such as Actor, makes writing filter programs easy, using several
common classes and methods.


Object-Oriented Programming


Object-oriented programming (OOP) treats a program as a set of objects, with
the code and data that are needed to make an object exhibit appropriate
behavior fused together in the program. By dividing a program into objects
that embody both data and operations, the structure of the program more
closely represents the structure of the problem being solved. As a result,
object-oriented programs are more easily written, read, and maintained than
programs in a procedural language.
The creation of abstract data types is an important task in object-oriented
programming. This means the actual implementation of an object is encapsulated
by high-level operations. Objects therefore have a clear separation into a
public and private protocol. For example, a queue object that defines a public
protocol based on the operations of adding data to or removing data from the
queue. The queue may be written as an array with separate variables (called
"instance" variables) that maintain the first and last positions in the queue,
but how this is done is private to the queue object. By adhering to the public
protocol elsewhere in the program, you could later change the implementation
of the queue (to a threaded list, for example) and not change the rest of the
program that used the queue.
Programming in an object-oriented language involves creating objects and then
sending them messages to do things. Messages in Actor resemble function calls
in other programming languages, but they are not. The first parameter in the
message is actually the receiver object of the message. The receiver object
determines how to respond to the message. Different objects can respond in
different ways to the same message name, a property called "polymorphism."
Polymorphism allows many objects within a program to respond to the same
common protocol, which makes the program easier to understand. It also allows
you to write more general, reusable code, because you don't have to worry
about the types of objects you're dealing with as long as they follow the same
public protocol.
Within an object, methods are defined that detemine how the object will
respond to a given message. Methods in Actor resemble functions or procedures
in C or Pascal: They have local variables and use control structures such as
do-while and if-then-else. Blocks can be defined along with methods. You can
think of these as methods without names that can be treated as separate
objects and passed to objects as parts of messages.
In most object-oriented languages, objects are organized hierarchically into
classes. Classes descend from one another in a kind of family tree, with the
most general classes being at the top. All classes have a common ancestor
called the "grandfather class." In Actor, the grandfather class is the Object
class. The Object class specifies safe default behavior for all objects, such
as what to do when an unknown message is received (in this case, print an
error message). As subclasses descend from classes, they inherit the functions
of their parent class. This allows the programmer to use existing behavior in
his new objects, specifying only what's different for the new application.
This "programming by difference" means that less duplicate code needs to be
written.
Actor provides a rich set of classes as part of the base language, including
Boolean values (which are similar to Booleans in Pascal), files, Strings
(similar to strings in Basic but with a 16K character limit), and Ordered and
Sorted Collections for data structures such as arrays and queues. Actor also
provides a set of classes for conveniently creating and handling Microsoft
Windows objects, such as windows, dialog boxes, and scroll bars. I'll use the
text-based objects in Actor only to demonstrate how inheritance and classes
can make writing common utility programs much easier.


Plan of Attack


My goal in this article is to develop two filter programs using Actor classes
and methods. The first program is an object-oriented version of the Unix
utility grep, which searches for and prints all occurrences of a regular
expression in a file. (The word "Grep" is a twisted acronym for "globally
search for regular expression.") I'll then extend Grep in order to replace the
pattern with a new string. This second program is called Reviser. By using
Actor's inheritance wisely, the Reviser program will be extremely easy to
write once Grep is working.
By using a straightforward loop in a procedural language, you could simply
create a Grep function and a Reviser function and write both programs. This is
a direct, but not very compact solution. A close look at this problem reveals
the many similarities between the two programs. The most obvious is that the
main function for both of them is the same, as illustrated in Figure 1.
Figure 1: Grep and Reviser have similar main functions

 initialize regular expression
 open input file
 open output file
 while not at end of input file
 read line from input
 process it
 end while
 close input file
 close output file

For Grep, the output file is the display where you want to see the lines of
text that match the regular expression. For the Reviser program, the output
file is another file that holds the revised text. The Reviser differs from
Grep in its output file, and in the processing that's done to each line. Grep
searches the line for the expression and copies it to the output file if it
matches, while Reviser goes a step further and changes it before copying it to
the output file.
As I said earlier, both Grep and Reviser are examples of a filter, a program
that reads an input file and filters it by some sort of processing into an
output file. The input file can be filtered a character or a line at a time.
For the purpose of this article, a line at a time is sufficient.
Because this is such a common design for a program, it's worthwhile to first
develop a Filter class of objects that can be used to write the Grep and
Reviser programs with. Later on, you can add more types of filters using this
class and save a great deal of code.
Further, because the Reviser is so similar to Grep, you should be able to
inherit most of its functions from Grep, add the part that uses change(-), and
send the output to a file instead of the display. This will save even more
code.
This approach will save the time and effort of developing both the Grep and
Reviser from scratch. The first step is to develop the Filter class, and then
develop the Grep so it uses a Filter. Most of the Grep can then be inherited
in order to write the Reviser.


The RegularExpression Class


Before writing the Filter class, however, let's take a closer look at the
problem of searching for strings in the file. This is an important, common
function in both Grep and Reviser. Searching for literal strings is easy, but
also limiting at times. It would be useful to search not only for a literal
string, such as Fred, but also for the four-letter words that begin with the
letter "F." This is especially true when editing programs, because you might
have used "If" in one place and "if" somewhere else. You want to be able to
search for both by entering a single string expression.
A convenient notation for these types of string searches exists, and can be
implemented here. Borrowed from the Unix world, these are called "regular
expressions." You already use a regular expression when entering a wildcard
file specification in MS-DOS, such as DEL*.*.

This regular expression format will be a bit more limited than that of MS-DOS,
but still powerful, and will allow regular expression strings to contain
normal alphanumeric characters, plus the special characters (sometimes called
metacharacters) listed in Table 1.
Table 1: Some special characters

 Character Meaning
 ---------------------------------------------

 ? match any character
 % match the start of the string
 $ match the end of the string

For example, the regular expression F??? matches any string that started with
the letter F and was followed by three characters. The regular expression
%F???$ matches any string that consists only of these characters, nothing
else.
You can match sets of characters by using the metacharacters [] to enclose a
set. For example, the string F[aeiou]?? will match any four-character string
that starts with the letter F and is followed first by a vowel, and then by
any two characters. If the first character in the set is tilde (~), then the
remaining characters enclosed by [ ] are excluded from matching. For example,
the string F [~ri]?? matches any strings that start with F but are not
followed by r or i. Sets can include ranges of characters: For example, the
set [A-Za-z] matches all the upper and lowercase alphabetic characters.
When searching for strings that contain metacharacters, a backslash (\) will
specify one of the other metacharacters literally afterwards. Table 2 provides
the complete list of metacharacters supported.
Table 2: Characters in regular expressions

 Character Meaning
 -----------------------------------------------

 ? match any character
 % match the start of the string
 $ match the end of the string
 [ begin list
 ] end list
 ~ exclude set of characters from match
 \ take next character literally

This is an ambitious project, but the rewards will be worth the effort as the
use of this class can add efficient pattern matching to virtually any
text-based program. The key, however, is to make the pattern allow for fast
searching and compact representation.
Although it's tempting to use one of the predefined Ordered or Sorted
Collection classes in the Actor language when writing this class, this is slow
and wasteful of storage for large patterns and requires a lot of searching.
The best approach is to encode and embed the characters shown in Table 2 as
control characters in the pattern string. This allows the encoded patterns to
be stored anywhere a String can be stored, and searched rapidly with
character-by-character comparisons.
I call this class RegularExpression. It contains a String as an instance
variable (the encoded pattern), and a Boolean that determines whether the case
of an alphabetic character is significant. You must immediately write several
methods that access the instance variables without resorting to dot notation:
setPattern( ) allows you to send a previously encoded pattern to the
RegularExpression as the new pattern to use, and setCaseMatch( ) allows you to
set the Boolean instance variable that determines if upper and lowercase
versions of the same character are to be treated as equal. So you can save an
encoded pattern for later use, pattern( ) will return the pattern instance
variable. This allows a single RegularExpression instance to handle multiple
patterns.
The first hurdle is encoding the pattern. This is done by makePattern( ). It
loops through the String that is the regular expression, building another
String that is the encoded pattern. When it finds a metacharacter in the
source, it places a control character into the pattern that will be used later
for comparisons. These control characters are kept in a header file called
REGULARE.H (see Listing One, page 112) and can be changed later if necessary.
Building the pattern is a big job. It requires another message, fillSet( ),
which takes a set enclosed in [] and encodes it. Use an INCLUDE_SET or
OMIT_SET character for the set delimiter (depending on whether the characters
that follow match), followed by a character that is the number of characters
in the set, and then by the characters themselves. If a - (hyphen) was used to
indicate a range of characters, such as A-Z, this set is expanded and copied
into the pattern string. This is done when the pattern is built, and it won't
be repeated when matching is performed because this would slow down the
program considerably. A sample encoded pattern is shown in Figure 2.
Though this seems like a lot of trouble, the result is that you can compare a
String against a pattern in a running program by characters only, which are
efficiently handled. There is, however, a limit of 255 characters in a set
enclosed by [], because that's the largest count you can place in a single
character.
The comparison of a RegularExpression against a String is handled by match( ).
It calls aMatch( ), which matches the encoded pattern against every possible
substring in the source string. If case matching is enabled (case- Match is
non-nil), then match( ) converts the pattern and source to uppercase before
calling aMatch. The method aMatch( ) in turn calls oneMatch( ), which matches
a single character in the pattern against a single character in the string.
When a comparison fails, oneMatch( ) fails, and aMatch goes on to the next
string.
Because of the way the pattern is encoded, a control character always
determines what type of comparison to perform. Even a single character has an
A_CHAR control character preceding it, which directs oneMatch( ) to compare it
against a single character in the source. As a result of this property,
oneMatch( ) can use a select/case statement to handle each type of control
character. Two instance variables, arrow and cursor, hold the indexes of the
source string and the pattern during the matching process.
In Actor, constants such as A_CHAR can be placed in separate files for easier
maintenance, just as in C. For the RegularExpression class, the header file is
called REGULARE.H. The header file and class file for this class are in
Listing Two (page 112).
You can test this code by first loading the header file and the class file,
then typing:
 Fred:= new(RegularExpression, "F???"); match(Fred, "My name is Fred"); 11
If the pattern was found in the source string, I could have simply returned
true or false. Instead, match( ) returns the position in the source string
where the match occurred, or nil if it didn't. Including this extra touch
allows the source string to be edited. The other methods in RegularExpression
make the change( ) method easy to write and allow a string matching the
pattern to be replaced by another:
 change(Fred, "My name is Fred", "Sam"); My name is Sam
The search-and-replace function of a word processor or text editor becomes
easy to write with a RegularExpression class.


The Filter Class


It's important to be able to create and compare regular expressions before
moving on to the Filterclass. Looking at the structure of a filter program,
(including Grep and Reviser) we can see it follows the general design shown in
Figure 3. The phrases "initialize processing," "process line," and "terminate
processing" will vary from program to program. Also notice that the input and
output files from the previous iteration of the design have now become
objects. Polymorphism allows an object to be used in place of an input or
output file as long as it obeys the protocol of a File, adding even more
generality to the idea of a Filter.
Figure 3: Structure of a typical filter program

 initialize processing
 open the input object
 open the output object
 while not at end of input object
 read a line from input object
 process line
 end while
 terminate processing
 close input object
 close output object


In order to change the parts of the Filter that will vary from program to
program, blocks can be used to pass the class-independent processing parts to
the Filter. This design requires three blocks, one for each of the processing
phrases. The first is initBlock, because it receives control before the input
and output files are opened. Its job is to initialize anything needed by the
object using the Filter. The second block is processBlock. It receives the
string read from the input object, plus the output object, and does whatever
processing is needed by the Filter. The third is closeBlock, which is called
before the input and output objects are closed. It performs any cleanup and
termination the Filter needs.
The file for the Filter class is in Listing Three, page 114. The run( ) method
actually does the Filter's processing, according to the design in Figure 3. It
also calls checkError( ) at the proper times to ensure that the input and
output objects are opened correctly, and that the end of the input object
hasn't been reached.
Like the RegularExpression class before it, the Filterclass is not
particularly useful by itself -- it needs a Grep class to take advantage of
the Filter's generality.


The Grep and Reviser Classes


The Grep class will be a descendant of Object, because it must use behavior
from three other classes: Files, RegularExpressions, and Filters. If Grep is
made a descendant of one of these, the others will lend a lot of code and
unwanted behavior. Make it a unique class and use all three of the other
objects as instance variables.
The goal of the Grep class is to create a Filter and provide it with the
initBlock, processBlock, and closeBlock needed to search a file for a regular
expression, and to print the lines that contain it. Formulating the blocks is
easy: The blocks will simply call the start( ), process( ), and finish( )
messages in the Grep class. Deciding what these three messages should do takes
a bit more time.
The finish( ) method is the easiest. It simply prints a count of the lines
that were found in the file. It doesn't even have to do that, but this is
easily accomplished with an instance variable (called matches) that keeps a
count of the lines that match.
The start( ) method opens the input file that was passed as an argument when
the Grep object was initialized with init( ). It also prints a message that
tells the user that the program has started, and lists the name of the file
being searched.
The process( ) method takes the string read from the input file and compares
it against a RegularExpression pattern held in an instance variable. If it
matches, it prints the line with printLine( ) and increments the matches
instance variable. If not, it doesn't do anything.
Finally, the init( ) method is responsible for setting up the Filter and
RegularExpression objects and the blocks that the Filter will use. Two other
instance variables are initialized by init( ): fileName, which holds the name
of the file being searched, and pattern, the RegularExpression used as the
search pattern.
Three other messages in this class might puzzle you. They are: create( ),
close( ), and checkError. All return self. These are here because the Filter
object treats the Grep object as the output file. The output isn't being
written to a real file, but to a Grep object that checks the lines read from
the input file against the regular expression. For this deception to work, the
Grep class must support the File object's public protocol, even down to
checkError( ). Because these messages don't have to do anything (the Grep has
already been created, and it can't be closed), they can return self. This is a
common technique in object-oriented programming, another use for fall-through
methods and polymorphism: Making one object obey the protocol of another so
they can be used interchangeably. The Grep class file is shown in Listing
Four, page 114.
A new( ) method is defined in this class that will allow you to create and
initialize a Grep with a single message, with which you specify the name of
the file and the regular expression to search for as arguments:
 run(Grep,"README.TXT", "[Ww]hite-water");
The sample output from Grep is shown in Figure 4.
Reviser can now be written. Most of Reviser is already done, because it does
the same thing as Grep except the process( ) method. In a Reviser, the line
read from the input file must be searched using match( ), then changed if the
pattern is found, and then written to the output file in either case.
Therefore Reviser should be a descendant of Grep.
The output file needs instance variables for its name and object, and an
instance variable for the text that's going to be substituted when the pattern
is found. The other instances variables will be inherited from Grep.
The start( ) and finish( ) messages use the Grep versions of these messages
and do their own unique processing. Early binding notation indicates that the
Grep version of start should be called:
 ^start(self:Grep, inFile);
In the Grep class, the create( ), checkError( ), and close( ) methods didn't
have to do anything, because the output file was really a Grep. In Reviser
they have to function as File methods, and are passed on to the File class by
the newFile instance variable.
With inheritance, the code for the Reviser is even shorter than for the Grep.
The class file is shown in Listing Five, page 115. The Reviser inherits the
run( ) message from Grep, so it can be used with a single message:
 run(Reviser, "README.TXT", "[Ff]red", "Sam");
Though building the RegularExpression and Filter classes took a while, Actor's
polymorphism and inheritance can now be used to write tools such as Grep and
Reviser easily. Both of these utilities take much longer to develop in
procedural programming languages, because the functions of the Filter and Grep
classes cannot be easily separated.

WRITING FILTERS IN AN OBJECT-ORIENTED LANGUAGE_
by Marty Franz


[LISTING ONE]

/* header file for RegularExpression class */

#define ANY_CHAR 1 /* ^A */
#define A_CHAR 3 /* ^C */
#define BEGIN_STR 2 /* ^B */
#define END_STR 5 /* ^E */
#define INCLUDE_SET 19 /* ^S */
#define OMIT_SET 14 /* ^N */





[LISTING TWO]

/* *************************************************
 * REGULARE.CLS: RegularExpression class file *
 ************************************************* */

/* Class used to hold regular expressions for string
matching. */!!

inherit(Object, #RegularExpression, #(
pattern /* a pattern to match */
caseMatch /* nil, case doesn't matter */
arrow /* used to scan strings */

cursor /* used to scan patterns */), 2, nil)!!

now(RegularExpressionClass)!!

/* Create a new RegularExpression from String s. */
Def new(self, s re)
{ re := init(new(self:Behavior));
 re.pattern := makePattern(re, s);
 ^re;
}!!

now(RegularExpression)!!

/* Change every occurrence of string s matching the pattern to string t.
 Does not allow recursion: searching occurs after the new string has been
 substituted. The target string t is not a pattern, just another string,
 inserted into the source string at the point of the match. Returns the
 changed source string s */
Def change(self, s, t from, last, source)
{ source :=
 if caseMatch
 then asUpperCase(s);
 else s
 endif;
 from := 0;
 loop
 while from < size(source)
 begin last := aMatch(self, source, from);
 if last
 then s := replace(s, t, 0, size(t), from, last);
 from := from + size(t);
 else from := from + 1;
 endif;
 endLoop;
 ^s;
}!!

/* Set case matching. Converts pattern to upper-case, too. You should
 save the old pattern if you plan on toggling case matching a lot.*/
Def setCaseMatch(self, c)
{ caseMatch := c;
 if caseMatch
 then pattern := asUpperCase(pattern);
 endif;
 ^self;
}!!

/* Match the String s against the pattern. Calls aMatch() for each
 possible substring in the String. If caseMatch flag is set, then fold case
 to upper before doing the search. */
Def match(self, s)
{ if caseMatch
 then s := asUpperCase(s);
 endif;
 do(size(s),
 {using(i)
 if aMatch(self, s, i)
 then ^i;
 endif;

 });
 ^false;
}!!

/* Compare string s against pattern starting at position from. Makes
 successive calls to oneMatch. Returns index of matching character, or
 nil. Remember that oneMatch() advances the arrow in the source string. */
Def aMatch(self, s, from found)
{ arrow := from;
 cursor := 0;
 found := true;
 loop
 while found cand (cursor < size(pattern))
 begin
 if oneMatch(self, s)
 then cursor := cursor + patternSize(self, cursor);
 else ^false;
 endif;
 endLoop;
 ^arrow;
}!!

/* Match a single character passed as an argument to the set pointed to by
 the pattern cursor. Returns nil if not in the set, otherwise non-nil. */
Def locate(self, c setSize, fromCurs, toCurs, found)
{ setSize := asInt(pattern[cursor+1]);
 fromCurs := cursor + 2;
 toCurs := fromCurs + setSize;
 found := false;
 do(over(fromCurs, toCurs),
 {using(i)
 if pattern[i] = c
 then found := true;
 endif;
 });
 ^found;
}!!

/* Match a single character in the target string against a single
 character in the pattern. Returns nil or non-nil. Also advances arrow
 scanning target string */
Def oneMatch(self, s next, c)
{ next := -1;
 if arrow < size(s)
 then c := asInt(pattern[cursor]);
 select
 case c = ANY_CHAR
 is next := 1;
 endCase;
 case c = BEGIN_STR
 is
 if arrow := 0
 then next := 0;
 endif;
 endCase;
 case c = A_CHAR
 is
 if s[arrow] = pattern[cursor+1]
 then next := 1;

 endif;
 endCase;
 case c = INCLUDE_SET
 is
 if locate(self, s[arrow])
 then next := 1;
 endif;
 endCase
 case c = OMIT_SET
 is
 if not(locate(self, s[arrow]))
 then next := 1;
 endif;
 endCase
 endSelect;
 else /* at end of string, check for $ */
 if asInt(pattern[cursor]) = END_STR
 then next := 0;
 endif;
 endif;
 if next >= 0
 then arrow := arrow + next;
 endif;
 ^(next >= 0);
}!!

/* Get the pattern String from a Regular Expression. */
Def pattern(self)
{ ^pattern;
}!!

/* Set the pattern String in a RegularExpression. */
Def setPattern(self, s)
{ pattern := s;
 ^self;
}!!

/* Initialize a RegularExpression. */
Def init(self)
{ ^self;
}!!

/* Return the size of the Pattern from position p. */
Def patternSize(self, p c)
{ c := asInt(pattern[p]);
 select
 case c = A_CHAR
 is ^2;
 endCase;
 case c = ANY_CHAR cor c = BEGIN_STR cor c = END_STR
 is ^1;
 endCase;
 case c = 14 cor c = 19
 is ^asInt(pattern[p+1]+2);
 endCase;
 default ^1;
 endSelect;
}!!


/* Return a set of characters for inclusion in a Pattern. The first and
 last characters should be [ and ]. Returns INCLUDE_SET for set or OMIT_SET
 for exluded set, followed by a count of the characters in the set (just a
 single character), followed by the characters themselves. */
Def fillSet(self, s work, i, count, from, to)
{ i := 1;
 count := 0;
 if s[i] = '~'
 then work := asString(asChar(OMIT_SET));
 i := i + 1;
 else work := asString(asChar(INCLUDE_SET));
 endif;
 work := work+asString(asChar(1)); /* holds count later */
 loop
 while i < size(s) and s[i] <> ']'
 begin
 select
 case s[i] = '\'
 is work := work + asString(s[i+1]);
 i := i + 2;
 count := count + 1;
 endCase;
 case s[i] = '-'
 is from := asInt(s[i-1])+1;
 to := asInt(s[i+1])+1;
 do(over(from, to),
 {using(c) work:=work+asString(asChar(c));
 });
 i := i + 2;
 endCase;
 default work := work + asString(s[i]);
 i := i + 1;
 count := count + 1;
 endSelect;
 endLoop;
 work[1] := asChar(count);
 ^work;
}!!

/* Convert a normal String into a pattern String. Needs error checking,
 and to use more OOP technique. Note that it does not set the Regular-
 Expression's instance variable. */
Def makePattern(self, s work, c, i, j)
{ work := "";
 i := 0;
 loop
 while i < size(s)
 begin c := s[i];
 select
 case c = '?'
 is work := work+asString(asChar(ANY_CHAR));
 i := i + 1;
 endCase;
 case c = '%'
 is work := work+asString(asChar(BEGIN_STR));
 i := i + 1;
 endCase;
 case c = '$'
 is work := work+asString(asChar(END_STR));

 i := i + 1;
 endCase;
 case c = '['
 is j := indexOf(s, ']', i+1);
 work := work+fillSet(self, subString(s, i, j+1));
 i := j + 1;
 endCase;
 default work := work+asString(asChar(A_CHAR))+asString(c);
 i := i + 1;
 endSelect;
 endLoop;
 ^work;
}!!






[LISTING THREE]

/* *************************************************
 * FILTER.CLS: Filter class file *
 ************************************************* */

/* This is a class that makes possible filter programs,
like Listers, Greps, etc. in and out are objects that
respond to Stream or File protocols. initBlock, processBlock,
and closeBlock are executed when the filter is started,
running, and done, respectively. */!!

inherit(Object, #Filter, #(inObj
outObj
initBlock
processBlock
closeBlock
), 2, nil)!!

now(FilterClass)!!

/* Create a new Filter. */
Def new(self, input, output, b1, b2, b3 f)
{ f := init(new(self:Behavior), input, output, b1, b2, b3);
 ^f;
}!!

now(Filter)!!

/* Once a filter has been set up, run it. Call initBlock to
 initialize anything other than opening inObj and outObj.
 Read a line from inObj and call processBlock. When done,
 close both objects. Note that outObj is optional. Also
 not the outObj is the receiver of the blocks. This is because
 it's the object that's programmer-defined. */
Def run(self str)
{ eval(initBlock, outObj, inObj);
 open(inObj, 0);
 checkError(inObj);
 if outObj

 then create(outObj);
 checkError(outObj);
 endif;
 loop
 while str := readLine(inObj)
 begin eval(processBlock, outObj, str);
 endLoop;
 eval(closeBlock, outObj, inObj);
 close(inObj);
 if outObj
 then close(outObj);
 endif;
}!!

/* Initialize a new Filter. Fill-in all its instance
 variables. */
Def init(self, input, output, b1, b2, b3)
{ inObj := input;
 outObj := output;
 initBlock := b1;
 processBlock := b2;
 closeBlock := b3;
 ^self;
}!!






[LISTING FOUR]

/* *************************************************
 * GREP.CLS: Grep class file *
 ************************************************* */

/* This is a class that holds a single method that will analyze
 a file for regular expressions. This method is an example of a
 generic Filter, too. */!!

inherit(Object, #Grep, #(fileName
pattern
matches), 2, nil)!!

now(GrepClass)!!

/* Create and run Grep for a file and a pattern. */
Def run(self, file, expr g)
{ g := init(new(self:Behavior), file, expr);
 ^g;
}!!

now(Grep)!!

/* The Filter will try to close us. Make sure something
 safe happens. */
Def close(self)
{ ^self;
}!!


/* We need a checkError message because the Filter will try to
 call one. For a Grep this doesn't do anything. */
Def checkError(self)
{ ^self;
}!!

/* A dummy method, needed because the Filter will try to
 create() a Grep object with a mode value. Ours doesn't do
 anything. If it were a real file, it would be created. */
Def create(self)
{ ^self;
}!!

/* When done, print the number of matches that were found. */
Def finish(self, inFile)
{ printLine(asStringRadix(matches, 10)+" lines matched pattern.");
 ^self;
}!!

/* Process a single line. Compare it against the pattern.
 If it matches, print it. */
Def process(self, s)
{ if match(pattern, s)
 then printLine(s);
 matches := matches + 1;
 endif;
 ^self;
}!!

/* Start grep. Assign the filename to the TextFile */
Def start(self, inFile)
{ setName(inFile, fileName);
 printLine("");
 printLine("Searching file: "+fileName);
 ^self;
}!!

/* Open the input text file and search it for the expression.
 If found, print it. */
Def init(self, file, expr f, start, process, finish,
 input, output)
{ input := new(TextFile);
 setDelimiter(input, CR_LF);
 output := self;
 pattern := new(RegularExpression, expr);
 fileName := file;
 matches := 0;
 start :=
 {using(me, inFile) start(me, inFile);
 };
 process :=
 {using(me, theLine) process(me, theLine);
 };
 finish :=
 {using(me, inFile) finish(me, inFile);
 };
 f := new(Filter, input, output, start, process, finish);
 run(f);

 ^self;
}!!





[LISTING FIVE]

/* *************************************************
 * REVISER.CLS: Reviser class file *
 ************************************************* */

/* A Reviser is like a Grep, but it takes an additional
instance variable: the new text to be revised when the
pattern is found. Another filename is also required,
to hold the revised text. Finally, there is a handle
for the new file. */!!

inherit(Grep, #Reviser, #(newText
newFileName
newFile), 2, nil)!!

now(ReviserClass)!!

/* Create and run a Reviser. */
Def run(self, file, outFile, expr, text r)
{ r := init(new(self:Behavior), file, outFile, expr, text);
 ^r;
}!!

now(Reviser)!!

/* Initialize what's different about the Reviser from
 the Grep. */
Def init(self, file, outFile, expr, text)
{ newText := text;
 newFileName := outFile;
 ^init(self:Grep, file, expr);
}!!

/* Process a single line. Compare it against the pattern. If it matches,
 change it. For simplicity, this does not handle multiple substitutions in
 the same line. */
Def process(self, s)
{ if match(pattern, s)
 then write(newFile, change(pattern, s, newText) + CR_LF);
 matches := matches + 1;
 endif;
 ^self;
}!!

/* Set the name for the input file, then do whatever else
 a Grep does. */
Def start(self, inFile)
{ newFile := new(TextFile);
 setName(newFile, newFileName);
 setDelimiter(newFile, CR_LF);
 ^start(self:Grep, inFile);

}!!

/* Do a checkError on the newFile after creation. */
Def checkError(self)
{ ^checkError(newFile);
}!!

/* Create the new file. */
Def create(self)
{ create(newFile);
 ^self;
}!!

/* Close the new file */
Def close(self)
{ close(newFile);
 ^self;
}!!












































December, 1989
A HOME-BREW C++ PARSER


Designing your own C++ compiler begins with a good parser




John M. Dlugosz


John is a free-lance writer and software developer. His upcoming book:
Advanced C and Object-Oriented Programming, (MIS) will be released in
mid-1990. He can be reached at P.O. Box 867506, Plano, TX 75086; or on
CompuServe at 74066,3717.


When it comes to C++, still relatively few tools are available. This fact is
never more apparent than when you need a cross-reference utility. If, for
instance, you want to examine the function foo( ), you'll discover that the
function could appear anywhere in your program. In other languages, it's
simple to find a function by using a text-search utility, but functions may be
overloaded in C++ -- you might find seven functions named foo( ). You must
then know which of those functions might be in scope. In order to do so, you
need to know about the class hierarchy. In addition, you may need to know the
types of the parameters.
That is usually no big deal -- but in extreme cases, for example, you might
not know that adding a Windmill and an int generates a result of type
exception_node. In short, you have to parse the entire file down to the
primitive level, maintain a symbol table, and figure out class relationships
-- you practically have to compile the program!
For the purposes of this article, I'll describe a parser generator. A
table-driven parser, such as Paul Mann's LALR Parser Generator, takes a
statement of the grammar (see CPP.GRM, Listing One, page 116), and produces
tables. So, let's start with the grammar.


How's Your Grammar


The grammar specification is really a language-design language. As the source
is being parsed, the parser calls actions in the program. This can be viewed
as an event-driven machine. The parser sends messages to the rest of the
program. The way in which the grammar is specified determines what messages
are sent, when they are sent, and in what order. This is the key to using LALR
to build a full parser program.
Look at the grammar in the declaration class. A declaration starts with a
storage class, followed by a type and the item being declared. In addition, a
declaration contains asterisks, parentheses, and brackets, as well as keywords
such as near, far, const, and volatile. The basic form of the declaration is
shown below:
 Declaration -> StorageClass Type Declarator ;
A declarator can be a simple name, or it can be another declarator modified by
adding "( )" (function call), "[]" (vector), "*" (pointer), or "&" (reference)
symbols.
A declarator is put together in very much the same manner as an expression. A
declarator contains operators with different levels of precedence, along with
parentheses used for grouping purposes. So, the grammar for declarations can
be modeled on the grammar of expressions.


Productions


In order to achieve the right order of precedence (including the use of
parentheses for grouping purposes) you need three levels of productions. The
innermost level (Decl3) is the simple Dname. The next layer (Decl2) is postfix
operators "( )" and "[ ]." The outermost level (Declarator) is prefix
operators "*" and "&." To provide for grouping, the innermost layer can start
over again with a declarator in parentheses. The three levels of productions
are shown in Figure 1.
Figure 1: Productions used to determine the order of precedence

 Declarator
 -> Decl2
 -> ReachAttribute * Const/Volatile? Declarator
 => TypeModifier 3
 -> ReachAttribute & Const/Volatile? Declarator
 => TypeModifier 4

 Decl2
 -> Decl2 (Arg-Declaration-List) => TypeModifier 1
 -> Decl2 [ConstExp?] => TypeModifier 2
 -> Decl3

 Decl3
 -> Dname => Dname 1
 -> (Declarator)

ReachAttribute is either a near or far keyword, or is empty. Const/Volatile?
is the keyword const, the keyword volatile, neither, or both. ReachAttribute
appears immediately to the left of the item that it modifies (in this case a
pointer or reference). The const keyword appears just after a "*" (or "&") to
indicate that the item that is pointing is constant.
So, how does this triple-decker definition work? Consider the test case *x[].
The "*" is read, and matches the second line of the declarator. So x[] must
match the rest of that line, which is expecting another declarator. Because
the "*" is in the outer level and the "[]" is in the inner level, the "*" has
precedence. Now x[] doesn't look like anything under declarator, but a
declarator can be a Decl2. If this is a Decl2, it can be turned into a
declarator when it is finished. And sure enough, Decl3 [] is listed under
Decl2. The xmatches Decl3, and an action is triggered. Then the Decl2[] is
finished, and an action is triggered. Now the "*" declarator is finished, and
another action is triggered.
Consider the order in which the program receives its actions -- the name x,
the modifier Vector, and the modifier Pointer. Notice that the order of
precedence is built into the grammar, and is the only order possible.
The grammar sends the modifiers in the correct order to the program. The
program builds a representation of the object in response to these messages.



Keeping Track of It


All of the information about the source being processed is kept in nodes.
Different kinds of nodes are derived from a base class node (Listing Two,
NODE.HPP, page 116). One of the derived nodes is a type_node. A type_node can
refer to a simple type such as int or char, or to fancy types such as arrays
and pointers. These fancy types correspond to the four modifiers. As a result,
your code contains an array of <something>, a function that returns
<something>, a pointer to <something>, and a reference to <something>. If the
primary type is one of these four modifiers, the to_what field points to
another type node. In this way, complicated types are stored in a linked list.
The type_node:: print( ) function in NODE.CPP shows this structure clearly.
All of the actions called by the parser are in the file ACTIONS.CPP ( Listing
Three, page 118). The actions are defined with an ellipsis so that the C++
compiler will generate right-to-left passing with the caller clearing the
stack, which is what the parser (written in C) is expecting to call.
The working_type structure contains all of the information about the item
being parsed. The first message is for the storage class, which is stored for
future reference. The next item in working_type is the type, which is also
stored. A const (or volatile) keyword can appear on either side of the type.
(Notice that the action is called even if the production is empty.) As a
result, a pair of calls to the ConstVol action take place. ConstVol can be
called in several places, so these calls simply stack the values received. The
purpose for ConstVol is recognized later, and the values for this addition are
popped from the stack when it is ready for them.


Atoms


The first call in the declarator is a call to Dname. This call looks up the
last scanned token and stores the name. (In the case of abstract declarators,
the name can be empty.) The name is stored in an atom table, and the atom
number is used by the rest of the program. The atom table can be used in the
same way that an array of strings is used -- when the atom table is
subscripted with an int, it gives a string. But, when the atom table is
subscripted with a string, it gives the number! This second technique is known
as an "associative array" and is quite handy. When a string that is not in the
table is given, that string is added to the table and a new number is
assigned. The files ATOM.CPP (Listing Four , page 120) and ATOM.HPP (Listing
Five, page 120) provide more details.


See How They Run


As explained earlier, the type is stored as a linked list. Dname starts off a
declaration, and a new list is created. type_root is the head of the list, and
this_type is the tail end. Each modifier adds a link and moves the tail down.
When TypeModifier is called, the correct primary (function, array, pointer, or
reference) is stored in the node, along with other information specific to
that type. Then a new node is added, and the pointer is moved down to that
node. In the expression *x[], Dname gets a name of x and creates the first
node; TypeModifier(2) then sticks in array of, chains on another node in the
to_what field, and moves the this_type pointer to point to the new node.
Type_Modifier(3) then plugs in pointer to and adds on another link. The linked
list now reads (starting at the root) array of pointer to <garbage>.
When the declarator is finished, a call to the Declaration is made. This call
takes the base type (stored way back when with the call to StoreType) and
fills it into the current node. For example, if the line read static int
x*[];, then the result is array of pointer to int. Other stuff is filled in,
and the complete declaration is ready.
The declaration must be stored in a symbol table for later reference. The
type, along with the storage class (static) and the name (x), is sent to
store_thing( ). Basically, that's now the end of the story -- the parser
continues.
The storage class and type are still remembered, so declarators that are
separated with commas use the same values. Each declarator is sent to
store_thing( ) when the declarator is encountered. The parser starts over with
StoreStorage for a new line, or with Dname if several declarators are
contained in the same statement (that is, extern char a, *b, &c; share "extern
char").
In DEFINE.CPP (Listing Six, page 120), store_thing( ) handles the "thing"
after the "thing" is parsed. Up to now, the only kind of node used was a
type_node. Now, another kind of node is introduced -- a def_node. def_node
holds the complete definition, which is built from the type, the storage
class, and the name, and is stored in a list of def_nodes.
Look back at NODE.HPP, and notice that all of the nodes are in a taxonomy. A
list of nodes is also very useful. A list of base class nodes can hold any
kind of node, but it is better to use lists with pedigree. To do so, node_list
class is defined to handle the maintenance of a variable-sized array of nodes,
and a dummy class is derived from node_list for each required pedigree. This
dummy class contains an in-line function that accesses the correct element and
casts the elements to the proper type. The use of a macro allows a list (for
whatever type is needed) to be defined in a single statement. (Note the use of
the token-pasting operator in the macro definition.)
After the entire file is parsed, the list contains several entries. The
AllDoneNow action is a simple loop that prints out the list. The C++
definitions are spit back out in English-like phrases.


Nested Definitions


You may now be wondering about nested definitions. Consider a case such as int
(*p)(int x[]);. In this example the parameter list is parsed after Dname for
the pointer modifier, p, and before the function modifier. This itself
contains a definition. Look at the grammar for Arg-Declaration-List -- this
nonterminal grammar appears between the parentheses in the function modifier
(see Figure 2).
Figure 2: The grammar for Arg-Declaration-List

 Arg-Declaration-List
 -> Start-Nested-Type A-Decl-List? Elipses? End-Nested-Type

 Start-Nested-Type ->=> NestedType 1
 End-Nested-Type ->=> NestedType 0

 A-Decl-List?

 ->
 -> A-Decl-List

 A-Decl-List
 -> A-Decl-List, Argument-Declaration
 -> Argument-Declaration

 Argument-Declaration
 -> StorageClass Type_w/const Declarator => Declaration 2

Start-Nested-Type and End-Nested-Type are empty productions, and exist only to
call the NestedType action at the right place. This action saves the state of
the definition parse in progress, and then restores the state again. This
second step is very simple because all of the information is stored in a
structure. The structure is placed into a linked list and a fresh structure is
created. The end call puts the old structure back.
The argument-declaration is similar to Declaration, except that it is
separated with commas. The same action Declaration is called when an
argument-declaration is finished, with the parameter defined as 2 instead of
1. This change causes the completed definition to be placed into a different
list. This new list is also stackable, because a parameter may itself be a
pointer to a function. The NestedType( ) action calls parameter_ list( ),
which creates a new list to store the upcoming parameters. The end call then
pops the list off, restoring any list that was in progress. This completed
list is placed in a global variable, which is read by TypeModifier( ) and will
be the next action called.
This mechanism for nested types is also used for processing class members.
Notice that the Const/Vol stack works across nested types. Consider char
*const (*p)(char const* s);, which is a pointer to a function that returns a
pointer. The first const is used by the last modifier call, and stays on the
stack while the nested type is being parsed.


Conclusion



This program is a far cry from a full compiler, but the program's framework
can be easily expanded to include additional language elements. The first
feature that you should add to the program is the use of initializers. (The
grammar presented in Figure 3 contains initializers, but no actions are called
yet.) As you can see, a small amount of code can produce a very robust
program.
Figure 3: The test data for the parser

 struct C1 * p;
 int *p= "hello there!";
 int*names[];
 union FOO *p;
 const char * const *(*w)()[];
 char a,b,*c,&d;
 int foo (int a, char* b);


_A HOME BREW C++ PARSER_
by John M. Dlugosz


[LISTING ONE]

/* TOKENS. */

 <error>
 <identifier> => KW_SEARCH
 <operator> => OP_SEARCH
 <punctuator> => OP_SEARCH
 <number>
 <string>
 <eof>
 <type>

/* KEYWORDS. */

 auto break case cdecl char class const continue default delete do
 double else enum extern far float for friend goto huge if inline int
 interrupt long near new operator overload pascal private protected
 public register return short signed sizeof static struct switch this
 typedef union unsigned virtual void volatile while

/* OPERATORS. */

 &&
 < <= == > >=
 + - * / %
 ? ++ -- '->'
 ! ~ ^ '' & >> <<
 = <<= != %= &= *= += -= /= = >>= ^=

/* PUNCTUATORS. */

 '...' . , : ; [ ] { } ( ) ::

/* NONTERMINALS. */

Input -> File_and_tell <eof>

File_and_tell -> File => AllDoneNow 1 /* normal completion */

File -> Item File Item

Item -> Declaration

 /* or Definition. not in yet. */


/**************************************************************
To recognize a declaration, the storage class and type appear
once. They are remembered. Each declaration is seperated by
commas, and share the same type. The FinishedDeclarator calls
an action for each one found.
****************/

Declaration
 -> StorageClass Type_w/const Declarators ;

Declarators
 -> FinishedDeclarator
 -> FinishedDeclarator , Declarators

FinishedDeclarator -> Declarator Initializer? => Declaration 1

/*********************************/


Initializer?
 ->
 -> = Expression
 -> = { Expression-List }
/* -> ( Expression-List ) */

Expression-List
 -> Expression
 -> Expression Expression-List


StorageClass
 -> => StoreStorage 0
 -> static => StoreStorage 1
 -> extern => StoreStorage 2
 -> typedef => StoreStorage 3
 -> auto => StoreStorage 4
 -> register => StoreStorage 5

Type_w/const /* const may appear before or after the type name */
 -> Const/Volatile? Type Const/Volatile? => StoreBaseConstVol

Type
 -> char => StoreType 1
 -> signed char => StoreType 2
 -> unsigned char => StoreType 3
 -> int => StoreType 4
 -> short => StoreType 4
 -> short int => StoreType 4
 -> signed int => StoreType 4
 -> signed short => StoreType 4
 -> signed short int => StoreType 4
 -> unsigned => StoreType 5
 -> unsigned int => StoreType 5
 -> unsigned short => StoreType 5
 -> unsigned short int => StoreType 5
 -> long => StoreType 6

 -> signed long => StoreType 6
 -> unsigned long => StoreType 7
 -> float => StoreType 8
 -> double => StoreType 9
 -> long double => StoreType 10
 -> void => StoreType 11
 -> enum Tag => StoreType 12
 -> Class Tag => StoreType 13
 -> union Tag => StoreType 14

Tag
 -> <identifier> => StoreTag 1
 -> <type> => StoreTag 2

Class
 -> struct
 -> class

OverloadableOp -> * / = + /* and all the others */

Elipses? -> '...'

/* Declarations */

Declarator
 -> Decl2
 -> ReachAttribute * Const/Volatile? Declarator => TypeModifier 3
 -> ReachAttribute & Const/Volatile? Declarator => TypeModifier 4

Decl2
 -> Decl2 ( Arg-Declaration-List ) => TypeModifier 1
 -> Decl2 [ ConstExp? ] => TypeModifier 2
 -> Decl3

Decl3
 -> Dname => Dname 1
 -> ( Declarator )

Const/Volatile? /* const or volotile, neither, or both */
 -> => ConstVol 0
 -> const => ConstVol 1
 -> volatile => ConstVol 2
 -> const volatile => ConstVol 3
 -> volatile const => ConstVol 3


ReachAttribute
 -> => ReachType 0
 -> near => ReachType 4
 -> far => ReachType 8

Dname
 -> SimpleDname
 -> <type> :: SimpleDname

SimpleDname
 -> <identifier>
 -> <type>
 -> ~ <type>

 -> Operator-FunctionName

Operator-FunctionName
 -> operator OverloadableOp /* overload operator */
 -> operator <type> /* conversion operator */
 /* this should really allow any abstract type definition, not just
 a simple type name. I'll change it later */
 -> operator <identifier> /* ERROR production */


/* Argument list for function declarations */

Arg-Declaration-List
 -> Start-Nested-Type A-Decl-List? Elipses? End-Nested-Type

Start-Nested-Type -> => NestedType 1
End-Nested-Type -> => NestedType 0

A-Decl-List?
 ->
 -> A-Decl-List

A-Decl-List
 -> A-Decl-List , Argument-Declaration
 -> Argument-Declaration

Argument-Declaration
 -> StorageClass Type_w/const Declarator => Declaration 2

/* Expressions */

ConstExp?
 ->
 -> ConstExp

ConstExp -> Expression /* semantics will check */

Expression
 /* stub out for now */
 -> <identifier>
 -> <number>
 -> <string>






[LISTING TWO]


// the node class is central to date representation.
// Everything it knows is in a node.

enum node_flavor { //state the derived type from a node
 nf_base, nf_type, nf_def
 };

class node {

protected:
 node();
 virtual ~node();
public:
 node_flavor flavor;
 virtual void print();
 };

enum primary_t { type_void, type_char, type_int, type_long, type_float,
 type_double, type_ldouble, type_enum, type_class, type_union,
 type_pointer, type_reference, type_array, type_function };

class def_node_list; //forward ref

class type_node : public node {
public:
 type_node* to_what;
 type_node ();
 ~type_node();
 void print();
 unsigned flags;
 primary_t primary;
 node* secondary() { return to_what; }
 atom tag;
 def_node_list* aggr;
 void stuff_primary (int x, atom tagname);
 bool isConst() { return flags&1; }
 bool isVol() { return flags&2; }
 bool isNear() { return flags&4; }
 bool isFar() { return flags&8; }
 bool isUnsigned() { return flags&16; }
 };

class def_node : public node {
public:
 atom name;
 int storage_class;
 type_node* type;
 void print();
 def_node (atom n, int store, type_node* t);
 };

/* /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ */
/* lists of nodes */
/* /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ */

class node_list {
 node** list;
 int capacity;
 int count;
public:
 node_list();
 ~node_list() { delete list; }
 node** access (int x);
 int size() { return count; }
 void add(node* n) { *access(count++) = n; }
 };

#define create_list(TYPE) class TYPE##_node_list : public node_list { \

public:\
TYPE##_node*& operator[] (int x) { return *(TYPE##_node **)access(x); } }

create_list (type);
create_list (def);







[LISTING THREE]

/*****************************************************
File: ACTIONS.CPP Copyright 1989 by John M. Dlugosz
 the Actions called from the parser
*****************************************************/

#include "usual.hpp"
#include <stream.hpp>
#include "atom.hpp"
#include "node.hpp"
#include "define.hpp"

// #define SHORT return 0
/* short out actions for testing parser only.
 if something suddenly goes wrong, I can stub
 out all the actions to make sure I'm not walking
 on data somewhere. */
#define SHORT
 // the normal case.

/* /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ */

static char last_token[81];

void get_last_token()
{
/* copy last token from scanner into a nul-terminated string */
int len= 80; //maximum sig. length
extern char *T_beg, *T_end; //from the scanner
char* source= T_beg;
char* dest= last_token;

//cout << (unsigned)T_beg << " " << T_end;
//for (int x= 0; x < 5; x++) cout.put(T_beg[x]);

while (len-- && source < T_end) *dest++ = *source++;
*dest= '\0';
}

/* /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ */

/* in a declaration, a storage class is the first thing I get. This starts
 it off. Then a ConstVol, a StoreType, and a second ConstVol. The const
 or volatile keyword may appear before or after the type, with equal
 effect. The two bits are ORed together for the final result.
 After this, I get one or more calls to Declaration.

*/

/* /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ */

// the type I'm building
struct working_type {
 type_node* this_type; //the tail
 type_node* root_type; //the head
 int base_type;
 atom tag_name; // if base_type has a name
 atom name; //The name of the thing being declared
 int storage_class;
 int const_vol;
 working_type* next;
 } MainType;

working_type* Tx= &MainType;
/* this is accessed through a pointer because a declarator can be encounted
 while parsing another declarator. This lets me stack them. */

/* /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ */

static int const_vol_stack[50];
static int const_vol_stacktop= 0;

/* /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ */

int Declaration (int x,...)
{
/* values for x: 1- global or local def.
 2- parameters
 3- struct/union list
*/
SHORT;
/* This finishes it off. A complete declaration has been found. */
Tx->this_type->stuff_primary (Tx->base_type, Tx->tag_name);
Tx->this_type->flags = Tx->const_vol;
// build the 'thing' from the type_node and the name.
store_thing (Tx->root_type, Tx->name, Tx->storage_class, x);
// Tx->root_type->print();
// cout.put('\n');
return 0;
}

/* /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ */

int StoreBaseConstVol (int x,...)
{
SHORT;
// the first two calls to ConstVol apply here.
Tx->const_vol = const_vol_stack[--const_vol_stacktop];
Tx->const_vol = const_vol_stack[--const_vol_stacktop];
return 0;
}

/* /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ */

int StoreType (int x,...)
{

SHORT;
Tx->base_type= x;
return 0;
}

/* /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ */

int StoreTag (int x,...)
{
SHORT;
/* called when a struct, union, or enum is parsed. The tag is the last
 token read. After this call, the StoreType call is made with 'union'
 or whatever. */
get_last_token();
Tx->tag_name= atoms[last_token];
return 0;
}

/* /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ */

int StoreStorage (int x,...)
{
SHORT;
/* this is the first thing called */
Tx->storage_class= x;
return 0;
}

/* /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ */

int Dname (int x,...)
{
SHORT;
/* if x==1, the last token is the name of a thing being declared. If
 x==0, there is no name and this is an abstact declarator. Either
 way, build a type node and store the name. This overwrites the type
 node, as it will be the first thing called. */

if (x) {
 get_last_token();
 Tx->name= atoms[last_token];
 }
Tx->this_type= new type_node;
Tx->root_type= Tx->this_type;
return 0;
}

/* /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ */

int TypeModifier (int x,...)
{
SHORT;
/* 1 t() 2 t[] 3 *t 4 &t */

switch (x) {
 case 1:
 Tx->this_type->primary= type_function;
 // attach parameter list
 Tx->this_type->aggr= completed_def_list;

 break;
 case 2:
 Tx->this_type->primary= type_array;
 // >> attach size
 break;
 case 3:
 Tx->this_type->primary= type_pointer;
 Tx->this_type->flags = const_vol_stack[--const_vol_stacktop];
 break;
 case 4:
 Tx->this_type->primary= type_reference;
 Tx->this_type->flags = const_vol_stack[--const_vol_stacktop];
 break;
 }
Tx->this_type->to_what= new type_node;
Tx->this_type= Tx->this_type->to_what;

return 0;
}

/* /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ */

int ConstVol (int x,...)
{
SHORT;
/* 1-const 2-volatile 3-both */
const_vol_stack[const_vol_stacktop++]= x;
return 0;
}

/* /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ */

int ReachType (int x,...)
{
SHORT;
/* 0-default 1-near 2-far */
return 0;
}

/* /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ */

int NestedType (int x, ...)
{
SHORT;
working_type* p;
if (x) { //start nesting
 p= new working_type;
 p->next= Tx;
 Tx= p;
 }
else { //restore old type
 p= Tx;
 Tx= Tx->next;
 delete p;
 }
parameter_list (x); //pass on to DEFINE module
return 0;
}


/* /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ */
/* /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ */
/* /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ */

int AllDoneNow (int x, ...)
{
SHORT;
cout << "parser complete. \n";
for (int loop= 0; loop < global_stuff.size(); loop++) {
 global_stuff[loop]->print();
 cout.put ('\n');
 }
return 0;
}






[LISTING FOUR]

/*****************************************************
File: ATOM.CPP Copyright 1989 by John M. Dlugosz
 store strings
*****************************************************/

#include "usual.hpp"
#include "atom.hpp"

extern "C" void* malloc (unsigned size);
extern "C" void free (void*);
extern "C" void* realloc (void*, unsigned);

atom_storage atoms(16);

/* /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ */

atom_storage::atom_storage (int size)
{
count= 0;
capacity= size;
list= (char**) malloc (size * sizeof(char*));
}

/* /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ */

atom_storage::~atom_storage()
{
free (list);
}

/* /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ */

extern "C" int strcmp(char*,char*);
extern "C" char* strdup(char*);

atom atom_storage::operator[] (char* s)
{

for (int loop= 0; loop < count; loop++) {
 if (!strcmp(s, list[loop])) return loop; //found it
 }
if (count == capacity) { // make it bigger
 capacity += capacity/2;
 list= (char**)realloc(list,capacity*sizeof(char*));
 }
list[count]= strdup(s);
return count++;
}








[LISTING FIVE]


typedef int atom;

class atom_storage {
 char** list;
 int count;
 int capacity;
public:
 atom_storage(int size);
 ~atom_storage();
 char* operator[] (atom x) { return list[x]; }
 atom operator[] (char*);
 };

extern atom_storage atoms;







[LISTING SIX]

/*****************************************************
File: DEFINE.CPP Copyright 1989 by John M. Dlugosz
 deal with definitions once they are parsed
*****************************************************/

#include "usual.hpp"
#include <stream.hpp>
#include "atom.hpp"
#include "node.hpp"
#include "define.hpp"

bool local_level= FALSE;

def_node_list global_stuff;
struct p_list_struct {

 def_node_list* l;
 p_list_struct* next;
 };
static p_list_struct *p_list;
def_node_list* completed_def_list;

/* /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ */

void store_thing (type_node* t, atom name, int storage_class, int param)
{
/* 'param' is passed through from the grammar. If 1, this is a declaration
 for a local or global object. If 2, this is part of a parameter list */
def_node* n= new def_node (name, storage_class, t);

// file it away somewhere
switch (param) {
 case 1:
 if (name == -1) {
 // >> I need to get a standard error reporter
 cout << "abstract declarator ignored\n";
 }
 global_stuff.add (n);
 break;
 case 2:
 // >> check it over
 p_list->l->add(n);
 break;
 }
}

/* /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ */

void parameter_list (int x)
{
p_list_struct* p;
if (x) {
 p= new p_list_struct;
 p->next= p_list;
 p->l= new def_node_list;
 p_list= p;
 }
else {
 p= p_list;
 p_list= p_list->next;
 completed_def_list= p->l;
 delete p;
 }
}














December, 1989
WRITING CORRECT SOFTWARE WITH EIFFEL


Assertion and exception techniques can aid in class correctness




Bertrand Meyer


Bertrand is the president of Interactive Software Engineering and is the main
designer of the Eiffel language. His book, Object-Oriented Software
Construction, was published by Prentice Hall in 1988. He can be reached at
805-685-1006, or through e-mail as Bertrand at Eiffel.com.


My aim in designing Eiffel was to produce a major programming language for the
1990s, catering to the needs of those software engineers willing to do what it
takes to produce high-quality software. A key aspect of Eiffel, which makes it
original in the world of object-oriented languages, and in the world of
programming languages at large, is its strong emphasis on techniques that help
produce highly reliable software.
Although, there are many more aspects to Eiffel (including those described in
my book Object-Oriented Software Construction, Prentice-Hall, 1988) the
reliability features deserve a presentation of their own. That is the focus of
this article. I will show how it is possible to write software that
programmers (and users) can place a much higher degree of confidence in than
that written with traditional techniques. In particular, I will discuss the
all important notion of assertion -- the specification element included within
the software itself. This will lead to a systematic view of exception
handling, and a look at techniques (such as those offered by Ada) that I find
somewhat unsafe.


Why All the Fuss?


The issue is simple. It is great to have flexible software that is easy to
build and easy to maintain, but we also need to be concerned that the software
does what it is supposed to do.
From reading most of the object-oriented literature, one would think this is
not a problem. Correctness concerns are hardly ever mentioned. Actually, it is
unfair just to pick on object-oriented programming. Take any standard
textbooks you have on programming, algorithms, data structures, and similar
topics. See how many of them list "correctness," "reliability," "invariant,"
or "assertion" in their indexes. I have quite a few textbooks on my shelves,
but could not find many that passed this simple test.
This apparent disregard for correctness issues cannot last forever. Even
barring the occurrence of a major catastrophe resulting from faulty software,
sooner or later someone will call the software engineers' bluff and ask them
exactly why they think their systems will perform as announced. It is
difficult to answer that question convincingly given the current state of the
art.
Eiffel won't provide the magical key to the kingdom of software reliability.
No existing method or tools will. I do believe, however, that the Eiffel
techniques are an important step in the right direction.
If you are expecting a sermon telling you to improve your software's
reliability by adding a lot of consistency checks, you are in for a few
surprises. I suggest that one should usually check less. According to
conventional software engineering wisdom, "defensive programming" is
considered to be a programmer's best shot at reliability. I believe that
defensive programming is a dangerous practice that defeats the very purpose it
tries to achieve. To program defensively is one of the worst pieces of advice
that can be given to a programmer.
That more checking can make software less reliable may seem foolish. Remember,
though, that in science common sense is not always the best guide. If you have
ever hit a wooden table with your fist, you probably found it hard to believe
the physics professor who told you that matter is a set of tiny atoms with
mostly nothing in-between.


Expressing the Specification


The ideas that help achieve correctness in Eiffel are much older than Eiffel
itself. They come from work on program proving and formal specification. Oddly
enough, research on these topics has remained estranged from most "real-world"
software development. Part of the reason, at least in the United States, is
the widespread view that formal specification and verification are specialized
research topics whose application is mostly relevant to "mission-critical"
software. Correctness, however, should be a universal concern. Eiffel looked
at specification and verification work to see how much of it could be made
part of a standard programming methodology.
Eiffel is a production language and environment. It is not a research vehicle.
Eiffel relies on the technology of the last part of the twentieth century. It
has to work now. This means that no miracles can be expected. In fact, the
techniques are modest and almost naive. They are the result of an engineering
trade-off between what is desirable in an ideal world and what can
realistically be implemented today. But they make a big difference and I can't
understand why no widespread language, other than Eiffel, has made any
significant attempt in a similar direction.
The basic idea is rather trivial. Correctness is a relative notion. No
software element is correct or incorrect per se; it is correct or incorrect
only concerning a particular specification, or statement of its purpose.
Correct elements cannot be written unless the time is taken to express all or
part of this specification.
Writing the specification will not guarantee that it is met. But the presence
of a specification, even one that is only partially spelled out, goes a
surprisingly long way toward helping produce elements that satisfy their
correctness requirements.
This idea was captured by the title of an article by Harlan Mills, then of
IBM, published in 1975: "How to Write Correct Programs and Know Why." If you
are a serious software engineer, you don't just want to hope that your
programs are correct because you have been careful, and done a lot of testing,
and so on. You need precise arguments that document the correctness of your
software.
In Eiffel, such arguments are expressed as assertions -- elements of formal
specification that can be attached to software components, classes and their
routines.


The Contract


Let's look at routines first. A routine is the description of some computation
on the instances of a class, made available by that class to its clients (to
other classes relying on its services). How do we specify the purpose of a
routine?
The view I find most helpful is that a routine provides clients with a way to
contract out for a certain task that the client's designer finds advantageous
not to implement within the text of the client. This is the same way that we
humans at times contract out for part or all of a task that we need to
perform.
Human contracts have two important properties:
Each party expects some benefits and is prepared to incur some obligations in
return. What is an obligation for one party is a benefit for the other.
The obligations and benefits are spelled out in a contract document.
Figure 1 illustrates an example of a contract between a publisher and an
author. The author's obligation is to bring in a manuscript before March 1st.
The benefit to the author is that the manuscript will be published before May
1st. The publisher's obligation is to publish the manuscript before the second
date.
The publisher is not bound by any obligation if the author violates his part
of the deal. In such a case the publisher may still publish the manuscript,
but does not have to. The situation is outside of the contract's bounds.


Routine as Contract


Specifying a routine is based on the transposition of these observations to
software. First, we need the equivalent of the contract document. It bewilders
me that no such concept exists in standard approaches to software
construction.
The specification consists of two parts:

The precondition of a routine states the obligations of clients, which are
also the benefits for the routine itself.
The postcondition states the obligations of the routine, which are also the
benefits for the clients.
The precondition is a set of initial conditions under which the routine
operates. Ensuring the precondition at the time of any call to the routine is
the clients' responsibility.
The postcondition is a set of final conditions the routine is expected to
ensure. Ensuring the postcondition at return time (if the precondition was met
on entry) is the routine's responsibility.
The concept of a contract is one of the most useful aids to understanding
Eiffel programming. The role of contracts in Eiffel can be compared to what
message passing represents in Smalltalk.
Figure 3 illustrates this idea. The function intersect1 in a class CIRCLE
(assumed to be part of some graphic package) returns one of the two
intersecting points of two circles (see Figure 2). We will look at how to
associate the precondition and the postcondition to the text of the function
in the actual Eiffel class. In this example:
The precondition is that the two rectangles should intersect.
The postcondition is that the function result is a point that is on both
circles.


Contract Variants


This is not the only possible specification. Programmers may feel uneasy about
the just mentioned "demanding" form of the routine, which only works in some
cases. Instead, a tolerant version implementing a different contract may be
designed. For example:
There is no precondition. More precisely, the precondition is true, and
automatically satisfied by any client. Here, the routines will be applicable
in all cases.
The postcondition is more difficult to express in this case. Either the two
circles intersect and the function result is a point on both circles; or the
two circles do not intersect, the function result is an arbitrary point, and
an error message has been displayed somewhere. The awkwardness of stating the
postcondition in such a way is the first sign of why "demanding" versions are
often better.


Expressing the Contract


Let's see how the preconditions and postconditions will be integrated. Listing
One, page 125, shows what a class CIRCLE might look like. Assume the
availability of a class POINT describing points, and a function distance, such
that p1.distance (p2) is the distance between any two points (p1 and p2).
Result is a predefined variable which, in a function, denotes the result of
that function. Create is the initialization procedure. It is automatically
exported.
The precondition of a routine, if any, is given by the require clause. The
postcondition is given by the ensure clause. Preconditions and postconditions
are assertions -- logical constraints expressed as one or more Boolean
expressions, separated by semicolons. They are essentially equivalent to
Boolean ANDs, but allow assertion components to be identified individually.
These components can be tagged for even better identification. For example,
consider Listing Two, page 125.
Note that the first clause in this precondition (as well as clauses in the
preconditions of inside and outside) express that the argument must be
nonvoid. Void is a predefined language feature expressing whether there is an
object associated with a certain reference.


Uses of Assertions


Along with invariants (discussed later), preconditions and postconditions play
a fundamental role in the design of Eiffel classes. They show the purpose of
routines and the constraints on their uses. A brief look at any well-designed
set of Eiffel classes shows how wide their application is. The Basic Eiffel
Library, which covers fundamental data structures and algorithms, is an
example of a set of carefully designed classes that come fully loaded with
expressive assertions.
The first application of assertions, perhaps the most powerful, is as a
conceptual design aid for producing reliable software. In this role,
preconditions and postconditions directly support the goal stated earlier:
Writing correct software and knowing why it is correct. When a routine is
written, its goal (contract) is expressed. If this goal cannot be expressed in
a formal way, it should still be expressed as formally as possible.
Documentation is another key application of assertions. One of the most
pervasive myths of software engineering literature is the idea that
documenting software is a worthy goal. Instead, documentation should be viewed
as an evil, made necessary by the insufficient abstraction level of current
tools, techniques, and languages. It is an evil not just because documentation
is tedious to produce, but also because it is almost impossible to maintain
the consistency of a software system with its documentation throughout the
system's evolution. Incorrect or out-of-date documentation is often worse than
no documentation at all.
In an ideal world, software should be self-documenting, with no need for
outside documentation. Failing this programmer's Eden, we should strive to
have as little need for external documentation as possible. Documentation
should be deduced from the software itself. "Self-documenting software" does
not mean that the software is its own documentation. Instead, self-documenting
software should contain part, or (ideally) all, of its documentation,
corresponding to various levels of abstraction, which can be extracted by
automatic tools.
Preconditions and postconditions play a key role because they document the
essential properties of routines: What each routine expects and what each
ensures in return. The Eiffel environment provides an automatic tool that
yields the documentation of a class based on its assertion. This tool, the
class abstracter, is implemented by a command called "short." Applying short
to a class yields the description necessary to determine whether the class can
be used in a certain situation, and, if so, how to use it effectively.
The result of short applied to class CIRCLE would be of the form shown in
Listing Three, page 125.
As shown in this example, short keeps, as a complement to formal assertions,
the natural language header comments of routines, if present, at a
well-defined place. Only exported features are kept by short.
short provides documentation "for free" -- it is extracted from the software.
short is the major tool for documenting Eiffel classes. A companion tool,
good, produces high-level system documentation in graphic form, showing the
class structure with client and inheritance relationships. Remember, though,
that short is meaningless without the presence of assertions in the language.


Invariants


Preconditions and postconditions can be used in a non-object-oriented context.
Another use of assertions that is inseparable from the object-oriented
approach is the class invariant. This is an optional clause of Eiffel classes.
An invariant is a consistency constraint that applies to all instances of the
class.
In the CIRCLE example, the invariant clause might state the following
assertion:
 radius >= 0;
 inside (center)
In larger examples the invariants can be much more extensive.
Invariants can be viewed as general clauses that are implicitly added to all
contracts of a certain class, without being expressly repeated for each of
these contracts. The precise definition of the class invariant is that it is
an assertion that:
Must be ensured by the Create of the class
Must be preserved by every exported routine of the class
In principle, we could do away with the invariant by adding its clauses to the
precondition and postcondition of every exported routine, and to the
postcondition of the Create. But, besides making these assertions unduly
repetitive, this would be losing sight of the role of the invariant as a
global integrity constraint on the class, independent of a particular routine.
The two properties used earlier to define the invariant imply that the
invariant is satisfied in all observable states in the life of every instance
of the class. Observable states are those immediately following the Create,
and before and after application of exported routines. The life of a typical
object is pictured in Figure 4, with observable states marked as square
blocks. The idea of an observable state is important in the context of
parallel programming.
In spite of its name, an invariant is not necessarily satisfied at all times.
It may be temporarily violated during execution of exported routines, so long
as it is restored for the next observable state.
An invariant captures the semantic properties of a class, independently of its
current implementation, by a set of attributes and routines. These properties
must be understood in a software engineering context in which software is
always subject to change. Invariants can help bring some order to a constantly
changing environment by expressing what does not change in a class -- the
basic semantics of the class.
Invariants can play a major part in establishing a scientific basis for
software activities that currently rest on a rather shaky basis: Quality
assurance, regression testing, and maintenance. Because an invariant expresses
the essential semantics of a class that should be preserved through successive
modification and extension, it provides a framework for making QA and
associated activities more systematic.


Limitations of Assertions



The Eiffel assertion techniques are only partial. The assertion sublanguage is
based on Boolean expressions with some extensions. Sometimes more is needed,
such as first-order predicates. In the CIRCLE class it would be nice to have
the invariant express that no point can be both inside and outside the circle,
or that any such point must also be "on" the circle. The notation for this
could be:
 for p: POINT then
 inside (p) and outside (p)
 implies on (p)
 end
This is not possible in current Eiffel, although properties involving
quantifiers ("for all," "there exists") can sometimes be expressed through
Boolean expressions involving function calls. These function calls require
some care. Other limitations of assertions are due to the reference-based
dynamic model used for objects.
The mechanism is the result of an engineering trade-off. Though limited,
assertions are a tremendous asset in Eiffel programming.


Assertions and Inheritance


Assertions also play an important role in the context of inheritance.
Invariants are always inherited. When a routine is redefined, its precondition
may be weakened, but not strengthened. Its postcondition may be strengthened
but not weakened. To understand these rules, the contracting metaphor must be
viewed in the context of inheritance, redefinition (subcontracting), and
dynamic binding.


Monitoring Assertions


The question of what happens when an assertion is violated (such as if
intersect1 is called on two circles that do not intersect) is secondary. The
main question is: How can we, as responsible software professionals, make sure
that we produce software that is correct?
The tendency to reverse the priorities and ask the secondary question first is
a sign of how insecure most of us in the software engineering profession feel
about our techniques and tools. This article won't reverse this situation.
Still, we must get our priorities straight.
The answer to what happens when an assertion is violated depends on how you
have compiled your class. If you have made the effort of spelling out the
mental hypotheses that underlie the correctness of your software, you could
expect a theorem prover to check the software against these hypotheses.
Unfortunately, this is beyond today's technology. The next best thing to
static proof is run-time monitoring. If you compile a class under the
ALL_ASSERTIONS mode, all assertions (preconditions, postconditions,
invariants) are checked at the appropriate times during execution. If one is
found to be violated, an exception is triggered. Unless you have made explicit
provisions to handle it, the exception will result in program termination with
a clear message identifying the context of the failure.
There is never a good reason to compile a class under any option other than
ALL_ASSERTIONS, except performance. If you are sure your software is correct
and do not want to incur the overhead of checking, use the NO_ASSERTION_CHECK
mode. If a bug does remain, though, you are on your own. The default is an
intermediate mode, which generates code that checks preconditions only.
Switching modes may be needed a number of times during development. This
switch is easy. Only the last stage of compilation is repeated for the
corresponding class.
Run-time monitoring of assertions provides a powerful debugging mechanism.
Assertions are a way to make explicit the otherwise implicit mental
assumptions that lie behind our software. It is typical for a bug to cause one
of these assumptions to be violated. When this occurs, run-time monitoring
will catch the violation. This debugging technique takes on its full meaning
in the object-oriented context. I used it when using the Algol W compiler in
the seventies. Its superiority over usual debugging methods is hard to imagine
until you have actually applied it.


Defensive is Offensive


If a routine has a precondition p, defensive programming would mean that the
text of the routine should test again for p, in case the client forgot. For
instance, consider Listing Four, page 125.
The form as shown in Listing Four is never acceptable. It is a sloppy style of
programming in which responsibility for ensuring various consistency
conditions (contract clauses) have not been clearly assigned. Because the
contract is unclear, the scared programmer includes redundant checks "just in
case." This is a self-defeating policy. Complexity is the single, worst enemy
of software reliability. The more redundant checks, the more complex the
software becomes, and the greater the risk of introducing new errors.
Reliability is not obtained by cowardly adding even more checks, but by
precisely delineating whose responsibility it is to ensure each consistency
requirement. A party in a contract may fail to meet the requirement imposed on
it. This is precisely what a bug is. The solution, however, is not to make the
software structure more complex by introducing redundant checking, which only
makes matters worse. For fault-tolerant design, you should be able to rely on
a general-purpose run-time checking mechanism. In Eiffel, this mechanism is
the monitoring of assertions as described above.
With redundant checking being unacceptable, we still face a choice between the
"demanding" (strong precondition) style and the "tolerant" (no precondition)
style, with the intermediate spectrum. Mathematically, tolerant routines
represent total functions and demanding routines represent partial functions.
Which one to use depends on the circumstances. The closer a routine is to
uncontrolled "end users," the more tolerant it should be. But even with
general-purpose library routines, there is a strong case for demanding
routines.
With a strong precondition, a routine can concentrate on doing a well-defined
job and doing it well, rather being concerned with other things. The
intersect1 routine becomes a mess if it isn't assumed that the circles do
intersect. Tolerant routines must address user interface concerns for which
the routines do not have the proper context. The intersect1 routine must
address problems of geometrical algorithmics (computing the intersection of
two intersecting circles in the best possible way). It is difficult to
reconcile these two aspects in a single routine. The solution that will ensure
reliability more certainly than blindly checking all constraints all the time,
is to separate the checking and the computation.
Conventional wisdom, which says "never assume anything, anywhere," is wrong
and dangerous. Its pervasiveness can only be explained by the absence of any
notion of contract in standard approaches to programming. If clients have no
precise specification of the conditions they are supposed to observe, they
can't be trusted to observe these conditions and there is no choice but to
include as many consistency checks as possible. In a systematic approach to
software construction, however, the contract is clearly and adequately
expressed, independently of its implementation, through assertions. By using
the short command to let client designers see this contract, you can
concentrate on doing your job rather than checking theirs.
Considered in the perspective of other engineering disciplines, the often
recommended ban on "partial" routines seems absurd. If you ask an electrical
engineer to design an amplifier that will work for any input voltage, or a
mechanical engineer to build a bridge that will hold any load, they will laugh
at you. Any engineering device has preconditions. There really is no good
reason why software routines should be required to be total.
The reference to electronic components is not coincidental. One of the most
exciting advantages of object-oriented techniques is the ability to work from
libraries of standardized, off-the-shelf, reusable components. These
components are similar to hardware components used in electrical engineering.
These libraries cannot be successful unless the components are specified in a
precise and standardized way. Trying to sell a class without its invariant,
preconditions and postconditions is like trying to sell an amplifier without
its engineering specs.


Programming by Prayer


Assertions are not a way to program the handling of special cases. An
exception violation is not an expected situation that you want to handle
separately from the others -- it is the manifestation of a bug. To handle
special cases, there is not much substitute for what you learned on day two of
Introduction To Programming 100 -- the if ... then ... else construct.
There seems to be another pervasive myth in the industry that one can forget
about special cases through a form of faith healing. This can be called
"programming by prayer." In Ada, the sacred word is raise. Whenever you
encounter a situation that threatens to disrupt the spiritual harmony of your
program, kneel down and say, raise some_exception and a saint or angel will
come and take your worries away.
It doesn't work this way. The "angel" has to be programmed, and usually by
you. Postponing a problem does not solve it.
In Ada, after a raise, a chain of calls that led to the exception is explored,
in reverse order, until a block is found that includes an exception clause of
the form:
 exception
 when
 some_exception=>some_action;
 when
 other_exception=>other_action; ...
One of the when branches names the current exception. Then the code
some_action is executed and control returns to the handling block's caller.
If your aim was to make your software simpler by separating the processing of
"normal" and "special" cases, you will be disappointed. Special cases will not
go away through the raise attempt at absolution. Such as old sins, they will
come back to haunt you in your exception clauses. In the program text, such
clauses are far away from the source of the exception. They usually lack the
proper context to deal with the exception.
There are two cases of exception handling. One is when the exception must be
handled identically for all calls of the routine. This type of exception is
much better handled by an if ... then ... else ... clause in the routine
itself. In other words, the routine should be made more tolerant.
The second is when the handling of the special case is different for each
client. This can be achieved by protecting each call with an if ... then ...
else. The routine itself remains demanding. In either case no special control
structure is needed.


Exceptions



Once the naive faith in exceptions as exorcism has been dispelled, there is
still room for an exception mechanism. Exceptions should not be used as
control structures. They have no advantage over standard control structures,
and have many drawbacks. Some mechanism is needed however, to deal with an
operation that might fail in such a way that it is difficult or impossible to
check for with a standard control structure. Following are three main
examples:
1. Bugs. By definition, a bug is unexpected. If you were able to test for its
occurrence, you would correct the bug in your software, not handle it at run
time. If, in spite of your best efforts, a bug does occur, you still want the
ability to recover from it somehow at run time, even if only to terminate the
execution gracefully.
2. Uncheckable consistency conditions. Some preconditions may be impossible to
check as part of an if ... then ... else, either because they are too complex
to express formally, or because the applicability of an operation can only be
ascertained by attempting the operation and seeing if it fails. For example, a
write to disk operation may fail, but it is not useful to ask first and then
write. The only way to know if you can write is to attempt to write. Then, if
something goes wrong, you must be able to recover. Another example, in an
interactive system, is the implicit precondition that the user will not hit
the BREAK key. Obviously, you cannot test for the occurrence of such events.
3. Impractical to check before each call. These are operations for which
expressible preconditions exist in principle, but for which it is impractical
to check before each call. For example, few programmers want to protect every
addition by a test for non-overflow, or every object allocation (Create) by a
check that enough memory remains. As in the previous case, but for practical,
rather than theoretical reasons, you want to be able to attempt the operation,
proceed as if everything went all right, but recover if something goes wrong.
These three cases are ones for which exceptions are needed. They are not
"special" or expected algorithmic cases, but abnormal situations that cannot
be properly handled by standard algorithmic techniques.
In Eiffel, an exception occurs in the following situations:
Assertion violations (if monitored). The violation of an assertion is always a
bug. A violated precondition reflects a bug in the client; a violated
postcondition reflects a bug in the routine.
Hardware or operating system signals, such as arithmetic overflow, memory
exhaustion, and so forth.
An attempt to apply an operation to a non-existent object (Void reference).
Failure of a called routine.
The range of such exceptions is much less extensive in Eiffel because of the
disciplined nature of the language. In particular, the static typing mechanism
of Eiffel implies that for a correctly compiled system there is no exception
for a "feature applied to an object that cannot handle it (a message sent to
an object that cannot process it)."


Dealing with Exceptions


What happens when an exception occurs? The Ada answer is dangerous. Because
you can do essentially anything you like in a when clause, there is no
guarantee that you will achieve anything remotely resembling the original
purpose of the routine that failed.
To obtain a satisfactory solution, it is necessary to think in terms of the
contract that a routine is meant to ensure. The routine initially tries to
satisfy its contract by following a certain strategy, implemented by the
routine's body (the do clause). An exception occurs when this strategy fails.
In the disciplined approach, only two courses of action make sense:
The routine (contractor) may have a substitute strategy. If so, it should
bring the target object back to a stable state and use this strategy. This is
the resumption case.
If no substitute strategy is available, the routine should bring the target
object back to a stable state, concede failure, and pass the exception to its
client. This is the failure case.
In the exception history table shown in Figure 5, some exceptions are dealt
with in each of these two modes. The table, shown as it is printed at run
time, is divided into periods, separated by double lines. Each period, except
the last, ended with a retry.
Figure 5: An exception history table

Object Class Routine Name of exception Effect
_____________________________________________________________________________

2FB44 INTERFACE m_creation Feature "quasi_inverse": Retry
 Applied to void reference

2F188 MATH quasi_inverse "positive_or_null": Fail
 (from BASIC_MATH) Precondition violated

2F188 MATH raise "Negative_value": Fail
 (from EXCEPTIONS) Programmer exception

2F188 MATH filter "Negative_value": Fail
 Programmer exception

2F321 MATH new_matrix "square_matrix": Fail
 (from BASIC_MATH) Invariant violated

2FB44 INTERFACE create Routine failure Fail

The absence of a clear-cut choice between resumption and retry is what makes
the Ada mechanism too general, and hence dangerous. Some Ada examples show
cases in which a routine reacts to an exception, fails to correct the cause of
the exception, and returns to its caller without signalling the exception.
This is extremely dangerous.
Eiffel enforces the choice between resumption and retry. The key idea is that
of routine failure -- a routine may succeed or fail. If it fails to achieve
its contract, it may either try again or give up. It should not conceal the
failure from its caller.
This explains the fourth case in the earlier list of Eiffel exceptions. The
failure of a routine automatically triggers an exception in its caller. This
is implemented by the optional routine clause rescue. If present, the rescue
clause is executed whenever an exception occurs during the routine's
execution.
If a rescue clause is executed to the end, the routine terminates by failing.
As noted, this automatically raises an exception in the caller, whose own
rescue clause should handle it. If a routine has no rescue clause, as will
typically be the case with most routines, then it is considered to have an
empty rescue clause -- any exception occurring during the execution of the
routine leads to immediate failure and an exception in the caller. If no
routine in the call chain has a rescue clause, the entire execution fails and
an appropriate message, recording the history of recent exceptions in reverse
order, is printed. Note the use of assertion tags, when present, in the
messages shown in Figure 5.
Not all exceptions cause failure. A rescue clause may execute a retry
instruction, in which case the body (do clause) of the routine must be tried
again, presumably because a substitute strategy is available. This is the
resumption case.
For example, consider the routine in Listing Five, page 125, for attempting to
write to disk, from a generic class C.
Here it is assumed that the actual write is performed by a lower-level
external routine attempt-to-write, written in another language, over which we
have no control. If this routine fails, it triggers an exception, which is
caught by the rescue clause. This results in a retry. Local routine variables
are initialized on routine entry. An integer variable such as attempts, is
initialized to 0.
The routine write never fails. Its contract says, "write if you can, otherwise
record your inability to do so by setting the value of attribute
write_successful to false, so that the client can determine what happened." It
is always possible to satisfy such a contract.
The version of write shown in Listing Six, page 125 is a variant of the class
that does not include attribute write_successful. It may succeed or fail.
In this version, after five attempts, the routine terminates through the
bottom of its rescue clause. This means the routine fails, triggering an
exception in the caller. This contract is more restrictive than the one shown
in Listing Three. It requires that the routine be able to write. If this
contract cannot be fulfilled, the only exit is through failure.


Formal Requirements



The deeper meaning of the rescue clause can be understood in the
object-oriented context, and with reference to the contract of a routine, as
expressed by assertions.
The following expresses the requirements on a contractor that implements
software element e:
 {P} e {Q}
This means the contractor must write e in such a way that, whenever P is
satisfied on entry, Q will be satisfied on exit. The stronger P is, the easier
the contractor's job (more can be assumed); the stronger Q is, the harder the
contractor's job is (more must be produced).
Consider routine r with body do, precondition pre, and postcondition post, in
a class with invariant INV. The requirement on the author of the do clause is:
 {pre and INV} do {post and INV}
In other words, the invariant and the precondition can be assumed, the
invariant must be preserved, and the postcondition must be ensured. Now,
consider a branch rescue, of the rescue clause, not ending with a retry. The
requirement here is:
 {true} do {INV}
The input condition is the weakest possible (hardest from the contractor's
viewpoint), because an exception may occur in any state. The rescue clause
must be prepared to work under any condition, but the output condition only
includes the invariant. Ensuring the invariant brings the object back to a
stable state. Integrity constraints play a similar role in data base systems.
The rescue clause is not, however, constrained to ensure the entire
postcondition. This is the sole responsibility of the do clause. If the
contractor satisfies the routine's contract, there is no need for the rescue
clause.
This shows the clear separation of concerns between the do clause and the
rescue clause. The former is responsible for achieving the contract when
possible. The latter takes over in case the do clause falters. The rescue
clause restarts the do clause under improved conditions, or closes the store
after putting things in order. The requirements on the rescue clause are both
harder (a weaker precondition) and easier (a weaker postcondition).


Fine-Tuning the Mechanism


Those are the basics of Eiffel exception handling. In practice, some
fine-tuning may be needed for particular applications. This is done not
through the language itself, but through the library class EXCEPTIONS. Classes
needing the corresponding facilities should inherit this class.
It is sometimes necessary to treat various exceptions differently. Attribute
exception in class EXCEPTIONS has the value of the code of the last exception
that occurred. Exception codes are integer symbolic constants (attributes)
defined in that class. Examples include Precondition (precondition violated)
and other assertion-related exceptions, No_object, No_more_memory, operating
system signals (Sighup and so on.) and others. A rescue clause may contain a
test of the form:
 if exception=No_more_memory then ... elsif and so on.
Generally, it is wise to resist the temptation to attach too much meaning to
the precise nature of an exception. An exception usually points to a symptom,
rather than a cause.
For programmers who want to define and raise their own exceptions, the routine
raise is available in class EXCEPTIONS. The default handling of certain
exceptions, especially operating system signals, can be changed by redefining
certain routines from class EXCEPTIONS. By using class EXCEPTIONS, application
software can access information about the last exception. This infomation
includes the exception type, its meaning expressed as a plain English string,
and so on. This is particularly useful for printing informative error
messages.


Why Not Make It Right?


Reliability is a primary concern in any serious view of software construction.
In the object-oriented approach, it is even more essential. Reusability of
software is meaningless unless the reusable components are correct and robust.
Static typing is an important aspect of Eiffel's contribution to this goal
(see the article "You Can Write, but Can You Type?" in the March 1989 issue of
the Journal of Object-Oriented Programming for more on this subject).
The assertion and exception techniques described in this article provide the
complement to static typing. They don't absolutely guarantee that your classes
will be correct and robust, but they sure can help.

_WRITING CORRECT SOFTWARE WITH EIFFEL_
by Bertrand Meyer


[LISTING ONE]

class CIRCLE export
 center, radius, intersect1, intersect2,
 on, inside, outside,
 translate, scale, rotate ...
feature
 center: POINT;
 radius: REAL;

 intersect1(other: CIRCLE): POINT is
 -- One of the intersections
 -- of current circle with other
 require
 not other.Void;
 center.distance(other.center)
 <=radius + other.radius
 do
 ... Computation of intersection ...
 ensure
 on(Result);
 other.on(Result);
 end; -- intersect1

 intersect2(other:CIRCLE): POINT is
 ...


 on(p:POINT) is
 -- IS p on circle?
 require
 not p.Void
 do ...
 end; -- on

 inside(p:POINT) is
 -- Is p inside circle?
 require
 not p.Void
 do ...
 end; -- inside

 outside (p:POINT) is
 -- Is p outside circle?
 require
 not p.Void
 do ...
 end; -- outside

 Create (c:POINT; r:REAL) is
 --Create circle with center c
 --and radius r
 require
 not c.Void;
 r>=0
 do
 center:=c; radius :=r
 end; -- Create

 ... Other features (translate, scale, ...) ...

 invariant
 ... See below ...
 end -- class CIRCLE







[LISTING TWO]

require
 other_not_void:not other.Void;
 circles_intersect:
 center.distance (other.center)
 <=radius + other.radius






[LISTING THREE]


class interface CIRCLE
exported features
 center, radius, intersect1, intersect2,
 on, inside, outside,
 translate, scale, rotate ...
feature specification
 center: POINT;
 radius: REAL;

 intersect1(other: CIRCLE): POINT
 -- One of the intersections
 -- of current circle with other

 require
 not other.Void;
 center.distance(other.center)
 <=radius + other.radius
 ensure
 on(Result)
 other.on(Result)

 intersect(other:CIRCLE): POINT
 ...

 on(p:POINT)
 -- Is p on circle?
 require
 not p.Void

 inside(p:POINT)
 -- Is p inside circle?
 require
 not p.Void
 ensure
 Result=(center.distance(p)<radius)

 outside(p:POINT)
 ...

 ...Specification of other features
 (translate, scale, ...) ...

invariant
 ... See below ...

end -- class interface CIRCLE








[LISTING FOUR]

r is
 require
 p

 do
 if not p then
 ... Deal with erroneous case ...
 else
 ... Proceed with normal execution ...
 end
 end; -- p








[LISTING FIVE]

class C [T] export
 write, write_successful,...
 feature

 write_successful: BOOLEAN;
 -- An attribute

 write (x: T) is
 -- Write x, if possible;
 -- make at most five attempts.
 -- Record result in write_successful
 local
 attempts: INTEGER
 external
 attempt_to_write (x:T)
 language "..."
 do
 if attempts <5 then
 attempt_to_write (x);
 write_successful:=true
 else
 write_successful:=false
 end
 rescue
 attempts:=attempts+1;
 retry
 end -- write
 ...
end -- class C





[LISTING SIX]

write(x: T) is
 -- Write x;
 -- make at most five attempts.
 local
 attempts:INTEGER
 external

 ... As before ...

 do
 attempts:=attempts+1;
 attempt_to_write(x);
 rescue
 if attempts < 5 then
 retry
 end
 end -- write




















































December, 1989
THE QUICKPASCAL IN QUICKPASCAL


A look at how the windows in QP's User Interface use their own object-oriented
technology




Joseph Mouhanna and Michael Vose


Michael Vose authors the monthly newsletter OS REPORT: News and Views on OS/2.
He can be reached at P.O. Box 3160, Peterborough, NH 03458. Joseph Mouhanna is
the program manager for Pascal products and has been with Microsoft since
1984. Prior to transferring to the language group in 1989, he spent 18 months
developing the Microsoft right-to-left technology. He can be reached at
Microsoft Corp., 1 Microsoft Way, Redmond, WA 98052-6399.


Writing the user interface for a compiler imposes a different set of problems
than creating a code generator. Presenting the applications programmer with an
orderly set of windows and menus streamlines his interaction with the
compiler. When designing this interface, the compiler's authors need to
consider visual elements, and not machine internals.
A windowing user interface uses a limited number of variations on a few visual
elements. These elements include windows, dialog boxes, and menus. When these
components are analyzed, one realizes that dialog boxes and menus are
specialized windows.
Therefore, a windowing interface lends itself to the hierarchical structure of
object-oriented programming (OOP). The match between graphical user interfaces
and OOP emerges because the techniques of OOP allow the creation of new
windows as children of existing windows. Doing this requires little additional
code. With a few generic window classes, specialized subclasses implementing
variations on the overall window theme can be created.
Microsoft used QuickPascal itself to write the QuickPascal user interface.
They did so to obtain the many advantages of OOP. In addition to code
reusability, OOP techniques cut down on debugging and testing time. By mixing
object-oriented and standard programming, Microsoft produced, in a short span
of time, a user interface that will be easy to update for future releases of
QuickPascal.


Design Considerations


Good OOP design uses a top-down approach to creating programs. Objects are
derived from general classes, and then inheritance is used to create ever more
specialized child-class objects for special purposes. OOP's use of inheritance
makes code reusable and, as a result, substantially reduces the size of
programs. OOP techniques reduce debugging time by encouraging the use of
hundreds of small, simple methods, rather than a few large, complex procedures
or functions as in standard structured programming.
One or more parent classes for the application reside at the top of the OOP
design hierarchy. The QuickPascal user interface uses a parent window class
and a parent event-handling class. These parent classes contain the
information and methods basic to all windows, as well as information about the
event-handling functions of the user interface. The inheritance of these
general qualities and actions by child classes allows the building of all the
objects needed to implement a consistent user interface.
Child classes inherit the properties of their parents without the writing of a
lot of additional code. Using classes and objects also streamlines event
handling. Window environments written in classical languages usually use very
large CASE statements to handle all possible events for each window type.
Microsoft Windows applications, for example, frequently sport CASE statements
with a dozen or more elements. An object-oriented event handler lets the
different window types inherit all the general event handling characteristics
of their parents, and only adds the new code needed to handle special events.
This inheritance usually makes code much more compact and efficient than the
more conventional programming approach.
However, this is not always true. A standard OOP event handler might send a
"redraw yourself" message to a window previously covered by a dialog box or
pull-down menu. QuickPascal does not always follow this standard approach for
one simple reason -- speed. Instead, QuickPascal's event handler sometimes
stores in a buffer a copy of the image that lies underneath the dialog box.
When the dialog box disappears, the event handler transfers the image stored
in the buffer back onto the screen.
QuickPascal uses this buffer-image technique to make its user interface more
responsive. Redrawing an image from a buffer updates only the portion of the
screen that has changed. Telling a window object to redraw itself means
redrawing the entire image, which is much more time consuming. The
buffer-image technique is an "optimization" of OOP practice.
This optimization addresses the few problems that result from using OOP
techniques. If creating a class and an associated method to perform a task
requires more bytes or machine time than a standard procedure, sticking with a
more traditional procedure may improve a program's performance.
For this reason, QuickPascal's user interface does not use objects and their
associated methods to the exclusion of standard procedures. Where standard
procedures complete a task with less programming overhead than a method,
QuickPascal's interface uses conventional procedures.


The Parent Classes


QuickPascal's user interface code uses two elementary classes. The subclasses
used to spawn the objects that comprise the user interface are derived from
these classes. One parent class receives information from the keyboard and
mouse (input), and the other displays information on the screen (output).
These two parent classes comprise the top of the class hierarchy.
The primary elements of these classes include event handling for input, and a
variety of windows, dialog boxes, and menus for screen output. These basic
parts interact as indicated in Figure 1.
The two parent classes in the QuickPascal environment are called TWindow and
TApplication. These names are familiar to anyone who has programmed in Object
Pascal or with the Macintosh's MacApp class library. The T in these class
names means, "type," and is a naming convention established by Apple Computer.
The TWindow class and its descendants handle all screen drawing, including
windows, menus, and dialog boxes. The TApplication class and its children
interpret keyboard and mouse events and send them to the appropriate window
object. All of the code classes in the QuickPascal user interface devolve from
these parent classes, as shown in Figure 2.
The TWindow class (see Listing One, page 126) contains variables and methods
basic to all window functions. These events include opening, drawing, moving,
mounting, unmounting, printing, coloring, scrolling, responding to mouse
events, and closing windows. Mounting refers to bringing a window to the front
of the screen and making it the active window.
TWindow serves only as a parent to other classes. QuickPascal's user interface
code declares no objects directly from TWindow. TWindow's child classes --
TDialogBox, TDocument, TMenuBar, and TSubMenu--create their own distinct types
of windows, and all of the QuickPascal interface window objects are derived
from these child classes.
The TWindow definition contains mouse handling methods that are comprised
simply of BEGIN and END statements. The child classes of TWindow inherit these
blank methods and override them with their own mouse handling method
definitions. This polymorphism allows a TApplication object to send the same
message to any TWindow object (described shortly.) The type of the
TWindow-derived object -- dialog box, document, or menu -- doesn't matter.
Each of these objects specifies how to deal with these mouse-event messages.


The TWindow Child Classes


Most of the dialog box objects used by the QuickPascal user interface come
from the TWindow child class TDialog Box. This child class inherits all the
qualities of TWindow, and adds new methods for drawing buttons, edit fields,
and list boxes (see Listing Two, page 126). It stores the window's current
state (open, closed, waiting for input, and so on) in its instance variables.
Because the QuickPascal interface uses some specialized dialog boxes, several
other child classes exist to deal with those selected cases. One of these
child classes is TFileDialogBox.
While inheriting all the method functionality it needs from TDialogBox,
TFileDialogBoxadds two more instance variables. These variables store the
default and current document extension and the path to the current directory.
These new variables make the file dialog box especially convenient to use.
Without them, a user would have to reenter the path name every time a file
dialog box opened, and the default document extension would always be the same
regardless of the most recently used extension.
TPrintDialogBox also adds instance variables to store pointers to the correct
document and to the text block location. They tell QuickPascal what window to
print from and what text to print in the active window.
Help dialog boxes use TDialogBox's methods, but need instance variables to
describe the help topic called and to indicate the help page, the first line
of the page, and the number of lines on a page.
The BP in TBPDialogBox means "Break Point." This object is used by QuickPascal
to let the user set break points in the code they are editing. The break point
dialog box adds three instance variables to the ones it inherits from
TDialogBox. The additional variables store a break point location, the
condition on which to halt execution, and the type of break point (conditional
and unconditional).
Dialog boxes that need custom keyboard and mouse actions use the
TModifyDialogBox class. Each of the methods in this class overrides the
methods of its parent.
TDocument and its children produce the screen items traditionally considered
windows: The program editing windows, the QuickPascal Advisor help window, and
the debug window. TDocument and its children control all editing windows. The
TDocument class clearly shows the advantages of OOP. It greatly simplifies the
moderately complex task of creating a custom text editor. TDocument builds on
TWindow's basic actions by adding the methods necessary for editing text.
TDocument's child classes then modify the editor features they inherit to suit
their own needs.
The TDocument class treats documents (program files, help files, and others)
as a series of lines made up of separate characters. TDocument uses instance
variables to track the relevant data about a document. The source code for
TDocument demonstrates how easy it can be to design a custom file format. All
that's required is the declaration of a new child of TDocument with an altered
data structure. QuickPascal will read files that use this new structure as
long as the GetPhysLine method still returns a single line of text.
THelp Window is identical to TDocument except that it overrides the
GetPhysLine method. THelpWindow's version of GetPhysLine retrieves data from
QuickPascal's help (.HLP) files.
THelpWindow also makes minor changes to the GetChar and LeftMousePress methods
to keep any user activities from altering the contents of the help screen. All
other window-related activities function normally.

The TEditWindow class describes QuickPascal's standard editing window, in
which users write their programs. The methods of this class override
TDocument's GetChar, LeftMousePress, and DrawScreen methods. This class
includes three new methods, which take care of opening files, closing files,
and saving data to a file.
A menu bar is a specialized window. TMenuBar inherits all of TWindow's methods
without modification, with one exception: TMenuBar alters the DrawWindow
method to handle special Menu cases, such as graying out of menu items that
cannot be accessed, colors, menu text, and so on. TMenuBar also uses a pointer
to any TMainAppli objects, the principal object performing menu management.
A submenu is the pull-down portion of a menu. It is the part that appears when
a user clicks on an item in the menu bar. Like TMenuBar, TSubMenu overrides
the DrawWindow method it inherits from TWindow. It also uses a pointer to an
object of type TMainAppli to obtain other menu management functions.


The TApplication Chidren


The TApplication class is a good example of a parent class serving as the
basis for other classes. QuickPascal's user interface declares objects from
the child classes of TApplication, but does not declare any objects directly
from TApplication itself.
When a window object of the TWindow subclass TDialogBox (or one of its
descendants) opens, an object of class TDialogBoxAppli also opens to handle
events within a dialog box. TDialogBoxAppli overrides its inherited GetEvents,
HandleEvents, and DrawStatusLinemethods in order to handle events special to
dialog boxes. TDialogBoxAppli also adds an instance variable that points to
the type of dialog box created.
Dialog box objects from both window and application classes exist only as long
as the dialog box remains open. When the dialog box closes, QuickPascal
disposes of both objects.
Like TApplication, TMainAppli illustrates one of the principles of good OOP
design. TMainAppli serves primarily as a foundation for its child, EditAppli,
but remains general enough to allow for easy future expansion. If Microsoft
decides to add another type of window to the QuickPascal user interface,
TMainAppli will make it easy to add a new event handling class for that
window.
EditAppli handles the bulk of the events in the QuickPascal environment. It
keeps track of the number of open windows, plus the handle of the active
window, and watches for menu activity. EditAppli's objects work with the
window objects for TDocument, TMenuBar, and TSubMenu. EditAppli inherits
almost all its methods and instance variables from TMainAppli.


Interacting Objects


TWindow and its child classes work in tandem with the TApplication classes.
Every TWindow-derived object has a companion TApplication-derived object that
handles the events for that window. A TApplication object can be associated
with more than one TWindow object. TApplication and its children have instance
variables that store mouse events, and timer events, and the current state of
the keyboard. The children's methods acquire and dispatch these various
events.
TApplication's child classes manage special event handling features (such as
TDialogBoxAppli not allowing mouse clicks outside of a dialog box).
TApplication, however, has methods to process all the normal user activities
that occur in a window and passes those methods on to its child classes
through inheritance.
The TApplication classes and TWindow classes work closely together. When
QuickPascal opens and initializes a new TApplication-derived object and its
corresponding TWindow-derived object(s), the methods set up a loop between the
objects. The TApplication object's GetEvents method waits until either a
keyboard or mouse event occurs. When something happens, GetEvents sets its
instance variables and then ends. The TApplication object then transfers
control to its associated TWindow object, activating its HandleEvents method.
The TWindow-derived object's methods (My Window.LmousePress, for example)
checks the state of the window's instance variables and responds to the event
that just occurred. This usually results in a call back to one of the
TWindow-derived methods, but can also mean transferring control to the
compiler or to some other activity.


How It All Works


A look at a fragment of the QuickPascal interface code provides a glimpse into
this object-oriented universe. Listing Three (page 127) shows a class
definition, an instance of an object, and the use of that object within a
Pascal procedure. The first part of Listing Three illustrates how child
classes become defined, inheriting behavior from their parent classes and
overriding the methods of those arents where necessary.
The two instances of EditWindow in the definition of class EditWindow set up a
linked list of edit windows that can be open at the same time.
EditWindow-class objects use (among others) the OpenFile method, which
specifies what happens when an object of this class needs to open a file.
OpenFile names the object of type EditWindow for which it can open a file (in
this case, Edit).
As the code fragment in Listing Three shows, a class is first defined by
describing its methods and naming its instance variables. Next, an object of
that class is created. This process is similar to declaring a variable. Then,
some memory for the named object is allocated using the New keyword and the
name of the object. Finally, a message is sent to the object by calling one of
its methods from within the main body of a program.
Using this technique, code modules that are easy to debug and modify can be
built. The internal operation of an object can later be changed without
worrying about "breaking" the rest of your program in its interaction with the
changed object. Also, new versions of existing objects can be created by
defining a child class with new or overriding methods. The building blocks
that classes and objects provide make creating programs a lot less risky than
it used to be.

_THE QUICKPASCAL IN QUICKPASCAL_
by Joseph Mouhanna and Michael Vose


[LISTING ONE]

{Top Window Object}
TWindow = OBJECT
 Client : Rect;
 Coord : Rect;
 TCoord : Rect;
 Handle : word;
 Shadow : boolean;
 Col : tWindowColors;
 Captured: boolean;
 Class : WindowClasses;
 Bitmap : pointer;

 TimeOutState: byte;
 ForceTimeOut: boolean;
 CursMode: CursorModes;
 CursX : word;
 CursY : word;

procedure TWindow.OpenWindow (Class: WindowClasses; x,y,w,h: word;
 Col: tWindowColors; Shadow: boolean);
procedure TWindow.DrawWindow (Clip: lRect);
procedure TWindow.CloseWindow;

procedure TWindow.MoveWindow (x,y: word);
procedure TWindow.ChangeWindow (x,y,w,h: word);
procedure TWindow.MountWindow;
procedure TWindow.UnMountWindow;
procedure TWindow.GetClientArea (var Client: Rect);
procedure TWindow.WrtChars (var C; Len,x,y: word);
procedure TWindow.WrtCharsAttr (var C; Attr,Len,x,y: word);
procedure TWindow.WrtExtendedCharsAttr (var C; Attr1,Attr2,Len,Width,x,y:
word);
procedure TWindow.WrtNAttr (Attr,Len,x,y: word);
procedure TWindow.WrtCells (var C; Len,x,y: word);
procedure TWindow.WrtFrame (F: FramePartsSet; Attr,x,y,w,h: word);
procedure TWindow.WrtSeparator (RightCorner: boolean; Attr,y: word);
procedure TWindow.ScrollUp (Lines, Attr,x,y,w,h: word);
procedure TWindow.ScrollDown (Lines, Attr,x,y,w,h: word);
procedure TWindow.ScrollLeft (Rows, Attr,x,y,w,h: word);
procedure TWindow.ScrollRight (Rows, Attr,x,y,w,h: word);
procedure TWindow.CaptureMouse;
procedure TWindow.UnCaptureMouse;
procedure TWindow.PressLMouse (DbleClicked: boolean; x,y: integer);
procedure TWindow.ReleaseLMouse (x,y: integer);
procedure TWindow.PressRMouse (DbleClicked: boolean; x,y: integer);
procedure TWindow.ReleaseRMouse (x,y: integer);
procedure TWindow.MoveMouse (x,y: integer);
procedure TWindow.PosCursor (x,y: word);
procedure TWindow.SetCursorMode (Mode: CursorModes);
procedure TWindow.GetChar (Key: word);
procedure TWindow.TimeOut;
procedure TWindow.RefreshStatusLine;
end;







[LISTING TWO]

TDialogBox = object (TWindow)
 First : pDialogBoxItem;
 Current : pDialogBoxItem;
 Command : function (Dialog: TDialogBox; ID:word): boolean;
 ID : word;
 HelpNO : word;
 UndoID : word;
 ValidID : word;
 Appli : TDialogBoxAppli;
 MouseAction : MouseActions;
 TitleColor : boolean;
{$IFDEF DEBUG}
 HeapPtr : pointer;
{$ENDIF}

procedure TDialogBox.OpenDialogBox (CheckPrevW : boolean;
w,h : word;
Command : pointer;
HelpNO : word;
UndoID : word;
ValidID : word);


procedure TDialogBox.Call (var ID: word);
procedure TDialogbox.GetChar (Key: word); override;
procedure TDialogBox.CloseWindow; override;
procedure TDialogBox.DrawWindow (Clip: lRect); override;
procedure TDialogBox.PressLMouse (DbleClicked: boolean; x,y: integer);
override;
procedure TDialogBox.PressRMouse (DbleClicked: boolean; x,y: integer);
override;
procedure TDialogBox.ReleaseLMouse (x,y: integer); override;
procedure TDialogBox.MoveMouse (x,y: integer); override;
procedure TDialogBox.SelectDialogBoxItem (ID: word);
procedure TDialogBox.TimeOut; override;
procedure TDialogBox.DelDialogBoxItem (ID: word);
procedure TDialogBox.AddComment (
 CommID : word;
 AltID : word;
 x,y,w : word;
 Text : String;
 justif : Justifications);
procedure TDialogBox.AddFrame (
 FrameID : word;
 AltID : word;
 x,y,w,h : word;
 Text : String);
procedure TDialogBox.AddSysButton (
 TabID : word;
 SysID : SysButtons;
 justif : Justifications);
procedure TDialogBox.AddDlgText (
 TabID : word;
 DlgID : word;
 Size : word;
 x,y,w : word;
 Text : String);
procedure TDialogBox.AddPushButton (
 TabID : word;
 PushID : word;
 x,y : word;
 State : boolean);
procedure TDialogBox.AddRadioButton (
 TabID : word;
 GroupID : word;
 RadioID : word;
 LinkID : word;
 x,y : word;
 State : boolean;
 Trace : pointer);
procedure TDialogBox.AddListBox (
 TabID : word;
 ListID : word;
 Columns : word;
 x,y,w,h : word;
 LinkID : word;
 Trace : pointer);
procedure TDialogBox.DrawComment (
 p : pDialogBoxItem);
procedure TDialogBox.DrawFrame (
 F : FramePartsSet;
 p : pDialogBoxItem);
procedure TDialogBox.DrawSysButton (

 Hilite : boolean;
 p : pDialogBoxItem);
procedure TDialogBox.DrawDlgText (
 p : pDialogBoxItem);
procedure TDialogBox.DrawPushButton (
 p : pDialogBoxItem);
procedure TDialogBox.DrawRadioButton (
 p : pDialogBoxItem);
procedure TDialogBox.DrawListBox (
 p : pDialogBoxItem);
procedure TDialogBox.SetComment (
 CommID : word;
 Text : String);
procedure TDialogBox.GetComment (
 CommID : word;
 var Text : String);
procedure TDialogBox.GetDlgText (
 DlgID : word;
 var Text : string);
procedure TDialogBox.SetDlgText (
 DlgID : word;
 Text : String);
procedure TDialogBox.SetLockState (
 DlgID : word;
 Lock : Boolean);
procedure TDialogBox.GetPushButton (
 PushID : word;
 var State : boolean);
procedure TDialogBox.SetPushButton (
 PushID : word;
 State : boolean);
procedure TDialogBox.GetRadioButton (
 GroupID : word;
 var ID : word);
procedure TDialogBox.SetRadioButton (
 GroupID : word;
 ID : word);
procedure TDialogBox.GetTextListBox (
 ListID : word;
 var Text : string);
procedure TDialogBox.GetOffsetListBox (
 ListID : word;
 var Offset : word);
procedure TDialogBox.SetTextListBox (
 ListID : word;
 offset : word;
 Text : string);
procedure TDialogBox.SetOffsetListBox (
 ListID : longint;
 offset : word);
procedure TDialogBox.ClearListBox (
 ListID : dword);
function TDialogBox.AddListBoxItem (
 ListID : dword;
 Text : string): boolean;
end;






[LISTING THREE]

EditWindow = object (TDocument)
 ewNext : EditWindow;
 ewPrev : EditWindow;
 ewWDW : word;
 ewDOC : word;
 ewType : DocTypes;
 ewUndo : record
 X : byte;
 Y : word;
 Len : word;
 Buf : MemHandle
 end;

 ewFirstMark ,
 ewCursMark : record
 XY: word;
 dX: word
 end;

procedure EditWindow.OpenFile (
var Status : QPErrors;
Col : tWindowColors;
DOC : word;
WDW : word;
Locked : boolean;
 dtType : DocTypes;
 IsNew : boolean;
 Time : LongInt;
 Coord : lRect;
 Title : PathStr;
 TextHandle: MemHandle;
 TextSize: word);

procedure EditWindow.DuplicateWindow;
procedure EditWindow.SaveFile (var Status: QPErrors; FileName: PathStr;
 DefaultExt: ExtStr);
procedure EditWindow.CloseWindow; override;
procedure EditWindow.CloseDocument;
function EditWindow.NbLines: word; override;
function EditWindow.NbColumns: word; override;
function EditWindow.GetLine (Column,Line,Len: word; var Buffer): word;
override;
function EditWindow.GetLineLength (Line: word): word; override;
procedure EditWindow.GetPhysLine (Column,Line,Len: word; var Cells); override;
procedure EditWindow.GetChar (Key: word); override;
function EditWindow.CopyBlock2ClipBoard: boolean; override;
procedure EditWindow.MountWindow; override;
procedure EditWindow.UnMountWindow; override;
procedure EditWindow.ModifyLine (Line: word; PosX,PosY: word);
procedure EditWindow.ModifyEOF (Line: word; PosX,PosY: word);
end;

{=====================
An instance of an object of type EditWindow; here, the object Edit is passed
to the procedure InternalReplaceChar.


procedure InternalReplaceChar (
 Edit : EditWindow; { <== object declared}
 AtX,AtY : word;
 Ch : Char);
var
 Pt : pFileText;
 Pl : pFileLines;
 Pi : pFileLinesInfo;
begin {InternalReplaceChar}
 with Documents[Edit.ewDOC] do begin
 __SaveLineForUndo (Edit, AtX,AtY);
 FarLockMem2 (dLines,Pl, dText,Pt);
 if __CheckAndExpandIfInTab (
 EditorError,
 Edit.ewDOC,
 Pt, Pl,
 AtX, AtX, AtY) then begin
 Pt^[Pl^[AtY]+AtX] := Ch;
 OneSourceModified := true;
 dUsed := true;
 dTime := GetDateTime;
 FarUnLockMem2 (dLines,dText);
 if dMarkModif then begin
 {$IFDEF DEBUG}
 Assertion (dInfo <> NullMemHandle, __LINE__, SourceName);
 {$ENDIF}
 FarLockMem1 (dInfo, Pi);
 Pi^[AtY] := 0;
 FarUnLockMem1 (dInfo)
 end
 end
 else
 ExtendedErrorBox (false, EditorError, true, dFileName)
 end
end; {InternalReplaceChar}
*)

{================
The Openfile [method?] [procedure declaration?].
================}

{-----------------------------------------------------------------------}
{ OpenFile}
{-----------------------------------------------------------------------}
procedure OpenFile (
 var Status : QPErrors;
 var Edit : EditWindow;
 Mode : OpenFileModes;
 dtType : DocTypes;
 FileName : PathStr;
 DefaultExt : ExtStr);
label Error;
var
 H : FileHandle;
 Size : dword;
 wdw : word;
 doc : word;
 Time : LongInt;
 Ht : MemHandle;

 Pt : pFileText;
 Dummy : QPErrors;
 Locked : boolean;
begin {OpenFile}
{Check if it's possible to open a new document}
 Edit := NIL; H := 0; Ht := NullMemHandle;
 if CheckFileNameNotAlreadyLoaded (FileName, DefaultExt) <> 0 then begin
 Status := ErrFileAlreadyLoaded;
 goto Error
 end;

{
===================
Using the OpenFile procedure
===================}

 New (Edit); {Memory Allocated for the object Edit}
 Edit.OpenFile (Status,
 wcEditWindows,
 Doc,
 wdw,
 Locked,
 dtType,
 Mode = ofUntitled,
 Time,
 NULL,
 FileName,
 Ht,
 Size);
 if Status <> ErrNone then begin
Error:
 if Ht <> NullMemHandle then FarFreeMem (Ht);
 if H <> 0 then DosCloseFile (Dummy, H);
 if Edit <> NIL then begin
 Dispose (Edit); Edit := NIL
 end;
 {ExtendedErrorBox (false, Status, true, FileName)}
 end
end; {OpenFile}























December, 1989
AN OBJECT-ORIENTED LOGIC SIMULATOR


Wire-wrapping and breadboarding made easy




Kenneth E. Ayers


Ken Ayers is a self-educated software engineer. He is currently employed by
Industrial Data Technologies of Westerville, Ohio. His interests include
computer graphics, machine intelligence, and OOP systems. He can be reached at
7825 Larchwood St., Dublin, OH 43017.


As a bench technician in a research and development laboratory, my primary job
was to build prototype circuits with digital logic devices, and to make sure
that the prototypes worked.
After migrating into the world of software, I saw no reason why my computer
could not simulate those same digital logic circuits. When I discovered
object-oriented programming, I became more and more convinced that it was both
possible and practical.
The project, which I named LogicLab, was intended from the start to simulate
not just logic devices, but the complete bench environment. LogicLab was
required to provide the simulated equivalent of four aspects of that
environment:
1. A well-stocked parts cabinet containing standard integrated circuit (IC)
devices (74LSOO, and so forth);
2. A method to connect the pins of the ICs together (commonly called a
"breadboarding system");
3. A source of signals (clocks, pulses, switches, and so on) that could be
used to simulate inputs to the circuit;
4. A logic analyzer that I could use to monitor the activity within my circuit
for debugging purposes.
At first glance, these four simulations appear simple. If implemented
properly, they would indeed emulate the workbench environment. But when these
simulations were expanded to the level of detail required for an effective
implementation, the task seemed overwhelming.
I had previously considered tackling this project with a combination of C and
assembly language, and estimated that the time required to produce a prototype
in those languages would be about 8 to 12 months. When I used Smalltalk to
write LogicLab, I had a working version of the project in roughly 100 hours! I
spent much of that time learning to use the incredibly complete environment
provided with Smalltalk/V. The environment includes more than 100 classes and
some 2000 methods, and source code is provided for everything (except the
compiler itself!). After building the prototype, I invested another 150 (or
so) hours on the project in order to give LogicLab a major overhaul that
improved both performance and the user interface.


A Glance Inside LogicLab


The heart of LogicLab -- the simulation kernel -- is shown in Listing One
(page 130). The system also includes more than 20 object classes. Nearly all
of these object classes correspond directly to real-life counterparts on the
design bench. Some of the typical simulated devices are a toggle switch
(Listing Two, page 136), a pulser switch (Listing Three, page 136), and a
logic device Listing Four, page 137). Other simulated devices include buttons,
switches, logic probes, input signals, a logic analyzer, and an assortment of
logic devices. A list of the class hierarchy for LogicLab is given in Figure
1. Figure 2 shows a typical display produced by LogicLab's logic analyzer.
Figure 1: LogicLab's class hierarchy

 Object
 Button
 MomentaryActionButton
 ButtonPanel
 VariableMenu
 EmptyMenu

 TimeInterval
 LogicNode
 LogicProbe

 LogicComponent
 LogicDevice
 N74LS00

 LogicSignal
 ClockInput
 ConstantInput

 LogicSwitch
 PulserSwitch
 ToggleSwitch

 LogicAnalyzer
 LogicLab


LogicLab is organized into two discrete application phases. The first phase is
the breadboard. In this phase, the user installs devices into the circuit and
makes the connections between pins. This process is very similar to the
process of wire-wrapping, and is driven by a series of pop-up menus and
prompts for device names and pin numbers.
The second application phase in LogicLab is the logic analyzer. In a hardware
workbench environment, the logic analyzer is used as a testing and debugging
tool. The logic analyzer has a number of probes (LogicLab's logic analyzer has
eight probes) that can be connected to various points in the circuit (usually
to pins on ICs). While the circuit is operating, the logic analyzer records
and displays the states monitored by each probe. The output is a timing
diagram that illustrates the relationships between signals in the circuit. A
technician uses this information to determine if the circuit is functioning
properly. If the circuit is not functioning properly, this information is used
to identify where a problem (a bug!) might be found.
Because LogicLab is a simulation, the process of using the logic analyzer is
handled a little bit differently than in a hardware circuit. LogicLab
simulates time in discrete steps. In essence, LogicLab examines all of the IC
input pins and calculates the corresponding output states. After LogicLab
completes the calculation, it pretends that one nanosecond (a billionth of a
second) has passed, and then repeats the process.
Between nanoseconds, LogicLab performs housekeeping chores -- such as updating
the timebase, recording the states at the probes, and updating the timing
diagram display.


The State Connection


One object class -- the LogicNode class -- performs the majority of the work
during a simulation. This class simulates the electrical connections within a
circuit. Each pin of a simulated logic device (most devices have 14, 16, or
more pins) is associated with a LogicNode; each input signal has a LogicNode;
and the private data for each switch or push-button device includes a
LogicNode.
The LogicNode class is a good example to use for demonstrating the close
correspondence between physical objects and their software counterparts.
Figure 3 shows the conceptual organization of pins on an IC.
A characteristic of hardware logic devices is that a change in an input signal
requires a finite amount of time before the corresponding change appears at
the output. This delay is known as the "propagation delay." The length of the
propagation delay is determined by examining the specifications (data sheets)
for a particular type of device.
In a hardware logic device, the propagation delay occurs within the logic
elements (transistors, diodes, and so forth) inside the device. Because of
different signal path lengths within an IC, the propagation delay at an output
can often differ, based upon which input signal is present. (For example, in a
typical flip-flop, the delay from the reset input to the data output is
different than the delay from the clock input to the data output.)
To accommodate this phenomenon, I have modeled the propagation delay as an
integral part of the electrical connection rather than as a part of the
device. When a new IC object is created, each of its pins is assigned a delay
value by the initialization method for that device class.
Each LogicNode maintains a record of both its internal and its external state.
In this case, the "external state" is the (high or low) state that actually
appears at the pin of the device. The "internal state" is the state that is
present at the logical elements of the device. Each LogicNode also keeps a
record of its most recent internal state. This extra state record is essential
to the simulation of edge-triggered devices, such as flip-flops and counters.
In the case of LogicNodes that are associated with output pins, a change in
the internal state (as the result of a logical computation) triggers a
propagation delay counter. If the internal and external states are still
different when the counter reaches zero, the internal state appears at the
output -- that is, the internal state's value is stored in the node's external
state variable. During each simulation cycle, an output state is broadcast to
all other nodes to which that node is connected.
This brings us to another point. An isolated IC pin is of little use. An IC
pin must be connected to other pins, input signals, or output devices in order
to perform a useful function. Each of LogicLab's simulated pins maintains a
list of the other LogicNodes that form its connection chain. The node's
private data contains the identity of its successor and its predecessor nodes.
This information allows an output pin to access all of the input pins that are
affected by the output pin's signal.
Listing Five (page 138) is the source code for the output method implemented
in the LogicNode class. This method detects changes in the node's internal
state, times the propagation delay, and broadcasts the current external state
along the node's connection chain. The output method is invoked at each
simulation cycle, once per simulated nanosecond.


Putting It All Together


Ultimately, LogicLab simulates a circuit. Somewhere along the line, it must
simulate logic devices. Listing Six (page 138) is the simulation method found
in the class N74LS00. A hardware 74LS00 device is commonly known as a "Quad
2-Input NAND gate." This name indicates that a single IC in this device
contains four logic gates. The output of each gate is determined by logically
ANDing tow inputs and then negating that operation -- hence the name
"NOT-AND," or "NAND."
The primary characteristic of this type of gate, which is exploited by the
simulation, is that its output is low only when both inputs are high.
In LogicLab, one of the variables in each logic device is an array of
LogicNodes. In Listing Six, the statement (pins at: 1) isHigh fetches the
LogicNode from the first element of the device's pins array. The message
isHigh is then sent to that LogicNode. In response to the isHigh message, the
LogicNode returns either a true or false value, depending upon whether the
node's internal state is high or low.
Most simple logic gates can be simulated in a similar manner. More complex
functions, such as flip-flops and counters, can also be simulated in the same
general way as long as you recognize that latched devices must remember their
previous state between simulation cycles.
As a test of LogicLab's ability to simulate an actual circuit, I breadboarded
the simple divide-by-three counter shown in Figure 4. The timing diagram that
is expected for this circuit is shown below the schematic. Compare this timing
diagram to the timing diagram in Figure 2, which was produced by LogicLab's
simulation of the circuit. The primary difference between the two timing
diagrams is the length of the propagation delays between inputs and outputs.


Objects Pay Big Dividends


This project demonstrated to me, beyond any doubt, the value of using an
object-oriented programming system to construct a simulation. The ability to
construct objects by using their physical counterparts as models meant that I
was able to work within a familiar framework -- I already knew the
relationships between logic devices, signals, logic probes, and analyzers.
Smalltalk let me express those relationships essentially intact. Other
languages would have required me to translate those relationships into a more
rigid and less expressive form. Smalltalk's integrated, fully interactive
development environment also assisted me at every step and let me produce the
prototype with relative ease.
The only drawback to Smalltalk is its performance. Not only is Smalltalk an
interpreted language, but the overhead associated with message sends inflicts
a significant penalty upon run-time performance. The end result is that
LogicLab requires approximately one second of real time for every ten
nanoseconds of simulated time. Blazing speed is not this simulator's claim to
fame! But the slow run-time performance is easily overlooked, once you've
experienced firsthand the ease with which Smalltalk lets you transform ideas
into functional extensions of the environment itself.

_AN OBJECT-ORIENTED LOGIC SIMULATOR_
by Kenneth E. Ayers


[LISTING ONE]

Object subclass: #LogicLab
 instanceVariableNames:
 'devices signals switches clashes changed topPane analyzer breadboard
 listSelector currentComponent '
 classVariableNames: ''
 poolDictionaries:
 'FunctionKeys CharacterConstants ' !

!LogicLab class methods !
description
 "Answer a String describing the
 application and version."
 ^'LogicLab (Version 1.0 -- 06/26/88)'.!
new
 "Answer an initialized LogicLab application."
 logicLab 
 logicLab := super new.

 logicLab initialize.
 ^logicLab.! !

!LogicLab methods !
addComponent: aComponent
 "Add aComponent to the circuit description.
 If there is an error, answer nil; otherwise
 answer aComponent."
 name 
 name := aComponent name.
 name size == 0
 ifTrue: [
 "
 User is installing -- get a name.
 "
 name := self getNewName.
 name isNil
 ifTrue: [^nil].
 aComponent name: name]
 ifFalse: [
 "
 A name has been supplied -- this implies
 that the component is being installed from
 a file. Need to check for a clash with
 an existing name.
 "
 ((self componentNamed: name) isNil)
 ifFalse: [
 "
 Had a name clash -- get a synonym
 from the user and stash both of them
 away in the clashes table. Then
 rename the component.
 "
 name := self getNewName.
 name isNil
 ifTrue: [^nil].
 clashes
 at: aComponent name
 put: name.
 aComponent name: name]].
 changed := true.
 aComponent isDevice
 ifTrue: [devices add: aComponent]
 ifFalse: [
 aComponent isSignal
 ifTrue: [signals add: aComponent]
 ifFalse: [
 switches add: aComponent.
 analyzer isNil
 ifFalse: [analyzer addSwitch: aComponent]]].
 ^aComponent.!
allNames
 "Answer an array of all of the
 names of installed components."
 ^((self deviceNames), (self signalNames), (self switchNames)).!
analyzer: aModel
 "Set the LogicAnalyzer Application model
 to aModel."

 analyzer := aModel.!
breadboardList
 "Answer an array of strings according to the
 current list selector."
 listSelector isNil
 ifTrue: [listSelector := #listDevices].
 ^(self perform: listSelector).!
breadboardMenu
 "Private -- answer the menu that processes
 breadboard functions."
 MenuPosition := Cursor position.
 ^(Menu
 labels: ('Load\Save\Erase\List\',
 'Install\Connect\Remove\Disconnect\',
 'Simulate\',
 'Quit') withCrs
 lines: #(4 8 9)
 selectors: #(load save erase list
 install connect remove disconnect
 run
 quit)).!
changed
 "Answer true if the circuit has changed."
 ^changed.!
changed: aBoolean
 "Set the circuit-changed flag to aBoolean."
 changed := aBoolean.!
close
 "Close the LogicLab breadboarding window."
 topPane dispatcher deactivateWindow closeWindow.!
closeIt
 "Close the breadboard application window."
 self close.!
componentNamed: aName
 "Answer the component (device, signal, or switch)
 whose name is aName. If no component can be found
 answer nil."
 realName 
 realName := aName.
 clashes isNil
 ifFalse: [
 (clashes includesKey: aName)
 ifTrue: [realName := clashes at: aName]].
 devices do: [:aDevice
 (aDevice name = realName)
 ifTrue: [^aDevice]].
 signals do: [:aSignal
 (aSignal name = realName)
 ifTrue: [^aSignal]].
 switches do: [:aSwitch
 (aSwitch name = realName)
 ifTrue: [^aSwitch]].
 ^nil.!
componentTypeMenu: selectorArray
 "Answer a user-selected action for a
 component type."
 ^((Menu
 labels: 'Device\Signal\Switch' withCrs
 lines: #()

 selectors: selectorArray) popUpAt: MenuPosition).!
connect
 "Make a user-specified connection."
 from to 
 from := self getNode.
 from isNil
 ifTrue: [^nil].
 to := self getNode.
 to isNil
 ifTrue: [^nil].
 from connect: to.
 changed := true.
 currentComponent := from model.
 listSelector := #listComponentConnections.
 breadboard update.!
description
 "Answer a string with a description of the receiver."
 ^(self class description).!
deviceNames
 "Answer a collection of all of the
 names of installed devices."
 list 
 list := OrderedCollection new: (devices size).
 devices do: [:aDevice list add: aDevice name].
 ^list.!
devices
 "Answer the list of installed devices."
 ^devices.!
disconnect
 "Remove a user-specified connection."
 node 
 node := self getNode.
 node isNil
 ifTrue: [^nil].
 node disconnect.
 changed := true.
 currentComponent := node model.
 listSelector := #listComponentConnections.
 breadboard update.!
erase
 "After user-verification, erase
 the circuit description."
 Cursor offset: MenuPosition.
 (self verify: 'Erase circuit description?')
 ifFalse: [^nil].
 self eraseCircuit.
 listSelector := #listDevices.
 changed := true.
 breadboard update.!
eraseCircuit
 "Erase the circuit description."
 devices do: [:aDevice
 self removeComponent: aDevice].
 signals do: [:aSignal
 self removeComponent: aSignal].
 switches do: [:aSwitch
 self removeComponent: aSwitch].
 self initialize.!
getExistingComponent

 "Answer a user-specified component."
 name component reply list 
 name := self getName.
 name isNil
 ifTrue: [^nil].
 component := self componentNamed: name.
 component isNil
 ifFalse: [^component].
 Cursor offset: MenuPosition.
 (Menu message:
 (name, ' not installed -- select from list?')) isNil
 ifTrue: [^nil].
 Cursor offset: MenuPosition.
 reply := self componentTypeMenu:
 #(deviceNames signalNames switchNames).
 Cursor offset: MenuPosition.
 reply isNil
 ifTrue: [^nil].
 list := self perform: reply.
 (list size == 0)
 ifTrue: [
 Menu message: 'None installed'.
 Cursor offset: MenuPosition.
 ^nil].
 name := VariableMenu selectFrom: list.
 name isNil
 ifTrue: [^nil].
 name := list at: name.
 ^(self componentNamed: name).!
getExistingName
 "Answer a user-specified name of
 an existing component."
 component 
 component := self getExistingComponent.
 component isNil
 ifTrue: [^nil].
 ^(component name).!
getFile
 "Answer a FileStream for a
 user-specified filename."
 name 
 name := self getFilename.
 name isNil
 ifTrue: [^nil].
 ^(File pathName: name).!
getFilename
 "Answer a user-specified filename."
 name 
 Cursor offset: MenuPosition.
 name :=
 Prompter
 prompt: 'Enter filename'
 default: ''.
 Cursor offset: MenuPosition.
 ^name.!
getName
 "Answer a user-specified name."
 name 
 Cursor offset: MenuPosition.

 name :=
 Prompter
 prompt: 'Enter component name'
 default: ''.
 Cursor offset: MenuPosition.
 ^name.!
getNewName
 "Answer a user-specified name for
 a new component."
 name 
 Cursor offset: MenuPosition.
 name :=
 Prompter
 prompt: 'Enter name for new component'
 default: ''.
 Cursor offset: MenuPosition.
 name isNil
 ifTrue: [^nil].
 [(self componentNamed: name) isNil]
 whileFalse: [
 name :=
 Prompter
 prompt: 'Name exists -- enter NEW name'
 default: name.
 Cursor offset: MenuPosition.
 name isNil
 ifTrue: [^nil]].
 ^name.!
getNode
 "Answer a user-specified LogicNode."
 component 
 component := self getExistingComponent.
 component isNil
 ifTrue: [^nil].
 ^(component getNode).!
initialize
 "Private -- initialize a new
 LogicLab application."
 devices := OrderedCollection new.
 signals := OrderedCollection new.
 switches := OrderedCollection new.
 changed := true.!
install
 "Install a user-specified component."
 component 
 component := LogicComponent install.
 component isNil
 ifTrue: [^nil].
 self addComponent: component.
 listSelector := self listSelectorFor: component.
 breadboard update.
 ^component.!
installClassFrom: aStream
 "Install a LogicComponent subclass
 whose name is the next word on aStream."
 className 
 className := aStream nextWord.
 (Smalltalk includesKey: className asSymbol)
 ifFalse: [

 self error: ('Class: ', className, ' not installed')].!
installComponentFrom: aStream
 "Install a LogicComponent instance
 whose name is the next word on aStream."
 className class component 
 className := aStream nextWord.
 class := LogicComponent classNamed: className.
 class isNil
 ifTrue: [
 self error: ('Unknown class: ', className).
 ^nil].
 component := class new installFrom: aStream.
 component isNil
 ifTrue: [^nil].
 ^(self addComponent: component).!
installConnectionFrom: aStream
 "Install a connection from aStream."
 fromName from toName to fromNode toNode 
 fromName := aStream nextWord.
 from := self componentNamed: fromName.
 from isNil
 ifTrue: [
 self error: ('Unknown component: ', fromName).
 ^nil].
 fromNode := from getNodeFrom: aStream.
 fromNode isNil
 ifTrue: [
 self error: ('Unknown node on: ', fromName).
 ^nil].
 toName := aStream nextWord.
 to := self componentNamed: toName.
 to isNil
 ifTrue: [
 self error: ('Unknown component: ', toName).
 ^nil].
 toNode := to getNodeFrom: aStream.
 toNode isNil
 ifTrue: [
 self error: ('Unknown node on: ', toName).
 ^nil].
 ^(fromNode connect: toNode).!
installFrom: aStream
 "Load a circuit from the description
 on aStream."
 keyWord 
 clashes := Dictionary new.
 [(aStream atEnd)
 or: [(keyWord := aStream nextWord) isNil]]
 whileFalse: [
 keyWord = 'LOAD'
 ifTrue: [
 self installClassFrom: aStream]
 ifFalse: [
 keyWord = 'INSTALL'
 ifTrue: [
 self installComponentFrom: aStream]
 ifFalse: [
 keyWord = 'CONNECT'
 ifTrue: [

 self installConnectionFrom: aStream]
 ifFalse: [
 self error:
 ('Unknown command: ',
 keyWord)]]]].
 clashes release.
 clashes := nil.!
list
 "Process a user-specified list request."
 selection 
 selection :=
 (Menu
 labels: ('Components\Connections\',
 'Circuit Description') withCrs
 lines: #()
 selectors: #(listComponents
 listConnections
 listCircuit))
 popUpAt: MenuPosition.
 selection isNil
 ifTrue: [^nil].
 listSelector := selection.
 breadboard update.!
listCircuit
 "Answer a collection of strings with
 the circuit description."
 name stream list 
 CursorManager execute change.
 name := 'logiclab.tmp'.
 stream := File pathName: name.
 list := OrderedCollection new.
 stream
 nextPutAll: '**** Circuit Description ****';
 cr;
 cr.
 self storeOn: stream.
 stream flush.
 stream position: 0.
 [stream atEnd]
 whileFalse: [list add: stream nextLine].
 stream close.
 File remove: name.
 CursorManager normal change.
 ^list.!
listComponentConnections
 "Answer a collection of strings listing
 the connection chain(s) for the
 'currentComponent'."
 currentComponent isNil
 ifTrue: [^#()]
 ifFalse: [
 ^(#('**** Connection List ****' ' '),
 currentComponent connectionList)].!
listComponents
 "Answer a collection of strings containing
 a list of installed components."
 selection 
 selection :=
 self componentTypeMenu:

 #(listDevices listSignals listSwitches).
 selection isNil
 ifTrue: [^#()].
 ^(self perform: selection).!
listConnections
 "Answer a collection of strings listing
 the connection chain(s) for a
 user-specified component."
 component 
 component := self getExistingComponent.
 component isNil
 ifTrue: [^#()].
 currentComponent := component.
 ^self listComponentConnections.!
listContaining: aComponent
 "Answer the list (devices, signals, or switches)
 that includes aComponent."
 (devices includes: aComponent)
 ifTrue: [^devices].
 (signals includes: aComponent)
 ifTrue: [^signals].
 ^switches.!
listDevices
 "Answer a collection of strings containing
 a list of all the installed devices."
 size list 
 size := devices size.
 size == 0
 ifTrue: [^#('No devices installed')].
 size := size + 1.
 list := OrderedCollection new: size.
 list add: 'DEVICES'.
 devices do: [:aDevice list add: (' ', (aDevice identify))].
 ^list.!
listSelectorFor: aComponent
 "Answer the list selector method used
 to produce the list for aComponent's type."
 aComponent isDevice
 ifTrue: [^#listDevices].
 aComponent isSignal
 ifTrue: [^#listSignals].
 ^#listSwitches.!
listSignals
 "Answer a collection of strings containing
 a list of all the installed input signals."
 size list 
 size := signals size.
 size == 0
 ifTrue: [^#('No signals installed')].
 size := size + 1.
 list := OrderedCollection new: size.
 list add: 'SIGNALS'.
 signals do: [:aSignal list add: (' ', (aSignal identify))].
 ^list.!
listSwitches
 "Answer a collection of strings containing
 a list of all the installed swithces."
 size list 
 size := switches size.

 size == 0
 ifTrue: [^#('No switches installed')].
 size := size + 1.
 list := OrderedCollection new: size.
 list add: 'SWITHCES'.
 switches do: [:aSwitch list add: (' ', (aSwitch identify))].
 ^list.!
load
 "Load a circuit description from
 a user-specified file."
 file 
 file := self getFile.
 file isNil
 ifTrue: [^nil].
 self installFrom: file.
 listSelector := #listDevices.
 breadboard update.!
noDelay
 "Setup all components to ignore
 propagation delays."
 signals do: [:signal signal noDelay].
 switches do: [:switch switch noDelay].
 devices do: [:device device noDelay].!
noMenu
 "Private -- answer an empty menu."
 ^(EmptyMenu new).!
open
 "Open the Breadboard and Analyzer windows."
 size position 
 size := (Display boundingBox extent * 3) // 4.
 position := Display boundingBox center - (size // 2).
 topPane :=
 TopPane new
 label: ((self class description),
 ' -- Breadboard');
 model: self;
 menu: #noMenu;
 yourself.
 topPane addSubpane:
 (breadboard := ListPane new
 name: #breadboardList;
 model: self;
 menu: #breadboardMenu;
 change: #doNothing:;
 framingRatio: (0 @ 0 extent: 1 @ 1)).
 topPane reframe: (position extent: size).
 topPane dispatcher openWindow scheduleWindow.!
quit
 "Quit this LogicLab."
 (self verify: 'Quit this LogicLab?')
 ifFalse: [^nil].
 self eraseCircuit.
 signals := switches := devices := nil.
 analyzer isNil
 ifFalse: [
 analyzer closeWindow.
 analyzer := nil].
 breadboard dispatcher deactivateWindow closeWindow.
 Scheduler systemDispatcher redraw.

 Scheduler resume.!
remove
 "Remove a user-specified component."
 component 
 component := self getExistingComponent.
 component isNil
 ifTrue: [^nil].
 changed := true.
 listSelector := self listSelectorFor: component.
 self removeComponent: component.
 breadboard update.!
removeComponent: aComponent
 "Remove aComponent from the circuit."
 analyzer isNil
 ifFalse: [analyzer removeComponent: aComponent].
 (self listContaining: aComponent) remove: aComponent.
 aComponent remove.!
reset
 "Reset all components."
 signals do: [:signal signal reset].
 switches do: [:switch switch reset].
 devices do: [:device device reset].!
restoreDelay
 "Setup all components to use
 propagation delays."
 signals do: [:signal signal restoreDelay].
 switches do: [:switch switch restoreDelay].
 devices do: [:device device restoreDelay].!
resume
 "Resume the breadboarding application
 after running the simulation."
 Cursor offset: breadboard frame center.
 topPane dispatcher scheduleWindow.!
run
 "Invoke the LogicAnalyzer to run the simulation."
 analyzer isNil
 ifTrue: [
 analyzer := LogicAnalyzer new.
 analyzer openOn: self]
 ifFalse: [analyzer activate].!
save
 "Store the circuit description in
 a user-specified file."
 file 
 file := self getFile.
 file isNil
 ifTrue: [^nil].
 CursorManager execute change.
 self storeOn: file.
 file
 flush;
 close.
 CursorManager normal change.!
selectName
 "Answer a user-selected name from a list
 of the names of installed components."
 names index 
 names := self allNames.
 (names size == 0)

 ifTrue: [^nil].
 index := VariableMenu selectFrom: names.
 index isNil
 ifTrue: [^nil].
 ^(names at: index).!
signalNames
 "Answer a collection of all of the
 names of installed signals."
 list 
 list := OrderedCollection new: (signals size).
 signals do: [:aSignal list add: aSignal name].
 ^list.!
signals
 "Answer the list of installed signals."
 ^signals.!
simulate
 "Simulate one pseudo-time interval."
 signals do: [:signal signal simulate].
 switches do: [:switch switch simulate].
 devices do: [:device device simulate].!
storeClassesOn: aStream
 "Write a record of each component class
 used by the circuit on aStream."
 classes 
 classes := Set new.
 devices do: [:aDevice classes add: aDevice class].
 signals do: [:aSignal classes add: aSignal class].
 switches do: [:aSwitch classes add: aSwitch class].
 classes do: [:aClass
 aStream
 nextPutAll: ('LOAD ', (aClass name));
 cr].!
storeComponentsFrom: aCollection on: aStream
 "Write a record of each logic component from
 aCollection installed in the circuit on aStream."
 aCollection do: [:aComponent
 aStream nextPutAll: 'INSTALL '.
 aComponent storeOn: aStream.
 aStream cr].!
storeConnectionsOn: aStream
 "Write a record of each connection
 in the circuit on aStream."
 devices do: [:aDevice aDevice storeConnectionsOn: aStream].
 signals do: [:aSignal aSignal storeConnectionsOn: aStream].
 switches do: [:aSwitch aSwitch storeConnectionsOn: aStream].
 devices do: [:aDevice aDevice unMark].
 signals do: [:aSignal aSignal unMark].
 switches do: [:aSwitch aSwitch unMark].!
storeDevicesOn: aStream
 "Write a record of each logic device
 installed in the circuit on aStream."
 self storeComponentsFrom: devices on: aStream.!
storeOn: aStream
 "Write a description of the circuit on
 aStream in a form that can be recovered
 by the 'installOn:' method."
 self
 storeClassesOn: aStream;
 storeDevicesOn: aStream;

 storeSignalsOn: aStream;
 storeSwitchesOn: aStream;
 storeConnectionsOn: aStream.!
storeSignalsOn: aStream
 "Write a record of each logic signal
 installed in the circuit on aStream."
 self storeComponentsFrom: signals on: aStream.!
storeSwitchesOn: aStream
 "Write a record of each logic switch
 installed in the circuit on aStream."
 self storeComponentsFrom: switches on: aStream.!
switches
 "Answer the list of installed switches."
 ^switches.!
switchNames
 "Answer a collection of all of the
 names of installed swithces."
 list 
 list := OrderedCollection new: (switches size).
 switches do: [:aSwitch list add: aSwitch name].
 ^list.!
verify: aPrompt
 "Ask the user to verify some condition."
 Cursor offset: MenuPosition.
 ^((Menu message: aPrompt) notNil).! !






[LISTING TWO]

LogicSwitch subclass: #ToggleSwitch
 instanceVariableNames: ''
 classVariableNames: ''
 poolDictionaries: '' !

!ToggleSwitch class methods !

type
 "Answer a string with the receiver's type."
 ^'Toggle Switch'.! !
!ToggleSwitch methods !
identify
 "Answer a string identifying the receiver."
 ^((self name),
 ' (', (self type), ')').!
push: aButton
 "Simulate pushing a toggle switch by
 inverting its state."
 node invert.
 node isHigh
 ifTrue: [aButton lampOn]
 ifFalse: [aButton lampOff].
 (model isNil or: [changeSelector isNil])
 ifFalse: [model perform: changeSelector].!
reset
 "Reset the receiver."

 button isNil
 ifFalse: [
 node isHigh
 ifTrue: [button lampOn]
 ifFalse: [button lampOff]].!
simulate
 "Simulate a toggle switch."
 node output.! !






[LISTING THREE]

LogicSwitch subclass: #PulserSwitch
 instanceVariableNames:
 'rest time timer '
 classVariableNames: ''
 poolDictionaries: '' !

!PulserSwitch class methods !

type
 "Answer a string with the receiver's type."
 ^'Pulser'.! !
!PulserSwitch methods !
identify
 "Answer a string identifying the receiver."
 ^((self name),
 ' (', (self type), ' -- ',
 (LogicNode
 statePrintString: (LogicNode not: rest)), ': ',
 (TimeInterval timePrintString: time), ')').!
initialize
 "Initialize a new PulserSwitch."
 super initialize.
 rest := false.
 time := timer := 0.!

install
 "Answer the receiver with user-specified
 rest state and pulse time."
 rest := LogicNode getState. "User will select pulse state"
 rest isNil
 ifTrue: [^super release].
 rest := LogicNode not: rest.
 time := TimeInterval getTimeFor: 'pulse'.
 time isNil
 ifTrue: [^super release].
 ^self.!
installFrom: aStream
 "Answer a new PulserSwitch initialized with
 parameters read from aStream."
 super installFrom: aStream.
 rest := LogicNode stateNamed: aStream nextWord.
 node state: rest.
 time := aStream nextWord asInteger.

 ^self.!
push: aButton
 "Simulate pushing a Pulser Switch."
 timer == 0
 ifTrue: [node state: (LogicNode not: rest)].
 timer := time.
 node isHigh
 ifTrue: [aButton lampOn]
 ifFalse: [aButton lampOff].
 (model isNil or: [changeSelector isNil])
 ifFalse: [model perform: changeSelector].!
reset
 "Reset the receiver's state to its resting
 state and its timer to zero."
 node state: rest.
 timer := 0.
 button isNil
 ifFalse: [
 node isHigh
 ifTrue: [button lampOn]
 ifFalse: [button lampOff]].!
simulate
 "Simulate a Pulser Switch."
 timer == 0
 ifTrue: [
 node state: rest.
 button isNil
 ifFalse: [
 node isHigh
 ifTrue: [button lampOn]
 ifFalse: [button lampOff]]]
 ifFalse: [timer := timer - 1].
 node output.!
storeOn: aStream
 "Store a record of the receiver on aStream."
 super storeOn: aStream.
 aStream
 nextPutAll: (' ',
 (LogicNode statePrintString: rest), ' ',
 (time printString)).! !





[LISTING FOUR]

LogicDevice subclass: #N74LS00
 instanceVariableNames: ''
 classVariableNames: ''
 poolDictionaries: '' !

!N74LS00 class methods !
description
 "Answer a string with a description
 of the receiver's function."
 ^'Quad 2-input NAND gate'.!
type
 "Answer a string with the receiver's type."

 ^'74LS00'.! !
!N74LS00 methods !
initialize
 "Private -- initialize the propagation delays
 for a new 74LS00 LogicDevice."
 super
 initialize;
 initializeDelays:
 #( 5 5 10 5 5 10 0
 10 5 5 10 5 5 0 ).!
simulate
 "Simulate a 74LS00 device."
 ((pins at: 1) isHigh and: [(pins at: 2) isHigh])
 ifTrue: [(pins at: 3) output: false]
 ifFalse: [(pins at: 3) output: true].
 ((pins at: 4) isHigh and: [(pins at: 5) isHigh])
 ifTrue: [(pins at: 6) output: false]
 ifFalse: [(pins at: 6) output: true].
 ((pins at: 10) isHigh and: [(pins at: 9) isHigh])
 ifTrue: [(pins at: 8) output: false]
 ifFalse: [(pins at: 8) output: true].
 ((pins at: 13) isHigh and: [(pins at: 12) isHigh])
 ifTrue: [(pins at: 11) output: false]
 ifFalse: [(pins at: 11) output: true].! !






[LISTING FIVE]

output: aState

 "Generate aState as an output from the node."

 old := int.
 int := aState.
 int == ext
 ifTrue: [
 "State is stable"
 timer := 0.
 ^self outputToConnections].
 "State has changed"
 timer == 0
 ifTrue: [
 "No delay in progress -- initiate prop delay"
 delay == 0
 ifTrue: [
 "No delay -- just change state"
 ext := int]
 ifFalse: [
 "Arm delay timer"
 timer := delay].
 ^self outputToConnections].
 "Propagation delay in progress"
 timer := timer - 1.
 timer == 0
 ifTrue: [

 "Timer has expired -- update state"
 ext := int].
 self outputToConnections.





[LISTING SIX]

simulate

 "Simulate a 74LS00 device."

 ((pins at: 1) isHigh and: [(pins at: 2) isHigh])
 ifTrue: [(pins at: 3) output: false]
 ifFalse: [(pins at: 3) output: true].
 ((pins at: 4) isHigh and: [(pins at: 5) isHigh])
 ifTrue: [(pins at: 6) output: false]
 ifFalse: [(pins at: 6) output: true].
 ((pins at: 10) isHigh and: [(pins at: 9) isHigh])
 ifTrue: [(pins at: 8) output: false]
 ifFalse: [(pins at: 8) output: true].
 ((pins at: 13) isHigh and: [(pins at: 12) isHigh])
 ifTrue: [(pins at: 11) output: false]
 ifFalse: [(pins at: 11) output: true].




































December, 1989
ARE THE EMPEROR'S NEW CLOTHES OBJECT ORIENTED?


Scott Guthery


Scott is a scientific advisor at Schlumberger's Austin System Center in
Austin, Texas, where he was the chief software architect of Schlumberger's new
family of wellsite data acquisition systems. He has a Ph.D. in probability and
statistics from Michigan State University and he has been programming since
1957. He can be reached through Internet: @guthery asc.slb. com.


A large number of sweeping claims, particularly with respect to programmer
productivity and code reuse, are being made for object-oriented programming
(OOP), many of them by firms that are in the business of making and selling
object-oriented programming tools. Before we bet a real programming project on
this freshly sewn technology, we'd like to ask some questions about
object-oriented programming just to make sure the emperor's new clothes are
really as fine as these tailors claim.


Where's the Evidence?


In scientific and engineering disciplines -- besides computer science and
programming -- we accumulate experience, gather evidence, and conduct
experiments, which we then generalize to make claims. If one doubts the claims
one is invited to reanalyze the data and replicate the experiments. For some
strange reason, programmers have a history of accepting claims blindly without
asking for proof. Hope springs eternal, I guess, and dire need grasps any
straw. To my knowledge there has been absolutely no evidence gathered or
experiments performed to validate the claims made for object-oriented
programming, particularly for object-oriented programming in the large.
The biggest OOP projects undertaken to date seem to be the OOP development
systems, and the news from this front is not all good [Harrison, 1989]. But
OOP isn't supposed to be an end in itself. Esperanto was wonderful for writing
books about Esperanto but not much else. If you want to write an OOP system,
then OOP is probably just the thing. But if you want to write, say, an
accounting system or a reservation system using OOP you're going where no man
or woman has gone before. In fact, you'll be performing on yourself the very
experiments that the OOP peddlers should have performed to substantiate their
claims. Would you accept this situation if OOP were, for example, a new
surgical procedure?


What is an Object?


The atomic element of object-oriented programming is, not surprisingly, the
object. But what is an object? The 51 papers in the IEEE "Tutorial on
Object-Oriented Computing" by Gerald Peterson [Peterson, 1987] contain many
definitions and descriptions of an object. These definitions come in two basic
flavors. One flavor talks about modeling reality and the other talks about
encapsulated collections of programming tricks.
Stripped of its fancy jargon, an object is a lexically-scoped subroutine with
multiple entry points and persistent state. Object-oriented programming has
been around since subroutines were invented in the 1940s. Objects were fully
supported in the early programming languages AED-0, Algol, and Fortran II. OOP
was, however, regarded as bad programming style by Fortran aficionados. As
Admiral Grace Hopper has so often observed, we don't actually do anything new
in computing we just rename the old stuff. Admiral Hopper (at the time Lt.
Hopper) was doing object-oriented programming on the Harvard Mark I in 1944
and probably didn't even know it.
Unfortunately, we have completely ignored Rentsch's 1982 plea [Rentsch, 1982]:
"Let us hope that we have learned our lesson from structured programming and
find out what the term means before we start using it." C++ was first
described in April, 1980 [Stroustrup, 1980]. Over nine years later an
incompatible Version 2.0 has just been released. The definition of C++ still
isn't complete. Can anything that's this hard to define be good for writing
understandable and maintainable programs? And if you think it's hard to pin
down the definition of an object, just try drawing a bead on the definition of
the inheritance links that connect objects.


What's the Cost of OOP Code Reuse?


One of the primary claims of object-oriented programming is that it
facilitates the reuse of code. Does it? And if so, at what cost?
The unit of reuse in object-oriented programming is the hierarchy. It is the
hierarchy and not the object that is the indivisible whole. Unlike a
subroutine library where you can just take what you need and no more, in
object-oriented programming you get the whole gorilla even if you just wanted
the banana.
The problem is that hierarchies are nonmodular. You can't just clip the
objects you want to reuse out of the hierarchy because you don't know (in
fact, aren't supposed to know) how the objects are entangled in the hierarchy.
So, the cost of OOP-style code reuse seems to be that you must (re)use a lot
more code than you want or need. Your system will be bigger, run slower, and
cost more to maintain than with subroutine-style reuse. Though there may be
situations in which the convenience of the programmer so completely outweighs
the interests of the users and the interests of the maintainers, I've never
seen one.


How to Combine Object Hierarchies?


If object hierarchies need to be small to control the cost of their reuse then
you must be able to get many of them to work together when you build your
program. You may, for instance, want to use a polynomial approximation
hierarchy, a linked list hierarchy, a communication hierarchy, an indexed
record hierarchy, a pop-up menu hierarchy, and a ray-tracing hierarchy all at
once.
But how do you combine object hierarchies? Can objects in a C++ mathematics
hierarchy send arguments to objects in an Objective-C ray-tracing hierarchy?
Sadly, no. What's worse is that you can't even send arguments from one C++
hierarchy to another. There are neither in theory nor in practice any OOP
hierarchy combiners.
It is left as an exercise for the OOP programmer to "impedance match" not only
between OOP technologies but between OOP hierarchies within a technology. This
means doing exactly what you were told you wouldn't have to do; open up the
objects and program with respect to representation of the state inside. The
object-oriented programmer must map from one internal representation to
another. There is, after all, no reason to suspect that one hierarchy's
internal representation of a compound object such as a matrix or a picture is
anything like another's. This clearly defeats one of the main advertised
benefits of object-oriented programming: Namely, hidden internal
representation. What you may have saved by not having to write code for
objects in the same hierarchy, you now must spend as you write code to map
between objects in different hierarchies.
One of the few things that we have learned (again and again) over the last 40
years of programming is that the hard part isn't getting code fragments to
work. The hard part is getting them to work together. The name of the game,
particularly when it comes to code reuse, is integration at scale.
Object-oriented programming makes building code fragments easier, but it makes
integration much more difficult. Making the easy parts easier but the hard
parts harder is not progress by my lights.


How to Tune an Object-Oriented Program?


Has any program you've ever written been too fast or even fast enough? What do
you do if your object-oriented program isn't fast enough? How do you
performance tune an object-oriented program? Indeed, how do you even answer
the question, "Where is the program spending its time?"
It's just possible you'll find yourself spending lots of time in one or two of
your own methods and can work on making those methods faster using classic
techniques. But it's much more likely that you'll find you're spending more
time than you care to running around the hierarchy.
There is only one thing you can do: Rearchitect and reorganize the hierarchy
itself to make it more efficient and to take into account the way you want to
use it. The semantics of the hierarchy thus become a twisted combination of
the descriptive reality that the objects came from and the profile of the use
your procedural code makes of them. This is not an attractive prospect.
The problem here, of course, is that while the semantics of classic
programming languages match the semantics of the underlying hardware, the
semantics of object-oriented languages do not. When using classic languages
like C or Fortran, if you couldn't bind the problem to the hardware tightly
enough to get the performance you needed, you could make this binding tighter
yet by resorting to assembly language or even microcode. You can't do this
with an object-oriented program because you can't get at the virtual machine
that implements the semantics of these languages. They're all hidden away from
you in the vendor's compiler and runtime library.
Once again, the programmer is being invited to pass the cost of expedience
onto the user of the system. The additional cost of supporting a runtime OOP
virtual machine can vary from as little as 50 percent [Thomas, 1989] to as
much as 500 percent of the cost of a non-OOP version of the system. This
wholesale sacrificing of runtime efficiency to programmer's convenience, this
emphasis on the ease with which code is generated to the exclusion of the
quality, usability, and maintainability of that code, is not found in any
production programming environment with which I am familiar.
Finally, before we leave the topic of hardware, let's not forget the Intel
432. The 432 was OOP in silicon and it failed because it was just too slow. If
we couldn't make OOP efficient when we implemented it in hardware why do we
think we can make it efficient when we emulate it in software?


How to Manage the OOP Development Team?



Real programs are built by large programming teams, not by individuals or
small, closely knit cliques. Because we certainly don't want to imagine that
every programmer on a project builds his or her own private object hierarchy,
we are faced with the prospect of many programmers working on the same tree.
Given something as flexible as an object to work with, it is almost certain
that each programmer working on the tree will want to implement a different
vision of the reality that the tree is attempting to capture.
One possibility is to appoint an object "czar," the direct analogy of a
database administrator. Databases need to be stable, so appointing an
administrator to watch over the database schema and carefully coordinate
changes to it makes good software engineering sense. Object hierarchies, on
the other hand, are deliberately not stable; the hierarchy is the program
after all and it's the program that we're developing. Imagine having to ask
the permission of the subroutine czar every time you wanted to write a
subroutine.
What really happens? What I've seen in three large (7000+ objects) OOP
projects is that because everyone is trying to get his or her job done with a
minimal number of dependencies on everyone else, subtrees and subrealities
spring up all over the place and new objects and new methods sprout like
weeds. There was an object in one of these systems which when printed went on
for 80 pages. One also finds lots of little private languages for
communicating between these subrealities.
Of course, good communication between the team members can attenuate the
growth of some of this gratuitous complexity. But, in projects on tight
schedules with programmers removed from one another in time, space, and
organization, predicating success on good communication adds more risk to an
already risky undertaking.
Another distressing property of these multi-programmer hierarchies is that
they're difficult to debug. If there is one overarching flaw in OOP, it's
debugging. As was noted recently in the OOP newsgroup on USENET, "It has been
discovered that C++ provides a remarkable facility for concealing the trivial
details of a program -- such as where the bugs are."
While we're passing through this analogy with database management systems,
recall that one of the raison d'etre for DBMSs was the separation of data and
program. Now, along comes OOP and we're told that mixing data and program is
the right thing to do after all. Were we wrong then or are we wrong now?


Do Object-Oriented Programs Coexist?


We have gotten used to mixing languages in our programs. This is
industrial-strength code reuse in action; if you can't access it at will, you
can't reuse it. You don't have to rewrite a Fortran subroutine into C to use
it in a C program, you just call it. Common or at least coercible calling
conventions and a uniform linking model have made this possible. One of the
many reasons that Lisp has failed as a programming language is that Lisp is a
language loner.
How about object-oriented languages? Can you mix Objective-C, Eiffel, CLOS,
Actor, Owl, and C++ objects in a tree? Not on your tintype. Object hierarchies
are isolated bunker realities just like the language technologies that
implement them.
We have learned that there is no all-singing, all-dancing anything in
computing. No one language, no one communication protocol, no one operating
system, no one graphics package -- no one anything is always right all the
time everywhere. We have learned again and again that closed systems are
losers. Successful systems have one thing in common -- they can coexist
peacefully and gracefully with other systems. Object-oriented programming does
not currently have this property, either in concept or in practice. If by
"reuse" OOP advocates really mean "reuse when the whole world is just like me"
then this is not reuse in any practical or useful sense.


What are the Consequences of Persistent State?


Persistent state means that data obtained from an object cannot be used
independently of that object. It means that the very act of obtaining a value
invalidates all other previously obtained values. Programmatically, this means
that every time you want to use a value you have to retrieve it from the
hierarchy. It is a programming error to make a local copy of a value.
Hierarchy chasing and the inheritance machinery are not only in the inner loop
of every orthodox object-oriented program, they are part and parcel of every
use of every value in the program.
But, persistent state isn't only a performance issue. It is much more
importantly a data consistency issue. The only correct way to get two or more
consistent values from an object hierarchy is to get them together in one
package in one response to one query. This is the only way you can be assured
that they are consistent each with the other. Not only have I not seen any
discussion of this property of OOP, I have seen example object-oriented
programs that don't understand the consequences of persistent state and simply
assume consistency between values obtained serially. Without explicit
assurances from the designers of the hierarchy in use, this is an error.
There was a very good reason why persistent state was regarded as bad Fortran
programming style -- it's a semantic mine field. Why do we have to completely
rediscover the principles of good programming with each new programming
language and paradigm we invent? In all but very restricted and tightly
controlled situations experience has shown that persistent state should be
avoided. For those who haven't taken a stroll in this mine field,
object-oriented programming offers the opportunity to avoid reusing other's
experience and learn for themselves. As with modularity and the separation of
programs and data, OOP seems content to simply ignore what we have learned in
40 years of programming.


Can You Get the Development System Out of the Production System?


A common failing of many programming aids such as OOP is that you can't get
rid of them when you're done with them. They're like training wheels on a
bicycle except you can't take them off when you've learned to ride.
Programming languages such as Lisp and methodologies such as OOP are
particularly painful because they are based on a virtual machine that sits
between you and the real machine. The virtual machine is a nice warm-fuzzy to
have during development but we simply can't afford to have it in our
production systems.
Structured programming was such a success because you got all the benefits of
enhanced software productivity without any runtime penalty. We don't know yet
what the minimal runtime cost of OOP is but our inability to measure it and
hence engineer it should certainly give us pause. I'm uncomfortable working
with a programming paradigm whose runtime cost I can't even estimate, let
alone eliminate.


Conclusion


Object-oriented programming runs counter to much prevailing programming
practice and experience: In allocating and controlling software costs, in
modularity, in persistent state, in reuse, in interoperability, and in the
separation of data and program. Running counter to prevailing wisdom does not,
of course, automatically make an innovation suspect but neither does it
automatically recommend it. To date, in my opinion, advocates of
object-oriented programming have not provided us with either the qualitative
arguments or the quantitative evidence we need to discard the lessons
painfully learned during the first 40 years of programming.


Bibliography


Harrison, William H., John J. Shilling, and Peter F. Sweeney, "Good News, Bad
News: Experience Building a Software Development Environment Using the
Object-Oriented Paradigm," IBM Technical Report RC 14493, March 3, 1989.
Petersen, Gerald E., "Tutorial: Object-Oriented Computing," Computer Society
Press of the IEEE, Order Number 821 and 822, 1987.
Rentsch, Tim, "Object-Oriented Programming," SIGPLAN Notices, Volume 17,
Number 9, (September 1982) pp. 51-57.
Stroustrup, Bjarne, "Classes: An Abstract Data Type Facility for the C
Language," Bell Laboratories Computing Science Technical Report No. 84, April
3, 1980.
Thomas, Dave, "The Time/Space Requirements of Object-Oriented Programs,"
Journal of Object Oriented Programming, March/April 1989, pp. 71-73.
















December, 1989
PDQ: LESS BAGGAGE, FASTER CODE


Doing away with bloated code




Bruce W. Tonkin


Bruce develops and sells software for TRS-80 and MS-DOS/PC-DOS computers. You
may reach him at T.N.T. Software Inc., 34069 Hainesville Rd., Round Lake, IL
60073.


To a user, the biggest difference between a compiled program written in a
high-level language and a compiled program written in assembly language is the
size of the executable (EXE) file. The majority of compilers, regardless of
language, produce large EXE files even when the size of the original program
is trivial. Most serious programmers would like to have an alternative to this
problem of bloated code.
QuickBasic programmers, take note: Crescent Software has found a way to put
those fat EXE files on a severe diet, shrinking some of them to less than 1K
bytes. As a bonus, Crescent's PDQ library allows Basic TSR programs, and
permits Basic programs to return an error level to MS-DOS.
The large size of most EXE files partially result from the way in which a
language's libraries are arranged. According to Crescent Software, the
Microsoft QuickBasic libraries are typical: If a program contains a single CLS
command (which clears the screen), the linker also includes routines for
COLOR, CSRLIN, POS(O), LOCATE, and SCREEN (x,y). Most of these routines also
include code that can manage screens in graphics mode, whether or not that
code is needed. The addition of other simple commands and functions adds even
more overhead. This library "clumping" factor is called "granularity."
I used LIB.EXE to investigate Crescent's claims. A simple BEEP command in
QuickBasic 4 costs 923 bytes of overhead, and a single ASC( ) function adds
554 bytes to the size of an EXE file! That's granularity with a vengeance.
Another part of a standard EXE file is taken up by routines that start a
program and determine the type of video adapter attached, the keyboard, the
current screen and graphics mode, and so on. Still more overhead is added in
order to handle the process of program termination. This overhead is easily
seen when a null program is compiled: In QuickBasic 4.5, a program that
contains only an END statement compiles to an EXE of 9842 bytes in size!
Crescent claims that the QuickBasic compiler actually produces very efficient
code. If this is true, it should be possible to produce very small executables
by making the library more efficient and by reducing the size of default
start-up and termination code. This method seems to be exactly what Crescent
has done.
Crescent's PDQ is a substitute library for those who use the Microsoft
QuickBasic compiler, Version 4.0, or higher. The PDQ library is intended to
replace the BCOM4x.LIB file with a subset library that has smaller granularity
and less sophistication.
"Less sophistication" means that no floating-point routines are available.
There are no routines to draw circles. Error handling is greatly simplified;
only eight error codes are supported. Disk files must be either sequential or
binary. All arrays must be static, however, an allocMEM function is provided
to simulate dynamic arrays. PDQ provides no random-number generator, no ON
anything, no DATA statement or READ/RESTORE commands, TAB, PRINT USING, or
WRITE.
Is this too much to give up? I don't think so, and I'll bet you won't, either.
I wouldn't use the PDQ library to write a billing system, but that's not PDQ's
intended use. Before the introduction of PDQ, I thought that only C or
assembly would be appropriate for the inevitable menus, filters, translators,
and miscellany that are part of most systems. Now, I intend to use PDQ to
develop most of them.
In fact, Crescent has done what many programmers have been requesting. For
years, compiler vendors have been increasing the size of the default code that
is automatically linked to each program. To handle the proliferation of
hardware options, vendors supply many library routines in the most general
form possible -- and that only makes matters worse.
It's hard to argue with the apparently sensible decision to supply general
routines. If I write a program, I want to be sure that it will run whether the
user has VGA, MDA, or anything in-between. On the other hand, many programs
are strictly character-mode and need nothing more than the most elementary TTY
functions. Some file utilities may require only the simplest I/O. In those
cases, why should the developer have to accept an extra 25K (or more) of
unnecessary run-time code?
The availability of different (individual selectable) versions of library
routines would provide a real advantage for developers. (I've long argued that
developers would be willing to pay for such libraries.) Even better, I'd like
the option to use some sort of meta-command to specify which libraries will be
used for which parts of a program. Given the current levels of complication
and sophistication in compilers and libraries, this may be a lot to ask.
At the very least, the libraries should have finer granularity. It's probably
true that the process of linking libraries that have a very fine granularity
takes more time, and that these libraries may even require the use of multiple
link passes. I'm willing to make that sacrifice, and I think that most
developers would be willing to do so too -- at least for final distribution
versions of software.
Another potential problem exists with respect to reduced granularity. Most
modules are aligned on paragraph boundaries, meaning that each module wastes
an average of half a paragraph -- 8 bytes. The more modules, the more wasted
memory. For speed reasons, modules should at least be aligned on word
boundaries, this technique causes an average waste of one byte per module.
This waste factor doesn't sound like much of a problem, but it can become one
if a program contains several thousand very small modules. The use of a more
intelligent linker or an integrated compiler/linker might cut at least some of
this waste.
I've told several compiler vendors that I would be willing to accept compile
times of an hour or longer on a 386 machine -- provided that the compiler
produced the tightest, most highly optimized code possible, and that it linked
only the routines that were absolutely necessary. I would be willing to pay a
good deal of money for such a Basic compiler and linker. (For whatever
language, I'm not the only developer with those opinions.) I would also be
willing to debug with something faster but less efficient.
From that viewpoint, Crescent's PDQ may signal the start of something big. If
PDQ becomes a success, it may spur the major compiler vendors to offer
something equivalent to PDQ or better. Most developers are not reconciled to
working with large executables that need a lot of memory. Speed and efficiency
will always have appeal.
To see how well Crescent has succeeded with PDQ, let's consider some examples
of simple programs written in QuickBasic and modified for the PDQ library.
Example 1lists the (hoary) standard SIEVE program, and Example 2 shows the
equivalent program modified for PDQ.
Example 1: Standard Sieve program

 DEFINT A-Z
 DIM flags (8190)
 CLS
 PRINT "25 iterations"
 x# = TIMER
 FOR j = 1 TO 25
 count = 0
 FOR i = 0 TO 8190
 flags(i) = 1
 NEXT i
 FOR i = 0 TO 8190
 IF flags (i) THEN
 prime = i + i + 3
 k = i + prime
 WHILE k <= 8190
 flags (k) = 0
 k = k + prime
 WEND
 count = count + 1
 END IF
 NEXT i
 NEXT j
 xx# = TIMER
 PRINT USING "#### primes in ##.###

 seconds";count;xx#-x#
 END

Example 2: PDQ version of the Sieve program

 rem $include: 'pdqdecl.bas'
 DIM flags (8190)
 CLS
 PRINT "25 iterations"
 start&=pdqtimer&
 FOR j = 1 TO 25
 count = 0
 FOR i = 1 TO 8190
 flags(i) = 1
 NEXT i
 FOR i = 0 TO 8190
 IF flags (i) THEN
 prime = i + i + 3
 k = i + prime
 WHILE K <= 8190
 flags (k) = 0
 k = k + prime
 WEND
 count = count + 1
 END IF
 NEXT i
 NEXT j
 done&=pdqtimer&
 tot&=(10*(done&-start&))\182
 'convert timer ticks to seconds.
 frac&=(10000&*(10* (done&-start&)
 mod 182))\182 'and decimal.
 PRINT count; "primes in ";rtrim$
 (str$(tot&));".";frac&;"seconds."
 END

I also compiled a modified version of the program (under PDQ and QB 4.5) with
all PRINT and TIMER statements removed. Table 1 shows the file sizes after
this program is compiled.
Table 1: File sizes after PDQ compilation

 PDQ QB 4.5
 Std. No Print Std. No Print

---------------------------------------------------------------


 SIEVE.BAS 590 352 452 352
 SIEVE.OBJ 1585 951 1226 951
 SIEVE.EXE 2620 1282 27544 12522

In each case, the EXE file created by PDQ is about a tenth of the size of the
EXE file created by the standard QuickBasic library. Few changes were
necessary, and the program ran just as quickly -- it ran in about 3.24 seconds
on my computer.
Notice the effects of library granularity. Removing the PRINT, TIMER, and
double-precision subtraction routines sliced the size of the QB 4.5 EXE file
by 15K bytes. A simple TTY print routine, plus the bytes contained in the
actual strings, might use a total of 150 bytes. TIMER ought to not use more
than a dozen or so bytes. Double-precision subtraction routines and the
numeric-to-ASCII conversions would surely require less than 2K bytes. The rest
of the 15K overhead in the QB 4.5 EXE file comprises all of the unnecessary
linked code.


A Small Utility Program


Consider a small filter utility. Most programmers would write such a utility
in C or in assembly language in order to generate the smallest EXE possible.
The filter utility simply inspects the text in the file named on the command
line and converts all lowercase letters to uppercase. This utility looks like
the program in Example 3. The filter utility is straightforward and the task
is trivial, but this type of program is the core of most file-filter
utilities. The file sizes produced with PDQ were:
Example 3: Sample filter utility


 defint a-z 'by default, all variables are integers
 call initstr(a$, 128) 'initialize a$ to 128 characters
 call initstr (b$, 8192) 'buffer for file is 8K bytes
 a$=command$ 'look at the command line
 if rtrim$(a$)="" then 'no file named; give help.
 print"Syntax:"
 print "UCASE filename"
 print "The named file will be converted in place to upper-case."
 end
 end if
 open rtrim$(a$) for binary as #1 'no random files in PDQ
 size&=lof(1) 'file size
 if size&=0 then close 1:kill rtrim$(a$): end 'no such file
 where&=1 'start with the first byte
 while where&<=size& 'as long as there are bytes to read
 remains&=size&-where&+1 'how much is left?
 if remains&<8192 then call setlength (b$, remains&) '< 8192 bytes
 get 1,where&, b$ 'grab the next chunk to convert
 b$=ucase$(b$) 'upper-case it
 put 1, where&, b$ 'write it back out
 where&=where&+len(b$) 'update the file position
 wend
 end

 Std. No Print

 UCASE.BAS 592 459
 UCASE.OBJ 1723 1553
 UCASE.EXE 2578 2348
Note: I added some remarks for publication, so the source code size will not
match the numbers shown above.
UCASE.BAS ran with impressive speed, converting a 104K-text file to uppercase
letters in about three seconds on my Tandy 4000 (16-MHz 80386, no math
coprocessor, Plus Development Hard Card 40, with 64K cache enabled). Further,
on this small program, PDQ was one-third the size of the equivalent program
written in QuickC and Turbo C.
PDQ also contains several alternate libraries. These libraries provide special
versions of various modules, optimized for the 80286 and 80386 processors. The
PDQ documentation mentions that the 80386 library is especially suitable for
use with long-integer calculations, most notably long-integer divisions.
To test PDQ's speed during various operations, I used a modification of the
benchmark programs presented in my earlier QuickBasic review (DDJ, November
1988). The results are shown in Table 2.
Table 2: Benchmark results

 Times per Million Long Integer Operations

 8088 80286 80386 QB4.5
 Raw loop: 2.76 2.75 2.24 13.63
 Assignments: 1.86 1.86 1.37 1.75
 Additions: 16.20 16.20 13.73 14.34
 Subtractions: 16.20 16.20 13.73 14.40
 Multiplications: 28.28 27.40 22.95 25.33
 Divisions: 32.46 32.46 25.04 32.20
 Comparisons: 12.02 12.08 11.25 10.54

 Times per Million Operations

 8088 80286 80386 QB4.5
 Fixed string assignments: 35.81 36.52 36.63 51.43
 Fixed string MID$ operations: 36.03 36.24 35.59 23.67
 Fixed string "concatenations": 36.03 36.24 35.59 24.26

 Miscellaneous Times and Size

 8088 80286 80386 QB4.5
 Print 1,000 70-byte strings to screen: 27.02 27.02 27.02 9.93
 Executable size (bytes): 7,346 7,344 7,126 46,139


Generally speaking, the PDQ routines are equivalent to the routines in the
standard QB 4.5 library. The PDQ 80386 routines are appreciably faster at
long-integer operations, as advertised.
Regardless of the library used, the long-integer assignment operations are
much faster with PDQ. I know no reason for this difference and few programs
use enough assignment operations to make this difference in speed noticeable
to the programmer.
TestHotkey StuffBuf
IntEntry 1 ResetKeyboard
PopUpHere PopDown
PointIntHere IntEntry2
GotoOldInt EndTSR
CallOldInt
Rather than explaining each of these commands in detail, I'll present a sample
TSR program that uses them.
I altered one of Crescent's many demonstration programs to output keyboard
directly to screen memory. This routine's output was not used because it
cannot be redirected. (The ability to direct output is of great use in utility
programs.) Also the routine does not automatically scroll.
The slower print speeds occur because PDQ's standard PRINT commands go through
DOS services. PDQ also contains a PDQPRINT routine that prints macros. The
sample program intercepts calls to interrupt 9 (the standard keyboard
interrupt). If the key combination pressed is ALT-A, ALT-S, or ALT-D, the TSR
program clears the keyboard buffer and inserts the defined sequence of
keystrokes into the same buffer. Non-trapped keys are passed to the standard
Int 9 handler. You may not stuff more than 15 characters into the keyboard
buffer (as mentioned in the PDQ documentation).
PDQPRINT is useful when a programmer needs to force screen printing (even when
the standard output has been redirected), or when greater speed is necessary.
In addition, PDQPRINT supports color attributes; the PRINT routine in PDQ
doesn't.
I also compared the speeds of the integer operations, and found little
difference between PDQ and QB 4.5. Realistically, integer operations are so
fast in nearly any compiler that much greater performance increases result
from the use of optimization techniques rather than from the use of tricks for
integer arithmetic. Because PDQ uses the code generation of QB 4, any
optimization (or lack of it) in QB's code generation is reflected in PDQ. The
performance changes from PDQ result only from changes in library routines.


TSR Power


The PDQ library does more than just remove functions and commands -- it also
adds some new ones. The most notable added commands and functions make TSR
programs possible. Some of these new commands and functions are:
The sample programs furnished with PDQ do work. If you have any doubts about
the correct way to do something with PDQ, look at those examples carefully. I
accidentally omitted the final call ReturnFromInt for ALT-D, and was sure I'd
found a bug. When my program was changed to a TSR program, all attempts to use
macros failed after the first time that ALT-D was pressed. I'm not exactly
sure why this happened, but the error was certainly mine and not PDQ's.
Even Basic's executables would make such TSRs rather wasteful. With PDQ, TSRs
are small enough to be practical. The sample program that I just presented
results in an EXE file of only 2404 bytes in size.


The Manual


The PDQ documentation supplied with the first release of the library was in
preliminary form, and will be changed soon. Ethan Winer of Crescent Software
told me that only 100 manuals were printed for the first release of PDQ, so
criticisms of the documentation may be moot.
It is important to note that if your program uses large static arrays, you
should use the /ex option on the link command line. (This option is not
mentioned in the first version of the manual.) The /ex option packs space used
by static arrays and results in much smaller EXE files. Without the /ex
option, the SIEVE program compiled to approximately 18K under PDQ. I also used
the /ex option when I compiled with QB 4.5.
This first version of the manual was disorganized in places, but was fairly
easy to use. The information about the new commands and the changes from QB 4
were well organized, but the discussion of TSR programs was incomplete. In
some places, the manual didn't match the syntax of the first release.
Fortunately, the distribution disk contains a generous number of helpful
example programs, all of which work. If you purchase PDQ, I recommend that you
read the manual and then pay very close attention to the examples on the disk.
From previous experience, I knew that Crescent Software's manuals are chatty
and easy to read. The manual for PDQ is typical. Ethan Winer clearly knows a
lot about the internals of QuickBasic, and he's not shy about sharing his
knowledge. The manual contains a number of performance hints, tips for
reducing the size of typical PDQ and QuickBasic programs, and even ways to
call QuickBasic library routines directly. One such example shows how to force
QuickBasic to do the equivalent of MAT READ.
On balance, I liked the manual very much despite its early flaws; later
versions of the manual should be better.


Overall Utility


PDQ is aimed at QuickBasic programmers, but it has clear applicability in
other areas. In the past, I resorted to the use of assembly language or C to
write small utilities. I resented the general unreadability of C and the more
complicated compilation process that it requires. Also, because of the
low-level nature of assembly and C, I found the debugging process to be far
more tedious than the process of debugging Basic, even with the new versions
of C and assembly from Borland and Microsoft. As long as I benefited from
efficiency and speed, I was willing to put up with those disadvantages.
Granted, there will still be times when C or assembly are the most suitable
tools for the job. PDQ simply makes those times less frequent, and makes my
work easier.
Programmers who are unfamiliar with either C or assembly can now attempt more
projects. With the advent of PDQ, Basic can only gain stature as a readable
and efficient language that is suitable for both small and large projects. As
a long-time Basic enthusiast, I can't pretend to be unbiased about the
prospect.
PDQ is a watershed product. I think it is the first in a series of efficient
compilers and alternative libraries. In the next few years, we'll almost
certainly see similar attempts for C, Pascal, and other popular languages.
Subsequent products will surely include ever more efficient compilers and
linkers, benefiting us all.
Whether or not you find PDQ appealing, I feel sure that the approach taken by
Crescent Software is the approach that the whole industry will follow in the
future. Other developments are doubtful -- this one is not. Bloated code may
finally be doomed.


Product Information


P.D.Q. Crescent Software, Inc.
11 Grandview Ave.,
Stamford, CT 06905
Includes full source code
For IBM PC, XT, AT, or compatible
$99


_PDQ: LESS BAGGAGE, FASTER CODE_
by Bruce W. Tonkin

[EXAMPLE 1]


DEFINT A-Z
DIM flags(8190)
CLS
PRINT "25 iterations"
x# = TIMER
FOR j = 1 TO 25
 count = 0
 FOR i = 0 TO 8190
 flags(i) = 1
 NEXT i
 FOR i = 0 TO 8190
 IF flags(i) THEN
 prime = i + i + 3
 k = i + prime
 WHILE k <= 8190
 flags(k) = 0
 k = k + prime
 WEND
 count = count + 1
 END IF
 NEXT i
NEXT j
xx# = TIMER
PRINT USING "#### primes in ##.### seconds";count;xx#-x#
END


[EXAMPLE 2]

rem $include: 'pdqdecl.bas'
DIM flags(8190)
CLS
PRINT "25 iterations"
start&=pdqtimer&
FOR j = 1 TO 25
 count = 0
 FOR i = 0 TO 8190
 flags(i) = 1
 NEXT i
 FOR i = 0 TO 8190
 IF flags(i) THEN
 prime = i + i + 3
 k = i + prime
 WHILE k <= 8190
 flags(k) = 0
 k = k + prime
 WEND
 count = count + 1
 END IF
 NEXT i
NEXT j
done&=pdqtimer&
tot&=(10*(done&-start&))\182 'convert timer ticks to seconds.
frac&=(10000&*(10*(done&-start&) mod 182))\182 'and decimal.
PRINT count;"primes in ";rtrim$(str$(tot&));".";frac&;"seconds."
END


[EXAMPLE 3]


defint a-z 'by default, all variables are integers
call initstr(a$,128) 'initialize a$ to 128 characters
call initstr(b$,8192) 'buffer for file is 8K bytes
a$=command$ 'look at the command line
if rtrim$(a$)="" then 'no file named; give help.
 print"Syntax:"
 print"UCASE filename"
 print"The named file will be converted in place to upper-case."
 end
 end if
open rtrim$(a$) for binary as #1 'no random files in PDQ
size&=lof(1) 'file size
if size&=0 then close 1:kill rtrim$(a$):end 'no such file
where&=1 'start with the first byte
while where&<=size& 'as long as there are bytes to read
 remains&=size&-where&+1 'how much is left?
 if remains&<8192 then call setlength(b$,remains&) '< 8192 bytes
 get 1,where&,b$ 'grab the next chunk to convert
 b$=ucase$(b$) 'upper-case it
 put 1,where&,b$ 'write it back out
 where&=where&+len(b$) 'update the file position
wend
end






































December, 1989
FUNCTIONAL PROGRAMMING AND FPCA '89


A conference report




Ronald Fischer


Ronald is a software developer for Software-Entwicklung & Consulting and can
be reached at Straubinger Strasse 20, D-8000 Munchen 21, W. Germany.


Editor's Note: Over the past few years, a new method of developing and writing
computer programs called "functional programming" has evolved without getting
much attention from the general public. Nevertheless, this novel paradigm may
influence programming five to ten years from now. Every second year, the
advances in this area are discussed at the Conference on Functional
Programming and Computer Architecture (FPCA). FPCA '89 was held in London,
England and DDJ correspondent Ronald Fischer was there; here is his report.
To start with, forget everything you have ever learned about computer
programming. During the last decade, a new way of writing programs has slowly
developed: The "Functional Programming Paradigm," which lacks some of the most
salient features of traditional programming languages. Most notably among
those missing features are looping constructs and the destructive assignment
to memory variables. In some respects, this new programming style appears to
be more like writing mathematical equations than developing algorithms. Its
proponents in fact hope that it leads to more bug-free software, because
programs may be proved correct by mathematical means, instead of by debugging.
As an example, consider a function SUM, which returns the sum of a list of
numbers:
 DEC SUM: LIST(NUM)-> NUM;
 --- SUM(NIL) <= 0;
 --- SUM(H::T) <= H + SUM(T);
This example is coded in a "real" functional language called "Hope+." I have
chosen this language because of its straightforward syntax. Hope+ programs are
easy to understand even if you are not yet comfortable with functional
programming (FP for short).
The first line DEClares the function SUM as a mapping from the set "list of
numbers" onto the set "numbers." The next two lines describe what SUM actually
does: If applied to an empty list, SUM returns zero. If SUM is instead applied
to a list consisting of a head H and a tail T, it sums up the tail and adds it
to the head element. For Prolog programmers, this way of thinking may be
familiar. Actually, there is a strong relationship between "pure" Prolog (that
is, no "cut," no side effects, no extra logic features) and Hope+, as you may
easily model one with the other.
Functional programs are appealing because they are free of side effects. They
are called "referential transparent," which means that two successive
invocations of a function with the same arguments always yield the same
results. In traditional languages such as C, this may or may not be the case.
Here is a C function that is not referential transparent:
 int foo(int n);
 { static int c=O;
 return n+getchar( )+(c++);
 }
You can argue that some programs are impossible to write without relying on
side effects. Interactive input/output comes to mind, or generation of random
numbers. As we will see shortly, this is not true: Every conventional program
can be written in functional style. But first, it is necessary to understand a
few little FP basics.


In the Beginning


The roots of FP can be traced back to the famous mathematician Alonzo Church,
who developed in 1941 a notation to reason about mathematical functions
without giving them explicit names, that is, anonymous functions. This
notation was called the "lambda calculus," and a function that returns the
square of its argument looks similar to this in Church's notation: lambdax.x*x
The argument x is listed to the left of the period. In order to calculate the
square of a particular number, say 5, you write (lambdax.x*x) 5.
It can be shown (but to do so would require a separate article the length of
this one) that every function and thus every computer program could be written
as a series of -- possibly nested -- lambda expressions. This is not obvious
at all; just think of how you would write a recursive function such as SUM as
a lambda expression. Note that you must not use names (for example, SUM) to
accomplish the recursive call, because functions in the lambda calculus are
unnamed. Besides, N.G. de Bruijn proved in 1972 that it is even possible to
dispense with the names of the variables, too!
The first application of Church's theory was incorporated into the programming
language Lisp, where the previously defined function to square its argument
could be written as (lambda X (* X X)). Lisp could be very close to functional
programming, but for efficiency reasons, Lisp designer McCarthy soon had to
abandon this concept in favor of a more traditional approach. For example:
(SETQ V 5) replaces the previous value of V with the new value 5. Destructive
assignments, however, are forbidden in any FP language.
It was not before the 1970s that researchers really began to investigate the
possibility of a pure functional language. In 1978, Fortran inventor John
Backus published his now historic paper, "Can Programming Be Liberated From
the Von-Neumann Style," where he described a new functional language called
"FP." Soon after, research activity took off.
Cheaper memory and faster CPUs made functional programming usable. Hope+ was
developed and used internally at universities for teaching purposes and for
application programming. The first full-featured and commercially available
compiler for a functional language was announced in 1985 at the International
Conference on Functional Programming and Computer Architecture (FPCA '85) at
Nancy, France, by researcher David Turner. The language implemented was called
"MIRANDA" and introduced a formalism called "currying" (named after the
theoretician Haskell B. Curry): A function of n arguments may be treated as a
concatenation of n single-argument functions. Thus, if PLUS expects two
arguments and returns their sum, the expression PLUS 1 denotes a new function
of one argument, which simply adds 1 to it.
Miranda and Hope+ are not only pure functional languages for use in research;
they are also intended to be used for application programming. As mentioned
earlier, there seems to be a conflict between referential transparency and
some real-world problems such as input/output. Of course, input from the
terminal could be regarded as a list of characters that can be passed at once
to a function for processing. Because output to the screen may be regarded as
a similar list, one may be tempted to define a program doing interactive I/O
in the following way: DEC DIALOGUE: LIST(CHAR)-> LIST(CHAR);
But this requires the program to anticipate the whole input at once and in
advance! In an interactive environment, however, most input will not be
available until at least part of the output has been written to the screen.
One solution to this problem is termed "lazy evaluation." During function
evaluation, the program consumes only as much from its (input) parameters as
is necessary for the communication with the terminal. Therefore, although
DIALOGUE conceptually accesses all of its input at the beginning and delivers
all of its output at the end of the computation, the code generated by the
compiler manages the interleaving of input and output. The programmer, of
course, need not worry.
More Details.
Look for instance at the program SILLY in Figure 1. It expects a list of
character digits and comments on each character received. If the character
received is indeed numeric (a digit), it says "That's fine." If it is not
numeric, it complains. SILLY may be connected to a terminal like this:
Figure 1: This program expects a list of character digits and comments on each
character received

 DEC silly: LIST(CHAR)->LIST(LIST(CHAR)):
 --silly(NIL)<=["Thank you for using <silly>"];
 --silly(onechar::remaining) <=
 (IF member(onechar, "0123456789") THEN "That's fine!\n"
 ELSE "Please type only digits!\n") :: silly(remaining);

 TYPEVAR any_type:
 DEC member: any_type #LIST (any_type)->TRUVAL;
 --member(_,NIL)<=FALSE;
 --member(a,a :: t)<=TRUE;
 --member(a,_ :: t)<=member(a,t);


 TOFILE(term_in,silly(FROMFILE (term_out));
term_in and term_out are not part of the language definition, but identifiers
such as these should be provided by the respective Hope+ installation. Due to
lazy evaluation, the user gets a response after each typed character.
Infinite data structures like the terminal input mentioned are called
"streams." Streams are not only useful for input/output. The following Hope+
function returns an infinite list of successive integers, starting with N: DEC
ILIST: NUM ->LIST(NUM); -- FROM(N) <= N :: FROM(N+1); This would be impossible
to accomplish without lazy evaluation, because a call to ILIST would lead to a
non-terminating loop. In Hope+, however, it is perfectly reasonable to work
with those functions and structures.


Using FP in Real Projects


So, what if you decide to use functional programming in real projects? There
are a lot of compilers for imperative languages such as Fortran, C, Pascal,
and even Smalltalk and Prolog, but if you go to a software store and ask for a
nifty little FP system for your work-station, you will be disappointed.
Indeed, the present situation is reminiscent of the time the first Fortran
compilers became available in the mid-1950s: Insiders knew that something
revolutionary was going on, but how were they to participate? The situation
with functional programming appears similar, but as it happens it could be
worse.
Presently, the most advanced products run under Unix. This comes as no
surprise because this operating system is prominent among universities. Hope+,
for instance, runs under Unix. This compiler is not freely available, however,
so you have to contact a university computer science department to get a copy.
For instance, Imperial College of London runs a copy of Hope+ in its
functional programming laboratory.
At present, only one language providing full laziness is commercially
available: Miranda, the programming language invented by David Turner. Turner,
who at the time worked at the University of Kent in England, founded his own
company, Research Software Ltd., in order to market his compiler, which is
currently available on Sun, Apollo, VAX (running Ultrix), and the
Hewlett-Packard 9000 series.
One of Miranda's attractive features is its provision for strong typing
without requiring the programmer to write any type declarations for variables.
The compiler simply deduces the type of a function from its applications, and
reports any inconsistencies it finds.
Data abstraction is realized by "constructor functions," which can be compared
to user-defined types in traditional languages. To define a type tree
representing a binary tree of numbers, you simply write: tree:: = niltree node
num tree tree.
In this equation, num is a predefined, primitive type, while the remaining
words are user defined and introduced to the system with this equation. A
function sumtree, which sums up all numbers stored in a tree, could be written
like this:
 sumtree niltree = 0
 sumtree (node number left right) = number + sumtree left + sumtree right
New trees are built by the application of the constructors. The expression:
node 5 (node 2 niltree niltree) niltree returns a tree whose root contains the
number 5, whose left branch contains 2, and whose right branch is empty.
As in Hope+, functions in Miranda can be polymorphic. This means that you need
not worry about a concrete type as long as some structural invariant is
observed. Figure 2 shows a polymorphic sort function that sorts anything as
long as it is represented as a tree and has an ordering relation defined on
it.
Figure 2: A polymorphic sort function that sorts anything as long as it is
represented as a tree and has an ordering relation defined on it

 MIRANDA example program
 defining a tree whose nodes can be any type
 the type is represented by the asterisk symbol tree*::=niltree
 node * (tree *) (tree *)

 Now implementing a sort algorithm
 To sort a tree, first flatten it to get a list of nodes
 Next build a sorted tree from the list sort = flatten buildsorted

 This defines how to flatten a tree
 flatten niltree = {} Flattening an empty tree gives the empty list
 To flatten any other tree, flatten the left subtree first,
 append the root, and append the flattened right subtree.
 flatten (node a left right) = flatten left ++ [a] ++ flatten right

 This defines how to build a sorted tree from an unsorted list
 Note: "<= and ">" must be defined on tree nodes!
 buildsorted = foldr insert niltree foldr is a predefined transformer!

 where
 insert a niltree = node a niltree niltree
 insert a (node b left right)
 = node b (insert a left) right, a<=b
 = node b left (insert a right), a>b

Miranda is unique in that it is already used for many industrial applications.
Among the steadily increasing community of Miranda users are Toshiba,
Signetics, Shell Netherlands, BP (UK), Olivetti, Logica Cambridge, and ICL.
The British company Logica Cambridge uses Miranda currently for the design of
Viper 2, a special-purpose microprocessor for real-time control.
For MS-DOS and OS/2, there are fewer possibilities for doing FP today. This is
understandable, because these operating systems are used extensively in the
scientific community. Also, due to memory limitations, porting a compiler such
as Miranda to MS-DOS is a non-trivial, if not impossible, task. The market for
OS/2, on the other hand, is not considered important enough yet to justify the
cost of a conversion.
If you are willing to sacrifice lazy evaluation, there are a some languages
available: Q'NIAL, for example, is an excellent implementation of the NIAL
(nested interactive array language) language, which not only incorporates an
equivalent of Backus's FP language as a subset (see Figure 3 for examples),
but also contains a lot of useful, imperative constructs. Because Q'NIAL is
not only offered for DOS, but also for OS/2, Unix, VMS, and several other
systems, it may be the language of choice for doing serious application
development.
Figure 3: Sample NIAL code

 # Defining the FACTORIAL function in NIAL
 # i.e., factorial 4 results in 16

 factorial IS FORK
 [1>=, 1 first,
 times [pass, factorial (1 CONVERSE minus)]
 ]


 # Defining the AVERAGE of a list of values
 # i.e., average 3 9 5 3 results in 4
 average IS/[sum,tally]

Some Lisp dialects also enforce, or at least enable, programming in a
functional style. PC-Scheme, an implementation of the Lisp-derivate Scheme by
hardware manufacturer Texas Instruments, is an example. The compiler is well
made, but the user interface lacks speed and sophistication. There is also a
functional Lisp derivate called "Le-Lisp," implemented at a French university
for Unix and DOS. A public domain version of Backus's FP was implemented in C
by Arch D. Robinson, Urbana, Illinois. It runs on Unix and MS-DOS, but may be
used only for educational purposes because it is so slow.


FPCA '89, the Haskell Language, And More


The papers presented at FPCA '89 covered a wide range of subjects, from purely
theoretical subjects to industrial applications. As with the introduction of
Miranda in 1985, this year's conference revealed a new and widely discussed
programming language called "Haskell," named after Haskell Curry (creator of
the "currying functions" of the Miranda language). So what's the advantage of
Haskell over Miranda?
While Miranda certainly is suitable for every kind of program development, it
does not particularly support large projects with tens of programmers writing
hundreds of modules concurrently. Haskell, on the other hand, supports modular
programming by requiring the definition of clean interfaces between modules
and of abstract data types. In some respects, Miranda is to Haskell as Pascal
is to Modula-2: The spirit is the same, but the latter is more advanced.
Second, and even more important: Haskell is free! This means that for a
nominal fee covering shipping and handling, anyone can get the original
software including the source code and port it to any operating system for
resale. Of course, there will be a version available ready to use that has to
be paid for.
In this manner, the Haskell research group, centered around Philip Wadler and
Simon Peyton-Jones of Glasgow University, hopes to spread the spirit of
functional programming around the world. Unlikely? Remember how Unix became
popular. Maybe it will work again. Figure 4 shows a small Haskell program that
types a file FOO to the console. Haskell is presently neither standardized nor
finished, so the actual syntax might vary slightly when the first compiler is
delivered.
Figure 4: A Haskell program that types a file FOO to the console

 main resps =
 [ ReadFile "FOO" Text,
 case resps of
 (Return Val:_)->
 AppendChan "stdout" Text Val]

Another interesting feature of Haskell is its relationship to object-oriented
programming (OOP). Haskell has objects, classes, multiple inheritance, and all
other OOP features except one, of course: Haskell objects don't have internal
"states," because this would contradict the referential transparency of FP.
Other papers covered the union of FP and OOP, and also of FP and logic
programming, the paradigm used in Prolog. The latter has been proved possible
by Erik Ruf, an ambitious young researcher from Stanford University. While
demonstrating his ability to speak close to the speed of light, he presented
an extension to the programming language Scheme. Scheme is a deviation of Lisp
that doesn't offer lazy evaluation, but enforces programming in FP style for
subprograms that don't rely on interactive input/output such as compilers.
Ruf's extension, Log-Scheme, simulates all constructs of Prolog, including
unification and extra logical features such as "The Cut."
Maybe the most interesting question from the conference is not, "Should I use
FP or OOP or logic programming in the future?" but "Why not use them all
together?"
A great deal of the presented papers focused on the difficulties experienced
during the implementation of FP compilers. Among the topics were the aggregate
assignment problem, strictness analysis, and abstract interpretation.
Adrienne Bloss from the Virginia Polytechnic Institute for example, worked on
the aggregate assignment problem. Aggregates (that is, arrays or records) are
necessary for storing huge amounts of data in an uniform way. Because
destructive assignment is forbidden in the functional programming paradigm,
how do I, then, "update" an array? In theory, the solution is easy: You just
have to define an "update function." The following example looks like Hope+
again, but because this language does not provide arrays at all (lists are
used instead), I simply invented them for the sake of clarity:
Thus, let ARRAY(T) be an (open ended) array of type T. The update function
must be declared like this: TYPEVAR T; DEC UPD: ARRAY(T) # NUM # T ->
ARRAY(T);
UPD(A, N, V) therefore returns a copy of the array A, except the Nth element
is replaced by V. Obviously this involves a lot of copying and memory
management at run time, especially when the arrays are large. In fact, this is
one of the main obstacles when writing efficient FP compilers. Adrienne Bloss
investigates the possibility of doing an in-place-update instead of copying,
for example, a destructive assignment of V to the Nth element of A. This is
not always safe. In the context LET A=some array IN TOFILE(term_out, UPD (A,
4,155)), the previous value of A need not be kept in memory, while a function
call like F(UPD(A, N1, V1), UPD-(A, N2, V2), G(A)), which involves some
auxiliary functions F and G, makes copying necessary. Of course, this decision
is made by the compiler. The programmer still thinks in terms of functions
free from side effects.
Another problem area in FP is strictness analysis. As mentioned earlier,
input,/output and infinite data structures are handled by lazy (delayed)
evaluation. This works fine in theory, but produces much overhead when applied
to every argument of every function in a functional program.
To optimize the code produced, the compiler should know when it can safely use
a traditional, non-lazy parameter passing mechanism. Such function parameters
are called "strict." of course, the programmer could provide the necessary
information. But this would introduce a new class of errors: Arguments
erroneously declared "strict" would cause the program to loop forever. On the
other hand, programmers themselves are "lazy" and tend to declare more
arguments lazy than necessary, just to avoid such errors. This would lead to
correct, but inefficient programs.
Enter strictness analysis. Here, the compiler tries to find out which function
arguments can safely be assumed to be strict by examining the source code.
This is not an easy task, because parameters may be passed through to other
functions, which in turn must be analyzed. Most problems regarding strictness
analysis are solved now, although not always in an optimal way: This kind of
optimization is still very time consuming.
A tutorial on the state-of-the-art of abstract interpretation was given by
John Hughes, also a co-author of the Haskell programming language. He defines
abstract interpretation as "a compile-time analysis technique to predict
information about a program's behavior from partial information about its
inputs." The idea behind this is simple: The more the compiler knows about the
program, the more efficient code may be produced. A typical example is the
SIGN function: SIGN(X) is defined to be -1 for negative X and +1 for positive
X. SIGN(O) equals zero.
Suppose a program contains the following function definition (the "sharp"
symbol, #, separates the function parameters):
 DEC F:NUM # NUM-> NUM; -- F(I,J)<=I*SIGN(I*J);
Now imagine that the compiler is able to prove that on every concrete
invocation of function F, both arguments supplied always have the same sign.
The compiler then can conclude that SIGN(I*J) always produces +1 and F
therefore reduces to:
 DEC F:NUM # NUM-> NUM;
 -- F(I,J)<=I*SIGN(I*J);
As a next step, the function F is obviously not necessary at all, because it
does not perform any computation. This means that no code for F is generated,
and that every application of, say, F(A,B) is simply replaced by its first
argument, A. The conference also attracted some researchers from other areas
who are normally not associated with functional programming in the scientific
community. Among the audience, there was for instance Professor Tim Teitelbaum
of Cornell University. Teitelbaum became quite famous due to his work on the
Cornell Program Synthesizer, an integrated program development system that
analyzes and compiles the program while it is being entered. What motivated
Professor Teitelbaum to make the trip to Europe and show interest in an
entirely different subject?
"The work with the synthesizer generator is motivated by an interest in
incremental computation," says Tim Teitelbaum, adding, "In the past, we have
used attribute grammars for this purpose. But attribute grammars are not the
only game in town. A lot of people consider functional programs a more natural
way for expressing incremental compilation." For the current version of his
program synthesizer, Teitelbaum and his students already use a special
technique derived from FP called "memorization," where function results are
automatically stored in a table after the first evaluation, and retrieved
quickly when the function happens to be called again with the same argument.


The Future of Functional Programming


Clearly, implementation problems are no longer stumbling blocks. Lazy
evaluation is a commonly-used technique, at least at universities, and FP
systems perform well enough to be used for practical purposes, fast
workstations and a few megabytes of memory provided. This contrasts with the
conference held at Nancy four years ago, where only two examples of very
special industrial applications were mentioned.
Despite its name, this year's FPCA covered many topics on functional
programming, but practically none on computer architecture. This is
surprising, because parallelism in hardware especially would gain a lot from
functional programming: Due to referential transparency, all arguments to a
function may be safely evaluated in parallel without worrying about possible
conflicts. Despite this, the resolution of many implementation problems has
obviously shifted towards a software instead of hardware solution.
The implications of this fact are not yet apparent. Perhaps most researchers
concentrate on sequential machines because those are available today and in
widespread use. The opinions are not undivided, however. Tim Teitelbaum, for
instance, assumes that, "The future will be dominated by parallel
architecture. The issue of how to program such machines is still very open. If
it turns out that the FP people really have the answer, then we will
necessarily switch over to functional programming in order to get the benefits
of the parallelism."
The next conference, FPCA '91, will be held in the United States. At that
time, the first version of Haskell should be up and running. Maybe the
revolution has already begun.


An Interview with Simon Peyton-Jones


DDJ: Simon, you promise that your new language called "Haskell" will be a
standard and will be freely available. In what way do you mean it's free and
everybody will have it? Will it be like SNOBOL4, so everybody can have the
source code for it?
P-J: Sure, the main aspect for it is that anybody can implement it and
distribute their implementations.
DDJ: Such as early Unix?
P-J: That's right. Anybody can use Haskell and distribute it.
DDJ: What do you think when it will be available?

P-J: Well, to the end of next year [i.e., 1990], we will have a first-time
beta version. Until the end of the following year, we expect to have a full
working version.
DDJ: What do you think will be the reaction from the industry? Will Haskell
become an established standard in 10 or 20 years?
P-J: Oh, we already have several industrial promises involving functional
projects, so the industry is showing some interest. One of them is British
Telecom. Another one has been ICL. They were recently involved in a large
European project to do with declarative [i.e., functional] programming.
DDJ: Do they actually use functional programming for applications work?
P-J: Mainly it's restricted at the moment to industrial research labs, because
available implementations just haven't been fast enough. Nowadays we are just
beginning to get to a state, where we've got implementations of functional
languages where you're only losing a small factor over writing in Fortran or
C, and the productivity benefits you pick up from writing in functional
languages then become sufficiently severe.
DDJ: Do you think that education will be an issue?
P-J: Well, I think education is an important thing. That's why Pascal became
so important, because it was taught in a lot of universities, so a whole
generation of students went out knowing Pascal. We have already started
teaching our students functional programming in their first year; we also
teach imperative languages later on as well. So, functional programming may
become more widespread now. -- R.F.



_FUNCTIONAL PROGRAMMING AND FPCA'89_
by Ronald Fischer

[FIGURE 1]


DEC silly: LIST(CHAR)->LIST(LIST(CHAR));
--- silly(NIL) <= ["Thank you for using <silly>"];
--- silly(onechar::remaining) <=
 (IF member(onechar,"0123456789") THEN "That's fine!\n"
 ELSE "Please type only digits!\n") :: silly(remaining);

TYPEVAR any_type;
DEC member: any_type # LIST(any_type) -> TRUVAL;
--- member(_,NIL) <= FALSE;
--- member(a,a::t) <= TRUE;
--- member(a,_::t) <= member(a,t);



[FIGURE 2]

 MIRANDA example program
 defining a tree whose nodes can be any type
 the type is represented by the asterisk symbol
tree * ::= niltree node * (tree *) (tree *)

 Now implementing a sort algorithm
 To sort a tree, first flatten it to get a list of nodes
 Next build a sorted tree from the list
sort = flatten . buildsorted

 This defines how to flatten a tree
flatten niltree = [] Flattening an empty tree gives the empty list
 To flatten any other tree, flatten the left subtree first,
 append the root, and append the flattened right subtree.
flatten (node a left right) = flatten left ++ [a] ++ flatten right

 This defines how to build a sorted tree from an unsorted list
 Note: "<=" and ">" must be defined on tree nodes!
buildsorted = foldr insert niltree foldr is a predefined transformer!
 where
 insert a niltree = node a niltree niltree
 insert a (node b left right)
 = node b (insert a left) right, a<=b
 = node b left (insert a right), a>b


[FIGURE 3]


# Defining the FACTORIAL function in NIAL
# i.e. factorial 4 results in 16

factorial IS FORK
 [ 1 >= , 1 first ,
 times [ pass , factorial (1 CONVERSE minus) ]
 ]

# Defining the AVERAGE of a list of values
# i.e. average 3 9 5 3 results in 4
average IS / [sum,tally]



[FIGURE 4]

main resps =
 [ ReadFile "FOO" Text,
 case resps of
 (Return Val: _) ->
 AppendChan "stdout" Text Val ]








































December, 1989
PROGRAMMING PARADIGMS


OOPSLA '89: Fourth Down And Goal to Go




Michael Swaine


As I walked through New Orleans's colorful French Quarter one hot night in
early October, a man caught my sleeve and offered to bet me five dollars that
he could guess where I got my shoes. I declined his tempting offer, assuming
that the answer was, in a shoe store. I was in town for the Fourth Annual
Conference on object-oriented systems, languages, and applications (OOPSLA
'89), and the next morning as I sat listening to a panel discussion, it struck
me that this was an object-oriented gag, deliberately confusing class with
instance. Maybe OOPS is taking over the world.


Talking About a Revolution


It was easy to believe that there was an object-oriented revolution underway
that day at the show. There was a lot of enthusiasm, and there were a lot of
people. Although claims of 2000 attendees at last year's OOPSLA may have been
exaggerated, this year there probably were about that many. The
preregistration and on-site registration attendee lists give 1626 names, by my
count. In any case, the figures don't reflect the mad rush when the exhibit
doors opened Tuesday morning. Although they gave different reasons, everyone I
asked said that this OOPSLA was the best they'd been to.
Some attendees had a greater presence than most. Apple announced its C++ and
talked around the edges of Apple Script, its planned user programming
language. Borland announced Turbo Pascal 5.5 with object-oriented support. But
there was more interest in what might be coming in the next few months from
Borland and from Microsoft, which was not present as an exhibitor, although a
number of Microsoft employees were there. The biggest presence at OOPSLA '89,
though, was definitely C++, already the most widely-used object-oriented
language and spreading like -- well, you can supply your own simile. Some
people think C++ is a disease that we will all soon be required to contract.
All the expected exhibitors were there: Digitalk, The Whitewater Group, and
Interactive Software Engineering in the booth right in front of the entrance.
But there were few announcements of importance; this was an ACM conference,
not Comdex, and the real action was in the presentations.
The tutorials began the conference, and to a certain extent, they reflect the
topics of greatest programmer interest or areas of greatest difficulty in OOP
today, because they cover the topics that teachers of OOP are finding a demand
for. Most of the tutorials were not specially created for the show but are
classes that people in the field have been putting on over the past year.
Jon Pugh of Carleton University set up the tutorials, which included
introductions to OOP concepts, MacApp, and C++, plus object-oriented issues in
databases and concurrency. There were also tutorials evaluating OOP
environments, including NextStep, C++, Smalltalk, MacApp, and a portable C++
class library for Unix called "ET++," along with more advanced tutorials on
object-oriented analysis and design, prototyping, and managing object-oriented
software projects. ET++ is likely to get a lot of attention, because there's
not much out there in the way of libraries for C++.
After two days of tutorials, the conference proper began. The emphasis in the
conference program shifted this year away from some peripheral issues such as
general software engineering, user interface design, and databases, and toward
more depth of coverage in theory, language design and implementation, and
concurrency. Conference Chair Kent Beck explained that the peripheral areas
were welcomed in the past because they would not have got a proper hearing in
any other venue until recently. Now, though, object-oriented work is invading
the general computer magazines and journals and other conferences, and OOPSLA
could get more focused.
The overall impression I got from the attendees, the exhibits, the tutorials,
and the conference program, was of a fringe thread in the process of being
pulled up into the general fabric of software research and development.
Except, of course, for the fact that there still does not seem to be a
universally accepted definition of object-oriented programming.


Views from the Navel Observatory


If you can believe Johnny Carson, some 20 percent of the people in this
country examine their belly buttons daily. A disproportionate number of these
navel observers were at the Hyatt Regency in New Orleans the first week in
October, asking themselves and each other what object-oriented programming is.
To be fair, I should add that just as many people were answering the question
as asking it; I just wish I could say that they were giving the same answer.
The answer involves some subset of these features, apparently: These data/
code hybrids called objects, object classes of which particular objects are
instances, an inheritance mechanism defined on these classes, and the ability
to define new object classes. At least one language has been called
object-oriented despite the lack of each of these features, although each has
been put forth as necessary by one authority or another, and all at OOPSLA
'89.
The keynote speech by Peter Wegner of Brown University was a proper question
raiser and territory mapper. Wegner wrote the first book on Ada and got
interested in object-oriented programming early on because of his perception
of Ada's deficiencies as a language for software engineering. In Wegner's
view, the goals of object-oriented programming are: Creating a technology of
off-the-shelf software components, developing distributed national and
international software libraries, and growing to a capital-intensive software
technology in which one can buy rather than build. He also sees, as a present
goal, the extending of OOP to encompass what he calls OOP in the large:
Object-oriented systems supporting concurrency and persistent objects for
multiple computers and multiple users, and distributed data and cooperative
computing.
Not all the presentations followed Wegner's map.
One session described SELF, a dynamically-typed object-oriented language. The
authors, Craig Chambers, David Ungar, and Elgin Lee, all of Stanford,
presented an object-oriented approach that substitutes prototypes for classes.
Although SELF has objects and inheritance, by Wegner's criteria it is not
object-oriented, because it has no classes. Objects are cloned from other
(prototype) objects, from which they inherit behavior directly. In another
session, Ungar predicted that prototype-based languages will ultimately
replace class-based ones.
Of more immediate interest than the implementation peculiarities of the SELF
language, though, is the fact that SELF runs twice as fast as the fastest
Smalltalk implementation, even though SELF is inherently less efficient than
Smalltalk. The authors have optimized SELF with techniques applicable (but not
yet applied) to any other object-oriented language, and have not had to
hardwire any user-level operations into the compiler, as some researchers have
done with Smalltalk implementations. They bluntly say "researchers seeking to
improve performance should improve their compilers instead of compromising
their languages."
That could be the answer to one of the other nagging issues in object-oriented
programming: Can an OOP system provide the strong typing needed for software
engineering and at the same time remain a good system for rapid prototyping?
Those of a software engineering bent and those who like OOPS for rapid
prototyping both like inheritance, but seem to use it for different purposes.
The differences revolve around type checking and the relationship between
types and classes. Panelists in one session argued that types and classes are
really orthogonal and should not be confused; and that programming
environments could be, but are not, constructed to support both production and
exploratory programming. But one dissenter, David Stempel of the University of
Massachusetts, said that the only way that one environment can support both
styles of programming is by really being two environments, with a switch to
turn off prototyping features when in production mode.
Several talks dealt with the teaching of OOP concepts. The simplest technique
presented was in some ways the most intriguing. Kent Beck of Apple and Ward
Cunningham of Wyatt Software Services presented an index-card-based approach
they have been using to teach OOP. The approach involves representing objects
with 3 x 5 index cards, and placing the cards in appropriate physical
relationship to one another. A card contains three kinds of information about
the object: Its class name, its responsibilities (problems it is to solve),
and its collaborators (what other classes it messages or is messaged by),
presumably written in pencil for easy modification. This simple technique
apparently underscores the objectness of the objects, and the authors report
students picking the cards up and waving them around to demonstrate their
interaction.


Getting Software Engineering on the Track


There's not even agreement that software engineering is the goal of OOP,
although many people not only believe it to be the goal but even speak as
though there were such agreement.
The early afternoon of the first day of the program was given over to software
engineering papers and a panel on the more-or-less SE-oriented topic of the
role of transactions in object-oriented systems. A couple of other panel
discussions directly addressed software engineering concerns as well,
including a discussion moderated by Brad Cox on the software industrial
revolution. Cox was of the opinion that the revolution will need more than
interchangeable parts -- it'll also need gauges or templates to ensure that
the parts fit. He stressed the importance of developing the gauges apart from
the parts they test - possibly even employing a different technology to
develop them.
There was one field report from pioneers trying to build a full software
development environment using object-oriented technology. William Harrison,
John Shilling, and Peter Sweeney described their system, which runs on a PS/2
under OS/2, an RT/PC under AIX, and even an AT under MS-DOS in 640K RAM. (Two
of them work for IBM.) The system has a persistent object store -- the
object-oriented version of a file system. With some 200,000 objects in their
store, the authors have some experience with the use of objects, and it is
suggestive that they found that organizing objects by type hierarchy did not
work well, because their actual use of objects did not reflect a type
hierarchy. They found some serious hurdles, too, and concluded that the
object-oriented paradigm needs some extensions if it is to be used for things
such as version control, and that the paradigm currently does not provide the
necessary support for modifying and extending the paradigm itself.
Lt. Col. John Morrison of the National Test Bed spoke briefly of one of the
more ambitious software projects to date: The Strategic Defense Initiative.
Whether one thinks SDI is a brilliant and achievable project or a fast one put
over on a gullible chief executive, SDI research should be a remarkable
testbed for software engineering and reusable components. One of the elements
is intended to be a large national library of reusable software components.


Real Applications


Others discussed how they were using object-oriented technology for real
applications in commercial software development, CAE/CAD, and scientific
computing.
Aldus personnel discussed how they are using object-oriented techniques to
manage software development in expanding teams of developers in the face of
competitive pressure to deliver. The Aldus VAMP system has helped them isolate
the developer from differences in Macintosh and Windows event-dispatching, for
example.
There were several presentations on business applications. Two that seemed
significant were an implementation of security in an object-oriented system at
MCC, and a language developed at GE Corporate R&D and Calma. The MCC system
implemented security levels from Unclassified through Confidential and Secret
to Top Secret, more than the average business requires, one would think, but
it was classified as a business presentation. The system from Calma and GE,
called DSM, is designed for commercial CAE/CAD applications. DSM supports most
of the features expected in an object-oriented language and is reported to be
nearly as efficient as writing straight C code. The only explanation offered
for this remarkable performance was that DSM trades memory for speed.
Papers on scientific computing came from three distinct object-oriented
backgrounds: CLOS (the Common Lisp Object System), Smalltalk, and C++. Besides
talks on systems for function minimization, automatic differentiation, and
linear algebra, there was an interesting discussion of scientific prototyping.
I guess what I found most interesting about it was the realization that the
ability to construct 3-D computer models of objects and processes and to
manipulate the models has become an essential tool of scientific research.
Sandra Walther and Richard Peskin presented a Smalltalk-based scientific
prototyping environment that uses C primitives for efficiency.
Is OOP seeing real use in embedded systems development? Design?
Implementation? And does it have a future there? The one case study of
embedded system development presented at the conference suggests what one
might expect: OOP was a good tool for managing the development process and
coming up with the basic design in a short time, but the actual implementation
required substantial recoding in C and assembler.



The State Space of the Art


Any description of the state-of-the-art in any developing field always sounds
wrong to some part of the audience. What looks close to SOA to one person may
be SF to another. To traditional DP professionals it may be a non-issue: Bill
Joy said he did not consider traditional DP professionals a market for OOP
because they would not understand it. Later, being fair, he added that he'd
say the same about most C programmers.
Bill Joy said that, not me. My only point is that the state-of-the-art has
thickness; it's not so much a time slice as a time loaf. At OOPSLA '89 there
were SOA talks on theory, and SOA talks at the implementation level.
A couple of the implementation issues on the edge are efficient garbage
collection and how to implement persistent objects and virtual memory for
objects. One of the concepts being usefully employed in garbage collection is
the notion of a generation: Several systems separate objects by age, focusing
garbage collection on the youngest generation, where turnover is likely to be
most rapid.
One of the more interesting theoretical papers showed that inheritance, rather
than being merely a feature of object-oriented programming, is a general
method that can be applied to any form of recursive definition.
Some presentations were state-of-the-art in any sense, focusing on concepts on
the edge, such concurrency, agents, and reflection.
Object-oriented programming decomposes problems in ways nicely suited to
communicating processes in a parallel architecture. The conference
demonstrated that there are real implementations of OOP systems that implement
fine-grained parallelism, although they probably won't run on your machine
yet.
Several talks dealt with agents that may turn out to be important for making
sharable, reusable software components work in practice in widely distributed
systems. Agents are independently executing entities sensitive to particular
stimuli. Such entities could be trained to search libraries for the software
components needed. In "Swaine's Flames" I speculate about the economic
consequences of such a system, but the need for some method of automating the
search for software components seems clear. If one of the goals of
object-oriented programming is reusability, and if objects are going to
multiply as fast as some of the presentations at the conference suggest,
automating the process of connecting the component with the person who needs
it will be essential.
Reflection is the ability of a system to represent and reason about itself. It
turns out to be relatively easy to implement some form of reflection in
Smalltalk and some other OOP languages. The benefits discussed at the
conference include modifying the behavior of the system, monitoring entities
of the system, and self reorganization (learning). Formerly viewed as an
academic topic, reflection is beginning to be considered for practical
implementations. One presentation discussed a way of implementing many of the
benefits of full reflection with no cost in efficiency.
Maybe reflective programs could debug themselves. That would be handy.
During one of the panel discussions, Bjarne Stroustrup, humble developer of an
arrogant language, told the audience that anyone claiming that object-oriented
programming will bring about bug-free software was probably spending his
evenings in the French Quarter guessing where people got their shoes. I didn't
notice if Lt. Col. Morrison laughed.















































December, 1989
C PROGRAMMING


Text Searching, C++ and OOPS, and ANSI Strings




Al Stevens


This month we are going to start a new "C Programming" column project. We are
going to examine some of the problems associated with the maintenance of large
text data bases. You might be familiar with the problem. You have a lot of
reference material and you do not always know how to find what you are looking
for. Sometimes the reference material is stored in text files. If you keep a
lot of word processing files at work, or if you download a lot of software
with .DOC or READ.ME files, your material is stored somewhere in ASCII text
and would be ready to be read if only you could find it. Our new project will
address the problems associated with locating the text files that have the
information we need.
Someday everyone will have Optical Character Recognition scanners, and we will
store all our books and magazines on disk. Maybe the magazines will begin to
sell their editorial content on diskettes or CD-ROM as well. When that
happens, then all our reference works will be ready for treatment by a
document processor similar to the one we will build here in this column.
There are a number of software packages that do what we will build here. Most
CD-ROM distributions of large text data bases include such a system. We will
build our own. That way each of us can customize it to our specific
requirements. This software will be generic standard C. If you want to
integrate it into a particular user platform, it should port with little
difficulty. Our project will build a system that we will call TEXTSRCH.
There are a number of disciplines that we must address in the development of
such a project. First, we must ask, given a large text data base and someone
to search it, how does the search proceed? What are the search criteria and
how will we express them to the software? These are the first requirements to
consider, because they will determine what techniques we use to store and find
things. Next, once we identify files of text that match our search criteria,
how must the system present them to us?


The TEXTSRCH Data Base


A document retrieval system such as TEXTSRCH works best in an environment
where the data files are static. There is a lot of text and it does not change
much. Users control changes with addendum or replacement documents at
scheduled intervals. Typical applications are engineering specifications,
maintenance manuals, and reference libraries.
To build the data base, you collect the documents in a controlled location,
perhaps its own subdirectory, and then you build an index. The index supports
searches. Some systems require you to manually examine every document and mark
all the phrases that will appear in the index. Others build the index by
automatically scanning the text and extracting words. We will use this second
approach.
You want to use this kind of system on a static data base because the indexing
operation can take a long time.


Retrievals


With the data base in place and the index built, you are ready to perform
retrievals. Let's first discuss how such searches will proceed. Someone will
want to find all references to a word or phrase in the data base. Perhaps you
will want every reference to the phrase, "object-oriented programming." You
must tell the system what you want, and it must tell you where you can find
those references. But suppose you are interested only in references to OOP
that are specifically about Smalltalk. Then you might want your search to
deliver only those files where both key phrases appear. Another search might
want to specifically exclude references to C++, and so you must be able to
tell the system about that requirement as well.


Queries


You can see where we are heading. It is apparent at the outset that we need
some form of structured query language with Boolean operators. We need to be
able to parse an expression such as this one:
 "object oriented" and "smalltalk" but not "C++"
Our expressions will have operators with precedence, so we will need to use
parentheses to override the default precedence assigned by the parser, making
an expression such as this one possible:
 "object oriented" and ("design" or "programming")


The TEXTSRCH Expression Analyzer


All of what just preceded leads us to our first installment. The first set of
functions that we will develop will parse such query expressions and prepare
them for a retrieval pass at the data base. Listing One, page 164, is
textsrch.h. This listing is one that will grow over the months. As we add
features, we will add macros, prototypes, and such to textsrch.h. This month
it contains what we need for expression analysis.
Listing Two, page 164, is express.c, the code that parses our expressions. It
begins with a modified BNF that describes the query language. As you can see,
it isn't very complex. A program that wants to parse an expression calls the
lexical_scan function in this module. The function returns a NULL if it finds
an error. If the expression is correct, the function returns a pointer to the
parsed expression, which is represented in the form of tokens suitable for
processing by an expression interpreter.
The lexical_scan function calls the expression function to check the
expression for syntax and to convert it into tokens. The syntax check uses a
recursive descent parser. That is, if the expression function finds a left
parentheses in the stream, it calls itself to validate the expression that is
in parentheses, and then, when it returns, it expects to find a right
parenthesis. If it finds the unary NOT operator, it calls itself to check the
expression that follows the operator. If it finds a word or phrase -- a phrase
in our context consists of a group of words surrounded by double quotes -- it
looks at the next element in the expression. If that element is the binary AND
or OR operator, the expression function calls itself to check the expression
to the right of the operator.
While validating the expression, the program converts it into tokens. Each
language element is a one-character token, and the tokens are written into an
array in the order in which they occur. When the token is a word or phrase,
which is assigned the OPERAND token, an associated string contains the value
of the word or phrase.
After the expression is found to be correct and the tokens are built, the
lexical_scan function calls the postfix function. Its purpose is to convert
the token stream from infix notation to postfix notation. An expression in
infix notation uses defined operator precedence with parentheses to override
the defaults. This is the notation we use when we compose the query. It is
also the notation used by the C language. The postfix notation (also called
Reverse Polish Notation) eliminates the parentheses and represents precedence
by the proximity of tokens to one another. Users of SNOBOL, Forth, and TI
scientific calculators will recognize the notation.
An infix expression places a binary operator between the operands as shown
here:
 this and that
The postfix version of this expression pushes the operands on a stack followed
by the operator as shown here:
 <and> that this
(In these examples, I've used the angle brackets to set operators apart from
the operands on the stack.)
A parenthesized infix expression such as this:
 (this and that) or what
is represented on the postfix stack this way:
 <or> what <and> that this

Evaluation of a postfix expression is a stack operation. The top element is
popped. If the element is an operand, we use the operand to derive the value
of the expression. If the element is a unary operator, the next element is
popped and evaluated, the unary operator is applied to it, and the result is
pushed. If the element is a binary operator, the next two operators are
popped, the binary operator is applied to them, and the result is pushed. Then
the procedure repeats until the element popped is a single result. Each pop is
a recursive repeat of the entire procedure. In our case the pushed results are
true or false values within the data base document depending on the search
results. Each time we pop a word or phrase, we see if it is in the data base
and, if so, in which files. The result of that search is pushed. We'll get
further into the details of expression evaluation next month.
Conversion from infix notation to postfix notation is a simple matter. The
procedure is described in detail in Fundamentals of Data Structures, Horowitz
and Sahni, 1976, Computer Science Press, on pages 91 - 97. We implement this
procedure in the functions postfix, isp, icp, and poststack.
Listing Three, page 167, is testexpr.c, a program to test our expression
analyzer and display the results. It is a throwaway program not to be used in
the project beyond its purpose as a test and demonstration tool. You type your
expression into the gets function. Use words and phrases with interspersed
operators. The valid operators are AND, OR, and NOT. The purpose of the query
is to express a pattern of words and phrases that do or do not appear in the
text data base. The TEXTSRCH program will find files that match the query
expression. Typical expressions are these:
 (fortran or cobol) and not pascal
 fortran or (cobol and not pascal)
The testexpr program will report a syntax error by displaying a pointer symbol
below the expression element where it found the error. If the expression is
correct, the program will display the postfix stack in this format:
Enter the search expression: fortran or (cobol and not pascal)
 Enter the search expression:
fortran or (cobol and not pascal)

 Token
------------------------------
 <or>
 <and>
 <not>
 pascal
 cobol
 fortran
------------------------------
 Enter the search expression:
Press the Enter key with no expression to terminate the program. Next month
we'll discuss how we will index the data base and how that index will deliver
our query results.


Book Report: Object-Oriented Program Design With Examples in C++


Are you looking for a stocking stuffer for yourself this season? There is a
new book called Object-Oriented Program Design With Examples in C++, by Mark
Mullins. The book is more about object-oriented programming and design than
about C++, but it uses C++ for examples and is an indispensable addition to
your library if you are trying to figure out OOP and C++.
Mullin takes a unique approach to these subjects. Instead of presenting just
another enhancement to the AT&T C++ reference documents and Stroustrup's book,
he teaches the basics of object-oriented design and uses C++ to demonstrate
the lessons. Follow this book from beginning to end, and you will have a
manageable grasp on object-oriented design and programming by the time you're
done.
Mullin's book also teaches valuable lessons about the general approach a
programmer uses in the design of software systems, object oriented or not, and
he does it with a writing style that informs and often entertains the reader.
Every now and then he slips in a dry joke that goes by almost unnoticed.
To show you how to design an object-oriented system, Mullin undertakes the
development of an integrated information management system for a fictional
company. This is the first extensive treatment of OOP I've seen that uses
real, believable examples of things that a programmer might encounter in the
real world. Most writers build contrived analogies that attempt to relate
abstract examples of toasters, fruit, or other silly non-automata to the
mysterious things that the object-oriented paradigm calls objects. Mullin uses
inventory, sales and personnel database records to demonstrate the development
of a class hierarchy to support his hypothetical design. These are things we
can all associate with.
Perhaps you've never worked on an inventory system, but you can readily see
what one might need to do. If there is a warehouse full of stuff and you need
to locate specific items, fill orders for items, and replace what you sell,
then you need an inventory system. I've worked on several. The smallest was
for a video tape store; the largest was for spare parts for the Space Shuttle.
Their respective functional requirements were similar. The inventory
management folks call it "material requirements planning." George Carlin calls
it "keeping track of your stuff." Mullin takes this mundane application, one
that everyone will recognize, and designs an object-oriented software system
to support it.
Mullin traces the design in the manner in which it might actually occur. The
false starts, the refinement of requirements, the incremental development of
an object-oriented data model, he presents them all in the sequence in which
you would encounter them during a real-design project. As he proceeds, Mullin
tells you how you should be thinking about the objects and their place in the
class hierarchy.
Mullin's design sometimes tends to build his data base by using views that
seem -- at least to me -- to be unnatural. Object-oriented design involves
building base classes and derived classes in a class hierarchy. Because every
entity in his data base has a name and address, Mullin starts out with a new
class named "Entity" that contains name and address members. Then he derives
everything else from that class. This is not the natural view of things. We
are not necessarily always subordinate to our common denominators. Because we
all have addresses does not mean that we are subordinate to those addresses. I
think we are seeing the tendency of the object-oriented paradigm to bend our
perspective in inappropriate directions. The solution to a problem should
resemble the problem, and names and addresses are components of their owners,
not the other way around. I would have created the Entity class with its name
and address members and then included an instance of that class in each of the
classes that require it. The functional result would be the same but we would
not be forced to perceive our data base from an illogical perspective just to
adhere to hierarchical dogma. But then, I do not know enough about
object-oriented design to write a book. Not yet, at least.
My only other criticism of this book is that Mullin and his editors do not
seem to know that the word "data" is a plural noun. But then, many of my
colleagues here at DDJ don't know that either, and I forget sometimes, too.
That criticism aside, this is a very good book. OOP and C++ have been begging
for literary treatments this good, and I'm happy to see it finally happening.


The ANSI Corner: Preprocessing and Strings


The ANSI standard C specification has introduced something the committee calls
"stringizing," not a real word, an abomination actually, but a new concept in
preprocessing, nonetheless. (If K&R can give us "initializer," also not a
word, I guess X3J11 similarly presumed that they could extend English while
they extended C.)
The trouble with the ANSI document is that it often fails to provide any basis
for a particular extension to classic C. In too many cases the committee
leaves it to us to figure out why they included a particular feature. They
will describe the feature and perhaps provide examples of how a conforming
compiler must behave, but they do not adequately provide the motive behind it
all. The rationale document that accompanies the specification is supposed to
fill these information gaps but too often it does not or tries and fails. Such
is the case with the preprocessor's # operator on #define macro parameter
substitutions, the so-called "stringizing" feature. They tried to explain it,
but I do not understand the explanation. If you do not already know how you
might use the # feature, you might not be able to guess its purpose from the
description in the specification even when you know what it does. What we can
do is look at what it does and see if this solution solves some problem we
might have. The # operator turns a parameter into a string. That's all it
does. Here is an example:
 #define str(x) # x
 printf(str(HELLO));
This macro expands the hello parameter into a string so that the printf call
looks like this after the substitution:
 printf("HELLO");
The preprocessor performs any other translations on the parameters before it
builds the string as shown here:
 #define HELLO goodbye
 #define str(x) # x
 printf(str(HELLO));
This sequence converts to this:
 printf("goodbye");
Why would you want to do any of that? If you know you want HELLO to be "HELLO"
or goodbye to be "goodbye," why not just code it that way in the first place?
Let's try to dream up a circumstance where this feature might have some use.
One possibility that comes to mind is in the realm of debugging. You can use
the # operator to create a TRACE macro that traces selected expressions in
your program on the console. You can turn the TRACE on and off with a global
compile-time variable. Here is the macro:
 #ifdef DEBUGGING
 #define TRACE(x)(printf("\n"# x),(x))
 #else
 #define TRACE(x) (x)
 #endif
Suppose your program has an expression that you want to trace on the console.
If this is the expression:
 b = strlen ("12345");

To trace this expression, you can recode it with the TRACE macro like this:
 b = TRACE(strlen ("12345"));
Now whenever the expression is executed with the DEBUGGING global variable
defined, the expression's code is displayed on the console, as well as being
evaluated because the statement is preprocessed to this sequence:
 b = (printf("\ n" "strlen (\ "12345 \")"),
 (strlen("12345")));
Notice that the two components of the macro expansion, the printf and the
strlen calls are comma separated. This guarantees that the strlen, which is
the rightmost expression, returns the value required when the expression is
evaluated. Notice also that the two are surrounded by parentheses. If they
were not, we would get the wrong value in the "b" integer because the
assignment operator has a higher precedence than the comma operator.
Next month the ANSI Corner will explore the ## token pasting operator. We
will, that is, if I can figure out a useful application for token pasting.

_C PROGRAMMING COLUMN_
by Al Stevens


[LISTING ONE]

/* ----------- textsrch.h ---------- */

#define MXTOKS 25 /* maximum number of tokens */

#define OK 0
#define ERROR !OK

struct postfix {
 char pfix; /* tokens in postfix notation */
 char *pfixop; /* operand strings */
};

extern struct postfix pftokens[];
extern int xp_offset;

/* --------- expression token values ---------- */
#define TERM 0
#define OPERAND 'O'
#define AND '&'
#define OR ''
#define OPEN '('
#define CLOSE ')'
#define NOT '!'
#define QUOTE '"'

/* ---------- textsrch prototypes ---------- */
struct postfix *lexical_scan(char *expr);








[LISTING TWO]

/* ---------- express.c ----------- */

#include <stdlib.h>
#include <string.h>
#include <ctype.h>
#include "textsrch.h"

/*

 * Parse a search expression into a valid postfix token stream.
 * The input expression has this form:
 * <expr> := <ident>
 * <ident> <op> <expr>
 * NOT <expr>
 * (<expr>)
 * <op> := AND
 * OR
 * <ident> := <character>
 * <character> <ident>
 * "<phrase>"
 * <phrase> := <ident>
 * <ident> <space> <phrase>
 */

#define iswhite(c) (c < 33 && c > 0)
#define isparen(c) (c == OPEN c == CLOSE)
#define isop(c) (c == AND c == OR)
#define ischaracter(c) (!isparen(c) && \
 !isop(c) && \
 c != TERM && \
 !iswhite(c) && \
 c != QUOTE)

/* ----------- prototypes --------------- */
static int getword(char *expr);
static int gettoken(char *expr);
static int expression(char *expr);
static void postfix(void);
static int isp(int tok);
static int icp(int tok);
static void poststack(void);

int xp_offset = 0; /* offset into the expression */
static char word[50]; /* word from the expression */

static char tokens[MXTOKS+1]; /* tokens in infix notation */
static char *operands[MXTOKS]; /* operand strings */
static int token_ptr = 0;

static char stack[MXTOKS]; /* stack for tokens */
static char *stopr[MXTOKS]; /* operand strings */
static int top = 0;

struct postfix pftokens[MXTOKS];
static int pf = 0;

/* ------ analyze the expression for valid syntax ----------*/
struct postfix *lexical_scan(char *expr)
{
 token_ptr = xp_offset = 0;
 if (expression(expr) == ERROR)
 return NULL;
 else if (gettoken(expr) != TERM)
 return NULL;
 postfix();
 return pftokens;
}


/* ---------- analyze an element of the expression ---------- */
static int expression(char *expr)
{
 int tok;

 tok = gettoken(expr);
 switch (tok) {
 case OPEN:
 if (expression(expr) == ERROR)
 return ERROR;
 if ((tok = gettoken(expr)) != CLOSE)
 return ERROR;
 break;
 case NOT:
 if (expression(expr) == ERROR)
 return ERROR;
 break;
 case OPERAND:
 break;
 default:
 return ERROR;
 }
 tok = gettoken(expr);
 switch (tok) {
 case TERM:
 return OK;
 case AND:
 case OR:
 return expression(expr);
 case CLOSE:
 --token_ptr;
 --xp_offset;
 return OK;
 default:
 break;
 }
 return ERROR;
}

/* ------- extract the next token from the expression ------- */
static int gettoken(char *expr)
{
 char tok;

 operands[token_ptr] = 0;
 if ((tok = getword(expr))== OPERAND) {
 operands[token_ptr] = malloc(strlen(word) + 1);
 strcpy(operands[token_ptr], word);
 }
 tokens[token_ptr++] = tok;
 return tok;
}

/* ------- extract a word, operator, parenthesis,
 or terminator from the expression ------------- */
static int getword(char *expr)
{
 int w = 0;


 /* ------- bypass white space -------- */
 while (iswhite(expr[xp_offset]))
 xp_offset++;
 switch (expr[xp_offset]) {
 case OPEN:
 case CLOSE:
 return expr[xp_offset++];
 case TERM:
 return TERM;
 case QUOTE:
 while (expr[++xp_offset] != QUOTE) {
 if (w == 50)
 return ERROR;
 word[w++] = tolower(expr[xp_offset]);
 }
 xp_offset++;
 word[w] = '\0';
 return OPERAND;
 default:
 while (ischaracter(expr[xp_offset])) {
 if (w == 50)
 return ERROR;
 word[w++] = tolower(expr[xp_offset]);
 xp_offset++;
 }
 word[w] = '\0';
 if (strcmp(word, "and") == 0)
 return AND;
 else if (strcmp(word, "or") == 0)
 return OR;
 else if (strcmp(word, "not") == 0)
 return NOT;
 return OPERAND;
 }
}

/* - convert the expression from infix to postfix notation - */
static void postfix(void)
{
 char tok = '*';

 top = token_ptr = pf = 0;
 stack[top] = '*';
 while (tok != TERM) {
 switch (tok = tokens[token_ptr]) {
 case OPERAND:
 pftokens[pf].pfix = tok;
 pftokens[pf].pfixop = operands[token_ptr];
 pf++;
 break;
 case NOT:
 case OPEN:
 case AND:
 case OR:
 while (isp(stack[top]) >= icp(tok))
 poststack();
 stack[++top] = tok;
 break;
 case CLOSE:

 while (stack[top] != OPEN)
 poststack();
 --top;
 break;
 case TERM:
 while (top)
 poststack();
 pftokens[pf++].pfix = tok;
 break;
 }
 token_ptr++;
 }
}

static int isp(int tok)
{
 return ((tok == OPEN) ? 0 :
 (tok == '*') ? -1 :
 (tok == NOT) ? 2 :
 1 );
}

static int icp(int tok)
{
 return ((tok == OPEN) ? 4 :
 (tok == NOT) ? 2 :
 1 );
}

/* --- remove a token from the stack, put it into postfix --- */
static void poststack(void)
{
 pftokens[pf].pfix = stack[top];
 pftokens[pf].pfixop = stopr[top];
 --top;
 pf++;
}






[LISTING THREE]

/* ----------- testexpr.c ------------- */

/*
 * A program to test the TEXTSRCH expression analyzer
 */

#include <stdio.h>
#include <process.h>
#include <string.h>
#include "textsrch.h"

static void disp_token(struct postfix *pf);

void main(void)

{
 char expr[80];

 do {
 /* ----- read the expression from the console ------ */
 printf("\nEnter the search expression:\n");
 gets(expr);
 if (*expr) {
 /* --- scan for errors and convert to postfix --- */
 if (lexical_scan(expr) == NULL) {
 while(xp_offset--)
 putchar(' ');
 putchar('^');
 printf("\nSyntax Error");
 exit(1);
 }

 /* ------ display the postfix tokens ------ */
 printf("\nToken");
 printf("\n--------------");
 disp_token(pftokens);
 printf("\n--------------");
 }
 } while (*expr);
}

static void disp_token(struct postfix *pf)
{
 if (pf->pfix != TERM) {
 disp_token(pf+1);
 printf("\n %s", pf->pfix == AND ? "<and>" :
 pf->pfix == OR ? "<or>" :
 pf->pfix == NOT ? "<not>" :
 pf->pfixop);
 }
}


























December, 1989
STRUCTURED PROGRAMMING


OOPs, I Stepped in Something GUI ...




Jeff Duntemann, K16RA


The scene: At an animal research laboratory, a beaming researcher directs
reporters' attention to a window in a room-sized box on the floor before them.
Inside the box, a bored-looking gorilla sits on the floor and picks fleas from
his tummy.
"We've created this experiment to measure the cleverness of gorillas," says
the researcher. "We've carefully designed this cage so that there are only
three ways to get out. We're monitoring the cage continuously to see how
quickly Grunion hits upon a way to get out, and which one he chooses."
Inside the cage, Grunion just sits there, scratching.
"He seems to be in no hurry to get started," observes one of the reporters.
"Gorillas can be stubborn," rejoins the researcher. "He'll get down to
business shortly."
But as minutes pass, Grunion shows absolutely no interest in finding his way
out of the box. The reporters begin to yawn. The researcher grows restive.
"Hmmm. He may not be feeling well today. I'd better go in and take a look at
him." The researcher unlocks the door in the side of the box and goes in.
Grunion immediately perks up, grabs the researcher, and throws him at the wall
so hard that the entire side of the box collapses outward. Grunion then lopes
out of the box and gives the reporters the High Five.
Moral: There is always another way out of the box. Plus: Even if you built the
box, you may not know the best way out. And finally: Never be too sure you're
smarter than the gorilla.


Forklift Toolkit


Way back in 1985, when I was a foot soldier at PC Tech Journal, each of us on
the editorial staff was given an early copy of Microsoft Windows. I loved it.
It was wonderful, the realization of Xerox's inaccessible PARC research for
the common man. Naturally, I immediately wanted to write applications that
used it. So I called Microsoft and asked for whatever it took to generate a
Windows app, affectionately known as a "winapp."
A couple of days later, the Federal Express man rolled this thing in on a
forklift and I was on my own.
Sheesh!
The Microsoft Windows Software Developers' Toolkit (SDK) is a massive
collection of facts and diverse utilities poured into a box with little
thought to organization or documentation clarity. It is the second most
daunting developers' tool I have ever confronted; the first (as I'll return to
a little later) is the SDK for OS/2 Presentation Manager.
The Windows SDK makes it pretty plain that you can write a winapp in one of
two languages: Microsoft C or MASM. Being neither a masochist nor suicidal, I
called Microsoft and complained. There were but two (brutally difficult) ways
out of the box, and I cared for neither. How about some bindings for Pascal or
Modula-2? And cripes, how about coming up with some reasonable documentation?
Their answer was and is most informative: The Windows SDK is intended for use
by experienced professional developers. In other words, if you have the head
muscle to design a major application that can compete on the open market,
you've got the head muscle it takes to untangle the unholy mess waiting inside
the SDK box. Programmers in kiddie languages need not apply. Ah, well. It was
a fond dream ... and like Grunion, I sat back in my cage and waited for
someone to provide another way out of the box.


The Nature of the Box


I was a while waiting. In the meantime, Microsoft was moaning about how nobody
was jumping on the Windows' bandwagon. Nobody was using it -- perhaps not
remarkably because there were no applications of consequence for it. This may
have seemed incomprehensible to Microsoft, but it was pretty plain to me: Most
professional developers were looking at the cost/ benefit equations presented
by the Windows SDK and deciding that the costs outweighed the benefits by
about fifty to one.
So I waited. While I was waiting, I thought a lot about how Windows
development could be done more easily. Consider the nature of the box: There
are about 520 different API calls for Microsoft Windows 2.0. This is cool;
Windows is a capable and complex gadget that does a lot of interesting and
difficult things. Furthermore, Windows is an event-driven system. At
unpredictable times, the Windows machinery alerts your winapp that significant
things have occurred: Mouse clicks, window overlaps, things like that. The
windows comprising your application must then react to those events somehow.
The main problem with the Windows SDK is that it makes little or no effort to
organize its API calls and other information in hierarchical fashion. It hides
nothing, in keeping with the C philosophy of giving the programmer control
over everything. However, the C folks never seem to catch on to the fact that
invisible does not mean inaccessible. The essence of structured programming,
according to old Nick Wirth, is the artful hiding of details: Hide the stuff
you don't need so that it doesn't overwhelm your perception and understanding
of the stuff you do need!
Most of the stuff in the SDK can be kept hidden and out of the way about 90
percent of the time. (Instead, it hangs in your face like cobwebs and makes
the whole process look murky and incomprehensible.) My suggestion has always
been that a carefully-crafted bindings library could provide a header file
full of useful defaults that don't need to be tweaked until necessity arises.
Very high-level procedures in the library would allow creation of windows and
dialog boxes with one call rather than twenty. A winapp should need to be no
larger than this:
PROGRAM Yo;

USES WinLib;

BEGIN
 InitWinApp;
 OpenWindow(WindowHandle, 10,10,40,5,'Window Title');
 WriteToWindow(WindowHandle, 'Yo, World!');
 WaitUntilWindowClose(Window-Handle);
END.

As part of the functioning of InitWinApp, the init code would fill all the
necessary descriptor records with usable default values. The programmer could
override those default values any time that the default values just wouldn't
serve. Alas, there is no Pascal library like this, so don't call me up at
three ayem asking me where to buy it.


The Way Out



Microsoft wasn't listening back then, and as best I know they're not listening
today. While waiting for someone to implement my obvious way out of the box, a
small company near Chicago came from nowhere and provided yet another way out,
a way that was years ahead of its time and flew right past nearly everybody in
the industry in 1986: objects. The company is The Whitewater Group, and the
product is Actor. If you have a hankering to do Windows, let me be pretty
blunt about it: Actor is the only way to develop applications for Microsoft
Windows that is worth the trouble it takes. Period.
Actor is a wholly synthetic language developed by Whitewater cofounder Charles
Duff, who earlier implemented the idiosyncratic Macintosh language called
Neon. Actor is not an extension of any earlier language, though it contains
elements of many. If you're familiar with either C or Pascal it won't seem
totally alien, and if you've learned Smalltalk as well, it'll seem downright
traditional by comparison.
Actor manages the complexity of the Windows API by modeling the elements of
the Windows GUI (graphics user interface) in object form. The Window class
models windows. The ScrollBar class models the scroll bar control that may
optionally be added to a window. Creating an instance of a Window class hides
the complexity of window creation behind a single easy-to-grasp metaphor: The
Window class acts as a "window factory" and whacks out a window object for you
to use. Very visual, very intuitive. All the ugly little details of setting up
a window are hidden behind that object metaphor.
Actually, the Window class is never really instantiated, because a
vanilla-flavored window isn't good for much. Window is a formal class in
Actor, which is the equivalent of the abstract class or abstract object type
in Quick and Turbo Pascal. Child classes of the Window class add additional
code to specialize the generic window: TextWindow can display (though not
edit) ordinary text. EditWindow is a child class of TextWindow that can edit
as well as display text, and so on.
Every important service Windows offers is modeled somewhere in the Actor class
hierarchy. The hierarchy manages the Windows API complexity in two dimensions:
First, Actor distributes the specification of Windows functionality up and
down the class hierarchy by degree of specificity. The most general concepts
(windows, controls, and so on) float to the top and the most specific concepts
(file window, button, polygon, and so on) move to the bottom. Second, Actor
hides the implementation of Windows functionality (that is, the gritty details
of calling the API and handling events) behind the graspable masks of the
individual classes.
This dual-mode management of complexity (idea complexity and implementation
complexity, both at once) is perhaps the greatest gift object-oriented
techniques offer the programmer. Actor uses it to the fullest, both to manage
the complexity of the Windows API and also Actor's own very rich class
library.


The Importance of a Class Library


The notion of a rich class library is extremely important, and I haven't
really taken it up so far in my discussion of OOP concepts. If you've seen
Whitewater's excellent ad describing how to write a text editor in two code
statements, you'll begin to get a feeling for the complexity-hiding leverage
Actor affords. It really is that easy to create a text editor in Actor:
 MyEditor:= New(EditWindow, ThePort,"editmenu", "Editor",nil);
 Show(MyEditor,1);
In the first statement the object MyEditor is instantiated from the EditWindow
class, by sending the New message to the class. In the second statement, the
new window object MyEditor is made visible by sending it the Show message.
(I'll get back to this notion of message passing again shortly.)
This is not just a matter of making Windows API calls. Actor actually comes
with a fully functional text editor object in its class library. This is the
rough equivalent of something such as the Turbo Pascal Editor Toolbox with all
the loose ends tied up and tucked in. Nothing ties up loose ends such as
encapsulation. The cleanness of interfacing to an object that may represent
tens of thousands of lines of code is one reason Actor can offer things like a
text editor class, and is a major reason earlier languages had a tough time
with toolbox products.
Actor offers lots of additional library classes, some with considerable power
and many with no analog in the purely structured world. Under the formal class
Collections (which is Actor's name for what we would simply call data
structures) are a number of interesting critters. One is Bag, which
metaphorically is just what its name implies: A container in which you can
toss any objects you like. The class keeps track of what's in the bag, and how
many are there.
Bag bears some relation to Pascal's set type. Sets in Pascal only deal in
values, not variables, however; and a value is either in a set or not in the
set -- Bag can tell you how many objects of a given class are present.
The TextCollection class resembles the melding of a Pascal text file held
completely in memory with a simple line-oriented editor such as EDLIN.
TextCollection data is some number of string values, and the class has a suite
of methods for inserting, deleting, and replacing lines. Actor uses
TextCollection as the mechanism for communicating with the Windows clipboard
resource. Text from a TextCollection may be written to or read from the
clipboard through methods defined for the purpose. You might think of
TextCollection objects as sheets torn off the Windows clipboard -- a handy
metaphor for what is a very handy thing to have in a library.
Another important collection class is the Stream class, which resembles an
array of objects, where the objects can be of any class at all. Streams can be
handy for things like lexical analysis when held entirely in memory, but are
probably most useful for writing objects to disk files. Turbo Pascal 5.5 has a
similar Stream type, as does C++.
It's possible (if not easy) to provide libraries of routines defining text
editors, windows, and buttons for ordinary structured languages. It takes the
power of late binding and polymorphism, however, to implement things such as
bags and streams in which the elements of the data structure can be any type
at all.
This is one of the numerous reasons that rich runtime libraries really come
into their own in an object setting. Not only does encapsulation make
libraries clean and easy to understand and use, but polymorphism allows a
richness of data expression that older purely structured languages simply
cannot match. Just try to code up a bag in preobject C or Pascal. Generic
pointers might give you a fighting chance, but the resulting Hackenstein's
monster would be an ugly thing indeed.


Polymorphism via Messages


By now, a good many people have sampled OOP concepts in the forms of Quick
Pascal and Turbo Pascal. People evaluating Actor will more and more be doing
so not from a platform of total ignorance but simply from different OOP
implementations. Actor has all the object-oriented features that C++ and
Object Pascal have, but as with Smalltalk (as I described a few columns ago)
the jargon can be confusingly different.
Polymorphism is a case in point.
In C++ and Turbo Pascal 5.5, polymorphism is an option rather than a way of
life. Your methods can be virtual (meaning that they support polymorphism) if
you choose, but they default to being early-bound (static), just as ordinary
procedure calls are. (Quick Pascal, on the other hand, makes all methods
virtual, period.)
In Actor and Smalltalk, everything is an object and all method calls are late
bound. Both languages use the same jargon in describing polymorphic method
calls: Instead of calling a method directly (which would imply early binding)
you call an object's method by sending that object a message indicating which
method you wish to call. The object then uses the message to select which
method code is actually invoked.
If you learned polymorphism in a Pascal or C++ context, this probably sounds
pretty weird. The mechanism is called message passing, and it deserves a
little elaboration.
Think of it this (rather literal) way: Imagine that you, as a farmer, find a
note in your mailbox one morning from the county agent. The note simply says:
KILLER FROST TONIGHT. HARVEST YOUR CROPS NOW!
Up and down the county roads, other farmers find the same message in the
mailbox. What does each one do?
It depends on the nature of the crops each farmer grows.
If you're a cucumber farmer, you crank up your cucumber-picking machine and
head off down the rows in your fields. If you own a cherry orchard, you haul
out the tall ladders, call out the kids and your hired hands, and start in on
the trees. The act of picking cucumbers is nothing like the act of picking
cherries, but both actions amount to a reasonable response to the directive
HARVEST YOUR CROPS.
The "many shapes" of crop harvesting are a form of polymorphism. The county
agent simply tells the farmers in his district what to do -- not how to do it.
The farmers each know how to harvest their own crops. And they do it, each in
an appropriate way.
Back in Actor, consider a collection (a bag, say) of various objects. (Keep in
mind that everything in Actor is an object, right down to characters, strings,
and numeric digits.) You want to print out the value of every object in the
bag. The way to do this is to send the Print message to every object in the
bag. Each object then reacts to the Print message by selecting the correct
Print method to invoke. A string object will print using one method, but a
numeric value object will need to use a different method -- one that first
converts the numeric value to a string and then displays the string to the
screen.
If you think about it, this is really no different than the situation in
Object Pascal. Remember, that when virtual methods are inherited down an
object hierarchy, the name of the virtual method is always the same, even when
the implementation of the method is different. In a sense, when you make a
virtual method call you pass the name of the method to the object in question,
which then looks the name up in a table to determine which implementation of
that named method to invoke. The name of the virtual method is thus a message:
PRINT YOURSELF. The receiver of the message must decide what code will in fact
do the job and satisfy the directive.
So beneath it all, message-passing and virtual method calls are functionally
the same thing. Don't let the jargon get you down!


Familiar Bones


Although, Actor uses some of the same jargon as Smalltalk, it's an easier
language to learn because the bones of language are implemented in more
familiar form. Passing a message to an object looks a heckuva lot like a
Pascal procedure call:
 MyStack:= New(OrderedCollection,64);
 Add(MyStack,17);
 Add(MyStack,42);
Here, we create an instance of the OrderedCollection class (think of it as an
array with a built-in stack pointer) and call it MyStack. Notice that the
assignment operator and syntax is identical to Pascal's. Two Add messages
amount to pushing two values onto MyStack. You could implement a stack type in
Pascal and create an Add procedure that would have the same sort of interface.
In a fashion similar (if not identical) to C, an Actor method is defined by
creating a header and then enclosing some number of statements between curly
brackets:
 Def MyMethod(self,Foo,Bar)

 {
 DoSomething;
 DoSomethingElse;
 AndSoOn;
 }

Actor's control structures, while using slightly different syntax, incorporate
most of the familiar Pascal keywords and are easy to recognize and understand.
Conditional statements are pretty plain:
 if (i < 10)
 then DoSomething;
 else DoSomethingElse;
 endif;
The looping construct can be set up in a number of different ways, but it
reads well in any configuration. This one is set up as a while/do loop:
 loop
 while x < 42
 x:= x + 1;
 print(x);
 endloop;
Actor's designers have taken considerable pains to keep "weirdness" out of the
language; the only weirdness is the inescapable weirdness of object-oriented
thinking, which like kudzu is destined over time to replace whatever else is
out there.


Actor on the Scales


Given the enormity of the things it does (such as taming the Windows API) I
found Actor almost ridiculously easy to learn. The trick, again, is not in the
language but in the paradigm. Once you learn to think object-ively, you
shouldn't have any more trouble than I had.
The documentation is solid, if not exhaustive. The single fat (700 page)
volume hasn't failed to provide me what I've needed to know so far, which is
my early measure of language documentation. I haven't yet tried to do anything
really tricky, such as using the early-binding option or create a stand-alone
application, but I hope to report on both in future columns, and we'll see how
well the doc stands up in a pinch.
On the low end, the manual is a so-so introduction to OOP thinking. I'll
forgive them that, and in fact I prefer that they'd give the language its due
in the manual and let us joinalists anoint the great unwashed in
object-oriented First Principles. If you've already made the conceptual
breakthrough to objects, on the other hand, you'll have no problem at all
picking up Actor itself.
There is also the book by Marty Franz, Object-Oriented Programming with Actor,
that takes a slower, more gentle path to OOP. Marty has a clean, plain style
and goes into the mindset of Actor programming in a way that the manual itself
doesn't have room for. If you're going to go the $500 to acquire Actor, it's
well worth shelling out another $25 for the book.
Which brings us to the question of Actor's success in the marketplace. Pretty
obviously, Actor has hitched its wagon to Windows, and won't do any better
than Windows does. Windows, however, is alluvasudden on a roll, fueled largely
by the availability of $2000-386 machines that make Windows not only fast
enough but almost sprightly. The Windows installed base has probably achieved
a sort of critical mass, with even Borland announcing that they will pursue
Windows development in their languages.
The way I see it, Actor has only one serious impediment to its success: Its
$500 price tag. Compared to the Windows SDK it's cheap enough, but an impulse
buy it is not. If a Tiny Actor (Child Actor?) were available for the canonical
$99, there'd be no stopping it. Microsoft has done well with its Big C/Little
C approach. Whitewater should pay heed. Actor is the way to break out of the
Windows software development box, but for Grunion the Gorilla to knock out a
wall with it, Actor first has to get in the door somehow.


OS/2, Mon Dieu!


And then there is Presentation Manager. PM is to OS/2 as Windows is to DOS:
The anointed user interface and graphics toolkit. Like Windows, it has an SDK,
which is larger, richer, and perhaps even a little better organized.
Unfortunately, because PM does lots more than Windows, the end result is the
same: You could spend years getting good enough at the SDK to write a
competitive product with it. (Not to mention the endless irritation of having
to write in C.)
Ever since PM was announced, I've been thinking, "Somebody should do an Actor
for it." The whole nature of PM, like that of Windows, cries out for an
object-oriented development language. Both of the current books I've obtained
on Presentation Manager programming describe PM internals in object-oriented
terms, even though the example code is in plain old-fashioned C. If you intend
to work in PM I'd suggest you get them both: Alan Southerton's Programmer's
Guide to Presentation Manager is nicely done, if a little thin on figures and
example code for my tastes. Even so, it runs to 750 pages -- there is a lot to
cover.
The book to have, though, is the one bundled in the Microsoft PM SDK:
Programming the OS/2 Presentation Manager by Charles Petzold. (Details on both
books are given at the end of the column -- but remember not to buy Petzold's
book if you're also going to buy the SDK!) Lord knows what Microsoft would
have done without a writer like Charles Petzold to make sense of it all. In
850 pages he makes PM development about as rational as it's ever going to be.
Lots of code, lots of figures, and beautifully written. To get a handle on PM
internals, start here.


Presenting Smalltalk


I've encouraged Whitewater to do an Actor for it, and they well might -- or
let us hope. In the meantime, however, Digitalk has gone and done the
remarkable, which is to produce a Smalltalk for PM -- one that adheres totally
to the PM user interface spec.
This is remarkable because Smalltalk, the language, was defined with its own
user interface, and up until now Digitalk has been relatively faithful to that
interface, perhaps for fear of being accused of "tinkering with the standard."
Their latest move is the right one. Smalltalk has its own user interface spec
because when it was designed (mid-seventies) Batch was King, and user
interfaces didn't exist. For the same reason that UCSD Pascal was designed
with its own operating system, Smalltalk-80 had its own user interface. But
operating systems have become standardized with user interfaces right behind.
I've had very little time to experiment with Smalltalk/V PM, mostly because
getting OS/2 to run on a Frankenclone is ticklish business. (The product
itself, even in its unfinished state, is pretty robust.) At this writing the
product is still in beta test form. I'll have a lot more to say about it when
I receive the shrinkwrapped product.
For now, here are some of the important points:
Where Smalltalk's original spec and PM conflict, PM wins. For example,
Smalltalk's original Bitblt class and its children have been replaced by the
new GraphicsTool class and its children, which correspond more closely to PM's
vector-oriented graphics.
PM's coordinate system places the default origin (0,0) in the lower-left
corner of a window's client area. Smalltalk uses the upper-left corner for the
origin, and some rewriting of existing Smalltalk code to "flop" the grid is
almost assured.
The Digitalk interactive environment now generates stand-alone .EXE files that
execute without the presence of any runtime system. This means simpler
logistics for distributing applications written in Smalltalk/V PM.
Undoubtedly, there is some internal interpretation going on (Smalltalk is at
heart an interpreter) but the sealed-off application is as independent as a C
module and all the apps I've seen have run unapologetically fast.
Yes, it's a dialect, but so was Turbo Pascal -- which has weathered the rain
of brickbats from sourpuss academicians. Digitalk has done the right thing,
both from a Smalltalk perspective and a Presentation Manager perspective. C is
not an appropriate language for use in developing for a GUI. (C++ is, of
course, but that's another person's column.) Smalltalk, as it was, seemed
chained to the late seventies, trapped in a set of assumptions that simply
don't apply to our modern machines and operating environments. In a future
column I'll use Smalltalk/V PM to show you just how utterly simple it can be
to develop for Presentation Manager. Thanks, Digitalk, for a bold step.
And welcome to the real world, Smalltalk.


PRODUCTS MENTIONED


 Actor V 1.2
 The Whitewater Group
 906 University Place
 Evanston, IL 60201
 312-491-2370
 $495.00

 Object-Oriented Programming with Actor
 by Marty Franz

 Scott, Foresman & Company, 1989
 ISBN 0-673-38641-4
 $24.95

 Programmer's Guide to Presentation Manager
 by Alan Southerton
 Addison-Wesley, 1989
 ISBN 0-201-19440-6
 $26.95

 Programming the OS/2 Presentation Manager
 by Charles Petzold
 Microsoft Press, 1989
 ISBN 1-55615-170-5
 $29.95
 (Bundled with Microsoft's OS/2 PM Software Developer's Toolkit)

 Smalltalk/V PM
 Digitalk Inc.
 9841 Airport Blvd.
 Los Angeles, CA 90045
 213-645-1082
 $499.95







































December, 1989
OF INTEREST





Apple Computer announced the Macintosh Programmer's Workshop (MPW) C++ at
OOPSLA '89. MPW C++ fully supports the AT&T 2.0 specification including
multiple inheritance, operator overloading, and protected members. Support of
the Macintosh Toolbox and the operating system as well as Object Pascal-based
methods found in MacApp are also provided. MPW C++ includes the MPW C++
translator, C++ interfaces, and libraries for complex math and stream
processing, an unmangler for CFront messages, and a collection of example
programs. In addition, Apple's Symbolic Application Debugging Environment
(SADE) can be used to debug programs at the C++ source level. A beta version
of MPW C++ (v 3.1B1 at the time of this writing) is available through the
Apple Programmers and Developers Association (APDA) for $175. MPW C++ also
requires MPW C, Version 3.0, or higher and the Macintosh System Software 6.0.2
or later. Reader service no. 20.
Apple Computer
20525 Mariani Avenue
Cupertino, CA 95014

Apple Programmers and Developers Association
(APDA) 800-282-2732
408-562-3910
ObjectVision Inc. of Berkeley Calif. has just released its visual
object-oriented programming environment of the same name. ObjectVision allows
the programmer to both organize object hierarchies and design objects visually
on the screen. Once an object is created, the programmer can implement its
methods through a pop-up editor by using an English-like syntax. Once the
design is completed, ObjectVision generates either C++ or Turbo Pascal 5.5
source code. A "hide" function encapsulates the details of a newly designed
object, keeping the screen free of clutter. In addition, a built-in browser
allows the designer to easily traverse the object hierarchy. ObjectVision also
provides a pixel editor for the creation of icons, the ability to import dBase
data into specialized database objects, built-in drawing functions, and a
rather slick UI that includes mouse support. Support for multiple inheritance
is not included in the current version, although such support is being
considered for future versions. System requirements for ObjectVision are
minimal -- IBM PC or compatible with at least 256K, either a VGA or EGA and a
mouse. ObjectVision sells for $399. A demo disk is also available at $30.
Reader service no. 21.
ObjectVision, Inc.
2124 Kittredge Street, Ste. 118
Berkeley, CA 94704
415-540-4889
To follow-up on last month's "Of Interest," Digitalk was demonstrating the
recently announced Smalltalk/V PM at OOPSLA '89. Smalltalk/V PM is an
object-oriented development environment for Presentation Manager under OS/2
and allows developers to take advantage of PM's capabilities while hiding its
complexities. The product is positioned as a prototyping tool for UI-intensive
applications, even though it is the first fully compiled Smalltalk under OS/2.
Developers can pass data to and from PM tools, as well as applications,
through Dynamic Link Libraries (DLLs) and Dynamic Data Exchange (DDE).
Digitalk also provides tools to organize and browse source code including PM
code. A source-level debugger provides most of the features you'd expect
including breakpoints, single-stepping, and an object inspector. The object
inspector additionally allows the programmer to change instance variables
during the debugging session.
Smalltalk/V PM comes complete with 150+ classes and is source code compatible
with Smalltalk/V 286 and Smalltalk/V Mac. The package requires OS/2
Presentation Manager 1.1 or later and is priced at $499.95. Reader service no.
22.
Digitalk, Inc.
9841 Airport Blvd.
Los Angeles, CA 90045
213-645-1082
800-922-8255
Stepstone's new release of their Objective-C compiler marks the fourth
generation of the product, which runs under DOS. Stepstone also offers
versions under other platforms including OS/2, Sun 3, 4, and Sun 386i under
SunOS, and the Apollo workstation.
Objective-C more closely follows the message sending paradigm found in pure
object-oriented languages such as Smalltalk than does C++. The strength of
Objective-C lies in its strong type checking. The compiler extends ANSI C
features such as function prototyping with method prototyping and type
mismatch detection for objects. Debugging tools include a message trace
facility that allows the programmer to view the message's receiver, the name
of the message, arguments to the message, and the sender of the message.
Another tool, back tracing, allows the programmer to back trace through the
current stack after an exception has occurred. Also available from Stepstone
is an object-oriented user interface toolkit, ICpak2O1, for workstations
running under X-Windows. The toolkit provides 58 classes with over 1100
methods to support pop-ups, scroll bars, dialog boxes, and the like. Other
classes support the UI by managing the environment, providing controllers,
fixtures, and other abstract classes. A set of container classes is also
included. Reader service no. 23.
Stepstone Corporation
75 Glen Road
Sandy Hook, CT 06482
203-426-1875
A new book from MIS Press, Object-Oriented Environment in C++, by David Hu,
presents concepts and code to create graphical environments such as those
found in Smalltalk or MacApp using C++. Hu covers Zortech, Guidelines, and
Advantage C++ (from Lifeboat) as well as Stepstone's Objective-C and
Smalltalk/V from Digitalk. The strengths and weaknesses of the various
languages are compared so that the reader can make informed decisions as to
which tool is best suited to a given project.
Hu also shows how to create windows, bit-mapped icons and menus, and how to
support the mouse using Zortech's C++. In addition, he presents schemes for
knowledge representation, and shows how to create an expert system including
methods for defining forward- and backward-chaining inference engines. In a
final chapter, the author presents object-oriented database concepts and
provides insights into the Gemstone Database system from Servio Logic. The
book is priced at 29.95, or with a source code disk at $59.95. Reader service
no. 24.
Management Information Source, Inc.
P.O. Box 5277
Portland, OR 97208
503-282-5215
Quintus Computer Systems and Logic Programming Associates have teamed up to
announce MacObject, a graphical package to generate object hierarchies in
Prolog++. Prolog++ is an extended version of Quintus's MacProlog that supports
meta-objects, super- and sub-objects (both static and dynamic). Prolog++ also
supports daemons, inheritance, message passing, broadcasting, methods, and
functions. In conjunction with MacObject is the announcement of version 3.0 of
the MacProlog compiler. MacProlog provides a complete development environment
that includes incremental compilation, multiple scrolling windows,
hierarchical menus, and pop-ups, compiled graphics windows and pictures, a
program optimization, access to Quick-Draw, and interfaces to C and Pascal
code. MacProlog 3.0 sells for $595. MacObject, which includes the Prolog++
compiler, is priced at $495.
Logic Programming Associates Ltd.
Studio 4, RVPB, Trinity Road
London, SW18 3SX
441-871-2016

Quintus Computer Systems
1310 Villa Street
Mountain View, CA 94041
415-965-7700
Franz Inc. recently announced Allegro CLiP; a Common Lisp designed for
parallel environments. Allegro CLiP, which is modeled after SPUR Lisp,
provides a set of low-level, parallel-programming primitives, while providing
high-level constructs from MultiLisp and QLisp. Access to the Sequent Parallel
Programming Library is also provided through a set of interface functions.
Allegro CLiP provides a complete integrated development environment that
features debugging facilities, an instrumentation style interface, and the
multiple entry-levels mentioned above. Reader service no. 26.
Franz, Inc.
1995 University Avenue
Berkeley, CA 94704
415-548-8253
Addison-Wesley has published The Renderman Companion, A Programmer's Guide to
Realistic Computer Graphics by Steve Upstill of Pixar (Steve wrote the article
"Photorealism in Computer Graphics," DDJ November 1988, on the Renderman
Shading Language). The book is a tutorial based on Pixar's Renderman Interface
for 3-D scene description, which can produce computer graphics that are
indistinguishable from actual photographs. Steve said he wrote The Renderman
Companion "to demonstrate that any programmer with some 3-D computer graphics
experience can produce stunning pictures with the use of our Renderman
Interface."
Some of the topics covered in the book are quadric surfaces, polygons,
parametric surfaces, hierarchical modeling, the digital camera, lighting and
shading, surface mapping, and the Renderman Shading Language. The paperback
sells for $26.95, ISBN 0-201-50868-0. Reader service no. 27.

Addison-Wesley Publishing Company
Reading, MA 01867
617-944-3700
Blaise Computing Inc. has announced C TOOLS PLUS/6.0, the latest version of
Blaise's library products for Microsoft C. This library of compiled C
functions supposedly gives programmers advanced routines for developing
high-powered C applications, and includes virtual, stackable menus and windows
with full mouse support and optional "drop shadows." It also has multiple
virtual pop-up help screens; a miniature multiline editor for gathering user
responses; a single function call that can move, resize, and promote a window
or menu on top of all others; the ability to update covered windows
automatically when they are written to; support for EGA, VGA. and MCGA text
modes including 30-, 43-, and 50-line modes; and support for the enhanced
(101/102 key) keyboard.
The library comes with adaptable source code for study or emulation. Blaise
claims that their attention to detail, like the use of function prototyping
and the const modifier, the pre-built libraries for the four standard memory
models, the organized header files, and an indexed manual makes C TOOLS
PLUS/6.0 appropriate for both experienced software developers and newcomers to
C.
The library also supports mouse detection, cursor control, and button presses
and releases, with mouse support integrated into the windows and menus.
Functions written in C can be installed to be called whenever certain classes
of mouse events occur; even DOS functions can be accessed in response to mouse
events.
Fully documented source code is included with the product, and the manual
gives a general overview for every function category and descriptions of each
function. C TOOLS PLUS/6.0 can be used for product development without
obligation to Blaise Computing. It requires the Microsoft C 5.0 or later or
QuickC, DOS 2.0 or later, and an IBM or compatible. The mouse functions
require a Microsoft-compatible mouse and its driver software. The product
sells for $149. Reader service no. 29.
Blaise Computing
2560 Ninth Street, Suite 316
Berkeley, CA 94710
415-540-5441
Zortech Inc. recently announced version 2.0 of its C++ compiler for MS-DOS.
C++ V2.0 Developer's Edition is fully compatible with the AT&T 2.0
specification, which includes support for multiple inheritance. Other features
include type safe linkage and built-in support for EMS. Version 2.0 also has
been enhanced for portability to other C environments including Microsoft C.
In addition, the Developer's Edition includes a C++ source level debugger, the
source code to its run-time library, and Version 2.0 of Zortech's C++ tools.
Each of these components may be purchased separately as well. The overall
system features compatibility with Microsoft Windows, a set of graphics
classes, and a TSR library that can make many applications resident through a
simple function call.
According to Scott Ladd, official beta tester, "its virtually 100 percent
compatible with AT&T 2.0, also supports Windows programming in OS/2 as well as
MS-DOS -- an impressive product."
Zortech has also announced the release of its OS/2 compiler upgrade, priced at
$149. The Zortech C++ Developer's Edition sells for $450, or the compiler can
be purchased separately for $199. Other components of the Developer's Edition
may also be purchased separately, including the new debugger, the run-time
library source code, and Version 2.0 of C++ Tools at $149 each. Updates to
existing users start at $40. Reader service no. 30.
Zortech, Inc.
1165 Massachusetts Avenue
Arlington, MA 02174
617-646-6703











































December, 1989
SWAINE'S FLAMES


It Takes More Than A Winning Smile




Michael Swaine


Paradigm: That collection of model problems, techniques, and specialized
knowledge that makes a specialist generally worthless.
I am a writer of prose, and when I venture outside the paradigm, I do so at
peril. Your peril, usually, but I did warn you. I'd like to talk about
marketing: The marketing of software innovation and the subversion of
marketing by software innovation.


First, Get a Million Dollars


How does a capable programmer with a good product idea become a successful
entrepreneur with a hot product? The standard truism, that you've got to have
money to make money, is no help and is disproved by the PC software industry
as a whole. It had nothing but ideas a decade ago, and now it's a
billion-dollar industry. Obviously, there's a cold start technique. What is
it? I've sniffed at the roots of the major personal computer software
companies without unearthing any rare truffles of wisdom. Maybe you've got to
know about business to learn about business. But here are two homely "morels"
I did dig up:
There appear to be two normal sources of development funding for those without
private fortunes or rich friends: Your current clients (or employer) and your
future customers. You can sell part of your enterprise, once you have one, to
investors or venture capitalists, but in the initial phase, you don't have
anything to sell but what's in your head. Venture capitalists don't want that,
and any private investors willing to take a chance on you are probably rich
friends, clients, or employers.
In the client-funded paradigm, you develop the components or early prototypes
of your product on contract to companies with big budgets, taking care to
retain all important rights to your work. (In the risky employer-funded
variation, you do product development on the job and hope you can take it away
with you without losing the product or your self respect or a lawsuit. It's
been known to work.) Client-funded development is probably the only way to
make money in the stackware market today. And it's the paradigm followed by
Microsoft, which got its start in Bill Gates's tireless refining of Microsoft
Basic under contract to MITS, Tandy, and others.
The most gutsy way to get funding is from future customers. This is how the
personal computer hardware industry was launched: Gullible electronic
hobbyists sent checks to post office boxes, and when there were enough checks,
the products got built. Legally, this technique gives you only a three-month
window, but a three-month interest-free loan is nothing to sneeze at. Many
software companies followed this mail order paradigm, some with more
scrupulous honesty than suggested by this scenario, some with less. The most
successful company to start out by mail orders is Borland International, which
made the transition to shelf space competitor with remarkable adroitness.


Agents are Subversive


One of the promises of object-oriented programming is that you will be able to
build software from reusable components: Brad Cox calls this "the software IC
model." But when it comes to finding the components, the software IC model
doesn't fit. Supplier catalogs are too narrow a channel. They work for real
ICs, because manufacturing costs already narrow the channel drastically, but
will be a problem for software components, for which there are likely to be as
many suppliers as customers. Some OOP supporters see agents as the answer.
An agent is a program that searches for information based on criteria you
supply. An agent might scan electronic news services to put together a custom
newspaper for you -- reflecting your interests. Or it might find reusable
software components. The search space could be a single software library or
all the sources for which you have e-mail addresses. Where the product sought
is information, agents could be essential in getting the product into the
hands of the person who needs it. And they could subvert the economy wherever
they are effectively used.
Usually, it is the vendor who takes responsibility for making the connection
with the customer. I can think of only two reasons why it would be appropriate
for the vendor to take responsibility for making the connection: 1. It is
easier for the vendor than for the customer; or 2. The vendor has more to gain
from the transaction. For the vast majority of goods, 1. can't be true: There
are just a lot more customers than vendors. I think that 2. is usually the
deciding factor: Most sales are driven by the vendor's needs rather than the
customer's.
Imagine a market managed by customers and driven by their needs. Imagine a
market in which there were no marketing costs to the vendor because there was
no marketing. If you wanted a tool to help you do your job, you would sketch a
description of its desired properties and dispatch your agent to search for
one. If you built a truly better mousetrap, the world really would beat a path
to your door.
Could we actually replace marketing by technology in this way? A lot of people
seem ready to try it in the area of reusable software components. If it works
there, the model might spread to other markets. Of course, one can think of
many products and services for which it obviously wouldn't work. And we know
that all those products and services serve real needs. Right?




























Special Issue, 1989
Special Issue, 1989
GUEST EDITORIAL


The Mac and the Mega-Micro Syndrome




Jef Raskin


It is a real pleasure and a distinct honor to have been invited to write this
introduction. You see, if it weren't for Dr. Dobb's, there probably would have
been no Macintosh. In fact, there might not be an Apple Computer Inc. today. A
bit of history will explain why this is so.
In 1976, Jim Warren, founder and then editor of Dr. Dobb's, sent a budding,
free-lance journalist and programmer to interview two guys who had been making
a bit of noise at the legendary Home-Brew Computer Club. He gave me a phone
number and an address. The journalist was me, and the two guys were both named
Steve, though one was always referred to as "Woz." They were working on the
Apple I, and I thought it had some of the slickest hardware ideas I had seen
in the two-year-old personal computer industry. I still have my Apple I, and
it still works.
I did the interview and offered to write the manuals for the early Apples.
Soon my newly-formed company, Bannister & Crun, and I ended up writing a lot
of the early Apple manuals, and eventually my whole crew was hired by Apple
Computer (in 1978) as their Publications Department. I was employee No. 31 and
I loved the Apple II, especially its pre-decoded bus. I own Apple II serial
No. 2 (on which I tested the manuals and early software), and it still works,
too.
By 1979 the company was gung ho on two projects spearheaded by Steve Jobs, the
Apple III and Lisa computers. I was worried that the Apple was not
forward-looking enough and that Lisa was too big and expensive for Apple's
customer base. So I created a project to make a small, relatively inexpensive,
small-footprint machine inspired by the screen architecture and some of the
interface elements I had come to love at Xerox PARC (at which I was a regular
visitor in the early '70s). I named the project after my favorite apple, the
McIntosh, and changed the spelling to avoid (so I hoped) conflict with the
hi-fi manufacturer of that name.
Thus Dr. Dobb's certainly deserves the credit for making a connection that led
to the creation of the Macintosh. And if Apple didn't have the Mac, would it
be in business today? Nobody can say for sure.


Enough History


As the following articles prove, the Macintosh has become a complex,
intimidating monster, requiring the same kind of convoluted understanding as
did the mainframes of a decade or two ago. We have reached the age of the
megamicro. Since "mega" stands for 10{6} and "micro" stands for 10{(-6)} this
term equals 1. In other words, we have canceled one of the advantages that
originally sparked the growth of microcomputers: Simplicity. We are back to
where we started. The other major advantage, namely low price, has also
eroded, but not to nearly the same extent, although the magazines once devoted
to "personal" computers no longer blush at prices easily exceeding $10,000 for
a single-user system.
On the other hand, we are exploring new ways to use computers. The Xerox/Apple
interface, and the growing understanding of object-oriented programming (OOP)
all lead to at least an external simplicity from the user's point of view. We
must learn to match it with internal simplicity as well. Some claim that our
brave, new graphic world is inherently complex. This is not true, but if we
believe it to be true we will not be able to do better.
In the meanwhile, articles like those in this issue will act like a guidebook
to an adventure game, and lead you ever deeper into the innards of Macintosh
software. Don the robes of a software wizard and you will find treasure here.



































Special Issue, 1989
PROGRAMMING WITH COLOR QUICKDRAW


Adding color and using multiple monitors




Chris Derossi


Chris is a member of the System Software group at Apple Computer and can be
reached at 20525 Mariani Ave., MS 27-AJ, Cupertino, CA 95014.


The large base of software available for the Macintosh it imperative that any
changes to the Mac, no matter how wonderful, not cause existing programs to
stop working. Consequently, any improvement to the Macintosh, small or large,
must be carefully designed to fit within the existing architecture. This is
hard enough with simple changes, such as adding a single new toolbox call.
With more extensive changes like adding color, maintaining compatibility can
be a monumental task.
The goal of compatibility is allowing older software to continue to function,
while the goal of new features is better software in the future. Well-written
applications can benefit from both compatibility and new features. However, it
is not always easy to create well-written applications. This is particularly
true when approaching the problem of color. With this in mind, this article
explains how you can write programs that are "color smart."


Maintaining Compatibility


One of the methods used to maintain compatibility is mimicry: New software
emulates its older counterpart. Older applications continue to work, because
the data structures and calls that they use are still supported. Often this
support is a facade, though, and the calls get translated into the new way of
doing things.
In Color QuickDraw, the CopyBits call continues to take the address of a
port's portBits field as a parameter, even when that field contains a
PixMap-Handle, and not a BitMap.
Sometimes the support for old calls and data structures is easy and natural.
Other times, however, this support requires hacks to an otherwise clean
design. And most times, the hacks would not be necessary if applications did
not make assumptions about the environment in which they were running.
Some assumptions are obvious, and can easily be avoided. Other assumptions,
though, are more subtle. When the Mac Plus was new, it was clear by then that
assuming the location of the screen RAM was bad. The assumption that each
pixel on the screen was represented by a single bit was less obvious. But
because multiple monitors for a single personal computer were not yet common,
it was natural to assume the presence of a single, rectangular, contiguous
display.
Because it is difficult to completely predict future changes, some assumptions
will always be made. Making new software compatible generally means supporting
these unavoidable assumptions.
Unfortunately, there is an adverse side effect to supporting such assumptions
with new software: Developers continue to write new programs that make the
same assumptions. Writing such programs is tacitly encouraged, because
software that assumes that it is running on the lowest common denominator of
the machines will always run on all of the machines. This practice is
acceptable, up to a point.
Programming for the lowest common denominator becomes undesirable when new
software takes advantage of some of the new features and yet still relies on
support for old assumptions. By their nature, assumptions valid for older
environments are usually limiting in the new environment. Using such
assumptions produces software that is unnecessarily limited.
One of the more common examples of this problem is color software, like paint
programs, that will not run unless a color monitor is the main display device.
Imagine the frustration such a program causes the Macintosh owner who has two
monitors -- a large black-and-white one as the main screen, and a color
display on the side.
The main screen on a Macintosh is the screen that is used to emulate the old,
single, rectangular display. Associated concepts are also supported by this
main screen, such as the QuickDraw global variable screenBits, which is
supposed to contain the BitMap that defines the display. Except for certain
hacks put in by Apple, programs that use screenBits find themselves restricted
to a single screen when several might be available.
Again, this limitation isn't so bad for older software, or software that needs
to run the same way on all models of the Macintosh. But programs written to
know about Color QuickDraw could go one step further and know about multiple
display devices. And programs that require Color QuickDraw to run have no
excuse for not taking advantage of all of Color QuickDraw's features.
As a rule, programs that utilize the newest routines and data structures have
greater functionality and flexibility. These programs also have a longer life
expectancy, as older features will not be supported forever. They also prevent
headaches for users who don't understand why the menu bar has to be on a
particular screen for some programs to work.


Handling Multiple Monitors


As you can probably see by now, an important feature of a color-smart program
is the ability to handle multiple monitors. The functionality required by your
program may be available on a screen other than the main one. Instead of
checking just the main screen for a particular configuration, such as pixel
depth, a program should check all available screens, or better yet, let the
user decide which screen to use.
If your window is displayed on a screen that can't support the program's
features, it is better to disable the use of those features than to
unconditionally exit the program. The user might change the situation at least
two ways: By dragging the window to a different monitor, or by changing the
screen attributes with the Control Panel. The program should explain to the
user why features are disabled, and should suggest possible solutions.
In addition to your program's environmental needs, the use of multiple
monitors affects the question of screen real estate. Historically, the size of
the screen would dictate the limits for dragging and sizing windows. The
common practice was to set these limits to an approximation of the screenBits
.bounds rectangle.
Because the screenBits variable represents only the main screen, the user of a
program developed before the advent of the Macintosh II would have been unable
to drag windows to other monitors. Fortunately, Apple put in a hack that
allows windows to be dragged onto other displays.
Changing the size of a window is another matter. In most cases, a 9-inch
monitor was not large enough to display an entire document, so limiting the
size of a window to the size of the screen was not a problem. With the
proliferation of large monitors, though, windows could easily become larger
than their documents. This resulted in garbled displays and sometimes in
crashes. In other cases, programs benefited from having as large a window as
they could get.
And so the relationship between window contents and window size must be
considered much more carefully than that between window contents and window
position. The limit you set for the size of a window should be based solely on
the nature of the document in the window. Windows set up this way will be able
to span multiple monitors when appropriate. For the user who purchases a
second or larger screen, the benefit will be automatic.


High-Level Calls and Data Structures


Apple has defined high-level calls and documented data structures for
determining the characteristics of display devices. You should take advantage
of these resources.
The linked list of display devices (called "GDevices") can be accessed and
traversed with the GetDeviceList and GetNextDevice calls. You can use the call
GetMainDevice to find the particular element of this list that represents the
main screen. Because each GDevice references a PixMap, you can determine the
current color settings of each monitor. Each GDevice also contains a rectangle
that represents that monitor's global position.
With this information, your program can locate the monitor which best supports
your program's requirements. You may, for example, wish to open new windows on
the most appropriate monitor. (An example of this technique is built into the
program presented later in this article.)


Handling Special Cases


In addition to increased functionality, programs that know about their
environment can provide greater efficiency and better-looking output.

Images designed to look great on an 8-bit color display don't always come out
as nicely when mapped by QuickDraw to a color device of lower resolution, or
to a black-and-white device. And color mapping incurs an overhead that can be
detrimental to high-performance programs. Frequently, however, you can make
the same images look just as good at lower color resolutions, either by using
a different set of colors, or by using patterns.
In general, software will work fine using a pure color model, and letting
QuickDraw do the best rendering possible on each device, but there is an
alternative if you wish to go that extra mile for appearance and performance.
In addition to imaging for the generic case, you can deal with special cases.
Performance and output will generally be enhanced for each special case,
although the amount of improvement will vary, depending on the application.
The best way to apply this technique is to use a generic case that extends to
the limits defined by ColorQuickDraw. In other words, use as many colors as
are appropriate for the application, and specify each color by using the full
48 bits available in the RGBColordata structure. Then handle the common
special cases of lower functionality, such as 8-bit color, 4-bit color, and
black-and-white.
It is important to deal with the high-end generic situation. After your
program has been written, more sophisticated hardware and software are bound
to come along. Even though it seemed superfluous to draw with more than 256
colors when the Mac II first came out, the programs that did now render
full-color images using Apple's 32-bit QuickDraw.
In order to deal with special cases, your program must know about the devices
to which it will be drawing. The program can traverse the list of GDevices and
compare their locations to the global coordinates of your window. Once the
GDevice that contains your window is found, the GDevice's PixMap will let the
program decide which, if any, special case to use.
A given window may intersect more than one display, and each display might be
set to a different configuration. When traversing the device list, the program
should treat each device window intersection as a separate case.
Even if a program is running on a Macintosh that has a single monitor, the
user can change the color environment at any time with the Control Panel. Such
changes may or may not be of concern to your program. If you use the technique
just described for all of your drawing, then everything will work
automatically, because this technique requires that you can actually draw
something meaningful on each device, no matter what its color settings. An
update event will be generated when the color environment changes, which will
prompt the program to execute its GDevice intersection and drawing loop.
If your program requires a certain minimum color environment, you should check
to see if the color environment has changed whenever your program gets an
update event. This test is easy, because the ctSeed field in the color table
of the GDevice's PixMap will change if that GDevice's color environment
changes. If the ctSeed has changed, you can then check other values, like the
pixel depth, to see if you should disable some features or just generate a new
set of colors to use.


Using the Palette Manager


Your color programs should always use the Palette Manager. The days when the
Palette Manager created more problems than it solved ended with the release of
System 6.0.2. Now, as part of 32-bit QuickDraw, the Palette Manager has even
more functionality and features that you will definitely want to use. In
response to voluminous feedback, the Palette Manager has been extended to
support the kinds of things that developers want to do.
For example, a single palette can now contain a different list of colors for
each kind of device. This is exactly what is needed to support the technique
described earlier for drawing the right thing to each GDevice.
Any old work-arounds that used to be required should be discarded. The support
provided by the system via the Palette Manager should be used instead. Apple
will continue to maintain the Palette Manager, but the same cannot be said
about non-Apple work-arounds.


The ShowColors Program


The program ShowColors demonstrates some of the concepts described in this
article. Listing One (page 57) gives the MPW Pascal version of this program,
Listing Two (page 58) gives the MPW C version, and Listing Three (page 60)
lists the Rez input for the application.
ShowColors is a color-smart application that displays something meaningful on
any CLUT (color lookup table) or Fixed device. It doesn't crash when it
encounters other types of devices. The display shown for each device is a
representation of that device's color table. To create the display, the
program uses the Palette Manager's pmExplicit entry type.
In order to show the right thing on each screen, the program uses the device
window intersection loop. If the area of intersection is large enough, then
the color table is drawn; otherwise, that portion of the window is left white.
For displays that are not CLUT or Fixed-type devices, the window is painted
with a 50 percent gray. When the program starts, it uses a simple algorithm to
find what it considers to be the best device. The window is centered on that
screen. Because this program has no fixed document size, the window can be
made as large as desired.


Conclusion


I hope that this article has encouraged you to write programs that take full
advantage of the ColorQuickDraw features. Your programs will live a longer
life, and your users will appreciate the added functionality and flexibility.
_PROGRAMMING WITH COLOR QUICKDRAW_
by Chris DeRossi


[LISTING ONE]

{[j=15-/40]}

PROGRAM ShowColors;

{ A color-smart application by Chris Derossi for Dr. Dobb's Journal.
 This nifty little utility displays the color table currently set for each
 display device. It uses a single window which can be moved onto any monitor,
 and grown to any size. The window is also allowed to lie across multiple
 monitors; each monitor-window intersection is drawn separately.
 If a particular device does not have a color table (it is not a
 CLUT or Fixed device), then that portion of the window is
 filled with 50% gray. If there is not enough room to display a
 device's entire color table, the window remains white.
 The window is initially made visible on what the program
 considers to be the best device for displaying color. }

USES
 Memtypes, OSIntf, ToolIntf, QuickDraw, Palettes;
 PROCEDURE _DataInit; {Declared so we can reference it later.}
 EXTERNAL;
 CONST
 appleID = 128; {Standard Apple menu}
 fileID = 129; {File menu for Quit command}

 editID = 130; {Edit menu for DAs}
 appleM = 1;
 fileM = 2;
 editM = 3;
 menuCount = 3;
 kWindowID = 128; {The ID of our single window}
 kAboutMeDLOG = 128; {The About ShowColorsI Dialog}
 kNoColorID = 129; {The ID of the error alert}
 undoCommand = 1;
 cutCommand = 3;
 copyCommand = 4;
 pasteCommand = 5;
 clearCommand = 6;
 aboutMeCommand = 1; {About ShowColorsI item in the Apple menu}
 quitCommand = 1; {Quit command in the File menu}
 mBarHeight = $BAA;
 TYPE
 IntPtr = ^Integer;
 VAR
 myMenus : ARRAY [1..menuCount] OF MenuHandle;
 dragRect : Rect;
 newSize : LongInt;
 doneFlag : BOOLEAN;
 wRecord : WindowRecord;
 myWindow : WindowPtr;

{$S Initialize}
 PROCEDURE SetUpMenus;
 VAR
 i : Integer;
 BEGIN
 myMenus[appleM] := GetMenu(appleID);
 AddResMenu(myMenus[appleM], 'DRVR');
 myMenus[fileM] := GetMenu(fileID);
 myMenus[editM] := GetMenu(editID);
 FOR i := 1 TO menuCount DO
 InsertMenu(myMenus[i], 0);
 DrawMenuBar;
 END;

{$S Initialize}
 FUNCTION FindBestDevice : GDHandle;
 { This function finds what it considers to be the best device from
 the list of screens connected to the Macintosh. For this program,
 best is considered to be more colors, and color is better than
 monochrome. The precise ordering of goodness is: 1-bit mono,
 2-bit mono, 2-bit color, 4-bit mono, 8-bit mono, 4-bit color,
 8-bit color. Non-CLUT or Fixed devices are not good at all for
 this program, even if they support more colors.
 To compare the devices, each device is assigned a rating. The
 higher the rating, the better the device. To convert a device's
 characteristics into a rating, we first convert the characteristics
 into a number, then use that number in a CASE statement to assign
 the rating. The mapping from characteristics to integer is as follows:
 bits 0..6 : pixel size = color depth (enough bits to handle
 127 bits/pixel)
 bit 7 : 0 = monochrome, 1 = color. }
 VAR
 aDevice : GDHandle;

 bestDevice : GDHandle;
 aRating : Integer;
 bestRating : Integer;
 BEGIN
 bestRating := 0;
 aDevice := GetDeviceList;
 bestDevice := aDevice; {In case we donUt find any good devices}
 WHILE aDevice <> NIL DO BEGIN
 IF (NOT TestDeviceAttribute(aDevice, screenActive)) OR
 ((aDevice^^.gdType <> clutType) AND (aDevice^^.gdType <> fixedType)) THEN
 aRating := 0
 ELSE
 CASE BAnd(aDevice^^.gdFlags, 1) * 128 + aDevice^^.gdPMap^^.pixelSize OF
 1 : aRating := 1; {1-bit monochrome}
 129 : aRating := 2; {1-bit color}
 2 : aRating := 3; {2-bit monochrome}
 130 : aRating := 4; {2-bit color}
 4 : aRating := 5; {4-bit monochrome}
 8 : aRating := 6; {8-bit monochrome}
 132 : aRating := 7; {4-bit color}
 136 : aRating := 8; {8-bit color}
 END;
 IF aRating > bestRating THEN BEGIN
 bestRating := aRating;
 bestDevice := aDevice;
 END;
 aDevice := GetNextDevice(aDevice);
 END;
 FindBestDevice := bestDevice;
 END;

{$S Main}
 FUNCTION PositionWindow(worldRect, windRect : Rect) : Rect;
 { This function centers the windRect over the worldRect in the horizontal
 direction, and places windRect one third of the way down over worldRect
 in the vertical direction. This positioned rectangle is then returned.}
 BEGIN
 OffsetRect(windRect, -windRect.left, -windRect.top);
 WITH worldRect DO
 OffsetRect(windRect, (right + left - windRect.right) DIV 2,
 (bottom - top - windRect.bottom) DIV 3 + top);
 PositionWindow := windRect;
 END;

{$S Initialize}
 PROCEDURE ShowColorsInit;
 { Initialize the standard Mac stuff and the application stuff. For the
 application, the window needs to be created and placed on the best
 available monitor.
 Since this program requires Color QuickDraw, we check for its presence with
 SysEnvirons before we try to open the window. If Color QuickDraw is not
 present, we set doneFlag to TRUE which causes the application to
 terminate right away.}
 VAR
 mySysStuff : SysEnvRec;
 bestDevice : GDHandle;
 aRect : Rect;
 mbhPtr : IntPtr;
 dummyItem : Integer;

 myPalette : PaletteHandle;
 BEGIN
 UnLoadSeg(@_DataInit); {Get rid of MPWUs data initialization segment}
 MaxApplZone;
 InitGraf(@thePort);
 InitFonts;
 FlushEvents(everyEvent, 0);
 InitWindows;
 InitMenus;
 TEInit;
 InitDialogs(NIL);
 InitCursor;
 IF SysEnvirons(1, mySysStuff) = 0 THEN {Nothing} ;
 IF mySysStuff.hasColorQD THEN BEGIN
 SetUpMenus;
 { Get our window and create a palette for it. Our palette needs to have
 256 explicit entries. We don't care what the palette color entries
 are, so we can pass NIL as the color table handle to NewPalette. }
 myWindow := GetNewCWindow(kWindowID, @wRecord, Pointer(-1));
 myPalette := NewPalette(256, NIL, pmExplicit, 0);
 NSetPalette(myWindow, myPalette, pmAllUpdates);
 { Find the best screen for our window. The window is markes as
 invisible in the resource template so we can move it before we
 show it. }
 bestDevice := FindBestDevice;
 aRect := bestDevice^^.gdRect; {Device's global rectangle}
 IF bestDevice = GetMainDevice THEN BEGIN {Take menu bar into account.}
 mbhPtr := IntPtr(mBarHeight); {Get ptr to low memory global}
 aRect.top := aRect.top + mbhPtr^; {Adjust size of rectangle}
 END;
 aRect := PositionWindow(aRect, myWindow^.portRect);
 MoveWindow(myWindow, aRect.left, aRect.top, TRUE);
 ShowWindow(myWindow);
 SetPort(myWindow);
 doneFlag := FALSE; {Will be set to true when user chooses Quit}
 END ELSE BEGIN
 dummyItem := StopAlert(kNoColorID, NIL);
 doneFlag := TRUE;
 END;
 END;

{$S Main}
 PROCEDURE ShowAboutMeDialog;
 VAR
 itemHit : Integer;
 theDialog : DialogPtr;
 savedPort : GrafPtr;
 aRect : Rect;
 mbhPtr : IntPtr;
 BEGIN
 GetPort(savedPort);
 theDialog := GetNewDialog(kAboutMeDLOG, NIL, WindowPtr( - 1));
 SetPort(theDialog);
 aRect := screenBits.bounds; {Main Device's global rectangle}
 mbhPtr := IntPtr(mBarHeight);
 aRect.top := aRect.top + mbhPtr^; {Adjust for the menu bar}
 aRect := PositionWindow(aRect, theDialog^.portRect);
 MoveWindow(theDialog, aRect.left, aRect.top, TRUE);
 ShowWindow(theDialog);

 REPEAT
 ModalDialog(NIL, itemHit)
 UNTIL (itemHit = ok);
 DisposDialog(theDialog);
 SetPort(savedPort);
 END;

{$S Main}
 PROCEDURE DrawWindowContents(aWindow : WindowPtr);
 { This is the procedure that loops through all of the screens and draws
 whatever is appropriate for that screen in the part of the window
 which intersects the screen. Each screen-window intersection is
 treated separately. }
 VAR
 grayRGB : RGBColor;
 aDevice : GDHandle;
 aRect : Rect;
 globalWindRect : Rect;
 mbhPtr : IntPtr;
 workRect : Rect;
 vCount : Integer;
 hCount : Integer;
 vBlockSize : Integer;
 hBlockSize : Integer;
 v : Integer;
 h : Integer;
 BEGIN
 { Create a 50% gray color for filling in non-clut/fixed devices }
 WITH grayRGB DO BEGIN
 red := $8000;
 green := $8000;
 blue := $8000;
 END;
 { Turn the window's portRect into global coordinates for intersecting
 the screens. }
 globalWindRect := aWindow^.portRect;
 LocalToGlobal(globalWindRect.topLeft);
 LocalToGlobal(globalWindRect.botRight);
 { Loop through all of the devices, seeing if we intersect each one }
 aDevice := GetDeviceList;
 WHILE aDevice <> NIL DO BEGIN
 aRect := aDevice^^.gdRect;
 IF aDevice = GetMainDevice THEN BEGIN {Exclude menu bar from draw. area}
 mbhPtr := IntPtr(mBarHeight); {Get a ptr to the low memory global}
 aRect.top := aRect.top + mbhPtr^; {Adjust size of working rectangle}
 END;
 IF SectRect(aRect, globalWindRect, workRect) THEN BEGIN
 GlobalToLocal(workRect.topLeft);
 GlobalToLocal(workRect.botRight);
 { Figure how many blocks to draw to show whole color table }
 IF (aDevice^^.gdType = clutType) OR (aDevice^^.gdType = fixedType) THEN BEGIN
 CASE aDevice^^.gdPMap^^.pixelSize OF
 1 : BEGIN
 vCount := 1;
 hCount := 2;
 END;
 2 : BEGIN
 vCount := 2;
 hCount := 2;

 END;
 4 : BEGIN
 vCount := 4;
 hCount := 4;
 END;
 8 : BEGIN
 vCount := 16;
 hCount := 16;
 END;
 OTHERWISE { Uh oh. A pixel size that we canUt handle. }
 vCount := -1; {Force the vBlockSize to be less than zero.}
 END;
 { Size of blocks be in the horizontal and vertical directions? }
 vBlockSize := (workRect.bottom - workRect.top) DIV vCount;
 hBlockSize := (workRect.right - workRect.left) DIV hCount;
 { Use the smaller dimension for both to keep the blocks square }
 IF vBlockSize < hBlockSize THEN
 hBlockSize := vBlockSize
 ELSE
 vBlockSize := hBlockSize;
 { If there is enough room to draw the color table, then do it. }
 IF (vBlockSize > 0) AND (hBlockSize > 0) THEN BEGIN
 FOR v := 0 TO vCount-1 DO
 FOR h := 0 TO hCount-1 DO BEGIN
 PMForeColor(v * hCount + h);
 SetRect(aRect, workRect.left + h * hBlockSize, workRect.top + v * vBlockSize,
 workRect.left + (h+1) * hBlockSize, workRect.top + (v+1) * vBlockSize);
 PaintRect(aRect);
 END;
 END;
 END ELSE BEGIN { Not a CLUT or Fixed device. Draw gray on screen }
 RGBForeColor(grayRGB);
 PaintRect(workRect);
 END;
 END;
 aDevice := GetNextDevice(aDevice);
 END;
 END;

{$S Main}
 PROCEDURE DoCommand(mResult: LongInt);
 VAR
 theItem : Integer;
 theMenu : Integer;
 name : Str255;
 temp : Integer;
 dummyBool : Boolean;
 BEGIN
 theItem := LoWord(mResult);
 theMenu := HiWord(mResult);
 CASE theMenu OF
 appleID:
 IF (theItem = aboutMeCommand) THEN
 ShowAboutMeDialog
 ELSE BEGIN
 GetItem(myMenus[appleM], theItem, name);
 temp := OpenDeskAcc(name);
 SetPort(myWindow);
 END;

 fileID:
 CASE theItem OF
 quitCommand : doneFlag := TRUE;
 END;
 editID:
 dummyBool := SystemEdit(theItem - 1);
 END;
 HiliteMenu(0);
 END;

{$S Main}
 PROCEDURE Mainloop;
 { This is the standard event polling procedure that finds out what needs to
 be done and handles the request or dispatches to the appropriate routine. }
 VAR
 dragRect : Rect;
 newSize : LongInt;
 theChar : CHAR;
 myEvent : EventRecord;
 whichWindow : WindowPtr;
 BEGIN
 SystemTask;
 IF GetNextEvent(everyEvent, myEvent) THEN
 CASE myEvent.what OF
 mouseDown:
 CASE FindWindow(myEvent.where, whichWindow) OF
 inSysWindow: SystemClick(myEvent, whichWindow);
 inMenuBar: DoCommand(MenuSelect(myEvent.where));
 inDrag: BEGIN
 { If the boundsRect parameter passed to DragWindow looks like it was
 derived from screenBits.bounds, DragWindow will substitute a region
 which represents active screens. This is what we want, since
 we can't pass a region to DragWindow, we have to rely on this hack.
 That's why we make dragRect equal to screenBits.bounds. }
 dragRect := screenBits.bounds;
 DragWindow(whichWindow, myEvent.where, dragRect);
 { Because dragging window may change the device-window
 intersections, we force the window to be redrawn completely.
 This causes a redraw even when the window is moved only a
 little bit on the same screen. A little more intelligence
 could be added here to avoid unneeded updating. That's left
 for the reader. }
 InvalRect(myWindow^.portRect);
 END;
 inGrow: BEGIN
 SetRect(dragRect, 32, 32, 32766, 32766);
 newSize := GrowWindow(whichWindow, myEvent.where, dragRect);
 IF LongInt(newSize) <> 0 THEN BEGIN
 SizeWindow(whichWindow, LoWord(newSize), HiWord(newSize), TRUE);
 InvalRect(myWindow^.portRect);
 END;
 END;
 inContent: BEGIN
 IF whichWindow <> FrontWindow THEN
 SelectWindow(whichWindow);
 END;
 END; {of mouseDown case}
 keyDown, autoKey:
 IF myWindow = FrontWindow THEN BEGIN

 theChar := CHR(BAnd(myEvent.message, charCodeMask));
 IF BAnd(myEvent.modifiers, cmdKey) <> 0 THEN
 DoCommand(MenuKey(theChar));
 END;
 activateEvt:
 IF WindowPtr(myEvent.message) = myWindow THEN BEGIN
 IF BAnd(myEvent.modifiers, activeFlag) <> 0 THEN BEGIN
 DisableItem(myMenus[editM], 0);
 END ELSE BEGIN
 EnableItem(myMenus[editM], 0);
 END;
 DrawMenuBar;
 END;
 updateEvt:
 IF WindowPtr(myEvent.message) = myWindow THEN BEGIN
 BeginUpdate(myWindow);
 EraseRect(myWindow^.portRect);
 DrawWindowContents(myWindow);
 EndUpdate(myWindow);
 END;
 END; {of myEvent.what cases}
 END;

{$S Main}
BEGIN {ShowColors}
 ShowColorsInit;
 WHILE NOT doneFlag DO
 Mainloop;
END.



[LISTING TWO]

/*
 * ShowColors - A sample color-smart application
 * by Chris Derossi for Dr. Dobb's Journal. MPW C Version.
 * This nifty little utility displays the color table currently set for each
 * display device. Uses a single window which can be moved onto any monitor,
 * and grown to any size. The window is also allowed to lie across multiple
 * monitors; each monitor-window intersection is drawn separately.
 * If a particular device does not have a color table (not a CLUT or Fixed
 * device), then that part of the window is filled with 50% gray. If there is
 * not enough room to display a deviceUs entire color table, window remains
 * white.
 * Window is initially made visible on what the program considers to be the
 * best device for displaying color.
 */

#include <types.h>
#include <memory.h>
#include <events.h>
#include <osevents.h>
#include <desk.h>
#include <toolutils.h>
#include <osutils.h>
#include <menus.h>
#include <windows.h>
#include <dialogs.h>

#include <Resources.h>
#include <QuickDraw.h>
#include <Fonts.h>
#include <Palettes.h>

#define appleID 128
#define fileID 129
#define editID 130
#define appleM 0
#define fileM 1
#define editM 2
#define menuCount 3
#define kWindowID 128
#define kAboutMeDLOG 128
#define kNoColorID 129
#define aboutMeCommand 1
#define quitCommand 1
#define MBarHeight (*((short *) 0xBAA))

MenuHandle myMenus[menuCount];
Rect dragRect;
long newSize;
Boolean doneFlag;
WindowRecord wRecord;
WindowPtr myWindow;

void SetUpMenus()

{
 short i;
 myMenus[appleM] = GetMenu(appleID);
 AddResMenu(myMenus[appleM], 'DRVR');
 myMenus[fileM] = GetMenu(fileID);
 myMenus[editM] = GetMenu(editID);
 for (i = 0; i < menuCount; i++)
 InsertMenu(myMenus[i], 0);
 DrawMenuBar();
}

GDHandle FindBestDevice()
/* This function finds what the best device from the list of
 * screens connected to the Macintosh. For this program, best is considered
 * more colors, and color is better than monochrome. The ordering of goodness
 * is: 1-bit mono, 2-bit mono, 2-bit color, 4-bit mono, 8-bit mono,
 * 4-bit color, 8-bit color. Non-CLUT or Fixed devices are not good at all
 * for this program, even if they support more colors.
 * To compare the devices, each device is assigned a rating. The higher the
 * rating, the better the device. To convert a device's characteristics into
 * a rating, first convert the characteristics into a number, then use that
 * number in a CASE statement to assign the rating. The mapping from
 * characteristics to integer is as follows:
 * bits 0..6 : pixel size = color depth (enough bits to handle 127
 * bits/pixel); bit 7 : 0 = monochrome, 1 = color.
 */
{

 GDHandle aDevice;
 GDHandle bestDevice;
 short aRating;

 short bestRating;

 aDevice = bestDevice = GetDeviceList();
 bestRating = 0;
 while (aDevice) {
 if ((!TestDeviceAttribute(aDevice, screenActive)) 
 (((**aDevice).gdType != clutType) && ((**aDevice).gdType != fixedType)))
 aRating = 0;
 else
 switch (((**aDevice).gdFlags & 1) * 128 + (**(**aDevice).gdPMap).pixelSize) {
 case 1:
 aRating = 1; // 1-bit monochrome
 break;
 case 129:
 aRating = 2; // 1-bit color
 break;
 case 2:
 aRating = 3; // 2-bit monochrome
 break;
 case 130:
 aRating = 4; // 2-bit color
 break;
 case 4:
 aRating = 5; // 4-bit monochrome
 break;
 case 8:
 aRating = 6; // 8-bit monochrome
 break;
 case 132:
 aRating = 7; // 4-bit color
 break;
 case 136:
 aRating = 8; // 8-bit color
 break;
 }
 if (aRating > bestRating) {
 bestRating = aRating;
 bestDevice = aDevice;
 }
 aDevice = GetNextDevice(aDevice);
 }

 return(bestDevice);
}

void PositionWindow(worldRect, windRect, resultRect)
 Rect worldRect;
 Rect windRect;
 Rect *resultRect;

/* This function centers the windRect over the worldRect in the horizontal
 * direction, and places windRect one third of the way down over worldRect
 * in the vertical direction.
 */

{
 *resultRect = windRect;
OffsetRect(resultRect, -resultRect->left, -resultRect->top);
OffsetRect(resultRect,(worldRect.right + worldRect.left -
resultRect->right)/2,

 (worldRect.bottom - worldRect.top - resultRect->bottom) / 3 + worldRect.top);
}

void ShowColorsInit() {
/* Initialize the standard Mac and the application stuff. For the application,
 * the window needs to be created and placed on the RbestS available monitor.
 * Since this program requires Color QuickDraw, we check for its presence with
 * SysEnvirons before opening window. If Color QuickDraw is not present,
 * set doneFlag to TRUE which causes the application to terminate right away.
 */

 SysEnvRec mySysStuff;
 GDHandle bestDevice;
 Rect aRect;
 PaletteHandle myPalette;

 MaxApplZone();
 InitGraf(&qd.thePort);
 InitFonts();
 FlushEvents(everyEvent, 0);
 InitWindows();
 InitMenus();
 TEInit();
 InitDialogs(nil);
 InitCursor();

 SysEnvirons(1, &mySysStuff);
 if (mySysStuff.hasColorQD) { // We're in good shape. Setup everything.
 SetUpMenus();

 /* Get window and create a palette. Our palette needs to have 256 explicit
 entries. Don't care what palette color entries are, so we can pass NIL
 as the color table handle to NewPalette. */
 myWindow = GetNewCWindow(kWindowID, (Ptr)&wRecord, (WindowPtr)-1);
 myPalette = NewPalette(256, nil, pmExplicit, 0);
 NSetPalette(myWindow, myPalette, pmAllUpdates);

 /* Find best screen for window. The window is markes as invisible in the
 resource template so we can move it before we show it. */
 bestDevice = FindBestDevice();
 aRect = (**bestDevice).gdRect; // Device's global rectangle
 if (bestDevice == GetMainDevice()) // Take menu bar into account.
 aRect.top += MBarHeight; // Adjust size of working rectangle
 PositionWindow(aRect, myWindow->portRect, &aRect);
 MoveWindow(myWindow, aRect.left, aRect.top, true);
 ShowWindow(myWindow);
 SetPort(myWindow);

 doneFlag = false; // Will be set to true when the user chooses Quit
 } else {
 StopAlert(kNoColorID, nil);
 doneFlag = true;
 }
}

void ShowAboutMeDialog()
{
 short itemHit;
 DialogPtr theDialog;

 GrafPtr savedPort;
 Rect aRect;
 GetPort(&savedPort);
 theDialog = GetNewDialog(kAboutMeDLOG, nil, (WindowPtr)-1);
 SetPort(theDialog);
 aRect = qd.screenBits.bounds; // Main Device's global rectangle
 aRect.top += MBarHeight; // Adjust for the menu bar
 PositionWindow(aRect, theDialog->portRect, &aRect);
 MoveWindow(theDialog, aRect.left, aRect.top, true);
 ShowWindow(theDialog);
 do
 ModalDialog(nil, &itemHit);
 while (itemHit != ok);
 DisposDialog(theDialog);
 SetPort(savedPort);
}

void DrawWindowContents(aWindow)
 WindowPtr aWindow;
/* The procedure that loops through all of the screens and draws whatever is
 * appropriate for that screen. Each screen-window intersection is treated
 * separately.
 */
{
 RGBColor grayRGB;
 GDHandle aDevice;
 Rect aRect;
 Rect globalWindRect;
 Rect workRect;
 short vCount;
 short hCount;
 short vBlockSize;
 short hBlockSize;
 short v;
 short h;

 // Create a 50% gray color for filling in non-clut/fixed devices
 grayRGB.red = 0x8000;
 grayRGB.green = 0x8000;
 grayRGB.blue = 0x8000;

 // Turn window's portRect into global coordinates for intersecting screens.
 globalWindRect = aWindow->portRect;
 LocalToGlobal((Point *)&globalWindRect.top);
 LocalToGlobal((Point *)&globalWindRect.bottom);

 // Loop through all of the devices, seeing if we intersect each one.
 aDevice = GetDeviceList();
 while (aDevice) {
 aRect = (**aDevice).gdRect;
 if (aDevice == GetMainDevice()) // Exclude menu bar from drawable area
 aRect.top += MBarHeight; // Adjust size of working rectangle
 if (SectRect(&aRect, &globalWindRect, &workRect)) { // Window intersects.
 GlobalToLocal((Point *)&workRect.top);
 GlobalToLocal((Point *)&workRect.bottom);
 // Figure out how many blocks to draw to show the whole color table
 if (((**aDevice).gdType == clutType) ((**aDevice).gdType == fixedType)) {
 switch ((**(**aDevice).gdPMap).pixelSize) {
 case 1:

 vCount = 1;
 hCount = 2;
 break;
 case 2:
 vCount = hCount = 2;
 break;
 case 4:
 vCount = hCount = 4;
 break;
 case 8:
 vCount = hCount = 16;
 break;
 default: // Uh oh. A pixel size that we canUt handle.
 vCount = -1; // Will force the vBlockSize to be less than zero.
 }
 // How big will the blocks be in horizontal and vertical directions?
 vBlockSize = (workRect.bottom - workRect.top) / vCount;
 hBlockSize = (workRect.right - workRect.left) / hCount;
 // Use the smaller dimension for both to keep the blocks square
 if (vBlockSize < hBlockSize)
 hBlockSize = vBlockSize;
 else
 vBlockSize = hBlockSize;
 // If there is enough room to draw the color table, then do it.
 if ((vBlockSize > 0) && (hBlockSize > 0)) {
 for (v = 0; v < vCount; v++)
 for (h = 0; h < hCount; h++) {
 PmForeColor(v * hCount + h);
 SetRect(&aRect, workRect.left + h * hBlockSize,
 workRect.top + v * vBlockSize,
 workRect.left + (h+1) * hBlockSize,
 workRect.top + (v+1) * vBlockSize);
 PaintRect(&aRect);
 }
 }
 } else { // Not a CLUT or Fixed device. Draw gray on this screen.
 RGBForeColor(&grayRGB);
 PaintRect(&workRect);
 }
 }
 aDevice = GetNextDevice(aDevice);
 }
}

void DoCommand(mResult)
 long mResult;
{
 short theItem;
 short theMenu;
 char name[256];
 theItem = LoWord(mResult);
 theMenu = HiWord(mResult);
 switch (theMenu) {
 case appleID:
 if (theItem == aboutMeCommand)
 ShowAboutMeDialog();
 else {
 GetItem(myMenus[appleM], theItem, name);
 OpenDeskAcc(name);

 SetPort(myWindow);
 }
 break;
 case fileID:
 doneFlag = true;
 break;
 case editID:
 SystemEdit(theItem - 1);
 break;
 }
 HiliteMenu(0);
}

void Mainloop()
/* Standard event polling procedure that finds out what needs to be done
 * and handles the request or dispatches to the appropriate routine.
 */
{

 Rect dragRect;
 long newSize;
 char theChar;
 EventRecord myEvent;
 WindowPtr whichWindow;
 SystemTask();
 if (GetNextEvent(everyEvent, &myEvent))
 switch (myEvent.what) {
 case mouseDown:
 switch (FindWindow(myEvent.where, &whichWindow)) {
 case inSysWindow:
 SystemClick(&myEvent, whichWindow);
 break;
 case inMenuBar:
 DoCommand(MenuSelect(myEvent.where));
 break;
 case inDrag:
 // If boundsRect parameter passed to DragWindow looks like it was
 // derived from screenBits.bounds, DragWindow substitutes region
 // which represents all of the active screens. We want this, since
 // we can't pass a region to DragWindow, we have to rely on hack.
 // That's why we make dragRect equal to screenBits.bounds.
 dragRect = qd.screenBits.bounds;
 DragWindow(whichWindow, myEvent.where, &dragRect);
 // Because dragging the window may change the device-window
 // intersections, window is redrawn completely. This causes
 // a redraw even when the window is moved on the same
 // screen. More intelligence could be added here to avoid unneeded
 // updating. That's left for the reader.
 InvalRect(&myWindow->portRect);
 break;
 case inGrow:
 SetRect(&dragRect, 32, 32, 32766, 32766);
 newSize = GrowWindow(whichWindow, myEvent.where, &dragRect);
 if (newSize) {
 SizeWindow(whichWindow, LoWord(newSize), HiWord(newSize), true);
 InvalRect(&myWindow->portRect);
 }
 break;
 case inContent:

 if (whichWindow != FrontWindow())
 SelectWindow(whichWindow);
 break;
 } // end of mouseDown case
 case keyDown:
 case autoKey:
 if (myWindow == FrontWindow()) {
 theChar = (myEvent.message & charCodeMask);
 if (myEvent.modifiers & cmdKey)
 DoCommand(MenuKey(theChar));
 }
 break;
 case activateEvt:
 if (myEvent.message == myWindow) {
 if (myEvent.modifiers & activeFlag) {
 DisableItem(myMenus[editM], 0);
 } else {
 EnableItem(myMenus[editM], 0);
 }
 DrawMenuBar();
 }
 break;
 case updateEvt:
 if (myEvent.message == myWindow) {
 BeginUpdate(myWindow);
 EraseRect(&myWindow->portRect);
 DrawWindowContents(myWindow);
 EndUpdate(myWindow);
 }
 break;
 } // end of myEvent.what cases
}

void main()
{
 ShowColorsInit();
 while (!doneFlag)
 Mainloop();
}



[LISTING THREE]

/* ShowColors.r - Rez source for the color-smart program
 * by Chris Derossi for Dr. Dobb's Journal
 */

 #include "Types.r"

/* These define's are used in the MENU resources to disable specific
 menu items. */
#define AllItems 0b1111111111111111111111111111111 /* 31 flags */
#define MenuItem1 0b00001
#define MenuItem2 0b00010

type 'ShCo' as 'STR ';

resource 'ShCo' (0) {

 "ShowColors Application. Copyright ) 1989 Chris Derossi"
};

resource 'WIND' (128, "Colors Window") {
 {40, 10, 200, 170},
 documentProc,
 invisible,
 noGoAway,
 0x0,
 "Colors Window"
};

resource 'DLOG' (128, purgeable) {
 {40, 40, 200, 340},
 altDBoxProc,
 invisible,
 noGoAway,
 0x0,
 128,
 "About ShowColorsI Dialog"
};

resource 'DITL' (128, purgeable) {
 { /* array DITLarray: 4 elements */
 /* [1] */
 {0, 0, 160, 300},
 UserItem {
 enabled
 },
 /* [2] */
 {17, 4, 37, 294},
 StaticText {
 disabled,
 "ShowColors Color-Smart Sample Application"
 },
 /* [3] */
 {113, 24, 132, 277},
 StaticText {
 disabled,
 "by Chris Derossi for Dr. DobbUs Journal"
 },
 /* [4] */
 {58, 129, 90, 161},
 Icon {
 disabled,
 128
 }
 }
};

resource 'ALRT' (129) {
 {36, 52, 114, 320},
 129,
 { /* array: 4 elements */
 /* [1] */
 OK, visible, sound1,
 /* [2] */
 OK, visible, sound1,
 /* [3] */

 OK, visible, sound1,
 /* [4] */
 OK, visible, sound1
 }
};

resource 'DITL' (129) {
 { /* array DITLarray: 2 elements */
 /* [1] */
 {50, 201, 70, 261},
 Button {
 enabled,
 "Okay"
 },
 /* [2] */
 {8, 87, 44, 261},
 StaticText {
 enabled,
 "ShowColors requires Color QuickDraw to be present."
 }
 }
};

resource 'MENU' (128, "Apple", preload) {
 128, textMenuProc,
 AllItems & ~MenuItem2, /* Disable item #2 */
 enabled, apple,
 {
 "About ShowColorsI",
 noicon, nokey, nomark, plain;
 "-",
 noicon, nokey, nomark, plain
 }
};

resource 'MENU' (129, "File", preload) {
 129, textMenuProc,
 AllItems,
 enabled, "File",
 {
 "Quit",
 noicon, "Q", nomark, plain
 }
};

resource 'MENU' (130, "Edit", preload) {
 130, textMenuProc,
 AllItems & ~(MenuItem2), /* Disable item #2 */
 enabled, "Edit",
 {
 "Undo",
 noicon, "Z", nomark, plain;
 "-",
 noicon, nokey, nomark, plain;
 "Cut",
 noicon, "X", nomark, plain;
 "Copy",
 noicon, "C", nomark, plain;
 "Paste",

 noicon, "V", nomark, plain;
 "Clear",
 noicon, nokey, nomark, plain
 }
};

resource 'ICON' (128) {
 $"00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00"
 $"00 00 00 00 00 00 00 00 00 00 00 00 7F FF 00 00"
 $"80 00 80 00 80 00 80 00 FF FF 80 00 AA AA 80 00"
 $"D5 55 80 00 AA AA 80 00 D5 FF 80 00 AB 00 80 00"
 $"D5 FF 80 00 AB 00 80 00 D5 00 BF FE AB 00 C0 6B"
 $"D5 00 C0 55 AB 00 C0 6B D5 00 C0 55 AB 00 C0 6B"
 $"D5 00 C0 55 AB 00 C0 6B D5 00 C0 55 AB 00 C0 6B"
 $"D5 FF FF D5 AA AA EA AB D5 55 D5 55 7F FF 3F FE"
};

resource 'ICN#' (128) {
 { /* array: 2 elements */
 /* [1] */
 $"00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00"
 $"00 00 00 00 00 00 00 00 00 00 00 00 7F FF 00 00"
 $"80 00 80 00 80 00 80 00 FF FF 80 00 AA AA 80 00"
 $"D5 55 80 00 AA AA 80 00 D5 FF 80 00 AB 00 80 00"
 $"D5 FF 80 00 AB 00 80 00 D5 00 BF FE AB 00 C0 6B"
 $"D5 00 C0 55 AB 00 C0 6B D5 00 C0 55 AB 00 C0 6B"
 $"D5 00 C0 55 AB 00 C0 6B D5 00 C0 55 AB 00 C0 6B"
 $"D5 FF FF D5 AA AA EA AB D5 55 D5 55 7F FF 3F FE",
 /* [2] */
 $"00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00"
 $"00 00 00 00 00 00 00 00 00 00 00 00 7F FF 00 00"
 $"FF FF 80 00 FF FF 80 00 FF FF 80 00 FF FF 80 00"
 $"FF FF 80 00 FF FF 80 00 FF FF 80 00 FF FF 80 00"
 $"FF FF 80 00 FF FF 80 00 FF FF BF FE FF FF FF FF"
 $"FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF"
 $"FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF"
 $"FF FF FF FF FF FF FF FF FF FF FF FF 7F FF 3F FE"
 }
};

resource 'BNDL' (128) {
 'ShCo',
 0,
 {
 'ICN#', {0, 128},
 'FREF', {0, 128}
 }
};

resource 'FREF' (128) {
 'APPL',
 0,
 ""
};

resource 'SIZE' (-1, purgeable) {
 dontSaveScreen,
 ignoreSuspendResumeEvents,
 enableOptionSwitch,

 cannotBackground,
 notMultiFinderAware,
 backgroundAndForeground,
 dontGetFrontClicks,
 ignoreChildDiedEvents,
 not32BitCompatible,
 reserved,
 reserved,
 reserved,
 reserved,
 reserved,
 reserved,
 reserved,
 50 * 1024,
 50 * 1024
};














































Special Issue, 1989
AVOIDING INIT COLLISIONS AT BOOT TIME


Loading device drivers with an INIT isn't the only way, but it's the best




John Rosford


John Rosford is an engineer at National Instruments and can be contacted at
12109 Technology Blvd., Austin, TX 78727-6204.


This article describes a Macintosh INIT that loads a complex set of device
drivers for a group of IEEE-488 interface and data acquisition cards. Even
though drivers vary widely in purpose, the code that loads them at boot time
is much the same, so this basic INIT will be able to load most driver
configurations. One of the main goals of this INIT code is to prevent
collisions with drivers previously loaded by other INITs, so this program will
be useful to Mac programmers who have drivers in use, as well as those writing
new ones.
The source code for the INIT program consists of initOpenDRVR.c ( Listing One,
page 62) and initOpenDRVR.h (Listing Two, page 68). Source code for other
resources can be found in DDJInit.r (Listing Three, page 70). The tools I used
include MPW 3.0 (to compile and copy the resources to the INIT) and Think C
3.0.


Loading Device Drivers


There are several methods for making a device driver available to an
application, the most common being to install the driver into the system file.
Several standard drivers are used by the Macintosh and are found in the System
file.
These drivers are installed when you upgrade or initially install the system
software by running the Installer.
Another method is to write an INIT to load the drivers at boot time, as
suggested by the "Macintosh Technical Note #14." All INITs in the System
folder are run at boot time by the system startup code. A third option, rarely
used, is to make the application load its own copy of the drivers. This last
option presents a difficult problem of hardware resource access control when
more than one application/driver is used under MultiFinder.


Device Drivers, Expansion Cards, and the INIT


In this section, I'll describe a set of drivers and their association to
subdrivers and the expansion cards that they drive. In doing so, I'll provide
an overview of the tasks performed by the INIT program.
Before an application can use either the NB-GPIB or NB-DMA-8G expansion
boards, the drivers must be initialized. These drivers share data because the
NB-GPIB and other data acquisition boards request DMA services from the DMA
driver.
The master driver, .LabDriver, is opened first (see Figure 1 and Listing
Three). It opens each board driver for the boards found in the computer, and
points them to the master global storage area, which they fill in with
information about the board. The board driver .NB-GPI does nothing more than
fill in information about the NB-GPIB board; the driver is never called by an
application and it does not manipulate the hardware. The .NB-DMA driver not
only fills in information about the NB-DMA-8G board, it also provides DMA
services for the NBHardwareCode driver and other data acquisition drivers.
Next, the INIT opens as many GPIB bus drivers as needed -- in this case, if
auto-configuration is set. The bus driver will be .gpib0 for the GPIB hardware
in the lower slot and .gpib1 will be for the GPIB in the higher slot.
Each bus driver opens the shared code and the hardware-variant code. The bus
driver opens hardware variant code that matches the hardware type for the slot
associated with the bus driver. These open calls link the bus driver's read,
write, and control calls to the hardware drivers.
One of the drivers is a hardware variant for the serial ports. This driver
can't be automatically configured at boot time, because there is no way to
find out what kind of device is attached without disturbing the serial port.
If you write commands out the port to see if it is a serial device that the
INIT has a driver for, you will get undefined results if the port is connected
to some other device.
Because the INIT sysz resource has requested allocation of a certain amount of
additional system heap memory, the drivers in the INIT should allocate during
the execution of the INIT all the memory that they will need. Memory not
allocated by the drivers can be taken by subsequent INITs, and thus will not
be available when an application executes a call to one of the drivers. There
is no recommended way to expand the system heap at run time, so you are stuck
with an inoperative driver until something, like a desk accessory or driver,
releases enough system memory, usually by being closed.
The NB-DMA-8G is a multifunction board. It has two distinct hardware sections,
the GPIB section and the DMA section. Making a driver for each section is the
best approach. The NBHardwareCode driver handles the GPIB section, and the
NB-DMA driver handles the DMA section. This approach minimizes the duplication
of code for this set of drivers. Any driver for the GPIB or data acquisition
can use DMA services by calling the DMA driver instead of manipulating the DMA
section of the NB-DMA-8G directly.
The NBHardwareCode is a multiboard driver. If the bus is associated with the
NB-DMA-8G, the drivers, as just described, are used. The NBHardwareCode
handles both the NB-GPIB board and the GPIB section of the NB-DMA-8G board;
the hardware is virtually identical.
The INIT code performs a number of simple steps to load a set of drivers.
Those steps are:
1. Initialize the Macintosh Toolbox Managers. Because most of the Toolbox
Managers should not be initialized until needed, you only need to initialize
and open the graph port (for plotting the icon) to show that the INIT is
running. This initialization makes the current grafPort the size of the
screen.
2. Show the icon in the bottom lower-left corner of the screen. The icon will
remain on the screen until after all INITs have run or InitWindows is called.
3. Check for a user abort. If Command Period is pressed, then give the option
of aborting the device driver installation. This should be standard practice
for INITs. (If you hold down the Command Period keys to prevent the INIT from
loading drivers and the Command key is still pressed when MultiFinder starts
to execute, you will also prevent MultiFinder from launching.)
4. Call SysEnvirons and check that the INIT is running on the right machine.
Some INITs should abort if running on the wrong type of machine, or if the
machine is missing needed resources.
5. Get the driver information tables from resources in the INIT file. There
are two tables of information: A driver table and an owned resource table.
Information in these tables includes the driver resource ID, which will change
if any resource ID collision occurs; the load status; the driver type, which
determines under what circumstances the driver is to be loaded; and the driver
name. The load status is determined by the INIT from the driver type and the
type of boards found in the computer. These tables allow the INIT to calculate
the number of drivers and give information about each driver.
6. Get the configuration data from the INIT file. This data allows the user to
customize the operating characteristics of the device drivers or the INIT.
7. Mark a driver for loading if the slot is for serial hardware; or if the
slot is for NuBus and one of our boards is in the slot; or if the driver is
called by a driver that is marked for loading.
8. Alert the user if the device drivers are already loaded. The user may have
two different revisions of the drivers in the system. To detect a duplicate
set of drivers, point the Resource Manager away from the INIT so it won't see
these device drivers. Then try to open the drivers that have been marked for
loading. Only these drivers would be previously loaded. Having two sets of
drivers in memory would be a waste of memory and driver IDs (a limited
resource).
9. Check that there is enough memory. Now that the drivers have been marked
for loading, the INIT must determine whether they will fit in memory. Read
each resource into memory. If a memory error occurs, abort the INIT process.
10. Make sure that each driver ID is unique within the system. There are two
places to check: The driver unit table, which has a device control entry (DCE)
for each open driver, and the System file. We are interested in those drivers
loaded by INITs that have run previous to our INIT, and drivers in the System
file, which may be opened later. Because all INITs test for collisions with
drivers loaded by previous INITs, there is no chance of a driver collision
with subsequently loaded drivers. Drivers loaded by applications must take the
same precautions as do those loaded by INITs. If a driver were loaded with an
ID that collides with that of an open driver, the new driver would replace the
open driver. If its ID were to collide with that of an unopened driver in the
System file, then the system driver would replace our driver when the system
driver was opened.
Initially, there are only 64 IDs available, but Inside Macintosh says that the
table will automatically expand to 128. Because some of the IDs are reserved
for the system, it is not clear whether the table expands when all user IDs
are used, or when all 64 IDs are used. If it is the former case, then which
IDs are user IDs? Do they include the desk accessory IDs? If a driver's ID is
changed, then the ID of each of its owned resources must also be changed.
11. Open the drivers. In this example, only the master driver and the bus
drivers are opened. These drivers are loaded only if needed as determined by
markLoad. All other drivers are subdrivers that are opened by the master
driver, bus drivers, or another subdriver. Opening a driver creates a DCE in
the driver unit table.
12. Detach the drivers. When an application quits, all of its resources are
removed from memory. So that the driver resources will not be removed, they
are detached from the INITs list of resources. Detaching the driver resources
affects only the memory copy of the resources -- it does not remove the
drivers from the INIT file. Because the driver resources are detached, the
memory copy of the driver resources won't be removed when the INIT file
closes. As long as the drivers remain open, their DCEs will remain in the unit
table, allowing your application to open and use the drivers without also
opening the INIT file. Configuration data can be passed to the drivers in the
open call.


Important Resources



Resources in the file DDJInit.r (Listing Three) are parts of the INIT file
that may be conveniently described with the MPW resource language and compiled
with the MPW Rez resource compiler. Resources, such as STR#, are standard to
the Macintosh OS. As such, there are ROM calls that can make use of these
resources directly. Other resources, such as dTbl, were designed specifically
for the INIT. Typically, resources for a specific application have a
corresponding C structure or Pascal record to identify each part of the data
stored in the resource. The structure for the dTbl makes it clear that the
data is a list of driver information: The driver ID, load status, type, and
name. Figure 2 shows which parts of the operating system and the INIT need
which resources. The Finder resources BNDL, FREF, DDJI, and ICN# are discussed
in detail in Inside Macintosh, Volume III and vers in "Tech Note #189."
The important resources include three that are user defined: dTbl, oTbl, and
busD. The type dTbl is a table of information about the drivers. There are
four elements in each entry of the table: The driver ID, the driver load
status, the load attribute, and the driver or board name. The type oTbl is a
table of information about resources owned by one of the drivers in the driver
table. There are three elements in each entry of the table: The type of owned
resource, the sub-ID of the owned resource, and the ID of the resource's
owner. In these two tables, you enter information for each driver that the
INIT must load. The type busD is a table of user configuration data. The
contents of this resource depends on the configuration data needed by the
drivers. I have included some example configuration data used by the bus
drivers.
The error messages are in a standard resource of strings, type STR#. This
allows the INIT to be localized for other countries, and it allows me to use
the standard system call GetIndString to read a string from the resource.


Compiling the Source Code and Resource Files


With all the source code files and resource files completed we can compile
them to create the various pieces of the INIT. First, we create the INIT
resource, which is the INIT's executable code run by the system's INIT runner,
INIT 31, at boot and restart times. Next, we append the resources created by
MPW Rez.
Rez is used because it identifies many system resource formats. You can define
your own resource types, and you can decompile resources with Derez that you
create or modify in ResEdit.
The Think C project should contain all source files and libraries needed.
(Read Think C's warnings about using library calls that need global storage.
If you are using library calls that need globals, then recompile the library
as described in the Think C manual.) From the Project menu, choose the menu
item Set Project Type ... Click the Code Resource radio button onto the File
Type to ????, and the Creator to ????. (See Figure 3.) The file type and
creator will be changed to their correct values after the INIT has been
completely built, because, by itself, this is not a working INIT file, so we
don't want the system to identify it as such. We don't want a custom header;
we will use Think C's standard header for code resources. Enter a name such as
DDJ Init (this is optional because INITs are not referred to by name). Set the
resource type to INIT, ID to 0, and resource attributes to Locked. Now choose
options from the Edit menu and select any options you want. I suggest MacsBug
Symbols for Code Generation and Check Pointer Types, and Require Prototypes
for Compiler flags.
Now create the INIT resource by choosing Build Code Resource ... from the
Project menu. Enter the file name of the INIT, such as *INIT.rsrc (I put an
asterisk as the first character of the name of any part of the INIT file for
easy identification). Move to the Build INIT folder and save. This save
command overwrites any existing file of that name.
Code resources, such as the INIT resource, can't be segmented with Think C
3.0, but with MPW C 3.0 you can make segments larger than 32K bytes. Unlike
code resources, Think C will let you create segmented drivers. If you create a
multisegment driver, then there will be one DRVR resource and, for each
segment, a DCOD resource. Each of these resources will have global data in an
owned DATA resource. If any owned DATA resource has zero size, you can delete
it. You must enter the information on all of these resources in the driver
table used by the INIT. Add them to the source code file compiled by MPW Rez.
The MPW Rez command compiles resources specified with the Rez language, which
is similar to the C language. The resource file, DDJInit.r, specifies various
Finder and INIT resources, not including the INIT and DRVR resources. Four
commands are used to build the final INIT file. The first two commands set
shell variables to the name of the build directory, where all the compiled
driver and INIT files are, and to the name of the INIT file to be created. The
Rez command specifies the output file name, the search directory for Include
statements, and the source file. On this line, you can define an identifier to
control conditional compilation. This is useful if you want different
resources for different computers. The last command sets the bundle bit, the
file type, and file creator. These are the commands used to compile this file;
set the file attributes, type, and creator; and copy the resources to the
INIT:
 Set BuildDir '::Build INIT:'
 Set InitName 'DDJ INIT'

 Rez -o "{BuildDir}{InitName}"
 -s "{BuildDir}" DDJInit.r
 Setfile "{BuildDir}{InitName}"
 -a B -t INIT -c DDJI
Near the top of the file are three Include statements that read the INIT and
DRVR resources into the INIT file. In this example, I put all my device
drivers in one file, *DDJ Driver, with ResEdit. The Include statements find
the driver file with the help of the search directory specified by the -s
option to the Rez command. The drivers used in this example must be locked in
the system heap. The first Include statement copies into the INIT file any
driver whose ID is in the range 0 - 64, changing the resource attributes of
each driver from Purgeable to System Heap and Locked, leaving the ID and name
as they were. There is no easy way to set these attributes with Think C. The
next statement copies all of the global data resources to the INIT file. Think
C sets the resource attributes to Locked. The last Include statement copies
the INIT resource to the INIT file. The attributes remain the same because you
set the resource attributes of the INIT resource to Locked in the Think C
project. Here are the Include statements:
 include "*DDJ Driver"'DRVR'(0:64)
 as'DRVR'($$ID, $$Name,
 SysHeap, Locked); include "*DDJ Driver"'DATA'
 ( - 15424:- 15200); as 'DATA'
 ($$ID, I'll, SysHeap, Purgeable);
 include "'INIT.rsrc";


Testing the INIT Program


As with any application, you should test the INIT code to make sure it
performs correctly. If an INIT bombs, then the user must abort the INIT before
it executes, or boot on another volume. Most INITs don't have a way for the
user to make the INIT abort execution.
Testing the INIT takes some planning because the actions performed by the INIT
depend on the user configuration data and the hardware configuration of the
computer when the INIT is executed by the system. The configuration is
determined by the expansion boards installed, the drivers required by the user
configuration data, the presence of drivers loaded by any previously run INIT
file, and the presence of drivers in the System file.
This INIT makes ID changes permanent, so when you are building the INIT, build
it in any folder other than the System Folder. The reason is that if an ID
collision occurs, both the driver and the INIT's driver and owner tables get
updated with the new driver ID. By keeping a master version of the current
INIT and copying it to the System Folder, you can be sure that the driver IDs
agree with the IDs in the INIT resources. Otherwise, if you recompile a driver
whose ID previously collided and add it to the INIT in the System Folder, the
resource tables will have a different ID than the driver.
The first test is to open all the drivers that the INIT can load. If your
application fails to open a driver, compile your drivers with MacsBug symbols.
After restarting the computer, break to the debugger and perform a symbol dump
of the system heap. If your driver's symbols don't appear, then any one of the
following may be true:
The INIT never marked the driver for loading.
The INIT never detached the driver resource.
The driver's System Heap attribute was not set. Accordingly, the driver was
loaded into the application heap, which was subsequently initialized.
The driver's Purgeable attribute was set, and the driver was purged before you
looked for the symbols.
The driver was successfully closed before you looked for the symbols.
If your driver's symbols do appear, then it may be that the driver was never
opened, either by the INIT or by another driver, before the driver resource
was detached by the INIT.
To test for duplicate drivers, simply copy the INIT file to the System folder,
select and duplicate it, and restart the computer. You will see the first INIT
complete normally, but the second driver will put up an alert to flag the
duplicate driver condition. You should perform this test for each separate
driver configuration.
The INIT should never report an "out of system heap memory error" because the
size resource in the INIT file requests enough additional memory from the
system to satisfy the highest memory requirement of the set of drivers.
However, an error will occur if the system does not have enough memory to
satisfy the request, or if you set the value of the size resource too low.
To test a memory error if your drivers use more than about 16K bytes of
memory, then set the size resource to zero, make sure enough drivers will be
loaded, and restart the computer. Because the system normally has no more than
16K bytes of free space in the system heap, the INIT should report the error
to cause a memory error under normal condition.
If your drivers are too small, then you may decide that testing for an
out-of-memory condition is not required. You can still perform the test by
making your drivers larger than 16K bytes. To do this, open a text file that
is larger than 16K bytes, such as the initOpenDriver.c source file; select all
of it; and copy (to the Clipboard). With ResEdit, select one of the drivers
that will be marked for loading, and choose Open General from the File menu.
Scroll to the bottom, put the cursor after the last byte on the right (ASCII)
side of the window, and paste (from the Clipboard). Close the window and
choose Get Info. You will see that your driver now occupies more than 16K
bytes. Now quit ResEdit, saving the changes, and restart the computer. This
time you should get the "out of system heap memory" error.
If testing the INIT requires that the INIT be modified, restore it to its
original condition before starting a new phase of testing. Just copy the
original INIT to the system folder. Make changes only to this copy of the INIT
when using a resource editor.
A major portion of the INIT code deals with driver resource ID collisions.
Testing this feature will tell whether the ID of our conflicting driver was
successfully changed to an unused ID, and whether the other driver was not
altered or opened.
The INIT has some conditionally compiled code that you can enable by defining
the identifier DEBUG2. This code will put up an alert when a driver conflict
occurs and will show the new ID number.
To cause a collision with a driver in the System file, copy any driver from
the INIT that will be loaded, and paste it in the System file; and use ResEdit
to change the name of this copy of the driver to an unused name. Now this
driver will conflict with the driver in the INIT, so restart the computer,
step through the alerts, and make a note of the new ID that the driver will be
given. With ResEdit, check that the driver you put in the System file has not
been altered. Check its name, size, and attributes. (MPW's ResEqual compares
all the resources between two files, so it would not be useful in this case).
Now check the driver in the INIT file: Its ID should be the new ID, and the
owner ID of all its owned resources, listed in the owned table, should also be
the new ID.
To cause a collision with a driver loaded by a previous INIT, compile the INIT
code with the identifier DEBUG3, which will disable checking for duplicate
drivers. Copy the resulting INIT file to the System Folder and then duplicate
it in the System Folder so that two copies of the INIT will execute. Rename
the copy so that it runs before the other (the system runs INITs in
alphabetical order). Change the name of each driver so that your driver test
program will have to open the set of drivers that were given new IDs.
Restart the computer and make a note of all the new IDs. Check that the INIT
that executes second changes the IDs of all its drivers that are marked for
loading, because these drivers will exist in the driver unit table from the
INIT that runs first. You can use ResEdit to check that the driver and owned
resources IDs have been changed correctly. The new IDs must not conflict with
any of the drivers in the System file. Now run a test program to exercise your
drivers. Check the drivers in the INIT that runs first. There should be no
changes in the resource IDs unless there was a conflict with some other
drivers. The resource attributes should also be at their default values.
If loading a driver depends only on the user configuration data, such as a
driver for hardware attached to the serial port, make sure all the needed
drivers get loaded when no other hardware variants are in the computer. This
test would verify that a driver could be marked for loading by the user
configuration data only.


Conclusion



Loading device drivers with an INIT should be the preferred method over
installing them into the System file. With this method, one file can
accomplish three things for you. It can make installation and upgrading
easier; it can contain a Control Panel device that can perform user
configuration; and it can expand the system heap to make room for the drivers.


Acknowledgments


The author would like to thank Lynda Gruggett for her help with the debugging
and the design of the NB Handler INIT.


References


Apple Computer, Inc. Inside Macintosh, Addison-Wesley, Reading, Mass., 1987.
Apple Computer, Inc. Macintosh Programmer's Workshop Development Environment,
Vol. 1, Ver. 3.0, Cupertino, Calif., 1988.
The Institute of Electrical and Electronics Engineers. IEEE Standard Digital
Interface for Programmable Instrumentation, ANSI/IEEE Std 488-1978 (New York:
The Institute of Electrical and Electronics Engineers, 1983).
Symantec Corp. Think C User's Manual Bedford, Mass, 1988.

_AVOIDING INIT COLLISIONS AT BOOT TIME_
by John Rosford


[LISTING ONE]

/* initOpenDRVR.c
# John Rosford, National Instruments.
# Copyright 1988,1989 National Instruments Corporation
# All rights reserved.
# This Macintosh INIT loads device drivers, checking
 for driver ID collisions.*/

#include <MacTypes.h>
#include <Quickdraw.h>
#include <WindowMgr.h>
#include <EventMgr.h>
#include <DialogMgr.h>
#include <OSUtil.h>
#include <MemoryMgr.h>
#include <Pascal.h>
#include <DeviceMgr.h>
#include <ResourceMgr.h>
#include <ControlMgr.h>
#include <ToolboxUtil.h>
#include <OSUtil.h>
#include <Strings.h>
#include <SlotMgr.h>


#include "initOpenDRVR.h"

#define NBICONID 128

#define NUM_DRVRS SizeResource(GetResource( DRVR_TBL_TYPE,
DRVR_TBL_ID))/sizeof(struct drvrStruct)
#define NUM_OWNED SizeResource(GetResource( OWNED_TBL_TYPE,
OWNED_TBL_ID))/sizeof(struct ownedStruct)

#ifdef LSC
/* setup address register A4 for global data */
#define SetUpA4() asm{ move.l a4,-(sp) \
 move.l a0,a4 }
#define RestoreA4() asm{ move.l (sp)+,a4 }

#endif

main()
{
 struct drvrStruct *pDrvrTbl; /* ptr to drvrStruct */
 struct ownedStruct *pOwnedTbl; /* ptr to ownedStruct */
 register short err; /* error code */
 int used_id[NUM_IDS]; /* true if device driver id is in use */
 char nilstr[1];
 char strBuf[256];
 SysEnvRec theWorld;
 config_t config;
 GrafPort myPort;

#ifdef DEBUG4
 DebugStr("\pBreak at MAIN+0");
#endif
#ifdef GLOBALS
 SetUpA4();
#endif
 InitGraf( &thePort); /* initialize quickdraw global variables */
 OpenPort(&myPort); /* make myPort the current grafPort */
 /* Other toolbox managers are initialized when needed for dialogs. */

 nilstr[0]='\0'; /* make a nil string */
#ifdef DEBUG4
 alertNote("Number of owned resources: %d", (int)NUM_OWNED);
#endif
 err = noErr; /* start with no error */
 showIcon(); /* display the icon */
 if( kill_key()) /* True if User abort key combination */
 /* question: abort the init? */
 if( err = qAbort( text_PStr(S_Q_USER_ABORT, strBuf), nilstr, nilstr))
 alert( text(S_USER_ABORT, strBuf)); /* yes, abort */
 if( !err)
 SysEnvirons( 1, &theWorld); /* what environment is the init running in? */
 if( !err){
 if( err = (theWorld.machineType<envMacII &&
theWorld.machineType!=envMachUnknown)) /*envMacII*/
 alert( text(S_NEED_MAC_II, strBuf)); /* my drivers need a Mac II */
 }
 if( !err)
 getTables( &pDrvrTbl, &pOwnedTbl); /* Get the driver tables. */
 if( !err)
 err = configInfo(&config); /* Copy in user configuration from resources */
 if( !err)
 err = markLoad( pDrvrTbl,&config); /* mark drivers that need loading. */
#ifndef DEBUG3
 if( !err)
 err = dupDriverInstalled( pDrvrTbl);/* check for duplicate drivers */
#endif
 if( !err)
 err = checkSysHeapSize( pDrvrTbl, pOwnedTbl); /* is there enough memory?*/
 if( !err)
 err = findUsedIDs( used_id); /* find all IDs used in the system */
 if( !err)
 err = checkIDs( pDrvrTbl, pOwnedTbl, used_id); /* check the IDs of the INIT's
drivers */
 if( !err){
 if( err = openMarkedDrivers( pDrvrTbl,&config)) /* open drivers that need
loading */
 alert( text(S_FAIL_OPEN_DRVRS, strBuf), err);

 }
#ifdef DEBUG1
 {
 short i;
 for( i=0; !err && i<NUM_DRVRS; ++i)
 if( pDrvrTbl[i].load)
 alertNote("After openMarkedDrivers, Will load %s", pDrvrTbl[i].name);
 }
#endif
 if( !err)
 err = detachDriver( pDrvrTbl, pOwnedTbl); /* detach the driver resources */
 ClosePort(&myPort); /* close my grafPort */
#ifdef GLOBALS
 RestoreA4();
#endif
}

char *
text(index, strBuf) /* return a C string in strBuf */
 int index;
 char *strBuf;
{
 return( PtoCstr( text_PStr(index, strBuf)));
}

char *
text_PStr(index, strBuf) /* return a Pascal string in strBuf */
 int index;
 char *strBuf;
{
 GetIndString(strBuf, SS_MSGS_ID, index); /* get string from STR# resource */
 return(strBuf);
}


/****************************************************************************/
/* Add code to show your icon */
showIcon()
{}
/****************************************************************************/

getTables( hDrvrTbl, hOwnedTbl) /* Get the driver tables. */
struct drvrStruct **hDrvrTbl; /* handle to the driver table. */
struct ownedStruct **hOwnedTbl; /* handle to the owned resource table. */
{
 Handle hData;
 int err;
 char strBuf[128];

#ifdef DEBUG_TRACE
 alertNote("getTables()");
#endif
 hData = GetResource( DRVR_TBL_TYPE, DRVR_TBL_ID); /* get driver table */
 if( err = ResError()){
 alert(text(S_FAIL_GET_DRVR_TBL, strBuf));
 return( err);
 };
 HLock( hData); /* table needs to be locked while the INIT runs */
 *hDrvrTbl = (struct drvrStruct*) *hData; /* send pointer back to caller */


 hData = GetResource( OWNED_TBL_TYPE, OWNED_TBL_ID); /* get owned resource
table */
 if( err = ResError()){
 alert(text(S_FAIL_GET_OWNED_TBL, strBuf));
 return( err);
 };
 HLock( hData); /* table needs to be locked while the INIT runs */
 *hOwnedTbl = (struct ownedStruct*) *hData; /* send pointer back to caller */
 return err;
}

/* Copy in user configuration from resources */
configInfo(config)
register config_t *config; /* pointer to the configuration data */
{
 register short err;
 Handle DefBoardsH; /* default board data resource */
 register BusResource *BP; /* pointer to bus data */
 char TheString[256]; /* storage for message string */

#ifdef DEBUG_TRACE
 alertNote("configInfo()");
#endif
 slotInfoTable(config); /* fill in brdName and brdID for each NB slot */
 /* Load default configuration */
 DefBoardsH = GetResource(BUS_DATA_TYPE,BUS_DATA_ID); /* get the bus data
resource */
 if (DefBoardsH == NULL) {
 alert(text(S_FAIL_GET_DATA, TheString));
 return -1;
 }
 HLock(DefBoardsH); /* lock data resource until we read it into local
variables */
 BP = (BusResource*) *DefBoardsH;
 err = configBrds( BP,config); /* read bus configuration data */
 if( err)
 return(err); /* failed */
 HUnlock(DefBoardsH); /* unlock data resource */
 ReleaseResource(DefBoardsH); /* release the resource */

 return noErr;
}

configBrds( BP,config)
register BusResource *BP;
register config_t *config;
{
 register short bus,MaxBuses;
 char strBuf[256];

#ifdef DEBUG_TRACE
 alertNote("configBrds()");
#endif
 MaxBuses = config->MaxBuses = BP->Cnt; /* get number of buses we have data
for */
 if( BP->Rev != BOARDREV){
 alert(text(S_FAIL_REVISION, strBuf));
 return E_BRDREV;
 }
 for (bus=0; bus<MaxBuses; bus++) { /* for each bus, copy the data */
 config->brdRsrc.BusData[bus].b_uflags = BP->BusData[bus].b_uflags;
 config->brdRsrc.BusData[bus].b_slot = BP->BusData[bus].b_slot;

 /* if the bus is associated with a serial slot, copy the extra data */
 if( isSerSlot(config->brdRsrc.BusData[bus].b_slot)){
 config->brdRsrc.BusData[bus].b_baud = BP->BusData[bus].b_baud;
 }

 }

#ifdef DEBUG1
 for (bus=0; bus<MaxBuses; bus++) {
 if(config->brdRsrc.BusData[bus].b_slot){
 /* alert: 4 int args max */
 alertNote("configBrds: bus %d with slot %d", bus,
config->brdRsrc.BusData[bus].b_slot);
 alertNote("configBrds: uflags %x",
 BP->BusData[bus].b_uflags);
 }
 }
#endif
 return(noErr);
}

/* Mark the drivers and owned resources that will need to be loaded.
 * Look at the boards to determine which to load.
 * All drivers will be mark either YES or NO.
 * Always returns noErr.
 */

markLoad( drvrTbl,config)
struct drvrStruct *drvrTbl;
register config_t *config;
{
 register short i, k, err, bus; /* loop counters, error, and bus number */
 char strBuf[128]; /* storage for message strings */

#ifdef DEBUG_TRACE
 alertNote("markLoad()");
#endif
 err = noErr;
 for( i=0; i<NUM_DRVRS; ++i){ /* Check boards before anything else. */
 short brdID; /* board ID variable */
 brdID = drvrTbl[i].type; /* driver type can be a board ID */
 switch( brdID){ /* which board ID? */
 case NB_GPIB:
 case NB_DMA_8:
 drvrTbl[i].load = getSlot( brdID,config); /* mark if board is in any slot */
 break;
 case T_SERIAL_HW:
 /* Load if any bus is configured as a serial slot: GPIB-422CT, GPIB-MAC */
 for( k=0; k<config->MaxBuses; ++k){
 if( isSerSlot(config->brdRsrc.BusData[k].b_slot)){
 drvrTbl[i].load = true;
#ifdef DEBUG1
 alertNote("Load serial slot %d, bus %d",config->brdRsrc.BusData[k].b_slot,k);
#endif
 break;
 }
 }
 break;
 default:
 /* Take no action here if type is not a brdID. Types will be checked below.
*/

 break;
 }
 }

 /* Loading the master driver depends only on the boards but not the serial
driver. */
 i = ndxDrvrType( drvrTbl, T_DRVR_MASTER); /* get index of the master driver
*/
 drvrTbl[i].load = L_NO;
 for( k=0; k<NUM_DRVRS; ++k)
 /* mark the master driver if any other driver except the serial or NB-GPIB
driver is marked */
 if( drvrTbl[k].load && drvrTbl[k].type!=T_SERIAL_HW && drvrTbl[k].load !=
NB_GPIB){
 drvrTbl[i].load = L_YES;
 break;
 }

 bus = 0;
 for( i=0; i<NUM_DRVRS; ++i){
 switch( drvrTbl[i].type){ /* scan through the other driver types */
 case T_SERIAL_HW: /* ignore serial, board, and master types */
 case NB_GPIB:
 case NB_DMA_8:
 case T_DRVR_MASTER:
 break;
 case T_DRVR_GPIB: /* load GPIB type if any GPIB board or serial driver is
marked */
 drvrTbl[i].load = ((drvrTbl[ndxDrvrType(drvrTbl,NB_DMA_8)].load
 drvrTbl[ndxDrvrType(drvrTbl,NB_GPIB)].load
 drvrTbl[ndxDrvrType(drvrTbl,T_SERIAL_HW)].load) ? L_YES : L_NO);
 break;
 case T_BUS_DRVR: /* Load GPIB bus driver if user or auto configured. */
 drvrTbl[i].load = config->brdRsrc.BusData[bus].b_slot;
 bus++;
 break;
 case T_BOARD_HW: /* Load if any NI-488 resource: NB-GPIB, NB-DMA */
 /* if any GPIB board is present, then bus 0, the first bus, will have a slot.
*/
 drvrTbl[i].load = autoSlot(0,config);
 break;
 default:
 /* error if there is an unknown type in the table */
 alertNote(text(S_NOT_DRVR_TYPE, strBuf), drvrTbl[i].type, i);
 err = 1;
 break;
 }
 }

#ifdef DEBUG1
 alertNote("Show drivers marked for loading");
 for( i=0; i<NUM_DRVRS; ++i)
 if( drvrTbl[i].load) /* all are YES or NO at this point */
 alertNote("Marked %s", drvrTbl[i].name);

#endif
 return err;
}

/* Input: driver type. Output: its drvrTbl index. */
ndxDrvrType( drvrTbl, type)
struct drvrStruct *drvrTbl;
short type;
{
 register unsigned short i;


 for( i=0; i<NUM_DRVRS; ++i)
 if( type == drvrTbl[i].type )
 return i;
 return 0;
}

/* Input: driver id. Output: its drvrTbl index. */
drvrID2index( drvrTbl, id)
struct drvrStruct *drvrTbl;
register short id;
{
 register short i;
 char strBuf[128];

 for( i=0; i<NUM_DRVRS; ++i)
 if( drvrTbl[i].id == id)
 return i;
 alert(text(S_FAIL_NO_DRVR, strBuf), id);
 return 0;
}

/* Slot number range 1-n while bus numbers range 0-m */
/* return slot if board whose id=brdID is found, else 0 */
getSlot( brdID, config)
register unsigned short brdID;
register config_t *config;
{
 register unsigned short i;

 for (i=1; i<=NSLOTS; i++){
 if( brdID == config->slotInfo[i-1].brdID )
 return i;
 }
#ifdef DEBUG4
 alertNote( "No slot for brdID: 0x%x", brdID);
#endif
 return 0;
}


/* return the slot of the (bus)th GPIB board in the machine. */
autoSlot( bus,config)
int bus;
register config_t *config;
{
 register unsigned short i, brdID;

 if( 0<=bus && bus<NBUSES){
 for (i=1; i<=NSLOTS; i++){
 brdID = config->slotInfo[i-1].brdID;
 if(brdID==NB_GPIB brdID==NB_DMA_8)
 if( !bus--) /* example: bus 1 will return slot of first match */
 return i;
 }
 }
#ifdef DEBUG4
 else
 alertNote("Bad parameter: autoSlot(%d)", bus);

#endif
 return 0;
}

/* Get NuBus slot info for each slot */
slotInfoTable(config)
register config_t *config;
{
 register int i;

 for (i=1; i<=NSLOTS; i++) {
 config->slotInfo[i-1].brdID = GetSlotInfo(i, config->slotInfo[i-1].brdName);
 }
#ifdef DEMO
 config->slotInfo[0].brdID = NB_DMA_8;
#endif

 /* serial slots are a different hardware variant */
 config->slotInfo[SSLOTA_NDX].brdID = GPIBMAC;
 config->slotInfo[SSLOTB_NDX].brdID = GPIBMAC;
}


/*
 * Return boardID and driver name of board in slot 'slot'.
 * returns one of the board IDs listed in brdIDs.h
 */

/* spIDs */
#define sRsrc_Name 2
#define BoardID 32
#define VendorInfo 36
#define VendorID 1

GetSlotInfo(slot, dName)
int slot;
char *dName;
{
 SpBlock sBlock;
 SpBlockPtr sp;
 int brdID;
 OSErr err;
 Ptr structZero;
 struct SPRAMRecord {
 short boardID;
 char VendorUse[6];
 }SPRAMRec;

 dName[0] = '\0';
 sp = &sBlock;
#ifdef DEBUG
 printf("\n###### slot #%d #######\n", slot);
#endif
 sp->spSlot = macSlotNum(slot); /* convert slot to NuBus slot number */
 sp->spResult = (long)&SPRAMRec;
 err = SReadPRAMRec(sp);
#ifdef DEBUG
 printf("SReadPRAMRec err: %d, boardID = 0x%x\n", err, SPRAMRec.boardID);
#endif


 if( err == smSlotOOBErr) /* slot does not exist, number is out of bounds. */
 brdID = SLOT_OUT_OF_BOUNDS;
 else if( err) /* unknown error, no board. */
 brdID = NO_BOARD;
 else
 brdID = SPRAMRec.boardID;

 if( err == noErr){

 sp->spSlot = macSlotNum(slot); /* convert slot to Mac slot number */
 sp->spID = 1;
 sp->spExtDev = 0;
 err = SRsrcInfo(sp);

 if( err == smNoMoresRsrcs) /* either no board or slot does not exist */
 brdID = NO_BOARD;

 if( err == noErr)
 structZero = sp->spsPointer; /* can be changed by other calls */
 }

 if ( err == noErr) {
 /* Get board name */
 sp->spsPointer = structZero; /* restore spsPointer from SNextsRsrc() */
 sp->spID = sRsrc_Name;
 if ((err=SGetcString(sp)) == noErr) {
 strcat(dName, (Ptr)sp->spResult);
 }else
 brdID = NOT_OUR_BOARD;
 }

 if ( err == noErr) {
#ifdef DEBUG
 printf("spSlot %d, spID %d, spExtDev %d\n",
sp->spSlot,sp->spID,sp->spExtDev);
#endif
 /* Track down vendor name */
 /* spsPointer from SNextsRsrc() */
 sp->spID = VendorInfo;
 if ((err=SFindStruct(sp)) == noErr) {
 /* spsPointer from SFindStruct() */
 sp->spID = VendorID; /* vendor name */
 if ((err=SGetcString(sp)) == noErr){
 long ourName[6];
 ourName[0]='Nati';
 ourName[1]='onal';
 ourName[2]=' Ins';
 ourName[3]='trum';
 ourName[4]='ents';
 ourName[5]=0L;
 if (err=strcmp((Ptr)sp->spResult, (Ptr)ourName))
 brdID = NOT_OUR_BOARD;
 }else
 brdID = NOT_OUR_BOARD;
 }else
 brdID = NOT_OUR_BOARD;
 }

 return brdID;

}

/* initOpenDRVRp2.c - DDJ part 2 */

/* Check for any existing NI Drivers. */
dupDriverInstalled(drvrTbl)
struct drvrStruct *drvrTbl; /* pointer to the driver table */
{
 short myRefNum, refNum; /* INIT's and driver's reference numbers */
 register unsigned short i; /* loop counter */
 register short openError, numDrivers, err; /* openDriver error, number of
drivers, other error */
 char strBuf[64]; /* storage for message string */

 err = noErr;
 /* if no drivers are marked for loading
 # then the flag should be anything besides noErr.
 */
 openError = !noErr;
 myRefNum = CurResFile();
 numDrivers = NUM_DRVRS; /* macro would return 0 if used after UseResFile(0)
*/
 UseResFile(0); /* Use the System File only */
 for( i=0; i<numDrivers && openError != noErr; ++i) {
 if( drvrTbl[i].load){ /* see if this driver already exists */
 openError = OpenDriver( CtoPstr( strcpy( strBuf, drvrTbl[i].name)), &refNum);
 }
 }
 UseResFile(myRefNum); /* go back to the INIT resource file so we can display
an alert */
 if( openError == noErr){
 alert( text(S_FAIL_REMOVE, drvrTbl[i].name));
 err = 1; /* In this case, no error is a failure. */
 }
#ifdef DEBUG4
 else
 alertNote("No duplicate master driver found, %d.", openError);
#endif
 return err;
}

/* see if there is enough memory to load the drivers */
checkSysHeapSize( drvrTbl, ownedTbl)
struct drvrStruct *drvrTbl; /* array of drvrStruct */
struct ownedStruct *ownedTbl; /* array of ownedStruct */
{
 register short i; /* loop counter */
 register OSErr err; /* OS error */
 register Handle hdl; /* resource handle */
 char strBuf[128]; /* storage for message string */

#ifdef DEBUG_TRACE
 alertNote("checkSysHeapSize()");
#endif
 /* Find out how much memory is needed to load the drivers */
 err = noErr;

 /* while not err, size DRVRs that need to be loaded */
 for( i=0; i<NUM_DRVRS && !err; ++i){
 if( !drvrTbl[i].load)
 continue;
 hdl = GetResource(DRVR_TYPE, drvrTbl[i].id); /* read resource into memory */

 err = ResError();
 if( !hdl err){
 alert(text(S_FAIL_GET_DRVR, strBuf), drvrTbl[i].name);
 if( !err)
 err = true;
 }
 }

 /* while not err, size owned resources that need to be loaded. */
 for( i=0; i<NUM_OWNED && !err; ++i){
 register short id;
 if( !drvrTbl[drvrID2index( drvrTbl, ownedTbl[i].owner)].load) /* fixed bug:
5/13/88 */
 continue;
 id = OWNED_ID(ownedTbl[i].owner,ownedTbl[i].id); /* calc owned id */
 hdl = GetResource(ownedTbl[i].type, id); /* read resource into memory */
 err = ResError();
 if( !hdl err){
 alert(text(S_FAIL_GET_OWNED, strBuf), drvrTbl[drvrID2index( drvrTbl,
ownedTbl[i].owner)].name);
 alertNote(text(S_FAIL_ID_OSERR, strBuf), ownedTbl[i].id, err);
 if( !err)
 err = true;
 }
 }

 if( err == -108) /* if out of system heap */
 alert(text(S_FAIL_HEAP, strBuf));
 else if( err)
 alert(text(S_FAIL_RSRC_MGR, strBuf), err);
 return err;
}

findUsedIDs( used_id)
int used_id[]; /* true if device driver id is in use */
{
 register short UNtryCnt, i, err;

#ifdef DEBUG_TRACE
 alertNote("findUsedIDs()");
#endif
 UNtryCnt = UnitNtryCnt; /* get number of entries in Unit Table */
#ifdef DEBUG2
 alertNote("Number of Unit Entries: %d.", UNtryCnt);
#endif
 for( i=START_OF_USER_IDS; i<UNtryCnt; ++i)
 used_id[i] = 0; /* clear used id's */
 /* check both the Unit Table and the system file */
 err = checkUnitTable( used_id); /* Any test after this should OR in the
results. */
 if( !err)
 err = checkSysRsrcs( used_id);
 return err;
}

/* check if a driver of the same id already exists in the driver Unit Table.*/
checkUnitTable( used_id)
int used_id[]; /* true if device driver id is in use */
{
 register DCtlHandle *UTable;
 register short UNtryCnt, i;


#ifdef DEBUG_TRACE
 alertNote("checkUnitTable()");
#endif
 UTable = UTableBase; /* get base address of Unit Table */
 UNtryCnt = UnitNtryCnt; /* get number of entries in Unit Table */
 /* Mark if this DCtl handle is not NULL */
 for( i=START_OF_USER_IDS; i<UNtryCnt; ++i){
 used_id[i] = UTable[i] != NULL;
#ifdef DEBUG2
 if( used_id[i])
 alertNote("UnitTable ID %d found.", i);
#endif
 }
 return noErr;
}

/* check if a resource of the same type & id already exists in the system
file.*/
checkSysRsrcs( used_id)
int used_id[]; /* true if device driver id is in use */
{
 register short i, count, UNtryCnt; /* loop count, number of resources, size
of Unit Table */
 short id, err; /* resource id, error */
 ResType type; /* res type */
 char name[32]; /* res name */
 register Handle hdl; /* res handle */
 char strBuf[128]; /* string buffer */

#ifdef DEBUG_TRACE
 alertNote("checkSysRsrcs()");
#endif
 UNtryCnt = UnitNtryCnt; /* get number of entries in Unit Table */
 /* UseResFile() has no effect on GetIndResource() */
 SetResLoad(FALSE);
 err = ResError();
 if( !err){
 count = CountResources('DRVR');
 err = ResError();
 }

 for (i = 1; i <= count && !err; i++) {
 hdl = GetIndResource('DRVR', i);
 err = ResError();

 if( hdl && !err){
 GetResInfo( hdl, &id, &type, name);
 err = ResError();
 /* Mark (with prev result) if this resource is from the system file. */
 if( id<UNtryCnt && id>=0 && !err){
 used_id[id] = HomeResFile( hdl) == 0;
 err = ResError();
#ifdef DEBUG2
 if (used_id[id])
 alertNote("Marked Driver ID %d used.", id);
#endif
 }
#ifdef DEBUG2
 else
 alertNote("Driver ID %d out of range.", id);
#endif

 }else
 alert(text(S_FAIL_RSRC_MGR, strBuf), err);
 }
 if( !err){
 SetResLoad(TRUE);
 err = ResError();
 }
 return err;
}

checkIDs( drvrTbl, ownedTbl, used_id)
struct drvrStruct *drvrTbl; /* array of drvrStruct */
struct ownedStruct *ownedTbl; /* array of ownedStruct */
int *used_id;
{
 register short i, err, k;
 register struct drvrStruct *drvr;
 Handle hdl;
 char strBuf[128];
 int init_id[NUM_IDS]; /* true if init device driver id */
 short UNtryCnt;

/* Check all drivers. If marked for loading, it must have a unique and unused
ID within the system.
 * Else, it must have a unique ID within the INIT.
 * check if a DRVR of the same id already exists.
 * Change the DRVR's ID and its owned rsrc's ID's.
 * Change table resources.
 */

#ifdef DEBUG_TRACE
 alertNote("Checking driver ID's.");
#endif
 err = noErr;
 /* table of driver ids used by INIT */
 UNtryCnt = UnitNtryCnt;
 for( i=START_OF_USER_IDS; i<UNtryCnt; ++i)
 init_id[i] = 0; /* clear init id's */
 for( i=0; i<NUM_DRVRS; ++i)
 init_id[drvrTbl[i].id] = true;

 for( i=0; i<NUM_DRVRS && !err; ++i){
 register short newID;
 if( !drvrTbl[i].load)
 continue;
 drvr = &drvrTbl[i];
 if( used_id[drvr->id]) {
#ifdef DEBUG2
 alertNote("Driver %d conflict.", drvrTbl[i].id);
#endif
 /* find a resource ID that doesn't conflict with system or init driver IDs.
*/
 newID = getUnusedID( used_id,init_id);

#ifdef DEBUG2
 alertNote("NewID %d.", newID);
#endif
 if( newID){
 hdl = Get1Resource( DRVR_TYPE, drvr->id);
 if ((hdl == NULL) (err = ResError())) {
 alert(text(S_FAIL_ID_CHANGE, strBuf), drvr->id, err);

 }else{
 used_id[newID] = TRUE;
 err = changeRsrcID( hdl, newID); /* change the resource's ID */
 }
 }
 if( !err)
 err = newOwner( ownedTbl, drvr->id, newID); /* update owner of rsrcs owned by
drvr */
 if( !err){
 drvr->id = newID; /* change data in the tabel rsrc */
 err = changeTableResources( DRVRTBL);
 }
 }
 }
 return err;
}

/* Return an unused DRVR_TYPE id (0 if none) from the previously built used_id
table. */
getUnusedID( used_id,init_id)
int *used_id,*init_id; /* true if device driver id is in use, true if init id
is in use. */
{
 register short id, UNtryCnt;
 char strBuf[128];

#ifdef DEBUG_TRACE
 alertNote("getUnusedID()");
#endif
 /* Find a resource ID that doesn't conflict. */

 UNtryCnt = UnitNtryCnt; /* Number of unit entries. */
 for (id=START_OF_USER_IDS;id<UNtryCnt && (used_id[id] init_id[id]);id++) /*
5/31/88 used_id[id] JR/LG */
 ;
 if (id == UNtryCnt) {
 alert(text(S_FAIL_NO_FREE_ID, strBuf));
 id = 0; /* failed */
 }
 return id;
}


/* update owner ID of rsrcs owned by drvr */
newOwner( ownedTbl, drvrOldID, drvrNewID)
struct ownedStruct *ownedTbl; /* array of ownedStruct */
short drvrOldID, drvrNewID;
{
 register short i, err, old_id;
 register struct ownedStruct *owned;
 register Handle hdl;
 char strBuf[128];

#ifdef DEBUG_TRACE
 alertNote("newOwner()");
#endif
 err = noErr;
 for( i=0; i<NUM_OWNED && !err; ++i){
 owned = &ownedTbl[i];
 if( owned->owner == drvrOldID){
 old_id = OWNED_ID(drvrOldID,owned->id); /* calc owned id */
#ifdef DEBUG2
 alertNote("Get1Resource( id = %d).", old_id);

#endif
 hdl = Get1Resource(owned->type, old_id);
 if (err = ResError()) {
 alert(text(S_FAIL_ID_CHANGE, strBuf), old_id, err);
 return(err);
 }
 err = changeRsrcID( hdl, OWNED_ID(drvrNewID,owned->id)); /* Change the
resource's owner's ID */
 if( !err){
 owned->owner = drvrNewID; /* Change table rsrc of the owner. */
 err = changeTableResources( OWNERTBL);
 }
 }
 }
 return err;
}


/* If resource is owned, then pass in the real ID, not the sub ID. */
changeRsrcID( rsrcHdl, drvrNewID)
Handle rsrcHdl;
short drvrNewID;
{
 short attr;
 register OSErr err;
 ResType type;
 char name[30];
 short drvrOldID;
 char strBuf[128];

#ifdef DEBUG_TRACE
 alertNote("changeRsrcID()");
#endif
 GetResInfo( rsrcHdl, &drvrOldID, &type, name); /* get name */
 if (err = ResError()) {
 alert(text(S_FAIL_GET_INFO, strBuf), err);
 return(err);
 }
#ifdef DEBUG2
 alertNote("SetResInfo NewID %d", drvrNewID);
#endif
 SetResInfo( rsrcHdl, drvrNewID, name); /* set new ID and name */
 if (err = ResError()) {
 alert(text(S_FAIL_SET_INFO, strBuf), err);
 return(err);
 }
 if( !err){
 ChangedResource( rsrcHdl); /* Mark the resource as changed. */
 err = ResError();
 }
 return(err);
}

changeTableResources( change) /* Change the driver table resources. */
int change; /* which driver table has changed. */
{
 Handle hData;
 int err;
 char strBuf[128];


#ifdef DEBUG_TRACE
 alertNote("changeTableResources()");
#endif
 if( change & DRVRTBL){
 hData = GetResource( DRVR_TBL_TYPE, DRVR_TBL_ID);
 if( err = ResError()){
 alert(text(S_FAIL_GET_DRVR_TBL, strBuf));
 return( err);
 };
 ChangedResource( hData);
 if( err = ResError()){
 alert(text(S_FAIL_RSRC_MGR, strBuf), err);
 return( err);
 };
 }

 if( change & OWNERTBL){
 hData = GetResource( OWNED_TBL_TYPE, OWNED_TBL_ID);
 if( err = ResError()){
 alert(text(S_FAIL_GET_OWNED_TBL, strBuf));
 return( err);
 };
 ChangedResource( hData);
 if( err = ResError()){
 alert(text(S_FAIL_RSRC_MGR, strBuf), err);
 return( err);
 };
 }
 return err;
}

/* openMarkedDrivers.c
 ".NB-DMA" and ".NB-GPI" and other subdrivers are opened as needed by
".LabDRIVER"
 */


/**********************************************************************/
/*
 * Opens certain GPIB drivers if they are marked for loading.
 * If no NI boards are in the system, then don't load any drivers.
 * Load will be either L_NO or NOT L_NO.
 * Only driver table types T_DRVR_MASTER and T_BUS_DRVR will be opened. Other
types are
 * opened by higher level drivers.
 */

openMarkedDrivers( drvrTbl,config)
struct drvrStruct *drvrTbl; /* array of drvrStruct */
register config_t *config;
{
 register short i;
 register short bus; /* 1 to n buses */
 short masterRefNum;
 register OSErr err;
 char strBuf[128];

#ifdef DEBUG_TRACE
 alertNote("openMarkedDrivers()");
#endif
 err = noErr;

 masterRefNum = 0;
 for( i=0; i<NUM_DRVRS && !err; ++i) {
#ifdef DEBUG1
 if(( drvrTbl[i].load && (drvrTbl[i].type == T_DRVR_MASTER
 drvrTbl[i].load && drvrTbl[i].type == T_BUS_DRVR)))
 alertNote( "Will open %s", drvrTbl[i].name);
#endif
 if( drvrTbl[i].load && drvrTbl[i].type == T_DRVR_MASTER){
 err = OpenDriver( CtoPstr( drvrTbl[i].name), &masterRefNum); /* name is now
Pascal */
 PtoCstr( drvrTbl[i].name); /* Must be a C string in the table. */
 if( err){
 masterRefNum = 0;
 alert(text(S_FAIL_OPEN_NAME, strBuf), drvrTbl[i].name);
 }
 break; /* only one master driver */
 }
 }
 for( i=0,bus=0; i<NUM_DRVRS && !err; ++i) {
 if( drvrTbl[i].type == T_BUS_DRVR){
 if( drvrTbl[i].load)
 err = openBusDriver(drvrTbl[i].name,bus,config);
 bus++;
 }
 }
 return err;
}

/* Detach the driver if the drivers load flag is not false */
detachDriver( drvrTbl, ownedTbl)
struct drvrStruct *drvrTbl; /* array of drvrStruct */
struct ownedStruct *ownedTbl; /* array of ownedStruct */
{
 register short i, id;
 register OSErr err;
 Handle hndl;
 char strBuf[128];

#ifdef DEBUG_TRACE
 alertNote("detachDriver()");
#endif
 err = noErr;
 /* Update the resource file because ResError will return resAttrErr if you
call
 * DetachResource to detach a resource whose resChanged attribute has been
set.
 */
 UpdateResFile( CurResFile()); /* Assumes that the current res file is the
init. */
 /* Here go the DRVRs */
 for( i=0; i<NUM_DRVRS && !err; ++i) {
 if( !drvrTbl[i].load)
 continue;
#ifdef DEBUG1
 alertNote("Detach %s driver", drvrTbl[i].name);
#endif
 hndl = GetResource( DRVR_TYPE, drvrTbl[i].id);
 if( err = ResError())
 alert(text(S_FAIL_GET_DRVR, strBuf), drvrTbl[i].name);
 DetachResource(hndl);
 if( err = ResError())
 alert(text(S_FAIL_DETACH_DRVR, strBuf), drvrTbl[i].name);
 }

 /* Here go the owned resources; Required for segmented drivers. */
 for( i=0; i<NUM_OWNED && !err; ++i){
 short drvrIndex;
 if( !drvrTbl[drvrID2index( drvrTbl, ownedTbl[i].owner)].load)
 continue;
 drvrIndex = drvrID2index( drvrTbl, ownedTbl[i].owner);
#ifdef DEBUG2
 alertNote("Detach owned type: %4s", &ownedTbl[i].type); /* adr of 4 chars */
 alertNote("id: %d, owner: %d", ownedTbl[i].id, ownedTbl[i].owner);
#endif
 id = OWNED_ID(ownedTbl[i].owner,ownedTbl[i].id); /* calc owned id */
 hndl = GetResource( ownedTbl[i].type, id);
 if( err = ResError())
 alert(text(S_FAIL_GET_OWNED, strBuf), drvrTbl[drvrID2index( drvrTbl,
ownedTbl[i].owner)].name);
 DetachResource(hndl);
 if( err = ResError())
 alert(text(S_FAIL_DETACH_OWNED, strBuf), drvrTbl[drvrID2index( drvrTbl,
ownedTbl[i].owner)].name);
 }
 return err;
}

openBusDriver(name,bus,config)
char *name;
short bus; /* 0 to n-1 buses */
register config_t *config;
{
 ioParam iopb;
 IBoard b;
 short err;
 char strBuf[128];

#ifdef DEBUG_TRACE
 alertNote("openBusDriver()");
#endif
 b.b_uflags = config->brdRsrc.BusData[bus].b_uflags;
 b.b_slot = config->brdRsrc.BusData[bus].b_slot;
#ifdef DEBUG1
 alertNote("bus %d, slot %d, uflags 0x%x", bus, b.b_slot, b.b_uflags); /* 4
args max */
#endif

 if( isSerSlot(b.b_slot)){
 b.b_slot = conf2initSerSlot(b.b_slot);
 b.b_baud = config->brdRsrc.BusData[bus].b_baud;
#ifdef DEBUG1
 alertNote("Serial slot %d, b_baud %d", b.b_slot, b.b_baud); /* 4 args max */
#endif
 }
 iopb.ioCompletion = NULL;
 iopb.ioPermssn = fsCurPerm;
 iopb.ioNamePtr =(StringPtr) CtoPstr(name);
 iopb.ioMisc =(Ptr) &b;
 err = PBOpen(&iopb,FALSE); /* name is now Pascal */
 PtoCstr( name); /* Must be a C string in the table. */
 if( err)
 alert(text(S_FAIL_OPEN_NAME, strBuf), name);
 return(err);
}

/* US keyboard only */

kill_key()
{
 KeyMap theKeys;

 GetKeys( &theKeys);
#ifdef TEST
 printf("0x%lx %lx %lx %lx\n", theKeys.Key[3], theKeys.Key[2], theKeys.Key[1],
theKeys.Key[0]);
#endif
 return( (theKeys.Key[1] == 0x808000L) && !(theKeys.Key[3] theKeys.Key[2]
theKeys.Key[0])); /* index and value for cmd-period. */
}
char *GetOSErrStr();

short firstInit = 0;

initMac()
{
 if( !firstInit){
 firstInit = true;
 /* InitWindows would erase previous INIT icons from the screen if called from
main(). */
 InitFonts(); /* These inits are required by Dialog Mgr.. */
 InitWindows();
 InitMenus();
 TEInit();
 InitDialogs( NULL); /* resume won't work for INITs */
 DeskHook = (ProcPtr)0; /* alert at init time will bomb if not initialized */
 FlushEvents( everyEvent, 0);
 InitCursor();
 }
}

alert(str, err1, err2, err3, err4) /* expects a C (null terminated) string */
char *str;
short err1, err2, err3, err4;
{
 char buf[256],strBuf[128];
 char nilstr[1];
 nilstr[0]='\0';

 sprintf(buf, str, err1, err2, err3, err4);
 note( text_PStr(S_FAIL_INSTALL, strBuf), CtoPstr( buf), nilstr);
}

alertNote(str, err1, err2, err3, err4) /* expects a C (null terminated) string
*/
char *str;
short err1, err2, err3, err4;
{
 char buf[256];
 char nilstr[1];
 nilstr[0]='\0';

 sprintf(buf, str, err1, err2, err3, err4);
 note( CtoPstr( buf), nilstr, nilstr);
}

note( msg1, msg2, msg3)
char *msg1, *msg2, *msg3;
{
 char nilstr[1];
 nilstr[0]='\0';


 initMac();
 ParamText( msg1, msg2, msg3, nilstr);
 Alert( 128, NULL); /* no beep */
}

caution( msg1, msg2, msg3)
char *msg1, *msg2, *msg3;
{
 char nilstr[1];
 nilstr[0]='\0';

 initMac();
 ParamText( msg1, msg2, msg3, nilstr);
 Alert( 129, NULL); /* one beep */
}

qAbort( msg1, msg2, msg3)
char *msg1, *msg2, *msg3;
{
 char nilstr[1];
 nilstr[0]='\0';

 initMac();
 ParamText( msg1, msg2, msg3, nilstr);
 return( Alert( 130, NULL) == 1); /* true if abort */
}


/* No global space is used.
 * fomats: \n, %s, %c, %d, %o, and %x.
 */

sprintf (buf, f, a1)
register char *f, *buf;
int a1;
 {
 register char *s;
 register int *args;
 register int length;
 int radix;

 args = &a1;
 for (; *f; f++)
 switch (*f) {

 case '\n':
 *buf++ = '\r';

 default:
 *buf++ = *f;
 break;

 case '%':
 length = 0;
 while (*++f >= '0' && *f <= '9')
 length = length * 10 + *f - '0';

 switch (*f) {


 default:
 *buf++ = *f;
 break;

 case 's':
 if( !length)
 length = 0x7fff; /* no maximum length */
#ifdef LSC
 s = *(char **)args;
 args+=sizeof(char*)/sizeof(int);
#else
 s = *(char **)args++;
#endif
 while (*s && length--)
 *buf++ = *s++;
 break;

 case 'd':
 radix = 10;
 goto cvt;

 case 'o':
 radix = 8;
 goto cvt;

 case 'x':
 radix = 16;
cvt: buf += itob (buf, *args++, radix);
 break;

 case 'c':
 *buf++ = *args++;
 break;
 }
 }
 *buf++ = '\0'; /* null end of string */
}

/* integer to base */
itob (buf, n, base)
char *buf;
register unsigned int n;
register unsigned int base;
{
 register unsigned int len, extra=0;

 if (base == 10 && (int)n < 0) {
 n = -n;
 *buf++ = '-';
 extra++;
 }

 len = n < base ? 0 : itob (buf, n / base, base);
 buf[len] = digits(n % base);

 return len + extra + 1;
}


/* returns the ASCII of the binary n without using a global table */
digits( n)
int n;
{
 int c;

 switch(n){
 case 0: c='0'; break;
 case 1: c='1'; break;
 case 2: c='2'; break;
 case 3: c='3'; break;
 case 4: c='4'; break;
 case 5: c='5'; break;
 case 6: c='6'; break;
 case 7: c='7'; break;
 case 8: c='8'; break;
 case 9: c='9'; break;
 case 10: c='a'; break;
 case 11: c='b'; break;
 case 12: c='c'; break;
 case 13: c='d'; break;
 case 14: c='e'; break;
 case 15: c='f'; break;
 default: c='?'; break;
 }
 return c;
}




[LISTING TWO]

/* initOpenDRVR.h
# John Rosford, National Instruments.
# Copyright 1988,1989 National Instruments Corporation
# All rights reserved.
*/

/* select compile configuration for Macintosh handler inits.*/
#define DEBUG_TRACE /* Alert for each function. */
#define DEBUG1 /* Show drivers marked for loading. */
#define DEBUG2 /* Show driver ID collisions. */
#define DEBUG3 /* Disable duplicate drivers error. */
#undef DEMO /* Fill slot info with NB-DMA-8 in slot 1. */
#define GLOBALS /* Using global variables. */

#define LSC /* Lightspeed C */

#define NSLOTS 6 /* number of NuBus slots. Don't include serial slots here. */

#define START_OF_USER_IDS 12 /* Inside Mac IV-215. Zero is used to flag a
failure. */

#define SSLOTS 2 /* serial slots */
#define SSLOTA_NDX (0+NSLOTS) /* index into slotInfo table - serial slot A,
modem */
#define SSLOTB_NDX (1+NSLOTS) /* index into slotInfo table - serial slot B,
printer */

#define OwnedMASK 0xC000 /* bit mask for the ID of a resource owned by a
driver */
#define OWNED_ID(owner,sub_id) (OwnedMASK (owner << 5) (sub_id))


#define NUM_IDS 128 /* max number of driver ids */

#ifndef NULL
#define NULL 0L
#endif

#define DRVR_TYPE 'DRVR'
/*----------------------------'dTbl'
Resources-------------------------------------*/
#define DRVR_TBL_TYPE 'dTbl'
#define DRVR_TBL_ID 128 /* driver table */

/*----------------------------'oTbl'
Resources-------------------------------------*/
#define OWNED_TBL_TYPE 'oTbl'
#define OWNED_TBL_ID 128 /* owned table */

/*----------------------------'busD'
Resources-------------------------------------*/
#define BUS_DATA_TYPE 'busD'
#define BUS_DATA_ID 128
#define BUS_DATA_NAM "GPIB-BusData"

/*----------------------------STR#
Resources-------------------------------------*/
#define SS_MSGS_ID 128 /* init message strings */

/* Load flags must not conflict with a slot number: 1-6 */
#define L_NO 0
#define L_YES -1

#define NB_GPIB 0x109
#define NB_DMA_8 0x10A

#define ourBoard(id) ( id==NB_GPIB id==NB_DMA_8)

/* Type flags. DRVR flags include dependency information. */
enum{
 T_DRVR_MASTER=1, /* Load/open if any NI board */
 T_BUS_DRVR, /* Load/open GPIB bus driver if configured or any NI-488 board in
slot */
 T_DRVR_GPIB, /* Load if any NI-488 board: NB-GPIB, NB-DMA, etc. */
 T_BOARD_HW, /* Load if any NI-488 resource: NB-GPIB, NB-DMA */
 T_SERIAL_HW, /* Load if any NI-488 resource: GPIB-422CT, GPIB-MAC */
 T_NB_GPIB, /* Load if hardware is installed */
 T_NB_DMA_8 /* Load if hardware is installed */
};

/* slot translation: s=1 to 6 -> s+8, s=7 to 14 -> s-6 */
#define macSlotNum(s) (s<=6?s+8:s-6)

/* identifies table resource */
#define DRVRTBL 1
#define OWNERTBL 2

/* NB-Series Board Types */
#define NOT_OUR_BOARD 0
#define NO_BOARD -1
#define SLOT_OUT_OF_BOUNDS -2

/* typedefs and macros created to aid compatibility between compilers. */
typedef char int8;
typedef short int16;

typedef long int32;
typedef unsigned char uInt8;
typedef unsigned short uInt16;
typedef unsigned long uInt32;

/* structure of the driver and owned resource resources. */
struct drvrStruct{ /* id changes if DRVR ID conflict in checkIDs(). */
 int16 id; /* All of these are DRVR_TYPE */
 int16 load; /* Set according to boards and drivers installed. Load driver if
true. */
 int16 type; /* name is either a board name or a driver name. */
 char name[16]; /* the name */
};

struct ownedStruct{ /* owner changes if DRVR ID conflict in checkIDs().*/
 ResType type;
 int16 id; /* sub id. Never changes. Resource id is OWNED_ID(owner_id,sub_id)
*/
 int16 owner; /* owner's id */
};

/* Parameter to first openDriver. */
typedef struct IBOARD
 {
 uInt16 b_uflags; /* user flags */
 uInt8 b_slot; /* NUBUS slot or serial port number */
 uInt16 b_baud; /* Baud for slots A & B */
} IBoard;

/* GPIB configuration structures, resource names, and constants */

#define NBUSES 2 /* number of buses in bus data */

#define BOARDREV 0x12 /* Rev 1.2 of BoardResource */

#define E_NBRDS -1 /* Errors returned by getBrds & getDevs */
#define E_BRDREV -2

/* Bus Structure */
struct busConf {
 uInt16 b_uflags; /* user flags */
 uInt16 b_slot; /* Slot for this GPIB bus. */
 uInt16 b_baud; /* Baud for slots A & B */
};


typedef struct {
 uInt16 Rev; /* hex value, for example: 0x10 for Rev 1.0 */
 uInt16 Cnt; /* number of buses in BusData */
 struct busConf BusData[NBUSES]; /* configuration data for each bus */
 } BusResource;

/* INIT configuration structures and constants */
#define isSerSlot(s) (s > NSLOTS) /* is serial slot */
#define conf2initSerSlot(s) (s-NSLOTS-1) /* is serial slot */

/* hardware variant types other than NuBus */
#define GPIBMAC 20000

typedef struct {
 char brdName[256]; /* max name length */

 int16 brdID;
} slotInfoType;

typedef struct {
 int16 MaxBuses; /* number of buses read from bus data. */
 slotInfoType slotInfo[NSLOTS+SSLOTS]; /* nubus+serial slots */
 BusResource brdRsrc;
}config_t;

/* Prototypes of functions defined in initOpenDRVR.c */
main(void);
dupDriverInstalled(struct drvrStruct *drvrTbl);
char *strcpy( char*, char*);
char *strcat(char*, char*);
int strcmp( char*, char*);
char *text_PStr(int16, char *);
char *text(int16, char *);
showIcon(void);
getTables( struct drvrStruct **, struct ownedStruct **);
configInfo(config_t *);
configBrds( BusResource*,config_t*);
markLoad( struct drvrStruct *, config_t *);
ndxDrvrType( struct drvrStruct *, int16);
drvrID2index( struct drvrStruct *, int16);
getSlot( unsigned int16, config_t *);
autoSlot( int16,config_t *);
slotInfoTable(config_t *);
calcNslots( void);
GetSlotInfo(int16, char *);
checkSysHeapSize( struct drvrStruct *, struct ownedStruct *);
findUsedIDs( int16*);
checkUnitTable( int16*);
checkSysRsrcs( int16*);
checkIDs( struct drvrStruct *, struct ownedStruct *, int16 *);
getUnusedID( int16 *,int16 *);
newOwner( struct ownedStruct *, int16, int16);
changeRsrcID( Handle, int16);
changeTableResources( int16);
openMarkedDrivers( struct drvrStruct *,config_t *);
detachDriver( struct drvrStruct *, struct ownedStruct *);
openBusDriver(char *,int16,config_t *);
kill_key(void);
initMac(void);
alert(char *, ...);
alertNote(char *, ...);
note( char *, char *, char *);
caution( char *, char *, char *);
qAbort( char *, char *, char *);
sprintf (char *, char *, ...);
itob (char *, unsigned int16, unsigned int16);
digits( int16);

/* indices to STR# */
enum{
 ZERO,
 S_FAIL_INSTALL,
 S_FAIL_REMOVE,
 S_FAIL_GET_DRVR_TBL,
 S_FAIL_GET_OWNED_TBL,

 S_FAIL_NO_DRVR,
 S_FAIL_GET_DRVR,
 S_FAIL_GET_OWNED,
 S_FAIL_ID_OSERR,
 S_FAIL_HEAP,
 S_FAIL_RSRC_MGR,
 S_FAIL_NO_FREE_ID,
 S_FAIL_ID_CHANGE,
 S_FAIL_GET_ATTR,
 S_FAIL_GET_INFO,
 S_FAIL_SET_INFO,
 S_FAIL_SET_ATTR,
 S_FAIL_OPEN_NAME,
 S_FAIL_OPEN_DRVRS,
 S_FAIL_DETACH_DRVR,
 S_FAIL_DETACH_OWNED,
 S_NOT_DRVR_TYPE,
 S_NEED_MAC_SE,
 S_NEED_MAC_II,
 S_USER_ABORT,
 S_Q_USER_ABORT,
 S_FAIL_GET_DATA,
 S_FAIL_GET_NAME,
 S_FAIL_NUM_BUSES,
 S_FAIL_REVISION,
 xS_LAST_STRING
};


[LISTING THREE]


/* File DDJInit.r

 Copyright ) 1989 National Instruments.

With MPW, to compile this file and copy the resources to the INIT:
Set BuildDir ':Build INIT:'
Set InitName 'DDJ INIT'
rez -o "{BuildDir}{InitName}" -s "{BuildDir}" DDJInit.r
Setfile "{BuildDir}{InitName}" -a B -t INIT -c DDJI
_________________________________________________________________________*/

/* include other resources */
#define INIT_RSRC
#ifdef INIT_RSRC
include "*INIT.rsrc"; /* INIT resource; flags OK */
#endif
include "*DDJ Driver" 'DRVR' (0:64) as
 'DRVR' ($$ID, $$Name, SysHeap, Locked); /* DRVR resource */
include "*DDJ Driver" 'DATA' (-15424:-15200) as
 'DATA' ($$ID, "", SysHeap, Purgeable); /* owned DATA resource */

/* include other text files */
#include "Types.r"
#include "SysTypes.r"

#define CREATOR 'DDJI' /* file creator */


/* Board IDs */
#define NB_GPIB 0x109
#define NB_DMA_8 0x10A

/* Resource IDs */

/*----------------------------'dTbl'
Resources-------------------------------------*/
#define DRVR_TBL_TYPE 'dTbl'
#define DRVR_TBL_ID 128 /* driver table */

/*----------------------------'oTbl'
Resources-------------------------------------*/
#define OWNED_TBL_TYPE 'oTbl'
#define OWNED_TBL_ID 128 /* owned table */

/*----------------------------STR#
Resources-------------------------------------*/
#define SS_MSGS_ID 128 /* init message strings */

/*----------------------------'busD'
Resources-------------------------------------*/
#define BUS_DATA_TYPE 'busD'
#define BUS_DATA_ID 128
#define BUS_DATA_NAM "GPIB-BusData"

/* Load flags must not conflict with a slot number: 1-6 */
#define L_NO_X 0
#define L_YES_X -1


/*----------------------------dTbl % driver
table-------------------------------------*/
type DRVR_TBL_TYPE {
 wide array {
 integer BUS0_ID=30, /* GPIB bus drivers */
 BUS1_ID,
 SHARE_ID, /* Shared code */
 SH_ID, /* GPIB serial port hardware */
 BH_ID, /* GPIB board hardware */
 LAB_ID, /* LabDriver */
 DMA_ID, /* NB-DMA subdriver */
 GPI_ID /* NB-GPIB subdriver */
 ; /* DRVR IDs */
 integer L_YES=L_YES_X,
 L_NO=L_NO_X; /* Load status */
 integer T_DRVR_MASTER=1, /* Load if any NI board */
 T_BUS_DRVR, /* Load GPIB bus driver if configured and any NI-488 board in
slot */
 T_DRVR_GPIB, /* Load if any NI-488 resource: NB-GPIB, NB-DMA, GPIB-422CT, etc
*/
 T_BOARD_HW, /* Loaded by LabDriver */
 T_SERIAL_HW, /* Load if any NI-488 resource: GPIB-422CT, GPIB-MAC */
 T_NB_GPIB, /* Load if hardware is installed */
 T_NB_DMA_8 /* Load if hardware is installed */
 ; /* DRVR Type flags */
 cstring[16]; /* board name else driver name */
 };
};


/*----------------------------oTbl % owned resource
table-----------------------------*/

type OWNED_TBL_TYPE {
 wide array {
 literal longint; /* type of owned resource */

 integer; /* id of owned resource */
 integer BUS0_ID=30, /* GPIB bus drivers */
 BUS1_ID,
 SHARE_ID, /* Shared code */
 SH_ID, /* GPIB serial port hardware */
 BH_ID, /* GPIB board hardware */
 LAB_ID, /* LabDriver */
 DMA_ID, /* NB-DMA subdriver */
 GPI_ID /* NB-GPIB subdriver */
 ; /* DRVR IDs */
 };
};


/*----------------------------DATA % Bus
Data-------------------------------------*/
type BUS_DATA_TYPE {
 hex integer; /* Revision */
 integer; /* Number of Buses */
 wide array {
 unsigned hex integer; /* uflags */
 unsigned integer; /* slot */
 unsigned integer; /* baud */
 };
};

data 'sysz' (0) {
 $"0001 0000" /* expand sys heap */
};

/* DRVR indices must match table resources. See initOpenDRVR.c */
resource DRVR_TBL_TYPE (DRVR_TBL_ID, "drvrTbl", preload) {
 {
 BUS0_ID, L_NO, T_BUS_DRVR, ".GPIB0",
 BUS1_ID, L_NO, T_BUS_DRVR, ".GPIB1",
 SHARE_ID, L_NO, T_DRVR_GPIB, ".GPIBSharedCode",
 SH_ID, L_NO, T_SERIAL_HW, ".GMHardwareCode",
 BH_ID, L_NO, T_BOARD_HW, ".NBHardwareCode",
 LAB_ID, L_NO, T_DRVR_MASTER, ".LabDRIVER",
 DMA_ID, L_NO, NB_DMA_8, ".NB-DMA",
 GPI_ID, L_NO, NB_GPIB, ".NB-GPI"
 }
};

resource OWNED_TBL_TYPE (OWNED_TBL_ID, "ownedTbl", preload) {
 {
 'DATA', 0, BUS0_ID, /* ".GPIB0" owned data resource */
 'DATA', 0, BUS1_ID, /* ".GPIB1" owned data resource */
 'DATA', 0, SHARE_ID, /* ".GPIBSharedCode" owned data resource */
 'DATA', 0, BH_ID, /* ".NBHardwareCode" owned data resource */
 'DATA', 0, SH_ID, /* ".GMHardwareCode" owned data resource */
 'DATA', 0, LAB_ID, /* ".LabDRIVER" owned data resource */
 'DATA', 0, DMA_ID, /* ".NB-DMA" owned data resource */
 'DATA', 0, GPI_ID /* ".NB-GPI" owned data resource */
 }
};


#define baud9600 10


/* No two buses should be assigned to the same slot. Slots range from 1 to
NSLOTS+SSLOTS with
 * zero for not assignment. Change slot to 7 for port A.
 */

resource BUS_DATA_TYPE (BUS_DATA_ID, BUS_DATA_NAM) {
 0x12, /* Revision */
 2, /* Number of Buses */
 { /* array: 2 elements */
 /* [1] */
 0x1C03,1,baud9600, /* uflags,slot 1,baud */
 /* [2] */
 0x1C03,7,baud9600 /* serial slot A */
 }
};


resource 'BNDL' (128, purgeable) {
 CREATOR,
 0,
 { /* array TypeArray: 2 elements */
 /* [1] */
 'ICN#',
 { /* array IDArray: 2 elements */
 /* [1] */
 0, 128,
 },
 /* [2] */
 'FREF',
 { /* array IDArray: 2 elements */
 /* [1] */
 0, 128,
 }
 }
};

type CREATOR as 'STR ';

resource CREATOR (0) {
 "DDJ Mac Init Version 1.0"
};

resource 'FREF' (128, preload) {
 'INIT',
 0,
 ""
};

resource 'vers' (1, purgeable) {
0x01, 0x00, final, 0x00, verUS,
"1.0",
"1.0, ) National Instruments 1989"
};

resource 'vers' (2, purgeable) {
0x01, 0x00, final, 0x00, verUS,
"1.0",
"DDJ INIT Release 1.0"
};


resource 'STR#' (SS_MSGS_ID, "INIT Messages") {
 {
 "The driver installation failed.", /* S_FAIL_INSTALL */
 "Remove old or duplicate NI drivers.", /* S_FAIL_REMOVE */
 "Failed to get the driver table resource.", /* S_FAIL_GET_DRVR_TBL */
 "Failed to get the owned table resource.", /* S_FAIL_GET_OWNED_TBL */
 "No driver with id = %d.", /* S_FAIL_NO_DRVR */
 "Failed to get the driver resource: %s.", /* S_FAIL_GET_DRVR */
 "Failed to get an owned resource of: %s.", /* S_FAIL_GET_OWNED */
 "ID %d, OSErr = %d", /* S_FAIL_ID_OSERR */
 "Out of memory in the system heap.", /* S_FAIL_HEAP */
 "Resource Mgr error: %d", /* S_FAIL_RSRC_MGR */
 "Failed to find a free driver ID.", /* S_FAIL_NO_FREE_ID */
 "Failed to get resource for ID change, id=%d, err=%d.", /* S_FAIL_ID_CHANGE
*/
 "Failed to get resource attributes, err=%d.", /* S_FAIL_GET_ATTR */
 "Failed to get resource info, err=%d.", /* S_FAIL_GET_INFO */
 "Failed to set resource info, err=%d.", /* S_FAIL_SET_INFO */
 "Failed to set resource attributes, err=%d.", /* S_FAIL_SET_ATTR */
 "Failed to open %s.", /* S_FAIL_OPEN_NAME */
 "OpenDriver error %d.", /* S_FAIL_OPEN_DRVRS */
 "Failed to detach %s driver.", /* S_FAIL_DETACH_DRVR */
 "Failed to detach %s owned resource.", /* S_FAIL_DETACH_OWNED */
 "Not a driver type: %d, at table index %d", /* S_NOT_DRVR_TYPE */
 "The handler requires a Macintosh SE.", /* S_NEED_MAC_SE */
 "The handler requires a Macintosh II.", /* S_NEED_MAC_II */
 "User aborted driver installation.", /* S_USER_ABORT */
 "Abort the driver installation?", /* S_Q_USER_ABORT */
 "Failed to get the default bus/device data resources.", /* S_FAIL_GET_DATA */
 "Failed to get the default name resources.", /* S_FAIL_GET_NAME */
 "Too many buses in default data resources", /* S_FAIL_NUM_BUSES */
 "Wrong revision in default data resources.", /* S_FAIL_REVISION */
 "End of strings." /* */
 }
};

resource 'ALRT' (128) {
 {50, 30, 150, 480},
 128,
 { /* array: 4 elements */
 /* [1] */
 OK, visible, silent,
 /* [2] */
 OK, visible, silent,
 /* [3] */
 OK, visible, silent,
 /* [4] */
 OK, visible, silent
 }
};

resource 'ALRT' (129) {
 {50, 30, 150, 480},
 128,
 { /* array: 4 elements */
 /* [1] */
 OK, visible, sound1,
 /* [2] */
 OK, visible, sound1,
 /* [3] */

 OK, visible, sound1,
 /* [4] */
 OK, visible, sound1
 }
};

resource 'ALRT' (130) {
 {50, 30, 150, 480},
 129,
 { /* array: 4 elements */
 /* [1] */
 OK, visible, sound1,
 /* [2] */
 OK, visible, sound1,
 /* [3] */
 OK, visible, sound1,
 /* [4] */
 OK, visible, sound1
 }
};

resource 'DITL' (128) {
 { /* array DITLarray: 4 elements */
 /* [1] */
 {60, 190, 78, 260},
 Button {
 enabled,
 "OK"
 },
 /* [2] */
 {10, 10, 24, 440},
 StaticText {
 enabled,
 "^0"
 },
 /* [3] */
 {25, 10, 39, 440},
 StaticText {
 enabled,
 "^1"
 },
 /* [4] */
 {40, 10, 54, 440},
 StaticText {
 enabled,
 "^2"
 }
 }
};

resource 'DITL' (129) {
 { /* array DITLarray: 5 elements */
 /* [1] */
 {60, 115, 78, 185},
 Button {
 enabled,
 "Abort"
 },
 /* [2] */

 {60, 265, 78, 335},
 Button {
 enabled,
 "Continue"
 },
 /* [3] */
 {10, 10, 24, 440},
 StaticText {
 enabled,
 "^0"
 },
 /* [4] */
 {25, 10, 39, 440},
 StaticText {
 enabled,
 "^1"
 },
 /* [5] */
 {40, 10, 54, 440},
 StaticText {
 enabled,
 "^2"
 }
 }
};

resource 'ICN#' (128) {
 { /* array: 2 elements */
 /* [1] */
 $"FF FF FF FE 80 00 00 03 80 00 00 03 8F E7 F1 F3"
 $"88 34 19 13 8D DE ED B3 85 6A B4 A3 85 2A 94 A3"
 $"85 2A 94 A3 85 2A 94 A3 85 2A 94 A3 85 2A 94 A3"
 $"85 2A 94 A3 85 2A 94 A3 85 2A 94 A3 85 2A 94 A3"
 $"85 2A 94 A3 85 2A 94 A3 85 2A 94 A3 85 2A 94 A3"
 $"85 2A 94 A3 85 2A 94 A3 85 6A B4 A3 8D DE EC A3"
 $"88 34 18 A3 8F E7 F0 A3 80 00 03 A3 80 00 02 63"
 $"80 00 03 C3 80 00 00 03 FF FF FF FF 7F FF FF FF",
 /* [2] */
 $"FF FF FF FE FF FF FF FF FF FF FF FF FF FF FF FF"
 $"FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF"
 $"FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF"
 $"FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF"
 $"FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF"
 $"FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF"
 $"FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF"
 $"FF FF FF FF FF FF FF FF FF FF FF FF 7F FF FF FF"
 }
};














Special Issue, 1989
MEMORY MANAGEMENT WITH MACAPP


Take control of your application's heap




Curt Bianchi


Curt is a software engineer at Apple Computer. He can be reached at Apple
Computer Inc., 20525 Mariani Ave., MS 22-AE, Cupertino, CA 95014, or on
Applelink at Bianchil.


Many Macintosh programmers equate MacApp with object-oriented programming and
vice versa. Although it's certainly true that much of MacApp is
object-oriented, it offers many services implemented with procedural
programming as well. Among those services are tools that help with the
difficult problem of managing an application's memory space. These tools range
from useful dubugging aids to techniques for detecting and recovery from
memory allocation failures, culminating in a comprehensive strategy for
implementing robust Macintosh applications in a dynamic memory environment.
Because these tools are implemented as conventional Pascal units and supplied
with source code, they can be used by any Macintosh program built with MPW,
including those written in C.
The purpose of this article is to describe some of these tools so that MacApp
and non-MacApp programmers alike can make use of them an perhaps even entice
some non-MacAppers into giving MacApp a try. By the way, I assume you have a
pretty good idea of how Macintosh Memory Manager works. If you don't, How to
Write Macintosh Software by Scott Knaster has an excellent description, and
it's funny, too! And there's always Inside Macintosh, though it lacks Scott's
humor (unless you count the Munger). Also, this article is based on MacApp
2.0B9. Some details may not apply to previous versions of MacApp, including
MacApp 2.0B5.


The Problem


One of the most difficult aspects of writing a Macintosh application is the
pervasive use of dynamic memory allocation. The Macintosh user interface and
the design of the Toolbox both encourage heavy use of dynamic memory
allocation. For example, good Macintosh programs allow the user to open
multiple documents and windows, limited only by available memory. Because
there is no way to know in advance how many windows or documents can be opened
on a given machine, dynamic allocation of window and document data structures
provides the natural solution. Given that available memory is indeed limited,
a major task in completing an application is ensuring that it properly handles
the time when memory fills up and dynamic memory requests are no longer
satisfied.
To complicate matters further, an application doesn't have complete control of
its own heap! (See the accompanying box.) The Macintosh system dynamically
allocates handles, such as code and system resources, in the application's
heap. And the system isn't very polite about such matters: It doesn't inform
the application when using its heap, nor does it recover gracefully if one of
its requests fails. Furthermore, any number of foreign objects cant take up
residence in your heap, the most common being the desk accessory, which uses
your heap when MultiFinder isn't available. In other words, your explicit
memory requests compete with the demands (they can hardly be called requests
because they can't fail) from the Macintosh system for the very same memory
space -- namely -- the application's heap.


MacApp's Strategy


Clearly, an overall strategy is needed for implementing a program in such an
environment. The goals os the strategy should be twofold. First, it should
implement a scheme that always maintains enough free space in the application
heap to satisfy the system's needs. And second, the heap should always have
enough space to carry out essential operations such as quitting, saving
documents, or deleting data from documents. Though more subtle, the latter
goal is just as important as the former. And it's easier said than done.
Quitting an application, for example, typically requires loading code segments
that are used only when terminating the program. If the program has allowed
the user to use up too much memory for data, then those termination segments
won't be loadable! (Don't laugh, I've seen this happen.)
MacApp's strategy is mostly implemented in a Pascal unit called "UMemory,"
which controls which memory requests succeed. The idea is to ensure that
essential requests always succeed, whereas non-essential requests may fail.
Examples of requests that must succeed are allocations for code segments
(strictly speaking, MacApp's strategy is smart enough that only certain
segments must be loadable), defprocs, and packages. An example of a request
that may fail is the allocation of data structures to open a second (or third,
and so on) document.
The way MacApp does this is by reserving enough space in the heap so that
essential memory requests can be satisfied at any point in the program.
Because the Macintosh Memory Manager doesn't know an essential request from a
nonessential one, it's necessary to intervene in the process and give the
Memory Manager some help.
Fortunately, the designers of the Memory Manager provided a mechanism that
lets you do just that. It's called the grow zone function. The way it works is
that whenever the Memory Manager finds that there isn't enough space in the
heap to satisfy a memory request, it calls the grow zone function in the hopes
of freeing additional space in the heap. The grow zone function is called
repeatedly until either there is enough space in the heap to satisfy the
request or the grow zone function is unable to free any more space, in which
case the allocation request fails. By installing a grow zone function, a
program can determine which requests succeed and which ones fail.


Permanent vs. Temporary Requests


MacApp's grow zone function distinguishes between essential and nonessential
memory requests. In MacApp terminology, these are classified as temporary
(essential) and permanent (non-essential). A useful way of thinking about this
is that temporary requests are program-driven whereas permanent requests are
user-driven. Memory allocated with a temporary request is freed when the
program has finished with it; memory allocated with a permanent request is
freed when the user has finished with it. Table 1 provides examples of each.
Given this distinction between temporary and permanent memory requests, how
does MacApp take advantage of it? If you look at Table 1, you'll notice that
the very nature of temporary requests is that they must succeed. For one
thing, most temporary requests come from the system, which assumes success!
Also, because these requests are program-driven, their success is usually
essential to keep the application running. In other words, failure of a
temporary request is usually fatal. On the other hand, the failure of a
permanent request simply indicates a lack of memory to carry out a user
operation such as opening another document. Assuming the application started
with enough memory to perform a reasonable set of operations, failure of
permanent requests isn't catastrophic. It simply indicates that the user needs
to reduce memory usage, say by closing a document or deleting some data.
Table 1: Examples of temporary requests and permanent requests
Temporary Requests
Temporary requests usually come from the system, when it attempts to use the
application's heap. Examples of such uses are:
Code segments, whose handles are allocated by the Macintosh operating system
when a segment has to be loaded into memory.
Defprocs for the Window Manager, Control Manager, Menu Manager, and List
Manager. They take the form of WDEF, CDEF, MDEF, and LDEF resources,
respectively.
Packages, which take the form of PACK resources.
Fonts, which take the form of FONT and FOND resources. Handles for Toolbox
data structures associated with windows, controls, etc.
Handles allocated by the Toolbox and operating system for temporary data such
as printing data structures, regions for QuickDraw, the file list for Standard
File dialogs, etc.
Handles allocated by the application for short-term use, such as handles that
are created and disposed of within the same routine.
Permanent Requests
Most explicit requests by the application are permanent, and are used for data
structures requested by the user, such as document data structures. These
include:
Allocation of objects.
Most handles allocated by the application.

MacApp ensures the success of temporary requests by reserving enough space in
the heap so that any temporary request succeeds (ultimately at the expense of
permanent requests). The space is reserved simply by allocating a handle large
enough to reserve it. This handle is known as the temporary reserve. Its size
is determined by taking the total amount of space reserved for temporary
requests and subtracting from it the sizes of the currently allocated
temporary handles. Becuase its purpose is to reserve space, the size of the
temporary reserve varies over time as the amount of temporary handles in
memory increases or decreases.
Now, when a memory request is made and there is no room in the heap, the
Memory Manager calls MacApp's grow zone function. It behaves differently
depending on whether the request is temporary or permanent. The algorithm is
shown in Table 2. The basic idea is never to let a permanent request use space
reserved for temporary requests, thereby always having enough space to satisfy
a temporary request. (The emerging reserve referred to in the algorithm will
be described later in this article.)
Table 2: MacApp's grow zone function algorithm
For a permanent request:

1. If the temporary reserve is too big, reduce its size to that actually
needed.
2. Otherwise, if there is a temporary handle that's not in use, purge it so
long as the space held by the temporary reserve and the other temporary
handles doesn't fall below the minimum that must be reserved.
3. Otherwise, if the emergency reserve is intact, purge it as a last-ditch
attempt to free space.
For a temporary request:
1. If the temporary reserve exists, purge it. (It will be reallocated later.)
2. Otherwise, if there is a temporary handle that's not in use, purge it. A
temporary handle is considered to be in use only if it's locked.
3. Otherwise, if the emergency reserve is intact, purge it as a last-ditch
attempt to free space.

The algorithm raises some interesting questions. First, how is it that
MacApp's grow zone function knows whether it's been called in response to a
temporary request or a permanent one? And second, how does the grow zone
function know if a handle is temporary and whether such a handle is currently
in use or not?


The Permanent Flag


The answer to the first question is that MacApp's UMemory unit maintains an
internal flag indicating whether it's in temporary or permanent mode. It is
used by MacApp's grow zone function to decide what to do. The normal state of
the flag is temporary, causing system requests (which tend to come at
unpredictable times) to be temporary.
For making permanent requests, MacApp defines four memory allocation routines,
each consisting of these steps:
1. Set the flag to permanent
2. Carry out the request
3. Restore the state of the flag
These routines are shown in Table 3. The inclusion of SetPermHandleSize and
SetPermPtrSize may seem odd at first, but increasing the size of a handle or
pointer results in a memory request no different from creating the handle or
pointer in the first place. An additional routine called PermAllocation is
available for setting the flag to affect any number of subsequent requests.
Thus, the two code fragments in Listing One, page 28, accomplish the same
thing -- they allocate two handles as permanent requests.
Table 3: Permanent requestors

 Permanent Request Inside Mac
 Routines Counterparts
 ______________________________________________________________________

 FUNCTION NewPermPtr (logicalSize: Size): Ptr; NewPtr
 FUNCTION NewPermHandle (logicalSize: Size) Handle; NewHandle
 PROCEDURE SetPermHandleSize (h:Handle; newSize; Size); SetHandleSize
 PROCEDURE SetPermPtrSize (p: Ptr; newSize: Size); SetPtrSize

Most allocations by the application itself are permanent. For this reason,
creating objects with NEW uses NewPermHandle to allocate the object's handle.
Likewise, most handles you create will be for long-lived data such as document
or window data structures, so they should be allocated with NewPermHandle.
Handles used for short-term use, such as those allocated and disposed of in
the same routine, can be allocated with NewHandle.


Temporary Handles


The second question that needs answering is how MacApp's grow zone function
finds temporary handles that are not in use. Actually, it's simple. MacApp
maintains four lists of handles. The handles in these lists are considered
temporary, and may be purged by MacApp's grow zone function if they aren't in
use. The four lists are:
1. gCodeSegs, which contains handles to all the code segments
2. gSysMemList, which contains handles to all the non-ROM system and
application LDEF, CDEF, MDEF, WDEF, and PACK resources
3. gApp1MemList and and gApp2MemList, which contain handles, usually to
resources, that the application wishes to be subject to purging by MacApp's
grow zone function
Note that handles to code segments and other resources exist whether or not
the segment or resource is loaded into memory. (If it's not in memory, the
handle's master pointer is NIL.)
The definition of "not in use" is simply that the handle is not locked. For
example, when code in a segment is executed, the segment's handle is locked.
When a segment is unloaded, it is simply unlocked and eligible for purging.
Likewise, when a defproc is actually used, it is locked. When it's not locked,
it's also eligible for purging. Normally all these unlocked handles would be
purged by the Memory Manager long before it got to the grow zone function. But
because MacApp wants to control when these handles are purged, it marks each
of them as nonpurgable to prevent the Memory Manager from interfering. Thus,
MacApp's grow zone function is able to choose when to purge them.
By the way, these lists are prioritized in terms of which handles get purged
first. The order is gCodeSegs, gApp1MemList, gSysMemList, and gApp2MemList.
That is, MacApp's grow zone function will purge unused code segments before
purging unused system resources. Similarly, those handles in gApp1MemList are
purged before those in gApp2MemList.


Determining How Much Space to Reserve for Temporary Handles


Essential to making this scheme work is ensuring that the right amount of
space is reserved for temporary requests. If too little space is reserved,
then the application could wind up being unable to satisfy a temporary
request, which usually results in a system error. (Despite its frequency of
appearance, the system error alert is not considered part of the Macintosh
look and feel.) On the other hand, reserving too much space is wasteful,
causing the application to require more memory than it really needs.
To determine how much space to reserve, MacApp uses the following formula:
1. Add up the sizes of the code segments whose names are listed in all seg!
resources seg! resources will be explained shortly -- their purpose is to list
the segments that must be in memory at the time the application uses the
greatest amount of code.
2. Add to this total an arbitrary amount from all mem! resources. This lets
you reserve space for noncode use, such as system resources; mem! resources
are also explained later.
The result of this formula should be the maximum amount of space that the
application will need to satisfy any temporary request. Note that I said
should be. That's because both steps require that you provide information to
MacApp -- namely, a list of code segments in step 1 and an additional padding
amount in step 2. Thus, this scheme is only as good as the information you
give it.


Accounting for Code Segments, or How to Use seg! Resources



Because allocating memory for a code segment is considered a temporary
request, it is necessary to know how much space to reserve for code. A
simple-minded approach would be to add up the sizes of all the code segments
in the application, but that would be wasteful because it's doubtful a large
program would ever need all of its segments in memory at one time. A better
way is to add up the sizes of the code segments that must be in memory when
the application uses the greatest amount of code. If enough space is reserved
for that case, then clearly there will be enough space for code in all cases.
To do this, you must determine which segments are required and the sizes of
those segments -- which is exactly what seg! resources are for.
A seg! resource is simply a list of strings, each of which is the name of a
segment. The segments listed in seg! resources are those segments that must be
in memory at the point of greatest code usage. When the application is
started, MacApp will add up the sizes of all the code segments listed in the
seg! resources, arriving at the amount of memory that it reserves for code.
The nice part is that you don't have to know the actual size of each segment,
just the names of the segments involved.
MacApp defines one seg! resource of its own, shown in Listing Two, page 28. It
lists MacApp's resident segments -- that is, the MacApp-defined segments that
are always in memory --and conditionally lists MacApp's debugging segments.
Normally you add one seg! resource of your own identifying which segments, in
addition to those in MacApp's lists, are in memory at the point of greatest
usage.
Obviously the key to making this work is knowing which segments to list in
your seg! resources. MacApp's debugger makes this easy to do. First, you can
request that the debugger interrupt the program any time the previous point of
greatest code usage is exceeded. And second, when the point of greatest usage
is identified, you can ask the debugger to list those code segments in use at
the time. Voila -- you've got your seg! list.
Here's the step-by-step procedure:
1. Build a debugging version of your application.
2. Start the application by double-clicking its icon (or better, by
double-clicking one of its documents). Immediately afterward, hold down the
Shift, Option, and Command keys; this is how you break into the MacApp
debugger. What you want to do is break into the debugger as soon as possible.
When the program break occurs, the MacApp debugger window will appear,
containing the cryptic message
stopped at Begin INITUMENUSETUP Seg#: 10
Command [BCDEFGHILMOPpiQRSbetaTWX?]:
The first line indicates where the program was interrupted, in this case at
the start of the routine InitUMenuSetup, which is in segment number 10. The
second line is the debugger's command prompt. Commands in the MacApp debugger
are issued by typing a single character. At any debugger prompt, the
characters accepted appear within brackets; typing a question mark displays a
description of each command.
3. Type x, then r. This causes MacApp to report via the debugger window each
time the amount of memory used by code and system resources surpasses the
previous maximum.
4. Type x, then b. This causes MacApp to also interrupt the program each time
the previous maximum is surpassed.
5. Type g to resume execution of the program. It will be interrupted
immediately with the debugger window showing a message similar to that in
Example 1.
Example 1: Breaking into the MacApp debugger

 = = New maximum resources usage: 211884 = =
 stopped at Break CHECKRSRCUSAGE Seg#: 1
 Command [BCDEFGHILMNOPpiQRSbetaTWX?]:

This indicates that a new all-time high has been achieved in the amount of
memory required for code and system resources. Each time a new high is
achieved, the program will be interrupted and this message displayed. Now, as
soon as you type g to resume the application, this will probably happen again.
In fact, this may happen several times until the application gets to the point
where it's waiting for a user event.
More Details.
6. Completely exercise the program, trying all its commands and options. What
you're trying to do is find the one point in the application that requires the
greatest amount of code and system resources. For many applications this
occurs at program start-up (when the user double-clicked a document) when
printing, or when quitting and saving a document. Of course, this will vary
greatly depending on the nature of the application, and you should thoroughly
exercise your program to make sure you've found the right place.
7. Now that you know where the high point is, run the program again using the
procedure just described and cause the known high point to be reached. When
the program is interrupted at this point, type the following commands into the
debugger: h, then s. This produces a list of the code segments in use at the
time and looks similar to Example 2. (You can move and resize the debugger
window in order to see the complete list). This list shows the set of segments
that should be included in the seg! resources. A dot in front of a segment
name indicates the segment is resident and is always in memory. An L in front
of the name indicates the segment is locked in memory because it's currently
in use. Note that other segments may be in memory too, but because they're not
in use (and could be purged from memory if necessary) they're not shown. (A
list of all segments, in memory or not, can be shown by typing h and
Option-s.)
Example 2: An example of the segment list displayed by MacApp's debugger

 Command [JBCDEFGHILMOPPiQRSbetaTWX?]: H

 Heap/Stack Cmd [+BDIMRSbeta?]: S
 Total # segments = 32
 * = resident, L = loaded
 $0008A22C Seg#: 1 * Main 24584 bytes
 $0008A184 Seg#: 4 * GDebug 10288 bytes
 $0008A18C Seg#: 6 * GNonRes 10004 bytes
 $0008A190 Seg#: 7 * GFields 16012 bytes
 $0008A19C Seg#: 10 L GInit 6984 bytes
 $0008A1A0 Seg#: 11 * GInspector 15600 bytes
 $0008A1A4 Seg#: 12 L GOpen 16640 bytes
 $0008A1AC Seg#: 14 L GSelCommand 9060 bytes
 $0008A1BC Seg#: 18 * BBRes 228 bytes
 $0008A1C0 Seg#: 19 * GWriteLn 16224 bytes
 $0008A1C4 Seg#: 20 * GDebugger 28932 bytes
 $0008A1C8 Seg#: 21 * GPerformanceTools 5988 bytes
 $0008A1CC Seg#: 22 * GMain 11436 bytes
 $0008A1D8 Seg#: 25 * GRes 31700 bytes
 $0008A1DC Seg#: 26 * ARes 6016 bytes
 $0008A1E0 Seg#: 27 * % MethTables 10940 bytes

 Total loaded code = 220764
 Current temp space: locked = 220764, unlocked = 0, total = 220764
 Command [JBCDEFGHILMOPpiQRSbetaTWX?]: O

You can either write these down or dump the list to a text file using the
debugger's output redirection feature. This is done by typing o, then a file
name such as Segment Listing. Type y to the prompt after entering the file
name to cause debugger output to be displayed in the window while it is
written to the text file. Then type h and s to list the segments again. This
time the list is copied to the text file. Type o to close the output
redirection file, and quit the application (by typing q in the debugger).
With the list of segments in hand, it's a simple matter to add your own seg!
resource to the application. Make sure you assign it a resource ID of 256 or
greater as anything less than 256 is reserved by MacApp. Also, list only those
segments that do not appear in MacApp's seg! lists. Repeating a segment causes
MacApp to reserve space for it each time it occurs in a seg! list.


Accounting for Noncode Usage (or How to Use mem! Resources)


As mentioned earlier, mem! resources can be used to increase the size of the
temporary reserve. Actually, each mem! resource consists of three numbers.
MacApp 2.0b9's mem! resources are shown in Listing Three, page 28. Like seg!
resources, there can be as many mem! resources as needed.

The first value of each mem! resource is added to the size of the temporary
reserve. This lets you account for noncode temporary requests, such as system
resources or system-allocated temporary data structures used for things such
as printing or perhaps complex region manipulation. MacApp includes three mem!
resources of its own. The result is that 4K is always allocated for the
temporary reserve, an additional 40K is added if the application prints, and
another 2K is added if MacApp's debugger is included. Your mem! resource can
reserve additional space if necessary to account for things like fonts, or it
may not need to add anything to this total. (It may take some work to
determine empirically how much your application needs. Of course, the more
your reserve, the safer you are.)
MacApp sums the second values of each mem! resource to determine the size of a
handle that reserves memory for emergency use. The emergency reserve was
alluded to in the algorithm for MacApp's grow zone function. As its name
implies, this handle is purged by the grow zone function as a last resort,
when all other avenues have been exhausted. By definition, once this reserve
is emptied, the application is in a low-memory situation, and MacApp will
display an alert to that effect. In such a low-memory situation, your
application should prevent the user from performing any operations that add to
the application's memory requirements. MacApp's mem! resources reserve 4K for
emergency use. You can add more if necessary.
The third values of the mem! resources are summed to determine the amount of
space for the application's stack, in addition to the 8K automatically
reserved by the Memory Manager. If you need a bigger stack, say because you're
passing large data structures as parameters or defining them as local
variables in procedures, you can do so by supplying the additional amount in a
mem! resource. You can also decrease the stack space by using a negative
number. Normally, it isn't necessary to change the size of the stack.


gApp1MemList and gApp2MemList


The lists gApp1MemList and gApp2MemList were described previously as
containing handles considered to be temporary and subject to MacApp's control.
The difference between the two is that handles in gApp1MemList are purged
before those in gApp2MemList. Typically the handles put in these lists are for
resources. For example, you may want to add the handles to font resources to
gApp2MemList so that the fonts stay in memory for as long as possible.
Normally these lists aren't used. If you want to use of them, you must first
create the list and then assign handles to it with MacApp's AddHandle routine:
 gApp1MemList := HHandle List(NewHandle(0));
 AddHandle(aResource, gApp1MemList);


Handling Low-Memory Conditions


A MacApp program is considered to be in a low-memory condition when its
emergency reserve handle cannot be allocated. In this condition there will
still be some free memory in the heap, giving the application a little
breathing room. In fact, the larger the emergency reserve, the more memory is
in a low-memory condition. Even in a low-memory condition, the application can
continue to request memory and the Memory Manager will oblige until every last
byte of the heap is allocated. After that, further requests will cause the
program to bomb.
Consequently, whenever the program is in a low-memory condition, it is
important that it not perform any operations that increase its memory usage.
To that end, MacApp automatically displays an alert informing the user of the
condition and disables the New and Open commands. You should take similar
precautions of your own. To find out if the application is low on memory, call
MacApp's MemSpaceIsLow function. If it returns true, then you should disable
any operations that increase the memory burden of the program. Obviously, any
commands that add data to a document, such as Paste, should be disabled.
On the other hand, commands that reduce the memory burden should be enabled,
particularly the Clear command. The Cut command may not be effective in a
low-memory condition because it sometimes increases the use of memory. This is
because in addition to copying data from a document or view to the Clipboard,
MacApp makes a copy of the existing Clipboard to make the Cut undoable.
Provided enough space has been reserved for temporary requests, a low-memory
condition will only affect permanent requests (because MacApp releases its
temporary reserve for temporary requests but not for permanent ones). Thus,
MacApp's strategy is geared toward the ultimate failure of permanent requests
as the user fills up memory, so it's important to test the result of every
permanent request to verify its success. Two MacApp routines are useful for
this purpose: FailNIL and FailMemError.
FailNIL has a single parameter, which is a handle, object, or pointer. If the
parameter is NIL, then FailNIL signals failure. FailNIL can be used to test
the result of NEW, NewHandle, NewPermHandle, NewPtr, and NewPermPtr, because
each of these returns NIL if there isn't enough space to complete the
allocation.
FailMemError can be used to test the result of any Memory Manager routine. It
signals failure if the value of the Memory Manager's MemError function returns
a nonzero value. MemError returns the result of the last Memory Manager
routine that was called. This is useful for testing the result of
SetHandleSize, SetPermHandleSize, SetPtrSize, and SetPermPtrSize.


Conclusion


Some people, when they see the amount of work that's gone into MacApp's memory
management scheme, wonder whether it's really necessary. The fact is that if
you want to build a truly robust Macintosh application, then a strategy of
some sort is necessary to address the problems described in this article. By
using MacApp, or its memory management scheme, your application has a proven
strategy in place that's relatively easy to take advantage of.


Bibliography


Inside Macintosh, Volumes I- V. Apple Computer, Cupertino, Calif., 1985-1988.
These are essential for writing any Macintosh program. Of particular interest
to this article are the chapters on the Memory Manager, Resource Manager, and
the Segment Loader.
Knaster, Scott. How to Write Macintosh Software. Rochelle Park, N.J.: Hayden
Book Company, 1986. An extremely useful book. Chapters 2 and 3 have valuable
information about life with the Macintosh Memory Manager.
MacApp, available from APDA at Apple Computer, Inc., 20525 Mariani Ave., MS
33-G, Cupertino, CA 95014-6299; 800-282-2732.


How the Macintosh System Uses an Application's Heap


The Macintosh system uses the application's heap for many purposes. It doesn't
notify the application when this happens, and if there isn't enough space in
the heap for the system's needs, the application will bomb. Here are the most
common system uses of the application heap:
1. The very code an application is made of gets loaded by the system into the
application's heap on a segment-by-segment basis whenever a routine is called
in a segment not currently in the heap.
2. When the application is not run under MultiFinder, the system loads various
system resources into the application's heap when the application uses them.
These include:
Defprocs (short for definition procedures), which implement the basic
low-level behavior of windows, controls, and menus. They take the form of
WDEF, CDEF, and MDEF resources, respectively. There are also LDEF resources
for the List Manager.
Packages, which are collections of code akin to libraries and get loaded into
the application's heap whenever the application requires their services. In
most cases the interface to the routines in a package provides no clue that
those routines are in fact in a package. For example, a simply utility such as
NumToString for converting integers to strings is contained in the
Decimal-to-Binary package.
Fonts used by your application that are not in ROM. Fonts appear as FONT and
FOND resources, and they can take up quite a bit of space.
3. The system uses the application's heap for short periods to maintain
various data structures. These uses include temporary regions for QuickDraw
operations, the creation of file lists for Standard File dialogs, saving the
screen bits behind a pull-down menu, and the allocation of data structures for
printing.
4. If the application is not run under MultiFinder, then desk accessories use
the application's heap for their memory requirements. Even worse, ill-behaved
desk accessories can leave handles in your heap and there's nothing you can do
about it!
-- C.B.











Special Issue, 1989
VISUAL OBJECT-ORIENTED PROGRAMMING


Standard C with home brewed OOP features




Rob Dye


Rob is a software engineer at National Instruments and a member of the LabView
2.0 development team. He can be reached at 12109 Technology Blvd., Austin, TX
78727.


Graphical, direct-manipulation interfaces are quickly becoming the standard
for today's software. Object-oriented programming (OOP) techniques can be
naturally applied to such interfaces, because the on-screen objects that users
see and manipulate can be directly and conveniently implemented as abstract
objects in an OOP language.
LabView is a program with just such an interface. It is a visual programming
and execution environment for data acquisition and laboratory automation.
Functions are wired together in LabView's diagrammatic, dataflow language to
produce executable programs. Graphical display objects provide both
interactive and programmatic input and output for the functions. (See Dr.
Dobb's Software Engineering Sourcebook, Winter 1988, 28-35 for a more detailed
description of LabView.) While LabView itself is not an OOP environment, our
implementation of Version 2 of LabView is very much object-oriented.
One thing noteworthy about the LabView object-oriented implementation is that
the language is standard C-flavored but incorporates our own home brewed OOP
mechanisms. This article describes these mechanisms -- how they were
implemented, how messages are dispatched, how inheritance is achieved, and how
objects are represented.


Why Not a Real OOP Language?


Why don't we just use a real OOP language to reap OOP benefits? Fundamentally
because when development began on LabView 2.0 in mid-1987, no high-performance
OOP language was available on the Mac. Several OOP environments have since
become available, yet we still feel our OOP implementation has advantages over
these others. By building our own OOP features into standard C, we have the
freedom to buy into as much object-orientedness as we need and can afford. We
can leave out those features that we feel negatively affect performance, and
yet build in and fine-tune those that we feel are worth the price of
implementing.
Our implementation of LabView 2.0 embodies two OOP concepts: 1. The close
binding of objects with the methods that operate on them, and 2. the
code-sharing framework accorded by inheritance. These concepts are manifested
in several features: A concise way of sending messages to objects and an
inconspicuous dispatcher for those messages; a mechanism enabling a class to
automatically inherit methods from its superClass; and a class hierarchy that
can be traversed at run time by a second dispatcher to allow a class to
forward messages to its superClass. In implementing these features, we
generally optimized for speed rather than space.
Together, these features give us almost all the benefits that any real OOP
language offers. A few features are missing, though. We have no true data
encapsulation because it must be provided by the language, and C's file
scoping and #include files are minimal encapsulation features. Garbage
collection is sometimes listed as a feature of OOP languages. It is not a
feature of our architecture -- our objects are responsible for cleaning up
after themselves. This is in keeping with the C philosophy that the programmer
retains all the power, not to mention all the responsibility.
A class browser like the one in Smalltalk, although certainly not a
requirement for an OOP language, would be a nice tool for managing our large
collection of classes. To perform this task, we use a group of tools that
treat the OOP system as a matrix, with classes labeling rows and messages
labeling columns. (See the accompanying box, "Managing Class Attributes," for
a discussion of these method table tools.)


The Programmer's View


These OOP mechanisms give us a more powerful language to work with, but its
successful use hinges on programmer discipline. Therefore, before getting into
the details of the implementation, we'll describe how these features appear to
the programmer. For the most part, they look like the familiar features of the
C language.
The template that defines an object's instance variables is created through
the use of nested macros, where the nesting mirrors the class ancestry of the
object. See Listing One (page 35) for an example adapted from LabView's source
code. The example shows the definition of object fields for a class hierarchy
four layers deep; the classes shown are involved in the display of input and
output values to functions. Figure 1 depicts the logical relationships between
these classes, and Figure 2 shows how some instances are graphically
represented in LabView.
At the root of the hierarchy is the data display object (DDO). Front panel
DDOs inherit all of their fields by nesting the DDO_ClassFields macro within
the FPDDO_ClassFields macro, and then supplying a few new fields of their own.
These two classes are examples of abstract classes because no object instances
of these classes are ever actually created; their purpose is to provide a base
class from which many sub-classes may inherit certain methods.
Numeric display objects inherit all the fields of the FPDDO class and supply
those fields specific to numbers, such as the numeric representation, a
digital display object, and its range of valid numbers.
Finally, we see the actual structure definition of several objects with three
typedefs. The standard numeric display object is defined simply in terms of
the NumericDDO_ClassFields. The Knob display object inherits all those fields,
plus some information relating to its graphical depiction and the scale around
its perimeter. The StringDDO inherits from the FPDDO and adds fields necessary
for displaying text.
Once a template for a class has been defined, the programmer must be able to
refer to an object of that class. In LabView, the objects allocated for a
single document (called a "virtual instrument") are contained within a data
structure called an ObjHeap. A particular object is therefore referred to
unambiguously by a pair of values: A handle (a doubly indirect pointer) to an
ObjHeap and an ObjID. The ObjHeap is a relocatable, dynamically sized block of
memory in which objects are allocated. An object's ObjID is its offset from
the beginning of its ObjHeap. (Handles and memory management are discussed
later in this article.)
Because a pair of values is used to refer to an object, accessing an object's
field requires dereferencing the heap's handle, then adding the offset to
yield a pointer to the object's structure. Macros are used to perform these
two steps in a consistent way for each class. The definition of KnobPtr in
Listing One is an example of such a macro. Example 1 shows an example of its
use.
Example 1: Defining the macro

 KnobDDORec *k;
 k = KnobPtr(heap, self);
 k->knobFlags = clockwiseFlag;

Another implication of this two-value identifier is that all messages must
take at least two arguments: An ObjHeap handle and an ObjID. It does not
necessarily mean that all stored references (such as one object holding a link
to another) need to have both values, because the ObjHeap is usually known by
the context. In LabView, the vast majority of inter-object links are within
the same ObjHeap.
One of the features most visible to the programmer is the syntax for sending
messages. The message is not an explicit argument to a function called, say,
MsgDispatch; rather, the message is the actual name of the function that is
called. The message is therefore emphasized, not the dispatching mechanism.
For example, to send a message asking an object to copy itself, you would
write code like this:
 theCopy = oCopy(h, o,...);
The name of the function or method that actually gets invoked need not be
known by the programmer, and frequently it isn't known. The ultimate
destination is determined by the class of the object described by the first
two arguments to the message. The O (for object) at the beginning of the
message name is a convention used to signal those reading the code that this
is no ordinary function call. (It also gives minor amusement to those who read
the message names as holy incantations.)
A similar syntax is used for forwarding a message to an object's superClass:
 theCopy = supCopy(myClass, h, o,...);
The extra argument myClass is used so that the message-dispatching mechanism
can correctly crawl up the class hierarchy tree to the parent class. This
mechanism is more complex than that used for normal messaging, and space
considerations do not permit its description in this article.
Writing a method is no different from writing any other function in C. The
method names, however, adhere to a convention so that the names can be
automatically generated by the method table tools. The name is generated by
the concatenation of the name of the class to which they belong; the character
O; the name of the message to which this method responds; and the word Method.
Example 2 shows such a method.
Example 2: A typical method

 KnobOCopyMethod(heap, self, I)
 OHHandle heap;
 ObjID self;

 ...
 {
 ...
 }



Memory Management of Objects


As mentioned earlier, object instances (Figure 3) physically reside in a data
structure called an ObjHeap (shown in the middle of the figure). ObjHeaps
themselves reside in a data structure, defined by the Macintosh Memory
Manager, called a "zone" (shown at the right side of the figure). ObjHeaps are
relocatable blocks of memory that must be accessed through a nonrelocatable
master pointer. Only one master pointer exists for each relocatable block in a
zone, and it belongs to the Memory Manager. When the block must be relocated,
the Memory Manager updates the master pointer to point to the block's new
location. All other references to the block are handles -- pointers to the
master pointer -- thus, we have double indirection. This arrangement allows
the Memory Manager to compact the zone by moving all the relocatable blocks to
one end of the zone when it becomes fragmented from numerous allocations and
deallocations.
ObjHeaps, therefore, live in the domain of the Macintosh Memory Manager.
Objects within LabView live in the domain of the ObjHeap Manager. LabView's
ObjHeap Manager takes care of object allocation and deallocation within
ObjHeaps. At first glance, this extra layer of memory management software may
seem to be a source of extraordinary overhead, but it actually results in a
significant increase in performance.
One of the lessons we learned (the hard way) from developing LabView 1.0 was
that the performance of the Mac Memory Manager begins to degrade severely when
the number of blocks allocated in a zone gets to be more than a couple of
thousand. This is because allocation of a new block may require searching
extensively through a fragmented zone before finding a free block large enough
to satisfy the memory request. If such a block can't be found, the zone must
be compacted and the search restarted.
All of the OOP languages that have become available recently on the Mac (the
Object Pascal environments, Think C 4.0, and the soon-to-be released MPW C++)
make the same mistake we made in LabView 1.0; each object is allocated in its
own pointer or handle block. Some ambitious programs written in LabView may
require as many as 50 linked documents totaling tens (perhaps hundreds) of
thousands of objects. Our heavy reliance on the Mac Memory Manager degraded
all aspects of performance that relied on memory allocation, even drawing.
Other OOP languages that rely on the Memory Manager can be expected to run
into the same problems.
Adding an extra layer of memory management improves the performance of both
layers. Each layer has to deal with fewer numbers of blocks. And because the
lowest layer of object management is our own, we are free to tweak the ObjHeap
Manager to enhance performance. As mentioned earlier, we avoided the framework
imposed by an OOP language in order to keep this kind of freedom.
More Details.
Another advantage of ObjHeaps has to do with saving all the objects of a
document to disk. As explained earlier, a virtual instrument is in large part
made up of two ObjHeaps. Each ObjHeap can be written out to disk as a single
block. Because inter-object references are simply offsets within an ObjHeap
and not memory addresses, the references need not be encoded or transformed in
any way to survive the move to disk and then back into memory. This goes a
long way towards achieving truly persistent objects, that is, objects that
maintain their identity and interrelationships from one invocation of the
program to another.
One disadvantage of maintaining objects within an ObjHeap is the expense of
calculating a pointer to an object from its ObjHeap handle and offset. This
disadvantage is somewhat exacerbated by the fact that ObjHeaps are
relocatable, which means that a pointer to an object inside an ObjHeap, once
calculated, can go stale across function calls, which can cause memory
relocations. Programmers must be careful to refresh such pointers at the
appropriate times. We consider this a minor penalty, given the massive global
savings in memory management. Judicious use of register variables can further
reduce the penalty.


Representation of Objects


An object has a logical extent that corresponds exactly to the struct that
defines the object's fields. However, an object's physical extent is larger
than its logical extent by three 32-bit integers (plus, perhaps, a few bytes
of internal fragmentation). These three integers are in a header that is at a
negative offset from the logical beginning of the object. They contain system
information that is normally invisible to the programmer: The actual physical
size of the object, a scratch field, and a pointer to the data structure
representing the class of which the object is an instance.
The physical size of the object is the private data of the ObjHeap Manager.
The scratch field is used both by the ObjHeap Manager (for example, during the
compaction of an ObjHeap) and by objects during certain message protocols (for
example, oCopy, oCompile).
The pointer to the class data structure is the most important field for our
purposes in this article. It is the link that binds the object to its
methods..


The defProc


All the defining information about a class is held in a data structure we call
a defProc. For the most part, it is a table of pointers to the class's methods
that is indexed (not searched) by message selectors. defProcs are the source
of a space trade-off inherent in our entire OOP mechanism. The average size
for a defProc is about 600 bytes.
At the beginning of the defProc is the range, which is the maximum selector
value allowed for this class. (See Figure 4.) The range is the offset in the
defProc to the last entry in the table of method pointers -- an error-handling
method. Should an object be sent a message that is beyond its ken, that is, a
message selector that indexes beyond the end of its method table, the
error-handling method is invoked. This method performs much the same function
as does the doesNotUnderstand method in Smalltalk.
Beyond the end of the method table in the defProc is a fixed-length structure
containing a variety of information, including the logical size of an instance
of this class, the ASCII name of the class and its superClass, and a pointer
to the superClass's defProc. The pointer to the superClass's defProc
establishes the inheritance hierarchy and is used at run time to route
superMessages.
defProcs for LabView's built-in classes are created and initialized at LabView
initialization time, not by the compiler at compile time. (The reason is an
unfortunate limit in the amount of static data allowed by the compiler.)
Classes may also be defined externally to LabView; their defProcs are read in
and initialized the first time an object of that class is instantiated. A
description of how external classes are defined is beyond the scope of this
article.
One of the steps in the initialization of a defProc establishes the
inheritance of all methods from its superClass's defProc. In this step, all
the method pointers of the superClass are copied into the defProc being
initialized. Once the pointers are copied, the class's initialization
procedure continues by poking the addresses of its own methods into the
appropriate places in the method table. Because inherited methods are copied
directly into the subClass's defProc, it is not necessary to climb up the
inheritance tree at run time to find the class that defines the method.
You might think that generating and maintaining these defProc initialization
procedures, as well as the indexes in the table for each method, would be
quite a nightmare. It could be, if you did it by hand. Our method table tools
automatically regenerate the source code for these initializers whenever a
change needs to be made in a class methods, or when new messages are
introduced.


Message Dispatching


What path does the code follow in getting from the point of sending an object
a message to the actual execution of that object's method? The path goes
through two functions: One could be called the message glue; the other is the
message dispatcher, an assembly language function called DefProcDispatch. Its
source is shown in Listing Two.
As we saw earlier, sending a message actually calls a function with the same
name as the message -- this is the message glue function. These functions are
quite small (as you can see in Listing Two), and their source is automatically
generated by the method table tools. All the function does is place its
message selector in data register zero and jump to DefProcDispatch.
DefProcDispatch knows the state of the stack upon entry. When the message was
originally sent, the parameters were pushed from right to left (as they appear
in the source code). Therefore the most recently pushed items on the stack
(besides the return address) are the objects ObjID and handle to the ObjHeap.
With these items, DefProcDispatch generates a pointer to the object being sent
the message and retrieves the pointer to the object's defProc from the
object's header. The message selector in DO3 is compared to the range at the
beginning of the defProc to make sure that the defProc contains an entry at
this index in its method table. If all is well, the method address is
retrieved from the table and DefProcDispatch jumps to it.
One of the nice features of this mechanism is that all message traffic goes
through DefProcDispatch. Therefore, it is a convenient place to put all sorts
of debugging hooks. We use it to check the validity of ObjHeap and ObjID
arguments, and to count messages for performance evaluation.


Conclusion


We feel the benefits of object-oriented programming are substantial. Extending
and modifying existing objects, as well as experimenting with new sub-classes,
is quite easy. Building in our own OOP mechanisms has caused a certain amount
of overhead in development time. We had to spend more time on mechanisms
rather than on the code that actually gets the job done, but we feel that the
benefits are worth this overhead.
The experience of both building and using these mechanisms has been
enlightening. We have all come to a greater appreciation of what
object-oriented programming is all about and what kind of design
considerations go into the making of an OOP language. Certainly this education
will be useful to us as we consider the development of future software.


Managing Class Attributes


During the development of a large object-oriented system, class hierarchies
and message protocols are frequently changed. OOP languages generally have
mechanisms that ease these modifications -- Smalltalk's Browser is perhaps the
best example of such a mechanism. With LabView, we had to invent our own
mechanisms external to the language, all of which depend on a spreadsheet that
contains a matrix of objects and messages.

This spreadsheet, which we call the method database, is essentially a
primitive browser. Each row defines a class, each column an attribute of the
class. The various attributes are the class's name, its superClass's name, the
name of the typedef that defines its objects' fields, and so on. In addition
to these attributes there is a column for every message in the system. The
entry at the intersection of a class and a message tells whether the class
defines its own method for that message, inherits a method from its
superClass, responds with an ErrorMethod, or uses a function with a special
name.
A program that we wrote reads the text version of this spreadsheet file and
generates from it six source code files (as well as a number of includes) that
are compiled into LabView. These files include all the class initialization
functions for all the built-in classes in LabView, as well as a function that
calls them in the correct order to assure that no class is initialized before
its superClass. It also generates the message glue functions and the indexes
into the defProc method pointer tables for each message selector.
Modifications to the method database, such as introducing a new message or a
new object, or changing an object's response to a message from inheriting a
method to overriding with its own method, are sufficiently infrequent that the
following three-step process is not too much of a hassle. 1. Modify the
spreadsheet and save as text. 2. Run the source code generator. 3. Remake all
the necessary files. One can easily imagine a program, however, that not only
would allow browsing in an easier way, but also would generate the required
source code. So much code to write, so little time.
--R.D.


























































Special Issue, 1989
WRITING MACINTOSH DEVICE DRIVERS


Here's a template program to get you started




Bryan Waters


Bryan Waters is a software engineer for Maynard Electronics and can be reached
at 460 E Semoran Blvd., Casselberry, FL 32707; phone: 407-263-3574.


Programming on the Macintosh has always been an exercise in research, usually
requiring bits of information from a dozen different sources. Device drivers
provide a good example of this process, and in this article I'll share some of
the information I've gathered and discovered, and describe the structure of a
Macintosh device driver. I'll then present a driver template written in Think
C 4.0.


The Device Manager


A set of system routines called the "Device Manager" gives access to device
drivers on the Macintosh. The calls provided by this Manager define the
application's interface to the drivers. The Manager includes functions for
opening and closing drivers, synchronous and asynchronous I/O, control and
status calls.
Opening and closing drivers are performed by two routines, OpenDriver and
CloseDriver, while the I/O routines are implemented through the FSRead and
FSWrite calls. The Control routine provides an interface for sending control
information to and from the driver, and the Status routine returns status
information about the driver and its current state. (These are all high-level
routines, meaning that the interfaces have been simplified at the expense of
functionality. Each of these routines in turn calls a low-level parameter
block-based routine to perform the desired function.)


Inside Device Drivers


Drivers are used for many purposes: Accessing block devices such as disk
drives, providing a common interface to many different printers, and even for
implementing networks. But drivers aren't always used for talking to a
hardware device. They can also be used for Inter-Application Communication,
and particularly for things such as implementing desk accessories on the
Macintosh.
There are four general classes of drivers on the Macintosh: System drivers,
desk accessories, slot drivers, and device-independent (general usage)
drivers. System drivers are used as network drivers, SCSI drivers, and printer
drivers, and most are stored in the Mac's ROM. Desk accessories, on the other
hand, are a special case of driver designed to be used as "mini-applications"
that can be accessed from any application. Though somewhat limited in size,
desk accessories can have their own menus and windows, and receive and handle
events in a manner similar to applications. But because they must coexist with
applications, desk accessories cannot take the same liberties with the system
as do some applications. NuBus slot drivers are usually loaded into memory
from devices on the NuBus. And general usage drivers, usually not associated
with any hardware, can be used to implement such things as Inter-Application
Communication or a network E-mail system.
Device drivers are accessed through the Unit Table stored in the Macintosh's
system heap. A pointer to the Unit Table is stored in the low-memory global
UTableBase ($11 C), and the size of the Unit Table is stored in the low-memory
global UntryCNt ($1D2). The Unit Table contains entries from which each
specific driver is referenced. When a driver is installed in the Unit Table, a
device control entry structure is allocated in the system heap, and its handle
is installed in the appropriate entry in the unit table. The device control
entry, commonly referred to as the DCE, is then filled out with the driver's
attributes. The format of the device control entry structure is shown in
Example 1. The high-order byte of the field dCtlFlags is set from the
drvrFlags field in the driver's header, while the low-order byte is set up at
driver installation time. The bits are defined in Figure 1.
Example 1: The format of the device-control entry structure

 typedef struct {
 Ptr dCtlDriver ; /* Pointer to ROM driver, or a handle to RAM
 driver*/
 short dCtlFlags ; /* Driver flags */
 QHdr dCtlQHdr ; /* Driver I/O queue header */
 long dCtlPosition ; /* Current position; used by block device
 drivers*/
 } DCtlEntry, *DCtlPtr, **DCtlHandle ;

Figure 1: The bits of the dCtlFlags field

 bit5: set if the driver is open
 bit6: set if the driver is RAM based
 bit7: set if the driver is currently executing

The driver itself is usually stored as a DRVR resource, although it can be
loaded in from a hardware device. The format of the DRVR resource is shown in
Figure 2. Table 1 provides further details on this format.
Figure 2: The format of the DRVR resource

 byte 0:drvrFlags (length 2 bytes)
 byte 2:drvrDelay (length 2 bytes)
 byte 4:drvrEMask (length 2 bytes)
 byte 6:drvrMenu (length 2 bytes)
 byte 8:drvrOpen (length 2 bytes)
 byte 10:drvrPrime (length 2 bytes)
 byte 12:drvrCtl (length 2 bytes)

 byte 14:drvrStatus (length 2 bytes)
 byte 16:drvrClose (length 2 bytes)
 byte 18:drvrName (length 1 byte)
 byte 19:drvrName + 1 (length n bytes)
 .
 .
 .
 byte 19+n:driver routines

Table 1: Detailed description of resource format

 Resource Description
 ----------------------

 drvrFlags The high order byte of the drvrFlags field contains the
 following bit fields:

 dReadEnable (bit 8) is set if the driver can respond to
 Read calls.
 dWriteEnable (bit 9) is set if the dirver can respond to
 Write calls.
 dCtlEnable (bit 10) is set if the driver can respond to
 Control calls.
 dStatEnable (bit 11) is set if the driver can respond to
 Status calls.
 dNeedGoodbye (bit 12) is set if driver needs to call be
 called before the application heap is reinitialized.
 dNeedTime (bit 13) is set if driver needs to be called
 periodically.
 dNeedLock (bit 14) is set if driver should be locked in
 memory.

 drvrDelay This field is used if the dNeedTime bit of the drvrFlags
 field is set. It defines how often the dirver will be
 called to perform periodic actions. This is done by a
 control call with csCode 65.

 drvrEMask This field is the event mask used by desk accessories. This
 contains flags for the desk accessory to define which events
 it can receive.
 drvrMenu This field is also used by desk accessories. If a desk
 accessory has it's own menu, then the menuID of the menu
 is stored here.

 drvrOpen Each of these fields contain an offset to their respective
 routines.
 drvrPrime
 drvrCtl
 drvrStatus
 drvrClose

 drvrName The device driver's name. The name of a non-desk accessory
 device driver always starts with a "." by convention.

Each driver can have five routines to handle calls from the device manager.
The routines are: open, prime, control, status, and close. The open and close
routines perform initialization and clean up functions for the driver. Read
and write calls are handled through the prime routine. The driver's status
routine returns the status of the driver to the device manager, and the
control routine is for control functions pertaining to the driver's task. When
one of the driver's routines is called, a pointer to the parameter block for
that call is passed in implementing function register A0, and a pointer to the
device control entry is passed in A1. The call-specific information is passed
in a parameter block as shown in Example 2.
Example 2: Call specific information

 Control/Status calls


 /* parameter block for Control/Status calls */
 typedef struct {
 QElemPtr qLink; /*Link to next parameter block in
 driver queue*/
 int qType; /*Queue type*/
 int ioTrap; /*Trap to make call;
 PBControl = $A004 */
 Ptr ioCmdAddr; /*Trap address*/
 ProcPtr ioCompletion; /* Completion routines address*/
 OsErr ioResult; /*Result of call*/
 StringPtr ioNamePtr; /*Driver name*/
 int ioVRefNum; /*Volume reference number*/
 int ioRefNum; /*Driver reference number*/
 int csCode; /*Type of Control/Status call*/
 int csParam(11); /*Control/Status information*/
 } cntrlParam;

 Prime (Read/Write) calls

 typedef struct{
 QElemPtr dLink; /*Link to next parameter block in
 queue*/
 int qType; /*Queue type */
 int ioTrap; /*Trap used to make call;
 PBRead = $A002*/
 Ptr ioCmdAddr; /*Trap address */
 ProcPtr ioCompletion; /*Completion routines address */
 OsErr ioResult; /*Result of call */
 StringPtr ioNamePtr; /*Driver name */
 int ioVRefNum; /*Volume reference number */
 int ioRefNum; /*Driver reference number */
 SignedByte ioVersNum; /*Not used */
 SignedByte ioPermssn; /*Read/Write permission (for block
 device drivers)*/
 Ptr ioMisc; /*Not used */
 Ptr ioBuffer; /*Pointer to data buffer */
 long ioReqCount; /*Requested number of bytes */
 long ioActCount; /*Actual number of bytes */
 int ioPosMode; /*Positioning mode (block device
 drivers) */
 long ioPosOffset; /*Positioning offset (block device
 drivers) */
 } ioParam;

The csCode field is used as a selector for requesting a specific function or
specific information from a control/status call. When defining the separate
control calls for a driver, the programmer must take into account that some
csCodes are predefined, such as the accRun (csCode 65), which is used to give
the driver time if the dNeedTime bit is set in the driver's DCE.
The ioPosMode, and the ioPosOffset fields are used for block device drivers to
position the current read/write. The valid modes are fsAtMark, fsFromStart,
fsFromMark. The current position is contained in the driver's dCtlEntryHandle.
If the mode is fsAtMark, ioPosOffset should be ignored and the operation
started at the current position. If it is fsFromStart, or fsFromMark, then
ioPosOffset is added to the beginning of the device, or the current position
respectively, to obtain the starting position for the operation. The constants
used to determine mode are: fsAtMark = 0, fsFromStart = 1, fsFromMark = 3. And
the ioTrap field can be used to determine whether operation is a read or
write, and whether it is synchronous or asynchronous.


I/O Management


I/O requests for drivers are, for the most part, managed by the Device
Manager, which calls the driver at the appropriate time to handle enqueuing
asynchronous requests. A call to the IODone routine informs the Device Manager
that the request was completed. The address of this routine is stored in the
low memory global jIODone at address $8FC. If the request was asynchronous and
our driver was unable to complete the call, then we must exit via an RTS. If
the call was completed we must JMP to the IODone routine, at which point the
Device Manager dequeues the request and calls the request's completion
routine. The Open and Close routines will always be called synchronously.
Two other routines that a driver may use are Fetch and Stash, which are used
as an aid for asynchronous I/O. The Fetch routine, called using the jFetch
vector at address $8F4, simply returns one byte at a time from the request's
data buffer pointed to by ioBuffer, while incrementing ioActCount by one. The
Stash routine is used for Read requests, and simply stuffs bytes into the
request's buffer, incrementing ioActCount by one. Its vector is jStash at
address $8F8. The setup for these calls is listed in Table 2.
Table 2: Setup for I/O calls

 Call Description
 _________________________________________________________________________

 IODone Asynchronous I/O completion call.

 Entry: store pointer to device control entry in A1.

 Fetch Fetches a byte fpr a wrote request (asynchronous only).
 Entry: pointer to device control entry in A1.
 Exit: one byte stored in DO (if bit 15 is set then it is
 the last character in the buffer).

 Stash States a byte for a read request (asynchronous only).
 Entry: pointer to device control entry in A1 byte to be
 stashed.
 Exit: if bit 15 of DO is set then last byte has been stashed.



Device Drivers in Think C


Think C provides a mechanism for developing drivers entirely in C using a stub
to call main() with the parameter block, the DCE, and a selector to specify
the type of call. The prototype for the main routine is:
 int main (cntrlParam *ioBlock_ptr, DCtlPtr dce_ptr, int call_type);
The call_type parameter is used to specify which type of Device Manager call
was actually made to the driver. The body of the driver should be set up as a
switch statement based on call_type, which is shown in Example 3.
Example 3: The body of the driver should be set up as a switch statement based
on call_type

 int main( cntrlParam *io_ptr, DCtlPtr dce_ptr, int call_type )
 {

 switch( call_type ) {
 case 0: /* open */
 case 1: /* prime */
 case 2: /* control */
 case 3: /* status */
 case 4: /* close */
 }
 return result ;
 }

The ioBlock_ptr parameter contains a pointer to the current request parameter
block, and the second parameter is a pointer to the driver's device control
entry. Think C also allows you to declare globals, and when the driver is
loaded, the stub will allocate the memory required and store a handle to it in
the dCtlStorage field of the device control entry. If the stub routine could
not allocate the memory for the globals, then dCtlStorage will be 0, and the
driver's open routine should return a negative error value. It is not
necessary to call the IODone routine, as the driver stub provided with Think C
does this automatically. If an asynchronous request could not be completed,
all that is necessary is to return a 1 to the stub.


Driver Template


The driver presented in this article was written using the Object C extensions
in Think C 4.0. Listing One (page 72) shows driver.h, the header file, while
Listing Two (page 72) lists driver.c, the source file. In order to use the
template, simply declare a subclass of the class driver, and override whatever
methods are necessary (see Example 4). For more information on Object C, see
the Think C User's Guide for Version 4.0. Also, a New() routine must be
provided to allocate the subclass as shown in Example 5.
Example 4: Overriding methods

 struct my_driver:driver {
 int my_storage ;

 /* overridden routines */
 void Open ( ) ;
 void Close ( ) ;
 void Read ( ) ;
 void Write ( ) ;
 } ;

Example 5: A New() routine allocates the subclass

 driver *New( )
 {
 return new( my_driver ) ;

 }

The Fetch() and Stash() routines were provided for use with asynchronous
calls, but should not be used for non-immediate calls. The async field of the
driver class can be used to determine whether a call was asynchronous. After
the driver is completed, all that needs to be done is to install it in the
unit table. Although it sounds simple in theory, this can be as much fun as
writing the driver itself, so I will leave driver installation for another
article.

_WRITING MACINTOSH DEVICE DRIVERS_
by Bryan Waters

[LISTING ONE]


/* driver class declaration */
#ifndef _driver_h_
#define _driver_h_

struct driver : indirect {

 cntrlParam *pb ;/* parameter block */
 DCtlPtr dc ; /* device control entry field */
 int async ; /* this is 1 if the call is asynchronous, else 0 */
 int status ; /* return status */

 /* methods */
 void Open( ) ;
 void Prime( ) ;
 void Read( ) ;
 void Write( ) ;
 void Control( ) ;
 void Status( ) ;
 void Close( ) ;
 void Idle( ) ;
 void Error( int ) ;
} ;

/* generic allocation routine */
driver *New(void ) ;

/* some useful defines */
#ifndef NULL
#define NULL 0L
#endif

/* define jump vectors */
#define jFetch 0x8f4
#define jStash 0x8f8
/*This is not needed since the IODone routine is automatically called by Think
C's driver stub
#define jIODone 0x8fc
*/

/*#define asyncTrpBit 0x0200*/ /* already defined in Think C MacHeaders */
#define ioNotCompleted 1

#endif


[LISTING TWO]

#include <DeviceMgr.h>
#include "driver.h"


#define OPEN 0
#define PRIME 1
#define CONTROL 2
#define STATUS 3
#define CLOSE 4

driver *drvr = NULL ;

int main( cntrlParam *paramBlock, DCtlPtr devCtlEnt, int rout )
{
 int result = 0 ;

 if( rout != OPEN && drvr != NULL ) {
 drvr->pb = paramBlock ;
 drvr->dc = devCtlEnt ;
 drvr->async = paramBlock->ioTrap&asyncTrpBit ; /* is it asynchronous */
 drvr->status = 0 ;
 }else if( rout != OPEN ){
 return badUnitErr ; /* driver not open, but installed */
 }

 switch( rout ) {
 case OPEN:
 if( drvr == NULL ) {
 if( devCtlEnt->dCtlStorage == NULL ) {
 result = -1 ;
 }else{
 drvr = New( ) ;
 if( drvr == NULL ) {
 result = -1 ;
 }else{
 drvr->pb = paramBlock ;
 drvr->dc = devCtlEnt ;
 drvr->Open( ) ;
 }
 }
 }
 break ;
 case PRIME:
 drvr->Prime( ) ;
 break ;
 case CONTROL:
 drvr->Control( ) ;
 break ;
 case STATUS:
 drvr->Status( ) ;
 break ;
 case CLOSE:
 drvr->Close( ) ;
 result = drvr->status ;
 delete( drvr ) ;
 drvr = NULL ;
 break ;
 }

 /* if the driver is open, then return result from the driver */
 if( drvr != NULL ) result = drvr->status ;

 return result ;

}
void driver::Open( )
{
 status = 0 ;
 return ;
}
void driver::Prime( )
{
 if( pb->ioTrap&0x00FF == aRdCmd ) { /* is it a read command */
 Read( ) ;
 }else{
 Write( ) ; /* no so it must be a write command */
 }
 return ;
}
void driver::Read( )
{
 /* override this routine to implement reads from the device */
 /* if the operation is asynchronous, and could not be completed right away,
the
 /* in order to find out whether a call is asynchronous, simply
}
void driver::Write( )
{
 /* override this routine to implement writes to the device */
 /* if the operation is asynchronous, and could not be completed right away,
then
 /* in order to find out whether a call is asynchronous, simpl
}
void driver::Control( )
{
 switch( pb->csCode ) {
 case accRun:
 Idle( ) ;
 break ;
 case goodBye:
 Close( ) ;
 break ;
 }
}
void driver::Status( )
{
 status = 0 ;
 return ;
}
void driver::Idle( )
{

 return ;
}
void driver::Close( )
{
 /* if close is successful, set status to 0, else set status to closeErr(-24)
*/
 /* eg. if( cannot close for some reason ) { Error( closeErr )
 status = 0 ;
 return ;
}
void driver::Error( int err_val )
{
 /* This routine sets status to the error number specified by err_val. Note:
this must be a negative number, preferably
 a valid Macintosh e

 status = err_val ;
}
/* Routine: Fetch

 Description: This routine gets the next character from the ioBuffer and
increments ioActCount by 1. If it is the last
 character to sen

 Result: returns the next character to send to the device.

 Note: this routine should only be used on asynchronous calls

*/
char Fetch( DCtlPtr dc, int *last_char )
{
 int data ;
 asm{
 move.l dc, A1
 move.l jFetch, A0
 jsr (A0)
 move.b D0, data
 }
 *last_char = data & 0x8000 ;

 return data&0x00FF ;
}
/* Routine: Stash

 Description: This routine takes care of stashing a byte of data read from a
device into the ioBuffer
 and incrementing the ioActCount.

 Result: returns true after stashing the last byte requested.

 Note: this routine should only be used on asynchronous calls

*/
int Stash( DCtlPtr dc, char stash_data )
{
 int data ;

 asm{
 move.l dc, A1
 move.b stash_data, D0
 move.l jStash, A0
 jsr (A0)
 move.b D0, data
 }

 return data&0x8000 ;
}


Example 1: The format of the device control entry structure.

typedef struct {
 Ptr dCtlDriver ; /* Pointer to ROM driver, or a handle to RAM driver*/
 short dCtlFlags ; /* Driver flags */
 QHdr dCtlQHdr ; /* Driver I/O queue header */
 long dCtlPosition ; /* Current position; used by block device drivers*/
 } DCtlEntry, *DCtlPtr, **DCtlHandle ;



Example 2: Call specific information

Control/Status calls


/* parameter block for Control/Status calls */
typedef struct{
 QElemPtr qLink; /*Link to next parameter block in driver queue*/
 int qType; /*Queue type*/
 int ioTrap; /*Trap to make call; PBControl = $A004 */
 Ptr ioCmdAddr; /*Trap address*/
 ProcPtr ioCompletion; /*Completion routines address*/
 OsErr ioResult; /*Result of call*/
 StringPtr ioNamePtr; /*Driver name*/
 int ioVRefNum; /*Volume reference number*/
 int ioRefNum; /*Driver reference number*/
 int csCode; /*Type of Control/Status call*/
 int csParam[11]; /*Control/Status information*/
} cntrlParam;


Prime (Read/Write) calls


typedef struct{
 QElemPtr dLink; /*Link to next parameter block in queue*/
 int qType; /*Queue type */
 int ioTrap; /*Trap used to make call; PBRead = $A002*/
 Ptr ioCmdAddr; /*Trap address */
 ProcPtr ioCompletion; /*Completion routines address */
 OsErr ioResult; /*Result of call */
 StringPtr ioNamePtr; /*Driver name */
 int ioVRefNum; /*Volume reference number */
 int ioRefNum; /*Driver reference number */
 SignedByte ioVersNum; /*Not used */
 SignedByte ioPermssn; /*Read/Write permission (for block device drivers)*/
 Ptr ioMisc;/*Not used */
 Ptr ioBuffer; /*Pointer to data buffer */
 long ioReqCount; /*Requested number of bytes */
 long ioActCount; /*Actual number of bytes */
 int ioPosMode; /*Positioning mode (block device drivers) */
 long ioPosOffset; /*Positioning offset (block device drivers)*/
} ioParam;


Example 3: The body of the driver should be set up as a switch
statement based on call_type

 int main( cntrlParam *io_ptr, DCtlPtr dce_ptr, int call_type )
 {
 switch( call_type ) {
 case 0: /* open */
 case 1: /* prime */
 case 2: /* control */
 case 3: /* status */
 case 4: /* close */
 }

 return result ;
 }


Example 4: Overriding methods

 struct my_driver:driver {
 int my_storage ;

 /* overridden routines */
 void Open( ) ;
 void Close( ) ;
 void Read( ) ;
 void Write( ) ;
 } ;



Example 5: A New( ) routine allocates the subclass

 driver *New( )
 {
 return new( my_driver ) ;
 }






































Special Issue, 1989
PERSISTENT OBJECTS


Dealing with storage in Smalltalk




Charles-A. Rovira


Charles is a system designer who can be reached on CompuServe 71230, 1217, BIX
crovira, or care of Adelphi, 3465 Wyman Crescent, Gloucester, Ontario, Canada
K1V OP3.


I recently had to develop several expert systems that, along with a host of
other considerations, had to be able to use files created by existing
applications. The expert system then needed to store the data from those files
in other files that could be shared on a network. This article describes a
technique that I used for building those requirements into my expert systems.
This technique, which uses persistent objects, allows programs to access files
from other applications and then store shareable persistent objects on disk.
Among the other constraints on this project were the type of PCs at my
disposal (a network of XT clones), user-interface considerations, and a
limited amount of time available to create the system. Consequently, I decided
that the best development environment for the job was Smalltalk -- in
particular, Digitalk's Smalltalk/V.
Why is an XT-based system being discussed in a magazine that focuses on
Macintosh programming? Because I do all of my work -- code development,
documentation creation, the writing of user's guides, the works -- on a
Macintosh, using Smalltalk/V Mac. I develop applications on my Mac at home,
copy them to a Compaq portable, and then port them to PCs at the client site
in Smalltalk/V or Smalltalk/V286. This process is convenient for two reasons:
The PC can be used effectively with a decent user interface, and I don't have
to recode my applications (apart from niggling details).


Streams and RecordStreams


Smalltalk is the language that defined all of the relevant concepts of
object-oriented programming and provided the first implementation of that
approach. Fundamentally, Smalltalk remains a single-user, memory-based, single
processor system. Furthermore, Smalltalk's file-handling system is quite
limited, satisfying all of its needs with Streams of bytes.
Most programmers never see beyond this limitation. During the course of their
programming experiences, they've developed the attitude that any language
without unit-record support can't be considered a real computer language. This
common misconception stems from the fact that languages are generally fixed,
and thus are limited by their original definition. Because file I / 0 is
usually the territory of compiler writers, we programmers are stuck with
whatever file-handling capabilities are already provided for us in a language.
What can be a revelation to programmers about Smalltalk is that this program
can be a superb file manager or database manager, if the appropriate objects
-- such as persistent objects -- are defined.
In general, persistent objects can be thought of as files that consist of
collections of bytes, one after another, that are normally read from beginning
to end as Streams of data. Streams may be positioned at any point in the
stream, thereby providing random access. The file-management system built into
Smalltalk handles Streams quite well.
In the simplest kind of persistent object, files are just repositories for
fixed-format data. Each file contains only one type of record. The files are
completely external to memory, and a window, one record wide, is provided for
viewing the files, as illustrated in Figure 1.
The process of extending Streams from files of bytes to files of unit records
is relatively simple. Listing One (page 74) provides all of the code needed to
implement a mechanism to access a file of any record type. This code is safe
to use, as long as you don't position, read, or write by any other methods
than these methods implemented in class RecordStream, which have not been
disinherited. It is still possible to position the file at points other than
at record boundaries. (If you write off boundaries, you will destroy your
file.) This implementation also has no end-of-file handling mechanism in its
file reads, and currently passes end-of-file detection to the atEnd message
that the file inherits from class File. The code is primitive, but usable.


Model Behavior


The definition of RecordStream incorporates a frequently used Smalltalk
concept -- the notion of a model. In a "model," objects manipulate other
objects. By providing the manipulative objects with the means to interrogate
any manipulated object about information relevant to a manipulative object, it
is possible to both make an object perform actions and to cause the object to
act upon any other object.
For example, say we need to read from and write to a file of employee data in
which fields are delimited by commas, and records are delimited by carriage
returns. We can define a model for employee records. The Employee class should
define all of the behavior of an employee, from the date of hire to the date
of termination. Listing Two (page 74) contains a sample class definition for
employees. This Employee model enables us to create an employee file, albeit a
very simple one. Let's say that the employee master for January 1989 is called
EmpMast.891. Access to the file could be performed in this way:
 empfile empRec 
 empfile := File pathName:
 'EmpMast.891' model: Employee
We can read the fifth employee record by sending the recordReadAt: message to
the file, and passing the record number as a parameter: empRec := empfile
recordReadAt: 5.
We read the next employee record with the code: empRec := empfile
recordReadNext.
We can write the fifth employee record by sending the recordWrite:at: message
to the file, and passing the employee to be written and that employee's record
number as the following parameters: empfile recordWrite: empRec at: 5.
We read the next employee record by sending the recordWriteNext: message to
the file, and passing the employee to be written as a parameter: empfile
recordwriteNext: empRec.
Now that we have provided a basic mechanism for dealing with flat files, the
unit record mechanism can be extended to include structured files such as
dBase files, B-tree files, and other allocation and indexing schemes. The use
of a record model class now allows us to define a unit record file as a file
that contains something.


Persistent Collections


A more interesting type of persistent object -- the persistent collection --
allows you to store instances of objects, regardless of their class or size.
This type of persistent object is implemented by extending the collection
classes beyond the bounds of memory. When such an object is accessed, the
collection is loaded into memory, and the instances that make up the
collection remain on disk.
With persistent collections, files can contain variable-length data, and they
are no longer limited to one class of object per file. The files are partially
resident in memory, and a window on the entire file is provided. Only the
specific instances within the collection are disk-resident, as shown in Figure
2.
The position of the class within the hierarchy also means that a fast-running
development version of an application can be produced quickly. You can easily
modify both test and production versions of the application in order to use
external storage. To do so, just change the initialization and termination
methods of the application.
The use of persistent collections lets you create and maintain objects that
can be orders of magnitude larger than the memory available to most computers
(640K in the PC world, and 1 Mbyte in the Macintosh world.) Most real-world
data is larger than a computer's available RAM memory, and these classes
enable Smalltalk to aspire to real-world applications on PCs.
Listings Three and Four (page 74 and 76, respectively) contain a sample class
definition for PersistentArray. The file is accessed in an initialization
routine:
 empPerArr empRec 
 empPerArr := PersistentArray
 open: 'EmpPA.891' of: 10.
After initialization, PersistentArrays are totally transparent and can be used
in the same way as any other array. To access the fifth instance, send the
array the message: empRec := empPerArr at: 5. To add or update the fifth
instance, send the array the message: empPerArr at: 5 put: empRec. This
definition of PersistentArray includes only three methods for managing the
array: at:, at:ifAbsent:, and at:put:.
The rest of the definition contains methods for managing the file-resident
portion of PersistentArray. These methods handle the following activities:

initially binding to a persistent object
optionally requesting a read-only mode
optionally requesting that the integrity of the read-only file be preserved by
coercing any read-write object bound to append any changes
optionally requesting that any changes to the object be posted to specific,
synchronized read-only instances
closing the object
optionally compressing read-write instances
closing all read-only instances of the same object
Because of recovery considerations, instances within a persistent collection
are composed of associations. Should you ever have to rebuild a persistent
object because of media failure or another fatal error, the task is made much
easier by the use of associations.
Of course, extending the flexibility of Smalltalk outside of its own memory
does not come without certain risks and costs. The synchronization between the
memory-resident copy and the instances on disk may be corrupted if one process
updates the object while another process is accessing the object, or if a
failure to return the memory-resident portion of a persistent collection to
disk occurs (because of a system crash, a software error, or whatever).
Also, a collection may no longer fit in the space reserved for it at the front
of the file. The file may contain large gaps, and may need garbage collection,
a simple but time-consuming activity. The entire process and the persistent
object structure can be optimized in order to minimize the need for garbage
collection.


Conclusion


Persistent objects and persistent collections allow access to existing data
and provide the flexibility of Smalltalk, while enabling you to store objects
on disk and even to share objects between applications. The synchronization of
views across operating system task boundaries can be accomplished with some
judicious intertask messaging. The entire problem can also be circumvented
through the use of a server-client mechanism, which, of course, presents its
own subtleties.


Bibliography


Persistent Object Tools. Knowledge Systems Corporation. Suite 270, 2000
Regency Parkway, Cary, NC 27511-8507, 919-481-4000.
Maier, David and Stein, Jacob. "Development and Implementation of an
Object-Oriented DBMS" Research Directions in Object-Oriented Programming,
Shriver, Bruce and Wegner, Peter (eds.), [Boston: MIT Press, 1987 ISBN:
0-262-19264-0, pp. 355-392.
Rovira, Charles-A. "Sequence Intolerance in Expert Applications." AI Expert
4:4 (April 1989): ISSN 0888-3785, pp. 56-59.

_PERSISTENT OBJECTS_
by Charles-A. Rovira

[LISTING ONE]

" *************************************************************************
 This system has been developped for Digitalk's Smalltalk/V. It was
 developed in /V Mac, ported to /V and tested in /V and /V Mac.
 Author: Based on some preliminary code by Steve Northover.
 Packer: This file, documentation and additional methods by Charles-A.
 Rovira.
 Install: fileIn this file.
 Globals: none.
 startUp: none required.
 shutdown: all files should be closed. None enforced.
 Usage: See notes below.
 Class(es): Object, File, RecordStream
 Class: Object
 Method(s): recordToString: stringToRecord: recordSize
 Class: File
 Method(s): path:model:
 Class: RecordStream
 Method(s): model: positionAt:
 recordReadAt: recordReadNext
 recordWrite:at: recordWriteNext
 These routines implement a file management system comparable to
 the COBOL random/sequential file system.
 In order to use unit-record files if is necessary to create a class
 which will respond to at least three messages:
 The first two are class methods:
 recordSize, which answers the size of unit-record on the file..
 stringToRecord, which translates a string loaded from disk into a
 Smalltalk internal representation of an object instance
 and answers an new instance of the modeled object

 the third is an instance method:
 recordToString, which translates the Smalltalk internal representation
 of an object instance into a string to be stored on disk
 The model class can manipulate the records in whatever manner is
 appropriate to the application in addition to these methods. The
 system in a bit fool-proofed in that the Object class defines
 a simple version of these three methods. For IBM card-image files
 it is only necessary to define a class for manipulating or extracting
 information from or putting information onto the card images.
Nota Bene: Files run from 1 to n record. They are not zero based
Examples:
 A unit record file can be accessed as follows:
 turf "TemporaryUnitRecordFile"
 turfRec "TemporaryUnitRecordFile-Record" 
 turf := File pathName: '<file>' model: AnExampleClass
 A unit record file can be read as follows:
 turfRec := turf recordReadAt: anInteger.
 Sequential reading is performed as follows:
 turfRec := turf recordReadNext.
 A unit record file can be written to as follows:
 turf recordWrite: turfRec at: anInteger
 Sequential writing is performed as follows:
 turf recordwriteNext: turfRec.
*************************************************************************** "

!Object methods !
recordToString
 "Private - Lowest level of unit-record file management.
 We assume that the object defining the content of the
 unit-record file will store its string image That's
 what's answered.
 This should be implemented in a unit-record model SubClass"
 ^self storeString! !
!Object class methods !
stringToRecord: aString
 "Private - Lowest level of unit-record file management.
 We assume that the object defining the content of the
 unit-record file is filled by a string image. In order.
 to provide for more flexibility we evaluate the string.
 That's what's answered.
 This should be implemented in a unit-record model SubClass"
 ^Compiler evaluate: aString! !
!Object class methods !
recordSize
 "Private - Lowest level of unit-record file management.
 We assume that the record length will be 80 bytes as
 that has been the standard size to assume for over a
 century.
 This should be implemented in a unit-record model SubClass class"
 ^80! !
!File class methods !
pathName: aString model: aClass
 "Answer a RecordStream. This is the entry point
 of the unit-record file management system. Access files
 by specifying the name of the file to use and the class
 which models the type of objects the file should contain"
 anArray dir file aDirectory aStream 
 "The following code has been duplicated from the Directory class
 to simplify creation of the RecordStream class "

 anArray := self splitPath: aString.
 dir := anArray at: 1.
 file := anArray at: 2.
 aDirectory := Disk.
 dir = '' ifFalse: [
 dir first = $: ifTrue: [
 dir := aDirectory pathName,
 (dir copyFrom: 2 to: dir size)]].
 "The preceeding code was been duplicated from the Directory class
 to simplify creation of the RecordStream class "
 aStream := RecordStream on: (File open: file in: aDirectory).
 ^aStream
 model: aClass! !
FileStream subclass: #RecordStream
 instanceVariableNames:
 'model'
 classVariableNames: ''
 poolDictionaries: '' !

!RecordStream class methods ! !

!RecordStream methods !

model: aClass
 "Private - Set the model for the objects in the file.
 Used only by the pathName:model: in class File "
 model := aClass!
positionAt: anInteger
 "Position the receiver before the object at anInteger.
 Unit-Record files run from 1..n not 0..n
 while not strictly speaking a private method it really
 seves no real purpose outside of the recordReadAt: and
 recordWrite:at: methods in this (ReadStream) class"
 self position: (anInteger - 1 * model recordSize)
recordReadAt: anInteger
 "Answer the unit accessible by the receiver at
 anInteger position in the file. Report an error if
 the receiver stream is positioned at the end. "
 self positionAt: anInteger.
 ^self recordReadNext!
recordReadNext
 "Answer the next record accessible by the receiver
 and advance the stream position. Report an error if
 the receiver stream is positioned at the end."
 bytes 
 self atEnd ifTrue: [
 ^self error: 'Read beyond end of file'].
 bytes := String new: model recordSize.
 CursorManager write showWhile: [
 1 to: model recordSize do: [:i 
 bytes at: i put: self next]].
 ^model stringToRecord: bytes!
recordWrite: anObject at: anInteger
 "Position the receiver before the object at anInteger.
 and write the object onto the file"
 self positionAt: anInteger
 self recordWriteNext: anObject!
recordWriteNext: anObject
 "Write anObject to the receiver stream. Report an error

 if its too big and pad with spaces if its too small"
 bytes size 
 bytes := anObject recordToString.
 bytes size > model recordSize
 ifTrue: [^error: 'record too big'].
 CursorManager write showWhile: [
 self nextPutAll: bytes.
 self next: (model recordSize - bytes size) put: $ .]! !




[LISTING TWO]

" *************************************************************************
A sample class to describe the behavior of any instance of an
Employee object.
*************************************************************************** "

Object subclass: #Employee
 instanceVariableNames:
 'lastName firstName socInsNum'
 classVariableNames: ''
 poolDictionaries: '' !

! Employee class methods !

recordSize
 "we will assume that employees each have:
 20 characters for the lastName
 1 character for the comma
 20 characters for the firstName
 1 character for the comma
 9 characters for the SocInsNum (social insurance number)
 1 character for the Carriage return/Line feed"
 ^52

stringToRecord: aString
 "Answer a new Employee object"
 ^Employee new initializeWith: aString!

! Employee methods !

initializeWith: aString.
 "Fill the receiver from the string"
 aStream 
 aStream := aString asStream.
 firstName := aStream upTo: ',' ;skip: 1.
 lastName := aStream upTo: ',' ;skip: 1.
 socInsNum := aStream upTo: CrLf.

recordtoString: anEmployee
 "answer a string for writing to disk"
 ^firstName, ',' lastName, ',', socInsNum , '\' withCrs! !



[LISTING THREE]


" *************************************************************************
Extending Collections in Smalltalk/V. The PersistentArray
 Install: Class Loader must be installed (See DL/4 in CIS AIExpert forum)
 fileIn this file.
 Globals: Class variable: OpenInstances.
 startUp: PersistentArray initialize
 Install in SystemDictionary start-up
 shutdown: PersistentArray shutdown.
 In /V Mac, regenerate ShutDownList
 Usage: See notes below.
 Class(es): PersistentArray
 Class: PersistentArray
 Method(s): Class -
 initialize
 open:of:
 open:of:readOnly:synchronized:
 shutdown:
 Instance -
 =
 associate:with:
 associationsDo:
 at:
 at:ifAbsent:
 at:put:
 close
 closeReadOnlys:
 coerce
 compress:
 contents
 contents:
 do:
 file:
 fileAppend:
 fileHeader
 fileReadAt:
 fileReadSizeAt:
 fileRemoveAt:
 fileReplace:at:
 fileWrite:of:at:
 includes:
 initHeader
 loadBy:
 loadFrom:as:and:
 loadHeader
 readOnly
 removeAt:notFound:
 reserve:
 synchronize:
 synchronized
 unloadBy:
 These routines extend collections outside the boundaries of Smalltalk
 memory.
 In order to use PersistentArray it is only necessary to create an
 instance of this class by issuing the class message 'open'. It can
 then be used like any other Array until you want to dispose of it.
 Then it must be closed.
Nota Bene: The loader class must allready be present.
Examples:
 A unit record file can be accessed as follows:

 tpa 
 tpa := PersistentArray open: '<file>' of: 10 .
 Access to and from the array is the same as for any other array.
 tpa at: 5.
 tpa at: 5 put anObject
 Instances can be removed by setting them to nil or by explicitely
 requesting a deletion:
 tpa at: 5 put: nil - or -
 tpa removeAt: 5 notFound: [].
 Saving the object to disk is accomplished by:
 tpa 
 tpa close.
*************************************************************************** "

Object subclass: #PersistentArray
 instanceVariableNames:
 'content file lostBytes headSize readOnly synchronized appendsCoerced '
 classVariableNames:
 'OpenInstances '
 poolDictionaries:
 'CharacterConstants ' !

!PersistentArray class methods !

initialize
 "Private - there are no OpenInstances, Make it so."
 OpenInstances := OrderedCollection new.!
open: aFileName of: anInteger
 "Open the persistent object in read/Write mode"
 temp 
 temp := super new.
 temp initialize: anInteger; loadFrom: aFileName as: false and: false.
 ^temp!
open: aFileName of: anInteger readOnly: aBoolean synchronized: aBoolean2
 "Open the Persistent Array
 in whatever mode
 and whatever synchronization"
 temp 
 temp := super new.
 temp initialize: anInteger; loadFrom: aFileName as: aBoolean and: aBoolean2.
 ^temp!
shutdown: aBoolean
 "Private - close OpenInstances"
 OpenInstances do: [:each 
 each close ]! !

!PersistentArray methods !
= aPersistentArray
 "Quickie to compare PAs"
 ^ (file pathName = aPersistentArray file pathName)!
associate: aKey with: aPosition
 "return an association for internal use"
 ^Association key: aKey value: aPosition!
associationsDo: aBlock
 "Answer the receiver. For each element in the receiver,
 evaluate aBlock with that element as the argument."
 index element 
 index := super size.
 [index > 0]

 whileTrue: [
 (element := super at: index) == nil
 ifFalse: [aBlock value: self fileReadAt: element value].
 index := index - 1]!
at: anInteger
 "Answer the value of the key/value pair at anInteger.
 If not found, report an error."
 ^ self at: anInteger ifAbsent:[self errorAbsentElement]!
at: anInteger ifAbsent: aBlock
 "Answer the value of the key/value pair at anInteger
 If not found, evaluate aBlock (with no arguments)."
 answer 
 ^ (answer := content at: anInteger) == nil
 ifTrue: [aBlock value]
 ifFalse: [(self fileReadAt: answer) value]!
at: anInteger put: anObject
 "Answer the object.
 If setting to nil
 remove the object."
 old 
 (anObject == nil)
 ifTrue: [^self removeAt: anInteger
 notFound: [self error: 'Persistent Array boundaries']].
 (old := content at: anInteger) == nil
 ifFalse: [content at: anInteger
 put: (self fileReplace: (self associate: anInteger
 with: anObject)
 at: old)]
 ifTrue:[content at: anInteger
 put: (self fileAppend: (self associate: anInteger
 with: anObject))].
 ^anObject!
close
 "closing the persistent object
 if its readOnly
 close it
 if its readWrite
 close all readOnlys on it
 see if it needs to be compressed
 update the header
 pull it off the OpenInstances OrderedCollection"
 aStream newSize 
 CursorManager execute showWhile: [
 readOnly
 ifTrue:
 [file close]
 ifFalse:
 [self closeReadOnlys: self.
 aStream := WriteStream on: ''.
 Loader new unload: content on: aStream.
 newSize := (2 + content size) * 32.
 (lostBytes > (file size /4) or:
 [headSize < newSize])
 ifTrue:
 [self compress: newSize].
 CursorManager write showWhile:
 [self fileHeader.
 file flush;
 close]].

 OpenInstances remove: self]!
closeReadOnlys: aPersistentObject
 "close read only objects tied to this read/write aPersistentObject"
 OpenInstances do: [:each 
 ((each = aPersistentObject) and:
 [each readOnly ])
 ifTrue: [each close]]!
coerce
 "the read/write instance has appendsCoerced"
 appendsCoerced := true!
compress: newKs
 "copies objects referenced by the old dictionary onto
 a new dictionary"
 window newPersistent newName 
 GrafPort push.
 window := Window dialogBox: (20 @ 50 extent: 450 @ 150).
 'Compressing a Persistent Object...'
 displayAt: 2 @ 2 * SysFont charSize
 font: Font menuFont.
 'CAUTION:'
 displayAt: 2 @ 5 * SysFont charSize.
 'Please do not interrupt this process with Control-Break'
 displayAt: 5 @ 6 * SysFont charSize.
 " not much to this, is there?"
 newName := file pathName.
 newPersistent := PersistentArray open: (newName, '$$$')
 of: (2 * content size).
 newPersistent reserve: newKs.
 self associationsDo: [:anOldAssociation 
 newPersistent at: anOldAssociation key
 put: anOldAssociation value].
 newPersistent file close.
 file close.
 File remove: newName.
 File rename: (newName,'$$$')
 to: newName.
 file reOpen.
 window release.
 GrafPort pop.!
contents
 "Answer the contents for this instance"
 ^content!
contents: anArray
 "This read only object has its indexes updated"
 content := anArray!
do: aBlock
 "Answer the receiver. For each value in the receiver,
 evaluate aBlock with that value as the argument."
 content do: [:each 
 each isNil ifFalse: [
 aBlock value: (self fileReadAt: each) value]]!
file
 "Answer the file for this instance"
 ^file!
fileAppend: anAssociation
 "Answer a position
 find a logical end of file
 derive the size of the unloaded assoclation
 write the resultiing collection and size at the end"

 aPosition aStream aSize 
 readOnly ifTrue: [
 ^self error: 'You cannot update this object'].
 CursorManager execute showWhile: [
 aPosition := file size max: headSize.
 aStream := WriteStream on: ''.
 Loader new unload: anAssociation on: aStream.
 aSize := aStream collection size].
 ^self fileWrite: aStream collection of: aSize at: aPosition!
fileHeader
 "write the header information to disk"
 temp 
 CursorManager write showWhile: [
 temp := Array with: headSize
 with: lostBytes
 with: content.
 file position: 0.
 Loader new unload: temp on: file]!
fileReadAt: aPosition
 "Answer an Association.
 read the size of and the association"
 size 
 CursorManager read showWhile: [
 file position: aPosition.
 size := (Loader new loadFrom: file) "asInteger".
 ^Loader new loadFrom: file]!
fileReadSizeAt: aPosition
 "Answer the size.
 read the size of the association which follows"
 CursorManager read showWhile: [
 file position: aPosition.
 ^(Loader new loadFrom: file) asInteger]!
fileRemoveAt: aPosition
 "Lose the bytes"
 self readOnly
 ifTrue: [^self error: 'You cannot update this object'].
 lostBytes := lostBytes + self fileReadSizeAt: aPosition.!
fileReplace: anAssociation at: aPosition
 "Answer aPosition or the logical end of file
 derive the size of the unloaded association
 look up the old size on disk
 if its still fits
 write it in place
 else
 find the logical end of file
 append it"
 aStream newSize oldSize aNewPosition 
 "(anAssociation isKindOf: Association)"
 readOnly
 ifTrue: [^self error: 'You cannot update this object'].
 aStream := WriteStream on: ''.
 Loader new unload: anAssociation on: aStream.
 newSize := aStream size.
 oldSize := self fileReadSizeAt: aPosition.
 (oldSize < newSize or: [appendsCoerced])
 ifTrue:
 [lostBytes := lostBytes + oldSize.
 aNewPosition := file size max: self header.
 ^self fileWrite: aStream collection

 of: newSize
 at: aNewPosition]
 ifFalse:
 [lostBytes := lostBytes + (oldSize - newSize).
 ^self fileWrite: aStream collection
 of: oldSize
 at: aPosition].!
fileWrite: aCollection of: bytes at: aPosition
 "write anAssociation size and anAssociation"
 CursorManager write showWhile: [
 file position: aPosition.
 Loader new unload: bytes printString
 on: file.
 file nextPutAll: aCollection.
 "file nextPutAll: '%'."
 file flush].
 ^aPosition!
includes: anObject
 "Answer true if the receiver contains the key/value
 pair whose value equals anAssociation, else answer false."
 self do: [ :element 
 (self fileReadAt: element) value = anObject
 ifTrue: [^ true]].
 ^ false!
initHeader
 "initialize the persistent object header"
 headSize := (2 + content size) * 32.
 lostBytes := 0.
 self reserve: headSize.!
initialize: anInteger
 "initialize the persistent object collection"
 content := Array new: anInteger!
loadBy: aLoader
 "Write out the instance variables of the receiver
 using the loader object aLoader."
 self error: 'Can''t (un)load a Persistent object (its pointless)'!
loadFrom: aFileName as: readOnlyStatus and: synchronizedStatus
 "initialize or load the Persistent object
 as readWrite [check for reopening of readWrite] or
 readOnly [as
 requiring appendsCoerced or
 requiring synchronization] "
 temp 
 file := (File pathName: aFileName).
 (readOnly := readOnlyStatus)
 ifFalse: [OpenInstances do:
 [:each (each file pathName = file pathName and:
 [each readOnly = false])
 ifFalse: [
 ^self error: 'Cant open: ',aFileName, ' twice for update']]]
 ifTrue:
 [File primitiveChangeModeOf: aFileName to: 1.
 "Change to read only mode. # from 'Inside Macintosh'"
 appendsCoerced := synchronizedStatus not.
 (synchronized := synchronizedStatus)
 ifFalse:
 [self openInstances do: [:each 
 each file pathName = file pathName and:
 [each readOnly not]]

 ifFTrue: [each coerce]]].
 OpenInstances add: self.
 file size = 0
 ifTrue:
 [self initHeader]
 ifFalse:
 [self loadHeader].
 ^self!
loadHeader
 "get the persistent object header from disk"
 temp 
 CursorManager read showWhile: [
 file position: 0.
 temp := (Loader new loadFrom: file).
 headSize := temp at: 1.
 lostBytes := temp at: 2.
 content := temp at: 3]!
readOnly
 "Answer the readOnly for this instance"
 ^readOnly!
removeAt: anInteger notFound: aBlock
 "Answer anInteger. Remove the key/value pair at anInteger
 If such a pair is not found, evaluate aBlock
 (with no arguments)."
 (content at:anInteger) == nil
 ifTrue: [^ aBlock value].
 self fileRemoveAt: (content at: anInteger)
 content at: anInteger put: nil.
 ^anInteger!
reserve: newSize
 "Answer newSize.
 Reserve newSize bytes for the persistent collection.
 Pad out the file as required."
 fileSize 
 headSize := newSize.
 fileSize := file size.
 1 to: (newSize-fileSize) do: [:junk file nextPut: Space "$!"].
 ^newSize!
synchronize: aPersistentObject
 "find the read instance(s) and synchronize keys"
 self openInstances do: [:each 
 each = aPersistentObject and:
 [each readOnly and:
 [each synchronized]]
 ifTrue: [each contents: aPersistentObject contents]]!
synchronized
 "answer synchronized"
 ^synchronized!
unloadBy: aLoader
 "Write out the instance variables of the receiver
 using the loader object aLoader."
 self error: 'Can''t (un)load a Persistent object (its pointless)'! !



[LISTING FOUR]

" *************************************************************************
Extending Collections in Smalltalk/V. The Loader

 This system has been developed from Digitalk's Smalltalk/V loader.
 The Loader, as defined in Smalltalk/V, required two modifications
 to operate in the /V Mac environment.
 The original /V Loader class is on the AIExpert forum on CompuServe
 in Data Library 4.
 Install: fileIn this file.
 Globals: none.
 startUp: none
 shutdown: none
 Usage: Internal to Persistent objects.
*************************************************************************** "

Object subclass: #Loader
 instanceVariableNames:
 'stream loaderIndex objectNumber loader loaderQueue classDict '
 classVariableNames: ''
 poolDictionaries:
 'CharacterConstants ' !

!Loader class methods ! !

!Loader methods !

classIndexFor: aClass
 "Unloading - Answer the string for the class of the next
 object to be unloaded."
 index 
 index := classDict at: aClass
 ifAbsent: [
 classDict at: aClass put: objectNumber.
 ^aClass name].
 ^'%',index printString!
getClass
 "Loading - Answer the class of the next object in the file"
 classString char index 
 (char := stream next) == $%
 ifTrue: [^(loader at: (stream upTo: Lf) asInteger) class].
 classString := (String with: char) , (stream upTo: Lf).
 ^Smalltalk at: classString asSymbol!
getSize
 "Loading - Answer the next object size"
 ^(stream upTo: Lf) asInteger!
load: anObject
 "Loading - Load the next object from the file"
 index 
 index := 1.
 anObject class isPointers
 ifTrue: [
 [index <= loaderIndex]
 whileTrue: [
 anObject instVarAt: index put: self nextInstVar.
 index := index + 1].
 ^self].
 anObject class isBytes
 ifTrue: [
 [index <= loaderIndex]
 whileTrue: [
 anObject at: index put: stream next asciiValue.
 index := index + 1].

 stream next.
 ^self].
 anObject class isWords
 ifTrue: [
 [index <= loaderIndex]
 whileTrue: [
 anObject at: index put:
 stream next asciiValue * 256 + stream next asciiValue.
 index := index + 1].
 stream next.
 ^self]!
loaderIndex
 "Loading - Indicates the size of the next object in the file.
 Unloading - Used as an object reference pointer"
 ^loaderIndex!
loadFrom: aStream
 "Loading - Load objects from aStream and return root"
 numOfObjects index anObject 
 stream := aStream.
 numOfObjects := (stream upTo: Lf) trimBlanks asInteger.
 loader := Array new: numOfObjects.
 index := 1.
 [index <= numOfObjects]
 whileTrue: [
 loader at: index put: String new.
 index := index + 1].
 index := 1.
 [index <= numOfObjects]
 whileTrue: [
 anObject := self loadInstance.
 (loader at: index) become: anObject."become: (loader at: index). ? ->
backwards ?"
 (loader at: index) loadBy: self.
 index := index + 1].
 loader do: [:each each rehash].
 ^loader at: 1!
loadInstance
 "Loading - Create an empty instance of the next object
 in the file"
 class 
 class := self getClass.
 loaderIndex := self getSize.
 class isVariable
 ifTrue: [^class basicNew: loaderIndex - class instSize]
 ifFalse: [^class basicNew]!
nextInstVar
 "Loading - Answer the next instance variable from the stream"
 char ptr answer classString size 
 char := stream next.
 char == $%
 ifTrue: [
 ptr := stream upTo: Lf.
 ptr = 't' ifTrue: [^true].
 ptr = 'f' ifTrue: [^false].
 ptr = 'n' ifTrue: [^nil].
 ^loader at: ptr asInteger].
 char == $-
 ifTrue: [^(stream upTo: Lf) asInteger negated].
 char isDigit
 ifTrue: [^((String with: char), (stream upTo: Lf)) asInteger].

 char == $$
 ifTrue: [
 answer := stream next.
 stream next.
 ^answer].
 char == $#
 ifTrue: [^(stream upTo: Lf) asSymbol].
 char == $!!
 ifFalse: [
 classString := (String with: char),
 (stream upTo: Lf).
 size := classString size.
 (size > 6
 and: [(classString copyFrom: size - 5 to: size) = ' class'])
 ifTrue: [
 classString := classString
 copyFrom: 1 to: size - 6.
 ^(Smalltalk at: classString asSymbol) class].
 ^Smalltalk at: classString asSymbol].
 stream peek == $!!
 ifTrue: [
 answer := (Compiler evaluate: stream nextChunk)
 fileInFrom: stream]
 ifFalse: [answer := Compiler evaluate: stream nextChunk].
 ^answer!
stream
 "Answer the stream of the receiver"
 ^stream!
unload: anObject
 "Unloading - unload anObject to the stream"
 size index 
 size := anObject class instSize + anObject basicSize.
 stream
 nextPutAll: (self classIndexFor: anObject class);
 nextPut: Lf;
 nextPutAll: size printString;
 nextPut: Lf.
 index := 1.
 anObject class isPointers
 ifTrue: [
 [index <= size]
 whileTrue: [
 stream nextPutAll: (self unloadIndexFor:
 (anObject instVarAt: index)).
 index := index + 1.
 stream nextPut: Lf].
 ^self].
 anObject class isBytes
 ifTrue: [
 [index <= size]
 whileTrue: [
 stream nextPut: (anObject at: index) asCharacter.
 index := index + 1].
 ^stream nextPut: Lf].
 anObject class isWords
 ifTrue: [
 [index <= size]
 whileTrue: [
 stream nextPut: ((anObject at: index) // 256) asCharacter.

 stream nextPut: ((anObject at: index) \\ 256) asCharacter.
 index := index + 1].
 ^stream nextPut: Lf]!
unload: anObject on: aStream
 "Unload anObject on aStream"
 class oldPos 
 oldPos := aStream position.
 (anObject isKindOf: Behavior)
 ifTrue: [^self error: 'cannot have ', anObject name, ' as root'].
 class := anObject class.
 (class == UndefinedObject
 or: [class == Symbol
 or: [class == Character
 or: [class isKindOf: Boolean]]])
 ifTrue: [^self error: 'cannot have ', class name, ' as root'].
 stream := aStream.
 loaderIndex := 1.
 objectNumber := 1.
 loader := IdentityDictionary new.
 classDict := IdentityDictionary new.
 loaderQueue := OrderedCollection new.
 stream
 nextPutAll: ' ';
 nextPut: Lf.
 loader at: anObject put: 1.
 loaderQueue addLast: anObject.
 [loaderQueue isEmpty]
 whileFalse: [
 loaderQueue removeFirst unloadBy: self.
 objectNumber := objectNumber + 1].
 aStream
 position: oldPos;
 nextPutAll: ((loaderIndex printString , ' ')
 copyFrom: 1 to: 5);
 flush!
unloadIndexFor: anObject
 "Unloading -- Answer the external string representation for
 anObject used in the unload stream."
 tempInt 
 (anObject isKindOf: Behavior)
 ifTrue: [^anObject name].
 anObject == nil
 ifTrue: [^'%n'].
 anObject == true
 ifTrue: [^'%t'].
 anObject == false
 ifTrue: [^'%f'].
 (anObject isKindOf: Integer)
 ifTrue: [^anObject printString].
 anObject class == Character
 ifTrue: [^String with: $$ with: anObject].
 anObject class == Symbol
 ifTrue: [^'#',anObject].
 tempInt := loader at: anObject
 ifAbsent: [
 loaderIndex := loaderIndex + 1.
 loaderQueue addLast: anObject.
 loader at: anObject put: loaderIndex].
 ^'%', tempInt printString! !

!Object methods !
loadBy: aLoader
 "Load the instance variables of the receiver using
 the loader object aLoader."
 aLoader load: self!
unloadBy: aLoader
 "Write out the instance variables of the receiver
 using the loader object aLoader."
 aLoader unload: self! !
!SortedCollection methods !
unloadBy: aLoader
 "Write out the instance variables of the receiver
 using the loader object aLoader. Convert the
 receiver to an OrderedCollection since blocks
 of code cannot be loaded or unloaded."
 self asOrderedCollection unloadBy: aLoader! !
!String methods !
loadBy: aLoader
 "Load the instance variables of the receiver using
 the loader object aLoader."
 aStream index size 
 size := aLoader loaderIndex.
 aStream := aLoader stream.
 index := 1.
 [index <= size]
 whileTrue: [
 self at: index put: aStream next.
 index := index + 1].
 aStream next!
unloadBy: aLoader
 "Write out the instance variables of the receiver
 using the loader object aLoader."
 aLoader stream
 nextPutAll: (aLoader classIndexFor: self class);
 nextBytePut: 10;
 nextPutAll: self basicSize printString;
 nextBytePut: 10;
 nextPutAll: self;
 nextBytePut: 10! !
!Object methods!
rehash
 "Rehash the receiver. the default is do nothing."! !
!Set methods!
rehash
 "Rehash the receiver."
 aSet 
 aSet := self species new: self basicSize.
 self do: [ :element aSet add: element].
 ^self become: aSet! !
!Dictionary methods!
rehash
 "Rehash the receiver."
 aDictionary 
 aDictionary := self class new: self basicSize.
 self associationsDo: [ :anAssociation 
 aDictionary add: anAssociation].
 ^self become: aDictionary! !
!IdentityDictionary methods!
rehash

 "Rehash the receiver."
 aDictionary 
 aDictionary := self species new.
 self associationsDo: [ :anAssociation 
 aDictionary add: anAssociation].
 ^self become: aDictionary! !
























































Special Issue, 1989
WIZARDCOPY FOR FAST BACKUPS


Fast disk backups save time and money




Don Gaspar


Don is a physicist and a senior software engineer for DIALOG Information
Services, Inc. in Palo Alto. He can be reached at 3460 Hillview Ave., Palo
Alto, CA 94304.


It's 8 a.m., and you need ten copies of a program you've been writing to hand
out to beta testers. The Finder is just too slow when copying and formatting
entire disks: A Kobayashi Maru scenario won't work here. Besides, you need the
disk now, not next year. So you try your disk copier, only to find it doesn't
allow you to make multiple copies. Oh no!
You decide that you'll make single copies by reading the disk into memory,
writing the copy to another disk, and then repeating both steps again, and
again. You know this process will be extremely time consuming, but the only
alternative is using the Finder, which will require more disk swaps and mouse
clicks than you can count. You launch your copier and find that it doesn't
support the new Macintosh 1.44-Mbyte disks. Nuts!
So you start the single-copy process, knowing full-well it's going to require
ten steps of swapping and clicking and almost as many minutes for each of ten
disks. Holy Bazookas! What are you going to do?
How about writing your own disk copier, over which you'll have complete
control? Too hard? No way.
This article explains how to write a sector-copying disk utility for your
Macintosh. A sector copier is a program that reads entire sectors on a disk
into a designated area in RAM, and then later writes them from RAM to a
destination drive. The data of all your programs, files, and utilities is
stored on your media in this way. The sector copier will read a disk into
memory, sector by sector (which is 512 bytes at a time), and then write this
data to the destination disk. Reading a sector at a time has the advantage of
speed since you're reading/writing sequences of 512 bytes from each read/write
process. The C source code for WizardCopy, a disk copier I wrote, is shown in
Listing One, page 80. (The Rez file that implements the WizardCopy application
is available on CompuServe, the DDJ on-line service, and on disk.) Programs
like WizardCopy are timesavers when it comes to multiple copies; I just made
15 copies of a beta version of a program I'm writing to send to beta testers,
and the whole copy process took only a few minutes! I would have aged a year
if I'd had to make the copies by other means.
The programming techniques presented in this article illustrate Macintosh
fundamentals for using the Device Manager and the Disk Driver. You'll quickly
understand these important fundamentals by looking through the source code in
Listing One. No more clicking the mouse again and again just to format a disk.
No more dragging disks' icons to other icons and waiting. How about
auto-formatting with minimal input? How about copying the disk in 17 to 45
seconds (depending on whether the disk is 400K, 800K, or 1.44-Mbyte)? These
speed advantages are another reason to make a sector copier; the more copies
you have to make, the better the performance gain will be.


The WizardCopy Program


WizardCopy is a sector-copying utility that will copy 400K, 800K, and
1.44-Mbyte disks for your Macintosh. It requires minimal input, is extremely
fast, and has only four controls (see the main window in Figure 1).
The check boxes let you tell WizardCopy to warn you before copying over an
existing disk, and whether or not to format every disk that you insert. These
items are simple to implement and are extremely useful. The check boxes are
represented in WizardCopy as Boolean flags that are either on or off; simple
semantic logic dictates how the program behaves after checking these flags.
This alert was written as a separate dialog box because it requires a response
of Yes or No, and the other dialog box offers only an acknowledgement.
For example, if you click 'Format All,' all the destination disks are
automatically formatted. If 'Warn Before' is active and you insert a
destination disk that already has data on it, WizardCopy will warn you, and
ask if you wish to proceed.
The 'New Master' and 'Quit' buttons are the other controls, and are also
available as commands in the File Menu. The 'New Master' button (or command)
tells WizardCopy that you would like to copy a new disk. WizardCopy will, of
course, free as much space as possible in the heap, and prompt you to insert
the disk. When you want to copy another disk, select this option again. The
Quit button (or command) does what its name implies: It frees all memory that
was used by WizardCopy and disposes of all windows, menus, and so on. It then
returns you to the Finder. The WizardCopy window also contains four status
items: A progress meter at the bottom of the window, and boxes labeled Status,
Copies Made, and In Memory. The progress meter (perhaps more decorative than
functional, some might argue) tells you how far along you are when reading and
writing sectors. The light gray rectangle (see Figure 1) fills to solid black
when the copying is done. This meter is simple; it uses a fundamental
algebraic relation between the number of sectors on the disk and the size of
the rectangle.
The progress meter is advanced each time through a read loop or a write loop
(which read and write ten 512-byte sectors, respectively). The right side of
the rectangle is equal to the left side plus the quantity of the sector just
read (or written) times the length of the rectangle divided by the number of
sectors.
The meter is implemented with Quick-Draw. To draw the meter, WizardCopy gets
the dialog user item (ID #11, in this case), fills it with a light gray
pattern, and then frames it. Another rectangle is created, equal to the first
one, but its right side is calculated by the equation just described -- it's
filled with black. Every time a sector (or ten sectors, in this case) is read,
the counter is incremented and the rectangle is drawn -- an accurate copy
progress meter.
It may be easier for some of you to write this meter as a control definition
procedure, which will have many advantages over the technique outlined here.
No matter which technique you use, the progress meter will have the same look
and will perform virtually the same way.
The 'Status' item tells the user what's going on. It displays the messages
"Writing, Reading, Formatting, Please insert a master disk," and so on. It
doesn't serve as an alert, but more as a message center for the basic tasks
our program executes. The strings for these operations are stored as a STR#
resource, and each is individually indexed.
The 'Copies Made' item tells the user exactly how many copies of a disk they
have made. This feature is useful when you need several copies and aren't
counting each one. A global integer, nCopies, is incremented each time a
successful copy is made; it's cleared when 'New Master' is selected, as the
counter must then be reset.
The 'In Memory' item displays the name of the disk being copied. If there is
no disk in RAM, WizardCopy displays the string 'Nothing' in this box. This
item is extremely helpful when you are making copies of multiple disks and
you're not sure what you're copying: just look at the In Memory item, and
you'll see the disk in action.
The Wizard icon in Figure 1 has been added to the program for aesthetic
purposes only, and can be removed from the resource fork if you don't like it.
I think it makes the program look great. (It was drawn by artist
extraordinaire Danny Green.) The alert boxes are as simple as possible.
The text for the alerts and the main window is stored as an STR# resource, and
accessed via the procedure Get-IndString. Remember that we're in the Macintosh
world, and all the strings we're working with are Pascal strings. To use this
alert for other problems that may occur, I simply get the appropriate string,
and then do a GetDItem followed by a SetIText to change the text to the
desired result. This dual operation is in a routine called SetDText, which is
short for Set Dialog Text. Having the strings in this format also simplifies
translating WizardCopy to foreign languages -- just access WizardCopy's
resource fork!


Event Loop


The main window is actually a modeless dialog box, which simplifies our coding
slightly. The event loop is structured as normal, and a function called
IsDialogEvent has been added. If Get-NextEvent returns TRUE, we then test
IsDialogEvent. IsDialogEvent will tell us if an event has occurred in our
dialog box; we can then take appropriate action. If IsDialogEvent is TRUE, we
then call DialogSelect.
DialogSelecttakes the appropriate action and tells us which item has been
selected. If DialogSelect is TRUE, we then call DoDialog. The events for the
dialog box are intercepted before going to DoDialog, which then processes the
required response -- a simple filtering operation. Two events are intercepted:
updateEvt and diskEvt.
The updateEvt is intercepted only because the dialog box has some nice boxes
around some of the status items. Intercepting the event here, we call our own
update procedure, which both updates the dialog box and draws our text.
For the copy process, the disk events are also intercepted. The two possible
levels are reading or writing disks, and are identified by two flags. A
function called DoDisk executes the appropriate action based on the level. For
example, if DoDisk is in masterLevel, it will know it is to copy a master
disk; after copying the disk, it moves into destLevel (destination level),
where it writes the copied disk to a destination volume. If New Master was not
selected, WizardCopy will then eject all inserted disks.
The event loop for other events, like menus or desk accessories, is then
processed as usual Listing One).


Formatting Disks


Formatting disks is a simple operation. Make a control call with csCode equal
to 6, and send the reference number of the disk driver (-5). We also tell the
driver what type of disk to format. For a single-sided or a double-sided disk,
we make csParam equal to either 1 or 2; for the 1.44-Mbyte disks, I had no
idea what to do, so I ran The Debugger (by Jasik Designs) while in the Finder,
and set a trap intercept at Control and Status. The calls for formatting
1.44-Mbyte floppies are identical to those for formatting 400K disks. The
Macintosh apparently realizes we have an HD floppy, and formats accordingly.
When formatting a 400K or 800K disk, I always format it as 80OK; I have found
no physical difference between the two types of disk except for the price.
Save some money and buy 400K disks, and use them as 800K disks -- you should
have no problems. You can also save by purchasing bulk SS disks and formatting
them as DS disks. The reason manufacturers differentiate between the two is
that it's cheaper to produce one kind of disk and label the disks either SS or
DS, rather than producing two different kinds of disks in the same factory.
Try formatting disks this way for yourself, and see if you ever have an
abnormally high number of failures.
The formatting routine is called DoFormat, and it works in conjunction with a
routine called CheckDisk. CheckDisk merely checks the format of the disk being
copied, so that we can allocate RAM and do other tasks.



Reading and Writing Disks


The calls for reading and writing disks are similar to the formatting calls,
except that we use some low-level read and write calls in the File Manager.
Essentially, we set up an ioParamBlock: We specify where to start the
read/write process, how many bytes to read/write, a buffer pointer for the
data, and the disk driver reference number (-5, again). When we receive a disk
event, DoDisk checks the appropriate level (masterLevel or destLevel) and
executes a read or write call.
WizardCopy executes a loop and reads/writes ten 512-byte sectors at a time.
The progress meter can thus be implemented accurately. We could have made one
read/write call and done the entire process in one swing, but it wouldn't have
been as exciting or as user friendly.
If you would like to modify the progress meter for greater accuracy, you can
have WizardCopy read a sector at a time (this will cause it to run a bit more
slowly). The routines ReadFloppy and WriteFloppy accomplish this task; see
Listing One.


Possible Enhancements


WizardCopy could use more error trapping. Error detection is provided in every
instance where it would be important, but it has not been implemented and
intercepted in each case, because of some laziness on my part. You can easily
see where to add your own dialog boxes. Or you can simply add strings in the
STR# resource and call the biker dialog box with strings added in it.
You could make WizardCopy check to see if the latest version of the disk
driver is available on the machine you're using. It would also be nice to see
if you're in System 6.0.3 or a later version. You can put this routine in the
procedure CheckThings.
WizardCopy reads and writes an entire disk in RAM in one pass; and will tell
users of smaller machines that more RAM is needed. You can easily make
WizardCopy a multiple-pass copier by putting some of the routines in a loop,
and repeating the loop n times until n passes are accomplished. The next
version of WizardCopy will include this feature.


References


Macintosh Developer Tech. Note #70 Venus Flytrap preliminary notes (available
from DTS) Inside Macintosh, Volumes I-III.


Acknowledgments


The products used to make this program were The Debugger, by Jasik Designs,
and THINK C. Thanks to my associate with Autosoft, artist extraordinaire,
Danny Green. The author is also heavily indebted to Macintosh Developer
Technical Support at Apple Computer, Inc. Not only are they technical wizards
that help, but they've also put up with stupid questions I've asked time and
time again. In particular, the author is indebted with soul to wizard Dennis
Hescox.


Modifying WizardCopy for Hard Disks


Kenneth Turner
While making multiple copies of a floppy diskette using the approach Don
presents here is certainly a handy utility, it does require that your Mac has
at least 2 Mbytes of RAM or that you modify the WizardCopy program for
multiple-passes (for single floppy drive systems). Besides being more complex,
such a modification requires more disk swapping (an undesirable side effect).
One alternative is to change the WizardCopy code so that you use a hard drive
instead of using valuable RAM to store the data from the master disk.
Modifying the existing programming scheme to do this involves little more than
using the Macintosh's File Manager in place of many Memory Manager routines.
Actual code modifications are simple and straightforward. Allocate space on
the hard drive volume instead of in memory. When the master floppy is read,
transfer the data to the newly created hard drive file instead of to memory.
Finally, when the duplicate diskettes are to be written, read the data from
the hard drive instead of from RAM.
The first change you can make to WizardCopy is to the MakeRAM routine. The new
code should create a work file on the hard drive with the File Manager
function Create. This replaces the call to the Memory Manager function NewPtr.
Remove the calls to DisposPtr, FreeMem, and CompactMem. These functions
managed the allocation of memory and reduced heap fragmentation. Instead, open
the file with FSOpen and use the function Allocate to ensure there is enough
free space on the hard drive volume. (Though hard drive fragmentation is also
an important issue, it is not crucial in this application and will be
ignored.)
In the existing ReadFloppy and WriteFloppy routines, a FORloop controls the
transfer of the master data to and from a 5K-byte buffer. A hard drive version
will still maintain this smaller, temporary data area. Another FORloop,
however, controls the transfer of data between the large, master buffer and
this intermediate buffer. Replace this loop with a single call to either
FSRead or FSWrite, depending on the situation. These, obviously, interface
with the master data file instead of the old master RAM buffer.
The Memory Manager requires careful maintenance of pointers and handles.
Though there is not a direct correspondence, the File Manager demands its own
bookkeeping. The disk copier needs to keep track of the access path number for
the work file it creates and the volume reference number of the hard drive
volume. These enable the program to precisely specify to the operating system
the file containing the master floppy data.
The volume reference number can be obtained with the GetVolfunction. The path
reference number is returned when the file is opened.
Also, WizardCopy must always know the position of the file mark. This is the
logical location of the last read or write. Whenever a new master diskette is
to be read or a new duplicate diskette is to be written, the mark must be set
to the beginning of the file with SetFPos. Finally, when the program is
finished, the file should be closed with FSClose and deleted from the volume
with FSDelete.
To put the finishing touches on the hard drive version, small changes should
be made to the user interface. Replace strings such as "In Memory" with "On
Disk" and change warnings such as "You need memory" to "The hard drive is
full."
With this modification, the door is now open to other attractive features. For
instance, the program could keep track of several master diskettes at once,
instead of having to reread a floppy that was duplicated previously. A library
of virtual master diskettes could be maintained on the hard drive and recalled
as needed. Or, the program might be modified to work in the background under
MultiFinderwithout requiring the heap space used by other concurrent
applications. This simple hard drive option is only the first step toward a
comprehensive disk copier.
Ken is a design engineer for Rodime Systems, Boca Raton, Fla., makers of hard
disk drives for the Mac. He can be contacted through DDJ's office.
--K.T.


_WIZARDCOPY FOR FAST BACKUPS_
by Don Gaspar


[LISTING ONE]

/*
 * WizardCopy v1.0.0 by Don Gaspar
 * A disk copier for the Apple Macintosh
 */

 #include <ControlMgr.h>
 #include <DeviceMgr.h>
 #include <DialogMgr.h>

 #include <DiskDvr.h>
 #include <EventMgr.h>
 #include <FileMgr.h>
 #include <FontMgr.h>
 #include <ListMgr.h>
 #include <MacTypes.h>
 #include <MemoryMgr.h>
 #include <MenuMgr.h>
 #include <OSUtil.h>
 #include <pascal.h>
 #include <Quickdraw.h>
 #include <ToolboxUtil.h>
 #include <WindowMgr.h>
 #include <math.h>

 #define beginMenu 880
 #define endMenu 882
 #define mainDialog 880
 #define ourStrings 880
 #define aboutDialog 881
 #define badDisk 882
 #define dataDialog 883
 #define useVol 8000
 #define nil 0L
 #define false 0
 #define true 1
 #define watchCursor 4

 #define newMaster 1
 #define aboutMe 1
 #define quit 2
 #define mQuit 3
 #define inMemory 3
 #define copiesMade 4
 #define status 5
 #define nothing 6
 #define none 7
 #define waiting 8
 #define pictItem 9
 #define warnBefore 10
 #define statMeter 11
 #define progress 12
 #define alwaysFormat 13

 #define SSDD 1
 #define DSDD 2
 #define MFM 3

 /* globals */
 Handle myMenus[3]; /* array of our menus */
 Boolean done = false, format = false, warn = true,
 masterLevel = false,destLevel = false;
 DialogPtr wizDialog;
 Ptr data = nil; /* here's the disk data */
 EventRecord theEvent; /* main event record */
 Point dummyPt;
 Str255 defaultName,volumeName;
 short temp,diskKind,nCopies = 0;
 long sectors; /* #of sectors of copying disk */

 int errno; /* because I don't want stdio */

 /* sets the text of our main dialog to the specified string */
void SetDText(item, text)
 short item;
 Str255 *text;
{
 short type;
 Handle h;
 Rect r;
 GetDItem(wizDialog,item,&type,&h,&r);
 SetIText(h,text);
}/* SetDText */

/* changes the status text in the main dialog box */
void SetAllDText(item1,item2,item3)
 short item1,item2,item3;
{
 Str255 myStr;/* a pascal string */
 if (item1 != useVol) {
 GetIndString(&myStr,ourStrings,item1);
 SetDText(nothing,&myStr);
 }
 else
 SetDText(nothing,&volumeName);
 NumToString((long)nCopies,&myStr);
 SetDText(none,&myStr);
 GetIndString(&myStr,ourStrings,item3);
 SetDText(waiting,&myStr);
}/* SetAllDText */

 /* simple routine to center the window; will make
 it visible if showIt is true */
 void CenterWindow(theWindow,showIt)
 WindowPtr theWindow;
 Boolean showIt;
{
 Point centerScreen,centerWind;
 Rect toRect;
 centerScreen.h = screenBits.bounds.right/2;
 centerScreen.v = screenBits.bounds.bottom/2;
 centerWind.h = (theWindow->portRect.right -
 theWindow->portRect.left)/2;
 centerWind.v = (theWindow->portRect.bottom -
 theWindow->portRect.top)/2;
 MoveWindow(theWindow,centerScreen.h - centerWind.h,
 centerScreen.v - centerWind.v,false);
 if (showIt)
 ShowWindow(theWindow);
}/* CenterWindow */

/* use this for updating the wizard dialog when the
 conditions are that an update event is not posted */
void DrawWizDialog()
{
 GrafPtr oldPort;
 SetAllDText(2,3,1);
 GetPort(&oldPort);
 SetPort(wizDialog);

 InvalRect(&(wizDialog->portRect));
 SetPort(oldPort);
}/* DrawWizDialog */

/* ejects the disk and unmounts the volume */
OSErr MyEject(drive)
 short drive;
{
 OSErr err;
 short vRef;
 Str255 name;
 long dummy;
 err = GetVInfo(drive,&name,&vRef,&dummy);
 err = UnmountVol(&name,vRef);
 err = DiskEject(drive);
 return(err);
}/* myEject */

/* tells you that you inserted a bad disk */
void BadDisk(index)
 short index; /* this is the string index # */
{
 DialogPtr badBox;
 short item,type;
 GrafPtr oldPort;
 Handle h;
 Rect r;
 Str255 myStr;
 GetPort(&oldPort); /* get the current port */
 badBox = GetNewDialog(badDisk,nil,((WindowPtr)-1));
 GetDItem(badBox,3,&type,&h,&r);
 GetIndString(&myStr,ourStrings,index);
 SetIText(h,&myStr);
 CenterWindow(badBox,true);
 SysBeep(3);
 SetPort(badBox); /* set current port */
 do
 ModalDialog(nil,&item);
 while (item != 2);

 DisposDialog(badBox); /* trash it since we're done with it */
 SetPort(oldPort); /* set it back */
 DrawWizDialog();
}/* BadDisk */

/* allocate sufficient space for copy */
OSErr MakeRAM(format)
 short format;
{
 long free,sizo;
 OSErr err;
 if (data != nil)
 DisposPtr(data);
 sizo = (format == MFM) ? 1474560:819200;
 sectors = sizo/512; /* globally keep track of this */

 if((long)FreeMem() < sizo)
 free = (long)CompactMem((long)1474560);/* ask for large block */


 data = NewPtr(sizo);
 err = MemError();
 if (err != noErr) {
 BadDisk(20);
 (void)EjectAllDisks();
 masterLevel = true;
 destLevel = false;
 DisposPtr(data);
 }/* if */
 return(err);
}/* MakeRAM */

/* disk has data on it. Proceed? */
Boolean DataAlert()
{
 DialogPtr box;
 short item;
 GrafPtr oldPort;

 GetPort(&oldPort); /* get the current port */
 box = GetNewDialog(dataDialog,nil,((WindowPtr)-1));
 CenterWindow(box,true);
 SetPort(box); /* set current port */
 do
 ModalDialog(nil,&item);
 while (item!=1 && item !=2);
 DisposDialog(box); /* trash it since we're done with it */
 SetPort(oldPort); /* set it back */
 return((item == 1) ? true : false);
}/* DataAlert */

/* formats the disk in the desired format */
OSErr CheckDisk(drive,format)
 short drive,*format;
{
 OSErr err;
 cntrlParam db;
 DrvSts sts;
 db.ioVRefNum = drive;
 db.ioCompletion = nil;
 db.csCode = 10; /* we want to inspect the disk and the drive */
 db.ioRefNum = -5; /* the disk driver */
 err = PBStatus(&db,false);
 *format = (db.csParam[0] == -1 && db.csParam[1] == -1)
 ? MFM : nil;
 if (*format != MFM) { /*call was invalid, get disk format */
 err = DriveStatus(drive,&sts);
 *format = (sts.twoSideFmt == -1) ? DSDD : SSDD;
 }/* if ... */
 return(err);
}/*DoFormat */

/* formats the disk in the desired format */
OSErr DoFormat(drive,format)
 short drive,format;
{
 OSErr err;
 cntrlParam db;
 int *dummy;

 db.ioVRefNum = drive;
 db.ioCompletion = nil;
 db.csCode = 6; /* we want to format the disk */
 db.ioRefNum = -5;
 dummy = &db.csParam;
 *dummy = (format == MFM) ? 1:format;/* gotta format it right */
 err = PBControl(&db,false);
 return(err);
}/*DoFormat */

/* Get the default disk back */
void GetDefaultVol()
{
 done = true;
 DisposDialog(wizDialog);
 if (data != nil)
 DisposPtr(data);
}/* GetDefaultVol */

/* toggle between off and on depending what iut is */
void ToggleItem(aWindPtr,item,new)
 WindowPtr aWindPtr;
 short item;
 Boolean new;
{
 ControlHandle h;
 Rect aRect;
 short type,val;
 GetDItem(aWindPtr,item,&type,&h,&aRect);
 SetCtlValue((ControlHandle)h,(GetCtlValue(
 (ControlHandle)h) ? false : true)); /* on-off or off-on */
 if (new)
 SetCtlValue((ControlHandle)h,true);
}

/* disable appropriate menu items */
void SetMenus()
{
 short index;
 DisableItem(myMenus[0],2);
 DisableItem(myMenus[1],2);
 for (index=0;index<8;index++)
 DisableItem(myMenus[2],index);
}/* SetMenus */

/* read an entire floppy into RAM */
OSErr ReadFloppy(drive)
 short drive;
{
 OSErr err;
 ioParam ioStuff;
 short index,i,delta;
 Rect r,x;
 Handle h;
 short kind;
 GrafPtr oldPort;
 GetDItem(wizDialog,statMeter,&kind,&h,&r);
 delta = r.right - r.left;
 x = r; /* this is our status rect. */

 x.right = r.left;
 GetPort(&oldPort);
 SetPort(wizDialog);

 ioStuff.ioVRefNum = drive; /* the drive # */
 ioStuff.ioReqCount = 5120; /* read 10 sectors at a time */
 ioStuff.ioMisc = nil;
 ioStuff.ioRefNum = -5; /* the disk driver */
 ioStuff.ioBuffer = NewPtr(5120); /* this is a temp. ptr*/
 ioStuff.ioPosOffset = 0; /* start at beg. of disk */
 ioStuff.ioPosMode = fsFromStart;

 for (index=0;index<sectors/10;index++) { /* read the disk */
 err = PBRead(&ioStuff,false); /* read it */
 x.right = floor((double)delta*(double)index/(double)(sectors/10))
 +r.left;
 FillRect(&x,&black);
 for(i=0;i<5120;i++)
 data[ioStuff.ioPosOffset+i] = ioStuff.ioBuffer[i];
 ioStuff.ioPosOffset += 5120; /* advance 512 bytes */
 }/* for... */
 DisposPtr(ioStuff.ioBuffer);
 FillRect(&r,&ltGray);
 FrameRect(&r);
 SetPort(oldPort);
 return(err);
}/* ReadFloppy */

/* write an entire floppy from RAM to a dest. disk */
OSErr WriteFloppy(drive)
 short drive;
{
 OSErr err;
 ioParam ioStuff;
 short index,i,delta;
 Rect r,x;
 Handle h;
 short kind;
 GrafPtr oldPort;
 GetDItem(wizDialog,statMeter,&kind,&h,&r);
 delta = r.right - r.left; /* diff. between left and right sides */
 x = r; /* this is our status rect. */
 x.right = r.left;
 GetPort(&oldPort);
 SetPort(wizDialog);

 ioStuff.ioVRefNum = drive; /* the drive # */
 ioStuff.ioReqCount = 5120; /* write 10 sectors at a time */
 ioStuff.ioMisc = nil;
 ioStuff.ioRefNum = -5; /* the disk driver */
 ioStuff.ioBuffer = NewPtr(5120); /* this is a temp. ptr*/
 ioStuff.ioPosOffset = 0; /* start at beg. of disk */
 ioStuff.ioPosMode = fsFromStart;

 for (index=0;index<sectors/10;index++) { /* write the disk */
 for(i=0;i<5120;i++) /* accuarate control meter */
 ioStuff.ioBuffer[i] = data[ioStuff.ioPosOffset+i];
 err = PBWrite(&ioStuff,false); /* write it */
 x.right = floor((double)delta*(double)index/(double)(sectors/10))

 +r.left;
 FillRect(&x,&black);
 ioStuff.ioPosOffset += 5120; /* advance 512 bytes */
 }/* for... */
 DisposPtr(ioStuff.ioBuffer);
 FillRect(&r,&ltGray);
 FrameRect(&r);
 SetPort(oldPort);
 return(err);
}/* ReadFloppy */

/* pop out all disks in all drives */
OSErr EjectAllDisks()
{
 OSErr err;
 DrvSts sts;
 short drive;
 for(drive=1;drive<3;drive++) {
 err = DriveStatus(drive,&sts);
 if (err == noErr)
 if (sts.diskInPlace > 0) /* is it there? */
 (void)MyEject(drive); /* pop it out */
 }/* for */
 return(err);
}/* EjectAllDisks */

/* which drive was the disk inserted into? */
short WhichDrive()
{
 DrvSts sts;
 OSErr err;
 short drive,index;
 do {
 err = DriveStatus(index,&sts);
 if (sts.diskInPlace>0) /* is it there? */
 drive = sts.dQDrive; /* this drive has the disk */
 } while(drive<0);
 return(drive);
}/* whichDrive */

/* returns the name of the inserted disk */
OSErr GetDiskName(drive,name)
 short drive;
 Str255 *name;
{
 OSErr err;
 long free;
 short vRef;
 err = GetVInfo(drive,name,&vRef,&free);
 return(err);
}/* GetDiskName */

/* we have to handle activate events too! */
void DoActivate(myEvent)
 EventRecord myEvent;
{
 WindowPtr targetWP;
 targetWP = (WindowPtr)myEvent.message;
 if (targetWP != FrontWindow())

 SelectWindow(targetWP);
 SetPort(targetWP);
}/* DoActivate */

/* set the text to tell you what's up */
void SetTheText(item,strInd)
 short item,strInd;
{
 Rect r;
 Handle h;
 short type;
 Str255 theStr;
 GetDItem(wizDialog,item,&type,&h,&r);
 GetIndString(&theStr,ourStrings,strInd);
 SetIText(h,&theStr);
}/* SetTheText */

/* here's the simple line drawings */
void DrawPseudoBoxes(r,x)
 Rect r;
 short x;
{
 MoveTo(r.left-3,r.top+7);
 LineTo(r.left-3,r.bottom+20);
 LineTo(r.right,r.bottom+20);
 LineTo(r.right,r.top+7);
 LineTo(r.left+x,r.top+7);
 MoveTo(r.left-3,r.top+7);
 LineTo(r.left,r.top+7);
}/* DrawPseudoBoxes */

/* is the disk write protected?*/
Boolean WriteProtected(drive)
 short drive;
{
 DrvSts sts;
 OSErr err; /* for err handling to be added later */

 err = DriveStatus(drive,&sts);
 return(BitTst(&sts.writeProt,(long)7)); /* is it? */
}/* WriteProtected */

/* simple QuickDraw McGraw stuff to make main dialog look good */
void BoxDialogThings()
{
 short type,item;
 Handle h;
 Rect r;

 GetDItem(wizDialog,inMemory,&type,&h,&r);
 DrawPseudoBoxes(r,72);
 GetDItem(wizDialog,copiesMade,&type,&h,&r);
 DrawPseudoBoxes(r,85);
 GetDItem(wizDialog,status,&type,&h,&r);
 DrawPseudoBoxes(r,50);
 GetDItem(wizDialog,statMeter,&type,&h,&r);
 FillRect(&r,&ltGray);
 FrameRect(&r);
}/* BoxDialogThings */


/* handle disk event */
OSErr DoDisk(message)
 long message;
{
 OSErr err;
 Boolean wp,flag = false;
 short drive,kind;
 wp = WriteProtected(LoWord(message));
 drive = LoWord(message);
 if (masterLevel) { /* are we reading a master disk? */
 if (HiWord(message) != noErr && HiWord(message) != volOnLinErr)
 BadDisk(19);
 else if (wp) {
 nCopies = 0;
 CheckDisk(drive,&diskKind);
 masterLevel = false; /* don't copy unless it's wp! */
 destLevel = true;
 (void)GetDiskName(drive,&volumeName);
 SetAllDText(useVol,3,16);
 if (MakeRAM(diskKind,data) == noErr) {
 ReadFloppy(drive);
 SetAllDText(useVol,3,12);
 }/* if MakeRAM ... */
 else
 SetAllDText(2,3,11);
 }/* else if !... */
 else
 BadDisk(6);
 (void)MyEject(LoWord(message));
 }/* if */
 else if (destLevel) { /* are we making copies??? */
 if ((HiWord(message)) == noErr && !wp ) {
 if (warn) {
 if (!DataAlert())
 flag = true;
 }
 if (!flag){
 CheckDisk(drive,&kind);
 if (kind != diskKind format) {
 if (kind != MFM format) {
 SetAllDText(useVol,3,17);
 (void)DoFormat(drive,diskKind);
 }
 else
 BadDisk(4);
 }
 }/* else... */
 }/* if... */
 else if ((format (HiWord(message) != noErr)) && !wp) {
 SetAllDText(useVol,3,17);
 (void)DoFormat(drive,diskKind);
 }/* if format */

 if (!wp && !flag) {/* write data to disk */
 SetAllDText(useVol,3,18);
 (void)WriteFloppy(drive);
 nCopies++; /* let's keep track here */
 }

 else if (wp)
 BadDisk(7);
 (void)MyEject(drive);
 SetAllDText(useVol,3,12);
 }/* else if */
 return(err);
}/* DoDisk */

/* handle the new master disk inserted */
void DoNewMaster()
{
 (void)EjectAllDisks();
 SetAllDText(2,3,11);
 masterLevel = true;
}/* DoNewMaster */

/* simple dialog box */
void DoAbout()
{
 DialogPtr aboutBox;
 short item;
 GrafPtr oldPort;
 GetPort(&oldPort); /* get the current port */
 aboutBox = GetNewDialog(aboutDialog,nil,((WindowPtr)-1));
 CenterWindow(aboutBox,true);
 SetPort(aboutBox); /* set current port to ours */
 ModalDialog(nil,&item);
 DisposDialog(aboutBox); /* trash it since we're done with it */
 SetPort(oldPort); /* set it back */
}/* DoAbout*/

/* process menu items that were selected */
void DoMenu(code)
 long code;
{
 short menuNum,itemNum;
 Str255 name;
 short temp;
 menuNum = HiWord(code);
 itemNum = LoWord(code);
 if (itemNum > 0) {
 switch (menuNum) {
 case beginMenu:
 switch (itemNum) {
 case aboutMe :
 DoAbout();
 break;
 default :
 GetItem(myMenus[0],itemNum,&name);
 temp = OpenDeskAcc(&name);
 }/* switch itemNum */
 break;
 case beginMenu+1:
 switch (itemNum) {
 case newMaster :
 DoNewMaster();
 break;
 case mQuit :
 GetDefaultVol();

 }/* switch itemNum */
 break;
 case endMenu:;
 }/* switch menuNum */
 HiliteMenu(false);
 }/* if */
}/* DoMenu */

/* update main window */
void DoUpdate(myEvent)
 EventRecord myEvent;
{
 WindowPtr tempPort,aWindPtr;
 aWindPtr = (WindowPtr)myEvent.message; /* get window */
 if (aWindPtr == wizDialog) {
 SetCursor(*GetCursor(watchCursor)); /* hold on a sec */
 GetPort(&tempPort);
 SetPort(aWindPtr);
 ForeColor(blueColor);
 TextFont(geneva); /* easy on the eyes */
 ForeColor(blackColor);
 BeginUpdate(aWindPtr);
 EraseRect(&aWindPtr->portRect); /* clean it out */
 DrawDialog(wizDialog); /* and redraw */
 BoxDialogThings();
 EndUpdate(aWindPtr);
 SetPort(tempPort);
 TextFont(systemFont); /* back to correct system font */
 InitCursor(); /* back to the arrow */
 }/* if */
}/* DoUpdate */

/* handle things for our main dialog */
void DoDialog(item)
 short item;
{
 switch (item) {
 case newMaster : /* a master disk was inserted, do it! */
 DoNewMaster();
 break;
 case quit : /* we're done */
 GetDefaultVol();
 break;
 case alwaysFormat : /* automatically format all disks */
 ToggleItem(wizDialog,alwaysFormat,false);
 format = (format==true) ? false : true; /* toggle */
 break;
 case warnBefore : /* warn before formatting an existing disk */
 ToggleItem(wizDialog,warnBefore,false);
 warn = (warn==true) ? false : true;/* toggle */
 }/* switch */
}/* DoDialog */

/* handle the key events */
void DoKeyEvent(myEvent)
 EventRecord myEvent;
{
 char theKey;
 long item;

 if (myEvent.modifiers & cmdKey) {
 theKey =
 myEvent.message & charCodeMask; /* which key */
 if (item = MenuKey(theKey))
 DoMenu(item); /* do it */
 }
}/* DoKeyEvent */

/* handle the mouse down event */
void DoMouseDown(myEvent)
 EventRecord myEvent;
{
 short windowLoc;
 Point mousePos;
 WindowPtr aWindPtr;
 Rect r;
 mousePos = myEvent.where;
 windowLoc = FindWindow(mousePos,&aWindPtr);
 switch (windowLoc) {
 case inMenuBar :
 DoMenu((long)MenuSelect(mousePos));
 break;
 case inSysWindow :/* for those pesky DAs' */
 SystemClick(&myEvent,aWindPtr);
 break;
 case inDrag :
 SetRect(&r,screenBits.bounds.left+4,screenBits.bounds.top+24,
 screenBits.bounds.right-4,screenBits.bounds.bottom-4);
 DragWindow(aWindPtr,mousePos,&r);
 break;
 case inContent :
 if (FrontWindow() != aWindPtr)
 SelectWindow(aWindPtr);
 break;
 default :;
 }/* switch */
}/* DoMouseDown */

/* Init. mgrs., allocate space, etc. */
 void InitThings()
 {
 InitGraf(&thePort);/* init appropriate mgrs */
 InitFonts();
 InitWindows();
 InitMenus();
 TEInit();
 InitDialogs(nil);
 InitCursor();
 MoreMasters(); /* get master pointers */
 MoreMasters();
 MoreMasters();
 MaxApplZone();
 FlushEvents(everyEvent,0); /* clear event queue */
 wizDialog = GetNewDialog(mainDialog,nil,((WindowPtr)-1));
 CenterWindow(wizDialog);

 DrawWizDialog();
 }/* InitThings */


 /* Handle any event that occurred */
 void HandleEvent(myEvent)
 EventRecord *myEvent;
 {
 switch (myEvent->what) {
 case mouseDown :
 DoMouseDown(*myEvent);
 break;
 case activateEvt :
 DoActivate(*myEvent);
 break;
 case updateEvt :
 break;
 case keyDown :
 case autoKey :
 DoKeyEvent(*myEvent);
 break;
 case diskEvt :
 (void)DoDisk(myEvent->message);
 }/* switch */
 }/* HandleEvent */

 /* remember the default volume */
 void RememberDefault()
 {
 short vRef;

 (void)GetVol(&defaultName,&vRef);
 }/* RememberDefault */

 /* check system, ram, etc. */
 Boolean CheckThings()
 {
 return(true);
 }/* CheckThings */

/* Setup menus, window, controls, etc. */
 void SetUpThings()
 {
 short index;
 for (index=beginMenu;index<endMenu+1;index++) /* get menus */
 myMenus[index-beginMenu] = GetMenu(index);
 AddResMenu(myMenus[0],'DRVR'); /* add desk accessories */
 for (index=beginMenu;index<endMenu+1;index++)
 InsertMenu(myMenus[index-beginMenu],0);
 DrawMenuBar();
 ToggleItem(wizDialog,warnBefore,true);
 SetMenus();
 }/* SetUpThings */

 void main()
 {
 InitThings();
 if (CheckThings()) {
 SetUpThings();
 RememberDefault();
 while (!done) {
 if (GetNextEvent(everyEvent,&theEvent))
 if (IsDialogEvent(&theEvent)) {

 if (theEvent.what == updateEvt)
 DoUpdate(theEvent);
 else if (theEvent.what == diskEvt) {
 if (masterLevel destLevel)
 DoDisk(theEvent.message);
 else
 (void)EjectAllDisks();
 }/* else if ... */
 else if (DialogSelect(&theEvent,&wizDialog,&temp))
 DoDialog(temp);
 }/* if */
 else
 HandleEvent(&theEvent);
 }/* while */
 }/* if CheckThings */
 else {
 }/* else */
 if ( data != nil )
 DisposPtr(data);
 }/* main */










































Special Issue, 1989
OBJECT C AND THE MACINTOSH CONTROL PANEL


Object-oriented cdev development




Bryan Waters


Bryan Waters is a software engineer for Maynard Electronics and can be reached
at 460 E Semoran Blvd., Casselberry, FL 32707.


With the addition of object-oriented extensions to Think C in Version 4.0,
Think Technologies adopted the notion of "Object C," which combines the
functionality of Object Pascal with the syntax of C++. While Object C supports
the declaration of classes, sub-classes, inheritance, and methods, it does not
support function and operator overloading, constructors and destructors, data
hiding through the use of public and private keywords, or in-line functions.
In this article, I'll show how Object C can be used to simplify the
development of Macintosh resources, and I'll use a control panel device as an
example.
A class in Object C is declared (as shown in Example 1) where class_name is
the name of the class, and storage_class can be either direct or indirect.
This determines whether the object is allocated as a pointer or a handle.
Example 1: Declaring an Object C class

 struct class_name : storage_type {
 variable and/or method declarations
 .
 .
 .
 } ;

To declare a method, simply include the routine's prototype in the class
declaration. Subclasses are defined using the syntax shown in Example 2.
Example 2: Declaring an Object C subclass

 struct subclass_name:class_name {
 variable and/or method declaration ;
 /* NOTE: in order to override a method in the class,
 simply redeclare the method in the ~ subclass */
 } ;

Methods are defined by using the class_name combined with the method name,
separated by the scope operator': :'. The object itself is passed implicitly
to the method, and can be referenced either explicitly by using the keyword
this, or implicitly by referencing the field or method directly. (All of the
new keywords are interpreted in context, and do conflict with variable naming;
for example, you can have a non-object variable named this, although I
wouldn't recommend it.)
Objects are allocated and destroyed using the functions new()and delete().
Memory that was not allocated using these functions can be treated like an
object, by using the functions bless() and blessD(), respectively, for
pointers and handles. The final function is a member(), which is used to
determine whether an object is a member of a specific class.


Control Panel Devices


With the release of the Macintosh SE, the Control Panel desk accessory became
extendible through the use of cdev, a control panel device. This allowed
programmers to add their own utilities to the control panel, without using a
precious desk accessory slot (currently a maximum of 15 desk accessories is
allowed). The cdev takes the form of a resource file that is placed in the
Macintosh's system folder. When the Control Panel DA is opened, it searches
the system folder for all files of type cdev, and adds them to its list. A
cdev must have a list of mandatory resources before being adopted by the
control panel. This list includes the 'cdev' code resource (an icon used to
represent itself in the control panel's list), a dialog item list and 'nrct'
resource (which determine the cdev's interface), and a 'mach' resource (a
bitmask used to determine which Mac the 'cdev' should appear on; for example,
a 'cdev' to change menu bar colors, would not be appropriate for a Macintosh
without color QuickDraw). The 'cdev' resource contains the code to implement
the cdev and is called from the control panel in the format shown in Example
3.
Example 3: Format for calling a cdev

 pascal long cdev ( message, Item, numItems, CPanelID, ev,
 cdevValue CPDialog )
 int message, Item, numItems, CPanelID;
 EventRecord *ev ;
 long cdevValue ;
 DialogPtr CPDialog ;

The control panel communicates with the 'cdev' by passing messages pertaining
to a specific event or requesting a specific action. For example, when the
cdev is first selected, the control panel calls the 'cdev' code resource with
the InitDev message. The messages are listed in Table 1.
Table 1: Messages passed between the control panel and cdev

 Message Meaning
 -------------------------------------------------------


 initDev initialization

 hitDev mouse down inside of a cdev dialog item

 closeDev cleanup

 nulDev idle time message

 updateDev update event message

 activDev activate event message

 deActivDev de-activate event message

 keyEvtDev key down event message

 macDev used for checking machine characteristics (this
 is only used if the 'mach' resource does
 not define the environment that the cdev
 should be used in).

 undoDev the user selected undo from the menu.

 cutDev the user selected cut from the menu.

 copyDev the user selected copy from the menu.

 pasteDev the user selected paste from the menu.

 clearDev the user selected clear from the menu.



OOP and the Control Panel


Included with Think C 4.0 is an object-oriented cdev development shell that I
will use to develop a typical cdev. The cdev development shell is based around
a cdev class, which implements the basics of a control panel device. To use
this to write a cdev, you must override the methods needed by your cdev. The
example I chose (see Listing One, page 86) uses the SysEnvirons() routine to
obtain the current system information for display purposes; basically a system
info cdev.
To implement this, you need to define a subclass that overrides the Init()
method, which corresponds to the initDev message passed to the cdev-resource
by the control panel. Also, you need to provide a routine named Runnable(),
which examines the environment and determines whether the cdev should be shown
or not. This corresponds to the macDev message, which cannot be implemented as
a method because it will be called before the object is allocated. Finally,
the New routine must be provided to allocate the subclass and return the
object reference.


Conclusion


The cdev shell included with Think C 4.0 makes cdev development relatively
easy. Essentially all you have to do is write the application-specific
portions of the control panel device, a technique that can be used for the
development of any Macintosh code resources like control and window definition
procedures, and even device drivers. (For an example of a driver shell in
Object C, see my article entitled "Writing Macintosh Device Drivers" in this
issue on page 37.)

_OBJECT C AND THE MACINTOSH CONTROL PANEL_
by Bryan Waters


[LISTING ONE]

#include <DialogMgr.h> /* for dialog routines */
#include <cdev.h> /* for cdev class */

/* define subclass of class cdev */
struct sys_info:cdev {
 void Init( ) ; /* override the initDev message */
};
/* dialog item numbers */

#define MACH_TYPE 1
#define SYS_VER 2
#define PROC_TYPE 3
#define HAS_FPU 4
#define HAS_CQD 5
#define KBD_TYPE 6
#define AT_VER 7

/* current system environment version */
#define curSysEnvVers 1

/* computer model */
#define envMachUnknown 0
#define env512KE 1
#define envMacPlus 2
#define envSE 3
#define envMacII 4
#define envMacIIx 5
#define envMacSE30 7

/* cpu types */
#define envCPUUnknown 0
#define env68000 1
#define env68010 2
#define env68020 3
#define env68030 4

/* keyboard types */
#define envUnknownKbd 0
#define envMacKbd 1
#define envMacAndPad 2
#define envMacPlusKbd 3
#define envAExtendKbd 4
#define envStandADBKbd 5

/* Will the cdev be usable in this environment? */
/* This routine should be used to determine whether the cdev
 should appear in the control panel for this environment. An
 example of this would be a cdev for setting the parameters for
 the virtual memory manager for System 7.0. This cdev
 would only be necessary for 68020 with the Paged Memory
 Management Unit, or a 68030. In this case, the Runnable
 routine would determine the type of machine it was on, and
 return either TRUE or FALSE, depending on whether the cdev
 serves a valid purpose on this machine */
Boolean Runnable( )
{
 return TRUE ;
}

/* This routine creates the cdev. In our cdev, it doesn't really
 serve any purpose, since we don't have any storage
 requirements, but we still need the structure allocated for
 purposes of calling the appropriate methods */
cdev *New()
{
 return new( sys_info ) ;
}


/* This routine overrides the initDev message, which is called
 upon selection of any item in the control panel list. For a
 more complex cdev, this routine would do all initialization
 required for the cdev, and the more complex interaction
 required by the cdev would be implemented using remainder of
 the messages, and their handlers. The remainder of the
 messages, and the related method, are listed below:

 hitDev - for mouse downs. Method: ItemHit( )
 closeDev - for de-initialization. Method: Close( )
 nulDev - idle routine, for whatever needs to be done on a
 regular basis, but is not really event related.
 Method: Idle( )
 updateDev - for update events. Method: Update( )
 activDev - for activate events. Method: Activate( )
 deActivDev - for de-activate events. Method: Deactivate( )
 keyEvtDev - for key downs. Method: Key( )
 undoDev - for handling undos. Method: Undo( )
 cutDev - for handling cuts. Method: Cut( )
 copyDev - for handling copys. Method: Copy( )
 pasteDev - for handling pastes. Method: Paste( )
 clearDev - for handling clears. Method: Clear( )

 To implement any of these messages, simply override the
 method, in your subclass.

*/
void sys_info::Init( )
{
 int type ;
 Handle hdl ;
 Rect box ;
 SysEnvRec sys_env ;
 long tmp ;
 int count ;
 char *curr1 ;
 char *curr2 ;
 char buffer1[10] ;
 char buffer2[10] ;

 /* first call inherited Init routine */
 /* NOTE: this routine must be called to set up the cdev class */
 inherited::Init();

 /* get system environment */
 SysEnvirons( curSysEnvVers, &sys_env ) ;

 /* set machine type */
 GetDItem( this->dp, MACH_TYPE + this->lastItem, &type, &hdl, &box ) ;
 switch( sys_env.machineType ) {
 default:
 case envMachUnknown:
 SetIText( hdl, "\punknown" ) ;
 break ;
 case envMacPlus:
 SetIText( hdl, "\pMacintosh Plus" ) ;
 break ;
 case envSE:
 SetIText( hdl, "\pMacintosh SE" ) ;

 break ;
 case envMacII:
 SetIText( hdl, "\pMacintosh II" ) ;
 break ;
 case envMacIIx:
 SetIText( hdl, "\pMacintosh IIx" ) ;
 break ;
 case envMacSE30:
 SetIText( hdl, "\pMacintosh SE/30" ) ;
 break ;
 }

 /* set system version */
 GetDItem( this->dp, SYS_VER + this->lastItem, &type, &hdl, &box ) ;
 if( sys_env.systemVersion == 0 ) {
 SetIText( hdl, "\punknown" ) ;
 }else{
 count = 0 ;
 curr1 = &buffer1[1] ;
 tmp = ( sys_env.systemVersion & 0xFF00 ) >> 8 ;
 NumToString( tmp, buffer2 ) ;
 curr2 = &buffer2[1] ;
 while( buffer2[0]-- ) {
 count++ ;
 *curr1 = *curr2 ;
 curr1++ ; curr2 ++ ;
 }
 count++ ;
 *curr1 = '.' ;
 curr1++ ;
 tmp = ( sys_env.systemVersion & 0x00FF ) ;
 NumToString( tmp, buffer2 ) ;
 curr2 = &buffer2[1] ;
 while( buffer2[0]-- ) {
 count++ ;
 *curr1 = *curr2 ;
 curr1++ ; curr2 ++ ;
 }
 buffer1[0] = count ;
 SetIText( hdl, buffer1 ) ;
 }

 /* set processor type */
 GetDItem( this->dp, PROC_TYPE + this->lastItem, &type, &hdl, &box ) ;
 switch( sys_env.processor ) {
 default:
 case envCPUUnknown:
 SetIText( hdl, "\punknown" ) ;
 break ;
 case env68000:
 SetIText( hdl, "\pMotorola 68000" ) ;
 break ;
 case env68010:
 SetIText( hdl, "\pMotorola 68010" ) ;
 break ;
 case env68020:
 SetIText( hdl, "\pMotorola 68020" ) ;
 break ;
 case env68030:

 SetIText( hdl, "\pMotorola 68030" ) ;
 break ;
 }

 /* set has FPU */
 GetDItem( this->dp, HAS_FPU + this->lastItem, &type, &hdl, &box ) ;
 SetIText( hdl, sys_env.hasFPU ? "\pyes":"\pno" ) ;

 /* set has colorQD */
 GetDItem( this->dp, HAS_CQD + this->lastItem, &type, &hdl, &box ) ;
 SetIText( hdl, sys_env.hasColorQD ? "\pyes":"\pno" ) ;

 /* set keybord type */
 GetDItem( this->dp, KBD_TYPE + this->lastItem, &type, &hdl, &box ) ;
 switch( sys_env.keyBoardType ) {
 default:
 case envUnknownKbd:
 SetIText( hdl, "\punknown" ) ;
 break ;
 case envMacKbd:
 SetIText( hdl, "\pstandard" ) ;
 break ;
 case envMacAndPad:
 SetIText( hdl, "\pstandard with key pad" ) ;
 break ;
 case envMacPlusKbd:
 SetIText( hdl, "\pMacintosh Plus Keyboard" ) ;
 break ;
 case envAExtendKbd:
 SetIText( hdl, "\pApple extended" ) ;
 break ;
 case envStandADBKbd:
 SetIText( hdl, "\pstandard ADB" ) ;
 break ;
 }

 /* set appletalk version */
 GetDItem( this->dp, AT_VER + this->lastItem, &type, &hdl, &box ) ;
 if( sys_env.atDrvrVersNum == 0 ) {
 SetIText( hdl, "\pnot loaded" ) ;
 }else{
 tmp = ( sys_env.systemVersion & 0xFF00 ) >> 8 ;
 NumToString( tmp, buffer1 ) ;
 SetIText( hdl, buffer1 ) ;
 }

 /* finished */
 return ;
}


Example 1: Declaring an Object C class

struct class_name : storage_type {
 variable and/or method declarations ;
 .
 .
 .
} ;


Example 2: Declaring an Object C subclass

struct subclass_name:class_name {
 variable and/or method declaration ;
 /* NOTE: in order to override a method in the class,
simply redeclare the method in the subclass */
} ;


Example 3: Format for calling a cdev

pascal long cdev( message, Item, numItems, CPanelID, ev,
cdevValue CPDialog )
int message, Item, numItems, CPanelID ;
EventRecord *ev ;
long cdevValue ;
DialogPtr CPDialog ;












































Special Issue, 1989
ON BEING OR BECOMING A MACINTOSH DEVELOPER


There's been some changes made




Janna Custer


Janna is an editorial assistant with DDJ and can be reached at 501 Galveston
Drive Redwood City, CA 94063.


While enthusiasm for the Macintosh as a programming platform has been largely
responsible for the more than 3000 currently available applications, Apple
realizes that programmers and developers need support if they are to continue
producing new and better software for Macintosh (and Apple II) systems.
Although the company has supported developers through their Certified
Developer program since 1983, it's just recently that Apple expanded that
service by creating three basic programs -- the Apple Programmers and
Developers Association (APDA), Apple Associates, and Apple Partners (formerly
Certified Developers)--through which third-party developers have access to
what previously were in-house resources.


APDA Update


APDA is an international direct distribution channel open to anyone interested
in developing Mac products. Members can order Apple development tools, like
the Macintosh Programmers Workshop (MPW) and MacApp, and documentation, such
as "Inside Macintosh," and "Macintosh and Apple II Technical Notes." Members
can also order third-party development products -- compilers, development
utilities, and books. The $20 annual membership fee includes a subscription to
the quarterly APDAlog product catalog.
Furthermore, says Wendy Tajima, APDA's marketing manager, APDA "offers
HyperCard for the Macintosh and Applesoft Basic or Basic compilers for the
Apple II" for end users interested in programming. Corporate in-house
developers can get MacWorkstation and MacAPPC, the tools for building network
and mainframe access software. And professional developers of in-house
products and products for resale have access to MPW for the Mac and APW for
the Apple II, not to mention development tools such as A/UX (Apple's Unix).
APDA, formerly administered by TechAlliance (a national user group based in
Renton, Wash.), was brought in-house by Apple to expand services and provide
additional resources, to get closer to Apple's development customers, and to
better coordinate with other developer-oriented groups within Apple. This
transition, according to Sue Espinosa of Apple's developer channels group, has
increased companywide awareness -- beyond the engineering department -- of the
value of programming development products.
APDAlog is billed as an information catalog for developers and programmers.
The first half of the magazine consists of short articles on programming
languages and development tools. What follows is a complete catalog of
development products, the majority of which are for the Mac. Through the Apple
Partners Program and the Apple Associates Program (described later), those who
intend to develop products for resale or for in-house use can directly connect
with the people at Apple, for a fee, of course (see Table 1).
Table 1: Apple's Developer Group program fees

 Partners Associates
 Program Program
___________________________________________________________________________

 Macintosh or Apple II development materials with the $750 $500
 developer library

 Macintosh or Apple II development materials without $600 $350
 the developer library

 Annual renewal fee $600 $350



The Associates Program


Apple created the Associates Program in order to lend support to a wider group
of developers than was formerly reached by the Certified Developer program.
The Associates Program targets educators, researchers, and in-house
developers, all of whom might develop software for the Mac or Apple II for
their own use, not for resale. (See Table 2 for a longer list of
participants.) Development information available to associates includes system
software and updates, the Technical Guidebook (containing datasheets about
Apple products), technical notes, sample code, hotline support
(non-technical), an AppleLink subscription (the proprietary worldwide
communication network, which includes current product and program information,
sample code, and technical libraries and notes), the AppleDirect monthly
newsletter, the Apple Viewpoints biweekly newsletter, an APDA subscription,
and the Developer Library discount (optional).
Table 2: Support to various groups

 Partners Program Associates Program
________________________________________________________________________

 Third-party developers Educators
 Value-added resellers Researchers
 Systems integrators MIS professionals
 Original equipment manufacturers In-house developers
 Software publishers Computer consultants
 Contract programmers Industry analysts
 Dealer-supported VARs Computer training providers
 Computer accessory manufacturers

 Writers of Apple-related books
 Distributors of Apple-related products

Two publications, Apple Viewpoints (biweekly) and AppleDirect (monthly), are
published to keep developers informed of the latest development-related
information, and contain technical and marketing information, news,
interviews, insights and perspectives on trends in the computer industry, and
information on future Apple directions.


The Partners Program


The Apple Partners Program replaces the Certified Developers Program, and
targets developers who intend to sell Apple-compatible products within two
years. Participants include third-party developers, VARs, and contract
programmers, among others ( Table 2). The benefits of this program include all
of those provided to associates plus technical support via electronic mail,
the Marketing Guidebook, an AppleLink subscription (plus 12 hours of connect
time, one hour per month), the Marketing Assistance Program (including the
Customer Mailing Program), an invitation to the annual Developer's Conference,
hardware discounts (30-50 percent off retail), and leasing and credit options
for equipment purchase.
One benefit of being a partner is that Apple provides marketing assistance,
sharing strategies and helps developers analyze potential markets.


Additional Services


Both partners and associates have the option of purchasing the Macintosh or
Apple II Developer Libraries, which consist of programming manuals, user
interface guidelines, technical notes, development guidelines, and sample
code, for an extra $150. And both receive subscriptions to AppleLink and its
electronic mail system that connects developers not only with Apple, but with
other developers, VARs, Apple dealers, and user groups. Apple partners can
connect with the Developer Technical Support through this service and reach
engineers who are there to help solve problems, answer questions, or address
concerns. Participants in both programs receive advance copies of new versions
of Apple system software in order to conduct compatibility testing.
Also available to partners and associates are the courses on Mac programming
at Apple's Developer University, which runs from two to five days, and costs
between $900 and $1420. Though all are available in Cupertino, Calif others
are available in locations throughout the U.S. The curriculum includes
Macintosh Programming Fundamentals, Advanced Macintosh Programming, MPW
(Macintosh Programmers Workshop development system), Technical Introduction to
AppleTalk, and MacApp and Object-Oriented Programming.
Beneath the umbrella of the Developer Group are Developer Services, including
programs, technical support, special events, the university, and the press.
This includes "evangelism," which targets developers who are implementing key
products and provides them with product-design assistance during all phases of
the product development cycle, such as guidance on standards compliance,
competitive analysis, and international marketing. The Developer Tools Product
Marketing is another branch; it works with Apple development tools engineers
and with third parties to make development systems and tools available. And
developer technical publications produces technical books and manuals that
describe Apple products and development techniques.
Apple currently has about 10,000 registered developers working on new software
and hardware. With an installed base of over two million Macintosh computers,
the market is definitely there.










































Special Issue, 1989
Special Issue, 1989
GUEST EDITORIAL


Yesterday, Today, and Tomorrow




Scott Robert Ladd


It has been an honor and a privilege to be involved in the production of this
issue of Dr. Dobb's Journal. DDJ has always been a resource for C programmers,
and it is only fitting that it produces an issue devoted to the future of C. I
think you'll find that the variety of articles in this issue reflect the
myriad uses of C you'll find today and tomorrow.
C was born in the 1970s, matured during the 1980s, and is headed for an even
better future in the 1990s. Going from an obscure systems programming language
at AT&T, C has grown to be a premiere language for the development of
sophisticated applications on nearly every hardware platform. C is not a
static language; the original Kernighan and Ritchie C has evolved into ANSI C
and C++. The world of programming changes so often that a language cannot
remain unchanged and still be viable over the long term.
Where to go from here? C++ is an obvious answer; while several other
object-oriented variants of C exist (that is, Objective C, C_talk), none of
them have garnered the following C++ has. C++ does have faults and detractors
-- but what language doesn't? C++ offers what C programmers have always liked:
Freedom of expression. An idea central to both C and C++ is that the
programmer knows how to program, and the compiler should not stand in the way.
As with any other form of creativity, programming is best done when the tools
being used have few limitations.
The eventual success of C++ depends on many things. Probably C++'s biggest
fault is the lack of a standard object class library. Smalltalk and other
"pure" object-oriented languages provide a cornucopia of classes for
everything from linked lists to windowing and graphics. These built-in classes
provide more than program building blocks: They show a programmer how
object-oriented programming should be done. Historically, languages without
standard libraries (such as Modula-2) have suffered in the marketplace. If
AT&T does not address this need, then those of us who use C++ must work
together to build publicly-available standard libraries.
In spite of C++, C will continue to be a popular programming language. Once
the ANSI standard is finalized, new directions for C include the development
of standards for mathematical extensions and international markets. An ANSI
subcommittee is working on the former, and an ISO committee is developing the
latter. As the requirements of programmers change, so will C.
Another change we can expect will occur in the programming environment. Just
as the days of card punches and batch compiles have passed, so too will our
current system of compile/edit/debug be replaced. Working with C++ has shown
that the current programming environment is too limited for sophisticated
software development. New tools are needed to help us understand, design,
document, and debug our programs.
You can already see the germination of some of these capabilities, with the
introduction of integrated environments and sophisticated debuggers. However,
these tools are often slow, cumbersome, and lack the "power" expert
programmers want. Also, the best programmers tend to be very particular about
the way their tools work, particularly when it comes to editors.
One concept that interests me is that of development environments where
individual development applications are linked together via intelligent
interfaces. We talk about integrating data bases, communications, and word
processing for users, by using operating environments that allow
interapplication communication. The same thing can be done with a program
development environment, where CASE tools, editors, compilers, and debuggers
can be linked together to form an integrated environment. It would be even
better if programmers could mix and choose components of the system,
integrating their favorite tools to make a customized environment.
No matter what the future of programming holds, you can rest assured that C
and its derivatives will continue to be a part of that future. And DDJ will be
there, every step of the way.







































Special Issue, 1989
FROM C TO C++: INTERVIEWS WITH DENNIS RITCHIE AND BJARNE STROUSTRUP


Interviews with Dennis Ritchie and Bjarne Stroustrup




Al Stevens


Al is a contributing editor and columnist for DDJ and the author of a number
of books on C. He can be reached at DDJ, 501 Galveston Drive, Redwood City, CA
94063.


Dennis Ritchie is the designer of the C language and is the "R" in K&R, the
nickname for The C Programming Language, co-authored by Brian Kernighan. He is
a member of the Computing Science Research Center at AT&T Bell Laboratories in
Murray Hill, New Jersey.
Dennis did his undergraduate and graduate work in physics and applied
mathematics at Harvard University. Since joining Bell Lab's Computer Science
Research Center in 1968, he has worked on the design of computer languages and
operating systems. Along with others at Bell Labs, Dennis created the Unix
operating system, and designed and implemented the C language. His current
research is concerned with the structure of operating systems.
DDJ: As the designer of the C language, you are no doubt the world's very
first C programmer in a world where the number of C programmers is inestimable
and growing fast. Yet, certainly C was not your first language. When did your
programming career begin and with what systems?
DR: I started when I was in college in 1961. I was a physics major. There was
no such thing as a Computer Sciences curriculum then. The Comp Center at
Harvard offered an informal course in programming on the Univac I, and I went
to the IBM office and got manuals. In graduate school in 1963 I was the
teaching fellow for the introductory programming course. For a while I worked
at Project MAC at MIT. My graduate work was theoretical in recursive function
theory. I lost interest in that aspect of things when I finished there, and
I've been spending most of my time programming ever since.
DDJ: Do you actively program now?
DR: It depends what you mean by programming. There's a fair amount of looking
at stuff and deciding how it should work. These days there's more bureaucratic
stuff. I'm not in management, but I write memos, look at proposals, complain
to the X3J11 C committee, and things like that. I'm definitely still involved
in the technical aspects of things.
DDJ: The ANSI X3J11 committee has been five-plus years in arriving at a
proposed standard for the C language. How long did it take you from the time
you had your original idea for a C language until you had the first compiler
running?
DR: The C language grew out of an earlier language. The syntax of the early C
language was essentially that of B. Over the period of a couple of years it
grew into something like its current form. The most significant milestone in
the growth of the language was when the Unix system was rewritten in C.
DDJ: How long did that take?
DR: Mostly it was done in the summer. There were two tries at it. This was in
1973. The summer before, Ken Thompson tried to do it, and gave up. The single
thing that made the difference was the addition of structures to the language.
When he first tried there were no structures. They were in by the next summer,
and this provided a way of encapsulating or describing the data structures
within the operating system. Without that it was too much of a mess.
DDJ: You mentioned complaining to the ANSI X3J11 C committee. What was the
extent of your participation in the development of the ANSI C standard?
DR: My participation in the committee was really quite minimal. I sent them a
couple of letters. One was to point out the consequences and difficulties of
the path they were taking with the new style function definitions and
declarations. It's clear that the new style -- function prototypes, as they
call them -- is a good thing. The language is better for having it, and it
should have been done that way the first time. The problem, however, is in the
interval before prototypes are universally accepted, while you still have both
the old and new styles. I pointed out that with that approach there will be
confusion and the possibility of errors. For example, if you think that
there's a prototype in scope, you might call the function and expect that the
arguments are going to be coerced as they would be in regular ANSI C. But it
might not happen.
DDJ: In the Rationale document, X3J11 has paved the way to eventually do away
with the old style of function declarations and definitions. Will that solve
the problem?
DR: Yes, but in this interval there is a sticky situation. There are
complicated rules for what happens when you mix the new and old styles. They
covered all the bases when they made the rules, but the rules are messy, and
most people couldn't reproduce them or explain what they mean. The letter I
wrote was to suggest that maybe they should think about not doing it if only
because it's too late, or as an alternative they should consider requiring an
ANSI compiler to have the new style only.
My second letter was related to this "no alias" business that came up about a
year and a half ago. I felt more strongly about this issue because I felt they
were about to make a bad mistake, and I was willing to spend a lot of time
getting them to reverse it.
Around December 1987, when they were intending to produce the penultimate
draft, the one that had all the technical things in it (with possibly some
language polishing needed, but nothing important), something that had been
simmering a long time came to the boil. Some people wanted to put in a
mechanism that would reduce the problems that optimizers have with aliasing.
Here's the problem. Suppose you have a single function that has two pointers
as arguments, and the function can never be sure that the pointers might not
point to the same thing. Or, suppose one of the pointers points to some
external place. The function cannot tell where the pointers are going to
clash. According to the language rules, this kind of thing is possible, and
optimizers have to be very conservative about it. In most functions it might
never happen, and so the conservative compiler will generate worse code than
it would otherwise. Languages such as Fortran have an easier job of this
because such aliasing is simply forbidden. There's no enforcement, of course,
but the compiler can take an optimistic point of view. If your program doesn't
work, someone can pull out the standard and say you shouldn't have done that.
Aliasing was a plausible thing for the committee to think about. It does, in
fact, make C somewhat harder to optimize. The mistake they made was in trying
to design a facility to allow the programmer to say that a particular function
has no aliasing problem. But they actually blew it. The language rules that
they developed, even after many sessions of hard work, really just weren't
correct. Their specification for how you say "no alias" was broken and would
have been much more dangerous than not having it. If this had happened three
or four years ago, people would have seen that this was wrong, fiddled with
it, and either thrown it out or fixed it one way or the other. But this was
supposed to be the next to the last draft, and all the technical requirements
were supposed to be already done, and it was just broken.
That December I drafted a long and strongly worded letter to them saying that
this just won't do, and pointed out the problems that I'd found. I even went
to the meeting, the first X3J11 meeting I'd been to, and argued against it.
What got me worried and annoyed was that this had happened when it did. If the
thing had gone ahead it would have been a real bug in the specification. On
the other hand, fixing it essentially meant a technical change, an important,
non-editorial change, and they would need another long public review period.
The point of view that I advanced was to get rid of it. I figured that my
argument had to be simple to understand. If I had said, "This 'no alias' is
broken, here's another thing that you should do instead," I could see us
getting bogged down endlessly worrying about the technical details, so I
figured it was better to argue that they should just throw it out altogether.
That was the only really detailed involvement I had with X3J11. The outcome
was that there is no specification for "no alias." They voted it out. Except
for some slight fiddles, the draft that is now before X3 is technically
identical to what it was nearly two years ago.
Aside from those two issues, I left them alone for two reasons. One is that to
take part in a standardization effort is an enormous amount of work. There are
three one-week meetings a year all over the world, a lot of detailed reading,
and I really didn't have the heart to do that. The second reason is that it
became clear early in the proceedings that the committee was on the right
track themselves. Their charter was to codify and to standardize the language
as it existed. They decided in advance to do that and that is what they did.
They did add some new things. The function prototypes are by far the most
obvious, and there are a lot more minor things, but mainly they stuck to their
charter.
I think they did a very good job, particularly when compared to the things
that are happening in the Fortran committee, X3J3, where there are wide swings
back and forth about the strange new things they're adding in, taking out, and
putting back in. They have great political arguments between customers and
vendors, Europeans versus North Americans, and it really seems to be a
free-for-all. Even though some wrangling went on in the C committee, with the
people involved seeming fairly fierce when you looked at it from outside, it's
obvious that X3J11 was a comparatively tranquil and technically wise group.
The upshot is that I think they did a good job. Certainly, though, if I'd
continued to work on things, some of the details would have been different.
DDJ: Are there any major areas where you disagree with the standard as it
exists now?
DR: There are some obvious weaknesses. For example, they have never worked out
what const really means. One of its intents is to say that this is some data
that can be put into some read-only storage because it's never going to be
modified. The definition that they have now is sufficient for that. But they
also had other ideas about what it should mean, having to do with
optimization, for example. The hope was that const is somehow a promise that
the compiler could assume that the data item wouldn't change underfoot.
If you have a pointer to a const, one might hope that what is const is not
going to suddenly change secretly. But, unfortunately, the way the rules read
that's not actually true. It can change, and this wasn't just an oversight. In
fact, they are potentially overloading the meaning of const. There are ideas
involved other than what people hoped to get, and they never really worked out
exactly which ones they wanted and which ones they didn't want. It's a little
confusing.
It's generally recognized that the standardization of the library was as
important as the standardization of the language. Among Unix systems there are
few variations on what's available in the library. Most things are pretty much
the same. In recent years, the use of C has spread far outside of Unix
systems, and the libraries supplied with compilers tend to vary a lot,
although many of them were based on what was available on Unix. So the
standardization of the library is important. On the other hand, I've heard
lots of complaints, both from users and implementors, that what they
standardized and some of the rules and interfaces for library routines were
not very well worked out. There may be more there than is necessary. Things
got too complicated.
DDJ: What was the rationale behind the decision to leave out the read, write,
open, close, and create functions?
DR: Those functions are viewed as being quite specific to the Unix system.
Other operating systems might have great difficulty in supplying things that
work the same way those do. The idea of the original pre-ANSI standard I/O
library was to make it possible to implement those I/O routines in a variety
of operating systems. It took us a couple of tries to reach that particular
interface. The machines we had here were the PDP-11 running Unix, a Honeywell
6000 running GECOS, and some IBM 360s running various IBM systems. We wanted
to have standard I/O routines that could be used in all the operating systems,
even those that didn't have anything like Unix's read and write. The committee
felt that it was better to let the IEEE and other Unix standardization groups
handle that. They specifically avoided putting things in the C library that
were Unix-specific unless they had meaning in other systems.
They did another thing that people don't quite understand. They explicitly
laid out the name space that a standard compiler is allowed to usurp or claim.
In particular, the guarantee is that there is a finite list of names that the
compiler and the compiler system take up. These are simple names, beginning
with underscore, and are listed in the back of the standard, something like
keywords. You are allowed to use any name that isn't on this list. In an
ANSI-conforming world, you are allowed to define your own routine called read
or write and even run it on a Unix system. It's guaranteed that this will be
your routine and that your use of the name does not conflict with any I/O that
the library itself does on your behalf. The Unix library authors will be
constrained to have an internal name for read that you can't see so that if
you bring a C implementation from a big IBM machine or from an MS-DOS machine
and you happen to use the name read for your own routine, it will still
compile and run on a Unix system even though there's a system call named read.
They have circumscribed the so-called name space pollution by saying that the
system takes these names and no others.
DDJ: How can they be sure, that an implementor won't need other than those
specific names from the list?
DR: There are rules for how the internal people can generate names, namely
these underscore conventions. The end user is not allowed to use underscore
names because any of these might be used internally. There is a problem,
though. They made a list of things and said they'll do this and no more, and
that helps. But there is still a problem for the writer of a library who wants
to sell or distribute it. You're in a bind because you don't know all the
underscore names that all the implementations are going to use. If you have
your own internal names, you can't be sure that they're not going to conflict
somewhere. If they have underscores, they might conflict with the underlying
implementation. If they don't, then they might conflict with things that your
end users are going to use. The C committee did not solve the problem that
other languages have tackled explicitly. There are other ways of controlling
the name space problem. They made a convention that helps, but it certainly
didn't solve the real problem. It solved it enough to improve the situation.
The basic problem with an uncontrolled name space is that if you write a
program, and it just uses some name that you made up, it may be actually
difficult to find out that this is not the same name as some random routine
that's used internally by your system library. Unfortunately, we here at Bell
Labs are in a bad position to notice this and do something about it because in
our group we've simultaneously developed the compiler and the library and the
Unix system, and so people here tend to know the names.
So, to summarize X3J11, the two largest things the committee did were function
prototypes and the standardization of the library. It was more work than
anybody expected, but I'm perfectly happy with what they did. The only problem
was it took twice as long as they thought.
DDJ: Do you see a potential for other standard extensions to C, beyond those
added by X3J11?
DR: One of the major excuses they give for not doing something is that there's
no practice, no prior art. So obviously people will try to create prior art
for the things they'd like to have happen. One such group is the Numerical C
Extensions Group.
There are people who have strong views about what should happen. The idea is
to get together with this group and agree that these are the things we need to
have, so let's make some rules so that when people try things out, we'll all
be trying it the same way, and we'll have a coherent story to tell, if there
are going to be these extensions.
Most of them have to do with IEEE arithmetic issues, exceptions and such, for
example. There's a core of things that are more general, and one that
interests me is variable arrays with adjustable sizes. One of the things C
does successfully is deal with single-dimension arrays that can be variable in
size, but it doesn't deal with multi-dimensioned arrays that are variable at
all. This is an important lack for numeric types, because it makes it hard to
write library routines that manipulate arrays. Multiplying two arrays is a bit
painful in C if the arrays are variable in size. You can do it but you have to
program it in detail and the interface doesn't look pleasant. That's an
obvious need, and I volunteered to look at how it might be done.
The NCEG will probably try to become official. They will affiliate themselves
either with X3J11 or as an IEEE standardization organization. This would give
them more clout. Also, many of the companies involved worry about legal
issues. Companies who are members of informal groups deciding standards worry
about anti-trust, whereas if they are members of official, blessed standards
organizations, then they can contribute. They worry about being accused of
going off into a corner and doing things behind other people's backs. It's
better to do it in the open. This may be just some lawyer's nightmare. NCEG
will probably become a subcommittee of X3J11.
DDJ: One non-ANSI extension to C is C++, a superset language that surrounds C
with disciplines and paradigms that go beyond its original intent as a
procedural language. Can you comment on how appropriate that is and how
successful it has been?
DR: Let me confess at the start that I know less about C++ than I probably
should. C is a very low-level language on a variety of fronts. The kinds of
operations that it performs are quite basic. The control over names and
visibility is basic. The defects or limitations of C in this area are most
evident when you get into a large project where you need strong standards,
rules, and mechanisms outside the language. Language developments such as C++
are trying to supply some of the structure within the rules of the language
for controlled visibility of name space and are trying to encourage various
kinds of modularization. This is good, I suppose.
C was designed in an environment where modularity was encouraged not so much
by the language but by the kinds of programs we wrote. In the Unix system, the
tradition is for small utilities that work together as tools, and the
interfaces between them were set by the conventions and rules of the operating
system, i.e., pipelines and so forth. The complexity of the pieces was kept
low by custom. Commands tend to be simple. In the world today, there's a
certain amount of admiration for that point of view. Certainly the
appreciation for that style is part of the reason for the growth of Unix.
People now are undertaking the building of much bigger systems, and things
that we handled by convention ten or fifteen years ago must be handled by more
explicit means. C++ is one such attempt.
Bjarne decided to design a compatible superset of C and to translate the C++
language into C code. That approach is not without its problems. First, having
decided that C++ is going to be largely compatible with C, every time he
departs from that he's under pressure either because of some accident or
because ANSI changed something. Or because he feels that there's something he
has to differ in, people are going to complain and get confused. Second, he is
constrained by the choice to make a C++ to C translator possible, that is, he
is constrained, as C was, by the existing tools of the various systems. The
whole separate compilation business in C++ is made a lot harder by the desire
to make it work with existing tools. If he could have simply designed a
language and implemented it, then a lot of the anguish would have been
avoided.
DDJ: Rumor is that within Bell Labs, C++ is now called C, and C is called "old
C." Any truth in that?
DR: I've asked Bjarne not to say "old C," and, as far as I know, he has
complied with that request.

DDJ: Colleges and universities have started offering courses in C. Some C
tutors have observed that many instructors either don't understand C well
enough or they don't understand teaching well enough to insulate the novice
student from the kinds of things you can do in C, things that the student
cannot grasp. In light of that, and as compared to Pascal, how do you view C
as a potential teaching language?
DR: Obviously, C was never designed to be a teaching language. It was designed
as a tool to express the kind of programs that we were trying to write at the
time. And it's fairly low level in that concepts, like pointers, have a
prominent role. I would not argue that C is a particularly good language for
teaching programming. As Pascal was explicitly designed for that.
Pascal's main fault is that you cannot use Pascal originally designed to
express all the things you need to, certainly not in a systems environment,
and not for general applications either because of explicit constraints that
are built into the language. C was, from the very start, designed to do all
the things that we found necessary in order to express ourselves, and little
design thought was given to preventing people from using its powerful
features.
Nevertheless, it's possible to teach C in a way that's reasonably safe if you
start with parts of the language that are similar to other procedural
languages. Then you can teach C's more unusual aspects -- pointers, for
example -- as cliches or set ways of expressing array manipulations and so
forth. Later you can gradually widen out into the more general things possible
with pointer manipulations.
I have not had the experience that the tutors have had. Part of the difficulty
with being in a position like this is that you have very little opportunity to
see what the novice really feels. But perhaps the reason there are not better
instructors is that things have grown fast, and there might be people teaching
C who only recently took the introductory course on the language themselves.
DDJ: Would you attempt a prediction for the future of the C language?
DR: I think the period of C's largest growth is over, although it will be
increasingly used and it probably will not change very fast. The new language
developments based on C will be on successors such as C++ or perhaps some
things we haven't heard of. In terms of what C tried to do, I think it
succeeded fairly well. The goals were reasonably modest. There's still plenty
of work to be done finding languages that have the touch of reality that C
has, work where you handle real problems in real environments as opposed to
dealing with elegant creations that can't be used. Sometimes things can't be
used just because the compilers don't exist on the machines people have.
Sometimes it's because there are simply flaws in the design, not from the
language point of view, but from the point of view of what the language ends
up doing in the real world. And in that respect, C seems to have worn fairly
well.
Bjarne Stroustrup is the creator of C++, the object-oriented extension to the
C language. He is a researcher at the AT&T Bell Laboratories Computing Science
Research Center where, in 1980, he began the development of the C++ extensions
that add data abstraction, class hierarchies, and function and operator
overloading to C. The C++ language has undergone several versions, and the
latest is Version 2.0. Dr. Stroustrup maintains an active presence in all
matters concerning the development, advancement, standardization, and use of
C++.
DDJ: Many experts are predicting that C++ will be the next dominant software
development platform, that it will essentially replace C.
BS: They're not alone. People were saying that five years ago.
DDJ: When you conceived the idea of C++ as an extension to the C language,
were you thinking about object-oriented programming in the way it's come to be
known, or were you looking to build a solution to a specific programming
problem that would be supported by the features that you built into C++?
BS: Both. I had a specific problem. All good systems come when there is a
genuine application in mind. I had written some simulations of distributed
computer systems and was thinking about doing more of them. At the same time I
was thinking about the problem of splitting Unix up to run on many CPUs. In
both cases I decided that the problem was building greater modularity to get
fire walls in place, and I couldn't do that with C. I had experience with
Simula, writing rather complex simulations, so I knew the basic techniques of
object-oriented programming and how it applied.
To solve the problem I added classes to C that were very much like Simula
classes. That, of course, was the solution to a particular problem, but it
included a fair amount of general knowledge about techniques, thoughts about
complexity and management, complexity of modularity and all the baggage that
you get from Simula. Simula is not ad hoc, especially not when it comes to the
class concept, which is what I was interested in.
DDJ: Are you familiar with any of the PC ports to C++, specifically Zortech
C++, Guidelines C++, and Intek C++?
BS: Only from talking to people and listening to discussions about them. They
all sound good. The CFRONT ports, Intek and Guidelines, have the advantage of
having the same bugs and features that you have on the bigger machines all the
way up to the Cray, whereas Zortech has the advantage of being native to the
PC world.
I walked around back in 1985 explaining why the current implementation of C++
couldn't be put on a PC. I designed the language under the assumption that you
had one MIPS and one Meg available. Then one day I got fed up with explaining
why it couldn't be done and did it instead. It was a pilot implementation, and
it wasn't ever used, but I proved that it was possible, and people went and
did the real ports. All the implementations are reasonably good, and they
could all be better. Given time, they will be.
DDJ: Do the PC ports accurately implement C++ the way you have it designed?
BS: We do have a problem with portability from one machine to another. If you
have a large program of ten to twenty thousand lines, it's going to take you a
day to move from one independent implementation to another. We're working on
that. Standardization is beginning. We're all sharing language manual drafts,
and so it's trying to pull together. But a large program port will still take
a day as compared to the ANSI standard ideal where you take something from a
PC to a Cray and everything works. Of course, you never really get to that
point even after full standardization.
DDJ: There are lots of rumors about Borland and Microsoft coming out with C++
compilers. Has any of this come to your attention?
BS: I've talked to people both from Microsoft and from Borland. They're both
building a C++ compiler, and it sounds as if they're building it as close to
the 2.0 specification as they jolly well know how to. Naturally, for their
machines they'll need something like near and far, which is not standard
language, but that's pretty harmless.
Both asked for a bit of advice and Microsoft asked for the reference manuals.
I've talked to the Borland guys. I'm sad to say they didn't ask for a manual,
but maybe they got one from other sources. The PC world is pretty cut-throat.
Maybe people get the impression everybody is cut throat. That's not quite the
case.
DDJ: One of the advantages of languages such as C and C++ is that they can be
implemented on a wide range of machines ranging from PCs to Crays. With more
and more people using PCs in their work, it's widely believed that acceptance
in the PC world is what spelled the overwhelming success of C as the language
of choice.
BS: That's widely believed in the PC world. In the minicomputer world it's
widely believed that the PDP-11 and the VAX spelled the success of C and that
is why the PC world picked it up. One of the reasons C was successful is that
it was able to succeed in very diverse environments. There are enough
languages that work in the PC world, enough that work in the minicomputer
world, enough that work on the large machines. There where not very many
languages that worked on all of them, and C as one of them. That's a major
factor. People like to have their programs run everywhere without too many
changes, and that's a feature of C. And it's a feature of C++.
DDJ: Do you see the PC as figuring as prominently in the acceptance of C++?
BS: Definitely. There are probably as many C++ users on PCs as on bigger
systems. Most likely the number of PC users will be growing the fastest
because the machines are smaller. People who would never use a PC for their
professional work -- there are still a lot of those -- nevertheless like to
play with things on a PC to see what it is, and that is where PCs come in.
Similarly, if you are working on a PC, sooner or later you run into a bigger
machine, and it's nice to be able to carry over your work. I'm very keen on
portability.
DDJ: Do you have opinions as to whether preprocessing translators, such as the
CFRONT implementation on Unix, have advantages over native compilers such as
Zortech C++?
BS: It depends on what you're trying to do. When I built C++ I felt that I
couldn't afford to have something that was hard to port, meaning it mustn't
take more than a couple of days. I thought if I built a portable code
generator myself, it would be less than optimal everywhere. So, I thought if I
generate C, I could hijack everybody else's code generators.
For the last 40 years we've been looking for a universal intermediate language
for compilation, and I think we've got it now, and it's called C. So, what I
built was something that was a full compiler, full semantic check, full syntax
check, and then used C as an intermediate representation. In essence I built a
traditional two-pass compiler. And two-pass compilers have great advantages if
you've got lots of code generators sitting around for them, and I had. They
have the advantage that they tend to find errors faster than one-pass
compilers because they don't start generating code until they have decided
that the program looks all right. But they tend to be slow when they actually
generate code. They also tend to be slightly larger and slightly slower
throughout the whole process because they have to go through the standard
intermediate form and out again.
And so, the advantages for the translator technology are roughly where you
have lots of different machines and little manpower to do the ports. I see the
one-pass compilers, the so-called native compilers, useful for machines and
architectures where the manpower available for support and development is
sufficient to make it worthwhile, which is when you've got enough users.
The two-pass strategy for translators was essential in the early days and will
remain essential as long as new machine architectures and new systems come on
the market so that you need to get a compiler up and running quickly. As the
C++ use on a given system matures, you'll see the translators replaced by more
specifically crafted compilers. On the PC, for example, CFRONT likes memory
too much; it was built for a system where memory was cheap relative to CPU
time. So once you know you are working for a specific architecture, you can do
intelligent optimizations that the highly portable strategy that I was using
simply mustn't attempt.
DDJ: Are there different debugging considerations when you are using a
preprocessing translator?
BS: One of the things that people have said about the translators is that you
can't do symbolic debugging. That's just plain wrong because the information
is passed through to the second pass and you can do debugging of C++ at the
source level. Using the 2.0 translator we're doing that. That 1.2 versions
didn't have quite enough finesse to do it, and people didn't invest enough in
modifying debuggers and the system-build operations to give good symbolic
debugging. But now you have it.
DDJ: Can you estimate the worldwide C++ user base today?
BS: Fifty thousand plus, and growing fast, and that is a very conservative
estimate.
DDJ: Have you formed plans to rewrite any or all of Unix with C++?
BS: Unfortunately, I haven't, despite that being one of my original thoughts.
I've been bitten trying to write software that was too complex for the tools I
had. When thinking about rewriting Unix, I decided that C wasn't up to the
job. I diverted into tool building and never got out of that diversion. I know
that there is an operating system written in C++ at the University of
Illinois. It has a completely different kernel, but it runs Unix and you can
make it look like System V, USD, or a mixture of the two by using different
paths through the inheritance trees in C++. That's a totally object-oriented
system built with C++. I know that AT&T and Sun have been talking about Unix
System V, Release 5, and that there are projects working on things like
operating systems rewrites, but whether they become real or not depends more
on politics and higher corporate management than anything else. Why should we
guess? All we can do is wait and see.
DDJ: Is what you do now primarily related to the development of C++ or the use
of it?
BS: Both. I write a fair bit of code, still. I do a lot of writing, and I
coordinate people, saying "Hey, you need to talk to that guy over there," then
getting out of the loop fast. I do a fair bit of thinking about what else
needs to be done with C++ and C++ tools, libraries and such.
DDJ: C is a language of functions, and a large part of the ANSI C standard is
the standardization of the function library. C++ has all that as well and adds
classes to the language. Is there a growing library of C++ classes that could
eventually become part of a standard?
BS: The problem is there are several of them. We use some inside AT&T, and
several of the other purveyors of C++ compilers and tools have their own
libraries. The question is to what extent we can pull together for a standard
library. I think that we can eventually get to a much larger standard library
and much better than what is available and possible in C. Similarly, you can
build tools that are better than what is possible with C because there is more
information in the programs.
But people, when they say standards, tend to think about intergalactic
standards, about things that are available in any implementation anywhere, and
I think that they think too small. There are good reasons for differences
between the ideal C++ environment for a Cray and the ideal C++ environment for
a PC. The orientation will be different as will the emphasis on what is
available. So we will see many standards, some for machine architectures, some
over ranges of machines, some national standards. You could imagine the French
having a whole series of libraries and tools that would be standard for people
doing French word processing, for instance. You will see national standards,
international standards, industry standards, departmental standards. A group
building things like telephone operator control panels would have the standard
libraries for everybody in the corporate department doing that kind of work.
But a token standardization of everything, you won't see. The world is simply
too big for that. But we can do much better than we're doing now.
DDJ: To the programmer, there is an event-driven or object-oriented appearance
to the graphical user interfaces (X Windows, the Macintosh, Presentation
Manager, MS-DOS Windows, and so on). These seem to be a natural fit for the
class hierarchies of C++. How would these facilities be best implemented, and
do you know of any recent efforts in these areas.?
BS: Some people have the idea that object-oriented really means graphics
because there is such a nice fit. That has not been my traditional emphasis.
The examples people have seen of object-oriented programming and
object-oriented languages have, by and large, been fairly slow. Therefore,
their use has been restricted to areas where there's a person sitting and
interacting. People are relatively slow.
My first aims were in areas where you had complex programs that had a very
high demand on CPU and memory, so that's where I aimed first. But people have
been building very nice toolsets for doing user interfaces with C++. There is
one from Stanford called "Interviews." Glockenspiel, in cooperation with
Microsoft, is selling Common Views, which is a C++ toolset that looks and
feels exactly the same whether you are under Presentation Manager, MS-DOS
Windows, or on a Mac. There are C++ libraries for Open look.
The problem with all these so-called standards is that everybody seems to have
their own standards, and then you start wondering how you can get toolsets
that give platform independence across all of these I think that's one place
where C++ comes in. Most of the differences between the major systems in the
areas of text handling -- as opposed to high-performance graphics -- seem to
be quite manageable as a common set of classes that could be standardized at
the language level. It's certainly something that's worth exploring because
the world is getting more fragmented.
DDJ: Are you familiar with Objective C, a C language extension that appears to
do at least a subset of what C++ does, and how do the two languages compare?
BS: Vaguely. The company that sells it will claim that it does a superset of
what C++ does, and that whatever C++ does that it does not, is not as
important, naturally, I disagree. There is much higher emphasis in C++ on
static type checking and on coherence of the type system. C++ is a rather
large affair with multiple purveyors and multiple libraries, where objective C
is a corporate language from a corporation that wants to make its fortune out
of it. That places a different emphasis on everything.
DDJ: Concurrent C and Concurrent C++ are, like C++, extensions to the
language. They add parallel processing operations to C and C++ for the
development of multitasking programs that are portable among multitasking
platforms. Do you have any comments on Concurrent C++? Is there a need for
portable parallel processing, and do you think that Concurrent C++ fills that
need?
BS: I don't like the idea of putting ADA tasking into C or C++. I think it
solves the issues dealing with concurrency at the wrong level, sort of a
medium-level thing. It doesn't give the transaction processing view and
transaction logging that you need in data bases. It doesn't give the machine
near world that you need when you write an operating system kernel or a
real-time application. Personally, I don't like that approach at all. The
approach I've taken with C++ is to provide con currency in the form of
libraries. We have the Task Library that provides a much lower-level system
that allows you to write multi-threaded programs. I've used it for simulations
where I needed a couple of thousand processes or tasks and I want them to run
with fairly minimal overhead. It has been used for robotics and such. I much
prefer the library route over the route of adding syntax to the language. I
think it serves more people better.
DDJ: Let's discuss the future of C++ and what programmers can expect in the
next decade. The immediate future of C++ is, of course, 2.0, which adds
multiple inheritance to the language. Can you summarize the other features
added by version 2.0?
BS: It's a reworking of the language, polishing off the little rough corners,
the unnecessary restrictions, and the problem areas we found. Even multiple
inheritance can be seen as removing a little odd restriction, which was that
you couldn't have more than one base class. We can argue how major an
extension it is. Some people think it's major. I think it's sort of medium. It
allows you to do things that you could do with C++ but noticeably cleaner and
easier. I don't think it allows you to do anything radically new, and most of
the other features I've added to 2.0 are of that ilk. It's meant to stabilize
the language, it's meant to increase the quality of the implementations, and
it's meant to remove unnecessary restrictions without destroying run-time or
space efficiencies.
So, in version 2.0 you have the multiple inheritance, you have a more
sensitive and better overloading resolution mechanism, you have type-safe
linkage to make sure you can link larger programs together more effectively,
and you have abstract classes. The list is fairly long but not radical. We
have a lot of users, and we have to make sure there is a certain stability in
the growth.
DDJ: Are you planning specific features for C++ beyond 2.0?
BS: We've been talking about exception handling and parameterized types for a
long time. It's universally agreed that we need them. We have a reasonably
good design for parameterized types that I presented at the last USENIX C++
conference, and that needs to be refined a little bit. Exception handling is
one stage behind that, but we need it badly. When you go to C++, you get more
ambitious. You want to have larger libraries, you want to use more of other
people's code, and so you need more support. That's what we're trying to
provide as quickly as we can without just throwing in everything at random.
The language has to be kept coherent.
DDJ: The Integrated Development Environment with its integrated editor,
compiler, and debugger has become the chosen software development suite in
small systems. Turbo C, QuickC, Turbo Pascal, etc., are examples. Stand-alone
symbolic debuggers that deal specifically with objects like C++ classes are
beginning to appear as well. Are such environments appropriate for C++, and do
you know of any current developments?
BS: Oh yes. They'll come. Some people will use them, and I believe you can buy
one from ParcPlace now. I'll assume that since Microsoft is working on C++,
they'll be working on a suitable environment. Borland is playing the same game
they'll be doing it, too. I know of several other people who are thinking
along those lines.
The thing that one has to seek is code portability across the different
platforms. Certainly you can provide better tools for a specific platform, a
better compiler, better compile times, a better program development
environment. Unless, however, you are able to take your programs out of their
environment and export them to something else, you have painted yourself into
a corner.
DDJ: To date there is no standard for C++. Hewlett-Packard formally requested
the ANSI X3J11 committee to undertake that standardization as a compatible
superset of ANSI C, but the committee was ambivalent about it, failing to vote
to begin the task. One reason for the request was to avoid the time-consuming
overhead of setting up a new committee. Opponents to the idea pointed out that
you are still in the process of the C++ definition and, as such, are not ready
for standardization. Do you see the committee's action as an impediment to the
acceptance of C++? Is C++ ready for standardization?
BS: Life isn't easy. Clearly we would like a fully standardized language, and
equally clearly we don't know how to do that. There are still some features
that we need to design. There were discussions that included me, other people
at AT&T, and people from Hewlett-Packard and Microsoft. Where do we go from
here, we asked? How can we get the most stable environment the fastest without
freezing the language at a level where everybody has to extend it themselves?
If we standardized C++ simply as it was, everybody would build their own
exception handling and parameterized types.

I don't think that the ANSI C committee would be at all a suitable forum for
standardizing C++. First of all, they are not C++ users. They may be C
experts, but they are not C++ experts. We might as well say that since the
Pascal committee did a good job on Pascal, let them do C++. We need a new
commit tee composed of people with C++ experience. We clearly need a standard
as soon as possible, but we simply have to figure out what "as soon as
possible" means.
We need a C++ that's as compatible with ANSI C as possible, but it can't be
one hundred percent compatible without destroying every C++ program ever
written. When ANSI turned down the proposal, Hewlett-Packard went to SPARC,
the policy committee for ANSI, with a new request to start a new ANSI C++
committee with the charter for standardizing C++, a committee composed of
people who know the problems of exception handling and parameterized types,
and who know that one hundred percent compatibility with ANSI C is not
desirable -- as close as possible but no closer. That was accepted by SPARC,
and they have sent a recommendation to X3 to start an ANSI C++ committee.
Presumably the first meeting of that will be next spring.
DDJ: "As close as possible to C but no closer." That was the title of a paper
by you and Andrew Koenig wherein you identified the differences between ANSI C
and C++. The tone of the paper was that those differences are proper and
necessary given the different purposes of the two languages. The paper also
stresses the inconsequential nature of most of the differences. Do you see
those differences as proof that C and C++ can not be combined? Should they be
combined?
BS: I don't think the two languages should be reconciled further than they
are. The differences are quite manageable by conditional expressions. If you
are writing C++, you use the new keywords and it isn't ANSI. If you are
writing ANSI C you could have an option in your compiler that suppresses the
C++ subset, and it will not affect the way you write code. It's much more an
issue for language lawyers than it is for programmers. The whole thing can be
exhaustively discussed on two pages and the differences can be listed in about
ten lines.
DDJ: Will you rewrite your book, the C++ Programming Language, to reflect the
new features of version 2.0?
BS: I will as soon as I get time. But it's more important to get the language
stabilized, and so I'm working on a new language definition, a new manual. I
am working on a book, as well, with Margaret Ellis that explains what C++ is,
not, as the first book does, how you go about using it. The book states what
the language is, what the implementation techniques are that make sensible
implementations of certain parts, and why certain decisions were made.
This is a "what" book, not a "how to" book. In theory, it's a book for
experts. If you have a question about what the language is, the answer should
be there. Lots of people prefer to learn languages from such explanations. You
never know who would want something like that. I learned ALGOL 60 out of the
ALGOL 60 Revised Report. I'm not the only one who is sort of semi-masochistic
in the way I read things.
DDJ: When will this new book be available?
BS: It's supposed to happen in December or January. The question is whether I
can make it. The manual work itself is taking longer than it should. It's
about twice the size of the original manual, not because version 2.0 is twice
the language as version 1.0, but because I need to go into much greater depth.
If you can assume your reader's culture, you can take shortcuts in
explanations, and there are certain words that you can forget to define
without getting into trouble. C++, however, is breaking out of the C ghetto.
People who were brought up with Pascal and wouldn't touch C with a barge pole
are getting on board with C++. They come in and they try to read some of the
standard C++ literature. When you get their comments, you learn what
assumptions you made without explaining them. Such things must not be done in
the manuals we are working on. It's getting harder to write a manual because
the audience is becoming more diverse.
DDJ: One of the problems we've observed when function-oriented programmers
attempt the transition to object-oriented systems is that there doesn't seem
to be any way to describe the new paradigm to them in a way that they can
learn it without actually using it.
BS: We'll eventually do better. The first books on C++ just said what the
language was, put a thin veneer on top of it, or described it primarily as a
better C. Lipmann's book and Dewhurst and Stark's book go beyond that and
demonstrate how things are done. We have a major education problem on our
hands. I've been saying that for years. You can write Fortran in any language,
and if you only use C++ as a better C, you'll see improvements in your
programs and productivity, but you won't get anywhere near what you can do or
what we have seen demonstrated with greater degrees of data abstraction and
object-oriented programming.
DDJ: The function-oriented programmer does not understand intuitively what the
advantage of that is.
BS: It's hard to explain how to bicycle. I can talk myself hoarse and still
you go up to a bicycle, and you fall off the first time. A certain amount of
practice must be done. Programming is an art like riding a bicycle. It's
learned by doing it. But you can certainly help the process. You don't just
say to somebody, "Here's a bicycle. Ride." Similarly we need better education
on how to use a language like C++ for object oriented programming. You can
write better articles, use videotapes, put training wheels on the new
programming environments, help programmers get started. It can be managed. It
will be managed. The gains in doing so have been demonstrated often enough.
DDJ: Programmers have been told that they must unlearn what they have learned
in order to use object-oriented programming, and they reject that.
BS: That's not what I am telling them. C++ is a better C, and it supports data
abstraction, and it supports object-oriented programming, and, yes, you can
forget all you have learned and jump into the deep end. But lots of people do
it differently. They start out using it like a better C, they experiment on
the side with the techniques they don't quite understand. Follow the
literature so that you know the syntax, the basic semantics, and your tools
before you take that big leap. Then try to focus on classes for a new project.
One of the advantages of C++ is that you can take some of the preliminary
steps without a paradigm shift from function to object-oriented programming.
You can learn the tools, the language, the basics, how to use debuggers,
whatever it takes. And then one day when you find the right project, you can
try the next step. We've seen that done quite a few times.
There are people who enthusiastically read all the literature, go to C++
conferences, and then go straight in, design huge class hierarchies, and
program them. I'm always amazed when it works, but it does quite often. There
are people who can write perfectly standard designed C programs, go away and
wait two months and come back and write their first program in C++, truly
object-oriented with tens of thousands of lines, and, lo and behold, it works.
By all laws, it oughtn't. But it's happened.
Of course, I'm sure there are also people out there who have gotten burnt
trying to do that.
DDJ: Would you offer your comments about the future of programming. What kinds
of things do we need to understand in order to deal with software development
in the near future.
BS: I think we'll see much more emphasis on the design of classes and the
formal interfaces between parts, and an increase in the reuse of existing
programs and libraries of classes. We will need tools that help us do this and
draw structure or draw inference on what the structure is. Things like
performance analysis and coverage testing will all be available. It's worth
remembering that it's not a solitary activity -- not just one guy sitting
there with one machine; many of the key activities are social.
We'll also need to develop ways of talking about programs that are ahead of
what we are doing today. It's no good if we can compose programs and
components out of classes if we can't talk about that activity in a sensible
way. We don't have the vocabulary. It would be nice if we knew what object
oriented design was. By and large, we don't know it yet. But that will emerge
in five years and will be virtually accepted almost universally after that.








































Special Issue, 1989
C++ STRING CLASSES


An exercise in class design




Scott Robert Ladd


Scott is a computer programming addict with 15 years of experience in a wide
variety of languages. You can contact him via MCI Mail (369-4376), or at 705
W. Virginia, Gunnison CO 81230. Scott also maintains a BBS system dedicated to
computer programming and scientific subjects; its phone number is 303-641-5125
(300/1200/2400 bps, 8 bits, no parity, 1 stop bit).


Object-oriented programming is the art of breaking down a program into its
fundamental data types and their associated operations. A class defines a
specific data type and encapsulates both the data definition and the
operations for objects of that type. The fundamental problem facing the
beginning C++ programmer is learning how to create classes. Almost a language
unto itself, class definition in C++ requires an understanding of the
fundamental philosophies behind object-oriented programming. Once you've
grasped the process of class definition, you are well on the road to becoming
an object-oriented programmer.
Object-oriented programming is not a task to be undertaken lightly; it
requires forethought and planning. Therefore, the first action to be
undertaken when designing a class is to determine exactly what its purpose is
and how it will be used. In these days of extensive programming environments
and high-pressure schedules, it might seem unrealistic to expect programmers
to spend time designing their programs before writing them. In the case of
developing classes, however, planning is of the utmost importance. The
programmer must have a clear idea of the goal of a class before writing the
first line of code.
A class in C++ is a definition of a new data type. For all intents and
purposes, this new data type is an analog to those which C++ predefines.
Similar to the int, float, and char types, the types defined by classes have
their own built-in rules, operations, and attributes. This is known as
"encapsulation," where form and function are linked. Encapsulation defines and
controls exactly what a type can and cannot do.
Furthermore, a class also offers the capability to do "data abstraction." When
we use a predefined type, such as a float, we are not concerned with exactly
how floating-point numbers are stored and manipulated. The actual
implementation of the float type will vary from architecture to architecture.
If we had to rewrite our floating-point code every time we ported it between
computers, there would be very little porting of code going on. Instead,
floating-point values are abstracted. We know that we can do assignments and
mathematical operations on them, and we write our programs using the
high-level definition of these activities. We don't need to understand the
complexities of binary multiplication to multiply floats; we assume the
compiler understands this already, and will do it for us automatically.
Classes give us the capability to "hide" the actual implementation of our own
data types so that the users of those classes can make assumptions about how
they work. Data abstraction frees the programmer from becoming involved in the
details.


A String Class


The first project many budding C++ programmers undertake is the development of
a character string class. One of C's primary faults is that it lacks the
sophisticated string handling available in languages such as Pascal and Basic.
Strings are quite useful: Nearly every program manipulates text data of one
type or another. A string class was one of my first projects, and during the
two years of its existence the class has undergone substantial changes. As my
understanding of C++ has grown, so has my ability to build a better class. I
am quite happy with the current incarnation, which works well in applications
ranging from data bases to text editors. The entire class is shown in Listings
One, string.hpp, (page 68) and Two, string.cpp (page 68). Listing Three,
strtst.cpp, (page 69) shows a program to exercise the String class.
My goal was to create a dynamically allocated string class that would provide
all the functionality of standard, NULL-terminated C character arrays (which I
call "C-strings"). However, I wanted to avoid the pitfalls of C-strings; for
example, errors often occur when working with C-strings because they fail to
do any sort of range or validity checking. In addition, the start library
functions defined in string.h are missing important features. In order for
strings to be useful in a wide variety of applications, they needed
manipulation routines not normally found in C function libraries, such as
those for inserting and deleting data.


The private Section


Listing One shows the file string.hpp, which contains the definition of the
String class. A String is defined as having three private instance variables:
Siz, Len, and Txt. Siz contains the currently allocated length of the Txt
pointer; Len holds the actual number of characters stored in String. The char
pointer Txt points to the location on the heap of the buffer containing the
String's text data. Every instance of String will have its own, unique
variables with these names.
AllocIncr is not an instance variable; rather, it is a private "static class
member." Any class data item defined as static has only one occurrence, shared
by the entire class. In this case, there is only one copy of AllocIncr, which
is common to all String objects. The purpose of private static class members
is to eliminate global variables; they should be used whenever there is a data
item that is accessed only from within a class scope. There's no need for
anything external to the String class to "see" AllocIncr, so it is safely
locked away from outside manipulation.


The public Section


In general, class methods are made public to facilitate their use by
user-defined objects. In the case the String class, however, the method Shrink
is used only internally by the class. Shrink adjusts the buffer space
allocation for a string in order to eliminate wasted space. It is not meant to
be called from outside the class scope, and thus it is declared in the private
section of the String class definition.
The public section of the String class begins by defining two enumerated
types: StrCompVal and StrCompMode. StrCompVal is the return value of the
Compare method; StrCompMode is used to indicate whether or not String
comparisons are case-sensitive. I use enumerations for these types so that I
can control the validity of values being passed to and returned from methods.
The types must be public so that the enumeration constants are available to
the user of the class.
The remainder of the public section defines all of the other methods
associated with the String class. The first four of these are constructors,
which are specialized methods used to initialize (that is, construct) new
String objects. Objects tend to be complex, and constructors allow the
programmer complete control over how the instance variables of an object are
loaded with values when an object is instantiated (comes into scope).
The constructor String() is used to create an empty, uninitialized string.
String(String & Str) is known as the "copy constructor," it copies one String
object into another. String(char * Cstr) allows a newly created String to be
initialized with the value of a C-string. The last constructor,
String(charFillCh, unsigned int Count), creates a new string, which contains
Count FillCh characters.
Copy constructors require a bit of explanation. The C++ compiler will generate
a copy constructor for you if you don't design one yourself. The generated
copy constructor simply assigns the instance variables of one object to
another. This default copy constructor won't work for most classes, including
String. Each String contains a pointer to a buffer. When we create a copy of a
String, we want it to have its own, unique buffer. The default copy
constructor will merely assign the address of the existing buffer to the new
Strings Txt instance variable, giving us two objects that are using the same
memory space. Therefore, we need to define a copy constructor that duplicates
the buffer from the original String for the new String.
The next method, ~String(), is a destructor. Destructors are called whenever
an object is deleted or goes out of scope. As with constructors, most complex
classes will require an explicitly defined destructor. A default constructor
is created, but it merely frees the space being used by the instance variables
of an object. This will not work for Strings; the Txt pointer locates spaces
allocated on the stack, and this space must be deallocated to avoid wasting
memory. The ~String() destructor handles this for us.
Remember that constructors and destructors are called automatically by the
compiler. Every time an object is created, a constructor is called for it;
every time an object is destroyed, a destructor is called. As we shall see
shortly, constructors tend to get called far more often than is immediately
apparent, and the programmer must be aware of these hidden method function
calls.
The Length() and Size() methods simply return the current length and
allocation size, respectively, of a string. Length corresponds to the string.h
function strlen(). Size was originally created to help in testing the class;
because it is so simple, I just left it in for future use. Another simple
method is Empty(), which clears a string to the blank -- or empty -- value,
with a length of 0 and an allocation of 8 bytes.
The remaining methods manipulate the value of String. Copy() and Dupe()
provide similar functions; Copy() copies the contents of String into a
pre-defined C-string, and Dupe() returns a pointer to a C-string (filled with
the value of its Txt buffer) it created on the heap. Copy() is useful when an
existing C-string needs to be filled with the value of String. Dupe() can be
used as the equivalent of the standard function strdup().
Next we come to a series of operator definitions. The first is the operator =
method, which handles assignments between Strings. Note that this is not
duplicating the function of the copy constructor discussed earlier. The copy
constructor is used when a new String is created; the assignment operator is
used when the value of an existing String is assigned to another extant
String.
You may wonder why I didn't create an assignment operator to copy a C-string
to an existing String. The truth is that there is no need for such a method.
Earlier, we defined a constructor (String(char * Cstr)) that created a String
from a C-string. Whenever a C-string is used in a method invocation that
expects a String as a parameter, the C-string is automatically converted via
the constructor to a temporary String. Once the temporary String has been
used, it is destructed.
This principle carries over to the methods for the two additive operators, +
and + =. Rather than define methods for every possible combination of adding
String and a C-string, these methods operated entirely on Strings. If a
C-string is passed as one of the arguments to these methods, it is
automatically converted to String by the String(char * Cstr) constructor.
The advantage of using conversion constructors such as String(char * Cstr) is
simplicity. Only one method needs to be defined for each operation, and the
constructor can take care of the rest. To add compatibility with another data
type (say, an alternative string class), you merely need to create a single
conversion constructor. The drawback is overhead: custom-written methods for
each combination of values will not incur the overhead of hidden constructor
and destructor calls.
The next method declared is Compare(). It compares two strings and returns an
enumeration of type StrCompVal, which indicates the relationship between the
two values. This works much like the standard function strcmp, with the
exception that the Case parameter determines whether the comparison is
case-sensitive. It's possible to define a series of operator methods for the
<, >, and = = operations as well; I've found that Compare() works well enough
for my purposes.
The method Find() duplicates the purpose of the standard function strstr(), by
locating a given String within another String. It returns an index indicating
where the substring begins. Similar to Compare, the Case parameter defaults to
SF_IGNORE to indicate a case-insensitive comparison. The search can be made
case sensitive by specifying the Case parameter as SF_SENSITIVE.
The Delete() method removes a specified number of characters from a string.
The parameter Pos provides the index of the first character to be deleted, and
the Count parameter indicates how many characters should be deleted.
Characters and strings can be inserted into String at any position with the
Insert() methods. The first method listed inserts a single character, and the
second inserts an entire String. Once again, the conversion constructor allows
us to use a C-string in place of the String parameter.
Occasionally, it is necessary to extract a section of a string. The SubStr
method accomplishes this by copying Count characters beginning at Pos into
another String.
Last but not least, the indexing operator is defined. This allows us to
extract single characters at specific positions within String in the same
fashion as we would if working with a C-string. If the position given is
beyond the last character of String, a NULL character is returned.



The Implementation


With the just mentioned definition and a copy of Listing One (string.hpp), a
programmer should be able to use and expand upon String objects without ever
seeing the implementation of the String class. There are, however, some
interesting facets of method implementation that can be seen by studying
Listing Two (string.cpp). It is particularly important to see the way in which
objects are returned from functions.
If you examine the methods that return String, you will see what looks like a
violation of proper programming practice. For example, SubStr() copies the
substring into a local String object -- TempStr. Then, it returns TempStr, a
seemingly dangerous act. After all, isn't the destructor for TempStr called
when the method is exited, meaning that the recipient of the method's return
value will get garbage?
Again, C++ is trickier than it looks. When an object is returned from a
function, the copy constructor is called first to copy the return value into
its destination, and then the destructor is called. This makes it much easier
to write methods that return objects.


Summing Up


As I mentioned earlier, it is possible to improve this class. The most obvious
improvement is to add methods for the comparison operators (<, >, and = =). I
haven't needed them, but perhaps your application will.
The String class does not contain any inline methods. I tend to be cautious
about making methods inline. Remember that under the C++ definition, inline
methods can be treated as regular method functions by the compiler. Such as
the register keyword, inline methods can be ignored by the compiler. No C++
compiler I know of will actually inline a method that contains complex control
statements like loops and switches; these methods are made into function
methods, regardless of their declaration as inline. Not all C++ translators
can inline methods containing if statements, either.
Inline functions also tend to be abused by programmers who are learning C++.
While an inline method is certainly faster than a function method, it can
cause severe code-size increases. You need to analyze which methods are being
used the most, and determine what the speed versus size trade-offs are. In the
String class, the most obvious candidates for inlining are the simple methods
like Length().
This class has served me well in a number of complex applications. Work with
and modify it; if you come up with interesting alternatives and changes, I'd
like to hear about your experiences.

_C++ STRING CLASSES_
by Scott Robert Ladd

[LISTING ONE]


// Header: String (Dynamic Strings)
// Version: 1.01 13-Sep-1989
// Language: C++ 2.0
// Environ: Any
// Compilers: Zortech C++
// Purpose: Provides a general dynamic string class.
// Written by: Scott Robert Ladd
// 705 West Virginia
// Gunnison CO 81230
// MCI ID: srl
// FidoNet: 1:104/45.2

#if !defined(STRING_HPP)
#define STRING_HPP

#include "stddef.h"

class String
 {
 private:
 // instance variables
 unsigned int Siz; // allocated size
 unsigned int Len; // current length
 char * Txt; // pointer to text
 // class constant
 static unsigned int AllocIncr;
 // private method used to shrink a string to its minimum allocation
 void Shrink();
 public:
 enum StrCompVal {SC_LESS, SC_EQUAL, SC_GREATER};
 enum StrCompMode {SF_SENSITIVE, SF_IGNORE};
 // constructor
 String();
 String(String & Str);
 String(char * Cstr);
 String(char FillCh, unsigned int Count);

 // destructor
 ~String();
 // value return methods
 unsigned int Length();
 unsigned int Size();
 // Function to return a blank string
 friend String Empty();
 // copy String to c-string method
 void Copy(char * Cstr, unsigned int Max);
 // create a c-string from String method
 char * Dupe();
 // assignment method
 void operator = (String & Str);
 // concatenation methods
 friend String operator + (String Str1, String Str2);
 void operator += (String Str);
 // comparison method
 StrCompVal Compare(String Str, StrCompMode Case = SF_IGNORE);
 // substring search methods
 int Find(String Str, unsigned int & Pos, StrCompMode Case = SF_IGNORE);
 // substring deletion method
 void Delete(unsigned int Pos, unsigned int Count);
 // substring insertion methods
 void Insert(unsigned int Pos, char Ch);
 void Insert(unsigned int Pos, String Str);
 // substring retrieval method
 String SubStr(unsigned int Start, unsigned int Count);
 // character retrieval method
 char operator [] (unsigned int Pos);
 // case-modification methods
 String ToUpper();
 String ToLower();
 };
#endif





[LISTING TWO]

// Module: String (Dynamic Strings)
// Version: 1.01 13-Sep-1989
// Language: C++ 2.0
// Environ: Any
// Compilers: Zortech C++
// Purpose: Provides a general dynamic string class.
// Written by: Scott Robert Ladd
// 705 West Virginia
// Gunnison CO 81230
// MCI ID: srl
// FidoNet: 1:104/45.2

#include "String.hpp"
#include "string.h"
#include "stddef.h"
#include "ctype.h"

// class-global constant intialization

unsigned int String::AllocIncr = 8;

// private function to shrink the size of an allocated string
void String::Shrink()
 {
 char * Temp;
 if ((Siz - Len) > AllocIncr)
 {
 Siz = ((Len + AllocIncr - 1) / AllocIncr) * AllocIncr;
 Temp = new char[Siz];
 memcpy(Temp,Txt,Len);
 delete Txt;
 Txt = Temp;
 }
 }

// constructor
String::String()
 {
 Len = 0;
 Siz = AllocIncr;
 Txt = new char[Siz];
 Txt[0] = '\x00';
 }
String::String(String & Str)
 {
 Len = Str.Len;
 Siz = Str.Siz;
 Txt = new char[Siz];
 memcpy(Txt,Str.Txt,Len);
 }
String::String(char * Cstr)
 {
 Len = strlen(Cstr);
 Siz = ((Len + AllocIncr - 1) / AllocIncr) * AllocIncr;
 Txt = new char[Siz];
 memcpy(Txt,Cstr,Len);
 }
String::String(char FillCh, unsigned int Count)
 {
 unsigned int Pos;
 Siz = ((Count + AllocIncr - 1) / AllocIncr) * AllocIncr;
 Len = Siz;
 Txt = new char[Siz];
 memset(Txt,FillCh,Count);
 }

// destructor
String::~String()
 {
 delete Txt;
 }

// value return methods
unsigned int String::Length()
 {
 return Len;
 }
unsigned int String::Size()

 {
 return Siz;
 }

// Function to return a blank string
String Empty()
 {
 static String EmptyStr;
 return EmptyStr;
 }

// copy String to c-string method
void String::Copy(char * Cstr, unsigned int Max)
 {
 unsigned int CopyLen;
 if (Max == 0)
 return;
 if (Len >= Max)
 CopyLen = Max - 1;
 else
 CopyLen = Len;
 memcpy(Cstr,Txt,CopyLen);
 Cstr[CopyLen] = '\x00';
 }

// create a c-string from String method
char * String::Dupe()
 {
 char * new_cstr;
 new_cstr = new char[Len + 1];
 memcpy(new_cstr,Txt,Len);
 new_cstr[Len] = '\x00';
 return new_cstr;
 }

// assignment method
void String::operator = (String & Str)
 {
 Len = Str.Len;
 Siz = Str.Siz;
 delete Txt;
 Txt = new char[Siz];
 memcpy(Txt,Str.Txt,Len);
 }

// concatenation methods
String operator + (String Str1, String Str2)
 {
 unsigned int NewLen, NewSiz, CopyLen;
 String TempStr;
 char * Temp;
 TempStr = Str1;
 CopyLen = Str2.Len;
 NewLen = TempStr.Len + Str2.Len;
 NewSiz = TempStr.Siz + Str2.Siz;
 Temp = new char[NewSiz];
 memcpy(Temp,TempStr.Txt,TempStr.Len);
 delete TempStr.Txt;
 TempStr.Txt = Temp;

 memcpy(&TempStr.Txt[TempStr.Len],Str2.Txt,CopyLen);
 TempStr.Len = NewLen;
 TempStr.Siz = NewSiz;
 TempStr.Shrink();
 return TempStr;
 }
void String::operator += (String Str)
 {
 unsigned int NewLen, NewSiz, CopyLen;
 char * Temp;
 CopyLen = Str.Len;
 NewLen = Len + CopyLen;
 NewSiz = Siz + Str.Siz;
 Temp = new char[NewSiz];
 memcpy(Temp,Txt,Len);
 delete Txt;
 Txt = Temp;
 memcpy(&Txt[Len],Str.Txt,CopyLen);
 Len = NewLen;
 Siz = NewSiz;
 Shrink();
 }

// comparison method
StrCompVal String::Compare(String Str, StrCompMode Case)
 {
 char * Temp1, * Temp2;
 Temp1 = new char[Len + 1];
 Copy(Temp1,Len+1);
 Temp2 = new char[Str.Len + 1];
 Str.Copy(Temp2,Str.Len+1);
 if (Case == SF_IGNORE)
 {
 strupr(Temp1);
 strupr(Temp2);
 }
 switch (strcmp(Temp1,Temp2))
 {
 case -1: return SC_LESS;
 case 0: return SC_EQUAL;
 case 1: return SC_GREATER;
 }
 delete Temp1;
 delete Temp2;
 }

// substring search methods
int String::Find(String Str, unsigned int & Pos, StrCompMode Case)
 {
 char * TempStr1, * TempStr2;
 unsigned int LastPos, SearchLen, TempPos;
 int Found;
 TempStr1 = new char[Len + 1];
 memcpy(TempStr1,Txt,Len);
 TempStr1[Len] = '\x00';
 TempStr2 = new char[Str.Len + 1];
 memcpy(TempStr2,Str.Txt,Str.Len);
 TempStr2[Str.Len] = '\x00';
 if (Case == SF_IGNORE)

 {
 strupr(TempStr1);
 strupr(TempStr2);
 }
 Pos = 0;
 TempPos = 0;
 Found = 0;
 SearchLen = Str.Len;
 LastPos = Len - SearchLen;
 while ((TempPos <= LastPos) && !Found)
 {
 if (0 == strncmp(&TempStr1[TempPos],TempStr2,SearchLen))
 {
 Pos = TempPos;
 Found = 1;
 }
 else
 ++TempPos;
 }
 delete TempStr1;
 delete TempStr2;
 return Found;
 }

// substring deletion method
void String::Delete(unsigned int Pos, unsigned int Count)
 {
 unsigned int CopyPos;
 if (Pos > Len)
 return;
 CopyPos = Pos + Count;
 if (CopyPos >= Len)
 Txt[Pos] = 0;
 else
 while (CopyPos <= Len)
 {
 Txt[Pos] = Txt[CopyPos];
 ++Pos;
 ++CopyPos;
 }
 Len -= Count;
 Shrink();
 }

// substring insertion methods
void String::Insert(unsigned int Pos, char Ch)
 {
 char * Temp;
 if (Pos > Len)
 return;
 if (Len == Siz)
 {
 Siz += AllocIncr;
 Temp = new char[Siz];
 memcpy(Temp,Txt,Len);
 delete Txt;
 Txt = Temp;
 }
 if (Pos < Len)

 for (unsigned int Col = Len + 1; Col > Pos; --Col)
 Txt[Col] = Txt[Col-1];
 Txt[Pos] = Ch;
 ++Len;
 }
void String::Insert(unsigned int Pos, String Str)
 {
 unsigned int SLen = Str.Len;
 SLen = Str.Len;
 if (SLen > 0)
 for (unsigned int I = 0; I < SLen; ++I)
 {
 Insert(Pos,Str.Txt[I]);
 ++Pos;
 }
 }

// substring retrieval method
String String::SubStr(unsigned int Start, unsigned int Count)
 {
 String TempStr;
 char * Temp;
 if ((Start < Len) && (Count > 0))
 for (unsigned int Pos = 0; Pos < Count; ++Pos)
 {
 if (TempStr.Len == TempStr.Siz)
 {
 TempStr.Siz += AllocIncr;
 Temp = new char[TempStr.Siz];
 memcpy(Temp,TempStr.Txt,Len);
 delete TempStr.Txt;
 TempStr.Txt = Temp;
 }
 TempStr.Txt[Pos] = Txt[Start + Pos];
 ++TempStr.Len;
 }
 return TempStr;
 }

// character retrieval method
char String::operator [] (unsigned int Pos)
 {
 if (Pos >= Len)
 return '\x00';
 return Txt[Pos];
 }

// case-modification methods
String String::ToUpper()
 {
 String TempStr = *this;
 for (unsigned int Pos = 0; Pos < Len; ++Pos)
 TempStr.Txt[Pos] = toupper(TempStr.Txt[Pos]);
 return TempStr;
 }
String String::ToLower()
 {
 String TempStr = *this;
 for (unsigned int Pos = 0; Pos < Len; ++Pos)

 TempStr.Txt[Pos] = tolower(TempStr.Txt[Pos]);
 return TempStr;
 }





[LISTING THREE]

#include "String.hpp"
#include "stdio.h"
#include "stream.hpp"

int main();
void print_string(String S);

String s1;
String s2("This is the second string!");

int main()
 {
 String ls;
 String ls2("Another local string");
 unsigned int pos, i;
 char ch;

 s1 = s2;
 ls = "This is the local string.";
 print_string(s1);
 print_string(s2);
 print_string(ls);
 print_string(ls2);
 cout << "\n";

 s1 = s2 + ls;
 print_string(s1);

 s1 = "String one has a value.";
 print_string(s1);

 s1 = s1 + "****";
 print_string(s1);

 s2 += "*****";
 print_string(s2);
 print_string(ls);

 s2 += ls;
 print_string(s2);

 cout << "\n";

 if (s2.Find("Burfulgunk",pos))
 printf("first search = %d\n",pos);
 if (s2.Find("*****",pos))
 printf("second search = %d\n",pos);
 if (s2.Find(ls,pos))
 printf("third search = %d\n",pos);

 if (s2.Find(s1,pos))
 printf("fourth search = %d\n",pos);
 ls2 = "&&";

 s1.Insert(10,'*');
 s1.Insert(15,ls2);
 print_string(s1);

 s1.Insert(s1.Length(),'%');
 s1.Insert(s1.Length(),'%');
 s1.Insert(s1.Length(),'%');
 s1.Insert(s1.Length(),'%');
 print_string(s1);

 for (i = 0; 0 != (ch = s1[i]); ++i)
 putchar(ch);
 putchar('\n');

 s1.Insert(2,"<><><><><>");
 print_string(s1);

 s1.Delete(2,10);
 print_string(s1);

 s2 = s1.ToUpper();
 print_string(s2);

 s2 = s1.ToLower();
 print_string(s2);

 s1 = Empty();
 print_string(s1);

 s1 = s2.SubStr(2,10);
 print_string(s1);

 return 0;
 }
void print_string(String S)
 {
 char * cs;
 cs = S.Dupe();
 cout << cs << " Len = " << S.Length() << " Siz = " << S.Size() << "\n";
 delete cs;
 }

















Special Issue, 1989
DISCRETE EVENT SIMULATION IN CONCURRENT C


Simulating discrete events with concurrent processing




N.H. Gehani and W.D. Roome


Narain and William are the architects of Concurrent C and authors of The
Concurrent C Programming Language (Silicon Press) from which this article is
adapted. They can be reached at AT&T Bell Labs, 600 Mountain Ave., Murray
Hill, NJ 07974.


In discrete event simulation, events in the simulated system happen at
discrete times, and these are the only times at which the system changes
state. For example, when simulating a queue of customers waiting for service
at a bank, the events include a new customer arriving, a customer reaching the
head of the line and being served by a teller, a customer leaving, and so on.
The key points are the times at which these events happen; as far as the
simulation is concerned, nothing happens between the events. Discrete event
simulation differs from "continuous time" simulation, which is used to
simulate continuous systems such as water flowing over a dam.
Concurrent programming can simplify the task of writing a discrete event
simulation program; the resulting program is easy to understand and modify. In
particular:
The simulation program is easy to write: For each entity type, a simple
sequential process that describes the steps performed by the entity is
written. These processes communicate when the entities they are simulating
need to communicate or synchronize. This is called the "process-interaction
model" of simulation.
It is easy to reuse entity processes from previous simulations or to create a
library of processes for standard entities.
Even if entity processes from previous simulations cannot be reused as is,
they can be used as templates that can be customized for new simulations.
It is easy to capture statistics for each entity: Just record them in local
variables in each entity process (that is, a running average). If there are
several instances of the same entity, each process automatically gets its own
versions of these variables.
The processes can be automatically distributed over the processors in a
multiprocessor system. This can speed up a computationally intensive
simulation program (that is, one in which the entity processes are
compute-bound between process interactions).
We will illustrate these points by showing how to write a Concurrent C program
that models a multi-stage, multiserver queuing network. Each queue and each
server is modeled by a Concurrent C process. This is a natural way to simulate
a queuing network: Each process runs independently, as do the queue and
servers in the real network, and they interact when necessary, for example,
when a server takes a job from its input queue. We'll present several general
processes that can be used in other simulation programs; these include an
event scheduler process and a queue manager process.


The Process-Interaction Model


Consider the queuing system in Figure 1 where the simulation uses three
processes: One for the source, one for the queue, and one for the server. The
source process generates items ("jobs"). The inter-arrival time -- the time
between item arrivals -- is a random variable. The body of the source process
consists of a simple loop that calculates the next inter-arrival time, waits
for that many time units, generates a new item, and then places the new item
on the queue as in Example 1.
Example 1: The server process

 while (1) {
 delay for random inter-arrival
 time;
 generate item;
 call queue process to put item
 in queue;
 }

Similarly, the server process repeatedly takes the next item from the queue,
processes it for the appropriate service time, and finally discards the item.
The queue accepts items from the source and gives them to the server.


General Method


The first step in the process-interaction model is to determine the
sequential, independent entities in the system being simulated. The entities
in a queuing network include sources, queues, and servers. Each entity
performs a well-defined series of operations. Some of these operations may
require interaction with another entity; one example is a server taking an
item from a queue. Other than that, each entity is independent of the other
entities.
The next step is to identify the types of interactions between the entities.
Each such interaction becomes a transaction call. It is convenient to divide
entities into two categories: Active entities, such as servers, and passive
entities, such as queues. In general, passive entities wait for requests, and
usually represent "resources" that are used by active entities. The process
that implements a passive entity has one transaction for each type of request.
For example, a queue process will have a put transaction to put an item into
the queue, and a take transaction to remove the next item from the queue. The
process for an active entity, such as the server, is not called by the other
processes, and thus does not have transactions. This is not an absolute rule.
For example, while handling a request, a passive process might actively
request service from another passive process. But it is often a useful
paradigm for structuring the processes in a simulation program.
For each distinct type of entity, we then write the specification and body for
the Concurrent C process that simulates it. If the simulation needs several
entities of the same type, we can create an instance of the corresponding
process type for each entity. The process for simulating an active entity is a
simple sequential program that performs the entity's operations. The process
for simulating a passive entity consists of a loop that repeatedly executes a
select statement with alternatives for all of its transactions. In general,
each process keeps statistics, such as the mean time in system, and prints
them out at the end of the simulation.
The final step is to write a main process that creates all the entity
processes and connects these processes appropriately.


Scheduler Process


Delays in simulated time -- such as the service time delay -- are handled by a
scheduler process. (Simulated time is different from actual time, so we cannot
use the Concurrent C delay statement to simulate service or arrival delays.)
This scheduler process maintains the current simulated time and advances it
appropriately. You can think of the scheduler as maintaining a clock that
gives the current simulated time. For each delay request from a process, the
scheduler determines the simulated time at which the process is to be
reactivated, and saves this request in an activation request list. When all
processes are waiting for delays to expire, the scheduler searches this list
for the entry with the lowest activation time. The scheduler then advances the
simulated clock to this time, removes this entity from the list, and
reactivates the selected process. If several processes are waiting to be
reactivated at the same simulated time, the scheduler awakens all of them
simultaneously. Any computation done by a process takes place in zero
simulated time.
One complication is that a process can be waiting for an event other than an
explicit delay request. For example, suppose that a server process tries to
take an item from an empty queue. The server process waits for the queue
process, which is waiting for a source process to put an item into the queue.
The source process is waiting for a delay to expire, at which point it will
place a new item in the queue. Thus, the scheduler advances the simulated time
when every process is either waiting for a delay request to expire or is
waiting for some event that will be generated by a process that is waiting for
a delay request to expire.
Before showing how the scheduler process can determine when this has happened,
we need to introduce some formalism. At any given time, each entity process is
in one of these three states:
waiting: Waiting for an explicit delay request from the scheduler


active: Computing in zero simulated time

passive: Waiting for something other than a delay request
As an example, consider a source process that waits for a random inter-arrival
delay and then places a job on the queue. The source process starts in the
active state. It enters the waiting state when it delays for the inter-arrival
time; it becomes active again when the scheduler reactivates it. If the Q is
not full, the source process stays active while it places the job in the
queue. However, if the queue is full, the source process enters the passive
state, and remains passive until some server process creates space by removing
a job from the queue.
Thus the scheduler accepts delay requests until there are no active entity
processes, at which time the scheduler advances the simulated clock and
reactivates the appropriate process(es). The scheduler knows the number of
processes waiting for the delay requests, so all it needs to know is the
number of passive processes. To give the scheduler this information, we will
call a scheduler transaction whenever a process changes state from active to
passive or vice versa.


A Two-State Queuing Network


We will develop a simulation program for the queuing network in Figure 2. In
this network, jobs enter queue 1 from a Poisson source (exponential
inter-arrival times). The arrows indicate the direction in which jobs flow.
The first stage has two servers (servers 1.1 and 1.2), each of which takes the
next available job from a common queue. The second stage has one server; it
takes jobs from the second queue, and discards them when done. The queues are
FIFO (first-in-first-out), and can hold at most 100 jobs. If a queue is full,
the server or source putting an item into this queue is blocked until space is
available. Similarly, if a queue is empty, the server taking an item from the
queue is blocked until an item is available (if more items are expected) or an
end-of-file is indicated to the server. The service times are exponentially
distributed, and are determined when a job enters the system. Servers 1.1 and
1.2 run at half the speed of server 2; that is, if server 2 takes x time units
to process a job, server 1.1 takes 2x units.


Structure of the Simulation Program


In this section, we present the interfaces to the processes in the simulation
program for the queuing network described in the previous section; the next
section describes how these processes are implemented. This simulation program
uses four types of processes:
sched: Simulated-time scheduler
process queue: Finite-capacity FIFO queue
source: Source of arriving jobs
server: Single-input, single-output server
The main process creates these processes and connects them appropriately.
The simulation program terminates after simulating the specified number of
jobs. The source process terminates after generating these jobs. Before
terminating, the source process informs its output queue that no more jobs
will arrive. When the queue is empty (and no more jobs are expected), the
queue process informs the servers that no more jobs will arrive. Each server
then prints its statistics, tells its output queue that it is done, and
terminates. In this way the queues are drained automatically and all processes
eventually terminate.
Strictly speaking, this causes our statistics to be distorted by the "edge
effects" of filling up the queues at the beginning of the simulation. And
draining them at the end. However, we will simulate enough jobs to mitigate
this distortion.


Scheduler Process Interface


The scheduler process manages simulated time. The time units are arbitrary. We
will refer to the processes that call the scheduler's transactions as the
"clients" of the scheduler. The specification of the scheduler process is in
Listing One, page 71.
A delay request requires calling the transactions reqDelay and wait. The
client process first calls reqDelay, giving the number of time units to delay,
and then calls wait, giving the value returned by reqDelay. The wait
transaction returns the simulated time as the end of the delay. Thus, if s is
the scheduler process, the following statement delays the calling process for
ten time units, and saves the time at the end of the delay in ts: ts =
s.wait(s.reqDelay(10));
The scheduler accepts reqDelay calls until it has received a request from
every active client process. Then the scheduler accepts the wait call from the
process with the smallest delay request.
For this to work, the scheduler must know the number of its client processes,
and must know how many of them are in the passive state. Each client process
calls transaction addUser when it starts, and calls transaction dropUser
before it terminates, so that the scheduler can maintain a client count.
Whenever an active client process becomes passive, the scheduler is informed
by calling transaction passive. This call is made by the server process that
forces a client to wait. Whenever a passive process becomes active, the
process that makes it active calls the scheduler's active transaction. Thus
the scheduler can determine the number of passive processes.


Source Process Interface


The source process has the specifications shown in Listing Two, page 71.
Arrivals are Poisson, and service times are exponentially distributed. The
process parameters define the output queue, the mean values for the
distributions, and so on.
The name parameter is a symbolic name, such as "source1," which the source
process uses to identify itself when printing statistics (a simulation program
could have several different source processes). The type name_t is a structure
containing a character string:
 typedef struct {char str[20];} name_t;
This structure is passed by value to the source process. A simpler alternative
would be to declare the parameter as a character pointer. However, if we do
that, our simulation program will work only when run on an implementation that
provides shared memory. Passing a structure by value, on the other hand, will
work on any Concurrent implementation.


Server Process Interface


The server process has the specifications shown in Listing Three, page 71. If
the output queue parameter is c_nullpid, then the server discards each job
after processing it. The speed parameter is the relative speed of this server;
this server takes xspeed time units to process a job whose service time is x.


Queue Process Interface


Each job in a queue is represented by a structure of type Item, as shown in
Listing Four, page 71. We save the job arrival time in this structure so that
we can calculate statistics for the time spent by jobs in the system.
The queue process simulates a FIFO queue, as shown in Listing Five (page 71).
A queue process can have several clients. Clients are either consumers
("takers") or producers ("putters"). A source process is a producer; a server
process is a consumer for its input queue and a producer for its output queue.
Each producer process calls transaction addProd when it starts, and calls
transaction dropProd before it terminates. Similarly, each consumer process
calls transactions addCons and dropCons. This allows the queue process to keep
track of the number of producer and consumer clients. When the queue is empty
and the last producer has terminated, all subsequent take requests return an
end-of-file indication. The queue process terminates when it has no more
producers or consumers.
Transactions putReq and putWait put an item onto the queue, and transactions
takeReq and takeWait take the next item from the queue. However, client
processes do not call these transactions directly. Instead, these processes
call functions qPut and qTake, which perform the put and take operations. Both
functions wait until the operation can be performed. If an operation cannot be
completed immediately, the queue process informs the scheduler process of the
client's change of state (active to passive or passive to active). Function
Take returns 1 if able to take (get) an item, or 0 on end-of-file. qPut and
qTake are the interface functions for the queue process; they hide a
complicated transaction interface.
Example 2 takes jobs from the queue qFrom and places them on the queue qTo and
continues until qFrom is empty and all its producers have terminated. The
implementations of the qPut and qTake functions, and the body of the queue
process, will be described later.
Example 2: Moving jobs between queues


 qItem item;
 process queue qFrom, qTo;

 while (qTake (qFrom, &item))
 qput (qTo, item);



Statistical Functions


We will use a simple statistical package that calculates the mean and standard
deviation of a set of values, as shown in Listing Six, page 71. The statistics
are kept in a structure of type stats. Each of the functions declared earlier
takes a pointer to such a structure as its first argument. Function stInit
initializes the structure. Function stVal updates the statistics to reflect a
new value. Functions stMean and stDev return the mean and standard deviation
of these values. The random number generator errand(m) returns an
exponentially distributed integer whose mean is the inter value m. (Strictly
speaking, errand cannot be exponential because it returns integer values
instead of floating-point values. However, we will use large mean values so
that the round-off effect will be insignificant.)
Example 3 shows the code fragment that generates 1000 random numbers and
prints their average.
Example 3: Generating random numbers

 stats mystats;
 int i;

 stInit (&mystats);
 for (i = 1; i <= 10000; i++)
 stVal (&mystats, erand(100));
 printf ("Mean is %lf\n", stMean
 (&mystats));



Process Implementations


The main process, shown in Listing Seven, page 71, creates the processes that
simulate the entities and connects them together. Function makeName creates
and returns a structure of type name_t that contains the string passed as an
argument.
The structure of the queuing network -- the number of stages, the number of
servers per stage, and so on -- is determined by main and can easily be
changed. For example, we can add a third server to the first stage just by
replicating the line that creates server 1.2, but with a different symbolic
name.
Notice that main sleeps for a few seconds before terminating. This delay
allows other processes to call the transaction addUser of the scheduler and to
make their initial delay request (if any). The scheduler will now allow any
other process to proceed until main calls transaction dropUser. Delaying main
is necessary to allow other processes enough time to register with the
scheduler. Otherwise, the scheduler will terminate immediately after main
calls transaction dropUser.
We have assumed that all processes will be dispatched and will perform their
initialization within two seconds. To avoid this assumption, we could modify
main to call an initialization transaction belonging to each of the processes
that it creates; this initialization transaction would call transaction
addUser of the scheduler.


Source and Server Processes


Listing Eight, page 71, shows the body of the source process. First, the
source process initializes the statistics counters, and tells the scheduler
and queue processes that they have one more client. Then the source process
repeatedly calculates an inter-arrival time, delays for this amount, generates
a service time, and places a job in the output queue. Finally, the source
process prints its statistics and tells the scheduler and the output queue
that they have one less client. This pattern -- initialization, main
processing, and termination -- is common in our processes.
Listing Nine, page 71, shows the body of the server process. It starts by
telling the scheduler, the input queue, and the output queue to increment
their client, consumer, and producer counts, respectively. The server process
then repeatedly takes a job from its input queue, delays for the job's service
time, and places the job on its output queue. Finally, the server process
prints statistics on the time that job spent in the system. The server process
also tells its input and output queues and the scheduler to decrement their
client counts.


Queue Process


Listing Ten, page 71, shows the source for the interface functions for the
queue process. A put or take request that can be done immediately requires one
transaction call; otherwise two calls are required. As mentioned earlier,
functions qPut and qTake, which are called by the clients of a queue process,
implement the necessary protocol for interacting with the queue process. To
putan item, qPtr first calls putReq. If the queue is not full, putReq puts the
item and returns -1. If the queue is full, putReq returns a non-negative
"ticket" value. In this case, qPutcalls the queue's putWait transaction,
giving the ticket returned by putReq. When space becomes available in the
queue, the queue process accepts transaction putWait and puts the item on the
queue.
Function qTake is similar; the only difference is that the takeReq transaction
returns a structure that contains either the item taken or the ticket value.
The body of the queue process uses several auxiliary functions to manipulate
the data structures that maintain the queue's state. All the information
necessary to define a queue is collected in a structure of type qInfo. The
queue process passes a pointer from this structure to the auxiliary functions
shown in Listing Eleven, page 72.
Member items of the structure type qInfo point to a circular buffer qItem
structure. The integer head is the index of the next item that can be taken
from the queue, and tail is the index of the next slot into which an item can
be put. We keep statistics on "time in queue" and "number in queue." The
latter is sampled when a job is placed in a queue.
Ticket numbers returned by putReq and takeReq are assigned circularly from 0
and 9999. Put and take tickets are assigned independently. Structure tInfo
contains information about pending tickets. We need one instance of this
structure for pending puts, and another for pending takes. Listing Twelve,
page 72, shows the body of the queue process.
The queue process starts by initializing the qInfo structure, and then waits
for the first producer's addProc request. The queue process then accepts
requests as long as it has any clients. Before terminating, the queue process
prints the statistics that it has recorded.
We will now analyze handling of the "take" requests by the queue process in
detail. There are two alternatives for takeReq with mutually exclusive guards.
The first of these alternatives accepts requests whenever there are items in
the queue and when there are no pending takeWait requests (that is, no pending
take tickets). This alternative calls takeItem, which removes an item from the
buffer, updates the statistics, and sets the ticket and gotItem fields to
indicate that an item was taken immediately. Also, if another client is
blocked on a put request, takeItem tells the scheduler that the process is now
active. This ensures that the scheduler will not honor another delay request
until the process that is blocked on the put request is able to execute its
pending putWait transaction. This takeReq alternative returns the item
selected by takeItem to the client process.
The second takeReq alternative accepts requests whenever the queue is empty.
It assigns a ticket, updates the ticket data, tells the scheduler that an
active client has just become passive, and returns the ticket to the client.
There are two alternatives for takeWait. When there are items in the queue,
the first alternative accepts the request with the oldest pending ticket. This
alternative takes an item from the buffer and returns it to the client. When
end-of-file has occurred (when there are no producers and the queue is empty)
the second takeWait alternative accepts any remaining takeWait calls. This
alternative returns an end-of-file indicator to the client.
The put requests are handled similarly. Transactions addCons, dropCons,
addProd, and dropProd just update the consumer and producer counts. Listing
Thirteen, page 72, shows the auxiliary functions called from the body of the
queue process.


Scheduler Process



The scheduler keeps a list of pending delay requests, ordered by the time at
which the client is to be reactivated. List entries are pairs (t, n) where t
is the simulated time and n is the number of processes to be awakened at that
time. Listing Fourteen, page 72, is an outline of the body of the scheduler
process sched.
After initialization, the scheduler process repeatedly accepts requests until
all clients become passive. For each delay request, the scheduler calculates
the absolute time at which the requesting process should be reactivated, and
adds an appropriate entry to the list of pending delay requests. This list is
ordered by increasing reactivation times. List entries are allocated from an
array; the reqDelay transaction uses the array index as a ticket value. When
all client processes are passive, the scheduler takes the next request from
the list, advances the simulated time, and accepts wait requests from all the
clients waiting for the new simulated time.
The suchthat clause performs a linear search through the pending requests;
this will be inefficient if there are hundreds or thousands of pending delay
requests. For small simulations, however, the simplicity of the scheduler
makes up for the inefficiency that may result from this linear search. If this
search proves to be expensive in large simulations, we can eliminate the
suchthat clause by making the client process supply a transaction pointer with
which the scheduler will call the client when it is ready to accept the
client's delay request.


A Feedback Queuing Network


Consider the feedback queuing network in Figure 3. This network differs from
the previous network in two ways:
Each server in the first stage has its own queue. When a job arrives, the
source places the job in the shorter of the two queues.
With probability p, the second stage server discards a completed job; with
probability 1-p, it returns the job to queue 2 for further processing.
To simulate this network, we can reuse the server, queue, and scheduler
processes. We will need two new process types: A source process that puts the
job in the smaller of two queues, and another server process that discards a
job with some probability.
The new source process will be similar to the original source process, except
that it has two output queue parameters, outQ1 and outQ2. After generating a
job, the new source process puts the job on the smaller of the two output
queues, which it can do with the code fragment shown in Example 4.
Example 4: Choosing queues

 if (out Q1.itemCnt() < outQ2.
 itemCnt())
 qPut (outQ1, item);
 else
 qPut (outQ2, item);

The new server process is similar to the original server process, except that
it has a new parameter probDone, which is the probability of discarding a job
after processing it. Because this new server is a feedback server, it does not
tell the output queue that it is a consumer process. (If it did this, the
simulation would never terminate. That is, the server will not terminate until
its input queue indicates end-of-file. This will not happen until all of the
input queue's producers have terminated. If the server process were one of
these producers, it would never terminate.)


Extensions and Modifications


We will now discuss some ways of extending our simulation program to model
other queuing programs.
Other Queue Disciplines -- Our queue process can be easily modified to use a
last-in-first-out (LIFO) discipline, or a more complicated queue discipline,
such as shortest-service-time-first. This would require simple changes to the
putItem and takeItem functions, but the basic structure of the queue process
would remain unchanged.
Servers with Multiple Output Queues -- Modeling a more general queuing network
in which servers output to multiple queues is straightforward. For example,
the server could be given an array of output queue process identifiers, and
the server could randomly select the output queue for each job. Alternatively,
the server could send each job to all of its output queues.
More General Networks -- The structure of the queuing network is determined by
how the main process creates the source, server, and queue processes, and how
it connects them together. This is easy to extend; we could even write a main
process that reads a network description at run time.
Statistics -- In general, we want to measure the steady-state performance of a
queuing system. Our processes do not. Instead, they measure the entire
simulation run, including the initialization phase (starting with an empty
queue) and the termination phase (draining the queues after the source stops).
We compensate for this by simulating many jobs, so that the initialization and
termination phases are small compared to the steady-state period.
A better technique is to monitor the statistics during the simulation and
terminate the simulation when the statistics have been stable for a long time.
We can accomplish this by adding a giveStats transaction to each entity
process. This transaction will return the current statistics of the process to
which it belongs. After starting all processes, main would repeat this cycle
until it decided that the simulation had reached a steady state. Thus, main
would look something like Example 5. Transaction giveStats is easy to add to
the queue process; it is just another alternative in the central select
statement. For the source and server process, the central loop would be
modified to conditionally accept a giveStats transaction (that is, accept a
call if one is available, but continue if no call is pending).
Example 5: An example of main( )

 main()
 {

 create all processes
 while (1) {
 s.wait (s.reqDelay (10000));
 for each entity process p
 entity-stats = p.
 giveStats ();
 if (statistics have been stable
 for long enough)
 break;
 }
 print final statistics
 for all processes p
 c_abort (p);
 }


_DISCRETE EVENT SIMULATION IN CONCURRENT C_
by Narain Gehani and W.D. Roome



[LISTING ONE]

File: sim-sched.h

process spec sched() {
 trans long now() /* return simulated time */
 trans long reqDelay(long del);
 /* request delay */
 trans long wait(long x); /* wait for delay */
 trans void addUser(); /* add client */
 trans void dropUser(); /* drop client */
 trans void passive(); /* client is passive */
 trans void active(); /* client is active */
};






[LISTING TWO]

File: sim-source.h

process spec source(
 process sched s, /* scheduler */
 process queue outQ, /* output queue */
 long meanIat, /* mean inter-arrival time */
 long meanServt, /* mean service time */
 long nGen, /* number to generate */
 name_t name); /* symbolic name */






[LISTING THREE]

File: sim-server.h

process spec server (
 process sched s, /* scheduler */
 process queue inQ, /* input queue */
 process queue outQ, /* output queue */
 double speed, /* speed of server */
 name_t name); /* symbolic name */






[LISTING FOUR]

File: sim-qItem.h

typedef struct {

 /* Public: */
 long servt; /* service time for job */
 long arrive; /* time job arrived */

 /* Private to queue process: */
 long qEnter; /* time entered queue */
 int ticket; /* ticket from takeReq */
 int gotItem; /* !=0 if item was taken */
} qItem;






[LISTING FIVE]

File: sim-queue.h

process spec queue(
 process sched s, /* scheduler */
 int maxSize, /* max queue size */
 name_t name) /* symbolic name */
{
 trans int itemCnt(); /* return queue size */
 trans void addProd(); /* add producer */
 trans void dropProd(); /* drop producer */
 trans void addCons(); /* add consumer */
 trans void dropCons(); /* drop consumer */

 /* start and finish put request:*/
 trans int putReq(qItem);
 trans void putWait(int, qItem);

 /* start and finish take request:*/
 trans qItem takeReq();
 trans qItem takeWait(int);
};






[LISTING SIX]

File: sim-stats.h

typedef struct {
 long nv; /* number of values */
 long maxv; /* max value */
 double sumv; /* sum of values */
 double sumsq; /* sum of squares */
} stats;

void stInit(); /* initialize structure */
void stVal(); /* add new value */
double stMean(); /* return mean value */
double stSdev(); /* return standard deviation */

long erand(); /* exponential random number */







[LISTING SEVEN]

File: sim-main.cc

name_t makeName(name) char *name;
{ name_t ret;
 strcpy(ret.str, name);
 return ret;
}

main()
{ process sched s;
 process queue q1, q2;
 long nGen=100000; /* number of jobs */
 long servt=500; /* mean service time */
 long iat=1000; /* mean inter-arrival */

/* Create virtual time scheduler. */
 s = create sched(); s.addUser();

/* Create queues and servers. */
 q1 = create queue(s, 100, makeName("Q1"));
 q2 = create queue(s, 100, makeName("Q2"));
 create source(s, q1, iat, servt, nGen,
 makeName("Src"));
 create server(s, q1, q2, 0.5,
 makename("Serv1.1"));
 create server(s, q1, q2, 0.5,
 makename("Serv1.2"));
 create server(s, q2, c_nullpid, 1.0,
 makename("Serv2"));

/* Wait for all processes to start. */
 delay 2.0; s.dropUser();
}







[LISTING EIGHT]

File: si-source.cc

process body source(s, outQ, meanIat,
 meanServt, nGen, name)
{ stats iat, servt;
 qItem item;
 long i, t;


/* Initialization phase */
 s.addUser();
 outQ.addProd();
 stInit(&iat); stInit(&servt);

/* Main processing phase: generate jobs. */
 for (i=1; i <= nGen; i++) {
 t = erand(meanIat);
 stVal(&iat, t);
 item.arrive = s.wait(s.reqDelay(t));
 item.servt = erand(meanServt);
 stVal(&servt, item.servt);
 qPut(outQ, item);
 }
/* Termination phase: print stats, etc. */
 print statistics;
 outQ.dropProd(); s.dropUser();
}







[LISTING NINE]

File: sim-server.cc

process body server(s, inQ, outQ, speed, name)
{ stats sysTime; /* time-in-system */
 qItem item;
 long ts;

 s.addUser(); stInit(&sysTime);
 inQ.addCons();
 if (outQ != c_nullpid)
 outQ.addProd();

 while (qTake(inQ, &item)) {
 ts = s.wait(s.reqDelay(item.servt/speed));
 stVal(&sysTime, ts - item.arrive);
 if (outQ != c_nullpid)
 qPut(outQ, item);
 }

 print statistics;
 if (outQ != c_nullpid)
 outQ.dropProd();
 inQ.addCons(); s.dropUser();
}






[LISTING TEN]


File sim-qPut.cc

/* Put item onto queue; wait if full. */
void qPut(q, item)
 process queue q; qItem item;
{ int ticket = q.putReq(item);
 if (ticket >= 0)
 q.putWait(ticket, item);
}

/* Set *itemp to next item; wait if empty. */
/* Return 1 if item was taken, 0 on EOF */
int qTake(q, itemp)
 process queue q; qItem *itemp;
{
 *itemp = q.takeReq();
 if (itemp->ticket >= 0)
 *itemp = q.takeWait(itemp->ticket);
 return itemp->gotItem;
}






[LISTING ELEVEN]

File: sim-qInfo.h

/* *tInfo: Describe outstanding tickets. */
typedef struct {
 int acc; /* next ticket to accept */
 int give; /* next ticket to give out */
 int nPass; /* pending passive clients */
} tInfo;

/* qInfo: Describe one queue. */
typedef struct {
 process sched s; /* scheduler process */
 int max; /* max queue size */
 int nProd; /* number of producers */
 int nCons; /* number of consumers */
 name_t name; /* name of queue */
 stats qTime; /* time-in-queue stats */
 stats qSize; /* for queue size stats */
 int nElem; /* items in queue */
 int head; /* index head of queue */
 int tail; /* index tail of queue */
 qItem *items; /* alloc'd array of items */

/* Describe pending put, take requests: */
 tInfo pPut, pTake;
} qInfo;







[LISTING TWELVE]

File: sim-queue.cc

process body queue(s, maxSize, name)
{ qInfo q; qItem x;
 initialize qInfo structure;
 accept addProd() { q.nProd++; }

 while (q.nProd+q.nCons >0) {
 select {
 (q.nElem<q.max && q.pPut.acc==q.pPut.give):
 accept putReq(item)
 { putItem(&q, &item); treturn -1; }
 or (q.nElem==q.max):
 accept putReq(item)
 { s.passive(); q.pPut.nPass++;
 treturn incTick(&q.pPut.give); }
 or (q.nElem<q.max):
 accept putWait(qt, item)
 suchthat (qt == q.pPut.acc)
 { putItem(&q, &item);
 incTick(&q.pPut.acc); }
 or (q.nElem>0 && q.pTake.acc==q.pTake.give):
 accept takeReq()
 { treturn takeItem(&q); }
 or (q.nElem==0):
 accept takeReq()
 { x.ticket = incTick(&q.pTake.give);
 s.passive(); q.pTake.nPass++;
 treturn x; }
 or (q.nElem>0):
 accept takeWait(qt)
 suchthat (qt == q.pTake.acc)
 { incTick (&q.pTake.acc);
 treturn takeItem(&q); }
 or (q.nProd==0 && q.nElem==0):
 accept takeWait(qt)
 { x.gotItem = 0; treturn x; }

 or accept itemCnt() { treturn q.nElem; }
 or accept addCons() { q.nCons++; }
 or accept addProd() { q.nProd++; }
 or accept dropCons() { q.nCons--; }
 or accept dropProd() { q.nProd--; }
 }
 /* On EOF, make pending takers active. */
 if (q.nProd==0 && q.nElem==0)
 for (; q.pTake.nPass > 0; q.pTake.nPass--)
 s.active();
 }
 print statistics;
}







[LISTING THIRTEEN]

File: sim-qAux.cc

/* Increment ticket, return prev value. */
int incTick(tp)
 int *tp;
{ int t = *tp;
 *tp = (t+1)%10000;
 return t;
}

/* Remove and eturn the next item in the queue. */
qItem takeItem(qp)
 qInfo *qp;
{
 qItem item;
 item = qp->items[qp->head];
 item.ticket = -1;
 item.gotItem = 1;
 stVal(&qp->qTime, qp->s.now() - item.qEnter);
 qp->nElem--;
 qp->head = (qp->head+1) % qp->max;
 if (qp-pPut.nPass >0)
 { qp->s.active(); qp->pPut.nPass--; }
 return item;
}

/* Add item *itemp to queue. */
void putItem(qp, itemp)
 qInfo *qp; qItem *itemp;
{
 qp->items[qp->tail] = *itemp;
 qp->items[qp->tail].qEnter = qp->s.now();
 stVal(&qp->qSize, qp->nElem);
 qp->nElem++;
 qp->tail = (qp->tail+1) % qp->max;
 if (qp->pTake.nPass > 0)
 { qp-<>s.active(); qp->pTake.nPass--; }
}






[LISTING FOURTEEN]

File: sim-sched.cc

process body sched()
{ int nUser, nAct, i;
 long curTime = 0; /* Current simulated time */
 ordered list of pending delay requests;

 initialize pending delay list data structures;
 accept addUser() { nUser = nAct = 1; }

 while (nUser >0) {
 select {
 accept addUser() { ++nUser; ++nAct; }
 or accept dropUser() { --nUser; --nAct; }
 or accept active() { ++nAct; }
 or accept passive() { --nAct; }
 or accept now() { treturn curTime; }
 or accept reqDelay(x)
 { add request for curTime+x to pending delay list;
 nAct--; treturn request-index; }
 }
 if (nAct == 0 && pending delay list is not empty) {
 curTime = time of request at head of list;
 nAct = number of processes waiting for that time;
 for (i = 1; i <= nAct; i++)
 accept wait(x)
 suchthat (x == index-of-head-request)
 { treturn curTime; }
 discard request at head of pending delay list;
 }
 }
}

Example 1: The Server Process

while (1) {
 delay for random inter-arrival time;
 generate item;
 call queue process to put item in queue;
}

Example 2: Moving Jobs Between Queues

qItem item;
process queue qFrom, qTo;

while (qTake(qFrom, &item))
 qput(qTo, item);



Example 3: Generating Random Numbers

stats mystats;
int i;

stInit(&mystats);
for (i = 1; i <= 10000; i++)
 stVal(&mystats, erand(100));
printf("Mean is %lf\n", stMean(&mystats));


Example 4: Choosing Queues


if (outQ1.itemCnt() < outQ2.itemCnt())
 qPut(outQ1, item);
else
 qPut(outQ2, item);



Example 5: An Example of main()

main()
{
 create all processes
 while (1) {
 s.wait(s.reqDelay(10000));
 for each entity process p
 entity-stats = p.giveStats();
 if (statistics have been stable for long enough)
 break;
 }
 print final statistics
 for all processes p
 c_abort(p);
}












































Special Issue, 1989
C PROGRAMMER'S GUIDE TO C++


Making the move




Al Stevens


Al is a contributing editor and columnist for DDJ and the author of a number
of books on C. He can be reached at DDJ, 501 Galveston Drive, Redwood City, CA
94063.


C++ is an object-oriented superset of C that Bjarne Stroustrup developed at
AT&T Bell Laboratories beginning in 1980 and continuing with successive
releases. Although C++ goes back almost ten years, its recent attention in the
PC community is largely due to the growing interest in object-oriented
programming and the availability of new C++ language development environments.
In this article, I will discuss some C++ extensions that make life easier for
C programmers and how C++ brings the object-oriented programming paradigm to
C. This article is not a comprehensive treatment of the subject; I neither
describe all the features of C++ nor explain every nuance of every feature.
Such coverage would fill a book, and C++ books and articles are appearing with
increased regularity as the language gains momentum.
Dr. Stroustrup developed the C++ language system as a preprocessor that
translates C++ code into C code, which is then compiled by a traditional C
compiler. Newer C++ language systems use native code compilers that translate
directly from the C++ source language to the native language of the target
machine. I will refer to all C++ language systems as "compilers," but these
references apply as well to preprocessing translators.


C++, an Improved C


Forget, for the moment, object-oriented programming. If you never define a
class or declare an object -- things I'll discuss shortly -- you can still
benefit from the improvements that C++ offers. Many of these improvements were
important enough that the ANSI X3J11 committee incorporated them into the C
language-standard specification, and you might know them now as ANSI C
additions rather than as the legacy of C++. Among the ANSI adoptions are the
function prototype, the const type qualifier, and the void type. There are
other C++ features that you can use in your traditional function-oriented C
programming, ones not included by ANSI, but definite enhancements,
nonetheless. I'll discuss some of these advantages before exploring the
object-oriented facets of C++. So, for now, let's look at C++ as an improved
C.
Mandatory Prototypes -- One of the C++ contributions to ANSI C was the
new-style function definition and declaration blocks. ANSI calls the
definition block a "function prototype." Both blocks contain the parameters'
type descriptions in the parenthesized parameter list.
ANSI C allows for both the old and new-style function blocks to protect
existing code, and you can use a mix of both styles in an ANSI C program. With
C++, however, the old style is not supported. C++ requires you to code with
the new style, and that is a strong advantage, because the new style enforces
stronger type checking of parameter and return data types.
Comments -- C++, as a superset of C, recognizes the standard C comment style
that delimits comments with the / * and */ tokens. But C++ has another comment
format. Wherever the // token appears, everything to the end of the line is a
comment.
Because C++ recognizes both formats, many programmers use the new format for
comments and the old format to temporarily disable blocks of code. Comments do
not nest in C, so disabling code by surrounding it with the /* and */
delimiters does not work if the code to be disabled itself contains comments.
In C++, old-style comments can include the new style comment format, so the /
* ... */ format is an effective way to "comment out" code. If you use the old
format exclusively for this purpose, finding all such commented-out code is a
simple matter of using a grep utility program or your editor to find all
instances of the /* token.
Default Function Arguments -- You can declare a default value for arguments in
a C++ function prototype in the following way:
 void func(int = 5, double = 1.23);
The expressions declare default values for the arguments. The C++ compiler
substitutes the default values if you omit the arguments when you call the
function. You can call the function by using any of these ways:
func(12, 3.45); // overrides both defaults
func(3); // effectively func(3, 1.23);
func( ); // effectively func(5, 1.23);
Variable Declaration Placement -- In C, you must declare all variables at the
beginning of the block in which they have scope. You may not intermix variable
declaration and procedural expressions. C++ removes that restriction, allowing
you to declare a variable anywhere before you reference it and making
expressions such as this one possible:
 for(int ctr = 0; ctr < MAXCTR; ctr++) // ...
This feature allows you to code the declaration of a variable closer to the
code that uses it.
Stand-alone Structure Names -- In C, you refer to structures with the struct
keyword as a prefix to the name. In C++, the structure is closely related to
the class (discussed later), and its name is similar to a keyword for as long
as the definition is in scope. You can code the following:
 struct linkedlist {
 /* -- whatever -- */
 linkedlist *previous_node;
 linkedlist *next_node;};
Observe that the two pointers are declared without the struct keyword. Later
declarations of the structure may use the linkedlist word only, as shown here:
 linkedlist mylist;
The effect here is as if you had used this typedef in C:
 typedef struct linkedlist linkedlist;
The typedef would not have worked for the pointers in the structure because
the typedef declaration is not complete when the pointer declarations occur,
so the C++ notation offers a distinct improvement.
Inline Functions -- You can tell the C++ compiler that a function is "inline."
This causes a new copy of it to be compiled in line each time it is called.
The inline nature of the individual copies eliminates the function-calling
overhead of a traditional function. Obviously you should use the inline
function qualifier only when the function itself is relatively small.
Global Scope Resolution - In C, if a local variable and a global variable have
the same name, all references to that name from within the block where the
local variable is declared will refer to the local variable. The local
variable name overrides the global name. C++ adds the "::" global scope
resolution operator with which you can explicitly reference a global variable
from a scope where a local variable has the same name. Consider the code in
Example 1.
Example 1: Global variables

 int amount = 123;
 main ()
 {
 int amount = 456;
 printf("%d", ::amount);
 printf("d", amount);;
 }


In this example, the output would be 123456 because the first printf refers to
the hidden global amount variable by virtue of the "::" global scope
resolution operator.
asm ( string ); -- C++ offers, as a standard way to incorporate assembly
language into a C++ program, the "asm" keyword. Many C compilers have ways to
do this, but there is no standard. ANSI sidestepped the issue as being
implementation dependent. C++ faces it head-on and provides a standard way to
pass the string of your choice to the assembler. Assuming all C++ compilers
use the standard, only the contents of the string are implementation
dependent. If you define the strings separately, the assembly language
components of your programs are more manageable.
Of course, this is not a perfect solution. Whenever you use assembly language,
you must be ready to deal with it when you undertake a port. Just because
assembly language was appropriate in your original program does not mean you
will be able to use it in the ported version. But at least the C++ approach
moves the layer of non-portable implementation dependence to a more distant
platform.
The Free Store -- C++ introduces an improved heap management facility called
the "free store." It is implemented with two operators named "new" and
"delete." The new operator is analogous to malloc in that it allocates memory
and assigns its address to a pointer. Its argument, however, is more
descriptive than the integer value that you pass to malloc. Its purpose is to
allow you to allocate memory for a semipermanent variable, one that remains in
scope beyond the block in which is declared. Here are some examples:
 char *cp = new
 char[strlen(whatever)+1];
 linkedlist *first_node = new
 linkedlist;
Observe that the array-like expression in the first example contains a
variable dimension. The second example assumes a structure (or class) named
"linkedlist," such as the one used earlier.
When you are ready to destroy the variable, you use the delete operator in
much the same way that you used the free function in traditional C.
 delete cp;
 delete first_node;
C++ allows you to override the new and delete operators by coding your own new
and delete functions. It also provides a global function pointer that the
system calls when any new operation cannot get the memory it needs. By calling
the set_new_handler function with the address of your heap exhaustion error
function, you can issue new operations without testing the pointer values for
a NULL return every time.
References -- C++ includes a derived data type called a "reference." It is a
form of alias and will seem oddly like a pointer to the C programmer. Here's
what a simple reference looks like:
 int sam;
 int& george = sam;
The & reference-to operator tells the C++ compiler that george is an alias for
sam. Wherever you say george you could have said sam. If you only use it that
way, you might just as well use a #define, but simple substitution is not the
strength of a reference.
If you pass a variable as an argument to a function that is expecting a
reference, the compiler actually passes the variable's address. The called
function acts upon the caller's copy of the variable through its address
rather than upon a local copy. This feature shows its promise when used with
structures. An example:
 void setheight(struct cube& box)
 {
 box.height = 5;
 }
This function will change the value of the structure owned by the calling
function rather than the value of a local copy. This procedure eliminates
unnecessary copying of data values back and forth between functions while
preserving the notation of a local variable, which is simpler and more
readable than de-referenced pointer notation.
A function can return a reference, as shown in this code:
 int& getwidth( )
 {
 /*...*/
 return newwidth;
 }
The caller of the getwidth function does not need to know that the called
function returns a reference. You can code the receiving variable as a
reference itself or an actual variable. Either way works. Some C++ experts
believe that you should never return a reference. Others disagree.
Overloaded Function Names -- In C++, you can have several functions with the
same name in the same scope. The compiler distinguishes the functions based on
their argument types. This feature, called "overloading," lets you use the
same name to represent the same generic operation on different data types.
Here is an example of an overloaded function name's prototypes:
void display(int); // display an int
void display(char); // display a char
void display(long); // display a long
void display(double); // display a double

These prototypes represent four distinct functions with the same name. When
you call the name, the argument you supply tells the compiler which one you
want.
Support for overloading is one reason why C++ needs prototypes for everything.
The compiler must see the argument types so it can distinguish functions that
have the same name.
Structures with Functions - C++ allows you to code functions as members of
structures. This practice binds a function call to an instance of a structure.
Consider the code in Example 2.
Example 2: Structures with functions

 struct cube {
 int length;
 int width;
 int height;
 int volume();
 };

You have defined a structure that has three integers and a function. You must
now complete the structure's definition by declaring the function:
 int cube::volume(void)
 {
 return length * width * height;
 }
Observe that you declare the function as a member of the cube structure by
attaching the cube:: prefix to the function name. Notice also that the
function does not need to name the instance of the structure when it refers to
one of the structure's members. The compiler knows that the function will be
called on behalf of an instance of the structure in which the function is a
member, and the compiler automatically makes the association for you.
Next you declare a cube object and get some values into the cube's dimensions:
 struct cube box = {5,4,3};
That looks like and is traditional C. You have declared a structure of type
cube and named it box. You have initialized the dimensions of the structure.
Finally, you call the cube's volume function to compute the volume of the
cube.

 printf("Volume: %d", box.volume( ));
Observe the form of the volume function call. The function name is prefixed
with the identifier of the declared structure in standard C structure member
addressing format.
This little exercise is a sneak preview of the C++ class, a more complex
version of the enhanced structure used here. It is also your introduction to
object-oriented programming in that the structure just shown is an abstract
data type including a method, and the structure declaration named box is an
object.
So far we've concentrated on the additive features of C++ that can enhance C
language programming. But, in doing so, we brushed against some new
object-oriented programming ideas. Now let's consider what those and other new
things mean when viewed from the perch of the object-oriented programming
platform.


Object-Oriented C


C++, we have said, adds the object-oriented programming paradigm to the C
language. C++ has "data abstraction," "encapsulation," "objects," "methods,"
"inheritance," and "polymorphism." These ingredients, we are told, are what
embody an object-oriented programming platform. Every paradigm wants its own
parlance, and you will find that these strange new terms have unexpected
parallels in your C experience, parallels that, once revealed, make the new
things easier to understand. Before getting into the object-oriented details
of C++, though, let's try to liken some of these new object-oriented concepts
to their counterparts in traditional function-oriented C.
Data Abstraction, Encapsulation, and Objects -- "Data abstraction" is the
ability to describe new data types in terms of their format and the processes
that act upon them. "Encapsulation" is the process by which you combine the
component data and function parts of an abstract data type into one
encapsulated definition. An abstract data type, called a "class" in C++, is
thus a collection of other data items and functions. The data items describe
the new class's format, and the functions describe how it behaves. An instance
of an abstract data type is called an "object."
C has its own built-in data types. In C, you use char, int, float, and double
and qualified variations. When you declare one of these, you are declaring an
instance of the type or an object. The developers of your C compiler
encapsulated these data types into the compiler by describing their formats
and by including the methods that act upon them. When you say:
 int answer = 10 + variable_integer;
you have declared the int object named answer and invoked the C compiler's int
method that sums the values from two integer data type objects and places the
result into the answer object. The int is not an abstract data type because it
is built into the compiler, but it is, nonetheless, an object.
In object-oriented programming, you add to the language's vocabulary of data
types with abstract data types encapsulated by you and, perhaps, by developers
of third-party data type libraries. So, in addition to the int and the float,
you can have the string, the blivot, the Bach_two_part_invention, and whatever
else you dream up as a new data type.
C Parallel to Abstract Data Types -- The closest things to abstract data types
in traditional C are the typedef and the struct, although they lack some of
the other qualities of object-oriented data types. If you define a structure
in C and assign a typedef name to it, you have encapsulated an abstract data
type. You complete the encapsulation by writing C functions that act upon
structures of the named typedef. The stdio.h definition of the FILE typedef
is, when combined with the fopen, fclose, fgets, and so on functions, a loose
encapsulation of the abstract FILE data type that manages stream input/output.
The encapsulation is not secure, however, and that is where object-oriented
programming comes in.
Methods -- We call the functions defined for an abstract data type its
"methods." In object-oriented programming, you make things happen by sending
messages to objects. In the encapsulation, someone defined the methods that
act upon the messages you send to the object.
C Parallel to Methods -- There is very little difference between the
object-oriented notion of sending a message to an object and the traditional C
notion of calling functions. The important single difference is that in
object-oriented programming, the message and the method are tightly bound by
encapsulation to the declared object, the instance of the abstract data type.
The encapsulation defines the binding, which occurs when you declare the
object.
Inheritance -- Having defined an abstract data type, you can define others
that inherit its attributes. This feature is called "inheritance." One
advantage is that all the work that went into the definition of the base type
passes down to the derived type. Another is that it encourages you to think of
and design your data system in a structured way.
C Parallels to Inheritance -- In C, a structure that has another structure as
an element effectively inherits the properties of the element structure and
adds its own unique elements to it.
The C array is a derived type. You describe a structure and then describe an
array of those structures. The structure is the base type. The array is the
derived type, inheriting the characteristics of the structure. Derived classes
in C++ are, however, more powerful than the simple addition of dimension to a
base type.
Other derived types in C are pointers, which are derived directly from the
base types to which they point; constants, which are derived from the types
assigned to them by the compiler; and structures, which are early examples of
derived types with multiple inheritance in that they are derived from the
various types of their multiple elements.
Function Overriding and Polymorphism -- Function overriding is the ability for
a type or method in a derived type to override a similarly defined type or
method in its base type. The different types up and down a type hierarchy can
define member types and methods according to their individual needs. The
programmer who is using an abstract data type does not need to know which
method will process the message being sent or in which abstract type the
referenced member type appears.
If you have a base data type and a derived data type, you send messages to an
object of the derived data type to get it to do what you want. Because the
derived type has inherited the attributes of the base, you can send messages
that are defined as belonging to the base data type, and the derived type will
accept them and use the methods of the base to act upon them.
With function overriding, you can define a member type or method in a derived
data type that resembles one in its base. Send a message via that method or
refer to that member type, and the one defined in the derived type gets used.
Not all of the base's derived types will duplicate the original. If you send a
message to an object way down the hierarchical ladder, the compiler will
select the method from the first type higher in the hierarchy where the method
is defined.
If, however, you address the derived type through the base type, perhaps by a
base-type pointer that contains the address of the derived type, then the base
function gets used, and overriding does not occur. When you do not want this
to happen, when you want the derived overriding function to be used regardless
of how you address the type, then you must invoke "polymorphism."
Polymorphism is a nuance of function overriding. If you declare the base
function as a "virtual" function, then it will always be overridden by
functions with the same name and characteristics in the derived types.
C Parallel to Function Overriding -- Traditional C uses a form of function
overriding in its arithmetic operations. If you view the integers, floating
point numbers, and pointers as a hierarchy of arithmetic types, the various
ways that you add to them, for example, can be thought of as similar methods
in a data model, methods that are invoked on the basis of the data types
rather than the operators.
There seems to be no parallel in C to the object-oriented concept of
polymorphism.
These, then, are the fundamentals of object-oriented programming, and we have
considered how you might associate your function-oriented programming
practices with them. Some of our analogies stretch the point to its limit, but
their purpose is to assure you that the paradigm called "object-oriented
programming" is largely a different way of looking at what you have been doing
all along.
Now let's look at some of the details of C++ and see how those principles are
applied.


C++ Classes


The basic unit of encapsulation in C++ is the class. What we called the
abstract data type before, we will now call the "class" because they are the
same thing. C++ uses the class keyword to describe its version of the
programmer-defined data type. When you encapsulate an abstract data type, you
define a class. When you declare an object, you declare an instance of a
class. Example 3 illustrates what a definition of a class looks like.
Example 3: A class definition

 class date {
 int day;
 int month;
 int year;
 public:
 date (void)
 {day = month = year = 0;}
 date (int da, int mo, int yr);
 date() { /* null destructor */ }
 void display (void);
 };

This definition describes a class named date. It resembles a structure in that
it has members consisting of data variables and functions. The members prior
to the public keyword are private parts. The rest are public. The difference
between the class and the struct shown earlier is that all the members of the
struct are visible to all parts of your program while only the public members
of a class can be seen by your program's functions. The private parts of a
class can be read and changed only by functions that are themselves members of
the class. (For an exception, see the discussion on friends.)
Usually, the private parts are variables and the public parts are functions.
Nothing says that this must be so, but it seems to be a good convention t
follow and works for most class definitions.
In addition to the public keyword, you can declare that certain members are
"private" and others are "protected." Members in a class are private by
default as are our day, month, and year members. Members in a structure are
public by default unless you declare them as private or protected.
The access control of a member indicates what kinds of functions can access
the members. A private member can be accessed by member functions and friends
only. Public members can be accessed by any function that declares an object
of the class or has the structure in scope. Protected members of classes are
private except that member functions and friends of derived classes can access
them. We will discuss member functions, friends, and derived classes soon.
Our date class's encapsulation is in complete because all the methods are not
there yet. Remember, methods are functions.
Classes usually have at least two member functions and often more. The usual
two are the "constructor" member function and the "destructor" member
function, although it is not required that you define them. The others are the
methods.
Constructors and Destructors -- When you declare an object as an instance of a
class, you do it in much the same way that you declare any other data type.
Compare this integer and this date:
 int days_on_board; date date_hired;

They look the same, and to the programmer using them, they are. But when you
design the class, you usually provide for one constructor function and one
destructor function. The constructor function executes when you declare the
object, and the destructor function executes when the object goes out of
scope. If you omit these functions, then no special processing occurs when an
object enters and leaves scope.
A constructor function has the same name as the class itself and has no return
value. In the date example above, we have defined two constructor functions,
both named date. They are distinguished by their different argument types; the
first constructor has no arguments, and the second constructor has three
integer arguments. These are overloaded constructor functions because they
have the same name and different parameters.
Observe next that the first constructor definition is not terminated with a
semicolon but has a brace-surrounded block of code following it. This format
is the member function's version of an inline function. By including the code
with the definition in this manner, you build an inline function, in this case
one that merely initializes the three date variable members to zero. This
constructor function executes when you declare a date object with no
initializing values as shown here:
 date date_hired;
The second constructor function is not an inline function (although it could
be), so you must provide its code somewhere. The function might look like that
in Example 4.
Example 4: Constructor function

 date:: date (int da, int mo, int yr)
 {
 day = da;
 month = mo;
 year = yr;
 }

Observe that the function declaration is prefixed with date:: to associate it
with the date class. Observe also that the function has free access to the
private members of the class. This particular constructor executes when you
declare a date object with three integer initializer values as shown here:
 date date_retired(25, 5, 88);
The destructor function has the same name as the class but with a tilde
prefix. In the date example class used here, the destructor function does
nothing, so it is coded as a null inline function. More complex classes will
require things to be done when the object goes out of scope. Perhaps some free
store memory needs to be deleted, for example, and the destructor function
would take care of that.
Member Functions -- Besides the constructor and destructor, a class can have
other member functions. These are the methods of the class. In our date
example, we show a member function named "display." To use this method, we
must code it something like this:
void date::display(void) { printf("%d/%d/%d", month, day, year); }
(Experienced C++ programmers might wonder why I use the old-fashioned printf
when the C++ stream facility is available. Because I haven't described the
stream classes yet, their use would tend to confuse those who are not familiar
with them. I'll discuss streams later.)
The program that declares a date object can then use a member function related
to the date class such as this:
 date_hired.display( );
In the parlance of object-oriented programming, we have sent a message to the
date_hired object to tell it to use its display method. It looks a lot like a
traditional function call, doesn't it?
Friends -- The private members of a class are visible only to the class's
member functions. The constructor, destructor, and display member functions in
the date class can read and write the month, day, and year integers, but no
outside function has that access. From time to time you will find a need to
provide outside access to the innards of one of your classes. A named function
or class can be a "friend" of the class being defined. You can make the
assignment of friend status only from within the definition of the class that
grants access as shown in Example 5.
Example 5: Making an assignment of friend status

 class time;
 class date {
 // ...
 friend void now (date&, time&);
 };
 class time {
 // ...
 friend void shownow (date&, time&);
 };

These are two classes named date and time (with the details omitted). We need
the extra time declaration at the top because the friend statement in the date
class has a reference to it. The two classes share a friend function named
shownow, which might display the current date and time like this:
void shownow (date& d, time& t)
{
printf("\n%d/%d/%d", d.day, d.month, d.year);
printf("\n%d:%d:%d", t.hr, t.min, t.sec);
}
Because the shownow function is a friend to both classes, it can read the
private members of both.
A class can have another class as its friend with this statement:
 class date {// ... friend class time;};


Operator Overloading


Operator overloading is one of the neater tricks you can do with C++. It is
what makes the language so extensible. You've already seen how you can add
data types by defining classes. Next, you might want to perform arithmetic,
relational, and other operations on your classes the same way that you do with
int and float variables and the like. We built a simple date class. Consider
Example 6.
Example 6: A simple date class

 date retirement_date, today;
 // ...
 if (retirement_date < today)
 // Keep working ...

 How about this?


 date date_married, today;
 // ...
 if (date_married + 365 == today)
 // Happy Anniversary ...

Both these forms are possible with C++. You can build class member functions
that the compiler associates with C operators. This feature is called
"operator overloading." When you use an operational expression such as the
ones just shown, your operator overloading member functions get called. Here's
an example:
 class date {
 // ...
 int operator<(date&);
 }
The date class definition says that a member function overloads the less-than
operator. When you code this expression:
 (retirement_date < today)
the function that associates the less-than operator to a date class with
another date class as the argument is called. It returns a true or false
integer. The member function might look like that in Example 7.
Example 7: A member function

 int date::operator<(date& dt)
 {
 if (year < dt .year)
 return TRUE;
 if (year == dt .year) {
 if (month < dt .month)
 return TRUE;
 if (month == dt.month)
 if (day < dt .day)
 return TRUE;
 }
 return FALSE;
 }

The function refers to the class on the left side of the expression by naming
its private parts without qualification (day, month, year). It refers to the
argument's private parts by way of the reference variable (dt.day, dt.month,
dt.year).
You can overload any C operator in this manner. You cannot create your own
operators such as")("or anything the compiler would choke on, and you may not
use overloaded operators in ways that the compiler cannot parse, such as A [B.
You could, however, use this feature to create some really confusing code. You
might, for example, use the + operator to logically subtract two classes. Try
to steer clear of such nonsense.
Operator overloading is a powerful feature. You can overload the [] array
operator to create your own array processing. You can overload the ( )
function call operator to make a class look like a function call. And, of
course, you can have several different functions overloading the same operator
with different argument types. You might want separate functions for adding
floats and longs to your class, for example.
The this Pointer -- Every member function has a built-in pointer named this.
It points to the object being processed by the member function. When a member
function wants to make a copy of the object or return it after perhaps
modifying it, the member function can reference the object as *this.
To illustrate the use of the this pointer, let's take another look at the
overloaded addition operator. Our less-than overloaded operator returned an
integer, but an arithmetic expression needs to return a copy of the class.
Consider the example we saw earlier:
 if (date_married + 365 = = today) //
 Happy Anniversary ...
There are two overloaded operations implied by this expression. One is the
equality = = operator, which would look a lot like the less-than operator we
already built. The other is the addition operator. Here they are in the class
definition.
 class date {
 // ...
 date& operator+(int);
 int operator= =(date &);
 }
The overloaded + operator must return a class that has the result of the
addition as its value. It does not add the integer to the class called out in
the expression. It performs the addition and returns the result, which is
itself a date class. Without the complex date arithmetic that checks for month
overflow, leap years, and all that, here is what the overloaded + operator
function looks like.
 date& date::operator+(int n)
 {
 date dt;
 dt = * this;
 //add n to dt.month,dt.day,dt.year
 // ...
 return dt;
 }
You will see that we copy the object being added to into a temporary date
named dt. The this pointer provides a way for us to do this.
If you wanted to overload the + = operator, the function would look like this:
 date& date::operator+=(int n)
 {
 //add n to month,day,year
 // ...
 return *this;

 }


Class Assignment


Observe in the overloaded + operator above that the object pointed to by this
is assigned to the object named dt. C++ includes a built-in assignment
operator for every class you define. It simply copies all the members from one
object to the other much the way that structure assignment works in C. You can
overload the assignment operator if you need to. You would do that if you
wanted to assign something other than another object of the same class.
Consider this, for example:
 #include <time.h>
 // ...
 date_hired = time(NULL);


stream.h


Throughout the examples so far, we have used printf to display data. C++
includes stream input/output classes already defined in stream.h. They have
several advantages over the traditional printf/scanf pair. To write something
to the console you say:
 cout << "Hello, Dolly";
The stream.h file not only includes the ostream and istream classes, it
includes external declarations of the standard objects, cout and cin, which
are assigned to the console. The example just given shows how the ostream
class has overloaded the << operator. This is not a bitwise shift operation in
this usage. The notation is meant to represent the direction of data flow. The
<< operator is overloaded several times to allow you to send different data
types to the console without worrying about their format. Here are examples:
 cout << "Blossom";
 cout <<' \ n';
 cout << 123;
The overloaded operator functions return references to the object being
processed, so you can write the above example in this way:
 cout << "Blossom" <<' \ n'<< 123;
The istream class works the other way with >> as the operator to signify data
flowing from the object to the argument variable as shown here:
 void main( )
 {
 char mystring[80];
 // ...
 cin >> mystring;
 }
You can declare streams and associate them with file buffers as well. There
are several standard methods that support file stream input/output included in
stream.h.
Conversion Functions -- Conversion functions allow you to provide for the
conversion of a class to another type, either a standard C data type or
another class as shown here:
class date {
 // ..
 operator long( );
 };
You would write the overloaded function such as the one shown in Example 8 and
can call the function one of several ways as shown in Example 9.
Example 8: An overloaded function

 long date::operator long () (void)
 {
 long days_since_creation;
 // compute the number of days...
 // ...
 return days_since_creation;
 }

Example 9: One way of calling the function listed in Example 8

 void main ()
 {
 date today;
 long eversince;
 // ...
 eversince = (long) today;
 eversince = today;
 eversince = long (today);
 }


The notation you choose would depend on how you are using the conversion. The
first format looks like a cast, and the second implies that a normal type
conversion is going on. You might use the last format if you want the code to
remind you that you are invoking a class conversion function.
This example shows how you convert a class to a standard C or C++ data type.
You can use the same technique to build class-to-class conversion functions,
but some programmers prefer to use specific constructor functions to perform
conversions of classes to other classes. When the target class comes into
scope, the constructor conversion function gets the data values from the
object that is its parameter. For example:
 date today(25, 12, 89);
 // ...
 Julian julian_today(today); // a class named Julian


Inheritance


Inheritance is a vital ingredient for object-oriented programming. Many C
programmers will use the advanced features of C++ classes to add data types
and never concern themselves with class hierarchies. Others will explore the
murky depths of inheritance and will build huge, exotic class systems, making
every new class a new wrinkle on an old. Somewhere in between is the probable
ideal.
A class in C++ can be defined as a derivative of another class. Given the
generic date class we've used so far, we might need a more specific date, one
that has all the properties of other dates but that has a few unique ones of
its own. Here is how you would code a derived class:
 class workday: date
 {
 int shift;
 public:
 workday(int da, int mo,int yr, int sh);
 // ...
 };
We have defined a class named workday that is derived from the class named
date. The workday class is called the "derived" class, and the date class is
called the "base" class. The derived workday class has its own private member,
the shift variable. What you do not see is that the class also has variables
named day, month, and year because it is derived from the date class which has
those variables, and that is how inheritance works. The workday constructor
function has a shift variable, one more variable than the corresponding date
constructor. Here is the constructor function:
 workday::workday(int da,int mo,int yr,int sh)
 :date( da, mo, yr)
 {
 shift = sh;
 }
The expression following the colon after the argument list shows what the
workday constructor function will pass to the date constructor function. The
values do not have to be taken from the argument list as we have done here,
they can be any valid expressions that match the argument types of the base
class's constructor function.
Member functions in a derived class can read and write the public members of
the base class if the derived class has not reused the member name as shown in
Example 10.
Example 10: Member functions

 class workday : date {
 public:
 // ...
 int isXmas (void);
 };
 int workday;;isXmas (void)
 {
 return month == 12 && day == 25;
 }

A derived class can reuse a member name that is used in a base. This is
function overriding. If the derived class has reused the base member's name,
any unqualified references to that name will point to the member in the
derived class. But if the member wants to access the base class's member, the
derived member function qualifies the name such as this:
 x = date:: month;
Non-member functions that declare objects of the derived class cannot access
the public members of the base class unless the public keyword qualifies the
inheritance as shown in Example 11.
Example 11: Non-member functions

 class workday : public date {
 // ...
 };
 void main ()
 {
 workday proj_dt;
 // ...
 int yr_comp = proj_dt.year;
 }

A derived class cannot access the private parts of its base unless it is
declared by the base as a friend. If you are adding a new class to a class
hierarchy and find that a derived class needs to get at a private member of a
base class somewhere up the line, you will need to rummage around in the class
definitions to find and modify the base class. You must either declare the
derived class (or one of its functions) as a friend of the base or move the
desired element into the public part of the base class.


Multiple Inheritance



The object-oriented world traditionally describes its type in what they call a
"class hierarchy." This is often a misnomer. In object-oriented design, a base
type can have many derived types. If you were to stop there, you would have a
hierarchy; in a true hierarchy each lower element is subordinate to only one
superior. Versions of C++ prior to 2.0, which is the latest, adhered to the
hierarchical model in that a derived C++ class could have only one base class.
In Version 2.0, as in many other object-oriented languages, a derived class
can have multiple base classes. This model is called "multiple inheritance,"
and it is not a hierarchy -- it is a network. Programmers love to mix,
confuse, and abuse metaphors. No doubt the "type hierarchy" metaphor is
destined to remain with us even though it misuses the base from which it
derives.
A C++ class can inherit the attributes of multiple base classes as shown in
Example 12.
Example 12: Inheriting attributes of multiple base classes

 class date {
 // ...
 };
 class time {
 // ...
 };
 class datetime : date, time {
 // ...
 public:
 determine (int da,int mo,int yr,
 int hr,int min, int sec);
 };

We have defined a derived class named datetime that inherits the attributes of
two base classes named date and time. The constructor for the datetime class
would look like this:
datetime:: datetime(int da, int mo, int yr, int hr, int min, int sec)
 : date (da, mo, yr), time (hr, min, sec)
 {
 // ...
 }
There are many things to consider when you build a network of
multiple-inheritance classes. You must be aware of the order in which base
constructors are called and you must guard against ambiguities when you refer
to members of the derived and base classes. The taller the family tree, the
harder it is to keep track of what is involved with distant, unseen relatives,
ancestors, and friends.


Virtual Functions


C++ uses the virtual function qualifier to declare a base-member function as
one that is always overridden by a derived class regardless of how the derived
class is addressed. If you declare a function in the derived class with the
same name and argument types, then any reference to that function -- even
through a reference to the base -- will invoke the derived class's copy of the
virtual function. Example 13 illustrates some of these concepts.
Example 13: Examples of C++ virtual functions

 class date {
 // ...
 public:
 virtual int isXmas (void);
 };
 class workday : date {
 public:
 int isXmas (void);
 };
 void main ()
 {
 date dt;
 workday wd;
 date = dat = &dt;
 // ...
 wd.isXmas (); // workday::isXmas
 dt.isXmas(); // date::isXmas
 dat->isXmas(); // workday::isXmas (!)
 wd.date::isXmas (); // date::isXmas
 }

This ability to redefine methods up and down the class network is what gives
your C++ class definitions their polymorphic characteristics.


Future Directions



Some of the features I discussed in this article are new to C++ 2.0. But the
language has not finished growing. For example, Dr. Stroustrup is working to
add parameterized data types and exception handling as intrinsic parts of the
language. And ANSI is about to field a committee whose task it will be to
write a standard description of C++.
With Microsoft and Borland moving in the object-oriented language direction
and with rumors that both will introduce C++ compilers in the '90s, the future
of C++ seems secure. Given the extensive improvements that C++ makes to the C
language, we can safely predict that C++ will eventually replace C as the
language of choice.


_C PROGRAMMER'S GUIDE TO C++_
by Al Stevens

Example 1: Global variables

 int amount = 123;
 main()
 {
 int amount = 456;
 printf("%d", ::amount);
 printf("%d", amount);;
 }


Example 2: Structures with functions

 struct cube {
 int length;
 int width;
 int height;
 int volume();
 };


Example 3: A class definition

 class date {
 int day;
 int month;
 int year;
 public:
 date(void)
 {day = month = year = 0;}
 date(int da, int mo, int yr);
 ~date() { /* null destructor */ }
 void display(void);
 };


Example 4: Constructor function

 date::date(int da, int mo, int yr)
 {
 day = da;
 month = mo;
 year = yr;
 }


Example 5: Making an assignment of friend status

 class time;
 class date {
 // ...

 friend void now(date&, time&);
 };
 class time {
 // ...
 friend void shownow(date&, time&);
 };



Example 6: A simple date class


 date retirement_date, today;
 // ...
 if (retirement_date < today)
 // Keep working ...

How about this?

 date date_married, today;
 // ...
 if (date_married + 365 == today)
 // Happy Anniversary ...


Example 7: A member function

 int date::operator<(date& dt)
 {
 if (year < dt.year)
 return TRUE;
 if (year == dt.year) {
 if (month < dt.month)
 return TRUE;
 if (month == dt.month)
 if (day < dt.day)
 return TRUE;
 }
 return FALSE;
 }


Example 8: An overloaded function

 long date::operator long()(void)
 {
 long days_since_creation;
 // compute the number of days ...
 // ...
 return days_since_creation;
 }


Example 9: One way of calling the function listed in Example 8

 void main()
 {
 date today;
 long eversince;

 // ...
 eversince = (long)today;
 eversince = today;
 eversince = long(today);
 }


Example 10: Member functions

 class workday : date {
 public:
 // ...
 int isXmas(void);
 };

 int workday::isXmas(void)
 {
 return month == 12 && day == 25;
 }


Example 11: Non-member functions

 class workday : public date {
 // ...
 };

 void main()
 {
 workday proj_dt;
 // ...
 int yr_comp = proj_dt.year;
 }


Example 12: Inheriting attributes of multiple base classes

 class date {
 // ...
 };

 class time {
 // ...
 };

 class datetime : date, time {
 // ...
 public:
 datetime(int da,int mo,int yr,
 int hr,int min, int sec);
 };


Example 13: Examples of C++ virtual functions

 class date {
 // ...
 public:
 virtual int isXmas(void);

 };

 class workday : date {
 public:
 int isXmas(void);
 };

 void main()
 {
 date dt;
 workday wd;
 date *dat = &dt;
 // ...
 wd.isXmas(); // workday::isXmas
 dt.isXmas(); // date::isXmas
 dat->isXmas(); // workday::isXmas (!)
 wd.date::isXmas(); // date::isXmas
 }












































Special Issue, 1989
AUTOMATIC MODULE CONTROL REVISITED


Adding power to an already powerful documenting utility


 This article contains the following listings: POOLE_CP.ARC


Ron Winter


Ron is a faithful Dr. Dobb's subscriber, and has been a microcomputer
aficionado for many years. He is a software engineer at SPEX in Edison, N.J.
You can contact him at P.O. Box 4143, Metuchen, N.J. 08840.


This article is a follow up to Stewart Nutter's article "An Aid To Documenting
C," DDJ, August 1988. In that article, Stewart presented a printer program he
called cp (for C printer), which allowed programmers to document C source-code
modules. When I first saw the program, I realized that it addressed my needs
in a general way, but as presented the program was just not sufficient.
After entering the source as printed (and adding a few missing ++s and fixing
a few other minor typos) I got cp to run. Two things struck me immediately:
Lowercase ls as variables are amazingly similar to 1s in small type, and the
program crashed when I gave my target program as input! By this time my
curiosity was sufficiently aroused, so I decided I had better understand the
program before changing it too much, or it might never work.
What appeared to be a simple task became instead a small research project into
the C language. It is amazing how much C code you can write without really
understanding why C is the way it is. The process of correcting the code so
that it would follow the rules of a C compiler and linker helped me understand
the role of functions, and brought me a deeper understanding of computer
languages and an appreciation for what they do. Nevertheless, you usually only
go far enough to get the job done. In the case of cp, I realized that if I
went much further, the program would start to become a compiler! That's a
thought I try not to consider too seriously.


Rewriting Without Rewriting


When I began rewriting Stewart's original program, all of my prejudices and
preferences in style came to bear upon the code in a fit of global search and
replacements. My own C style is "pascalean" in indenting and braces, with
highly descriptive functions and variable names, especially globals.
Every function in the original program is in my own version of cp, but the
names have been changed to make them a little clearer (at least to me). Some
new functions have been added. Function declarations are included, and two new
files were added: cpbuild.c and cpinput.c. cpbuild.c contains all the code
needed by the original function xref( ). The second file, cpinput c, handles
the prompting and parameter parsing that the program does at start-up.
My cp program consists of the following files: CPHEADER.H, the header file
(Listing One, page 73); CP.C, the main source file (Listing Two, page 73);
CPINPUT.C, the command-line parser (Listing Three , page 78); CP BUILD.C, the
front-end section of the program (Listing Four, page 79); CPFUNCTS.C, the
back-end section (Listing Five, page 83); CP, the make file (Listing Six, page
85); and CPLIST, the input file to execute the program on itself (Listing
Seven, page 85).
The make file now uses the /packcode directive in the linker command. This
allows me to make all the functions "near," even in different files in large
model programs, as long as the code size is less than a segment. The effect is
to have the speed and size of a small model program with respect to the code.
The arguments to the program are much the same as they were in the original
program. Some new parameters are: n for normal (as opposed to IBM) character
graphics, f for size of called function array, l for library call statistics,
q for quiet mode, d to show declarations and definitions on the console as
they are found, h to show more help than is shown when cp is executed with no
arguments, and x to show some technical information. The program now can take
uppercase or lowercase parameters with either the "-" or "/" switch character.
Figure 1 shows a sample printout of the the cp program; Figure 2 shows a
portion of a typical report the program produces.


Repairing the Algorithms


A minor deficiency of the original code is that it incorrectly assumes that a
function definition ends with a new line. A more correct algorithm scans
forward from possible identifiers; if an open parenthesis is found, it then
scans for the matching close parenthesis and checks the next non-white
character. As in "Find That Function!" by Marvin Hymowech (also in August
1988), the key is to note that if this next character is a comma or semicolon,
a declaration or prototype has been found. In either ANSI style or standard C,
definitions have either an open brace or variable declaration(s) following the
close parenthesis.
I had to overcome two major deficiencies in the original code. The first was
the lack of treatment of static functions. The second was the way leading
comments were stored. In the original code, the strdup( ) was not checked for
failure at the end of the function xref( ), so it crashed on my large program.
I made a minor change to the program that dressed up the form of the
tree-structured output. The connecting lines did not stop when there was
nothing underneath to connect them to. Finally, I added a toggle for IBM
character graphics to draw the tree in fine style.
I discovered that you must really understand what constitutes a correct C
program first before you can parse it for function definitions and function
calls. The scanner, called getnext( ) in the original, incorrectly scanned
quoted (") strings and pound sign (#) statements. They both allow the line
continuation form of backslash followed immediately by a new line. Also,
comments are considered white space in a #statement construction. The function
is now called get_to_next_possible_token( ), and it handles these situations
correctly.
Figure 2: Sample report from the cp program

 Function Index:
 function in file references
 ______________________________________________________________________

 static allocate_arrays cp.c 1
 binary_search_sorted_data_base cpfuncts.c 4
 build_box_parts cpfuncts.c 1
 static build_records_from_list cp.c 1
 build_the_data_base cpbuild.c 1
 static bump_line_count cp.c 29
 check_for_new_page cpfuncts.c 4
 static count_all_defined_references cp.c 1
 static deallocate_arrays cp.c 1
 static do_top_of_page cp.c 7
 doprint cpfuncts.c 3
 static draw_output_block cpfuncts.c 4
 static get_chars cpbuild.c 6
 static get_to_next_possible_token cpbuild.c 4
 static initialize_globals cp.c 1

 static is_legal_identifier_character cpbuild.c 1
 main cp.c 1
 static mark_as_static cpbuild.c 1
 nasty cpinput.c 1
 process_arguments cpinput.c 1
 static recursion_check cpfuncts.c 1
 scan_for_static_or_global cpfuncts.c 2
 static setpage cpfuncts.c 3
 static show_files_leading_comments cp.c 1
 static show_function_relationships cp.c 1
 static show_library_functions cp.c 1
 static show_line_and_byte_counts cp.c 1
 static show_page_references cp.c 1
 static show_sorted_function_list cp.c 1
 static show_unused_if_any cp.c 1
 static sort_the_data_base_array cp.c 1
 tab_to_left_margin cpfuncts.c 9
 static test_and_add cpbuild.c 1
 static unget_chars cpbuild.c 8
 Un-used function list: -cpfuncts.c
 static stop


I corrected the treatment of static functions. First, you have to recognize
them, and then mark them when they are entered into the data base. It is also
necessary to mark them in the called-function list with their file name, if
they are called at any time in the file under analysis. In C, you can have
many functions with the same name in a program, as long as only one of them is
not static and each is in a different file. This requires a change to the way
the defined function call count is done and the way the output tree is
checked. Basically, the binary search of the sorted data base must be called
with the understanding that there might be more than one defined function with
the same name. If the called function is in a different file from the defined
function and the defined function is static, it should not match. You must
search adjacent names in the ASCII sorted list for a called function that is
(first) static in the defined functions file, (second) not static and in any
file, else it must be a library function. This also reflects upon the
recursion check. All of these issues are addressed in the new version of the
program. A function is recursive if it calls a function with the same name
(perhaps through other functions) and the test checks that the file name
associated with the called function is the same as the file name of the
calling function. This avoids the trap of function x() say in file 1, calling
function y() in file 2, and y() calling static function x( ) in file 2. The
original code would say incorrectly that function x() was recursive.
More Details.
*


A Few Extras


The point of making the called function array programmable is to allow you to
shrink its storage requirements and thus allow the program to run in less
memory. This allows the program to be launched with impunity from within
editors, TSRs, Windows, DESQ-view, and so on.
As stated in the code comments, this program will not see the relationship
between functions when they are called indirectly via pointers to functions.
Also, functions in the body of a #define will be missed. Code in both #if and
#else will also be scanned, possibly noting more function calls and perhaps
even getting out of sync with respect to opening and closing braces. This may
be annoying but as long as you are aware of it, I don't think it's too
serious. If needed, you could pass the source through the preprocessor first
before processing it with cp.
The code now catches all of the initial comments in a file. I am not sure what
it did in the original code. The temporary buffer for it is 3K. Compile it
bigger if you wish. The look-ahead buffer that scans between matching
parentheses in a function declaration or definition is the manifest constant
c_line_length, which is set to 512. If you are even more verbose in your style
than I am, you may want to make this larger (a co-worker of mine is, so his is
2048!). The called function array size is the number of unique functions per
function definition for all function definitions in the program, so the number
of function calls can exceed this number. These arrays are mallocd so that if
they need to exceed a full segment, one only has to change the malloc()s to
halloc()s, the associated free()s to hfree()s, and then the really tedious
part, chasing all associated global and local variables (usually pointers to
these arrays) and changing all their definitions to huge, such as some_type
*some_type_pointer to some_type huge*some_type_pointer. A large model
recompile is all that is required. This is all Microsoft C 5.x specific talk,
though I assume similar constructs exist in other Cs on the IBM PC/XT/AT
platform. I would expect these huge arrays are not needed until one must
analyze a really large C program, perhaps something on the order of 1-2-3
release 3!


The Loose Ends


The parser still worries me. It catches all the stuff I have thrown at it, but
because I am still not sure how it works (!), I feel that it may still have
some black holes. I can't follow its execution by looking at it; I just go on
faith. The next version will probably return a space for any white space
character. That is, put the white space testing into
get_to_next_possible-token(). This should clean up build_the_data_base() a
little.
I would like to contemplate the data structures a little and perhaps come up
with something a little less ugly than what exists now, especially the
structure elements that mark statics. There must be a neater way of doing
this. You may want to add yet another input toggle to plot unused functions.
This is useful in a C program that uses pointers to functions. You may not be
able to see who calls the function, but you can plot it anyway.
For those of you with really big programs, you could go to halloc() for the
arrays. After qualifying the declarations of pointers with the huge attribute,
a large model recompile is all that is required to allow arrays larger than
the 64K limit imposed by malloc().
Two more items complete the wish list. The first is hooking in the page linker
to the library calls in doprint() so that a cross reference to library calls
may be generated. The last is to sort the defined functions on entry rather
than at the end. This would free up run-time dynamic memory and would not
create a significant time penalty; in fact, it might actually speed things up!


Late Additions


At the request of a co-worker (yes, the same one!) the input buffer for the
input file name list was extended from 20 bytes to 128 bytes. Apparently his
sources were really scattered about his directory tree, and so some of his
pathnames were quite long, hence the change to the buffer size. A small change
was also made to allow an optional string following each input file pathname.
This string is atoi()d, and if it is not 0, it is added to the tree structure
defining box and added as a column in the sorted function list. Its intended
usage is the overlay number of the function. Overlays are the trick that
allows you to write code and .exe files that may wildly exceed 640K or
whatever your memory space is. This is yet another argument to the program,
its default is off. The reason for this is to study the program flow of the
tree diagram to check for one overlay, calling another in order to prevent
thrashing in a loop, speeding up execution by combining them into one overlay
and so on.
I imagine other uses could be put into this extra information, the
interpretation is up to the user. The same (!) co-worker also asked that the
unused function list be sorted by filename. Yes, he had a lot of them! Because
the sorting routine was already in place for the used functions, it was simple
enough to clone it into the unused list display function.


Last Words


I trust that more C wizards will pop up and carry this ball a little further.
I only carried it as far as I needed for my purposes. The C program that cp is
now maintaining is over 4.2 million bytes, over 117,000 lines, 198 files, uses
993 defined functions called 4600 times, with 192 library functions called
3118 times! As you can imagine, this program does a wonderful job at wearing
out printers! Thanks to Stewart for the great idea and for doing all the
initial dirty work.


C Printer for VMS and Unix



Kevin E. Poole
Kevin is a software design engineer on the Boeing automated software
engineering (BASE) project. The BASE project increases quality and
productivity in the development of embedded computer software. The
capabilities presented in the article are part of the BASE documentation
production system. Kevin can be contacted at 14424 34th Ave. S., #2, Seattle,
WA 98168.
I modified Ron Winter's PC/MS-DOS C Printer Utility (CP), presented in the
accompanying article, to run on two additional operating systems: VAX VMS and
VAX Unix. The modifications are divided into four sets of steps: The first set
involves creating the makefiles, the second set involves modifying the C
source code, the third involves compiling and running the new CP, and the
fourth set provides two optional enhancements to CP. The listings in my
version were developed using DEC VAX VMS 5.1-1 and DEC VAX Unix BSD 4.3.
Creating the makefiles
Step 1: An MMS utility description file was created for the VMS operating
system, as shown in Listing One (page 86).
Step 2: The make utility makefile shown in Listing Two (page 86) was created
for the Unix operating system.
Step 3: CP uses command-line options, so the following line should be added to
the VMS login.com file to create a foreign command to run CP:
 CP == "$DEVICE:[YOUR_ ACCOUNT .CP]CP.EXE"
Step 4: Because the C Printer executable is called "cp," it was necessary to
change it to something else so that it would not be confused with the Unix cp
command. CP (printed in bold in my Listing Two) was changed to mycp. You can
name it whatever you like.
Modifying the C Code
Because of space constraints, the entire source code for the VMS and the Unix
version of the cp program are not included with this article. Instead, I've
provided code that should be inserted into Ron's program, as well as code that
should replace portions of his listings. Ron's listings will be prefaced by
the letters "RW" (RW-Listing One, for example) so as not to be confused with
the listings in this article. However, the complete system is available on the
DDJ listings disk, the DDJ Forum on CompuServe, and on the DDJ Listing
Service.
Step 1: Define one constant per operating system using C preprocessor #define
commands. These constants are used in conjunction with the #if and #endif
structures to surround code that is to be conditionally compiled. The lines in
Listing Three (page 86) should be inserted into RW-Listing One between lines 3
and 4.
Step 2: The Microsoft C function declarations contain the reserved word
"near," which is not supported by VMS or Unix. The function declaration blocks
at the top of each source file ending in ".c" must be duplicated. Surround one
block with the #if MSDOS and #endif pair. Surround the other block with the
#if VMS and #endif pair. From the VMS block remove all instances of the "near"
reserved word. The lines in Listing Four (page 86) should replace lines 27
through 52 in RW-Listing Two. The other source files must also be modified in
this way. The Unix compiler produces syntax errors on function declarations,
so they were omitted.
The Microsoft C function definitions also contain the reserved word "near."
Function definitions containing the "near" reserved word must be duplicated.
Remove the "near" reserved word from the duplicated definitions to support VMS
and Unix. Surround the original and duplicate definitions with the #if MSDOS,
#elseif, and #endif structure as in Listing Five (page 86), which replaces
line 56 in RW-Listing Two. All function definitions in source files ending in
.c should be changed in this way.
Step 3: Each compiler supplies a different set of include files. Where the
include file names are the same, the contents most often differ. The lines in
Listing Six (page 86) replace lines 5 through 9 in RW-Listing One and the
lines in Listing Seven (page 86) replace line 25 in RW-Listing Two to make the
necessary include-file modifications.
Step 4: A few library functions in the MS-DOS version were not found in either
of the VAX C libraries. The strdup( ) function must be added to the code for
VMS and Unix to duplicate the functionality of the missing library function.
The function in Listing Eight (page 86) should be inserted between lines 824
and 825 in RW-Listing Two.
The CP report header contains the current date and time that is provided by
the MS-DOS _strdate() and _strtime() functions. Although not in the same
format, this information is provided by the time() and localtime() functions
in VMS and Unix. A new function was created for VMS and Unix using the
appropriate library calls. The MS-DOS code was moved into a function body and
replaced by a call to the new function. The function call in Listing Nine
(page 86) replaces lines 233 through 256 of RW-Listing Two and the functions
in Listing Ten (page 86) should be inserted between lines 210 and 211 in
RW-Listing Two.
Step 5: The MS-DOS and VMS tolower() library function returns the lowercase
equivalent of an uppercase character or the same character if it is already
lowercase. In Unix the function works the same if the character is uppercase,
but it makes a lowercase character even lower, returning some character that
is not a valid command-line option. Replace line 93 in RW-Listing Three with
the code in Listing Eleven (page 88), which uses the islower() function to
check the case of a character and passes the character to the tolower()
function only if it is uppercase.
Step 6: CP directs output to the CRT by specifying that the outfile be "CON."
On PC/MS-DOS, CON is a reserved system device and is therefore automatically
assigned. On VMS and Unix the code in Listing Twelve (page 88) must replace
line 850 in RW-Listing Two to facilitate this option.
Step 7: The Unix C compiler detected an error that neither the Microsoft nor
the VMS compiler found. Replace line 868 in RW-Listing Two with the line in
Listing Thirteen to remove the use of a variable called "errno" that was used
but never declared and could cause a run-time error.
Step 8: Memory size limitations are not a concern on the virtual machines for
the requirements of the C printer. Replace lines 95, 117, 140, 163, 188 in
RW-Listing Two with the line in Listing Fourteen (page 88) to use the MS-DOS
constant to inhibit CP's byte limit errors on VMS and Unix.
Compiling and Running CP
Step 1: After completing the steps required to modify the C source code, the
code can be compiled on all of the supported operating systems using the make
utilities and files.
Step 2: CP does not interpret C preprocessor directives, so compile the code
with the C preprocessor and then run CP on the preprocessed code. Listings
Fifteen (page 88) and Sixteen (page 88) contain the commands needed to run the
C preprocessor on the CP source files on VMS and Unix, respectively.
Step 3: The cp.cpi file shown in Listing Seventeen (page 88) must be used as
the list file to run CP on the preprocessed source code. If run on VMS or
Unix, the -n option should be used to suppress IBM-type graphics characters in
the output.
Enhancing CP
The following enhancements to the C Printer were developed to support the
construction of design documents for software created by The Boeing Company.
Step 1: Path names can be very long in hierarchical directory structures, so
it is necessary to modify CP in order that it will accept these long names in
its list file. Insert the line in Listing Eighteen (page 88) between lines 18
and 19 of RW-Listing One and change the value of the LEN_FILE constant to the
maximum size needed.
The rest of the changes needed are made by replacing line 262 of RW-Listing
Two with the code in Listing Nineteen (page 88) and line 270 of RW-Listing Two
with the code in Listing Twenty (page 88).
Step 2: When using long file names, the boxes that are displayed by CP get
overrun. The b option has been added to CP to allow the size of the box to be
varied at run time. See Listings Twenty-One through Twenty-Nine (pages 88-89)
for the code and instructions needed to implement this option. The
bounds-checking values (Listings Twenty-Eight and Twenty-Nine) and the default
value (Listing Twenty-Five) can be changed to suit your needs. Listing
Twenty-Seven will correct two errors with the help screen that were most
likely inadvertently left out of the original program. Due to lack of space,
the details of the code will not be discussed. If you have any questions about
this option or about any aspect of the C Printer write me at the address given
at the beginning of this article.

_AN AID TO DOCUMENTING C REVISITED_
by Ron Winter


[LISTING ONE]

/*********************************************************************
 cpheader.h
 *********************************************************************/

#include <malloc.h>
#include <conio.h>
#include <stdio.h>
#include <string.h>
#include <stdlib.h>

#define Max_unget_buffer 20000
#define Max_general_buffers 3000
#define MAX_functions 5000
/* #define Max_functions 4000 */
#define Max_defined_functions 1400
#define Max_files 1400
#define Max_Recursion 50

#define false 0
#define true 1

#define completed 2

#define Escape 0x1b
#define Control_z 0x1a

/*********************************************************************/
typedef struct the_Pages
 {
 int on_this_page;
 struct the_Pages *next_page_ptr;
 }linked_pages_list;
/**********************************************************************/
typedef struct
 {
 char *functions_name;
 char *its_filename;
 int is_referenced;
 int static_function;
 }function_type;
/**********************************************************************/
typedef struct
 {
 char *source_filename;
 char *source_file_comment;
 unsigned int line_count;
 long size;
 }file_record_type;
/**********************************************************************/
typedef struct /* this is the main data base record */
 {
 file_record_type *file_record_ptr;
 char *defined_function;
 function_type *ptr_to_function_table;
 int number_of_function_calls;
 linked_pages_list *ptr_to_page_list;
 int number_of_references;
 int static_definition;
 int overlay_number;
 }data_base_record_type;
/**********************************************************************/

#if MAIN != 0
/***********************************************************************/

function_type /* 6 */
 **sorted_called_list_ptrs,
 *function_list,
 *function_list_ptr;
int
 Max_functions,
 count_of_functions = 0;
/********************************/
file_record_type /* 14 */
 *file_record_array,
 *file_record_array_ptr;
int
 count_of_source_files = 0;
/********************************/
data_base_record_type /* 20 */

 *data_base_array,
 *data_base_array_ptr,
 **array_of_unused_ptrs_to_records,
 **array_of_ptrs_to_records;
int
 count_of_valid_records = 0;
/********************************/

char *recursion_array[ Max_Recursion ];
int recursion_depth = 0;

char nesting_display_buffer[ Max_general_buffers ];

char target[ 40 ] = "main";
FILE *output = NULL;

char push_buffer[ Max_unget_buffer ] = { 0, 0, 0, 0 };
char *push_buffer_ptr;

char file_comment_buffer[ Max_general_buffers ];
int first_comment;

int effective_width;

int
 page = 1,
 line = 0,
 defined_page_width = 80,
 defined_page_length = 60,
 defined_left_margin = 1,
 defined_right_margin = 1,
 stats_only = false,
 g_lib_flag = false,
 g_comment_flag = false,
 g_dec_def_flag = false,
 g_help_flag = false,
 ibm_flag = true,
 g_quiet_flag = false,
 g_tech_flag = false,
 g_ov_flag = false,
 g_un_flag = false,
 target_flag = false;
int top_of_form_done;
char title[] =
/* mm/dd/yy0 hh:mm:ss0 */
 { " C PRINTER - (c) 1987, 1988 rev. 1.3" };

/********************************************************************/

#else
/*********************************************************************/

extern function_type
 **sorted_called_list_ptrs,
 *function_list,
 *function_list_ptr;
extern file_record_type
 *file_record_array,
 *file_record_array_ptr;

extern data_base_record_type
 *data_base_array,
 *data_base_array_ptr,
 **array_of_unused_ptrs_to_records,
 **array_of_ptrs_to_records;
extern char *recursion_array[ ];
extern int
 count_of_valid_records,
 Max_functions,
 count_of_functions,
 count_of_source_files;
extern int page, line, recursion_depth;
extern int first_comment;
extern char nesting_display_buffer[ ];
extern char top_bottom_line_of_box[ ];
extern FILE *output;
extern char push_buffer[ ];
extern char *push_buffer_ptr;
extern char file_comment_buffer[ ];
extern int defined_page_width;
extern int defined_page_length;
extern int defined_left_margin;
extern int defined_right_margin;
extern int effective_width;
extern char target[ ];
extern int
 stats_only,
 g_lib_flag,
 g_comment_flag,
 g_dec_def_flag,
 g_help_flag,
 ibm_flag,
 g_quiet_flag,
 g_tech_flag,
 g_ov_flag,
 g_un_flag,
 target_flag;
extern int top_of_form_done;
extern char title[];
/*********************************************************************/

#endif
/**********************************************************************/





[LISTING TWO]

/*********************************************************************
 cp.c

static void near bump_line_count( void );
static void near do_top_of_page( void );
static void near deallocate_arrays( void );
static void near allocate_arrays( void );
static void near initialize_globals( void );
static void near build_records_from_list( FILE * );

static void near sort_the_data_base_array( void );
static void near count_all_defined_references( void );
static void near show_function_relationships( void );
static void near show_line_and_byte_counts( void );
static void near show_sorted_function_list( void );
static void near show_page_references( void );
static void near show_unused_if_any( );
static void near show_library_functions( void );
static void near show_files_leading_comments( );
 int main( int, char ** );

 ***************************************************************************/

#define MAIN 1
#include "cpheader.h"
#include "time.h"

extern int near binary_search_sorted_data_base( char * );
extern void near build_box_parts( int );
extern int near build_the_data_base( char *, char * );
extern void near check_for_new_page( void );
extern int near doprint( int );
extern void near nasty( int );
extern void near process_arguments( int, int, char **, int );
extern void near scan_for_static_or_global( int *, int, char *, char * );
extern void near tab_to_left_margin( FILE * );

static void near allocate_arrays( void );
static void near build_records_from_list( FILE * );
static void near bump_line_count( void );
static void near count_all_defined_references( void );
static void near deallocate_arrays( void );
static void near do_top_of_page( void );
static void near initialize_globals( void );
static void near show_files_leading_comments( void );
static void near show_function_relationships( void );
static void near show_library_functions( void );
static void near show_line_and_byte_counts( void );
static void near show_page_references( void );
static void near show_sorted_function_list( void );
static void near show_unused_if_any( void );
static void near sort_the_data_base_array( void );
 int main( int, char ** );

/***************************************************************************/

static void near bump_line_count( )
{
top_of_form_done = false;
++line;
check_for_new_page();
tab_to_left_margin( output );
}
/***************************************************************************/
static void near do_top_of_page( )
{
if( !top_of_form_done )
 {
 top_of_form_done = true;

 line = 9999;
 check_for_new_page();
 tab_to_left_margin( output );
 }
}
/***************************************************************************/
static void near deallocate_arrays( )
{
if( function_list )
 free( function_list );
if( file_record_array )
 free( file_record_array );
if( data_base_array )
 free( data_base_array );
if( sorted_called_list_ptrs )
 free( sorted_called_list_ptrs );
if( array_of_ptrs_to_records )
 free( array_of_ptrs_to_records );
}
/***************************************************************************/

static void near allocate_arrays( )
{
unsigned long length;

length = (unsigned long)Max_functions * sizeof( function_type );
if( length > 65535 )
 {
 (void)printf( "too many called functions ( go to huge model code )\n" );
 exit( 1 );
 }
else
 if(
 !( function_list =
 (function_type *)malloc( (unsigned int)length )
 )
 )
 {
 (void)printf( "No room for function_list\n" );
 exit( 1 );
 }
 else
 {
 if( !g_quiet_flag && g_tech_flag )
 (void)printf( "function list = %lu bytes long\n", length );
 }

length = (unsigned long)Max_files * sizeof( file_record_type );
if( length > 65535 )
 {
 (void)printf( "too many files ( go to huge model code )\n" );
 exit( 1 );
 }
else
 if(
 !( file_record_array =
 (file_record_type *)malloc( (unsigned int)length )
 )
 )

 {
 (void)printf( "No room for file_record_array\n" );
 exit( 1 );
 }
 else
 {
 if( !g_quiet_flag && g_tech_flag )
 (void)printf( "file record array = %lu bytes long\n", length );
 }

length =
 (unsigned long)Max_defined_functions * sizeof( data_base_record_type );
if( length > 65535 )
 {
 (void)printf( "too many defined functions ( go to huge model code )\n" );
 exit( 1 );
 }
else
 if(
 !( data_base_array =
 (data_base_record_type *)malloc( (unsigned int)length )
 )
 )
 {
 (void)printf( "No room for data_base_array\n" );
 exit( 1 );
 }
 else
 {
 if( !g_quiet_flag && g_tech_flag )
 (void)printf( "data base array = %lu bytes long\n", length );
 }

length =
 (unsigned long)Max_defined_functions * sizeof( data_base_record_type * );
if( length > 65535 )
 {
 (void)printf(
 "too many defined functions pointers( go to huge model code )\n"
 );
 exit( 1 );
 }
else
 if(
 !( array_of_ptrs_to_records =
 (data_base_record_type **)malloc( (unsigned int)length )
 )
 )
 {
 (void)printf( "No room for *array_of_ptrs_to_records\n" );
 exit( 1 );
 }
 else
 {
 if( !g_quiet_flag && g_tech_flag )
 (void)printf( "array of ptrs to data base = %lu bytes long\n",
 length );
 }


length = (unsigned long)Max_functions * sizeof( function_type * );
if( length > 65535 )
 {
 (void)printf(
 "too many called function ptrs ( go to huge model code )\n"
 );
 exit( 1 );
 }
else
 if(
 !( sorted_called_list_ptrs =
 (function_type **)malloc( (unsigned int)length )
 )
 )
 {
 (void)printf( "No room for ptr function_list\n" );
 exit( 1 );
 }
 else
 {
 if( !g_quiet_flag && g_tech_flag )
 (void)printf( "sorted called list ptrs = %lu bytes long\n", length );
 }
}
/***************************************************************************/

static void near initialize_globals( )
{
int i;
char *cp;

function_list_ptr = function_list;
data_base_array_ptr = data_base_array;
file_record_array_ptr = file_record_array;

for( i = 0; i < Max_Recursion; ++i )
 recursion_array[ i ] = NULL;
build_box_parts( ibm_flag );
effective_width = /******** set global output width ***********/
 defined_page_width - defined_left_margin - defined_right_margin;
if( effective_width < 40 )
 {
 (void)printf( "\nThe page width is too narrow( needs > 40 )." );
 exit( 1 );
 }

cp = &title[ 0 ]; /* insert date and nice time into title */
(void)_strdate( cp );
title[ 8 ] = ' ';
cp = &title[ 10 ];
(void)_strtime( cp );

title[ 15 ] = ' '; /* knock off seconds */
title[ 16 ] = ' '; /* put am, pm here */
title[ 17 ] = 'm';
title[ 18 ] = ' ';

i = atoi( &title[ 10 ] ); /* f/ military to civilian time */
title[ 16 ] = ( i < 12 )? (char)'a': (char)'p';


if( i == 0 )
 i = 12;
if( i >= 13 )
 i -= 12;

(void)sprintf( &title[ 10 ], "%2d", i );
title[ 12 ] = ':';

if( title[ 10 ] == '0' )
 title[ 10 ] = ' ';
}
/***********************************************************************/
static void near build_records_from_list( stream )
FILE *stream;
{
char input_list_filename[ 129 ], input_line[ 129 ], overlay_number[ 129 ];
int l;

while( !feof( stream ) )
 {
 input_list_filename[ 0 ] = '\0';
 input_line[ 0 ] = '\0';
 overlay_number[ 0 ] = '\0';
 fgets( input_line, 128, stream ); /* ends at \n or eof */

 if(
 ( l = strlen( input_line ) ) > 1 /* ie not nul string */
 )
 {
 if( input_line[ l - 1 ] == '\n' )
 input_line[ l - 1 ] = '\0';

 l = sscanf( input_line, " %s %s ",
 input_list_filename, overlay_number
 );
 if( !g_quiet_flag && g_tech_flag )
 {
 (void)printf( "pathname = %s ", input_list_filename );
 if( l )
 (void)printf( "overlay # = %s ", overlay_number );
 }
 (void)build_the_data_base( input_list_filename, overlay_number );
 }
 }
}
/***************************************************************************/

static void near sort_the_data_base_array( )
{
int i, still_sorting_flag;

for( i = 0, data_base_array_ptr = data_base_array;
 i < count_of_valid_records;
 ++i
 )
 array_of_ptrs_to_records[ i ] = data_base_array_ptr++;

if( !g_quiet_flag )

 {
 (void)printf( "\n\nSorting the function list...\n" );
 (void)printf( " of %d functions\n", count_of_valid_records );
 }
still_sorting_flag = true;
while( still_sorting_flag )
 {
 still_sorting_flag = false;
 if( !g_quiet_flag )
 {
 (void)printf( "." );
 }
 for( i = 0; i < count_of_valid_records - 1; ++i )
 {
 if( strcmp( array_of_ptrs_to_records[ i ]->defined_function,
 array_of_ptrs_to_records[ i + 1 ]->defined_function ) > 0 )
 {
 still_sorting_flag = true;
 data_base_array_ptr = array_of_ptrs_to_records[ i ];
 array_of_ptrs_to_records[ i ] = array_of_ptrs_to_records[ i + 1 ];
 array_of_ptrs_to_records[ i + 1 ] = data_base_array_ptr;
 }
 }
 }
}
/************************************************************************/

static void near count_all_defined_references()
{
register int count;
int found;
register function_type *f_list_ptr;

f_list_ptr = function_list; /* the full list */

for( count = 0; count < count_of_functions; ++count )
 {
 found = binary_search_sorted_data_base( f_list_ptr->functions_name );
 if( found >= 0 )
 scan_for_static_or_global( &found,
 f_list_ptr->static_function,
 f_list_ptr->functions_name,
 f_list_ptr->its_filename
 );
 if( found >= 0 )
 array_of_ptrs_to_records[ found ]->number_of_references +=
 f_list_ptr->is_referenced;
 ++f_list_ptr; /* for all defined functions */
 }
if( !g_quiet_flag && g_dec_def_flag )
 (void)printf( "\n" );
}
/***************************************************************************/

static void near show_function_relationships( )
{
int found;
int record_index;


found = binary_search_sorted_data_base( target );/* w/o knowing filename */
 /* note if static, will find random one if more than */
 /* one with same name */
if( found >= 0 )
 {
 recursion_depth = 0;
 if( !g_quiet_flag )
 {
 (void)printf( "Checking for usage...\n" );
 }
 count_all_defined_references();
 nesting_display_buffer[ 0 ] = '\0';
 if( !g_quiet_flag )
 {
 (void)printf( "Starting the printout...\n" );
 }
 if( !target_flag ) /* main is only called once */
 array_of_ptrs_to_records[ found ]->number_of_references = 1;
 line = 0;
 if( !stats_only )
 {
 (void)doprint( found ); /* of target function */
 for( record_index = 0;
 record_index < count_of_valid_records;
 ++record_index
 )
 {
 (void)fprintf( output, "\n" );
 ++line;
 if( array_of_ptrs_to_records[ record_index ]->number_of_references >
 1
 )
 (void)doprint( record_index );
 }
 }
 }
else /* cant find target */
 {
 (void)printf( "cant find %s, exitting\n", target );
 exit( 1 );
 }
}
/***************************************************************************/

static void near show_line_and_byte_counts( )
{
long int total_byte_count;
long int total_line_count;
int i;

file_record_array_ptr = file_record_array;

do_top_of_page();
(void)fprintf( output, "File statistics:\n" );
bump_line_count();
total_byte_count = 0l;
total_line_count = 0l;
for( i = 0; i < count_of_source_files; ++i )
 {

 (void)fprintf( output,
 "%-40s - %8u lines, %12ld bytes\n",
 file_record_array_ptr->source_filename,
 file_record_array_ptr->line_count,
 file_record_array_ptr->size
 );
 bump_line_count();

 total_byte_count += file_record_array_ptr->size;
 total_line_count += file_record_array_ptr->line_count;
 ++file_record_array_ptr;
 }
(void)fputc( '\n', output );
bump_line_count();
(void)fprintf( output, "Totals:\n" );
bump_line_count();
/******** "%-40s - %8u lines, %12ld bytes\n", *******/
(void)fprintf( output, "%4d files%-30s - %8ld lines, %12ld bytes\n",
 count_of_source_files, " ", total_line_count, total_byte_count
 );
bump_line_count();
(void)fputc( '\n', output );
bump_line_count();
(void)fprintf( output,
 " %d defined functions found.\n", count_of_valid_records
 );
bump_line_count();
(void)fprintf( output, "Averages:\n" );
bump_line_count();
(void)fprintf( output,
 "%6d lines/file, %6d functions/file, %6d lines/function\n",
 (int)( total_line_count / count_of_source_files ),
 (int)( count_of_valid_records / count_of_source_files ),
 (int)( total_line_count / count_of_valid_records )
 );
}
/***************************************************************************/

static void near show_sorted_function_list( )
{
int i, record_index;
long reference_total = 0;

do_top_of_page();

(void)fprintf( output, "Function index:\n" );
bump_line_count();

if( g_ov_flag )
 (void)fprintf( output, "%-39s %-28s %s %s\n",
 "function", "in file", "ov#", "refs" );
else
 (void)fprintf( output, "%-39s %-28s %s\n",
 "function", "in file", "refs" );

bump_line_count();

for( i = 0; i < effective_width; ++i )
 (void)fputc( '_', output );

(void)fprintf( output, "\n" );
bump_line_count();

for( record_index = 0;
 record_index < count_of_valid_records;
 ++record_index
 )
 {
 data_base_array_ptr = array_of_ptrs_to_records[ record_index ];
 if( data_base_array_ptr->number_of_references > 0 )
 {
 if( g_ov_flag && data_base_array_ptr->overlay_number )
 (void)fprintf( output, "%-7s%-32s %-28s %3d %d\n",
 ( data_base_array_ptr->static_definition )?
 "static": "",
 data_base_array_ptr->defined_function,
 ( data_base_array_ptr->file_record_ptr )->source_filename,
 data_base_array_ptr->overlay_number,
 data_base_array_ptr->number_of_references
 );
 else
 (void)fprintf( output, "%-7s%-32s %-28s %d\n",
 ( data_base_array_ptr->static_definition )?
 "static": "",
 data_base_array_ptr->defined_function,
 ( data_base_array_ptr->file_record_ptr )->source_filename,
 data_base_array_ptr->number_of_references
 );
 reference_total += (long)data_base_array_ptr->number_of_references;
 bump_line_count();
 }
 }
(void)fprintf( output, "%-7s%-32s %-28s %s\n",
 " ", " ", " ", "____"
 );
bump_line_count();
(void)fprintf( output, "%-7s%-32s %-28s %ld\n",
 " ", " ", "total ", reference_total
 );
bump_line_count();
}
/***************************************************************************/

static void near show_page_references( )
{
int pmax; /* max x ref columns */
int i, pcnt;
linked_pages_list *p;

if( !stats_only && ( defined_page_length > 0 ) )
 {
 pmax = (int)( effective_width - 7 - 32 - 2 ) / 5;
 do_top_of_page();
 (void)fprintf( output, "Function cross reference:\n" );
 bump_line_count();

 for( i = 0; i < count_of_valid_records; ++i )
 {
 data_base_array_ptr = array_of_ptrs_to_records[ i ];

 if( data_base_array_ptr->number_of_references > 0 )
 {
 (void)fprintf( output, "%-7s%-32s- ",
 ( data_base_array_ptr->static_definition )?
 "static": "",
 data_base_array_ptr->defined_function );
 p = data_base_array_ptr->ptr_to_page_list;
 if( p )
 {
 pcnt = 0;
 while( p->next_page_ptr )
 {
 (void)fprintf( output, "%4d,", p->on_this_page );
 p = p->next_page_ptr;
 ++pcnt;
 if( pcnt >= pmax )
 {
 (void)fputc( '\n', output );
 bump_line_count();
 (void)fprintf( output, "%7s%32s ", " ", " " );
 pcnt = 0;
 }
 }
 (void)fprintf( output, "%4d\n", p->on_this_page );
 }
 else
 (void)fprintf( output, "\n" );
 bump_line_count();
 }
 }
 }
}
/***************************************************************************/

static void near show_unused_if_any( )
{
int i, unused_count, unused_index, count, still_sorting_flag;
data_base_record_type **unused_list_ptr_ptr, *unused_list_ptr;

do_top_of_page();
(void)fprintf( output, "Un-used function list:\n" );
bump_line_count();

unused_count = 0;
for( i = 0; i < count_of_valid_records; ++i )
 {
 data_base_array_ptr = array_of_ptrs_to_records[ i ];
 if( !data_base_array_ptr->number_of_references )
 {
 ++unused_count;
 if( !g_un_flag )
 {
 (void)fprintf( output,
 "%-7s%-32s- %-33s\n",
 ( data_base_array_ptr->static_definition )?
 "static": "",
 data_base_array_ptr->defined_function,
 ( data_base_array_ptr->file_record_ptr )->source_filename
 );

 bump_line_count();
 }
 }
 }
if( g_un_flag ) /* show sorted */
 {
 if( unused_count )
 {
 if(
 !( array_of_unused_ptrs_to_records =
 (data_base_record_type **)malloc( (unsigned int)unused_count )
 )
 )
 (void)printf( "No room for *array_of_unused_ptrs_to_records\n" );
 else
 {
 unused_index = 0;
 for( i = 0; i < count_of_valid_records; ++i )
 {
 data_base_array_ptr = array_of_ptrs_to_records[ i ];
 if( !data_base_array_ptr->number_of_references )
 { /* first just collect them */
 array_of_unused_ptrs_to_records[ unused_index++ ] =
 data_base_array_ptr;
 }
 } /* so now there are unused_index of them */
 unused_list_ptr_ptr = array_of_unused_ptrs_to_records;
 still_sorting_flag = true;
 if( unused_count > 1 )
 {
 while( still_sorting_flag )
 {
 still_sorting_flag = false;
 if( !g_quiet_flag && g_tech_flag )
 (void)printf( ".%d \r", count );
 for( count = 0; count < unused_count - 1; ++count )
 {
 if( strcmp( unused_list_ptr_ptr[ count ]->
 file_record_ptr->source_filename,
 unused_list_ptr_ptr[ count + 1 ]->
 file_record_ptr->source_filename
 ) > 0
 )
 {
 still_sorting_flag = true;
 unused_list_ptr = unused_list_ptr_ptr[ count ];
 unused_list_ptr_ptr[ count ] =
 unused_list_ptr_ptr[ count + 1 ];
 unused_list_ptr_ptr[ count + 1 ] = unused_list_ptr;
 }
 }
 }
 }
 for( i = 0; i < unused_count; ++i )
 {
 (void)fprintf( output,
 "%-7s%-32s- %-33s\n",
 ( unused_list_ptr_ptr[ i ]->static_definition )?
 "static": "",

 unused_list_ptr_ptr[ i ]->defined_function,
 ( unused_list_ptr_ptr[ i ]->file_record_ptr )->source_filename
 );
 bump_line_count();
 }
 }
 }
 }
if( !unused_count )
 {
 tab_to_left_margin( output );
 (void)fprintf( output, "No un-used functions in the list.\n" );
 bump_line_count();
 }
else
 {
 (void)fprintf( output, "%-7s%-39s- %d\n", "", "totals", unused_count );
 bump_line_count();
 }
}
/************************************************************************/

static void near show_library_functions( )
{
register int count;
int found, total, still_sorting_flag, x_count, final_count, final_call;
function_type **f_list_ptr_ptr, *f_list_ptr;

if( g_lib_flag )
 {
 if( !g_quiet_flag && g_tech_flag )
 (void)printf( "collecting library functions...\n" );
 do_top_of_page();
 (void)fprintf( output, "Library functions:\n" );
 bump_line_count();

 total = 0;
 f_list_ptr = function_list;
 for( count = 0; count < count_of_functions; ++count )
 {
 if( !f_list_ptr->static_function )
 {
 if(
 ( found =
 binary_search_sorted_data_base( f_list_ptr->functions_name )
 ) < 0
 )
 sorted_called_list_ptrs[ total++ ] = f_list_ptr;
 }
 ++f_list_ptr; /* for all called functions */
 }

 if( !g_quiet_flag && g_tech_flag )
 (void)printf( "gathering identical library functions...\n" );
 final_count = total; /* number of calls to be collected and sorted */
 f_list_ptr_ptr = sorted_called_list_ptrs;
 for( count = 0; count < ( total - 1 ); ++count )
 {
 for( x_count = count + 1; x_count < total; ++x_count )

 {
 if( ( f_list_ptr_ptr[ count ]->functions_name[ 0 ] != '\0' ) &&
 !strcmp( f_list_ptr_ptr[ count ]->functions_name,
 f_list_ptr_ptr[ x_count ]->functions_name )
 )
 {
 f_list_ptr_ptr[ count ]->is_referenced +=
 f_list_ptr_ptr[ x_count ]->is_referenced;
 f_list_ptr_ptr[ x_count ]->functions_name[ 0 ] = '\0';
 --final_count;
 }
 }
 }

 if( !g_quiet_flag && g_tech_flag )
 {
 (void)printf( "\nSorting the library function calls...\n" );
 }

 f_list_ptr_ptr = sorted_called_list_ptrs;
 still_sorting_flag = true;
 while( still_sorting_flag )
 {
 still_sorting_flag = false;
 if( !g_quiet_flag && g_tech_flag )
 (void)printf( ".%d \r", count );
 for( count = 0; count < total - 1; ++count )
 {
 if( strcmp( f_list_ptr_ptr[ count ]->functions_name,
 f_list_ptr_ptr[ count + 1 ]->functions_name ) > 0 )
 {
 still_sorting_flag = true;
 f_list_ptr = f_list_ptr_ptr[ count ];
 f_list_ptr_ptr[ count ] = f_list_ptr_ptr[ count + 1 ];
 f_list_ptr_ptr[ count + 1 ] = f_list_ptr;
 }
 }
 }
 if( !g_quiet_flag && g_tech_flag )
 (void)printf( "\n" );

 (void)fprintf( output, "%-32s %-28s\n",
 "library function", "calls" );
 bump_line_count();

 for( count = 0; count < effective_width; ++count )
 (void)fputc( '_', output );
 (void)fprintf( output, "\n" );
 bump_line_count();

 final_call = 0;
 f_list_ptr_ptr = sorted_called_list_ptrs;
 for( count = 0; count < total; ++count )
 {
 if( ( *f_list_ptr_ptr )->functions_name[ 0 ] != '\0' )
 {
 (void)fprintf( output, "%-32s %d\n",
 ( *f_list_ptr_ptr )->functions_name,
 ( *f_list_ptr_ptr )->is_referenced

 );
 final_call += ( *f_list_ptr_ptr )->is_referenced;
 bump_line_count();
 }
 ++f_list_ptr_ptr;
 }
 (void)fprintf( output, "Totals:\n" );
 bump_line_count();
 (void)fprintf( output, "%6d %-25s %d calls.\n",
 final_count, "library functions,", final_call
 );
 bump_line_count();
 }
}
/************************************************************************/

static void near show_files_leading_comments( )
{
int i;
char *cp;

if( g_comment_flag )
 {
 do_top_of_page();
 (void)fprintf( output, "File comments:\n" );
 bump_line_count();
 file_record_array_ptr = file_record_array;
 for( i = 0; i < count_of_source_files; ++i )
 {
 (void)fprintf( output, "%40s\n",
 file_record_array_ptr->source_filename
 );
 bump_line_count();
 cp = file_record_array_ptr->source_file_comment;
 while( *cp )
 {
 (void)fprintf( output, "%c", *cp );
 if( *++cp == '\n' )
 {
 bump_line_count();
 }
 }
 ++file_record_array_ptr;
 do_top_of_page(); /* one page per comment at least */
 }
 }
}
/**********************************************************************/

int main( argc, argv )
char **argv;
int argc;
{
int index, in_error = false, out_error = false;
FILE *stream;

nasty( argc );

(void)printf( "\ncp - ver. 1.3, (C)1987, 1988 Stewart A. Nutter\n" );

(void)printf( " extended and corrected by Ron Winter\n" );

index = 1;
if( !( stream = fopen( argv[ index ], "rt" ) ) )
 in_error = true;
else
 ++index;
if(
 ( argc > index ) &&
 (
 ( argv[ index ][ 0 ] != '/' ) && ( argv[ index ][ 0 ] != '-' )
 )
 )
 {
 output = fopen( argv[ 2 ], "w+" ); /******* wt+ <<<<<<<< ******/
 ++index;
 }
else
 output = fopen( "prn", "w+" ); /******** wt+ <<<<<< ********/

if( !output )
 out_error = true;

Max_functions = MAX_functions;
process_arguments( index, argc, argv, in_error out_error );
if( in_error )
 {
 (void)printf( "\n can't open input list %s\n", argv[ 1 ] );
 exit( 1 );
 }
if( out_error )
 {
 (void)printf( "\n can't open output file, error %s\n", strerror( errno ) );
 exit( 1 );
 }
allocate_arrays( );
initialize_globals( );
(void)printf( "\n" );

build_records_from_list( stream );
sort_the_data_base_array( );
if( !g_quiet_flag )
 {
 (void)printf( "\n" );
 }
top_of_form_done = false;
show_function_relationships( );
show_page_references( );
show_line_and_byte_counts( );
show_sorted_function_list( );
show_unused_if_any( );
show_library_functions( );
show_files_leading_comments( );
deallocate_arrays( );

/************* done *****************/
(void)fprintf( output, "%c", 0x0c ); /* ff */

return false; /* ok */

}
/********************************************************************/






[LISTING THREE]

/***********************************************************************
 cpinput.c
 void near nasty( int );
 void near process_arguments( int, int, char **, int );
************************************************************************/
#define MAIN 0
#include "cpheader.h"

 void near nasty( int );
 void near process_arguments( int, int, char **, int );
/************************************************************************/

void near nasty( argc )
int argc;
{
if( argc < 2 )
 {
 (void)printf( "\ncp listfile [ outfile ] [\n" );
 (void)printf(
 " /p:nn /w:nn /m:nn /r:nn /t:main /f:nnnn\n"
 );
 (void)printf(
 " /l /n /s /q /d /c /h /x\n"
 );
 (void)printf( " ]\n\n" );
 (void)printf(
 " outfile = prn\n" );
 (void)printf(
 " p: page length = %3d [ 0, 50 -255 ]\n", defined_page_length
 );
 (void)printf(
 " w: page width = %3d [ 80 - 255 ]\n", defined_page_width
 );
 (void)printf(
 " m: left margin = %2d [ 0 - 30 ]\n", defined_left_margin
 );
 (void)printf(
 " r: right margin = %2d [ 0 - 30 ]\n", defined_right_margin
 );
 (void)printf(
 " t: target function = %s\n", target
 );
 (void)printf(
 " f: # of function calls = %4d [ 2 - 5461 ]\n", MAX_functions
 );
 (void)printf(
 " n: normal characters( ie not ibm character graphics )\n"
 );
 (void)printf(

 " l output library functions\n"
 );
 (void)printf(
 " c output file\'s 1st comment\n"
 );
 (void)printf(
 " s output statistics only\n"
 );
 (void)printf(
 " d show declarations and definitions\n"
 );
 (void)printf(
 " o show overlay information\n"
 );
 (void)printf(
 " u show unused sorted by filename\n"
 );
 (void)printf(
 " q show no messages\n"
 );
 (void)printf(
 " h show more help\n"
 );
 (void)printf(
 " x show tech info\n"
 );

 (void)printf( "\n" );
 exit( 0 );
 }
}
/**********************************************************************/
void near process_arguments( index, argc, argv, an_error )
int index, argc, an_error;
char **argv;
{
char c;
int i, tmp;

for( i = index; i < argc; ++i )
 {
 if( ( argv[ i ][ 0 ] == '/' ) ( argv[ i ][ 0 ] == '-' ) )
 {
 c = (char)tolower( (int)argv[ i ][ 1 ] );
 switch( c )
 {
 case 'n':
 ibm_flag = ( ibm_flag )? false: true;
 break;
 case 'l':
 g_lib_flag = ( g_lib_flag )? false: true;
 break;
 case 'c':
 g_comment_flag = ( g_comment_flag )? false: true;
 break;
 case 'd':
 g_dec_def_flag = ( g_dec_def_flag )? false: true;
 break;
 case 's':

 stats_only = ( stats_only )? false: true;
 break;
 case 'q':
 g_quiet_flag = ( g_quiet_flag )? false: true;
 break;
 case 'o':
 g_ov_flag = true;
 break;
 case 'u':
 g_un_flag = true;
 break;
 case 'h':
 g_help_flag = true;
 break;
 case 'x':
 g_tech_flag = true;
 break;
 default:
 if( ( strlen( argv[ i ] ) > 3 ) && ( argv[ i ][ 2 ] == ':' ) )
 {
 tmp = atoi( &argv[ i ][ 3 ] );
 switch( c )
 {
 case 'p':
 if( ( ( 50 < tmp ) && ( tmp < 256 ) ) ( tmp == 0 ) )
 defined_page_length = tmp;
 break;
 case 'm':
 if( ( 0 <= tmp ) && ( tmp <= 30 ) )
 defined_left_margin = tmp;
 break;
 case 'r':
 if( ( 0 <= tmp ) && ( tmp <= 30 ) )
 defined_right_margin = tmp;
 break;
 case 't':
 (void)strcpy( target, &argv[ i ][ 3 ] );
 target_flag = true;
 break;
 case 'w':
 if( ( 79 < tmp ) && ( tmp < 256 ) )
 defined_page_width = tmp;
 break;
 case 'f':
 if( ( 1 < tmp ) && ( tmp < 5462 ) )
 Max_functions = tmp;
 break;
 default:
 (void)printf(
 "\nUnknown argument character: %c, ignored!\n",
 argv[ i ][ 1 ]
 );
 break;
 } /* end of switch on character after / or - */
 } /* end of if :something */
 else
 (void)printf( "\nMissing : for argument %s, ignored!\n",
 argv[ i ] );
 break;

 } /* end of switch on character after / or - */
 } /* end of if / or - */
 else
 (void)printf( "\nUnknown argument: %s, ignored!\n", argv[ i ] );
 } /* end of for loop on arguments */

if( g_tech_flag )
 {
 (void)printf( "\n" );
 (void)printf( "Notes: 1. Max recursive function displacement of %d.\n",
 Max_Recursion
 );
 (void)printf(
" 2. Max # of unique function calls per defined function\n\
 for all defined functions is %d.\n",
 Max_functions );
 (void)printf( " 3. Max # of defined functions is %d.\n",
 Max_defined_functions );
 (void)printf( "\n" );
 (void)printf( "sizeof()\'s:\n" );
 (void)printf(
" function table = %u, contents = %u, data base = %u,\
 database = %u, lib = %u\n",
 sizeof( function_type ),
 sizeof( file_record_type ),
 sizeof( data_base_record_type ),
 sizeof( array_of_ptrs_to_records ),
 sizeof( sorted_called_list_ptrs )
 );
 (void)printf( "\n" );
 (void)printf(
 "The program will tend to show certain \'c\' functions as unused.\n" );
 (void)printf(
 "1. defined functions assigned to declared pointers to function names\n" );
 (void)printf(
 " and executed as pointers to those function names won't be seen.\n" );
 (void)printf(
 "2. #if(s) controlling the generation of code especially with\n" );
 (void)printf(
 " braces( { } ) in the conditional code section will especially\n" );
 (void)printf(
 " screw up if there is an #else code part. This program will work\n" );
 (void)printf(
 " on both code parts of the conditional and most probably get out\n" );
 (void)printf(
 " of sync with the braces. One might do a preprocessor pass compile\n" );
 (void)printf(
 " and heave it\'s output files as input files at this program.\n" );
 (void)printf(
 "3. #define(s) that expand to functions and call functions will also\n" );
 (void)printf(
 " be neglected. The preprocessor may be used as stated above.\n" );
/******
 (void)printf(
 "\n" );
******/
 (void)printf( "\n" );
 }


if( g_help_flag )
 {
 (void)printf( "\n" );
 (void)printf(
 "The listfile argument is an ascii text file containing the list of\n"
 );
 (void)printf(
 "filenames to process, one filename per line (optional overlay number.)\n"
 );
 (void)printf(
 "The output file may be a device or a filename. If there is no\n"
 );
 (void)printf(
 "output filename, \'prn\' is assumed. Note that one may put \'con\'\n"
 );
 (void)printf(
 "here and view the output of the program before printing or saving\n"
 );
 (void)printf(
 "to a filename.\n"
 );
 (void)printf(
 "Also note that the output filename and the input filenames in the\n"
 );
 (void)printf(
 "listfile may be full pathnames with drives and or paths.\n"
 );
 (void)printf( "/ arguments accept the alternate - form.\n" );
 (void)printf( "For example: cp x y -s, cp /h, cp x -x /d -t:junk\n" );
 (void)printf( "arguments may be in upper or lower case.\n" );
 (void)printf( "Note that the target function is case sensitive\n" );
 (void)printf( "since it is a \'c\' function name.\n" );
 (void)printf( "\n" );
 }
if( an_error )
 {
 if( g_help_flag g_tech_flag )
 exit( 0 );
 else
 (void)printf( "Oops..." );
 }
}
/***********************************************************************/






[LISTING FOUR]

/***************************************************************************
 cpbuild.c
static void near mark_as_static( function_type *, char*, int );
static int near test_and_add( function_type *, char *, int );
static void near unget_chars( char );
static char near get_chars( FILE * );
static char near get_to_next_possible_token( FILE * );
static int near is_legal_identifier_character( char );

 int near build_the_data_base( char * );
***************************************************************************/

#define MAIN 0
#include "cpheader.h"

 int near build_the_data_base( char * );
static char near get_chars( FILE * );
static char near get_to_next_possible_token( FILE * );
static int near is_legal_identifier_character( char );
static void near mark_as_static( function_type *, char*, int );
static int near test_and_add( function_type *, char *, int );
static void near unget_chars( char );

/***************************************************************************/
static void near mark_as_static( ptr_to_function_list,
 name_of_static_function, count
 )
char *name_of_static_function;
function_type *ptr_to_function_list;
int count;
{
int i;

for( i = 0; i < count; ++i )
 {
 if(
 !strcmp( name_of_static_function, ptr_to_function_list->functions_name )
 )
 ptr_to_function_list->static_function = true;
 ++ptr_to_function_list;
 }
}
/***************************************************************************/
#define KEYS 7

static int near test_and_add( ptr_to_function_list, string, count )
function_type *ptr_to_function_list;
char *string;
int count;
{
int i, is_a_new_function_name;
static char *keywords[ KEYS ] =
 { /* must catch do (void)printf, while(), else (void)... etc. ***/
 "do", "while", "if", "else", "for", "switch", "return"
 };

for( i = 0; ( i < KEYS ) && ( strcmp( string, keywords[ i ] ) != 0 ); ++i )
 ;
if( i < KEYS )
 is_a_new_function_name = false; /* ie a reserved word match */
else /* is a function name */
 {
 for( i = 0; i < count; ++i )
 {
 if( !strcmp( string, ptr_to_function_list->functions_name ) )
 { /* function name matches */
 if( !ptr_to_function_list->static_function )
 break; /* and isn't static */

 else
 {
 if( !strcmp( ptr_to_function_list->its_filename,
 file_record_array_ptr->source_filename
 )
 )
 break; /* only statics in same file match */
 }
 }
 ++ptr_to_function_list;
 }
 if( i == count )
 { /* new function name */
 is_a_new_function_name = true; /* add function name to list */
 if( ( function_list_ptr->functions_name = strdup( string ) ) == NULL )
 {
 (void)fprintf( stderr, "Ran out of memory.\n" );
 exit( 1 );
 }
 function_list_ptr->static_function = false;
 function_list_ptr->its_filename =
 file_record_array_ptr->source_filename;
 function_list_ptr->is_referenced = 1;

 ++function_list_ptr; /* point to next empty cell */
 ++count_of_functions; /* increase current size */
 if( count_of_functions > Max_functions )
 {
 (void)fprintf( stderr, "Too many functions.\n" );
 exit( 1 );
 }
 }
 else /* string already in function list */
 {
 is_a_new_function_name = false;
 ptr_to_function_list->is_referenced++;
 }
 }
return is_a_new_function_name;
}
/***************************************************************************/
static void near unget_chars( c )
char c;
{
if( ( push_buffer_ptr - push_buffer ) < Max_unget_buffer )
 *push_buffer_ptr++ = c;
else
 {
 (void)fprintf( stderr, "\nProgram syntax error:" );
 (void)fprintf( stderr, " Too many pushed characters.\n" );
 exit( 1 );
 }
}
/***************************************************************************/
static char near get_chars( stream )
FILE * stream;
{
register char c;


if( push_buffer_ptr != push_buffer )
 c = *--push_buffer_ptr;
else
 {
 c = (char)fgetc( stream );
 if( c == EOF )
 c = Control_z;
 if( c == 0x0a )
 {
 file_record_array_ptr->line_count++;
 file_record_array_ptr->size++; /* count the unseen <cr> */
 }
 file_record_array_ptr->size++;
 }
return c;
}
/***************************************************************************/
static char near get_to_next_possible_token( stream )
FILE *stream;
{
register char
 c;
char
 next_char_peek;
int
 done;

static int /* the only apparent reason these are static is for speed */
 quotes_flag = false,
 comment_flag = false,
 escape_sequence_flag = false,
 pound_sign_flag = false,
 ascii_quote_flag = false;
static int
 fp = 0; /*****<<<<< */
static char *cp;

done = false;
do {
 c = get_chars( stream );
 if( c != Control_z )
 {
 if( comment_flag )
 {
/**************************
 process /* comment sequence of characters
***************************/
 if( first_comment == true )
 {
 if( fp < ( Max_general_buffers - 2 ) )
 {
 if(
 ( c != '\n' ) &&
 ( strlen( cp ) < effective_width )
 )
 {
 file_comment_buffer[ fp++ ] = c;
 file_comment_buffer[ fp ] = '\0';
 }

 else /* c == \n or length >= width */
 {
 file_comment_buffer[ fp++ ] = '\n';
 file_comment_buffer[ fp ] = '\0';
 cp = (char *)&file_comment_buffer[ fp ];
 if( c != '\n' )
 {
 file_comment_buffer[ fp++ ] = c;
 file_comment_buffer[ fp ] = '\0';
 }
 }
 }
/* else /* 1st comment exceeds buffer */
 } /* end of if first_comment == true */
 if( c == '*' )
 {
 next_char_peek = get_chars( stream );
 if( next_char_peek == '/' ) /* close comment */
 {
 comment_flag = false;
 unget_chars( ' ' ); /* comments are white space in 'c' */
 if( first_comment == true )
 {
 first_comment = completed;
 fp = 0;
 cp = (char *)&file_comment_buffer[ fp ];
 }
 }
 else /* next_char_peek != '/' ie close comment */
 unget_chars( (char)next_char_peek );
 } /* end of if c == '*' */
 }
 else /* not /* */
 {
/**************************
 process \sequence character, hoping \" \' \\ etc inside " or '
***************************/
 if( escape_sequence_flag )
 escape_sequence_flag = false;
 else /* not /*, not \ */
 {
/**************************
 process " string sequence of characters
***************************/
 if( quotes_flag )
 {
 if( c == '\\' ) /* check for \'\n' */
 {
 next_char_peek = get_chars( stream );
 if( next_char_peek != '\n' ) /* so not \'\n' */
 {
 escape_sequence_flag = true;
 unget_chars( (char)next_char_peek );
 }
/******* else /* \'\n' continuation */
 }
 else /* not \ */
 if( c == '\"' )
 quotes_flag = false;

 }
 else /* not ", not /*, not \ */
 {
/**************************
 process ' ascii character sequence
***************************/
 if( ascii_quote_flag )
 {
 if( c == '\\' )
 escape_sequence_flag = true;
 else
 if( c == '\'' )
 ascii_quote_flag = false;
 }
 else /* not ', not ", not /*, not \ */
 {
/**************************
 process # sequence of characters, ie #if, #define, etc.
 define causes code sequencing problems it would seem!
***************************/
 if( pound_sign_flag )
 {
 if( c == '/' ) /* comments override #defines etc */
 {
 next_char_peek = get_chars( stream );
 if( next_char_peek == '*' )
 comment_flag = true;
 else
 unget_chars( (char)next_char_peek );
 }
 else
 {
 if( c == '\n' )
 pound_sign_flag = false;
 else /* c != \n */
 {
 if( c == '\\' ) /* check for \'\n' continuation */
 {
 next_char_peek = get_chars( stream );
 if( next_char_peek != '\n' ) /* it aint \'\n' */
 unget_chars( (char)next_char_peek );
/* else /* \'\n' means continue # */
 }
 }
 }
 }
 else /* not ', not #, not ", not /*, not \ */
 {
/**************************
 process anything else
***************************/
 done = false; /* assume a ' or " or # or /* */
 switch( c )
 {
 case '\"':
 quotes_flag = true;
 break;
 case '\'':
 ascii_quote_flag = true;

 break;
 case '#':
 pound_sign_flag = true;
 break;
 case '/':
 next_char_peek = get_chars( stream );
 if( next_char_peek == '*' )
 {
 comment_flag = true;
 if( first_comment == false )
 { /* the 1st comment of the file */
 first_comment = true;
 fp = 0;
 cp = (char *)&file_comment_buffer[ fp ];
 }
 }
 else
 {
 unget_chars( (char)next_char_peek );
 done = true;
 }
 break;
 default: /* a worthy character to return */
 done = true;
 }
 } /* end of else not ascii */
 } /* end of else not # */
 } /* end of else not " */
 } /* end of else not /* */
 } /* end of else not \ */
 } /* end of if c != Control_z */
 }
while( !done && ( c != Control_z ) );
if( c == Control_z )
 {
 ascii_quote_flag = false;
 pound_sign_flag = false;
 quotes_flag = false;
 escape_sequence_flag = false;
 comment_flag = false;
 fp = 0;
 }
return c;
}
/***************************************************************************/
static int near is_legal_identifier_character( c )
char c;
{
if(
 ( ( 'A' <= c ) && ( c <= 'Z' ) ) 
 ( ( 'a' <= c ) && ( c <= 'z' ) ) 
 ( ( '0' <= c ) && ( c <= '9' ) ) 
 ( c == '_')
 )
 return true;
else
 return false;
}
/***************************************************************************/

#define C_line_length 512
#define C_identifier_length 80

int near build_the_data_base( the_filename )
char * the_filename;
{
static char fake_comment[ ] = "no room!";
int found_a_possible_function;
int brace_count, body_found;
int open_parenthesis, parenthesis_count;
int at_end_of_source_file;
int dummy_index, total_called_count;
int function_definition_flag, static_flag;
int analyze_buffer_flag = false;
char c;
char *function_name_buffer_ptr;
char function_name_buffer[ C_identifier_length ];
char look_ahead_buffer[ C_line_length + 1 ];
FILE *stream;
data_base_record_type *data_base_ptr, *starting_data_base_ptr;
function_type *starting_called_function_ptr;

if( !g_quiet_flag )
 {
 (void)printf( "Processing file: %-12s\n", the_filename );
 }
if( !( stream = fopen( the_filename, "r" ) ) ) /***** rt <<<<<<<<<< */
 {
 (void)printf( "Cant open %s\n", the_filename );
 return -1;
 }

push_buffer_ptr = push_buffer; /* reset input character stack */
 /* add file name to data base */
if( !( file_record_array_ptr->source_filename = strdup( the_filename ) ) )
 {
 (void)printf( "Ran out of memory.\n" );
 exit( 1 );
 }

starting_called_function_ptr = function_list_ptr;
starting_data_base_ptr = data_base_array_ptr; /* mark start of defined list */

look_ahead_buffer[ 0 ] = '\0';

first_comment = false;
file_comment_buffer[ 0 ] = '\0';

file_record_array_ptr->line_count = 0; /* clear it's variables */
file_record_array_ptr->size = 0l;

function_name_buffer_ptr = function_name_buffer;
function_name_buffer[ 0 ] = '\0';

static_flag = false;
found_a_possible_function = false;
open_parenthesis = false;
body_found = false;


brace_count = 0;
parenthesis_count = 0;

at_end_of_source_file = false;
while( !at_end_of_source_file )
 {
 c = get_to_next_possible_token( stream );
 switch( c )
 {
 case '{':
 ++brace_count;
 break;
 case '}':
 --brace_count;
 break;
 case Control_z:
 at_end_of_source_file = true;
 analyze_buffer_flag = true;
 break;
 case '(':
 if( !open_parenthesis )
 ++open_parenthesis;
 analyze_buffer_flag = true;
 break;
 case ' ': /* this is where we eat white space */
 case '\v':
 case '\b':
 case '\f':
 case '\t':
 case '\r':
 case '\n':
 do {
 c = get_to_next_possible_token( stream );
 }
 while(
 ( c == '\f' ) ( c == ' ' ) ( c == '\v' ) 
 ( c == '\b' ) ( c == '\t' ) ( c == '\r' ) 
 ( c == '\n' )
 );
 unget_chars( c ); /* put next non white character back */

 if( c != '(' )
 analyze_buffer_flag = true;
/*** else /* c == '(' and next pass will find it */
 break;
 default:
 if( is_legal_identifier_character( c ) )
 { /* it's a good identifier character */
 *function_name_buffer_ptr++ = c;
 *function_name_buffer_ptr = '\0';
 }
 else /* it aint, so toss it */
 {
 if( static_flag && ( c == ';' ) )
 static_flag = false;
/* if( c != '*' ) */
 analyze_buffer_flag = true;
 }
 break;

 } /* end of preliminary character parse */
/*****************
 start checking characters accumulated in function_name_buffer[]
******************/
 if( analyze_buffer_flag )
 {
 analyze_buffer_flag = false;
 if(
 function_name_buffer[ 0 ] && /* ie not null string */
 ( /* & not number */
 ( function_name_buffer[ 0 ] < '0' ) 
 ( function_name_buffer[ 0 ] > '9' )
 )
 )
 found_a_possible_function = true;
 else /* it aint an identifier */
 { /* so erase buffer */
 function_name_buffer_ptr = function_name_buffer;
 function_name_buffer[ 0 ] = '\0';
 if( static_flag && ( c == ';' ) )
 static_flag = false;
 open_parenthesis = false;
 }
 } /* end of analyze_buffer_flag */
/*****************
 if function_name_buffer[] has legal function name, scan ahead
******************/
 if( found_a_possible_function )
 {
 found_a_possible_function = false;
 *function_name_buffer_ptr = '\0'; /* append nul char to end */
 if( !static_flag ) /* don't retest if true */
 if( !strcmp( function_name_buffer, "static" ) )
 static_flag = true;
 if( open_parenthesis )
 {
 open_parenthesis = false;
 if( !brace_count )
 { /* ie outside any function body */
 parenthesis_count = 1;
 for( dummy_index = 0;
 ( dummy_index < C_line_length ) && parenthesis_count;
 ++dummy_index
 )
 { /* scan ahead for function() */
 c = get_to_next_possible_token( stream );
 if( c == Control_z )
 break; /* dummy_index not bumped */
 look_ahead_buffer[ dummy_index ] = c;
 look_ahead_buffer[ dummy_index + 1 ] = '\0';
 switch( c )
 {
 case '(':
 ++parenthesis_count;
 break;
 case ')':
 --parenthesis_count;
 break;
 } /* dummy_index is bumped */

 } /* end of for loop scanning for (...) */
 if( ( c == Control_z ) ( !parenthesis_count ) )
 --dummy_index;
 function_definition_flag = false;
 for( ++dummy_index;
 ( dummy_index < C_line_length ) && !function_definition_flag;
 ++dummy_index
 )
 { /* what happens past (..) */
 c = get_to_next_possible_token( stream );
 if( c == Control_z )
 break; /* w/ function_definition_flag == false */
 look_ahead_buffer[ dummy_index ] = c;
 look_ahead_buffer[ dummy_index + 1 ] = '\0';
 switch( c )
 {
 case ' ': /* this is where we eat white space */
 case '\v':
 case '\b':
 case '\f':
 case '\t':
 case '\n':
 case '\r':
 break;
 case '{':
 ++body_found;
 break;
 case ';':
 case ',':
 case '(': /* at (*)() type declaration */
 if( !body_found )
 {
 function_definition_flag = true; /* declaration */
 if( !g_quiet_flag )
 {
 if( g_dec_def_flag )
 {
 if( static_flag )
 (void)printf( " static" );
 else
 (void)printf( " " );
 (void)printf( " declaration " );
 (void)printf( "%s(%s\n",
 function_name_buffer,
 look_ahead_buffer );
 }
 }
 }
 break;
 default: /* any other non white character means */
 function_definition_flag = completed;
 if( !g_quiet_flag )
 {
 if( g_dec_def_flag )
 {
 if( static_flag )
 (void)printf( "static " );
 else
 (void)printf( " " );

 (void)printf( "define " );
 }
 }
 break;
 } /* dummy_index is bumped */
 } /* end of for loop parsing character after ) */
 body_found = false;
 if( function_definition_flag == false )
 {
 (void)printf( "\nSyntax error: " );
 (void)printf( "Function description.\n" );
 look_ahead_buffer[ dummy_index ] = '\0';
 (void)printf( "\n%s\n", look_ahead_buffer );
 exit( 1 );
 }
 while( dummy_index )
 { /* put all characters after ( back */
 unget_chars( look_ahead_buffer[ dummy_index - 1 ] );
 --dummy_index;
 }
 if( function_definition_flag == completed )
 {
 if( !g_quiet_flag )
 {
 if( g_dec_def_flag )
 (void)printf( "%-40s\n", function_name_buffer );
 }
/*******************
 this element can distinguish static functions
 in different files with the same name
 *******************/
 data_base_array_ptr->file_record_ptr = file_record_array_ptr;
 data_base_array_ptr->number_of_function_calls = 0;
 data_base_array_ptr->ptr_to_function_table = function_list_ptr;
 data_base_array_ptr->static_definition = static_flag;
 static_flag = false;

 if(
 !( data_base_array_ptr->defined_function =
 strdup( function_name_buffer )
 )
 )
 {
 (void)printf( "\nRan out of memory( for strdup() )." );
 exit( 1 );
 }
 data_base_array_ptr->number_of_references = 0;
 data_base_array_ptr->ptr_to_page_list = NULL;

 data_base_ptr = data_base_array_ptr; /* save current pointer */
 ++data_base_array_ptr; /* next entry */
 ++count_of_valid_records;
 if( count_of_valid_records > Max_defined_functions )
 {
 (void)printf( "\nToo many new functions\n" );
 exit( 1 );
 }
 } /* end of function definition */
 static_flag = false;

 }
 else /* brace_count is not zero */
 { /* so inside a function */
 data_base_ptr->number_of_function_calls +=
 test_and_add( data_base_ptr->ptr_to_function_table,
 function_name_buffer,
 data_base_ptr->number_of_function_calls
 );
 }
 look_ahead_buffer[ 0 ] = '\0'; /* reset tail buffer */
 static_flag = false;
 } /* end of parenthesis */
 function_name_buffer_ptr = function_name_buffer; /* reset buffer */
 *function_name_buffer_ptr = '\0';
 } /* end of found_a_possible_function */
 } /* end of while !at_end_of_source_file */
(void)fclose( stream );
if( !g_quiet_flag )
 {
 (void)printf( "\n" );
 }

if(
 !( file_record_array_ptr->source_file_comment =
 strdup( file_comment_buffer )
 )
 )
 file_record_array_ptr->source_file_comment = fake_comment;

/***** mark called functions in list as static if in defined list *******/
total_called_count = 0;
data_base_ptr = starting_data_base_ptr;
while( data_base_ptr != data_base_array_ptr )
 {
 total_called_count += data_base_ptr->number_of_function_calls;
 ++data_base_ptr;
 }
data_base_ptr = starting_data_base_ptr;
while( data_base_ptr < data_base_array_ptr )
 {
 if( data_base_ptr->static_definition )
 mark_as_static( starting_called_function_ptr,
 data_base_ptr->defined_function,
 total_called_count
 );
 ++data_base_ptr;
 }
++file_record_array_ptr; /* next file name entry */
++count_of_source_files;
if( count_of_source_files >= Max_files )
 {
 (void)printf( "\nError: too many files to process.\n" );
 exit( 1 );
 }
return at_end_of_source_file;
}
/***************************************************************************/







[LISTING FIVE]

/***************************************************************************
 cpfuncts.c
 void near build_box_parts( int );
 void near tab_to_left_margin( FILE * );
static void near stop( void );
static void near setpage( data_base_record_type * );
static int near recursion_check( char *, int );
 void near check_for_new_page( void );
static void near draw_output_block( char *, char *, char *,
 char *, int, int, int );
 int near doprint( int );
 void near scan_for_static_or_global( int *, int, char *, char * );
 int near binary_search_sorted_data_base( char * );

 ***************************************************************************/

#define MAIN 0
#include "cpheader.h"

static char
 top_line_of_box[ 37 ], bottom_line_of_box[ 37 ],
 wall, ibm_line, bottom_attach,
 upper_left_corner, lower_left_corner,
 upper_right_corner, lower_right_corner,
 left_attach, right_attach;

static char *recursion_filename, *test_filename;
static int static_recursion;

 int near binary_search_sorted_data_base( char * );
 void near build_box_parts( int );
 void near check_for_new_page( void );
 int near doprint( int );
 void near scan_for_static_or_global( int *, int, char *, char * );
 void near tab_to_left_margin( FILE * );

static void near draw_output_block( char *, char *, char *,
 char *, int, int, int );
static int near recursion_check( char *, int );
static void near stop( void );
static void near setpage( data_base_record_type * );

/***************************************************************************/
void near build_box_parts( is_ibm )
int is_ibm;
{
int i;

if( is_ibm )
 {
 wall = '\xb3';
 ibm_line = '\xc4';
 bottom_attach = '\xc2';

 upper_left_corner = '\xda';
 lower_left_corner = '\xc0';
 upper_right_corner = '\xbf';
 lower_right_corner = '\xd9';
 left_attach = '\xb4';
 right_attach = '\xc3';
 }
else
 {
 wall = '';
 ibm_line = '-';
 bottom_attach = '+';
 upper_left_corner = '+';
 lower_left_corner = '+';
 upper_right_corner = '+';
 lower_right_corner = '+';
 left_attach = '+';
 right_attach = '+';
 }

top_line_of_box[ 0 ] = upper_left_corner;
bottom_line_of_box[ 0 ] = lower_left_corner;
for( i = 1; i <= 34; ++i )
 {
 top_line_of_box[ i ] = ibm_line;
 bottom_line_of_box[ i ] = ibm_line;
 }
top_line_of_box[ i ] = upper_right_corner;
bottom_line_of_box[ i ] = lower_right_corner;
top_line_of_box[ ++i ] = '\0';
bottom_line_of_box[ i ] = '\0';
}
/***************************************************************************/
void near tab_to_left_margin( output )
FILE *output;
{
register int i;

for( i = 0; i < defined_left_margin; ++i )
 (void)fputc( ' ', output );
}
/***************************************************************************/
static void near stop()
{
(void)printf( "hello" );
}
/***************************************************************************/
static void near setpage( data_base_ptr )
data_base_record_type *data_base_ptr;
{
linked_pages_list *page_list_ptr;

page_list_ptr = data_base_ptr->ptr_to_page_list;
if( page_list_ptr == NULL )
 {
 if(
 !( page_list_ptr =
 (linked_pages_list *)malloc( sizeof( linked_pages_list ) )
 )

 )
 {
 (void)fprintf( stderr, "Ran out of memory for page # list.\n" );
 exit( 1 );
 }

 data_base_ptr->ptr_to_page_list = page_list_ptr;
 }
else
 {
 while( page_list_ptr->next_page_ptr )
 page_list_ptr = page_list_ptr->next_page_ptr;

 if(
 !( page_list_ptr->next_page_ptr =
 (linked_pages_list *)malloc( sizeof( linked_pages_list ) )
 )
 )
 {
 (void)fprintf( stderr, "Ran out of memory for page # list.\n" );
 exit( 1 );
 }

 page_list_ptr = page_list_ptr->next_page_ptr;
 }
page_list_ptr->next_page_ptr = NULL;
page_list_ptr->on_this_page = page - 1;
}
/***************************************************************************/
static int near recursion_check( string, static_call )
char *string;
int static_call;
{
register char **recursion_array_ptr;

recursion_array_ptr = recursion_array;
if( static_recursion )
 { /* defined function is static */
 while(
 *recursion_array_ptr && /* not null */
 /* and different function names */
 ( strcmp( *recursion_array_ptr, string ) 
 /* or same function names and */
 /* in different files */
 strcmp( test_filename, recursion_filename )
 )
 )
 ++recursion_array_ptr;
 }
else
 { /* defined function is not static */
 while(
 *recursion_array_ptr && /* not null & */
 /* and different function names */
 ( strcmp( *recursion_array_ptr, string ) 
 /* or same function names and */
 static_call /* called is static */
 )
 )

 ++recursion_array_ptr;
 }
return ( *recursion_array_ptr )? true: false;
}
/***************************************************************************/
void near check_for_new_page()
{
int i;

if( defined_page_length == 0 && line == 9999 )
 {
 (void)fprintf( output, "\n\n\n\n" );
 line = 0;
 }
else
 {
 if( defined_page_length != 0 )
 {
 if( line > ( defined_page_length - 5 ) )
 {
 (void)fprintf( output, "\f" );
 line = 0;
 }
 if( line == 0 )
 {
 top_of_form_done = true;
 tab_to_left_margin( output );
 (void)fprintf( output, "%s", title );
 for( i = strlen( title ); i < ( effective_width - 10 ); ++i )
 (void)fputc( ' ', output );
 (void)fprintf( output, "Page:%4d\n", page );
 tab_to_left_margin( output );
 for( i = 0; i < effective_width; ++i )
 (void)fputc( '_', output );
 (void)fprintf( output, "\n\n" );
 line = 3;
 ++page;
 }
 }
 }
}
/***************************************************************************/
static void near draw_output_block( lead_in_string,
 name_of_function,
 description,
 name_of_file,
 either_count,
 tail_flag,
 kill_flag
 )
char *lead_in_string,
 *name_of_function,
 *description,
 *name_of_file;
int either_count, tail_flag, kill_flag;
{
unsigned int string_length;
static char alternate_lead_in[ 140 ];


/******* 1st line ***********************************************************/
tab_to_left_margin( output );
(void)fprintf( output, "%s %s\n", lead_in_string, top_line_of_box );

/******* 2nd line ***********************************************************/
tab_to_left_margin( output );
string_length = strlen( lead_in_string );
if( string_length ) /******* ie not main or defined function box ***/
 {
 (void)strncpy( alternate_lead_in, lead_in_string, --string_length );
 alternate_lead_in[ string_length++ ] = '\0'; /* restore string_length */
 }
if( string_length ) /******* ie not main or defined function box ***/
 (void)fprintf( output, "%s%c%c%c%-33s %c\n",
 alternate_lead_in,
/*** if( kill_flag ) /****** last line to this box ******************/
/*** else /****** line continues downwards ***************/
 ( kill_flag )? lower_left_corner: right_attach,
 ibm_line, left_attach, name_of_function, wall );
else /****** main or defined box starting ***********/
 (void)fprintf( output, "%c%c%-33s %c\n",
 ibm_line, left_attach, name_of_function, wall );

/******* 3rd line ***********************************************************/
tab_to_left_margin( output );
if( string_length-- ) /***** kill outside vertical line on last box ****/
 lead_in_string[ string_length++ ] = ( kill_flag )? (char)' ': wall;
(void)fprintf( output, "%s %c%-20s %8s%3d %c\n",
 lead_in_string, wall, description,
 name_of_file, either_count, wall );

/******* 4th line ***********************************************************/
tab_to_left_margin( output );
bottom_line_of_box[ 2 ] = /******** if defined box has calls ***********/
 ( tail_flag && either_count )? bottom_attach: ibm_line;
(void)fprintf( output, "%s %s\n", lead_in_string, bottom_line_of_box );

line += 4;
top_of_form_done = false;
}
/***************************************************************************/
static char library_string[] = { "(library)" };
static char usage_string[] = { "Used=" };
static char funct_string[] = { "Functs=" };

int near doprint( index )
int index;
{
int
 loop_counter,
 max_count,
 starting_index,
 found,
 return_value;
data_base_record_type *record_ptr;
function_type *f_list_ptr;

static int kill_flag = false;


starting_index = index;
record_ptr = array_of_ptrs_to_records[ starting_index ];

recursion_array[ recursion_depth ] = record_ptr->defined_function;
if( !recursion_depth )
 {
 recursion_filename = record_ptr->file_record_ptr->source_filename;
 /* add function to list for recursion check */
 static_recursion = record_ptr->static_definition;
 }
check_for_new_page();
setpage( array_of_ptrs_to_records[ starting_index ] );

return_value = page - 1; /* must be a relic! */
 /* start w/ target function */
draw_output_block( nesting_display_buffer,
 record_ptr->defined_function,
 ( record_ptr->file_record_ptr )->source_filename,
 funct_string,
 record_ptr->number_of_function_calls,
 true,
 kill_flag
 );

++recursion_depth;
 /**** mystic width = 4 *****/
(void)strcat( nesting_display_buffer, " " );
nesting_display_buffer[ strlen( nesting_display_buffer ) - 1 ] = wall;

max_count = record_ptr->number_of_function_calls;
for( loop_counter = 0, f_list_ptr = record_ptr->ptr_to_function_table;
 loop_counter < max_count;
 ++loop_counter, ++f_list_ptr
 )
 {
 kill_flag = ( loop_counter == ( max_count - 1 ) )? true: false;
 check_for_new_page();
 /* is called function defined? */
 found = binary_search_sorted_data_base( f_list_ptr->functions_name );
 if( found >= 0 )
 {
 scan_for_static_or_global( &found,
 f_list_ptr->static_function,
 f_list_ptr->functions_name,
 f_list_ptr->its_filename
 );

 }
 if( found >= 0 ) /* yes */
 {
 test_filename = f_list_ptr->its_filename;
 if( recursion_check( f_list_ptr->functions_name,
 f_list_ptr->static_function )
 )
 {
/* tab_to_left_margin( output );
/* (void)fprintf( output, "%s\n", nesting_display_buffer ); */
 setpage( array_of_ptrs_to_records[ found ] );
/* ++line; */

 top_of_form_done = false;
 draw_output_block( nesting_display_buffer,
 f_list_ptr->functions_name,
 "(recursive)",
 "",
 0,
 false,
 kill_flag
 );
 }
 else /* not recursive and found >= 0 */
 {
 if( array_of_ptrs_to_records[ found ]->number_of_references == 1 )
 { /* got a new function */
/* tab_to_left_margin( output );
/* (void)fprintf( output, "%s\n", nesting_display_buffer );
/* ++line;
/* top_of_form_done = false; */
 doprint( found ); /* used only once */
 }
 else
 { /* a previously defined function */
/* tab_to_left_margin( output );
/* (void)fprintf( output, "%s\n", nesting_display_buffer ); */
 setpage( array_of_ptrs_to_records[ found ] );
/* ++line;
/* top_of_form_done = false; */
 draw_output_block( nesting_display_buffer,
 f_list_ptr->functions_name,
 "(defined)",
 usage_string,
 f_list_ptr->is_referenced,
 false,
 kill_flag
 );
 }
 }
 }
 else /* found = -1 ie not defined means */
 { /* a library function */
/* tab_to_left_margin( output );
/* (void)fprintf( output, "%s\n", nesting_display_buffer );
/* ++line;
/* top_of_form_done = false; */
 draw_output_block( nesting_display_buffer,
 f_list_ptr->functions_name,
 library_string,
 usage_string,
 f_list_ptr->is_referenced,
 false,
 kill_flag
 );
 }
 } /* end of loop on all called functions */

 /* remove function f/ recursion list */
recursion_array[ recursion_depth ] = NULL;
 /**** mystic width = 4 *****/
nesting_display_buffer[ strlen( nesting_display_buffer ) - 4 ] = '\0';

--recursion_depth;
return return_value;
}
/***************************************************************************/
void near scan_for_static_or_global(
 index_ptr, is_static, function_name, file_name
 )
int *index_ptr, is_static;
char *function_name, *file_name;
{
int index;

index = *index_ptr;
if( index )
 while( index-- )
 if( strcmp( function_name,
 array_of_ptrs_to_records[ index ]->defined_function )
 )
 {
 ++index; /* exit at last matching defined function */
 break;
 }
do {
 if(
 ( !is_static && !array_of_ptrs_to_records[ index ]->static_definition
 ) 
 ( is_static &&
 array_of_ptrs_to_records[ index ]->static_definition &&
 !strcmp( array_of_ptrs_to_records[ index ]->
 file_record_ptr->source_filename,
 file_name
 )
 )
 )
 break;
 }
while(
 ( ++index < count_of_functions ) &&
 !strcmp( function_name,
 array_of_ptrs_to_records[ index ]->defined_function
 )
 );
if(
 ( index >= count_of_functions ) 
 strcmp( function_name, array_of_ptrs_to_records[ index ]->defined_function
 )
 )
 index = -1;
*index_ptr = index;
}
/***************************************************************************/
int near binary_search_sorted_data_base( key )
char *key;
{
int lo, hi, index;
int doesnt_match;

lo = 0;
hi = count_of_valid_records - 1;

index = ( hi - lo ) / 2;

while( true )
 {
 doesnt_match =
 strcmp( key, array_of_ptrs_to_records[ index ]->defined_function );
 if( !doesnt_match ) /* a match found at index */
 break;
 if( lo >= hi ) /* no match found */
 {
 index = -1;
 break;
 }
 if( doesnt_match < 0 ) /* key < choice so go downwards */
 hi = index - 1;
 else /* key > choice so go upwards */
 lo = index + 1;
 index = ( hi + lo ) / 2; /* new choice */
 }
return index;
}
/***************************************************************************/






[LISTING SIX]

cp.obj : cp.c cpheader.h cp
 cl -AL -c cp.c

cpinput.obj : cpinput.c cpheader.h cp
 cl -AL -c cpinput.c

cpfuncts.obj : cpfuncts.c cpheader.h cp
 cl -AL -c cpfuncts.c

cpbuild.obj: cpbuild.c cpheader.h cp
 cl -AL cpbuild.c -c

cp.exe : cp.obj cpinput.obj cpfuncts.obj cpbuild.obj cp
 link cp+ cpinput+ cpbuild+ cpfuncts/packcode/st:16000,,cp;







[LISTING SEVEN]

cpheader.h
cp.c
cpbuild.c
cpfuncts.c
cpinput.c




===============================================================

_C PRINTER FOR VMS AND UNIX_
by Kevin E. Poole

SIDEBAR TO _AUTOMATIC MODULE CONTROL REVISITED_ BY RON WINTER



[LISTING ONE]

!
! VAX/VMS MMS Description File
!
! DEFINITIONS:
H = cpheader.h
!
! CP
!
cp DEPENDS_ON cp.obj cpbuild.obj cpfuncts.obj cpinput.obj
 LINK/EXEC=cp cp.obj,cpbuild.obj,cpfuncts.obj,cpinput.obj
 PURGE *.obj, *.exe

cp.obj DEPENDS_ON $(H) cp.c
 cc cp.c

cpbuild.obj DEPENDS_ON $(H) cpbuild.c
 cc cpbuild.c

cpfuncts.obj DEPENDS_ON $(H) cpfuncts.c
 cc cpfuncts.c

cpinput.obj DEPENDS_ON $(H) cpinput.c
 cc cpinput.c






[LISTING TWO]

#
# VAX/Unix Makefile
#
# DEFINITIONS:
#
H = cpheader.h
#
# CP
#
cp : cp.o cpbuild.o cpfuncts.o cpinput.o
 cc -o cp cp.o cpbuild.o cpfuncts.o cpinput.o

cp.o : $H cp.c
 cc -c cp.c


cpbuild.o : $H cpbuild.c
 cc -c cpbuild.c

cpfuncts.o : $H cpfuncts.c
 cc -c cpfuncts.c

cpinput.o : $H cpinput.c
 cc -c cpinput.c






[LISTING THREE]

#define MSDOS 0 /* Set the appropriate constant */
#define VMS 0 /* to one (1) before compiling. */
#define UNIX 1 /* All others are zero (0) */






[LISTING FOUR]

#if VMS
extern int binary_search_sorted_data_base( char * );
extern void build_box_parts( int );
extern int build_the_data_base( char * );
extern void check_for_new_page( void );
extern int doprint( int );
extern void nasty( int );
extern void process_arguments( int, int, char **, int );
extern void scan_for_static_or_global( int *, int, char *, char * );
extern void tab_to_left_margin( FILE * );

static void allocate_arrays( void );
static void build_records_from_list( FILE * );
static void bump_line_count( void );
static void count_all_defined_references( void );
static void deallocate_arrays( void );
static void do_top_of_page( void );
static void initialize_globals( void );
static void show_files_leading_comments( void );
static void show_function_relationships( void );
static void show_library_functions( void );
static void show_line_and_byte_counts( void );
static void show_page_references( void );
static void show_sorted_function_list( void );
static void show_unused_if_any( void );
static void sort_the_data_base_array( void );
static char* strdup( char * );
static void timedate( char * );
 int main( int, char ** );
#endif

#if MSDOS

extern int near binary_search_sorted_data_base( char * );
extern void near build_box_parts( int );
extern int near build_the_data_base( char * );
extern void near check_for_new_page( void );
extern int near doprint( int );
extern void near nasty( int );
extern void near process_arguments( int, int, char **, int );
extern void near scan_for_static_or_global( int *, int, char *, char * );
extern void near tab_to_left_margin( FILE * );

static void near allocate_arrays( void );
static void near build_records_from_list( FILE * );
static void near bump_line_count( void );
static void near count_all_defined_references( void );
static void near deallocate_arrays( void );
static void near do_top_of_page( void );
static void near initialize_globals( void );
static void near show_files_leading_comments( void );
static void near show_function_relationships( void );
static void near show_library_functions( void );
static void near show_line_and_byte_counts( void );
static void near show_page_references( void );
static void near show_sorted_function_list( void );
static void near show_unused_if_any( void );
static void near sort_the_data_base_array( void );
static char* near strdup( char * );
static void near timedate( char * );
 int near main( int, char ** );
#endif







[LISTING FIVE]

#if MSDOS
static void near bump_line_count( )
#else
static void bump_line_count( )
#endif






[LISTING SIX]

#if MSDOS
#include <malloc.h>
#include <conio.h>
#include <stdlib.h>
#endif

#include <ctype.h> /* this is for the 'tolower' function */
#include <stdio.h>

#include <string.h>







[LISTING SEVEN]

#if MSDOS
#include "time.h"
#else
#include <time.h>
#endif






[LISTING EIGHT]

#if !MSDOS
char *strdup(orig)
char *orig;
{
 char *ptr;

 ptr = (char *) malloc( (strlen(orig) * sizeof(char)) + 1);

 if(ptr != NULL)
 {
 strcpy(ptr,orig);
 }

 return(ptr);
}
#endif






[LISTING NINE]

 timedate(title);






[LISTING TEN]

/************************************************************/
#if !MSDOS
static void timedate(ret_time)

char *ret_time;
{
 struct tm *time_structure;
 int time_val, i;
 static char *hour[2] = {"am","pm"};
 char temp[19];

 time(&time_val);
 time_structure = localtime(&time_val);

 i = 0;
 if((time_structure->tm_hour >= 12)&&(time_structure->tm_hour<24)) i=1;

 if(time_structure->tm_hour > 12)
 time_structure->tm_hour = (time_structure->tm_hour)-12;
 sprintf(temp,"%d/%d/%d %d:%02d %s",
 time_structure->tm_mon,
 time_structure->tm_mday,
 time_structure->tm_year,
 time_structure->tm_hour,
 time_structure->tm_min,
 hour[i]);
 i=0;
 while(temp[i]!='\0')
 {
 ret_time[i] = temp[i];
 i++;
 }
}
#endif

#if MSDOS
static void near timedate(ret_time)
char *ret_time;
{
 char *cp;
 int i;

 cp = &ret_time[ 0 ]; /* insert date and nice time into ret_time */
 (void)_strdate( cp );
 ret_time[ 8 ] = ' ';
 cp = &ret_time[ 10 ];
 (void)_strtime( cp );

 ret_time[ 15 ] = ' '; /* knock off seconds */
 ret_time[ 16 ] = ' '; /* put am, pm here */
 ret_time[ 17 ] = 'm';
 ret_time[ 18 ] = ' ';

 i = atoi( &ret_time[ 10 ] ); /* f/ military to civilian time */
 ret_time[ 16 ] = ( i < 12 )? (char)'a': (char)'p';

 if( i == 0 )
 i = 12;
 if( i >= 13 )
 i -= 12;

 (void)sprintf( &ret_time[ 10 ], "%2d", i );
 ret_time[ 12 ] = ':';


 if( ret_time[ 10 ] == '0' )
 ret_time[ 10 ] = ' ';
}
#endif

/************************************************************/






[LISTING ELEVEN]

 if(islower((int)argv[i][1]))
 c = argv[i][1];
 else
 c = (char)tolower( (int)argv[ i ][ 1 ] );






[LISTING TWELVE]

 if( strcmp(argv[2],"con") == 0)
 output = stderr;
 else
 output = fopen( argv[ 2 ], "w+" ); /******* wt+ <<<<<<<< ******/







[LISTING THIRTEEN]

 (void)printf( "\n can't open output file.\n");






[LISTING FOURTEEN]

if( (length > 65535) && (MSDOS) )






[LISTING FIFTEEN]

$ CC/PREPROCESS_ONLY CP.C

$ CC/PREPROCESS_ONLY CPBUILD.C
$ CC/PREPROCESS_ONLY CPINPUT.C
$ CC/PREPROCESS_ONLY CPFUNCTS.C
$ CC/PREPROCESS_ONLY CPHEADER.H






[LISTING SIXTEEN]

cc -E cp.c >cp.i
cc -E cpbuild.c >cpbuild.i
cc -E cpinput.c >cpinput.i
cc -E cpfuncts.c >cpfuncts.i
cc -E cpheader.h >cpheader.i






[LISTING SEVENTEEN]

cp.i
cpinput.i
cpbuild.i
cpfuncts.i
cpheader.h







[LISTING EIGHTEEN]

#define LEN_INFILE 256








[LISTING NINETEEN]

char input_list_filename[ LEN_INFILE ], input_line[ LEN_INFILE ];
char overlay_number[ LEN_INFILE ];






[LISTING TWENTY]


fgets( input_line, LEN_INFILE-1, stream ); /* ends at \n or eof */






[LISTING TWENTY-ONE]

Replaces line 20 of Listing Five

 *top_line_of_box, *bottom_line_of_box,






[LISTING TWENTY-TWO]

Replaces lines 73 thru 75 of Listing Five

if( !( top_line_of_box =(char *)malloc( defined_box_width * sizeof(char) ))
 )
{
 (void)fprintf( stderr, "Ran out of memory for top line of box.\n" );
 exit( 1 );
}

if( !( bottom_line_of_box =(char *)malloc( defined_box_width * sizeof(char) ))
 )
{
 (void)fprintf( stderr, "Ran out of memory for bottom line of box.\n" );
 exit( 1 );
}

top_line_of_box[ 0 ] = upper_left_corner;
bottom_line_of_box[ 0 ] = lower_left_corner;
for( i = 1; i <= (defined_box_width - 3); ++i )






[LISTING TWENTY-THREE]

Replaces lines 215 and 216 of Listing Five

 name_of_file,
 description,








[LISTING TWENTY-FOUR]

Replaces lines 228 thru 274 of Listing Five

unsigned int string_length;
int x;
static char alternate_lead_in[ 140 ];

/******* 1st line *****************************************************/
tab_to_left_margin( output );
(void)fprintf( output, "%s %s\n", lead_in_string, top_line_of_box );

/******* 2nd line ******************************************************/
tab_to_left_margin( output );
string_length = strlen( lead_in_string );
if( string_length ) /******* ie not main or defined function box ***/
 {
 (void)strncpy( alternate_lead_in, lead_in_string, --string_length );
 alternate_lead_in[ string_length++ ] = '\0'; /*restore string_length*/
 }
if( string_length ) /******* ie not main or defined function box ***/
 {
 if( g_ov_flag && ov_num )
 {
 (void)fprintf( output, "%s%c%c%c%s %3d",
 alternate_lead_in,
 /*** if( kill_flag ) /****** last line to this box ******************/
 /*** else /****** line continues downwards ***************/
 ( kill_flag )? lower_left_corner: right_attach,
 ibm_line, left_attach, name_of_function, ov_num);
 for(x=strlen(name_of_function);x < defined_box_width-7;x++)
 putc(' ',output);
 putc(wall,output);
 putc('\n',output);
 }
 else
 {
 (void)fprintf( output, "%s%c%c%c%s ",
 alternate_lead_in,
 /*** if( kill_flag ) /****** last line to this box ******************/
 /*** else /****** line continues downwards ***************/
 ( kill_flag )? lower_left_corner: right_attach,
 ibm_line, left_attach, name_of_function);
 for(x=strlen(name_of_function);x < defined_box_width-7;x++)
 putc(' ',output);
 putc(wall,output);
 putc('\n',output);
 }
 }
else /****** main or defined box starting ***********/
 {
 if( g_ov_flag && ov_num )
 {
 (void)fprintf( output, "%c%c%s %3d",
 ibm_line, left_attach, name_of_function, ov_num);
 for(x=strlen(name_of_function);x < defined_box_width-7;x++)
 putc(' ',output);
 putc(wall,output);
 putc('\n',output);

 }
 else
 {
 (void)fprintf( output, "%c%c%s ",
 ibm_line, left_attach, name_of_function);
 for(x=strlen(name_of_function);x < defined_box_width-7;x++)
 putc(' ',output);
 putc(wall,output);
 putc('\n',output);
 }
 }
/******* 3rd line *****************************************************/
tab_to_left_margin( output );
if( string_length-- ) /** kill outside vertical line on last box **/
 lead_in_string[ string_length++ ] = ( kill_flag )? (char)' ': wall;
(void)fprintf( output, "%s %c%s %8s%3d",
 lead_in_string, wall, name_of_file, description, either_count);
 for(x=strlen(name_of_file);x < defined_box_width-17;x++) putc(' ',output);
 putc(wall,output);
 putc('\n',output);






[LISTING TWENTY-FIVE]

Insert between 109 and 110 of Listing One

 defined_box_width = 40,






[LISTING TWENTY-SIX]

Insert between 160 and 161 of Listing One

extern int defined_box_width;






[LISTING TWENTY-SEVEN]

Replaces lines 20 through 25 of Listing Three

 " /p:nn /w:nn /m:nn /r:nn /t:main /f:nnnn /b:nn\n"
 );
 (void)printf(
 " /l /n /s /q /d /o /u /c /h /x\n"
 );
 (void)printf( " ]\n" );








[LISTING TWENTY-EIGHT]

Insert between 45 and 46 of Listing Three

 (void)printf(
 " b: width of func. box = %2d [ 20 - 255 ]\n", defined_box_width
 );






[LISTING TWENTY-NINE]

Insert between 155 and 156 of Listing Three


 case 'b':
 if( ( 20 < tmp ) && ( tmp < 255 ) )
 defined_box_width = tmp;
 break;


































Special Issue, 1989
C LIST MANAGER


Lisp-like lists in C




Robert F. Starr


Bob is a software engineer for Halliburton Geophysical in Houston, Texas. This
project was completed while still employed at Positron Corp. Bob can be
reached at 2639 Valley Field Dr., Sugarland, TX 77479.


List management buys you a convenient way to create randomly accessible linked
lists with low-storage overhead. The ability to store objects of any type
generalizes the problem of list management so that each new problem doesn't
find you writing virtually the same code over again (to handle a problem only
moderately different from the previous one).
A list is defined as a collection of information of the same type, accessible
in either sequential or random order. The type of information on the list can
be anything, but within a given list, the type must be consistent (for
example, all int, float, struct, pointers, and so on).
Generally, lists are represented as vectors of pointers to data. Most popular
is probably the linked list, where each element of the list contains a pointer
to the next element (a regular linked list), and possibly a pointer to the
previous element (a doubly linked list).
In a language such as C, which provides for dynamic memory allocation,
programmers generally opt for the linked list approach over pointer lists or
arrays of fixed size. This allows the list to consume a minimum of memory to
start with, and presents no artificial upper limit on the length of the list.
Traditionally, every new problem carries with it the burden of defining new
data structures and building the linked list management code from scratch.
This process is time-consuming, not to mention error-prone. The basic tasks
required to implement a linked list are:
1. Describe data structure for the linked list.
2. Write code to add an element to the list. This involves building a new data
structure, populating it, and linking it onto the list.
3. Free all memory consumed in the linked list.
Wouldn't it be nice if linked lists could somehow be generalized, so that a
canned package would easily handle current and future needs? That's what I'm
presenting in this article -- a general list management system written in C
that can hold any type of objects, even other lists. The coding was done under
DOS 3.3 in Microsoft C 5.1, and the program has been compiled and run with no
modifications on Unix System V, VMS 4.7, and a Sun 4/280. The source code for
the program is presented in Listing One, page 90, while the header file is
listed in Listing Two, page 91.
One thing many programmers have discovered is that although the general memory
allocation routines offered with C (malloc, calloc, and so on) are very nice,
they can be terribly inefficient for many problems. This is especially true
when the memory allocation is performed in several small chunks (as is often
the case when building linked lists): Each call to malloc takes time (to find
the requested amount of memory) and actually consumes more memory than you
requested (for bookkeeping overhead). For large projects, this can get so bad
that overall performance of the application suffers greatly.
I have seen several programs gain substantial speed by adding a higher-level
memory allocation routine. If the programmer finds himself constantly calling
malloc to get chunks of memory of the same size, he can call this higher-level
function to get memory from the memory allocator in larger chunks than needed.
The higher-level allocator then doles out the space in smaller chunks as it is
requested.
This limits the fragmentation of memory, and adds to the efficiency of the
code. But, as with linked list implementations, the code to handle this must
be written especially for each new application.
The list management system described here gives you this kind of higher-level
memory allocator. To test the efficiency of the program, I took code that made
many calls to malloc and replaced it with this list management system. The
performance of the package increased by a factor of eight. You will realize
better performance than this in some cases, worse in others.
The code generalizes the list management problem in such a way that problems
of this kind are easily handled. If building and managing lists is very easy
to do, you will find yourself using them in ways you never dreamed possible.
You will not resist building a list as an elegant solution to a problem merely
to avoid the coding complexity, or due to project time constraints.
I will first discuss the list management package from the subroutine level and
then show you how it can be used. There will be enough short examples to serve
as an easy go-by for virtually any project you can imagine. The examples will
demonstrate how to make and maintain lists of numbers, strings, structures,
lists, and functions.


The Program Interface


A whole suite of subroutines makes up this list management package. They are
described in fairly general terms here.
As much as I tried to make the list management system problem independent,
there is one small facet that must be considered each time based upon the
problem at hand.
As mentioned earlier, calling malloc to get several small chunks of memory can
be very wasteful. Because avoiding memory fragmentation was one of the design
goals for this project, two items of information are required to set up a
list:
1. The size of each entry to be placed on the list.
2. The number of entries to grab at a time with each internal call to malloc.
The choice of the number of entries is generally problem dependent, and it is
the toughest choice you will have to make. Note that the number of entries
presents no upper bound on the size of the list. It is merely used for the
efficient allocation of memory.
A list is created using makelist. The return value is a void pointer, which is
passed on to other routines in the package. Consider the return value from
makelist like a FILE pointer. You needn't be concerned with what it points to.
After a list has been created, data is put onto the list using appendlist.
List data always gets added to the end of the list in this implementation.
appendlist requires two arguments: The list pointer to which the data is being
appended, and a pointer to the data to be put on the list. Because appendlist
expects a pointer to the data to be put on the list, and because it knows the
size of the data item (you provided that in the call to makelist), it performs
a copy of the pointed-to data into the internal data space it has reserved.
It is important to remember that the pointed-to data is copied. If you are
putting integers onto the list, the integers themselves will be copied. There
is no reason for the data being put on the list to be static, in this case. If
you are appending pointers to strings, or pointers to structures, these
pointers must point to malloced or static data space, if you expect to be able
to recover the data from the list. At the very least, the data must remain
static throughout the life of the list.
You might think that having to have malloced data space to put on a list is
wasteful, as I claim that this package makes memory allocation more efficient.
In reality, the package is a list management tool that manages memory for
lists much more efficiently than you could do with malloc. A clever user could
use the same tool for getting data space for his string storage (if that is
what the list must contain).
Getting data off the list is easy. You have two choices: fetchlist allows you
to get data items off the list in random order (as though the list were an
array). fetchlist provides array-bound checking, and NULL is returned if your
indices go outside the limits of the list being accessed.
walklist allows you to walk down the list, from beginning to end, without
regard for list indices. walklist expects you to traverse the entire list. If
you happen to find what you are looking for before walklist is done, you can
use rewindlist to reset internal pointers so that walklist will start at the
beginning on the next call.
On the first call, walklist always starts reading at the beginning of the
list. It will return subsequent entries in the list until the end of the list
is seen or a call is made to rewindlist. Calling fetchlist does not interfere
with walklist calls. When walklist gets to the end of the list, NULL is
returned.
What walklist and fetchlist return are void pointers to the internal data
space containing your data. To actually get at the data can be a little
tricky. For this reason, I have provided examples of storing and retrieving
many different kinds of things on lists. The easiest thing to remember is
this: What you get back is a pointer to a data area where your data resides.
You must first cast this pointer to be what you expect and, in general,
dereference the result.
If you stored integers, for example, the pointer returned would be a pointer
to an integer. Cast it as such, then dereference to get the integer value.
This is demonstrated in Example 1.
Example 1: A list of integers

 #ifdef Explanation
 -------------------------------------------------------------------------
 Make a list to hold integers, and put 100 integers onto the list. Then, play
back the list.
 -------------------------------------------------------------------------

 #endif


 #include <stdio.h>
 #include "makelist.h"

 main () {
 void *list,*dp;
 int ii;
 /* make a list to hold integers */
 list = makelist (sizeof (int),10);
 /* put 100 integers on list */
 for (ii =0 ; ii < 100 ; ii ++)
 appendlist (list,&ii);
 /* use fetchlist to read back list */
 for (ii=0 ; ii < 100 ; ii++) {
 void *dp = fetchlist(list,ii);
 printf("Entry %2d = %d\n",ii,*(int *)dp);
 }
 freelist(list);
 }

If you stored a pointer to a string, the returned value is a pointer to a
pointer to a string. This is demonstrated in Example 2. If you wish to store
data structures on a list, what you get back will be a pointer to the saved
data structure. All you need to do is cast the returned pointer -- you needn't
dereference it. That is because the data pointer returned holds the actual
data structure. This is demonstrated in Example 3.
Example 2: A list of strings

 #ifdef Explanation
---------------------------------------------------------------------------
 Demonstrates the putting of strings on a list and use of gclist.
---------------------------------------------------------------------------
 #endif

 #include <stdio.h>
 #include "makelist.h"

 extern char *strdup();

 main() {
 char buffer[132];
 void *list,*dp,*ptr;
 int ii;
 int cnt;

 /* make list to hold strings */
 list = makelist (sizeof (char *), 10);
 /* build the list of strings */
 for (ii=0 ; ii < 30 ; ii++) {
 sprintf (buffer, "text string %d",ii);
 dp = strdup (buffer);
 appendlist(list,&dp);

 }
 /* use waklist to view each string saved in list */
 while (dp = walklist(list))
 printf("%s\n*,* ((char **)dp));
 /* free up list, as well as all malloced data */
 gclist(list);
 }

Example 3: A list of structures

 #ifdef Explanation

 --------------------------------------------------------------------
 Saving and retrieving structures on lists.
 --------------------------------------------------------------------
 #endif

 #include <stdio.h>
 #include "makelist.h"
 extern char *strdup();
 /* here is our data structure */
 struct foo {
 char *name;
 int ndx;

 } foo;

 main() {
 void *list;
 void *vp;
 struct foo *fp;
 int ii,jj,kk;
 char buffer[256];
 /* make list to hold struct foo */
 list = makelist (sizeof (struct foo), 10);
 /* build list of 30 instances of struct foo */
 for (ii=0 ; ii < 30 ; ii++) {
 sprintf (buffer, "foo element %d", ii);
 foo. name = strdup (buffer);
 foo.ndx = ii;
 appendlist (list,&foo);
 }
 /* play back list. note that vp is merely cast */
 while (vp=walklist (list)) {
 fp = (struct foo *)vp;
 printf("%2d) %s\n", fp->ndx, fp->name);
 }
 /* free up each string malloced during list creation */
 while (vp=walklist (list)) {
 fp = (struct foo *)vp;
 free(fp->name);
 }
 /* free the list itself */
 freelist (list);
 }

To free a list (return all memory used by the list to free store), call
freelist. freelist makes no assumptions about the data you have saved on the
list. If what you have saved is pointers to malloced data space, when the list
goes away, so do your pointers to the malloced space. In this case, merely use
walklist to traverse the list, freeing each malloced pointer as you go.
malloclist is a function that performs a traditional malloc, with the malloced
data address automatically added onto a list. By utilizing this function, you
can easily free up all memory malloced during a particular function by calling
gclist. This is different from freelist in that all data in the list is
assumed to be malloced, and it is freed before the list itself is freed.
Example 4 shows how to use malloclist.
Example 4: Using malloclist ( )

 #ifdef Explanation
 ------------------------------------------------------------------------
 We use malloclist to get memory space, rather than malloc. This will
 automatically append malloced data onto a list (which you must provide),
 so that a single call to gclist will free not only the list, but all
 malloced space as well.
 ------------------------------------------------------------------------
 #endif

 #include <stdio.h>
 #include "makelist.h"


 main() {
 void *list, *dp, *ptr;

 printf ("Exercising malloclist...\n");
 list = makelist (sizeof (void *),10);

 ptr = "string #1";
 dp = malloclist (list,strlen(ptr)+1);
 strcpy(dp,ptr);
 ptr = "string #2";
 dp = malloclist(list,strlen(ptr)+1);
 strcpy(dp,ptr);
 ptr = "string #3";
 dp = malloclist (list,strlen(ptr)+1);
 strcpy(dp,ptr);
 /* play back list */
 while (dp = walklist (list))
 printf("%s\n",*(char **)dp);
 /* free up the list */
 gclist(list);
 }

pushlist and poplist allow you to easily use a list as a stack. toplist allows
you to examine the topmost stack item (the last entry pushed onto the stack).
NULL is returned if the list is empty. Using these functions is demonstrated
in Example 5.
Example 5: Pushing and popping a list

 #ifdef Explanation
 --------------------------------------------------------------------
 Demonstrate how to push & pop data off of a list.
 --------------------------------------------------------------------
 #endif

 #include <stdio.h>
 #include "makelist.h"

 main() {
 void *list, *dp, *ptr;
 int ii,jj,kk;
 int cnt;

 /* build list to hold integers */
 list = makelist (sizeof(int),5);
 /* push 10 integers onto list */
 for (ii=0 ; ii < 10 ; ii++)
 pushlist(list,&ii);
 /* walk down list 10 times, popping an entry each time */
 for (ii=0 ; ii < 10 ; ii++) {
 extern void *toplist ();
 while (dp = walklist(list))
 printf("%d\n",*(int *)dp);
 dp = toplist(list);
 printf("popping off %d\n",*(int *)dp);
 poplist(list);
 }
 /* list is now empty */
 printf("pushing onto list again\n");
 for (ii=0 ; ii < 10 ; ii++)
 pushlist(list,&ii);
 /* verify entire list */

 while (dp = walklist(list))
 printf("%d\n",*(int *)dp);
 /* pop off an item at a time off list */
 for (ii=0 ; ii < 10 ; ii++) {
 extern void *toplist();
 dp = toplist(list);
 printf("popping off %d\n",*(int *)dp);
 poplist(list);
 }
 /* ensure list is indeed empty */
 printf("walking list again... should be empty\n");
 while (dp = walklist(list))
 printf("%d\n",*(int *)dp);
 freelist(list);
 }

As an experiment, I added Lisp-like property manipulation to the stack. (Lisp
allows you to associate a property with any data value. In Lisp, a property
has a textual name and a value, which can be anything.)
In my implementation, a property is like an environment variable in DOS or
Unix, where there is a name and an associated value (both of which are
strings). For property lists, each property is associated with a single data
item, and different data items may have the same property name with different
values. There is no conflict.
When data is put onto a list (using appendlist, for example), a pointer is
returned. This pointer is a pointer to the actual storage location for the
data. This pointer is used to associate a property with a data item (for
either saving or retrieving property information). Many properties may be
associated with a single data item. The property value may be passed as NULL.
Effectively, this just hangs a string name onto the specified data item.
To append data onto the list and associate property information with it
simultaneously, use the function pappendlist. In its most common usage, this
function allows you to hang names on data values as they are put on the list,
which can be a very powerful feature.
To find a property on a list, use findprop. This function will find the first
occurrence of a property with the specified name, and return a pointer to the
data to which that property has been associated.
Property list handling is not done as efficiently as it could be, but it will
do for small applications. Basic property list usage is demonstrated in
Example 6 . Property list usage using pappendlist is shown in Example 7.
Example 6: Property list usage

 #ifdef Explanation
 ----------------------------------------------------------------------
 Put 100 integers on a list. Every 5th element on the list, add a special
property value.
 ----------------------------------------------------------------------
 #endif

 #include <stdio.h>
 #include "makelist.h"

 extern char *strdup();

 main() {
 void *list,*dp,*ptr;
 int jj;
 int cnt;
 void *nl = makelist(sizeof(int),20);

 for (jj=0 ; jj < 100 ; jj++) {
 void *dp = appendlist(nl,&jj);
 /* every 5th element, put something on property list */
 if (!(jj%5)) {
 char buffer[80];
 sprintf(buffer,"list[%d]",jj);
 putproplist(nl,dp,"MSG",strdup(buffer));
 }
 }
 for (jj=0 ; jj < 100 ; jj++) {
 void *dp = fetchlist(nl,jj);
 char *ptr;
 printf("%d\n",*(int *)dp);
 if (!(jj%5)) {
 void *p = (jj) ? dp : (void *)NULL;
 printf("PROP: %s\n",ptr = getproplist
 (nl,p,"MSG"));
 free(ptr);

 }
 }
 freelist(nl);
 }

Example 7: Property list usage w/pappendlist

 #ifdef Explanation
 ----------------------------------------------------------------------
 Make a list to hold integers, and put 100 integers onto the list.
 Then, play back the list. For each integer put on the list, add
 a property value which uniquely identifies the data value.
 ----------------------------------------------------------------------
 #endif

 #include <stdio.h>
 #include "makelist.h"

 extern char *strdup();

 main() {
 void *list, *dp;
 char *ptr,buffer[80];
 int ii;

 list = makelist(sizeof(int),10);

 /* put 100 integers on list, w/ prop to tell their index in ASCII */
 for (ii=0 ; ii < 100 ; ii++) {
 sprintf(buffer,"index %d",ii);
 pappendlist(list,&ii,"OP",strdup(buffer));
 }
 /* get each integer off list, and show its property */
 for (ii=0 ; ii < 100 ; ii++) {
 void *dp = fetchlist(list,ii);
 ptr = getproplist(list,dp,"OP");
 printf("Entry %2d = %d, prop = %s\n",ii,*(int *)dp,ptr);
 free(ptr);
 }
 freelist(list);
 }



Creative List Usage


After reviewing a few of the examples, you should realize that the list
management package is easy to use. The biggest stumbling block is remembering
always to pass pointers for information to be put onto the list, and to be
careful in casting and dereferencing the items returned off the list. Once you
get the hang of this, usage is a snap.
The simplest use for lists is for saving data in malloced data space much more
efficiently than malloc alone can do. Speed-ups of a factor of eight of this
package over malloc are fairly typical.
An interesting use of stacks is to initialize a list of lists. Build a list to
hold your data, and then push the list onto the list of lists to save context.
This is useful in recursive contexts, where a lot of dynamic information must
be saved during recursion. An example of how a list of lists can be built and
manipulated is shown in Example 8.
Example 8: A list of lists

 #ifdef Explanation
 ------------------------------------------------------------------------
 Build and populate a list of lists
 ------------------------------------------------------------------------
 #endif

 #include <stdio.h>

 #include "makelist.h"

 main() {
 void *listlist;
 int ii,jj,kk;

 /* create a list of lists */
 listlist = makelist(sizeof(void *),10);
 /* build 10 lists to hold integers */
 for (ii=0 ; ii < 10 ; ii++) {
 void *vp = makelist(sizeof(int),10);
 appendlist(listlist,&vp);
 }
 /* populate each of the 10 lists */
 for (ii=0 ; ii < 10 ; ii++) {
 void *list = *(void **)fetchlist(listlist,ii);
 for (jj=0 ; jj < 20 ; jj++) {
 kk = ii*100 + jj;
 appendlist(list,&kk);
 }
 }
 /* replay each of the 10 lists */
 for (ii=0 ; ii < 10 ; ii++) {
 void *list = *(void **)fetchlist(listlist,ii);
 for (jj=0 ; jj < 20 ; jj++)
 printf("List %d, data = %d\n",ii,*(int
 *)fetchlist (list,jj));
 }
 /* free all lists */
 for (ii=0 ; ii < 10 ; ii++) {
 void *list = *(void **)fetchlist(listlist,ii);
 freelist(list);
 }
 freelist(listlist);
 }

One utility for property lists is building a list that contains a variety of
different pointers (pointers to different things). This does not violate our
definition of a list: Even though the pointers point to different things, the
list is still merely a list of pointers. You can use the property tag to tell
what type of pointer each list item is within the function that must interpret
the list.


Real-World List Usage


As far as I am concerned, two types of functions are developed during any
programming project. They are:
1. Throw-away routines, written to make your code cleaner and easier to
understand.
2. Routines that are well thought-out and have usefulness beyond the scope of
the current project (here, I invoke the buzzword "reusable").
Functions of the second type will make life easier for you down the road. They
will provide you with a new set of tools for future development, and they have
already been tested and debugged.
The real world can often be quite different, though. Most programmers have
been at the following design crossroad. You have a project that you need to
get working and time is running out. Meeting project deadlines can often
impair your ability to dedicate the time required to generalize functions to
the point that they are useful outside the scope of the current project.
A good programmer can easily tell what functionality lends itself to being
made into a subroutine. He sees and anticipates the need for the special
coding before he actually begins writing it. He will often ponder a few
minutes, deciding what arguments need to be provided with the subroutine, and
what the subroutine can return that will be most useful.
He will try to arrange the calling arguments in a reasonably logical fashion,
so that it will be easy to remember how to call the function (without having
to scrape around for the documentation). He tries to give the function a name
that conveys the functionality, so that the subsequent coding which uses the
subroutine will be easy to use, maintain, read, and understand.
As every experienced programmer knows, one of the best things that can come
out of any programming project is a collection of useful subroutines that can
be used in other projects down the road. But have you provided the generality
necessary to make such a routine truly useful in a lot of different
environments?
In some applications, it is often clear that what you have written will
suffice for all applications one could envision. Standard library routines
like strlen and strcat are good examples. Routines like sprintf are less
clear. There is a reasonably good case to be made supporting the idea that
what sprintf should return is a pointer to the string that it built, rather
than the number of characters written. But it doesn't. You can do little about
standard library routines.
But what about the examples where the task is not so clear-cut? Given the time
constraints for a project, you may not be able to devote the time necessary to
envision all of the possibilities, and code an appropriate solution.
Often, it will occur to you that there are features you'd like to add to your
new function, but time constraints and satisfying the project goals limit your
creativity. You know what has to be done now, and realize the potential of
this routine in future applications.
Often the need arises to add extensibility to a function in a simple,
straightforward manner, with no impact on existing code. This is a tough
problem.
A good example is a menu function, wherein the user is presented a list of
options on the screen. The topmost entry is highlighted. The user moves the
cursor over the item of choice with the arrow keys on his keyboard. When the
highlighted bar is over the appropriate menu item, he hits Enter.
Should the menu routine handle lists that are longer than the box on the
screen? That requires extra code and extra time to write. Would you like to
let the user leap to a menu entry by merely hitting the first character of the
menu item? More code. Allow the user to fill in responses to certain menu
items? Allow him to sort the menu a variety of ways? And on and on. More code.
How can you generalize the implementation so that gobs of code don't have to
be replicated to handle future situations, while at the same time meeting your
goal for getting your project out the door on time? Assume that you realize
that a function you are coding will be expanded or enhanced at a later date.
Merely add a void pointer as the last argument of the function call. In
current usage (until you get the appropriate code written), merely pass a
(void*) NULL as the final argument. You can even put out documentation,
telling people that the final argument is required but is reserved for future
expansion. You needn't tell them anything else at this point.
What could be better than a generic list to add as the final argument? Your
code can always check for NULL as the last argument and take the appropriate
default action. If the pointer is non-NULL, consider it a list, and interpret
is as such.
This allows adding virtually endless expansion opportunities to a function
without having to change a single line of existing code. And because you have
used this list management package, the user isn't burdened with having to
write any complicated code. His job is easier, maintenance headaches are
lessened, and you can release a half-finished project before its time!

This approach is vastly different from the exec functions approach of merely
passing a list of strings as arguments, representing argv to a function, where
the last argument is a NULL pointer. Adding NULL as the final argument is an
easy solution to simple problems. When the variety of information required to
be passed to the function is more than simple strings, the problem becomes
much more difficult.
In the menu example just mentioned, you might want to add arguments at a later
date having to do with the menu-placement coordinates, how long the menu is
allowed to be, the maximum width of the window, the functions to call when
various keys are hit, and a slew of other parameters that can change how the
menu works.
This is where a generic list would prove useful as the last argument of the
subroutine calling sequence. A suggestion would be to create a list of void
pointers (that is, pointers to anything). As an item is put on the list, give
it a special property name with no property value. This tells the function
what the item is. To facilitate this operation, use the pappendlist function.
In your application function which must deal with the list, you can easily
search the list for a data item with a particular property name by calling
findprop.
Another approach to adding extensibility to a function can be gotten by adding
another suite of functions, which I'll call "preparatory functions."
Preparatory functions are used to "set things up" in preparation for a call to
the function you are actually interested in. You write each function to set
certain internal static variables so that when your actual function of
interest is called, the function performs to the user's specifications. This
is not a new trick, although naming the preparatory functions can be rather
ugly.
If preparatory functions are your choice for expandability of a given
function, they can easily be handled by creating a list of functions. In
Example 9, I show a function called proc. I have created an auxiliary function
called f_proc. The application calls f_proc to get a list of the functions
that are applicable to proc. f_proc builds a list, and puts the address of
several functions on the list. It then returns a pointer to this list.
Example 9: A list of functions

 #ifdef Explanation
 -----------------------------------------------------------------------
 Demonstrate how to use build a list of functions and call them
 directly off of the list, with arguments.
 -----------------------------------------------------------------------
 #endif

 #include <stdio.h>
 #include <varargs.h>
 #include "makelist.h"

 /* This function expects integer arguments, zero terminated */
 static
 foo (argmark)
 va_list argmark; {
 int i;
 while (1) {
 i = va_arg (argmark, int);
 if (!i) break;
 printf ("%d\n", i);
 }
}

/* This function expects string arguments, NULL terminated */
static
goo (argmark)
va_list argmark; {
 char *ptr;
 while (1) {
 ptr = va_arg (argmark, char *);
 if (!ptr) break;
 printf ("%s\n", ptr);
 }
}

/* This function expects a string, followed by an integer */
static
poo (argmark)
va_list argmark; {
 char *ptr;
 int i;
 ptr = va_arg (argmark, char *);
 i = va_arg (argmark, int);
 printf (*string = %s, int = %d\n",ptr, i);
}

/* build list for functions internal to proc, and return pointer to list */
static void *flist = NULL;
void *
f_proc() {
 int (*func) ();

 if (!flist) flist = makelist (sizeof (int (*) ()); 10);
 /* put functions on flist, and give 'em names */
 func = foo; pappendlist (flist, &func, "FOO", NULL);
 func = goo; pappendlist (flist, &func, "GOO", NULL);
 func = poo; pappendlist (flist, &func, "POO", NULL);
 return flist;
}

/* out proc function */
void
proc () {
 printf ("In function proc\n");
}

main () {
 void *1_proc;
 /* get list of proc's functions */
 1_proc = f_proc ();
 /* invoke each of the functions with arguments */
 funclist (1_proc, "FOO", 1,2,3,4,0);
 funclist (1_proc, "GOO", "line 1", "line 2", "line 3", NULL);
 funclist (1_proc, "POO", "some text", 666);
 /* finally, call proc */
 proc ();
}

Each of the functions (foo, goo, and poo) declared static inside Example 9.
This is not a requirement; it just makes for a cleaner interface.
Note that each of the three functions expects a variable length argument list,
so varargs is used to get each argument. This is required because the user
interface to these functions is made available indirectly through funclist.
When funclist actually calls the function, it passes a pointer to the stack
location of the argument list. If you aren't familiar with the varargs
interface, the three provided functions should give you sufficient examples of
its proper usage.
In the main routine in Example 9, you will see an example of how the user
would access the extended functionality of the proc function. As seen in
f_proc, the functions have been given a property tag which mnemonically
indicates what the functions are. This makes for a cleaner, easier to
understand interface for the user.
There is one caveat in providing an enhanced interface like this. If the
function proc was initially designed to run without arguments (as in our
example), be sure that it takes the default action as initially documented. If
calling any of the preparatory functions will permanently modify the behavior
of proc, be sure to tell the user about it. But by keeping the behavior
predictable upon default, you will not need to recompile or relink any
existing code to add functionality at a later date.


Summary


General list management is a useful addition to your arsenal of programming
tools. Once you become adept at using the tools in this package, you will find
yourself building and using lists in ways you never thought of. The best way
to understand the package is to review the many examples.

_C LIST MANAGER_
by Robert Starr


[LISTING ONE]

/* makelist- list management package
 RF Starr
 2639 Valley Field Dr.
 SugarLand, TX 77479
*/

#include <stdio.h>
#include <varargs.h>
#ifdef MSDOS
#include <stdlib.h>
#include <malloc.h>
#else
#define void char
extern char *malloc();
#endif


/*#define DEBUG*/

#ifdef DEBUG
#define Debug(x) x
#else
#define Debug(x)
#endif

typedef struct data DATA;
typedef struct list LIST;
typedef struct prop PROP;

struct data {
 void *data; /* space for list->nentries instances of data */
 DATA *next; /* next list->nentries collection of data */
};

struct prop {
 void *dataptr; /* to what data item this property associates */
 void *propval; /* property value to associate with the data */
 void *propsym; /* optional symbol (usually char *) to associate */
 PROP *next;
};

struct list {
 int entrysize; /* size of each data entry in bytes */
 int nentries; /* # entries to grab per malloc call */
 int empty_slots; /* empty slots left in current data block */
 int nitems; /* total items saved in this list */
 int ecount; /* where we are when reading back list */
 int fblock;
 DATA *fdata;
 PROP *prop; /* optional property list for this list */
 DATA *data; /* linked list for the actual data of this list */
 DATA *hidata; /* highest allocated data block (for efficiency) */
};

/* Internal malloc routine */
/*#define MEMCHK*/
#ifdef MEMCHK
static FILE *memfp = NULL;
#endif
static void *
imalloc(size)
int size; {
 void *ptr = malloc(size);
#ifdef MEMCHK
 if (!memfp) memfp = fopen("meminfo","w");
#endif
 if (!ptr) {
 fprintf(stderr,"malloc error: no free memory left.\n");
 fflush(stderr);
 }
#ifdef MEMCHK
 fprintf(memfp,"%x malloc\n",ptr);
 fflush(memfp);
#endif
 return ptr;
}


static void *
ifree(addr)
void *addr; {
 free(addr);
#ifdef MEMCHK
 fprintf(memfp,"%x free\n",addr);
 fflush(memfp);
#endif
}

/* Build, initialize, and return an empty list */
void *
makelist(esize,nentries)
int esize,nentries; {
 LIST *list = imalloc(sizeof(LIST)+sizeof(DATA)+esize*nentries);
 void *dp = imalloc(sizeof(DATA)+esize*nentries);
 if (!list !dp) return NULL;
 list->data = (DATA *)dp;
 list->data->data = (char *)dp + sizeof(DATA);
 list->entrysize = esize;
 list->nentries = nentries;
 list->empty_slots = nentries;
 list->nitems = 0;
 list->ecount = 0;
 list->fblock = 0;
 list->fdata = NULL;
 list->prop = (PROP *)NULL;
 list->hidata = list->data;
 list->data->next = NULL;
 return (void *)list;
}

/* Put items on property list for this data item. Propsym is the
 property symbol, and val is a pointer to a _static_ are where the
 data for this property resides.
*/
putproplist(list,dataptr,propsym,val)
LIST *list;
void *dataptr,*val;
char *propsym; {
 PROP *newprop = (PROP *)imalloc(sizeof(PROP));
 PROP *topprop = list->prop;
 if (!list) return NULL;
 if (!newprop) return;
 newprop->dataptr = dataptr;
 newprop->propsym = propsym;
 newprop->propval = val;
 newprop->next = topprop;
 list->prop = newprop;
}

/* Read an item off of the property list for a particular data
 item. NULL returned if there is none.
*/
void *
getproplist(list,dataptr,propsym)
LIST *list;
void *dataptr,*propsym; {

 PROP *p;
 void *propval = NULL;
 if (!list) return NULL;
 p = list->prop;
 while (p) {
 int fsym = !strcmp(p->propsym,propsym);
 if ((!dataptr && fsym) (p->dataptr == dataptr && fsym)) {
 propval = p->propval;
 break;
 }
 p = p->next;
 }
 return propval;
}

/* Find data item associated with a property name */
void *
findprop(list,propsym)
LIST *list;
char *propsym; {
 PROP *p;
 if (!list) return NULL;
 p = list->prop;
 while (p) {
 if (!strcmp(p->propsym,propsym)) return p->dataptr;
 p = p->next;
 }
 return NULL;
}

/* Append data to the specified list */
static void
whereis(list,ndx,walk,put)
LIST *list;
int *walk,*put; {
 *walk = ndx / list->nentries;
 *put = ndx % list->nentries;
}

/* Remove last entry on the specified list... adjust struct accordingly */
void *
poplist(list)
LIST *list; {
 DATA *org;
 void *dp = NULL;
 unsigned char *data;
 int put;
 if (!list) return NULL;
 if (!list->nitems) return dp;
 org = list->data;
 list->empty_slots++;
 list->nitems--;
 if (list->empty_slots == list->nentries) {
 while (org->next) org = org->next;
 put = list->nitems % list->nentries;
 data = (unsigned char *)org->data;
 dp = (void *)(data+(list->entrysize*put));
 if (org->next) ifree(org->next), org->next = NULL;
 list->hidata = org;

 }
 return dp;
}

/* calculate data pointer for the ndx entry */
static void *
calcdp(list,ndx)
LIST *list;
int ndx; {
 DATA *pdata;
 unsigned char *dp;
 int size = (list) ? list->entrysize :0;
 int max = (list) ? list->nitems : 0;
 int walk,put;
 if (!list ndx >= max) return (void *)NULL;
 whereis(list,ndx,&walk,&put);
 pdata = list->data;
 while (walk--) pdata = pdata->next;
 dp = (unsigned char *)pdata->data;
 return (char *)(dp+(size*put));
}

/* Return pointer to data which is last on the list */
void *
toplist(list)
LIST *list; {
 void *dp = NULL;
 void *fetchlist();
 unsigned char *data;
 int put;
 if (!list !list->nitems) return NULL;
 return calcdp(list,list->nitems-1);
}

/* append a data item onto specified list */
void *
appendlist(list,data)
LIST *list;
void *data; {
 DATA *pdata;
 void *where;
 unsigned char *dp;
 int walk,put,size;
 if (!list) return NULL;
 size = list->entrysize;
 whereis(list,list->nitems,&walk,&put);
 pdata = list->hidata;
 if (!list->empty_slots) {
 void *mem = imalloc(sizeof(DATA)+size*list->nentries);
 if (!mem) return NULL;
 list->hidata = pdata = pdata->next = (DATA *)mem;
 pdata->data = (void *)((char *)mem+sizeof(DATA));
 pdata->next = NULL;
 list->empty_slots = list->nentries;
 }
 dp = (unsigned char *)pdata->data;
 where = (char *)(dp+(put*size));
 memcpy(where,data,size);
 list->empty_slots--;

 list->nitems++;
 return where;
}

/* return index of NEXT list entry */
listindex(list)
LIST *list; {
 return list->nitems;
}

/* append a data item onto specified list with property value information */
void *
pappendlist(list,data,propsym,val)
LIST *list;
void *data,*val;
char *propsym; {
 void *dataptr = appendlist(list,data);
 if (dataptr) putproplist(list,dataptr,propsym,val);
 return dataptr;
}

/* Just like appendlist... new name for compatibility with poplist */
void *
pushlist(list,data)
LIST *list;
void *data; {
 if (!list) return NULL;
 return appendlist(list,data);
}

/* Fetch a specific data item off of list... ndx is 0-based */
void *
fetchlist(list,ndx)
LIST *list;
int ndx; {
 DATA *pdata;
 unsigned char *dp;
 int size = list->entrysize;
 int max = list->nitems;
 int walk,put;
 if (!list) return NULL;
 if (ndx >= max) return (void *)NULL;
 whereis(list,ndx,&walk,&put);
 if (walk == list->fblock)
 list->fdata = pdata = (list->fdata) ? list->fdata : list->data;
 else {
 pdata = list->data;
 list->fblock = walk;
 while (walk--)
 pdata = pdata->next;
 list->fdata = pdata;
 }
 dp = (unsigned char *)pdata->data;
 Debug(printf("fetchlist: getting data from %x\n",dp+(size*put)));
 return (char *)(dp+(size*put));
}

/* reset pointer used by walklist */
rewindlist(list)

LIST *list; {
 if (list) list->ecount = 0;
}

/* walk down the list, returning each data item 'till there ain't no more */
void *
walklist(list)
LIST *list; {
 DATA *pdata;
 unsigned char *dp;
 int size = list->entrysize;
 int max = list->nitems;
 int index = list->ecount;
 int walk,put;
 if (!list) return NULL;
 dp = (unsigned char *)fetchlist(list,index);
 if (!dp)
 list->ecount = 0;
 else
 list->ecount++;
 return (list->ecount) ? (void *)dp : (void *)NULL;
}

/* malloc size bytes, and add address of malloced space to the list */
void *
malloclist(list,size)
LIST *list;
int size; {
 void *dp;
 if (!list) return NULL;
 dp = imalloc(size);
 if (dp) appendlist(list,&dp);
 return dp;
}

/* Free up all malloced data associated with the specified list */
freelist(list)
LIST *list; {
 fl(list,0);
}

/* garbage collect list... assume all data in list is malloced ptrs */
gclist(list)
LIST *list; {
 fl(list,1);
}

/* Free the list up. If freedp != 0, free each data pointer as well */
static
fl(list,freedp)
LIST *list; {
 DATA *pdata,*ppd;
 PROP *p = list->prop;
 if (!list) return;
 pdata = list->data;
 if (freedp) {
 void *dp;
 rewindlist(list);
 while (dp=walklist(list))

 ifree(*(char **)dp);
 }
 /* free all malloced data */
 while (pdata) {
 DATA *next = pdata->next;
 ifree(pdata);
 pdata = next;
 }
 /* free all malloced property info */
 while (p) {
 PROP *n = p->next;
 ifree(p);
 p = n;
 }
 ifree(list);
}

/* given a list of functions, invoke the function with the specified
 property tag with the arguments supplied.
*/

/*funclist(list,prop,args) (actual calling argument list)*/
int
funclist(va_alist)
va_dcl {
 void *list;
 void *prop;
 void *vp;
 int (*func)();
 va_list argmark;
 va_start(argmark);
 list = va_arg(argmark,void *);
 prop = va_arg(argmark,void *);
 vp = findprop(list,prop);
 if (!vp) return 0;
 func = *(int (**)())vp;
 return (*func)(argmark);
}






[LISTING TWO]

/* Header file for makelist */

/* Remove this if you don't have ANSI C compiler */
#define ANSIC

#ifndef ANSIC
#define void char
#endif

/* Usage: e.g. deref(int,x) or deref(char *,x) */
#define deref(type,x) *((type*)(x))

/* Function prototypes/declarations */

/*----------------------------------------------------------------------*/
#ifdef ANSIC
extern void *makelist(int esize,int nentries);
extern int putproplist(void *list,void *dataptr,char *propsym,void *val);
extern void *getproplist(void *list,void *dataptr,void *propsym);
extern void *toplist(void *list);
extern void *poplist(void *list);
extern void *appendlist(void *list,void *data);
extern void *pappendlist(void *list,void *data,char *propsym,char *val);
extern void *pushlist(void *list,void *data);
extern void *fetchlist(void *list,int ndx);
extern void *walklist(void *list);
extern void rewindlist(void *list);
extern void *malloclist(void *list,int size);
extern int freelist(void *list);
extern int gclist(void *list);
extern int funclist(void *,...);
extern int listindex(void *list);
#else
extern void *makelist();
extern int putproplist();
extern void *getproplist();
extern void *toplist();
extern void *poplist();
extern void *appendlist();
extern void *pappendlist();
extern void *pushlist();
extern void *fetchlist();
extern void *walklist();
extern void rewindlist();
extern void *malloclist();
extern int freelist();
extern int gclist();
extern int funclist();
extern int listindex();
#endif
/*----------------------------------------------------------------------*/


Example 1: A list of integers


#ifdef Explanation
----------------------------------------------------------------------
Make a list to hold integers, and put 100 integers onto the list.
Then, play back the list.
----------------------------------------------------------------------
#endif

#include <stdio.h>
#include "makelist.h"

main() {
 void *list,*dp;
 int ii;
 /* make a list to hold integers */
 list = makelist(sizeof(int),10);
 /* put 100 integers on list */
 for (ii=0 ; ii < 100 ; ii++)

 appendlist(list,&ii);
 /* use fetchlist to read back list */
 for (ii=0 ; ii < 100 ; ii++) {
 void *dp = fetchlist(list,ii);
 printf("Entry %2d = %d\n",ii,*(int *)dp);
 }
 freelist(list);
}


Example 2: A list of strings


#ifdef Explanation
----------------------------------------------------------------------
Demonstrates the putting of strings on a list and use of gclist.
----------------------------------------------------------------------
#endif

#include <stdio.h>
#include "makelist.h"

extern char *strdup();

main() {
 char buffer[132];
 void *list,*dp,*ptr;
 int ii;
 int cnt;
 /* make list to hold strings */
 list = makelist(sizeof(char *),10);
 /* build the list of strings */
 for (ii=0 ; ii < 30 ; ii++) {
 sprintf(buffer,"text string %d",ii);
 dp = strdup(buffer);
 appendlist(list,&dp);
 }
 /* use walklist to view each string saved in list */
 while (dp = walklist(list))
 printf("%s\n",*((char **)dp));
 /* free up list, as well as all malloced data */
 gclist(list);
}


 Example 3: A list of structures


#ifdef Explanation
----------------------------------------------------------------------
Saving and retrieving structures on lists.
----------------------------------------------------------------------
#endif

#include <stdio.h>
#include "makelist.h"

extern char *strdup();
/* here is our data structure */

struct foo {
 char *name;
 int ndx;
} foo;

main() {
 void *list;
 void *vp;
 struct foo *fp;
 int ii,jj,kk;
 char buffer[256];
 /* make list to hold struct foo */
 list = makelist(sizeof(struct foo),10);
 /* build list of 30 instances of struct foo */
 for (ii=0 ; ii < 30 ; ii++) {
 sprintf(buffer,"foo element %d",ii);
 foo.name = strdup(buffer);
 foo.ndx = ii;
 appendlist(list,&foo);
 }
 /* play back list. note that vp is merely cast */
 while (vp=walklist(list)) {
 fp = (struct foo *)vp;
 printf("%2d) %s\n",fp->ndx,fp->name);
 }
 /* free up each string malloced during list creation */
 while (vp=walklist(list)) {
 fp = (struct foo *)vp;
 free(fp->name);
 }
 /* free the list itself */
 freelist(list);
}


Example 4: Using malloclist()

#ifdef Explanation
----------------------------------------------------------------------
We use malloclist to get memory space, rather than malloc. This will
automatically append malloced data onto a list (which you must provide),
so that a single call to gclist will free not only the list, but all
malloced space as well.
----------------------------------------------------------------------
#endif

#include <stdio.h>
#include "makelist.h"

main() {
 void *list,*dp,*ptr;

 printf("Exercising malloclist...\n");
 list = makelist(sizeof(void *),10);

 ptr = "string #1";
 dp = malloclist(list,strlen(ptr)+1);
 strcpy(dp,ptr);
 ptr = "string #2";

 dp = malloclist(list,strlen(ptr)+1);
 strcpy(dp,ptr);
 ptr = "string #3";
 dp = malloclist(list,strlen(ptr)+1);
 strcpy(dp,ptr);
 /* play back list */
 while (dp = walklist(list))
 printf("%s\n",*(char **)dp);
 /* free up the list */
 gclist(list);
}


Example 5: Pushing and poping a list

#ifdef Explanation
----------------------------------------------------------------------
Demonstrate how to push & pop data off of a list.
----------------------------------------------------------------------
#endif

#include <stdio.h>
#include "makelist.h"

main() {
 void *list,*dp,*ptr;
 int ii,jj,kk;
 int cnt;

 /* build list to hold integers */
 list = makelist(sizeof(int),5);
 /* push 10 integers onto list */
 for (ii=0 ; ii < 10 ; ii++)
 pushlist(list,&ii);
 /* walk down list 10 times, popping an entry each time */
 for (ii=0 ; ii < 10 ; ii++) {
 extern void *toplist();
 while (dp = walklist(list))
 printf("%d\n",*(int *)dp);
 dp = toplist(list);
 printf("popping off %d\n",*(int *)dp);
 poplist(list);
 }
 /* list is now empty */
 printf("pushing onto list again\n");
 for (ii=0 ; ii < 10 ; ii++)
 pushlist(list,&ii);
 /* veryify entire list */
 while (dp = walklist(list))
 printf("%d\n",*(int *)dp);
 /* pop off an item at a time off list */
 for (ii=0 ; ii < 10 ; ii++) {
 extern void *toplist();
 dp = toplist(list);
 printf("popping off %d\n",*(int *)dp);
 poplist(list);
 }
 /* ensure list is indeed empty */
 printf("walking list again... should be empty\n");

 while (dp = walklist(list))
 printf("%d\n",*(int *)dp);
 freelist(list);
}


Example 6: Property list usage


#ifdef Explanation
----------------------------------------------------------------------
Put 100 integers on a list. Every 5th element on the list, add a
special property value.
----------------------------------------------------------------------
#endif

#include <stdio.h>
#include "makelist.h"

extern char *strdup();

main() {
 void *list,*dp,*ptr;
 int jj;
 int cnt;
 void *nl = makelist(sizeof(int),20);

 for (jj=0 ; jj < 100 ; jj++) {
 void *dp = appendlist(nl,&jj);
 /* every 5th element, put something on property list */
 if (!(jj%5)) {
 char buffer[80];
 sprintf(buffer,"list[%d]",jj);
 putproplist(nl,dp,"MSG",strdup(buffer));
 }
 }
 for (jj=0 ; jj < 100 ; jj++) {
 void *dp = fetchlist(nl,jj);
 char *ptr;
 printf("%d\n",*(int *)dp);
 if (!(jj%5)) {
 void *p = (jj) ? dp : (void *)NULL;
 printf("PROP: %s\n",ptr = getproplist(nl,p,"MSG"));
 free(ptr);
 }
 }
 freelist(nl);
}


Example 7: Property list usage w/pappendlist


#ifdef Explanation
----------------------------------------------------------------------
Make a list to hold integers, and put 100 integers onto the list.
Then, play back the list. For each integer put on the list, add
a property value which uniquely identifies the data value.
----------------------------------------------------------------------

#endif

#include <stdio.h>
#include "makelist.h"

extern char *strdup();

main() {
 void *list,*dp;
 char *ptr,buffer[80];
 int ii;

 list = makelist(sizeof(int),10);

 /* put 100 integers on list, w/ prop to tell their index in ASCII */
 for (ii=0 ; ii < 100 ; ii++) {
 sprintf(buffer,"index %d",ii);
 pappendlist(list,&ii,"OP",strdup(buffer));
 }
 /* get each integer off list, and show its property */
 for (ii=0 ; ii < 100 ; ii++) {
 void *dp = fetchlist(list,ii);
 ptr = getproplist(list,dp,"OP");
 printf("Entry %2d = %d, prop = %s\n",ii,*(int *)dp,ptr);
 free(ptr);
 }
 freelist(list);
}


Example 8: A list of lists


#ifdef Explanation
----------------------------------------------------------------------
Build and populate a list of lists
----------------------------------------------------------------------
#endif

#include <stdio.h>
#include "makelist.h"

main() {
 void *listlist;
 int ii,jj,kk;

 /* create a list of lists */
 listlist = makelist(sizeof(void *),10);
 /* build 10 lists to hold integers */
 for (ii=0 ; ii < 10 ; ii++) {
 void *vp = makelist(sizeof(int),10);
 appendlist(listlist,&vp);
 }
 /* populate each of the 10 lists */
 for (ii=0 ; ii < 10 ; ii++) {
 void *list = *(void **)fetchlist(listlist,ii);
 for (jj=0 ; jj < 20 ; jj++) {
 kk = ii*100 + jj;
 appendlist(list,&kk);

 }
 }
 /* replay each of the 10 lists */
 for (ii=0 ; ii < 10 ; ii++) {
 void *list = *(void **)fetchlist(listlist,ii);
 for (jj=0 ; jj < 20 ; jj++)
 printf("List %d, data = %d\n",ii,*(int *)fetchlist(list,jj));
 }
 /* free all lists */
 for (ii=0 ; ii < 10 ; ii++) {
 void *list = *(void **)fetchlist(listlist,ii);
 freelist(list);
 }
 freelist(listlist);
}


Example 9: A list of functions


#ifdef Explanation
----------------------------------------------------------------------
Demonstrate how to use build a list of functions and call them
directly off of the list, with arguments.
----------------------------------------------------------------------
#endif

#include <stdio.h>
#include <varargs.h>
#include "makelist.h"

/* This function expects integer arguments, zero terminated */
static
foo(argmark)
va_list argmark; {
 int i;
 while (1) {
 i = va_arg(argmark,int);
 if (!i) break;
 printf("%d\n",i);
 }
}

/* This function expects string arguments, NULL terminated */
static
goo(argmark)
va_list argmark; {
 char *ptr;
 while (1) {
 ptr = va_arg(argmark,char *);
 if (!ptr) break;
 printf("%s\n",ptr);
 }
}

/* This function expects a string, followed by an integer */
static
poo(argmark)
va_list argmark; {

 char *ptr;
 int i;
 ptr = va_arg(argmark,char *);
 i = va_arg(argmark,int);
 printf("string = %s, int = %d\n",ptr,i);
}

/* build list for functions internal to proc, and return pointer to list */
static void *flist = NULL;
void *
f_proc() {
 int (*func)();
 if (!flist) flist = makelist(sizeof(int (*)()),10);
 /* put functions on flist, and give 'em names */
 func = foo; pappendlist(flist,&func,"FOO",NULL);
 func = goo; pappendlist(flist,&func,"GOO",NULL);
 func = poo; pappendlist(flist,&func,"POO",NULL);
 return flist;
}

/* our proc function */
void
proc() {
 printf("In function proc\n");
}

main() {
 void *l_proc;
 /* get list of proc's functions */
 l_proc = f_proc();
 /* invoke each of the functions with arguments */
 funclist(l_proc,"FOO",1,2,3,4,0);
 funclist(l_proc,"GOO","line 1", "line 2", "line 3", NULL);
 funclist(l_proc,"POO","some text", 666);
 /* finally, call proc */
 proc();
}

























Special Issue, 1989
DEBUGGING C PROGRAMS


Don't forget assert( ) and the stack




Bob Edgar


Bob is an experienced C programmer from Britain and can be reached through the
DDJ office.


Testing and debugging are probably the most time-consuming and neglected
aspects of the programmer's trade, and we programmers need all the help we can
get to figure out what our programs are really doing, as opposed to what we
think they should be doing.
The well-known assert( ) macro can be extended in a number of ways as a useful
debugging tool. The basic idea is to have a statement that checks for an
illegal condition in the program; the statement can be completely
"deactivated" by defining the NDEBUG macro, presumably in the compiler command
line. A simple version is in Example 1. Or, if a function do_str( ) takes a
single string argument, which must never be NULL, you might use the assertion
shown in Example 2. (Notice that a semicolon is not required: The assert macro
defines a complete statement.) The definition in Example 1 assumes that the C
macro processor will substitute cond in the string argument to printf, with
the condition given as an argument to assert; different implementations of the
macro processor differ on their rules for argument substitution inside
strings, and you should check the documentation for your compiler. ANSI C has
defined new operators (#, ##) for strings in macros, which would be the
preferred solution. The first improvement is to make an arbitrary printf( )
call if a given condition was (or was not) satisfied. The problem is that the
macro processor does not allow macros with a variable number of arguments. The
solution is to use parentheses, so that the entire argument list to printf is
a single argument to the macro, as shown in Example 3. (In this and the
following macro definitions, it is assumed that the macro is defined to be
empty when NDEBUG is turned on.) A typical use of this technique is shown in
Example 4.
Example 1: Defining the NDEBUG macro

 #ifndef NDEBUG
 #define assert(cond) {if (!(cond)) \
 printf("ASSERT cond, file %s line %d\n", \
 __FILE__, __LINE__); exit(1);}
 #else
 #define assert(cond) {;} /* Empty block */
 #endif

Example 2: Using an assertion

 do_str(s)
 char *s;
 {
 assert(s != NULL)
 /* rest of do_str... */

Example 3: Using parentheses so that the entire argument list to printf is a
single argument

 #define assertp(cond, args) {if (!(cond) \
 printf("ASSERT cond, file %s, line %d\n", \
 __FILE__, __LINE__); \
 printf args; exit(1);}

Example 4: A typical use of assert

 assertp(n > 3, ("Surprise! n > 3, is %d\n", n))

In a major, complicated program it is often an advantage when analyzing
problems to have statements that print out critical variables and data
structures when required, but are normally suppressed so that the output from
the program is not cluttered with a lot of irrelevant information. You can
extend the idea of the assert macro to provide a convenient mechanism for
making such output.
I introduce the idea of a "trace level," which determines how much debugging
output is produced. Permissible values might range from 0, meaning that no
debugging output is printed, to 9, the maximum amount of output. A macro that
unconditionally prints when the trace level is high enough is shown in Example
5.
Example 5: A macro that unconditionally prints when the trace level is high
enough

 #define assert1(level, args) {if (tlevel >= (level)) \
 printf("file %s, line %d\n", \
 __FILE__, __LINE__); \
 printf args; exit(1);}

An external integer tlevel should be visible and should contain the trace
level. The macro in Example 6 prints the values of i and j when the value of
tlevel is two or more. A convenient way of setting tlevel is to provide a
command line argument to the executable program, such as Ln where n is a digit
from 0 to 9.
Example 6: A typical use of Example 5


 assert1(2, ("i=%d j=%d\n", i, j))

To avoid having to check for this command-line argument in all programs using
these macros, write a standard version of main( ) that checks for the
argument, performs any other initializations required by your libraries, and
calls the user's main, which has (transparently to the user) been renamed.
Your version of the assert.h header file could include code similar to that
shown in Example 7.
Example 7: An assert.h header file

 #ifndef NDEBUG
 extern short tlevel;
 #define main mymain
 #endif

The standard version of main( ) would be in a library file, which would then
only be included at link time if the user's version of main( ) had been
compiled without NDEBUG. An elegant solution would remove the -L command-line
option from the argument list so that the user program need not check for it.
An example of this can be seen in Listing One, page 93.
A trace level alone is probably too crude for controlling debugging output
from a large program; to improve it, you could identify major program modules
and data structures that might be traced, and provide, in addition to tlevel,
an integer considered as a bit vector, tbits. A module or data structure in
the program is assigned a bit in tbits, if the bit is set, output related to
that module or data is printed. Imagine, for example, a compiler. You might
define the bits as shown in Example 8.
Example 8: Typical bit definitions

 #define TBIT_LEXER 0x0001 /* Lexical analysis functions */
 #define TBIT_PARSER 0x0002 /* Parser */
 #define TBIT_EXPTREE 0x0004 /* Expression tree */
 #define TBIT_CODEGEN 0x0008 /* Code generator */
 #define TBIT_OPTIM 0x0010 /* Optimizer */

You can now define assert-like macros with a new argument to specify the tbits
bits, which must be set for the information to be printed. This will be a
bit-wise OR of one or more of the bits that the user assigns to parts of the
program, as shown in Example 9. A general macro, say assertbl, would include
both bits and level arguments. The -L command line argument could be extended
to the form -Lnxxxx, where n is the trace level and xxxx is four hex digits
specifying the bits to set in tbits.
Example 9: Defining assert-like macros

 #define assertb(bits, args) {if ((bits) & tbits) \
 printf args;}

 which can be used in this way:

 assertb(TBITS_PARSER : TBITS EXPTREE,
 ("Nodes in expression tree = %d\n", nnode))

Putting all these ideas together provides an easy-to-use package for the
programmer to add selective debugging print (or other) statements into code
without any cost in object code in a production version. Because the overhead
of keeping the debugging statements is usually low, you might consider keeping
the statements in a "final" version of a program; a large program is rarely
completely bug free, and it's nice to be able to turn on selected debugging
statements when a user of the program rings and complains about an apparent
error.
A beneficial side effect to using such a method to facilitate testing and
debugging is that it encourages the programmer to think about the legal and
illegal states of the data structures in the program, and to find easily
understood ways of displaying the program state. This discipline in itself
tends to produce tighter, better thought-out code.


Don't Forget the Stack


Consider the following problem. You have a big program that makes many calls
to a low-level function called do_str( ). The program fails, and you suspect
that do_str( ) is called with a NULL pointer, so what do you do? Try adding a
statement similar to that in Example 10 to trap the error.
Example 10: Adding a statement to trap an error

 do_str(s)
 char *s;
 {
 if (s == NULL)
 printf("do_str(NULL)!\n");

OK, so you get the error message, but where was do_str( ) called? Wouldn't it
be wonderful to have a standard function in your library, say caller( ), which
would give you the name (as a C string) of the calling function? You could
then write the code shown in Example 11.
Example 11: Using caller

 do_str(s)
 char *s;
 {
 char *caller();
 if (s == NULL)
 printf("%s() calls do_str(NULL)\n", caller());
 }

An impossible dream? Not so, as I will show. By accessing the stack you can
reconstruct the entire calling sequence from main( ) down to the current
function, and you don't need a debugger -- you use the same techniques that a
debugger uses, but you turn an introspective eye on the program as it is
running. The details are dependent on the operating system and compiler that
you are using, but the ideas can be applied to almost all implementations of C
(and to many other procedural languages), although a couple of assembler
subroutines might be needed. To begin with, you have to understand how your
compiler uses the stack to make function calls.



Looking at the Stack


The program showstack (see Listing Two, page 93), perhaps with some simple
modifications, will reveal the secrets of the stack on most computers.
The program makes a couple of function calls with easily recognized values for
arguments, and prints out the contents of the stack. I use SCO Xenix on an
80386 computer with the Microsoft C compiler, and the output on my machine is
shown in Figure 1 (with annotations).
The stack contains local variables ("automatic" variables in official C
jargon, although no one seems to use the term), function arguments, and data
required to manage the call/return sequences. Under Xenix, the stack grows
toward lower addresses, so the macro STACKLOW is defined. To see whether the
stack grows toward higher or lower addresses on your box, print out &argv and
&lower, if &argv < &lower, then your stack grows toward higher addresses. Each
call to a function has its own area on the stack, sometimes called the "frame"
for that function. While that function is being executed, a register is
typically reserved to point at the frame on the stack. This register is often
referred to as the base pointer (BP), and will typically contain the value of
the stack pointer (SP) when the function was called. To call a function, the
computer does something like that in Example 12.
Example 12: Calling a function

 push n'th argument
 ...
 push 2nd argument
 push 1st argument
 push current instruction pointer
 (ie. return address)
 jump to start of function
 copy stack pointer (SP) to base
 pointer (BP)
 push BP

Your computer may do other things -- push some registers to save their values,
for example -- but will almost certainly perform these operations, possibly in
a different order. When this sequence is completed, the top of the stack will
look like that in Example 13. (The stack grows down the page, so the "top" of
the stack is the last line -- confusing, but conventional.) The base pointer
is now used as a base to locate function arguments and local variables. Notice
that the arguments are pushed in reverse order; the first argument is then a
known offset from BP, allowing function calls with variable numbers of
arguments. Other schemes are occasionally used, such as pushing the number of
arguments onto the stack. Local variables will come after the caller's BP on
the stack.
Example 13: The top of the stack after functions have been called

 2nd arg
 1st arg
 Return addr.
 BP--> Caller's BP

To return from a function, the caller's BP is restored (in pseudo C: BP =
*BP), and the return address is then a known offset away.
Frames for each function make a linked list on the stack, with the current BP
as the head of the list. It is easy to follow the links: g's BP --> fs BP -->
main's BP in the output from showstack, as indicated by the arrows. The frame
for the call to main( ) points back to the C startup function, which performs
chores such as setting argc and argv before calling main( ). A debugger uses
this linked list of function invocations to trace the call sequence in a
program, and you can exploit it to create some powerful debugging tools. The
function shown in Listing Three, page 93, follows the list; it may need
adjustment for other compilers.
The stack frame for the call to ctrace( ) itself is found by taking the
address of its first argument: The output from showstack shows that the
caller's BP is at an offset of 2 from the first argument, at address (&arg_1 -
2). The return address is at an offset of 1 from the BP. The output in Example
14 was produced by modifying showstack so that g( ) does nothing except call
ctrace( ).
Example 14: Modifying showstack so that g( ) calls only ctrace( ) results in
this output

 &main=00000094 &f=000000db
 &g=000000f9
 BP=0187ed20 RET. ADDR=0000010a
 BP=0187ed38 RET. ADDR=000000f1
 BP=0187ed50 RET. ADDR=000000d3
 BP=0187ed6c RET. ADDR=0000057d

The values of &main, &f, and &g show the address ranges of the code for these
functions in memory (see Table 1).
Table 1: Values of &main, &f, and &g

 Function Addresses
 -----------------------

 main() 0x94 to 0xda
 f() 0xdb to 0xxf8
 g() 0xf9 to ???

By knowing these address ranges, it is possible to work out where a given
return address points. You need an array of function address and function name
to make the conversion of an address in the code area of a program to the name
of the function. Listing Four, page 93, shows how this is done.
The function atoname( ) finds the function closest to, but starting before,
the address given as its argument. If the printf( ) statement in ctrace( ) is
changed to that shown in Example 15, the output will appear as the example
then shows. The C startup function is erroneously identified as g( ), because
g( ) is the closest function that atoname( ) knows.
Example 15: The function atoname( ) finds the function closest to, but
starting before, the address given as its argument

 printf("BP=%081x FUNCTION=%s\n",
 bp, atoname(*(bp + 1)));

 then the output looks like this:


 &main=00000094 &f=000000db
 &g=000000f9
 BP=0187ed28 FUNCTION=g
 BP=0187ed40 FUNCTION=f
 BP=0187ed58 FUNCTION=main
 BP=0187ed74 FUNCTION=g

Function arguments are also accessible at a known offset from the BP. With my
compiler there is no way of determining the number of arguments made in a
call, so I assumed that there are two arguments of type int, and changed
ctrace( ) again, as shown in Example 16. If the symbol table used by atoname(
) was extended with the number and types of the arguments to each function,
the output could be further improved.
Example 16: Changing ctrace( ) again

 printf("%s(%x, %x)\n", atoname(*(bp + 1)), *(newbp + 2),
 *(newbp + 3));

After this change, the output of the test program became:

 &main=00000094 &f=000000db &g=000000f9
 g(55556666, 187eda8)
 f(11112222, 33334444)
 main(1, 187eda8)
 g(1, 187ee20)

Most software development environments provide a symbol table of the type
required by atoname( ), which can be read at run time. This might be the
"memory map" provided by a linker or binder, or, as in the Unix or Xenix
environment, a table included in the executable run-time file itself. This
table may provide little more than the address of each function entry point,
or may include considerable details such as the source file name, addresses of
statements identified by source file line number, numbers and types of
function parameters, and other information that could be exploited by tracing
functions such as those described here.

_DEBUGGING C PROGRAMS_
by Bob Edgar


[LISTING ONE]

short tlevel = 0;

main(argc, argv, envp)
char *argv[];
char **envp;
 {
 int n;

 for (n = 0; n < argc; n++)
 {
 if (argv[n][0] == '-' && argv[n][1] == 'L')
 {
 int i;
 char digit;

 /* Found -L argument - process it */

 digit = argv[n][2];
 if (digit < '0' digit > '9')
 {
 printf("Bad -L option\n");
 exit(1);
 }
 tlevel = digit - '0';

 /* Delete this element from argv[]. */
 /* We assume that argv[] has argc+1 */
 /* elements (most systems set argv[argc] */
 /* to zero). */


 argc--;
 for (i = n; i < argc; i++)
 argv[i] = argv[i+1];
 }
 }
 mymain(argc, argv, envp);
 }






[LISTING TWO]

/* showstack: Show layout of host machine's stack */

#define STACKLOW 1
int *stacktop, *ip;
int f(), g();

main(argc, argv)
char **argv;
 {
 stacktop = (int *) &argv;
 printf("&argc=%08lx &argv=%08lx\n", &argc, &argv);
 printf("&main=%08lx &f=%08lx &g=%08lx\n", main, f, g);
 f(0x11112222, 0x33334444);
 }

f(arg_1, arg_2)
 {
 g(0x55556666);
 }

g(arg_2)
 {
 int local;

 local = 0x77778888;
#ifdef STACKLOW /* Stack grows towards LOWER addresses */
 for (ip = stacktop; ip >= &local; ip--)
#else /* Stack grows towards HIGHER addresses */
 for (ip = stacktop; ip <= &local; ip++)
#endif
 printf("%08lx\t%08x\n", ip, *ip);
 }







[LISTING THREE]

calltrace(arg_1) /* Trace calling sequence */
 {

 int *bp, *newbp;

 bp = (int *) (&arg_1 - 2);

 while (bp < stacktop) /* "<" because STACKLOW */
 {
 newbp = (int *) *bp; /* next link in list */
 printf("BP=%08lx RET. ADDR=%08lx\n", bp, *(bp + 1));
 bp = newbp;
 }
 }







[LISTING FOUR]


struct func
 {
 int (*addr)();
 char *name;
 };

int main(), f(), g();

struct func funcs[] = /* symbol table */
 {
 main, "main",
 f, "f",
 g, "g",
 };

int nfuncs = sizeof(funcs)/sizeof(funcs[0]);

char *atoname(a) /* convert address to function name */
int (*a)(); /* address */
 {
 char *s;
 int (*maxa)();
 int n;

 maxa = 0;
 s = "?";
 for (n = 0; n < nfuncs; n++)
 if (funcs[n].addr < a && funcs[n].addr > maxa)
 s = funcs[n].name, maxa = funcs[n].addr;
 return s;
 }


Example 1: Defining the NDEBUG macro

 #ifndef NDEBUG
 #define assert(cond) {if (!(cond)) \
 printf("ASSERT cond, file %s line %d\n", \

 __FILE__, __LINE__); exit(1);}
 #else
 #define assert(cond) {;} /* Empty block */
 #endif


Example 2: Using an assertion

 do_str(s)
 char *s;
 {
 assert(s != NULL)
 /* rest of do_str... */



Example 3: using parentheses so that the entire arugment list to
printf is a single argument

 #define assertp(cond, args) {if (!(cond)) \
 printf("ASSERT cond, file %s, line %d\n", \
 __FILE__, __LINE__); \
 printf args; exit(1);}


Example 4: A typical use of assert

 assertp(n > 3, ("Surprise! n > 3, is %d\n", n))


Example 5: A macro which unconditionally prints when the trace
level is high enough

 #define assertl(level, args) {if (tlevel >= (level)) \
 printf("file %s, line %d\n", \
 __FILE__, __LINE__); \
 printf args; exit(1);}


Example 6: A typical use of Example 5

 assertl(2, ("i=%d j=%d\n", i, j))


Example 7: An assert.h header file

 #ifndef NDEBUG
 extern short tlevel;
 #define main mymain
 #endif


Example 8: Typical bit definitions

 #define TBIT_LEXER 0x0001 /* Lexical analysis functions */
 #define TBIT_PARSER 0x0002 /* Parser */
 #define TBIT_EXPTREE 0x0004 /* Expression tree */
 #define TBIT_CODEGEN 0x0008 /* Code generator */
 #define TBIT_OPTIM 0x0010 /* Optimizer */



Example 9: Defining assert-like macros

 #define assertb(bits, args) {if ((bits) & tbits) \
 printf args;}

which can be used in this way:

 assertb(TBITS_PARSER TBITS_EXPTREE,
 ("Nodes in expression tree = %d\n", nnode))


Example 10: Adding a statement to trap an error

 do_str(s)
 char *s;
 {
 if (s == NULL)
 printf("do_str(NULL)!\n");


Example 11: A function for a calling function

 do_str(s)
 char *s;
 {
 char *caller();
 if (s == NULL)
 printf("%s() calls do_str(NULL)\n", caller());
 }


Example 12: Calling a function


 push n'th argument
 ...
 push 2nd argument
 push 1st argument
 push current instruction pointer (ie. return address)
 jump to start of function
 copy stack pointer (SP) to base pointer (BP)
 push BP


Example 13: The top of the stack after functions have been called

 2nd arg
 1st arg
 Return addr.
 BP--> Caller's BP


Example 14: Modifying showstack so that g() calls only ctrace()
results in this output

 &main=00000094 &f=000000db &g=000000f9
 BP=0187ed20 RET. ADDR=0000010a

 BP=0187ed38 RET. ADDR=000000f1
 BP=0187ed50 RET. ADDR=000000d3
 BP=0187ed6c RET. ADDR=0000057d


Example 15: The function atoname() finds the function closest to,
but starting before, the address given as its argument.

 printf("BP=%08lx FUNCTION=%s\n", bp, atoname(*(bp + 1)));

then the output looks like this:

 &main=00000094 &f=000000db &g=000000f9
 BP=0187ed28 FUNCTION=g
 BP=0187ed40 FUNCTION=f
 BP=0187ed58 FUNCTION=main
 BP=0187ed74 FUNCTION=g


Example 16: Changing ctrace() again

 printf("%s(%x, %x)\n", atoname(*(bp + 1)), *(newbp + 2), *(newbp + 3));

After this change, the output of the test program became:

 &main=00000094 &f=000000db &g=000000f9
 g(55556666, 187eda8)
 f(11112222, 33334444)
 main(1, 187eda8)
 g(1, 187ee20)
































Special Issue, 1989
C CUSTOMIZED MEMORY ALLOCATORS


When malloc( ) et al won't do




Paul Anderson


Paul is a consultant and coauthor of Advanced C Tips and Techniques, published
by Howard W. Sams, from which this article was adapted. Paul can be reached at
1212 Eolus Ave., Leucadia, CA 92024.


C programs often need to allocate storage at run time. The C library routines
malloc( ), calloc( ), realloc( ), and free( ) use pointers to heap memory for
run-time storage. Although these routines are easy to use, there's no error
checking if you call them incorrectly. Subtle bugs can make you lose valuable
development time.
In this article, I'll show you how to customize the memory allocator routines
to help you during the development cycle. Along the way I'll introduce several
techniques that help pinpoint memory allocator errors as they occur. The
approach is to provide front ends to the standard allocators for debugging.
After the system is running and becomes stable, you can then replace the
customized allocators with the standard ones. You may, however, want to
continue to use the custom allocators in your system, depending upon the
overhead imposed on your system. This article should at least give you some
ideas on how to better C's run-time management routines for your own
applications.


Negative Subscripts


The basic rule for arrays that the compiler uses for a pointer offset is:
 &a[n] = a + n = (char *)a + n* sizeof(object)
Substituting 0 for n, you have
 &a[0] = a = (char *)a
This explains why C array subscripts start at zero. The compiler always uses
the base address for the start of the array. What happens when n is negative?
The formula for a negative pointer offset is only slightly different:
 &a[-n] = a - n = (char *)a - n * sizeof(object)
The pointer offset is below the base address of the array. Negative subscripts
are legal in C because the index is converted to a pointer offset.
Example 1 is a program that demonstrates out-of-bounds array references in C.
The program uses a negative subscript and also references an array element
beyond its last member. On the Xenix system I use, this program produces a
core dump, but I've seen it run fine under DOS and other Unix systems. Note
that this program would not compile if rewritten in Pascal or Algol. These
languages include compile-time checks that complain if an array subscript is
out-of-bounds. In C, however, array references simply convert to pointer
offsets. This allows programs with negative subscripts to compile and
sometimes to run.
Example 1: Out-of-bounds references

 /* twzone.c = array out of bounds */

 main()
 {
 int buf[10];

 buf[-4] = 1; /* negative subscript */
 buf[10] = 2; /* one step beyond */

 printf("%d %d\n", *(buf - 4), *((buf + 10));
 }

Negative subscripts can be put to work in a useful way. Here, I'll use them in
a custom version of the C library routine calloc( ). The routine, called
xcalloc( ), provides heap memory for data storage at run time, and provides
features that calloc( ) does not. For example, xcalloc( ) checks for errors
instead of leaving this job to the calling routine. This makes it easier to
use, and it helps centralize error messages. xcalloc( ) also stores an integer
that indicates the number of elements you request in heap storage along with
the data.
Programs call xcalloc( ) with the same protocols as calloc( ). xcalloc( ),
however, allocates memory for an integer value in addition to space for the
data. The routine stores the number of elements allocated from the heap and
returns a pointer to the space allocated for the data. Programs use a negative
offset from the heap pointer to access the number of elements.
Suppose, for example, you call xcalloc( ) to allocate heap storage for ten
integers and assign the heap address to p (a pointer to an integer).
int *p; . . . . p = (int *) xcalloc(10, sizeof(int)); . . . .
xcalloc( )'s first argument is the number of elements and the second argument
is the size of each element. sizeof(int) is used for portability. Figure 1
shows a modified heap configuration for a machine with 16-bit integers and
addresses. p points to ten integers (with initial values of zero) on the heap.
The number of elements (ten) is stored at address 8612, in front of the data.
p, however, points to address 8614, assuming that integers are 2 bytes.
Programs use a negative offset from p to access the number of elements (ten).
Example 2 is a program called neg.c that demonstrates xcalloc( ). neg.c calls
xcalloc( ) twice for heap storage. The first call allocates storage for ten
integers and the second for fifteen integers. You can assume the heap pointers
p and q are valid because xcalloc( ) checks for heap errors. The program calls
routines to assign integers to the heap and display their values. The first
call to fill( ) stores ten integer numbers on the heap, and the second call
stores fifteen integers. display( ) prints the data from the heap area pointed
to by its argument. Neither of these routines requires the number of elements
as a function parameter -- this is available from the heap pointer.
Example 2: Program that demonstrates xcalloc( )

 /* neg.c - negative subscripts with xcalloc */

 #include <stdio.h>

 main()
 {


 char *xcalloc();
 int *p, *q;

 p - (int *) xcalloc(10, sizeof(int));
 q = (int *) xcalloc(15, sizeof(int));

 fill(p); /* fill with 10 numbers */
 display(p); /* print 10 numbers */

 fill(q); /* fill with 15 numbers */
 display(q); /* print 15 numbers */

 }

 $ neg
 1 2 3 4 5 6 7 8 9 10
 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Example 3 shows the code for xcalloc( ). The variable blksize contains the
amount of memory you request from the heap. The function calls malloc( ) from
the standard C library to allocate blksize bytes plus one integer
(sizeof-(int) bytes) from the heap. Note that xcalloc( ) displays an error
message and terminates the program if heap space is not available. The
statement

 *(int *)pheap = nitems;
 /* store no. of items in heap */
stores the number of elements in the heap by casting the heap pointer to an
integer pointer before the size is stored. memset( ) zeros the heap area to
make xcalloc( ) compatible with calloc( ). The statement

 return pheap + sizeof(int);
 /* pointer to data */
returns the heap pointer to the calling routine, which is offset by the
integer containing the number of elements. sizeof( ) makes xcalloc( )
portable.
Example 3: xcalloc( ) routine

 #include <stdio.h>
 #include <malloc.h>
 #include <memory.h>

 char *xcalloc(nitems, size) /* custom calloc() */
 unsigned nitems, size;
 {

 char *pheap;
 unsigned blksize;

 blksize = nitems * size; /* size of chunk */

 if ((pheap = malloc(blksize + sizeof(int))) == NULL) {
 fprintf(stderr, "Can't malloc on heap\n");
 exit (1);

 }
 *(int *)pheap = nitems; /* store no. of items in heap */

 memset(pheap + sizeof(int),
 0, blksize); /* zero the area */

 return pheap + sizeof(int); /* pointer to data */
 }

Now for the rest of the routines. Example 4 shows fill( ) and display( ),
which use negative subscripts. Both routines access the heap to determine the
number of elements before they loop through the data. The statement p[-1] uses
a negative subscript. From the basic rule for C arrays, it's as if you typed
*(p - 1). This evaluates to an integer, which is the number of elements at the
heap address.
Example 4: fill( ) and display( ) routines


 fill(p) /* fill heap with integers */
 int *p;
 {

 int i;
 int nints = p[-1]; /* number of items from heap */

 for (i = 0; i <nints; i++)
 p[i] = i + i;

 }

 display(p) /* display integers from heap */
 int *p;
 {
 int i;
 int nints = p[-1]; /* number of items from heap */

 for (i = 0; i < nints; i++)
 printf("%3d", p[i]);
 putchar('\n');
 }

neg.c demonstrates only integer pointers. Programs may use xcalloc( ) to
allocate heap pointers to any C data type. Suppose, for example, you want to
allocate space for 20 structures of type something. You may call xcalloc( ) as
follows:
 struct something *ps;
 . . . .
 ps = (struct something *) xcalloc(20, sizeof(struct something));
This shows that xcalloc( ) is independent of the allocated pointer's data
type. The routine still stores the number of elements as an integer, however,
which means you have to cast the pointer appropriately to access it. I'll
elaborate.
Suppose sfill( ) and sdisplay( ) store data into heap structures and display
data, respectively. A call to sfill( ), for example, looks like
 sfill(ps); /* fill 20 structures */
Inside sfill( ), you access the number of elements with the following
statements:
 sfill(p)
 /* fill structures from heap */
 struct something *p;
 {
 int nitems = ((int *)p)[-1];
 /* number of items from heap */
 . . . .
 }
You must cast p before you apply the negative subscript. The parentheses are
necessary in order to make the statement compile. Without them, the compiler
tries to cast an array reference. The same changes would apply to sdisplay( ).


Customized Memory Allocators


Many of the standard C library routines have no error checking. When you use
these routines and errors occur, you may want to report error information that
the standard routines can't (or won't) provide. One way to get this extra
level of error checking is to write front ends to the standard library
routines. These custom routines check for errors and handle system specifics,
then call the standard routines if they don't discover problems.
The dynamic storage allocators from the standard C library are good candidates
for custom front ends. I've found that software systems that make heavy use of
dynamic storage allocation are harder to debug. Calls to malloc( ), calloc( ),
and realloc( ), for example, provide some information about errors, but you
rarely get enough to find the cause of the problem. What's worse, free( ) may
destroy the integrity of the heap if its argument is not a heap address from a
previous call to the allocator.
With a little effort and some overhead, you can improve this situation. In
this section, I'll present front ends for malloc( ), realloc( ), and free( ).
A front end for calloc( ) follows from the same approach.


Front Ends


Programs include a file called xalloc.h for using the front end routines. The
statement #include "xalloc.h" provides the interface. I'll examine this file
shortly.
I'll start with xmalloc( ), a front end for malloc( ). The use of xmalloc( )
allocates storage dynamically in the same way as malloc( ). Suppose, for
example, a file called pgm.c contains the following statements.
 #include "xalloc.h"
 . . . .
 char *pheap;
 . . . .

 pheap = xmalloc(100);
 /* 100 bytes */
 . . . .
xmalloc( ) allocates 100 bytes of memory from the heap. If all is well, it's
as if you called malloc( ). If there is an error, however, the program
terminates and displays the following message on standard error:
 file pgm.c - line 56: malloc error for 100 bytes
The error message displays the file name and the line number where malloc( )
fails. This helps you locate the exact spot in your source file where the
problem occurred.
The front ends xrealloc( ) and xfree( ) work in a similar way. Programs use
these front ends with the same parameters as their C library counterparts.
Note that none of the front ends require you to test the return value from an
allocator call.
Example 5 shows what's inside the include file xalloc.h. The header file
reveals that the front ends are actually macros that call separate functions.
The use of the special names _FILE_ and _LINE_ makes the C preprocessor
substitute a filename and line number each time the macro is called. This
allows each function to display an error message with a file name and a line
number when a C library allocator routine fails.
Example 5: xalloc.h header file

 /* xalloc.h = header file for customized memory allocators */

 #define xmalloc(N) ymalloc(_FILE_, _LINE_, N)
 #define xcalloc(N, S) ycalloc(_FILE_, _LINE_, N, S)
 #define xrealloc(P, N) yrealloc(_FILE_, _LINE_, P, N)
 #define xfree(P) yfree(_FILE_, _LINE_, P)

 char *ymalloc(), *ycalloc(), *yrealloc();
 void yfree();

Note that xmalloc( ) (a macro) calls ymalloc( ) (a function). Example 6 shows
the first version of ymalloc( ). It's not very fancy yet, but there's more to
come. At this point, ymalloc( ) calls malloc and checks the return pointer
value. If there's an error, an error message appears on standard error and the
program terminates.
Example 6: First version of ymalloc.c

 /* ymallocl.c - front end for malloc()
 Version 1

 */

 #include <stdio.h>
 #include <malloc.h>

 char *ymalloc(file, lineno, nbytes)
 char *file;
 int lineno;
 unsigned int nbytes;
 {

 char * pheap;

 pheap = malloc(nbytes);
 if (pheap == (char *) NULL) {
 fprintf(stderr, "file %s - line %d: malloc error for %u bytes\n",
 file, lineno, nbytes);

 exit(1);

 }
 return pheap;
 }

Now put xmalloc( ) to use. Suppose you want to maintain a symbol table of
different data types. For simplicity, limit data types to only strings and
doubles. One way to handle different data types is with a union of pointers to
the data. Example 7 contains a header file called defs.h which defines a node,
called "Symbol," for a symbol table linked list. The pointer pnext links each
symbol in the table to the next one. Suppose you declare
 Symbol *p; /* pointer to symbol table */
in a program. If p points to a symbol and p->dtype is equal to STRING, then
p->val.pstring is a pointer to a string. Otherwise, p->val.pdouble points to a
double.
Example 7: Symbol table example

 /* defs.h - symbol table definitions */

 #define STRING 1
 #define DOUBLE 2


 typedef struct Symbol {
 int dtype;
 union {
 char *pstring;
 double *pdouble;
 } val;
 struct Symbol *pnext;
 } Symbol;

Listing One, page 94, is a program that calls xmalloc( ) and creates storage
for a string symbol, a double symbol, and a large number of integers. For the
symbols, the first call to xmalloc( ) allocates storage for a symbol, and the
second call reserves storage for the data. I'll omit the details of linking
the two symbols together. The last call to xmalloc( ) doesn't have anything to
do with the symbol table, but it shows what happens if you try to allocate
heap storage for too many objects.
The header file defs.h contains the definition of Symbol. The first xmalloc( )
allocates storage for a string symbol and assigns a heap address to pointer
p1. The second xmalloc() allocates storage for the string member pointed to by
p1. The program sets p1's data type to STRING and copies the constant string
"test string" pointed to by ps to the heap.
The third xmalloc( ) allocates storage on the heap for a double symbol, and
the fourth xmalloc( ) creates storage for a double. The program assigns a
double precision constant (6.7e - 13) to p2, whose data type is DOUBLE. The
last xmalloc( ) attempts to allocate 30,000 integers on the heap. The program
fails on the Xenix machine I use because there's not enough room. The error
message indicates which xmalloc( ) fails.
Before I move on to the front ends for realloc( ) and free( ), let's include
another level of error checking. This time I'll have the calls to xmalloc( )
and realloc( ) save heap pointers, so that xfree( ) can check them before it
calls free( ). With this arrangement, the front ends flag an error if a
program tries to free an invalid heap pointer.
This requires a data structure to hold heap pointers. For simplicity, I use an
array that holds a maximum of 256 pointers. Listing Two, page 94, contains the
C source code for this arrangement. Note that I modified ymalloc to install
the heap pointer in the array of pointers called dbuf. All that's different
from the first version is a call to install( ) to save the heap pointers.
The install( ) routine searches the dbuf array for an empty spot (NULL). If
none is found, the program exits with an error message that says the debug
buffer is full. Otherwise, the heap pointer is stored in a vacant slot. For
simplicity, I make a linear search through the dbuf array and terminate the
program if it's full, but other approaches are possible (see the bibliography
at the end of this article).
Recall that the macro xrealloc( ) calls the function yrealloc( ), which is a
front end to realloc( ). yrealloc( ) verifies that the pointer that's passed
to it has been previously allocated on the heap, however. It checks the debug
buffer for the pointer, and if it's there, calls the standard realloc( ) to
replace it with a new pointer.
Now let's look closer at the yrealloc( ) routine. First, yrealloc( ) checks
oldp to make sure that it's not NULL and in the buffer. Otherwise, the program
displays an error message along with the illegal address, and exits. If all is
well, yrealloc( ) calls realloc( ) and verifies that the new heap pointer is
not NULL. Before yrealloc( ) returns the heap pointer, the function installs
the pointer in the same debug buffer slot as the old pointer.
The last front end is xfree( ), which calls yfree( ) and verifies that a
pointer has heap storage associated with it. This is the important level of
error checking that's missing from the free( ) library routine.
The yfree( ) routine displays an error message and exits if the heap pointer
is not stored in dbuf or if it is NULL. The error message displays the bad
address. If the pointer contains a valid heap address, the routine makes the
pointer's location in the dbuf array available (NULL), before it calls free(
). This arrangement guarantees that you never free storage on the heap that
hasn't been previously allocated.
I tested these new routines by modifying the symbol table example. Listing
Three, page 94, is a program called "sym2.c," which reallocates the heap to
hold a longer string for the string symbol and calls xfree( ) to free heap
storage. The first part of the program is the same as the previous version.
However, sym2.c calls xrealloc( ) with the old pointer, p1->val.pstring, to
allocate enough storage to accommodate a longer string, ps2. The program
copies the longer string to symbol p1 on the heap and displays the new string.
Note that sym2.c calls xfree( ) twice. The first call works fine because the
pointer contains a valid heap address. The second call, however, displays an
error message because ps2 is not a heap pointer.


Performance Issues


How about performance? Although the front ends provide error checking, be
aware that these routines introduce additional overhead and may cause problems
with time-critical software, for several reasons. First of all, you're passing
more parameters to the front end routines than the C library modules require.
Secondly, it takes time to save heap pointers and search for them. What you
have, therefore, is a trade-off between error checking and performance.
In the early stages of development, the front ends can help considerably
during debugging. After the software becomes stable, you can always remove the
front ends and return to the standard allocators. One way to do this is by
modifying xalloc.h, as shown in Example 8. This makes your code execute the C
library calls instead of their front end counterparts. Bear in mind, however,
that this approach does not check return values.
Example 8: Header file for standard allocators

 /* xalloc.h - header file for standard memory allocators
 (No error checking)

 */

 #define xmalloc(N) malloc(N)
 #define xcalloc(N, S) calloc(N, S)
 #define xrealloc(P, N) realloc(P, N)
 #define xfree(P) free(P)

 char *malloc(), *calloc(), *realloc();
 void free();



Bibliography


Anderson, Paul and Anderson, Gail; Advanced C: Tips and Techniques.
Indianapolis, Ind.: Howard W. Sams & Company, 1988.


Acknowledgments


Id like to thank Gail Anderson, Marty Gray, and Tim Dowty for their
assistance.

_C CUSTOMIZED MEMORY ALLOCATORS_
by Paul Anderson



[LISTING ONE]


/* sym1.c - symbol table data types */

#include <stdio.h>
#include "xalloc.h"
#include "defs.h"

main()
{
 Symbol *p1, *p2;
 char *ps = "test string";
 int *p5;

 p1 = (Symbol *) xmalloc(sizeof(struct Symbol));
 p1->dtype = STRING;
 p1->val.pstring = xmalloc(strlen(ps) + 1);
 strcpy(p1->val.pstring, ps);

 p2 = (Symbol *) xmalloc(sizeof(struct Symbol));
 p2->dtype = DOUBLE;
 p2->val.pdouble = (double *) xmalloc(sizeof(double));
 *p2->val.pdouble = 6.7e-13;

 printf("%s\n", p1->val.pstring);
 printf("%g\n", *p2->val.pdouble);

 p5 = (int *) xmalloc(30000 * sizeof(int));
}

$ sym1
test string
6.7e-13
file sym1.c - line 26: malloc error for 60000 bytes






[LISTING TWO]

#include <stdio.h>
#include <malloc.h>

#define MAXBUF 256 /* size of debug buffer */
static char *dbuf[MAXBUF]; /* debug buffer */

/* ymalloc2.c - front end for malloc()
 Version 2
*/

char *ymalloc(file, lineno, nbytes)
char *file;
int lineno;
unsigned int nbytes;

{
 char *pheap;
 void install();

 pheap = malloc(nbytes);
 if (pheap == (char *) NULL) {
 fprintf(stderr,"file %s - line %d: malloc error for %u bytes\n",
 file, lineno, nbytes);
 exit(1);
 }
 install(pheap); /* place in debug buffer */
 return pheap;
}

void install(pheap) /* store heap pointer in debug buffer */
char *pheap;
{
 register char **pbuf;

 for (pbuf = dbuf; pbuf < dbuf + MAXBUF; pbuf++)
 if (*pbuf == (char *) NULL) {
 *pbuf = pheap;
 return;
 }
 fprintf(stderr, "No room left in debug buffer\n");
 exit(1);
}

char *yrealloc(file, lineno, oldp, nbytes)
char *file, *oldp;
int lineno;
unsigned int nbytes;
{
 char *newp;
 register char **pbuf;
 short found = 0;

 if (oldp != (char *) NULL)
 for (pbuf = dbuf; pbuf < dbuf + MAXBUF; pbuf++)
 if (*pbuf == oldp) { /* find oldp's slot */
 found = 1;
 break;
 }
 if (!found) {
 fprintf(stderr,"file %s - line %d: realloc error for address %x\n",
 file, lineno, oldp);
 exit(1);
 }
 newp = realloc(oldp, nbytes);
 if (newp == (char *) NULL) {
 fprintf(stderr,"file %s - line %d: realloc error for %u bytes\n",
 file, lineno, nbytes);
 exit(1);
 }
 *pbuf = newp; /* replace in debug buffer's old slot */
 return newp;
}

void yfree(file, lineno, pheap)

char *file, *pheap;
int lineno;
{
 register char **pbuf;

 if (pheap != (char *) NULL)
 for (pbuf = dbuf; pbuf < dbuf + MAXBUF; pbuf++)
 if (*pbuf == pheap) {
 *pbuf = NULL;
 free(pheap);
 return;
 }
 fprintf(stderr,"file %s - line %d: free error for address %x\n",
 file, lineno, pheap);
 exit(1);
}






[LISTING THREE]


/* sym2.c - more symbol table data types */

#include <stdio.h>
#include "xalloc.h"
#include "defs.h"

main()
{
 Symbol *p1, *p2;
 char *ps = "test string";
 char *ps2 = "much longer test string";

 p1 = (Symbol *) xmalloc(sizeof(struct Symbol));
 p1->dtype = STRING;
 p1->val.pstring = xmalloc(strlen(ps) + 1);
 strcpy(p1->val.pstring, ps);

 p2 = (Symbol *) xmalloc(sizeof(struct Symbol));
 p2->dtype = DOUBLE;
 p2->val.pdouble = (double *) xmalloc(sizeof(double));
 *p2->val.pdouble = 6.7e-13;

 printf("%s\n", p1->val.pstring);
 printf("%g\n", *p2->val.pdouble);

 p1->val.pstring = xrealloc(p1->val.pstring, strlen(ps2) + 1);
 strcpy(p1->val.pstring, ps2);
 printf("%s\n", p1->val.pstring);

 xfree((char *) p2->val.pdouble);
 xfree(ps2); /* free a bad pointer */
}

$ sym2

test string
6.7e-13
much longer test string
file sym2.c - line 31: free error for address 2634



Example 1: Out of bounds references

/* twzone.c - array out of bounds */

main()
{
 int buf[10];

 buf[-4] = 1; /* negative subscript */
 buf[10] = 2; /* one step beyond */

 printf("%d %d\n", *(buf - 4), *(buf + 10));
}


Example 2: Program that demonstartes xcalloc()


/* neg.c - negative subscripts with xcalloc */

#include <stdio.h>

main()
{
 char *xcalloc();
 int *p, *q;

 p = (int *) xcalloc(10, sizeof(int));
 q = (int *) xcalloc(15, sizeof(int));

 fill(p); /* fill with 10 numbers */
 display(p); /* print 10 numbers */

 fill(q); /* fill with 15 numbers */
 display(q); /* print 15 numbers */
}

$ neg
 1 2 3 4 5 6 7 8 9 10
 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15


 Example 3: xcalloc() routine



#include <stdio.h>
#include <malloc.h>
#include <memory.h>

char *xcalloc(nitems, size) /* custom calloc() */
unsigned nitems, size;

{
 char *pheap;
 unsigned blksize;

 blksize = nitems * size; /* size of chunk */

 if ((pheap = malloc(blksize + sizeof(int))) == NULL) {
 fprintf(stderr, "Can't malloc on heap\n");
 exit(1);
 }
 *(int *)pheap = nitems; /* store no. of items in heap */

 memset(pheap + sizeof(int), 0, blksize); /* zero the area */

 return pheap + sizeof(int); /* pointer to data */
}














































Special Issue, 1989
VIEWPOINT: WHAT'S WRONG WITH C


What's Right with C?




David Carew


David works for Berger and Company in Denver, Colorado. David can be reached
at 1623 N. El Paseo St. Colorado Springs, CO 80907


n the June 1986 issue of Dr. Dobb's Journal, I wrote a "Viewpoint" entitled
"What's Wrong With C." Have things changed enough to make me alter my opinion?
While the title of this "Viewpoint" probably gives away my position today, it
is nonetheless time to reassess the state of C, and see how it stacks up for
the 1990s.
Just to recap: My two principle objections to C in the mid-1980s were: a. The
quality of C compilers and their output; and b. The productivity (including
the maintenance part of the software life cycle) of programmers using C. Let's
put the easy one to bed quickly -- the quality of today's C compilers is
dramatically better than it was a few years ago.
The popularity of a language correlates well with the quality of its
compilers. The compiler vendors can afford to spend more of those expensive
man-hours improving a product that sells well -- and they are compelled by
competition to do so. This is why nasty (but popular) languages, such as
Fortran and Cobol, often produce the tightest code in the environments where
they run.
The size and quickness of the object code is (mostly) a consequence of the
implementation, not the language itself. The real point is that C, with its
inherently low-level nature, is much harder (but not impossible, as my 1986
article implied) to optimize to the same degree as "better" languages. I'm
still willing to bet that Microsoft has many more man-months' effort invested
in its C compilers' optimizing technology than, for example, JPI has in the
optimization portions of its excellent Modula-2, which produces comparable
code. Perhaps many times more. But, as long as someone has gone to the trouble
for us, and as long as people are willing to amortize the extra cost by buying
and using C in droves, then who cares? C compilers nowadays are generally
"right and tight." They are production quality tools in a way much hyped and
hoped for in the early 1980s, but seldom seen.
Somewhat of a corollary to C's inherently low-level, operator-rich, "portable
assembler" character was an ethos I thought of as "small is beautiful." The
idea was that obscure idioms are O.K. because "the notational convenience is
considerable" (to quote K&R); and that C compilers did not have much to do
because a truly good programmer could always build his own (ideally designed
and reusable) libraries; and unerringly find the appropriate, optimal
algorithm; and without fail put a finger on the right ten percent of a program
to convert into assembler; or do something else beautiful (while expending
practically no time) to compensate for the elegant simplicity of his tools.
Tens of thousands of people are now using C full time; it should be obvious
that we are not all legendary coding paragons such as Jon Bentley, Dennis
Ritchie, and Brian Kernighan. Let "small is beautiful" rest in peace. Codified
ANSI C seems much less the small, informally specified, gratuitously
"elegant," personal use language that was so objectionable to those facing
large-scale, serious systems implementation projects. ANSI C is larger, with a
more "designed by committee" feel. ANSI C is internally more complex, for the
purpose of providing real-world production services, such as compile-time
parameter checking, which production coding shops have come to expect and
depend on.
C compilers are now presented as environments that do lots of things for the
"pilot." State-of-the-art debuggers and code profilers and front ends with
hypertext online documentation are conveniently (or instantly) available, as
are function libraries, prepackaged to do nearly any part of an application
the programmer doesn't wish to tackle. The infrastructure, which has grown up
around C, has contributed enormously to the productivity of C programmers. For
my money, this is as it should be and (almost) all to the good.
In most programming situations, bottom-line productivity -- man-hours to a
result with some acceptable functionality -- is the most important factor
separating the quick from the dead. C programmers are probably more productive
than ever before; I believe it is safe to say, even without citing
corroborating studies. It may be ironic that the plethora of productivity aids
surrounding C grew up because raw "early C" was so idiomatic and difficult
that help had a market value. Or it may be that the utilities and subordinate
tools appeared for reasons having nothing to do with alleged difficulties in
C. Again, who cares, as long as we got the boost we needed?
A tremendous strength of C, which I totally overlooked, is evidenced by the
growth of new technologies that leverage on C's portability and low-level
robustness. The very terseness and richness of C's operator set, which I
bemoaned as distracting to the human programmer, makes C a good target
language to be emitted from translators evincing new concepts and powers. C++
is perhaps the most famous example of this. The first C++ emitted ordinary C
as its output, bringing "object oriented-ness" to C in the same way that
RATFOR brought "structured-ness" to Fortran.
Even for those purists who believe that tacking on "object orientation" to an
existing language is the wrong approach, C has made its contribution. For
example, Bertrand Meyer's Eiffel language emits C as its object code. Eiffel
is a promising, practical (not interpreted or Smalltalk-like), purely
object-oriented programming system. Meyer's (recommended) book, The Design of
Object Oriented Systems, uses Eiffel as its "presentation language."
When employed as an intermediate language, C's terse and impenetrable
tendencies may be a decided plus. No less an authority than Knuth advises that
if the emitted code is deliberately unfriendly, then people will use the front
end for modifications, rather than tweaking the intermediate (C) code and
causing maintenance headaches. Knuth's WEB document compiler system uses this
approach, deliberately emitting "unreadable" Pascal. Arranging for your new
translator to emit unreadable C should be easy; getting human beings (or
anything else) to write halfway readable C appears to be the real trick.
C still is (in my mind) the consummate virtuoso's ax for "programming in the
small." For large teams working on large problems, even ANSI C's improvements
are not enough. I am intrigued by how much of the interest in C++ and other C
variants (such as Object C and Objective C) may be due not to the attractions
of object-oriented technology itself (which truly requires a shift in world
view) but to the possibility that "C objects" might be used in the context of
traditional design (as Modula-2 modules and Ada packages). That is, to render
"programming in the large" more practical and productive, and design software
modules more "plug compatible" and reusable. The perceived need is not for
(yet another) fundamental paradigm shift, but rather for a way to manage
implementation of large traditional designs in a language that is neither
obscure, nor unwieldy, nor disdained by the best programmers. Many people
appear to be examining the "right technology" (such as object oriented-ness)
for the "wrong reason" (or the hope of data abstraction and reliable
modularity) just because both the reason and the technology are connected with
C.
It is a monument to C's mutability, resilience, and popularity among the best
and brightest that C seems to be carrying the freight of the world's hope for
a language which is great for the individual, nice for the team, and
economical for the life cycle of a system.


                                                                                                                               r
applications on the same platform, provided that the application is mail aware
and OMI-conforming.
Many OMI API implementations will be function libraries. Programs will include
the library when they link. Others will be implemented as Dynamic Link
Libraries. For example, a Windows OMI API will work through calls to an OMI
DLL. Any OMI-conforming Windows application will execute properly no matter
what underlying message delivery system is in place. You won't need to relink
the application to a different OMI library, either. The user's Windows
installation would load the correct copy of the OMI DLL, and your application
would work the same no matter which one Windows loads.


OMI Functions



There are about 50 OMI functions divided into these seven categories: the
Standard Send function, Session Management functions, Message Creation and
Submission functions, Message Store functions, Message Access and Attribute
functions, Address Book functions, and Common Object functions. The OMI API
specification describes functions, data types, constants, and error codes in a
generic C-language context. It does not say whether the function definitions
have such things as far or pascal specifiers, no doubt leaving such
platform-dependent details to the implementer.


The Standard Send Function


The Standard Send function is one of the most interesting parts of the OMI
specification. It is a boon for applications developers and an albatross for
library implementors. With it, any application can be mail aware with little
more than an extra line of code. It works on the sending side of electronic
mail and includes in one function call everything an application needs to send
electronic mail.
To send a message, an application calls the Standard Send function. The
parameters include a list of recipients, an attachment file specification, and
text for the message and its subject. Here's where the magic comes in. If
those parameters are NULL, the OMI library takes over by prompting the user
for whatever is missing. This means that an implementation of OMI must include
the ability to pop up windows with which the user can type message text,
select recipients, and specify file specifications for attachments. This is a
significant feature if you are an application developer and you want to add
electronic mail to your application. To gain that e-mail check mark on the
bullet lists of magazine reviews, you simply add an option that calls the
Standard Send function with NULL parameters. Voila. E-Mail. Your application
will be able to send user-composed mail messages with user-specified
attachments to user-selected recipients. Although the user cannot receive any
mail, your application qualifies nonetheless as being mail aware. That's why
the Standard Send function is a boon to applications developers. Upon close
inspection, however, one might conclude that the capability has no more power
than a memory-resident mail program that pops up over the application. In my
opinion, the OMI specification could do as well without the user-prompting
requirement. Here's why.
If the Standard Send function is gravy to the application developer, it is a
burden to OMI library developers. They have to write the user interface code
for the selection list boxes, a text editor, and video window pop-ups over the
application, code that many applications will never use. It is apparent that
the OMI designers targeted environments such as Windows and the Macintosh
where the GUI manages a common user interface. But library developers for
text-mode environments such as DOS will have to handle the user interface
without help from the operating system. Those processes will add to the size
and complexity of the OMI library, and their looks and feels will seldom be
the same as those of the applications that they support.
Do not assume, however, that the Standard Send function has no value. The
strength of the Standard Send function is not in its ability to wedge an
electronic mail function into just any application but in its support for
nonmail applications to send data files with no other electronic mail
requirements. Such an application would develop the file and text parameters
for the Standard Send function based on the application's knowledge of the
data. There are several options presented by the function. The application
might omit the recipient parameters and let OMI prompt the user for that
information, or it could use its own user interface along with the OMI Address
Book functions to build a recipient list. A closed system might use embedded
recipient data, and the Standard Send function would be the application's only
interface to OMI.
The Standard Send function is the only part of OMI that requires a user
interface. The OMI specification does not say what that interface should look
like, only what it should do.
Although the OMI specification does not address the issue of partial
implementations, a minimal OMI library implementation might be no more than a
Standard Send function without the user interface. This implementation would
support those applications that p