Saturday, February 2, 2013

Language and Development Tool for Microchip Technology "PIC" Microcontrollers


Figure 1: A typical development setup. A text editor, output from a build script, and Microchip's MPLAB simulator are visible. 

Introduction    

The article at hand describes a cross compiler and run time environment designed by the author. This compiler is Windows-hosted, and targets Microchip Technology's PIC microcontrollers, in particular those that use 14-bit instructions.

These are inexpensive devices with an 8-bit word size.  The PIC devices employed here were selling for less than one US dollar each when this article was published.  

Microchip Technology refers to the devices targeted by this compiler as "mid-range core" devices. More recently, the company has released a series of "enhanced mid-range core devices." These are fully supported as well.

The language accepted by the compiler is a new one, named "Frost" by the author. A full-featured and modular library is also provided. Taken as a whole, the development tool suite released with this article offers, among others, the advantages listed below:    
  • Subroutines, functions and parameters    
  • 16-bit, general-purpose real number type ("SFP")    
  • Event handlers, with automatic preemption and constant interrupt latency 
  • Serial I/O and ANSI terminal support   
  • Support for string literals and look-up tables  
  • Modular, expandable runtime library 
  • Macros  
  • Full support for PIC features (timers, analog sensing and signal generation, etc.) 
  • Extensive facilities for portability, e.g. between "mid-range" and "enhanced mid-range" devices, and to other future Microchip Technology devices 
Equally important, in my view, is the design of the new language used here. Programmers do not always enjoy learning new languages. Even more often, they do not feel they have the time to do so.

Below, I attempt to show why a programmer might consider learning the Frost language. I make every effort to reference both a theoretical basis, and practical experience. Much of the latter is documented in my other, earlier PIC articles.

One significant advantage of Frost is the way it deals with concurrency. Rather than add new concepts to deal with concurrency, the Frost language's entire syntax is designed around being concurrency-friendly. Obvious syntactic markers exist to guide the developer toward usage patterns that enable concurrency.

In fact, the author has attempted to extend this minimalist approach to the entire language design. Rather than strive to be unique, or inventive, the author has striven to selected well-known idioms and bend them to his particular requirements as gently as possible. 

Background    

This is my fifth PIC microcontroller article at this site. Its predecessor articles have tried to show off the capability of the PICs, and have attempted to demonstrate how one can apply high-level techniques even in this somewhat primitive realm.

The most recent of these predecessor articles, titled Scalable Processor Arrays for Cybernetic Control, showed a technique for using multiprocessor groupings of PIC 16F690 devices to attack problems in the area of robotic control. The material in this article relies on a mathematical foundation built in the article titled Minimalist Floating Point Type, and uses a multi-processor network described in Synchronous Bus for Control Systems.

The application code presented is exceedingly well-tested, deterministic, and efficient. The operator implementations provided in Minimalist Floating Point Type, for example, were exhaustively tested using brute force means. In Synchronous Bus for Control Systems, nominal values for things like network throughput and GUI refresh rate were calculated using dimensional analysis and then confirmed empirically. This code is well-known and well-documented by any reasonable measure.

Scalable Processor Arrays for Cybernetic Control excels from a functional standpoint. It presents an architecture for scalable arrays of PIC processors; each processor is responsible for all aspects of control in a single dimension, using a  PID (Proportional - Integral - Differential) control algorithm algorithm. These are connected in parallel and in series, in large numbers if this is necessary, to create an overall robotic control system. The robotic control loop described in this article operates quickly, at about 200 hertz, and the economic and power consumption advantages of such a system are undeniable.

The development of such a profusion of quality application code rested on those high-level structures mentioned a few paragraphs above. In the assembly language code provided with Scalable Processor Arrays for Cybernetic Control, the following aspects of this architecture are already evident:  
Because of the limited nature of the cheap hardware in use, care had to be taken to conserve resources during development. The most obvious limit associated with the use of high-level techniques on these PIC devices is probably the small RAM size; the devices targeted here have between 128 and 384 bytes of RAM, and the attainment of a fairly high level of abstraction (and predictability) is an accomplishment in which the author takes pride. 

This overall infrastructure, which evolved over time, eventually received the rather prosaic name "HLOE" (for High-Level Operating Environment).   

A High-Level Language   

While it did make considerable use of high-level structures, Scalable Processor Arrays for Cybernetic Control also pointed out in detail where opportunities for further abstraction were still present in the article code. These opportunities are evident in the way that literal floating point values are placed atop the main stack, in the repetitive code used to handle the stack base pointer for parameterized function calls, and so on.  

Such issues were mitigated, to some extent, using macros. The result, though, is not ideal. So, Scalable Processor Arrays for Cybernetic Control went on to suggest the need for a high-level language to rationalize application development using the techniques it demonstrated.

In fact, Scalable Processor Arrays for Cybernetic Control took pains to isolate and define those portions of the code most amenable to being written in a higher-level language. These were designated HLOE "user" code and this term is formally defined in Scalable Processor Arrays for Cybernetic Control. Here, suffice it to say that this code implements the algorithm for the application at a high level of abstraction, using function calls built around the main stack.  

A New Language 

The author anticipates that many readers will focus immediately on the use of a new language. Most developers can learn new languages, but to learn more than a few seems a waste of time.

The next large section of this article is devoted to a detailed discussion of the author's rationale for engaging in the development and release of a new computer language. Readers uninterested in this portion of the article should skip to one of the subsequent sections, e.g. "Using the Frost Compiler," or "Frost Language Basics."

The best reason the author can give for using this new language is that its design grew organically from a set of abstractions, techniques, and libraries that revealed themselves as optimal over the course of years of assembly language development. Ultimately, the compiler presented here was actually written, and the new language used here was actually defined, because the author's work reached a point where these things were just trivial additions to what had already been done in assembly language.  

Other compilers that target the same devices as the Frost compiler presented here accept source code in the C language. The PIC processors, and in particular the "mid-range" and lower-end devices, though, have a reputation as being a difficult target platform for the developer attempting to create a C compiler (see here).

The author's experience reflects this fact. This development work did not evolve in a direction that would have supported a C compiler. At least, they would not have supported a C compiler and runtime as easily as they have supported the Frost compiler and run time; and because of the success of his development work, the author saw no reason to bend them in that direction.

Not incidentally, the author wants to point out that, while the specific computer language created by Kernighan and Ritchie is consciously rejected here, their overall thought process is, it is hoped, followed. Just as they rejected the heavy-handed design of Multics and PL/I in favor of a thinner set of abstractions and specifications that were C and UNIX, the goal of Frost and HLOE is to abstract over the PIC 8-bit processors in a way that is natural and easy for those devices, without allowing preconceived notions to unduly affect the overall design process.
  
The centerpiece of the initial Frost code provided with this article is an implementation, in this new language, of the PID controller presented in Scalable Processor Arrays for Cybernetic Control. In considering whether existing higher-level languages adequately serve devices like the PIC 16F690, it is worthwhile to consider whether such an application as this PID controller could be implemented in such a language, and, in cases where such a thing is possible, what form the necessary code would take. 

Most basically, the author must evaluate other higher-level alternatives in terms of their numeric data types. Microchip's own C compiler does not support the mid-range devices like the 16F690 used for the author's robotics application, but there are a few C compilers that do.

The SourceBoost C/C++ compiler, for instance, does support the 16F690, but does not offer any floating point data type at all. This basically rules out the possibility of writing programs around well-defined, general-purpose real number using this compiler. Using this compiler to write the PID controller described above would therefore require the programmer to address a host of issues abstracted away by Frost / SFP.

Another popular development tool, the Hi-Tech C compiler, offers only 24-bit and 32-bit IEEE floating point data types; these might prove too large for the PID application, which barely fits onto the 16F690 even after the savings associated with using the smaller (and non-IEEE-compliant) SFP data type. Or, they might operate more slowly than the 200 hertz rate achieved in Scalable Processor Arrays for Cybernetic Control.

One recent addition to the set of available compilers for the "mid-range" PICs is the CC5X compiler. The CC5X compiler is a commercial product priced starting at around $200 (US).  In general CC5X is an offering that seems serious about supporting the lower-end PICs. Wisely, it does include a 16-bit floating point type.

CC5X is a very worthwhile entry into the development market, especially for developers with $200 to spend and/or those who are unwilling to learn a purpose-built new language. I hope that Frost will prove an attractive alternative for developers falling outside these categories.

Like the 16-bit numeric type provided by CC5X, the author considers the use of the SFP data type to be an enabling factor for a wide variety of applications. The 8-bit PIC devices are already at a disadvantage in doing mathematics, because of their small word size, and because they lack hardware divide and multiply instructions. The SFP data type is a sufficiently small and simple type for these most rudimentary of devices.

Beyond its small size, the SFP type strives for simplicity in its bit layout; neither the exponent nor the mantissa used for the SFP representation exceed 8 bits, simplifying the logic necessary for floating point operations, and reducing their execution time.

Many existing development tools for the PIC devices offer integrated development environments (IDEs). The absence of an IDE for the Frost language is a drawback that could potentially be addressed in future work, if Frost itself is well-received. 

Designed for Concurrency  

In implementing a system like the one described in Scalable Processor Arrays for Cybernetic Control, any of these tools would face the issue of concurrency. The robotics Frost code presented here, and in Scalable Processor Arrays for Cybernetic Control, consists not just of a PID controller. It also contains a communications and user interface section which much transmit GUI-related information to the user at very exact intervals. 

As a Scrapnet client application, the robotics code presented in Scalable Processor Arrays for Cybernetic Control engages in Time Division Multiple Access networking, and it must transmit properly when its time slice occurs, with errors resulting in potential corruption of network data.

This aspect of the application design implies that a timer event must preempt the execution of the main PID routine at very exact intervals. Furthermore, it is probably not acceptable for one of these timer events to get skipped. This is an undesirable possibility in a real time control application, and one that makes it difficult to make formal guarantees about refresh rate and system response time.

The robotics application presented here, and in Scalable Processor Arrays for Cybernetic Control, exhibit some fundamental design aspects designed to facilitate the achievement of exact timing. In particular, these applications take a Functional Programming (FP) approach to the related problems of timing and concurrency.   

Functional Programming

FP in its purest form implies that code consists entirely of calls to pure functions, with no concept of a direct assignment into a memory variable, for example. Such code not exhibit the side effects inherent to static allocation (vs. parameters and automatic variables, as are used here). That is, such code exhibits referential transparency: any reference evident in the code refers unambiguously ("transparently") to a single actual parameter of a standalone function call instance.

So, functional code is inherently reentrant, with the caveat that things like I/O have their own built-in management issues, especially in a concurrent environment. FP does not obviate the need for the developer of Frost code to manage resources, but it does absolve the developer of the need to worry about race conditions involving static variables, among other trivialities. 
Frost code follows the FP ideal closely, in that it allocates its storage as needed, on the main stack, in the form of function parameters and automatic variables. In this way concurrency issues associated with the use of static memory locations are avoided. This is a fundamental way in which the functional programming approach facilitates concurrency.

In practice, it seems probable that no useful development tool can really enjoy all of the advantages of a pure functional approach. This is true if, for example, it intends to be useful for systems programming. If a function call results in I/O, say, or if it makes some hardware resource unavailable or unreliable for interrupting code, or even if code accesses a named PIC register, then this code exhibits a side effect, in a very real way, despite the fact that a superficially functional approach may have been followed.

The fact that a practical functional language still has these unavoidable side effects that must be managed does not really eliminate the usefulness of the functional approach. There are inherent concurrency issues, but under FP they are well-bounded. The application developer can use the PIC datasheet as a checklist for potential inherent concurrency issues (access to the EUSART must be arbitrated, access to each ADC channel must be arbitrated, and so on). While perhaps not completely foolproof, the FP approach was considered by the author to be preferable to the more open-ended set of potential concurrency issues present under many other paradigms. 

A New Formalism 

Without getting into the issue of how the various C language tools mentioned in this article handle all of these issues, it should be noted that the C language itself offers such concurrency issues in abundance. Their existence is a byproduct of the simple, portable nature of the language.

The C language's approach to concurrency seems to be to pretend that it does not exist. C programmers typically spawn worker threads by getting a function pointer and passing it into some OS or API-specific method that schedules execution of the code pointed to by this function pointer.

Of course, naïvely written C code will wreak havoc when made concurrent in this way. Access to static storage, and logic written around access to static storage, becomes unpredictable in its behavior. At least, this is true for any static accessed concurrently (e.g. by a main task and by an event handler).

It is possible to patch up the resultant issues individually, by relying on automatic storage more (as is done in FP), and by using things like locks and critical sections. These are also handled using OS or API-specific calls1 .

In an application like the robotics application presented here, and in Scalable Processor Arrays for Cybernetic Control, though, locks and critical sections are probably not an option. Their existence would imply that interrupts would, at times, get disabled. The timer interrupt that handles the rendering of the GUI, for example, might get disabled, resulting in a lag. Disabling interrupts is therefore probably unacceptable.

The author cannot say with certainty whether these obstacles could be overcome by a C program, having not tried to do so. It can be safely said, though, that C does not facilitate the goal of running with interrupts enabled 100% of the time. It offers too many pitfalls, in its static and dynamic (as opposed to automatic) allocation systems, and it does not offer much native support for things like events and processes.

To achieve the timing goal of running with interrupts enabled at all times, one must be confident not only that his or her own application code is running with interrupts enabled, he or she must consider whether or not any libraries in use happen to disable interrupts, and must further address any concurrency issues that arrive from the static and dynamic allocation facilities used by his of her code without disabling interrupts, along with the concurrency issues inherent to the shared resources in play for the application.

Such constraints conspire to push developers of applications requiring exact timing toward assembly language. In cases where a high level of confidence in the code is required, the lack of proven high-level abstractions can really slow down the development process. Over its long lifetime, designers of electronic systems have learned how to exploit the PICs for applications requiring exact timing, but it seems safe to say that the basic incompatibility of these techniques with higher-level abstractions has limited the creativity with which they have been applied.

The development tool offered with this article is designed to offer a better way. The introduction of high-level structures explicitly built around the things that lead to exact timing - like having a low, constant interrupt latency, and knowing that interrupts are always enabled - promises to reduce development time and cost by providing know building blocks with which application developers can work.  

Similarly, the SFP data type is designed to introduce a new formalism into PIC 8-bit development, helping to ensure quality. The limited nature of the native PIC 8-bit data types and operations no doubt have given rise to a wide variety of assembly language mathematics techniques. In cases where these need to be provably correct, costs are high. It is not inexpensive to demonstrate the correctness and reliability of a mathematics library built around 8-bit add, subtract, and shift operations. The construction of application-specific fixed point data types is also expensive, and truly general fixed point data types end up being huge, like the 128-bit C# decimal type.

The availability of the SFP type is thus a boon for application development. It has sufficient dynamic range for any application, is small, and has relatively fast operators. SFP is therefore an enabler of applications that would otherwise be too large. It is worth noting, too, that the SFP code was exhaustively tested in distributed testing prior to its publication. Few ad hoc PIC data types exhibit this advantage. SFP can be reused, and counted upon, over and over again by the application developer.  

Using the Frost Compiler  

A single ".zip" file archive is supplied at the top of the article. This is done instead of having "source" and "binary" archives, because the materials provided do not divide neatly into source and binary sections. They are too layered; there are at least four distinct computer languages in play here, and binary files for two different hardware platforms.

The ".zip" archive contains several top-level subfolders. One of these folders, "frostcompiler," contains the compiler itself. Each of the other folders contains the source for a single Frost application, along with scripts "winbuild.bat" and "winclean.bat." The first of these builds the source into a target ".hex" file. The second cleans the folder, i.e. removes output files relating to previous build operations. All of these scripts are designed to be run from Windows Explorer, e.g. by double-clicking their icons.

These top-level folders can be stored anywhere on your system, but the location of the "frostcompiler" folder is stored in each copy of "winbuild.bat." So, some editing of those files is necessary. An example build script, from folder "frostloopproject," is shown below:   
set frost=f:\beau\frostcompiler\
del hloe.hex
%frost%frost.exe file loop.frost proc 16f687 board lpc
pause      
This build script is very similar to all of the other build scripts supplied in the archive. It is the first line of the script shown above that requires editing to reflect the location of the "frostcompiler" folder (unless you use my own folder location).

The third line of the script shown above will also require editing under certain circumstances. It is here that the processor for the build is set. This is the 16f687 in the example shown above. A value called "board" is also set on line three above. This is equal to "lpc" in the example supplied, meaning that it is the Microchip Technology "Low Pin Count" Demo board (part number DM164120-1) that is being targeted. These settings are discussed in detail in the sections about portability and configuration management presented further below. Some information about extending the Frost tool suite to support additional processors is also given.

When the user of this new compiler begins a new application, it is advisable to clone one of these top-level project folders as a starting point. The main ".frost" file will typically need to be renamed, and this change will need to be reflected in the build script.

The author used version 8.60 of the MPLAB IDE for this work. Microchip has released a new, very different IDE called "MPLAB X" but they are continuing to support MPLAB 8, to avoid breaking existing code. When this was written, MPLAB 8 was available as a free download from Microchip Technology. However, it should also be stated that any assembler capable of handling Microchip's official assembly language could easily be coupled with the Frost compiler supplied, with minimal adjustments to the automatically generated build scripts. 

The MPLAB Simulator   

The reader without access to physical PICs and a PIC programmer can use the simulator integrated into the MPLAB IDE. Even the reader who does have access to such hardware will at times want to run his object code in the emulator. Inside of the MPLAB IDE, the simulator is selected using the menu item shown below:

Figure 2: Configuring MPLAB to use the MPLAB SIM simulator 


After this configuration item is set, it is possible to load a ".hex" Frost object file for debugging using the "Import..." option of the "File" menu. The "processor" and board "settings" present in the build script should be acceptable as-is. It may be necessary to set the process in MPLAB; this is done using the "Select Device" item of the "Configure" menu. The processor name is evident on the third line of the build script. 

Also, the "Settings" item in the "Debugger" menu should be used to ensure that EUSART output is enabled. The relevant settings are on the "Uart1 IO" tab. "Enable Uart1 IO" and "Rewind Input" should be checked, along with the "Window" output option. To bring up the serial output, select "Output" from the "View" menu, go to the "Output" child window of the IDE, and select the "SIM Uart1" tab.

Execution can be started using the "F9" key or by pressing a triangular icon. An example of the "Output" window is shown beneath this paragraph. At times, it will be helpful to right click in the "Output" window, and select "Clear Page" to ensure that the latest output is visible when the program is run. The "F5" key can be used to halt the debugger. 

Figure 3: MPLAB SIM EUSART Output   

Frost Language Basics  

Most basically, the Frost language is free form; that is, whitespace can be added and (mostly) omitted as the programmer desires. Function calls are initiated with a dot. The function parameters, if any, are contained within square brackets. To print character 13 and then character 10 (decimal) to the serial output device, for example, one could use the code shown below:
.printch[13].printch[10]   

The values "13" and "10" in the code snippet shown above are numeric literals. Because they do not contain a decimal point, they are interpreted as base-10 integers of single-byte size. These literal values translate into "push" operations onto the top of the main call stack. 

In fact, all of the things that come between the square brackets of a Frost function call are designed to leave values atop the stack for the function to access and/or consume. Consider an expression like this one:
.add[10 .add[2 6]]
At the highest level, this snippet represents the construction of a set of parameters for add. One of these, pushed atop the stack first, is a simple byte having value 10. The other is a composite creation. It is the result of a function call that adds 2 and 6. So, these are first pushed atop the stack, in that order, and then a call to add is made, which leaves 8 atop the stack. This is right on top of 10, and addnow gets called to replace these two arguments with 18.

This is how the Frost call stack operates, and it is similar to many other high-level languages. The fact that add expects two bytes and yields just one is its signature, in the same sense that a C or Pascal function has a signature.  

"Blinking LED" Demo  

I will begin my presentation of Frost programs with two canonical examples. First, a "Blinking LED" demo is given. Then, a "Hello, World!" program (using ASCII serial output) is shown. The "Blinking LED" program, below, targets Microchip's "Low Pin Count" (LPC) demo board (part number DM164120-1); unlike the "Hello, World" program, it is not portable.
event 1 is
 subi(1)(@portc) > portc
;

insert clock
0>trisc
0>portc
insert slowtimer
insert starttimer 

First, this code begins by creating an event handler. This will run every time "timer 1" (the 16-bit timer, in Microchip's nomenclature) elapses. It sets the value of the PORTC register, whose lower nibble ties to a set of LEDs on the LPC board, to one minus the previous value of PORTC. This is done using the integer subtraction macro subi. Because subi is a macro, not a function, it uses the parentheses instead of the square brackets. The differences between functions and macros are explained in great detail further down in the article.

The @ and > operators are key to the "Blinking LED" demo. The first of these, @, takes a value from the PIC SRAM file and copies it to the stack top. The second, >, performs the opposite operation. It removes a single byte from the top of the stack and stores it at a location in the PIC SRAM file. To adjust PORTC, one instance of each operation is necessary. We load PORTC to the stack top using @, then perform subtraction, and finally copy the stack top back to PORTC

Note: Certain Microchip-defined names contain an underscore, e.g. OPTION_REG. When accessing these from Frost code, replace the underscore with a dash e.g. option-reg.

On the LPC board, this will result in the rightmost LED of the four-LED set blinking. There will be a state change approximately six times per second, meaning that there will be three flashes of light per second.

After the event handler(s) in a Frost program comes the main task. So, the rest of the program shown above after the semicolon that ends the event handler is the main task. As in a C program, execution begins at the top of the main task.

Here, we begin with insert clock. This results in board-specific clock oscillator setup code getting inserted; almost all Frost programs will have this. Next, TRISC is cleared. This sets PORTC to output mode. Then, PORTC is cleared. Its value at power-up is unknown; zero will turn off all LEDs to start. Manipulation of these registers is very typical of PIC code in any language.

Finally, the 16-bit timer is set to slow mode and then started. Both of these actions are accomplished by inserting predefined library snippets supplied as part of the Frost infrastructure. The supplied libraries provide easy access to two timer modes: slow 16-bit (approximately 6 hertz) and fast 16-bit (approximately 48 hertz). Of course, Frost is a systems programming language, so it is possible to access any PIC timer option with the proper code.

This "Blinking LED" program is one of the Frost projects with a top-level folder in the ".zip" archive provided with the article. The folder name is "frostledproject" and the source code is in "led.frost." This can be built into a ".hex" file using script "winbuild.bat." This ".hex" file can be used to program the LPC board. My first PIC article gives some more information about setting up this demo board and programming its 16F690 CPU. 

"Hello, World!"   

The canonical "Hello, world" program is shown beneath this paragraph. 
calls printch
 
insert clock
frost115200bps
.puthelloworld[]
 
blurb (helloworld) ('Hello, World!')  
A complete explanation the "Hello, world" program shown above is given further below. Here, since this program uses uses some advanced Frost features, let us begin with a simpler version. This is from "hello.frost" in folder "frosthelloproject":  
calls printch
insert clock
frost115200bps
 
.printch[13].printch[10]

/Doing it this way for illustrative purposes only
/Later in the article, real strings are discussed
.printch['H'].printch['e'].printch['l'].printch['l'].printch['o'].printch[',']
.printch[' ']
.printch['w'].printch['o'].printch['r'].printch['l'].printch['d'].printch['!']   
The top of this program is identical to the last program shown. In order, these statements 1) tell the Frost compiler that printch is an external function containing HLOE "kernel" code which needs to be located and linked into the finished product; 2) insert some code related to the CPU clock signal, e.g. that enables or disables the internal clock crystal depending on the configuration selected and 3) turns on 115,200 baud asynchronous serial communications.

The remainder of the last program shown consists of calls to method .printch[]. These print out the typical "Hello, World" output, in somewhat tedious fashion. It should be clear from the code shown above that the single-quote character is used to construct ASCII character literals in Frost. Also, simply typing numbers in Frost code, like 13 and 10 in the last program shown, results in the creation of a numeric literal, and the numbers typed in are assumed to represent decimal numbers. (The codes used above - 13 and 10 - are used in ASCII communications to advance to the next line.)

The next portion of this article shows several different Frost programs that use progressively more advanced language features. Eventually, in these discussions, all of the original, shorter "Hello, World" program shown even further above are explained. 

Functions   

Frost functions are defined in a section of the program that lies below all of the code shown thus far. They are introduced using the d keyword. Consider the sample of code below, which prints the same "Hello, World" output as the last two programs. This is from file "function.frost" in folder "frostfunctionproject":  
calls printch 
insert clock
frost115200bps
.dowork[]
 
d .dowork is
.printch[13].printch[10]

/Doing it this way for illustrative purposes only
/Later in the article, real strings are discussed
.printch['H'].printch['e'].printch['l'].printch['l'].printch['o'].printch[',']
.printch[' ']
.printch['w'].printch['o'].printch['r'].printch['l'].printch['d'].printch['!'] 
;  
Above, the function is named dowork, and all three sections of a full Frost program are evident. First, we have the header section, which contains non-executable code, like configuration settings and macro declarations. In the example shown above, this consists of a single calls directive2


The middle section of the example program shown above is the main task, or entry point, of the program. First, insert clock takes care of configuring the oscillator. Ultimately, it sets some special-purpose register bits in a way that is determined by the Frost configuration system described below. Then, frost115200bps configures the EUSART for 115,200 baud operation, in similar fashion.  Finally, the main task concludes with a call to dowork, which handles the actual printing.

The final section of the program begins with the d keyword, and consists of the definition of dowork. In a more sophisticated program, other functions could be defined as well, and ultimately functions can accept parameters, and return structure values, of any type. These three sections- header, main, and definition- delineate all Frost programs, although any section except the main section can be absent.    

Configuration Management     

The last section did not dwell on the insert clock and frost115200bps portions of the main task. Both of these were included in the "main" section, as they ultimately evaluate to executable code that runs at program startup. However, neither of these is a function call, despite the fact that Frost has been presented as a "functional" language. Now that a little code has been shown, it is possible to explain these things further, but this requires a detour into the topic of configuration management.

The insert clock directive uses the insert keyword to insert Frost code from a file. This seemingly simply description belies a fair amount of important logic. At a high level, the insert clock directive means what it says: it inserts the code necessary to configure the clock oscillator, whatever that may be.

Frost directives like this one manage their generality by way of a two-tier configuration management system. First, there is a system of configuration based on what are termed "boards." The name "board" is used in the sense that an electronics designer might use it. The "board" is the overall final application, consisting of the PIC plus any auxiliary hardware that happens to be present.

The "boards" used in the example Frost code provided therefore have names like "lpc" (a Microchip "Low Pin Count" demo board, part number DM164120-1) and "robot" (the circuit described in Scalable Processor Arrays for Cybernetic Control).

The second dimension of the configuration system is the name of the processor itself, e.g. "16f690" or "16f1872." These two tiers complement each other, in that the "board" tier allows the applications programmer to establish named configurations, while the "processor" tier allows a mechanism for dealing with the processor taxonomy established by Microchip Technology.

In the case of the insert clock directive, it is the "board" system that actually comes into play. If running using the "lpc" board configuration, the Frost file that will get inserted into the application program will be file "clocklpc.frost" (from a system folder). If running using the "robot" board, the Frost file that will get inserted into the application program will be file "clockrobot.frost" and so on.

This is appropriate; some processors can run from an internal clock source, or from a variety of external clock sources. Which of these systems is actually in use is an aspect of the particular board that is being designed. 

Other insertion files will be named based on the "processor" dimension of the configuration system. This is the case for many libraries. Files "analog.frost" and "analog16f1827.frost" exist in parallel. The former is a default version, and the latter is specific to the 16f1827 processor.

These versions exist to account for differences in the number of analog sample values available on the 16f1827 compared to devices that use the default version of the file (e.g. the 16f690), and these differences are inherent to the processor itself, not the particular application.

It is worth noting that the "processor" dimension comes into play after the "board" version is searched for and found not to exist. If a "processor" version does not exist, the default version of the file in question is used (e.g. plain "analog.frost"). This is appropriate, in that the most specific possible file is searched for first, followed by a less specific one, followed by one that is not specific at all.

The two-tier configuration management system just described operates in many places throughout the overall system. It is used internally, by the compiler, to generate its object code, and this provides a pathway for major changes like the one between the 16f690 and the 16f1827. The HLOE "kernel" libraries contain parallel versions that use the "processor" naming system.

The linker files present in the "frostcompiler" folder are even named using the "processor" configuration system, e.g. "hloe16f1827.lkr." This provides for extension of the tools provided here beyond the four CPUs for which support is initially provided. Microchip Technology makes a host of LKR files available, e.g. in folder "C:\Program Files\Microchip\MPASM Suite\LKR\," and these can be copied to the "frostcompiler" folder and renamed as needed to provide Frost support for new devices.  

Portable "Frost"     

User-constructed libraries driven by insert can also make use of this version system, e.g. by having parallel versions of user libraries with board-specific or processor-specific names. The insertion files will declare macros, functions, etc., in parallel file versions. This will enable the code that invokes or calls these library entities to do so without even considering the underlying hardware differences. These are abstracted over by the "board" and "processor" settings.

The file insertion system is built for maximum flexibility. An inserted file can insert additional files, for example. Insertions can take place anywhere is a Frost source code file, and the files that get inserted can contain anything. The calls directive is allowed to appear anywhere in the source code, so library Frost files can include calls directives to bring in necessary dependencies.

If the Frost library designer has done his or her job (and tools are certainly provided to do so), many things will work silently and seamlessly for the higher-level application developer.  Frost code can use macros, function and other entities declared in insertion files without having to consider the ways in which the configuration selections he or she has made will affect them.

Here, we see this with frost115200bps, which works equally well on all of the hardware explicitly supported by the code base provided. It is written in Frost, with several implementations existing in parallel. Version "baud.frost" is the default file.

Library "baud.frost" and its alternate versions consist purely of macro declarations. These therefore incur no incremental cost on the application developer. Because of this, and because of the potential shared value present in "baud.frost," the author has stored these files in a system folder (named "frostcompiler" in the provided archive) and has made the insertion of "baud.frost" (or some version thereof) automatic.  

Library "freelib.frost" is treated similarly. It includes some macro-based capabilities that are discussed in further detail later in this article.

The code base supplied provides explicit support for a small but diverse set of devices. These are enumerated beneath this paragraph:  
  • Board  "lpc" - Microchip Technology's "Low Pin Count" demonstration board, part number DM164120-1. 
  • Board "robot" - The 16f690-based circuit used in Scalable Processor Arrays for Cybernetic Control
  • Board  "nextsys" - A rudimentary 16f1827-based demonstration board built by the author, which is described in detail later in this article.   

New Computing Devices   

Note that many other devices are supported, on a de facto basis. The MPLAB emulator provided by Microchip Technology is not one of the designated target platforms listed below, for example, but it is capable of running all of the programs supplied here, so long as the correct CPU is selected in MPLAB.

In creating a brand new circuit, it is possible to begin work with an existing configuration that at least uses the same processor family as the target device. Eventually, though, a true board name will need to be created.

At this point, it will likely be necessary to explicitly create a properly-named version of "clock.frost" in the "frostcompiler" folder. One alternative is to simply set with clock register bits using Frost code typed directly into the main task. The existing versions of "clock.frost" should be helpful. Another would be to rely on the power-up defaults quoted in the data sheet for a given device.

There is no default version of "clock.frost," though, so a simple insert clock will not work for a completely new board name unless some board-specific version of "clock.frost" is created. Processor-specific versions of "clock.frost" are not provided, since each processor can accept multiple types of clock source.

A similar situation exists for "baud.frost." Applications that use this, and attempt to use a new board name, will not compile unless a board-specific version of this library is created. There are no processor-specific or default versions, though, because sufficiently general code could not be written. In the case of "baud.frost," this is not possible because some applications will want send and receive interrupts enabled, some will not; some applications will want receiving of data enabled, some will not, etcetera. 

The author suggests these strategies for the development of portable Frost code:
  • Typically, non-library Frost files should contain portable code only. Areas of the code that are device-specific should be extracted into their own Frost library files. Then, as new hardware needs to be supported, board-specific or processor-specific versions of these libraries can be created. Having this code separated out into dedicated files helps highlight the areas that need to be dealt with in a porting effort. Exceptions to this guideline are probably justified for smaller projects, or projects that will never need to run on different hardware (remembering that statements made about scope and platform early in a project are notoriously unreliable). 
  • Code that will always be the same for a processor should be in a processor-specific file. Library "slowtimer.frost" is an example. The 16F1827 has slightly different register names associated with its 16-bit timer from those used for the other processors targeted here. This will always be true regardless of the specific application's board design. So, "slowtimer.frost" has multiple processor-specific versions (and future versions should be based on / named like these existing versions).
  • Code that will vary from application to application should be in a board-specific file. Clock signal setup is the most obvious example. Even a single chip, like the 16F690, can have many different clock configurations. So, library "clock.frost" has board-specific version, not processor-specific versions, even though this does make things slightly more difficult for the application developer setting up a new project.  

Looping Structures    

Loops in Frost are handled using the function call mechanism, rather than using dedicated language keywords like do or for. Most basically, the code shown below illustrates an endless loop. This is from "loop.frost" in folder "frostloopproject":  
calls printch
 
insert clock
frost115200bps
.many-Bs[]
    
d .many-Bs is
 .printch['B']
 .many-Bs[] 
;  

The code begins like all the other programs, by setting up the clock and EUSART. Then, a call to function many-Bs is executed. This prints a single "B" character, and then calls itself. As a result, this program simply prints an endless stream of "B" characters.  

Tail Recursion    

In a traditional, procedural or object-oriented language like C++, Java, or C#, this recursive attempt at looping would have resulted in several "B" characters getting printed. However, this would quickly and inevitably be followed by a stack overflow and crash.  

Frost, though, benefits from something called the tail recursion optimization. In short, this means that recursive function calls that are at the end of the function can be translated into simple jumps back to the top of the function. More formally, this optimization is possible, i.e. it transforms a recursive call into a jump, whenever nothing remains to be done after the call returns but to return to the original caller.

This will not always be possible for recursive Frost function calls, even for those located at the physical bottom of a function definition. Frost functions that have a signature, for example, require a cleanup routine to run after each function call returns, and this prevents the tail recursion optimization from taking place.

When the tail recursion optimization does get applied, though, the use of a simple jump allows the recursive function call to proceed without allocating storage atop the call stack. This is the key point which allows the last Frost program shown above to operate indefinitely, without crashing.  

Conditionals   

The next program shown, "forloop.frost" from folder "frostforloopproject," demonstrates iterative looping, analogous to the for loop in C++ or Java. As before, this is based not on a language keyword like for, though, but instead on the function call mechanism. As before, there is a recursive function call, but this time around the call is predicated on a condition.

Frost offers a conditional facility based on the if keyword. Such a conditional is a thin addition to the pure functional language described earlier, and invalidates none of the FP-related guarantees about concurrency, etc., made earlier in the article (see source [1]).  
In the Frost language, the conditional structure consists of the if keyword, followed by whitespace, followed by a sequence of code, similar in nature to a function body definition. Then, there is a comma. Then, there is a segment of code similar in nature to the first, followed by a second comma. Then, there is a third such segment of code, followed by a third comma. 

In this structure, the first segment of code is evaluated to see if the condition is true. If a non-zero value is present in the byte atop the main call stack at the end of the first segment of code at run time, then the condition is considered to be true. In this case, the second, comma-delimited code segment of the conditional structure is executed. If it is not the case, i.e. if the condition is false, then the third comma delimited segment of code executes instead. In either case, the test value atop the stack at the end of the first segment of code is consumed by the conditional logic.

The snippet of code shown below is an example of such a conditional structure:  
   if .parm[0],.printch['A'] .add[-1] .some-As[],,  

Above, the parm function is called. It is passed a value of zero. If it returns a non-zero value, then an "A" is printed, and -1 is added to the value atop the stack. If the condition is false, nothing is done, since there is no code present between the second and third commas of the conditional structure. The next section explains how such a conditional can be used to build an iterative loop.

Developers are advised to place functional calls that should use tail recursion into the second (else) clause of the conditional. Otherwise, there will be code after the call that may prevent the compiler from applying tail recursion (even if the call might still be a potential candidate for recursion from a purely conceptual standpoint). 

Iterative Looping     

The iterative loop is built around a single-byte unsigned integer counter present atop the main call stack. At the start of the loop, it is equal to the desired iteration count. At loop start, a function is called. This function simply returns if the counter has reached zero. Otherwise, it executes the loop body, decrements the counter, and calls itself recursively. This approach is used in program "forloop.frost," shown below:  
calls printch,add,parm
 
insert clock
frost115200bps
.printch['.']
.some-As[10]
  
d .some-As * is
  if .parm[0],.printch['A'] .add[-1] .some-As[],,
;    

Above, several features of the code are new. The call .parm[0] accesses parameter 0, i.e. the byte value atop the stack at the beginning of the function call. This is important to the looping logic described in the paragraph above the code sample. By "accesses," I mean to say that .parm[0] takes that parameter value and makes a copy of it at the current stack top, wherever that may be.

For the first few examples, I pass literal numbers to parm. Later on, I show how a system of parameter names can be used instead, which is less obscure in appearance.

In this case, .parm[0] accesses the iterative loop's counter, which remains atop the call stack throughout the evolution of the loop. The conditional structure makes this copy because its testing logic consumes the single byte value atop the stack after the call, and we must maintain the loop counter in its original position, not consume it.

In cases where the counter has not reached zero, it gets decrements by the call .add[-1]. This call passes only one argument to a two-argument function. So, only one byte gets pushed atop the stack, and whatever was already there is also consumed by the function. As a result, this call serves as an easy shorthand for simply decrementing whatever is atop the stack in place, e.g. a loop counter.

Functions that access parameters in this way (in place, using parm) must be declared specially, so that the Frost compiler will know to do the bookkeeping work that is required. Above, this is done using the "*" character, which is newly present before the is keyword. There is one alternative: two numbers can be placed at this point in the code, to establish a function signature. This mechanism is described further below. The use of the "*" character tells the Frost compiler to make parameters available, but not to enforce any particular signature for the function being declared.

This looping structure is unconventional compared to commercially popular languages like C# and Java. It is brief and effective, though.

The looping technique just described can even be extended into something resembling a while or do loop. The looping shown in the last program above is already contingent upon a condition, and the test clause of this condition can be constructed in widely varying ways to create any type of loop. It does not have to be a test of a decrementing counter. It can be an arbitrary conditional expression, analogous to the condition of a while or do loop.    

Parameters  

The concept of parameters is central to the HLOE architecture. HLOE defines the main stack used for function parameters and return values, which is shared by both Frost and "kernel" code. Frost code (i.e. application code written around the HLOE kernel) can use function parm to access function parameters, while HLOE "kernel" functions avoid using parm, for efficiency reasons (in particular, because of the limited depth of the PIC hardware stack used for subroutine calls).

Just as Frost code adds a layer of functionality compared to HLOE "kernel" code, the Frost language introduces further abstractions. For one thing,  the repetitive function prologues and epilogues used in Frost code are generated automatically by the Frost compiler. Beyond that, Frost adds the system of function declarations described in the last section, which make explicit that which was obscure in Frost code.

Like a Frost function that uses parameters, each Frost function that uses parameters ends with an epilogue section that essentially enforces its signature, while allowing the function body above it to freely build values atop the stack as necessary.

When a Frost function's operations are complete, it is required to have placed its return value(s) at the stack top. But it does not necessarily have to leave the stack pointer in the correct spot for its signature. The epilogue section mentioned above performs the manipulations necessary to ensure that these return bytes are indeed returned to the caller, and that this is done with the stack pointer in the "correct" spot for its signature. Other than placing its proper return values, the body of a Frost function has leeway to operate freely upon the stack, provided that it contains the epilogue and prologue sections described here. Ultimately, this represents a system of automatic storage.

The next program shown below ("parms.frost") demonstrates the Frost facilities for parameters and automatic variables:    
calls printu,negti

insert clock

frost115200bps

.printu[  .subtr[4 3] ]

d .subtr 2 1 is
 .add[ .parm[1] .negti[.parm[0]] ]
;
Here, a Frost function to perform integer subtraction, subtr, is defined and implemented, and used to subtract three from four. The result is then printed.

The subtraction function operates by negating its right operand (.parm[0]) using kernel function negti, and adding the result to its left operand (.parm[1]). Note that the numbering of parameters is from the top of the stack down; .parm[0] is the byte atop the main stack when the function is called, .parm[1] is the byte immediately beneath it on the main stack, and so on.  

The direct use of numbered parameters (.parm[0].parm[1], etc.) is one aspect of the last program shown that makes it less readable. Later in the article, ways to correct this deficiency are shown. This is done when the syntactical elements that support these techniques are presented.
Earlier, it was mentioned that function signatures play a role in the allocation and reclamation of storage space. In the last program shown, the implementation of subtr would leak memory (storage on the main stack) if this were not so. Consider the fact that, of all of the stack operations inherent to the subtr implementation, only the final call to add results in any net decrease in the number of bytes on the stack. The rest of the operations (which are function calls and pushes of numeric literals) are either stack-neutral, or they place values atop the stack.  

If the last example shown is too subtle, the next is exaggerated; it does not leak memory, despite the wasteful way it places literal threes onto the main stack: 
calls printu,negti

insert clock

frost115200bps

.printu[  .subtr[4 3] ]

d .subtr 2 1 is
 3 3 3 3 3 3 3 3
 .add[ .parm[1] .negti[.parm[0]] ]
;    

In either case, when the subtr function reaches its conclusion, the proper return value is atop the stack. The potential problem is that there are newly allocated values beneath it that need to be discarded. The fact that this gets done automatically is a central part of the definition of the Frost language. 

Automatic Storage 

Automatic storage can be allocated from the main stack during function execution. It will get reclaimed during the function epilogue. The code shown below repeats the functionality of the first two programs, but makes use of a single automatic stack byte in the subtr function:  
calls printu,negti

insert clock

frost115200bps

.printu[  .subtr[4 3] ]

d .subtr 2 1 is
 .negti[.parm[0]]
 .add[.parm[1] .parm[-1]]
;
Such storage is accessed using the same method as are parameters: the parm function. This is used with a negative integer parameter to obtain the value of an automatic storage location. The first location allocated is .parm[-1], the second allocated is .parm[-2], and so forth.  

Frost Macro Facility  

The Frost macro facility is full-featured (e.g. parameterized) and can fill a variety of roles. Most simply, it can be used to declare constants. The line shown below is from "bot.frost," a Frost language implementation of the robotic controller described in Scalable Processor Arrays for Cybernetic Control. It defines a constant which holds the maximum setpoint for the robotic controller (which is 435.0): 
macro max-setp is 435.0 end macro      
If such a directive is present in the Frost code being compiled, then every occurrence of max-setp is replaced with the literal text 435.0. To be technical, the post-replacement value begins after the single whitespace character occurring after the is keyword, be it a tab, space, or newline. Similarly, a single whitespace character is assumed before the end macro token, i.e. is not part of the post-replacement value. 

A simple parameterized macro declaration is shown below: 
macro subi is
 .add[$0 .mul[-1 $1]]   
end macro
Like the declaration of subtr shown earlier, this declaration creates a new identifier that can be used to effect integer subtraction. Here, though, invoking said identifier (subi) does not result in a function call, but of the emission of direct calls to add and mul into the code being compiled. In fact, the expanded macro will be evident in the output of "Frost.exe" during the Frost built process.  

The $0 and $1 designators expand out to the parameters passed into the macro invocation, from left ($0) to right.  So, the macro invocation 
subi(4)(3)  
would expand to 
.add[4 .mul[-1 3]]    
yielding the expected integer result of 1 atop the stack.  

Macros versus Functions 

In comparison with the use of functions, the use of macros has advantages and disadvantages. Both are likely necessary parts of most full applications. While functions are the ultimate low-level building block of Frost object code, macros also offer some powerful advantages.   

Each function call is typically smaller (in the object code and eventual code segment) than its parameterized macro equivalent. However, the PIC microprocessors have a limited call depth, e.g. 7 calls on the PIC 16F690. So, using macros can allow for certain operations that would not otherwise work. A macro invocation also saves the time necessary for a function call and return. 

An advantage of functions, on the other hand, is that their ultimate compilation is built around a formally specified context-free grammar expressed in Backus-Naur Form or BNF and handled by the GNU BISON parser. Macros are expanded by an ad hoc parser whose operation is described in natural language in the text of this article. In practical terms, any errors that happen will be shown by the compiler in conjunction with fully expanded code; but there are probably advantages to relying on language features that can be specified formally in BNF.

Finally, macros have one major advantage that is specific to how Frost handles dependencies: their inclusion in a code base does not incur any runtime cost unless they are invoked. This is fundamentally different from a function declaration, which will get compiled and emitted into the object output independent of whether or not it is actually called. Any concept of macros is gone before the Frost compiler proper (vs. the preprocessor) starts running, and any unused macros are simply discarded. 

This advantage of using macros is magnified further by the modular way in which Frost handles dependencies. While the Frost developer must specify HLOE "kernel" functions being called from each ".frost" file, these functions are not included in the final object code unless they are actually called.

So, if a particular ".hloe" file is called by one or more Frost functions, it gets included. Again, though, Frost functions get compiled end emitted regardless of whether or not they are actually called in the final object code. As a result, whole ".hloe" libraries can get included as a result of uncalled Frost functions. They will not get included as a result of unused macros. 

"Freelib.frost"    

Because macros are low-cost entities, a large suite of macro declarations is included by default by the Frost compiler. These reside mostly in file "frostcompiler/freelib.frost." The library is "free" not so much in its licensing, but in the sense that its inclusion, unlike actual, compilable Frost code or HLOE "kernel" code, does not incur a runtime cost. 

Macro subi already shown is the first macro declared in "freelib.frost." The next one is a similar subtraction macro, but for SFP real numbers:
macro sfp-sub is
 .addf[$0 .mulf[-1.0 $1]]   
end macro 
After that, "freelib.frost" continues with the declaration of the blurb macro. This is designed to create printable text literals. (Because of the significance of the parentheses and the slash character in the Frost language, these are not currently supported within blurb string literals.)

Strings and Lookup Tables   

The creation of multi-character string literals in the Frost language rests on the available of lookup tables that are present in the code segment. The data segment in SRAM is too small for string literals of any size, but fairly large string literals (and other lookup tables) can be constructed in the Flash-based code segment, which holds 4 kilowords of 14-bit instructions. 

When constructing a lookup table, the code segment is loaded with opcode RETLW, "return with literal value in W."  An 8-bit literal value is embedded in the instruction, and RETLW accepts a single 8-bit argument. So, when each RETLW is executed, a return-to-caller operation is performed, with the main accumulator W loaded with the specified 8-bit literal.  

A table of multiple RETLWs in sequence only makes sense if a branch into this sequence is performed.  Typically, the program counter is incremented by a variable value to arrive at the correct RETLW to get the table element value required. This requires quite a bit of work in PIC assembly language; the problems relate to the paging necessary for the relatively limited PIC to access its relatively large Flash code segment. Reference [2], an application note from Microchip Technology, explains the necessary process in depth.

In the Frost language, lookup tables are identical to functions in the way they accept and return values. Functions and tables differ in that the implementation of a table is not Frost code, but is rather a simple list of literal byte values.

Tables have distinct-looking declarations compared to functions; there is never a numeric signature, and even the is keyword is omitted. Where the d keyword is used in a function declaration, a t is used instead. Table declarations do end with a semicolon, like function definitions do.

The literal values allowed in a table take two forms. First, decimal values ranging from 0-255 are allowed. Second, text literals, delineated by single quotes, are allowed. Together, these two modes of declaration allow for the creation of zero-terminated strings, optionally including ASCII control characters, e.g.
t .helloworld
 13 10 'Hello, World!' 0
;    
Table definitions like the one shown above do not have a signature because all tables share a common, implicit signature. They accept a single byte index and return a single byte element value. Above, the Frost code .helloworld[0] returns 13 (the ASCII control character for carriage return), .helloworld[1] returns 10 (an ASCII line feed), .helloworld[2] returns an ASCII uppercase "H," and so on. (Because of the significance of the / character in the Frost language, this character is not currently supported within the single-quote-delimited parts of lookup tables.)

In general, all characters are allowable between the single quotes of a literal like the string shown in the last code snippet. Single quotes cannot be inside such a string. So, for maximum flexibility, a double quote is interpreted as a single quote. Double quotes can thus be fabricated using two double quotes inside the string; or, numeric literals can be interspersed at any point as shown above. 

The table shown in the last examole can be leveraged into a printable literal with a Frost looping structure:
d .puthelloworld 0 0 is
 .putihelloworld[0]
;

d .putihelloworld * is
 if .eq[ .helloworld[.parm[0]] 0 ],,
  .printch[.helloworld[.parm[0]]]  .add[1] .putihelloworld[],
 ;    
The code that needs to print the literal calls puthelloworld. This is turn calls another, helper function, with 0 on the stack top. This is an index into the table. The helper function prints characters from the table, and increments the counter variable. It calls itself using tail recursion, repeatedly, until it finds the terminating zero. 

The blurb macro automates the construction of the literal/printing declarations shown above, by way of the Frost macro facility. This allows code like this:  
calls printch
insert clock
frost115200bps
.puthelloworld[]
blurb(helloworld)(13 10 'Hello, World!')  
Recall that is the original "Hello, World" example shown near the top of the article. It has been explained over the course of the preceding sections, except for one peculiarity in the declaration of the recursive helper function called by the printing function puthelloworld

Stack Strings 

Character strings can also reside on the main stack. Single-quoted strings outside of a table definition result in characters getting emitted to the main stack. Program "stackstring.frost" holds an example:
calls printch
insert clock
frost115200bps

0 10 13 'Hello, World!' 
.stackprint[]

/For the 0
.dispose[]

d .stackprint * is
 if .parm[0],.printch[] .stackprint[],,
;
Above, the line 0 10 13 'Hello, World!' results in 'H' at the top of the stack, with 'e' beneath it, etc., until finally there is a 0 as a marker at the bottom of the entire string literal. The stackprint function pops these characters off of the stack and prints them until the zero is found. 

The Star Token   

In the printing code shown in the last section, the helper function is declared using the star character. This reflects the fact that it uses parameters (i.e. uses the parm function and requires a base pointer) but does not require a specific signature to be enforced for stack cleanup purposes.
Functions that use tail recursion must either use the star character, or must have no signature at all. The complete absence of a signature indicates to the compiler that the function does not use parameters at all.

Functions with a numeric signature will not be subject to tail recursion optimization by the Frost compiler. Such functions require a stack cleanup epilogue that prevents the recursive function call from being converted to a simple branch.

An additional burden is imposed upon the developer when using the star token compared to what is required when using other signature declaration syntax. The runtime infrastructure will enable the use of parameters for the function, but it will not do any stack cleanup when a call into the function returns. The developer must therefore ensure that the stack does not grow out of control.

One strategy for doing this is to call the star-based function from a function with a numeric signature, e.g. a wrapper function. When a function with a numeric signature returns, it always leaves the main stack pointer in the correct position, as calculated by applying its own signature to the position of the stack pointer when it was called. 

Floating Point Parameters  

After the declaration of blurb, "freelib.frost" continues with the declaration of a macro for accessing floating point parameters:  
macro parmf is .parm[$0+].parm[$0] end macro   
This allows for the first (left) floating point parameter to be accessed as a single unit, as parmf(0).  
The parmf macro represents the first time the + Frost token has been shown. When an integer macro parameter is used, the macro expansion can post-increment it using this plus token. So, parmf(0) ends up expanding to .parm[0].parm[1], which is the correct code to get the 16-bit floating point parameter value.  

The "freelib.frost" library continues with some ASCII-related macros. These are analogous in operation to the functions present in the old C header "ctype.h": 
macro isalpha is 
 .andb[.geu[ $0 64] .geu[156 $0 ]] 
end macro

macro isdigit is 
 .andb[.geu[ $0 47] .geu[58 $0 ]]  
end macro
(A comment is visible between the two declarations shown above. The forward slash character indicates that the remained of the line is a comment.) 

Loop Unrolling 

Finally, "freelib.frost" wraps up with a macro that is used to create unrolled iterative loops. For example, if a certain operation will happen, say, 15 times, one approach is to use a counter variable and a conditional branch to construct something like a for loop. If RAM constraints do not allow for a counter variable, or time constraints do not allow for a conditional branch, though, then it may be possible to proceed instead by simply having the compiler emit fifteen copies of the necessary operation.

The ffor macro is provided by "freelib.frost" to facilitate this. For example, ffor(.printch['B'])(9) expands out to nine calls to printch with 'B' as the parameter. The declaration of this macro is shown below: 
macro ffor is 
 $0 
 when! $1 1
  ffor ($0)($1-)
 end!
end macro    
The expansion begins with simply $0, the first iteration of the loop body passed to ffor in the first parameter. Then, a test is performed, using the when! token. This token allows its contents (i.e. everything between it and the end! token) to get expanded if the two whitespace-delimited values after it are not equal. This is a string comparison, for maximum flexibility, but in ffor the capability is used for an integer counter. Note that when! has a counterpart, when=, which expands its contents when the two things after it are equal.

Recursive Macros   

In the last macro example shown above, expansion of the text that is after when! takes place unless the second parameter to ffor is 1. Note that the inner text of the expansion itself contains an invocation of the ffor macro.

Such recursion is allowable; the macro expansion process will repeat indefinitely until there are no more macros found. As with runtime, function-based recursion, it is necessary that there be a base case, and here the base case is reached when the iterator counter reaches 1.

In the ffor declaration, the counter variable (which exists at compile time, on the development PC, not at runtime) is decremented using the minus token. With each iteration, $0, the inner operation of the unrolled loop, is again emitted, and this results in the correct number of operations.   
More commonplace nesting of macros is also possible, e.g.: 
   sfp-sub
    ( control-fnewtime )
    ( control-ftime )    
This code, which is used to calculate the time delta in the robotics proof-of-concept, consists entirely of macro invocations. All three of the identifiers visible in the code are macro names.

In the last example shown, the order in which expansion occurs is not significant. However, in cases where order matters, the developer can assume that macros are applied in the order they were declared, i.e. that macros declared nearer to the top of the Frost source code file are applied first. 

"Freelib.frost" is inserted at the top of the Frost source code file, and its macros get applied first. If it is necessary to know what order the macros in "Freelib.frost" are applied, it can be opened from the "frostcompiler" folder and examined.

Reentrancy and the "Shift" Key 

There is a certain logic to the selection of Frost symbols like the dot, star, and brackets. The choices made go beyond mere appearance or prior convention.    

The logic used to choose these symbols rested on two observations made by the author in his career as an every day programmer: 
  • Having to use the "Shift" key for any purpose slows down the development process (and this is true both of the process of entering data, and the overall thought process of the developer). 
  • Most languages contain both stack-based operations that offer a certain inherent safety, as well as less elegant / more unpredictable operations like assignment and dereference. 
The author's philosophy has been to allow for use of the safest language features without using the "Shift" key at all.  

Consider, for example, the snippet of code shown below:
d .subtr 2 1 is
 .add[.parm[1] .negti[.parm[0]]]
;     
On a US keyboard, this function declaration can be entered entirely without using the "Shift" key. It is also completely reentrant and will not leak memory. Really, the snippet shown above is emblematic of the basic way programs ought to be written, at a very low level at least. They should pass and return values on a stack, with function call unwinding cleaning up things in memory automatically. 

A couple of caveats are necessary, though. First, the promise just made applies to the US keyboard; the author apologizes for his inability to completely internationalize this aspect of Frost development. Second, the convention that "Shift" implies less safe syntax does not apply to macro declarations. Almost all of the custom syntax associated with more complex macro declarations relies on the "Shift" key, e.g. $0$1, and when!. This is so because of necessity; all of the default characters available without "Shift" were already used up by the Frost language proper. 

Many languages use parentheses where Frost uses square brackets. This forces the developer to constantly use the "Shift" key, and the more the programmer relies on the (inherently safe) function call facility, the more negative feedback he or she is given in the form of jammed keys and mis-capitalized letters, courtesy of that inexplicably important "Shift" key. This is a subtle, but unfortunate aspect of any language that uses the parentheses for function calls, i.e. most languages that the author has worked in commercially. 

The operations listed below, already discussed in depth, are inherently safe with respect to memory allocation and concurrency, and can be typed without using "Shift":  
  • ' (single quote) - emission of static lookup tables 
  • , (comma) - conditional 
  • ; (semicolon) - declaration terminator 
  • . (period) - function invocation 
  • [ (square bracket) - function invocation 
  • ] (square bracket) - function invocation 
  • / (forward slash) - comment 
Numeric function signatures are also typed without using "Shift" and are inherently safe. In fact, they are the backbone of the automatic systems that prevent memory leaks in Frost programs.

Function names can be typed without using "Shift," unless the developer elects to use capital letters in his or identifiers. This is not a recommended practice, unless the significance of the "Shift" key selected for the Frost language itself is continued. For example, the use of a capital letter to call attention to a function that introduces a potentially dangerous side effect, or allocates some resource that requires reclamation by other code, is endorsed. 

Unsafe Tokens 

Those parts of the Frost language that threaten reentrancy, or otherwise impose some sort of burden or risk on the developer, require use of the "Shift" key. The star token falls into this category; it requires that the Frost developer consider stack cleanup issues that are not otherwise present. 

Two of the unsafe tokens involve access to static memory locations. Accessing any static defined memory location deviates from the stack-based FP paradigm. Functions that use static or global variables are inherently non-reentrant. Assignments into static memory location represent side effects.

Functions that read from static memory are perhaps safer, in some sense, but also often involve volatile memory, e.g. an analog sensor register or timer counter. As such, operations involving static memory location require use of the "Shift" key in the Frost language. 

The first of these static memory-related tokens is the @ character. When followed by any assembly language register file identifier, e.g. @PORTA, it copies the value at that location to the stack top.  The "greater than" sign, >, copies the value atop the stack out to a register file location. These two features can be combined to create Frost CPU setup code, e.g.
.setbit[dc1b1 .setbit[dc1b0 @ccp1con]] > ccp1con 
Here, the DC1B1 and DC1B0 bits of the CCP1CON register are set to 1. To be precise, the value of the CCP1CON register is copied to the stack top. Then, it is passed to setbit, along with the bit number designated as "DC1B0" by Microchip Technology.  (Unlike in assembly language, lowercase letters are allowed in the Frost register and bit names. This is more consistent with the rest of the language.)

The resultant value is left atop the stack and passed into setbit again, this time to turn on DC1B0. Here, note that the parameter orders for setbit and clearbit have been selected with such chained expressions in mind. Finally, the whole expression is loaded back into CCP1CON using the (unsafe) assignment operator >

Static Variables  

In addition to the named registers specified by Microchip Technology, it is possible to allocate new, Frost static variables. This is done using the var keyword. Access is done using @ and >. Of course, this is a fundamentally unsafe and non-reentrant practice, and this is highlighted by the use of the "Shift" key that is required each time the variable is actually used.

An example of some static variable declarations is shown below:
var setf,setg,ticked 
This is taken from the robotics proof-of-concept discussed later in the article.  

Floating Point Statics

SFP real number values occupy two bytes. As a rule, functions that deal with these values expect for the exponent byte of the two-byte pair to appear atop the stack at the appropriate location, with the mantissa byte immediately beneath it. 

When writing Frost code that uses SFP literals (like 3.14 or 0.0) and calls to SFP library functions only, one does not generally need to think about this byte ordering. When allocating and dealing with static variables, though, a bit more thought is necessary.

First, when allocating static storage for an SFP value, two bytes must be requested, e.g.


var tempfloatm,tempfloate

When accessing that static's value, two @ operations are necessary:


.printf[ @tempfloatm @tempfloate ]

Finally, assignment of a single SFP value into static storage requires two > operations:


3.14159 > tempfloate > tempfloatm

Note that these dual assignment operations place the relevant identifiers in the opposite order of what is seen for the dual @ operations.

Naming and the Minus Sign 

The minus and plus signs are used in macro declarations, as discussed in the appropriate section above. The minus sign has an additional use outside of macro declarations (i.e. in the precompiler's pure Frost output). It is used to set apart identifier declarations, e.g.  control-fnewtime and control-ftime from the robotic controller code. 

The minus sign never implies a subtraction operation in Frost. While subtraction is presumably no more nor less frequent than any other mathematical operation, the need to set apart words in identifier declarations is ever-present. Other languages again resort to finger-twisting applications of the "Shift" key to deal with this need. The author's feeling is that control-ftime is both easier to look at and easier to type than controlFTimeControlFTimefControlTime, or anything of that sort. 

Tick and Backslash  

Macro invocation, which does rely on parentheses, is therefore deemed an unsafe operation. After all, a macro can expand out to anything.

At the same time, though, the design of the Frost language is such that the things that make a macro hazardous would presumably have been highlighted by use of the "Shift" key during the entry of that macro's definition. So, it is perhaps not necessary to again call out these dangers in the macro's invocations. 

Frost developers are therefore allowed an alternate syntax for invoking macros, in which the tick mark, `, stands in the the left parenthesis, and the backslash, \, stands in for the right parenthesis, e.g.  
   sfp-sub
    ` control-fnewtime \
    ` control-ftime \        

The HLOE Kernel       

This section gives an overview of the complete HLOE kernel as released with this article. Glimpses of the kernel's capabilities were evident in the code samples shown above. Function printch is a HLOE kernel function, for example.


Naming

Something similar to the Hungarian notation seen in low-level Windows programming is present in the identifiers used to name the HLOE kernel functions. In contrast with Windows programming's system of prefixes, though, a system of suffixes is used. This was considered by the author to be less invasive. Making the type-determined part of the identifier a suffix downplays it compared to the programmer-selected part of the identifier, and this is appropriate, in the author's view. These suffixes are limited to one letter, and are given in the table below:   

Table 1: HLOE Notation Suffixes    
  • F : This suffix applies in the many cases where 16-bit SFP data is primarily involved, e.g., functions divfaddf, and mulf.  
  • U : This suffix is used when single-byte unsigned data is involved. HLOE "kernel" function divu is an example. It works properly for 8-bit unsigned integers ranging from 0-255, but does not divide 8-bit signed integers properly.     
  • I: This suffix is used when single-byte signed data is utilized. HLOE "kernel" function negti is an example. It negates a signed 8-bit integer. 
  • B: This suffix applies whenever a boolean value is used. These are single-byte values, where 0 implies false and all other values are true. One example of this suffix is "kernel" function andb. This function is distinguished from andu (which performs a bitwise AND operation) only by its suffix. 
  • No Suffix: No suffix is used in situations where the identifier involves single-byte data, and there is no need to make any further distinction about type. HLOE "kernel" function add, for instance, performs single-byte addition. This works for both signed and unsigned values, and there is no other similarly-named function creating an ambiguity.
Bitwise functions generally have no suffix, but when it is necessary to distinguish between a bitwise operation and a logical operation (e.g. between "and" and "or"), suffix "B" is used for the logical operator, and suffix "U" is used for the bitwise operator. In such cases, naming the bitwise version of the operation simply "and" or "or" would have been misleading in the judgment of the author. 
      The HLOE kernel's conversion functions follow a set of naming conventions built around these suffixes as well. Examples are utof, which converts an unsigned byte to its SFP equivalent, and ftou which reverses this operation. 
          Finally, note that much of the HLOE user code, and the Frost code, presented by the author will use these suffixes as well. The macros in "freelib.frost," e.g. subi, follow these conventions.  


          Kernel Functions     

          The tables below list, and categorize, the functions that comprise the HLOE kernel as initially released. After each table, detailed information is given about the functions listed, particularly those with non-obvious signatures.    

          Table 2: Integer and Bitwise Operations     
          Name Description    Input BytesOutput Bytes
          andu Performs a bitwise AND operation  21
          xoru Performs a bitwise XOR operation  21
          oru Performs a bitwise inclusive OR operation  21
          add Adds bytes (signed or unsigned)21
          mul Multiplies bytes (signed or unsigned) 21
          modu Divides the first byte by the second and returns the remainder (unsigned) 2
          negti Negates a signed integer1
          divu Divides the first byte by the second and returns the result (unsigned)   21
          safeaddi Adds signed bytes; results bump into type extrema21
          setbit Sets a single bit of a byte (and returns the result) 21
          clearbit Clears a single bit of a byte (and returns the result)2

          The first parameter to setbit /clearbit is the number of the affected bit (0-7). The second parameter is the input value to which the bit manipulation is applied. A modified version of this value is returned atop the stack. The other operations listed above are conventional unary or binary operators.

          Table 3: Logical and Comparison Operations   
          Name Description    Input BytesOutput Bytes
          eqTests bytes for equality 21
          andbPerforms a boolean AND operation on two bytes (non-zero is true) 21
          orbPerforms a boolean OR operation on two bytes (non-zero is true) 21
          notbNegates the boolean value atop the stack 1
          geu Returns non-zero if the first byte is at least as large as the second (unsigned comparison)21
          gti Returns non-zero if the first byte is larger than the second (signed comparison)  21
          gtfReturns non-zero if the first SFP value pushed is greater than the second  41

          Function eq is a raw, bit-by-bit test for equality. It can stand in for an integer comparison, boolean comparison, pointer comparison, and so on. 

          Table 4: Serial Input / Output     
          Name  Description  Input BytesOutput Bytes
          printu 
          Prints an unsigned byte, in decimal format (ASCII) 10
          printchxyPrints a character at an X,Y location (ANSI terminal)30
          printfPrints a floating point number (decimal format / ASCII)20
          getchPolls the serial port for a character (ASCII)01
          day Applies "day" color palette (ANSI terminal)0
          nightApplies "night" color palette (ANSI terminal)00
          cls Clears the display (ANSI terminal)0
          graphx Draws a horizontal bar graph (ANSI terminal)40
          graphy Draws a vertical bar graph (ANSI terminal) 40
          printchPrints an ASCII character 10   

          Function printchxy expects the character to be printed to be pushed first, followed by the X coordinate, and then the Y coordinate. Functions graphx and graphy expect the graph width to be pushed first, followed by the current position (ranging from 0 to the width minus one), followed by the X and Y coordinates of the graph's left edge. 

          Table 5: Real Number Operations 
          Name  Description  Input BytesOutput Bytes
          mulf Performs SFP multiplication42
          divf Performs SFP division 42
          addf Performs SFP addition42
          powfRaises 2.0 to an SFP power22
          logfTakes the base 2.0 logarithm of an SFP number22
          utof Converts an unsigned byte to its SFP equivalent12
          iszerofReturns non-zero if the SFP parameter is 0.021
          ftou Attempts to convert an SFP value to its unsigned byte equivalent 1

          Table 6: Stack Operations
          Name Description Input BytesOutput Bytes
          copyf Copies the SFP real number value atop stack 024
          debugpeeku Prints the byte atop the stack to the serial port (ASCII / unsigned decimal format)   00
          debugptru Prints the stack pointer (ASCII / unsigned decimal format)0
          remove Pops the stack 0 top and pushes it atop stack 110
          restorePops the stack 1 top and pushes it atop stack 001
          dispose Discards the value atop the main stack 
            
          parm Accesses function parameters   1  

          Table 7: EEPROM Operations

          Name Description 
          Input Bytes
          Output Bytes
          readeeprom Reads a byte from on-board EEPROM storage
          1
          1
          writeeeprom Writes the byte in the right-hand parameter to the EEPROM location whose ordinal is in the left-hand parameter  
          2
          0

          Events and I/O  

          In general, the developer of Frost code can expect the serial I/O kernel to handle calls from both the main task and the event handlers as transparently as possible. Frost code must consider how serial output originating from the event handler(s) will interact with serial output from the main task, if both of these portions of the high-level code do emit output. However, the kernel functions relating to serial I/O are preemptible, reentrant, and do guarantee that each character output by the high-level code will be emitted on the bus. Interference in constructing character strings evident in the UI, e.g. ANSI positioning commands, must therefore be considered by the code developer.    

          Frost Libraries 

          Much library code written in Frost itself is provided in the "frostcompiler" folder. A file-by-file listing of the Frost libraries provided is given below: 
          • "analog.frost" - functions for reading analog inputs 
          • "baud.frost" - macros for setting serial port baud rate (e.g. frost115200bps)  
          • "clock.frost" - executable clock signal setup sequence for emission into main task 
          • "freelib.frost" - macros included by default 
          • "pwm.frost" - functions for pulse width modulated analog output 
          • "slowtimer.frost" - 16-bit timer setup for direct insertion into executable code; long period (~6 hertz) 
          • "timer.frost" -  16-bit timer setup for direct insertion into executable code; short period (~48 hertz)  
          • "starttimer.frost" - starts the 16-bit timer; for direct insertion into executable code 
          • "propeller.frost" - more functions for pulse width modulated analog output
          Note that "baud.frost" is, like "freelib.frost", a zero-cost dependency. It is therefore included in the Frost build process by default. 

          It is important to note that the names of the Frost libraries present in the "frostcompiler" folder should not be duplicated in application code. For example, in using Frost to make a digital clock application, you should make your own "clock.frost" file. Doing so will result in undefined behavior.

          File "clock.frost" does not actually exist. Only board-specific versions exist, e.g. "clockrobot.frost". Generalizations about clock setup are not possible to make; one must know the board design to say whether the clock signal is generated internally (as in board "lpc", the Microchip Techonology "Low Pin Count" demo board) or externally (as in board "robot").   

          Good examples of how to use "analog.frost" and "pwm.frost" are present in the robotics proof-of-concept application in "bot.frost". These functions therein are straightforward in operation. They generally accept an integer channel number parameter.

          For example, channel 0 means pin A0 for sensing functions. For PWM (analog out) functions, channel 0 is PWM ouput P1A and channel 1 is PWM output P1B. (The PIC is put into "pulse steering" mode by calling .pwmon[]; this allows for two analog out signals, e.g. "left" and "right" or "rich" and "lean.")

          In all cases, signal values are dealt with as SFP numbers having a range of 0.0 to 1023.0. This reflects the 10-bit resolution of the PIC ADC.  

          Library "propeller.frost" is similar to "pwm.frost," with two key differences. First "pwm.frost" is written for the 16F690 and related devices having a  portc register, which "propeller.frost" targets the 16F1827, which does not have this register, but does have some additional periperhals.

          Second, "pwm.frost" uses PIC "PWM steering" mode, in which a PWM command value is steered to one of two outputs (e.g. "left" vs. "right"), while "propeller.frost" uses the PIC "half bridge" PWM mode. In this latter mode, a command issued to a PWM channel generates a signal of the requested strength on one pin, but also generates its complement signal on a second pin. 

          All of this code has a 1024-value resolution, so, when using "propeller.frost," a request to apply a signal of value 600 to a channel will also apply a signal of value 424 to its complement. The code in "propeller.frost" (like the code in "pwm.frost") offers both channels 0 and 1, so, this "propeller.frost" code offers the possibility of controlling four motors from a single 16F1827 processor.

          For example, the code shown below will result in a square wave averaging to 600/1024 of the processor voltage being applied to pin RB3, a square wave averaging to 424/1024 of the processor voltage being applied to pin RB5, a square wave averaging to 700/1024 of the processor voltage being applied to pin RA7, a square wave averaging to 324/1024 of the processor voltage being applied to pin RA6:

          .prop0on[] 
          .prop1on[]
          .propf[600 0]
          .propf[700 1] 
           insert propeller

          Robotics Application   

          The robotics application is fundamentally similar to the one described in Scalable Processor Arrays for Cybernetic Control, but with the application firmware written in Frost instead of assembly language. The algorithms used in these two articles are identical. In fact, the names of the specific functions used, and their parameters, are practically identical. A few names are altered to conform to Frost naming conventions; conform_i becomes conform-i in the Frost, for example. 

          The physical hardware used is completely identical, and this applies both to the twin-processor demo board described in detail in Scalable Processor Arrays for Cybernetic Control and to the overall expansion and installation possibilities of the system. This robotics application is present in the source code archive provided with this article, in the "frostbotproject" folder.  

          The Frost language lends itself well to the robotics application. The code is fairly brief , at about 500 lines, and intuitive, despite the fact that (as was the case with the original assembly language application) some tricks are necessary to make everything fit.

          However, this does not make any great overall statement about the Frost language as a general-purpose language. The Frost language evolved from the author's experience writing a robotics application for networks of PIC microcontrollers. It should be good at that. 

          Still, a full exposition of the code used for the Frost robotics application lends insight into how Frost can be used to write larger applications. This is given below.

          In the most basic demo, the robotics application firmware runs on two PIC 16F690s, each of which controls a single degree-of-freedom using a PID algorithm.These processors run their control loops in parallel, and send output to an ANSI terminal using a Scrapnet serial bus. Each processor connects to analog inputs (joystick and position) and a single position commanding analog output.  Scalable Processor Arrays for Cybernetic Control also gives instructions for expanding the number of control loops beyond 2, and for nesting control loops in series. 

          In the two-processor demo, each processor runs a distinct version of the firmware, whose main code is present in file "bot.frost."  Throughout "bot.frost," control and preprocessing structures related to processor number are evident. In fact, this is a really central aspect of the code that handles the timing of the Scrapnet communications undertaken by each processor. Scrapnet is a time-division protocol with a shared, synchronous clock signal, and much of the Frost code relates to making sure that node 0 and node 1 each emit their output onto the Scrapnet bus during the correct time slice.   

          The PID Algorithm

          Note: the next three sections are condensed from the sections of the same name in  Scalable Processor Arrays for Cybernetic Control. Readers especially interested in these parts of the system should refer to that article. Also, that article contains detailed information about setting up hardware for the robotics system described here, tuning the system, etc.

          The main task consists of an infinite loop. Each iteration of this loop results in the calculation of a single command value. This command value represents the action that the PID algorithm has determined will bring the measured system position closer to the position setpoint established by the end user. Expressed mathematically, this command value u is calculated as shown below:  



          The construction above can be explained as the sum of three terms: the first term is a product of KP, a constant, and another quantity, the second a product of KI, another constant, and another quantity, and the third a product of a third constant, KD  and some other quantity. 

          The quantity by which KP is multiplied is e(t), the error, i.e. the distance between the user-commanded position or setpoint and the actual position, at the present time t. This term is probably the most obvious of the three; it makes sense that the command, u, should vary in direct proportion to the error.

          This first term by itself is used to construct the command u in the most basic "proportional" controllers. In such a controller, an error of 1.0 might translate into a command of 5.0, an error of 2.0 into a command of 10.0, an error of 3.0 into a command of 15.0, and so on. Or, in a controller wired and scaled differently, an error of 1.0 might result in a command of -1.0, an error of 2.0 in a command of -2.0, and so on. In the former example, KP would be equal to 4.0; the command equals 4.0 times the error. In the latter example, KP would equal -1.0. In all such controllers some such value KP exists, and it remains constant during normal operation (as opposed to setup). 

          The second term consists of KI  multiplied by an integral expression. The integral expression represents the sum of all net error observed in the system from time 0 (in practice, the time when the "Go" button was pushed) to present. At first glance it might seem that adding up the error from time 0 to present throughout the entire operation period of the controller would quickly result in a very large sum. In practice, this sum is minimized by the fact that the errors being added up can have either positive or negative sign. Over time, the error present in a well-operating system therefore tends to cancel itself out, and the second, integral term of the PID equation tends toward zero.

          If this does not happen, for example if actual position is persistently less than the commanded setpoint over some period of time, then the action of the second, integral term will tend to command position higher, to a degree that increases over time. 

          This integral term thus has a sort of memory, which serves to fight against any bias imparted into the system by its environment. In a vehicle heading control system, this bias could be due to a stiff breeze or current, or even an asymmetrical vehicle design. Whatever the case may be, the action of the second term of the PID equation serves to automatically correct against the bias present in the control system.

          The action of the integral term can even overwhelm the action of the other two terms if necessary; a highly negative second term may be of greater magnitude than a positive first term, resulting in a net negative command despite the natural command direction indicated by a purely proportional calculation.   

          The final, differential term consists of constant KD multiplied by a derivative. In discrete terms, this derivative represents the change in the error term observed in each iteration of the main loop compared to the prior iteration. The differential term, as typically configured, fights against sudden position movements by imparting into the overall command a slight opposite action. This term is often described as having a "damping" action; it imparts a certain stickiness or hesitancy into the otherwise crisp movement of the command signal. The principle benefit of this damping action is that it prevents overshoot during position changes. This is an undesirable effect in which the action of the proportional and integral terms ends up being too aggressive, and the attempt to follow the setpoint move ends up moving the position too far.

          Digitization


          The PID equation shown in the last section is continuous; it deals in exact real number quantities. The continuous nature of the time dimension implies that the values used to calculate u are being measured over an instantaneously small period of time. In practice, the best that can be hoped is that the actual period of time is small enough to allow for good performance in a given application. If this time period is constant, or near enough to constant for an application to assume constancy, then this further simplifies the calculations necessary to construct the integral and differential terms of the PID command value u. The discrete equation shown below is designed to approximate the continuous PID equation shown earlier, in an ideal system where the time period between each iteration of the main loop is constant.


          The first term is as before. The second term uses a sigma (simple summation of individual terms) instead of a continuous integral. It conveys a technique for approximating the integral shown in the ideal PID: to add up the error terms observed over time into a single sum, and multiply this by KI. Summation of this sort is a task that the processor is well-equipped to perform.
          The third term expresses an approximation technique for the derivative used in the ideal formula. Note that it is not precisely the change in error that is used to construct this approximation; rather, it is the change in position. This distinction only exists for main loop iterations where the setpoint has changed, i.e. most likely only for a small portion of the overall main loop iterations. The fundamental purpose of the third term - to impart a certain hesitancy to the control signal, and thus avoid overshoot - is arguably served equally well by the position-based calculation shown above.

          Time Measurement 

          A more general version of the discrete PID shown above would contain not just a subtraction in the third term, but a division as well, to account for potential variations in timing. If two main loop consecutive iterations happen to execute relatively far apart at runtime, then the difference calculated in the third term must be reduced in its impact (divided), since it happened over a relatively long period of time, and thus does not constitute sudden movement in need of damping to the extent that it would had it occurred over a shorter period of time.

          In practice, the main loop does not iterate at a constant rate, and this additional division step must indeed be performed. Variation in iteration time occurs for several reasons. Different parameter values will result in different execution times for the SFP operators (e.g., addf and mulf). Also, the main task is subject to preemption by the communications / GUI event, and this will at times delay the execution of the main loop.

          In addition to the division step necessary to properly calculate the differential term of the PID, the error term added into the overall sigma expression with each main loop iteration must be modified based on time. To be specific, in the implementation used here this error term is multiplied by the elapsed time since the last loop iteration.  

          After these time considerations are addressed, the discrete PID implementation ends up taking the form shown below this paragraph. Note that, in this formula and in the diagram above, a lower case Greek "phi" is used to represent a function returning elapsed time for a given main loop iteration.


          If the main loop takes 400.0 units of time to execute, for example, and an error of 1.5 is measured after the loop iteration, then the algorithm implemented here multiplies 400.0 by 1.5 and adds the result of 600.0 into the running error total. If the main loop takes 50.0 units of time to execute, but an error of -11.0 is observed after its execution, then -550.0 is added into this total.  

          The approximation that results from this technique is a type of Riemann sum. In particular, it is a "right" Riemann sum. This is a simple but effective approach to approximating the integral expression present in the ideal PID equation.  

          Frost Implementation  

          The main task of "bot.frost" begins thus:  
          .main-setup[] 
          .clearbit[tmr1on @t1con] > t1con /timer off 
          .wait-button[] 
          0 > tmr1h  
          0 > tmr1l 
          0 > ticked  
          .setbit[GIE @intcon] > intcon /GIE on

          Function main-setup tweaks some PIC configuration registers. Then, the timer interrupt is turned off. Function wait-button polls the designated digital input until the user of the demo presses the demo board's "start" button. After that occurs, and wait-button returns, the timer counter is reset and interrupts are enabled.

          Timer interrupts still do not start happening at this point, because of the second line above. Ultimately, each timer interrupt will result in the processor updating its portion of the user interface display. The 16-bit PIC timer is used; the nodes communicate on the Scrapnet bus only about 6 times per second, so a fairly long period is necessary.  

          After these preliminaries, the main task continues with some timing-related code: 
          macro maybe-stagger is
           when= $0 0
            0 > tmr1h 
           end=
           when= $0 1
            8 > tmr1h 
           end=
           when= $0 2
            16 > tmr1h 
           end=
           /And so on, for stations 3 through 31; fill in as needed...
          end macro
          maybe-stagger(netid) 
          The code above begins by declaring a macro, maybe-stagger. This can be done right in the middle of the main task, as is done above, or anywhere else in a Frost source code file. Then, the code above invokes this macro, passing in the processor number (0 or 1, stored in constant netid). 

          The maybe-stagger macro loads the high byte of the timer counter appropriately, such that the processor will communicate within its designated Scrapnet time slice; Scalable Processor Arrays for Cybernetic Control explains the specific numbers and registers involved.  

          Here, it is important to note that the use of a macro above, instead of a more conventional if-based conditional, results in smaller object code. (This is one of those tricks that is necessary to get everything to fit.) No test will be omitted, and the clauses that do not execute on a given processor will not be emitted into the object code for that processor. This is especially important in light of the large number of processors that could conceivably be added to the network.  

          Finally, the main task code from "bot.frost" concludes thus: 
          .setbit[tmr1on @t1con] > t1con /timer on 
          .longf[ 
             0.0 /Time zero
             .addf[ 1.0 .samplef[4]]  /Last position... code below assumes at least 1.0 
             0.0                      /Running I
             init-setp                /Setpoint (70 assumes 50-90 range)
          ]    
          Here, the timer is finally turned on. Then, a call into recursive function longf is made. This function never returns. Rather, it runs continuously until power is removed from the CPU, and executes the control loop.

          There is one iteration of longf for each iteration of the PID control loop. Its parameters basically describe the state of the controller at any point in time. First, it expects a time parameter. This is a number ranging from 0-255 (it is the top byte of the timer counter), but SFP format is used for convenience. Then, the last measured position is passed in; this allows for calculation of the position change, which is key to the PID algorithm (in particular to its damping component). Finally, the user-requested setpoint is passed in.  

          The Control Function  

          The control function in "bot.frost" is where most of the PID calculations are actually done. It begins as shown below: 
          macro control-ftime is parmf(8) end macro
          macro control-prev is parmf(6) end macro
          macro control-I is parmf(4) end macro
          macro control-setp is parmf(2) end macro
          macro control-sample is parmf(0) end macro
          
          d .control 10 8 is  

          Here, what amounts to a function signature is declared. The control function is declared as accepting 10 bytes and returning 8 bytes (the latter of which end up being updated copies of longf's parameters). The syntax used for this purpose on the last line above should be a familiar one.

          What is new above is the use of macros to give names to the parameters accepted by control. A macro is used to name parmf(0); recall that parmf(0) is itself a macro from "freelib.frost" which pushes a copy of the last (rightmost) SFP value passed into the function. The value parmf(2), the previous SFP number passed into the function, is given a name as well, as are all other parameters. The names thus created contribute immeasurably to the readability of the control function. 

          These macro declarations occupy the global scope of the Frost language. Care must therefore be taken to avoid overlap. Above, the function name is placed at the front of the macro name, and this convention is recommended for further development work. 

          The executable implementation of the control function begins like this: 
            macro control-fnewtime is parmf(-2) end macro 
             .utof[@tmr1h] 

          Again, we immediately see the use of a macro to give a name to an item on the stack. This time, a negative subscript is used to access a byte of automatic storage. Immediately after the declaration of this name, the corresponding stack position is filled. This idiom is analogous to the declaration and initialization of an automatic variable in a language like C (although automatic variables are not really a part of Frost). Taken from top to bottom, from subscript 8 to subscript -2, these declarations paint a metaphorical picture of the stack's state at this point in the function. 

          The initialization expression shown on the second Frost line above simply captures the current value of the TMR1H clock counter register. The @ operator is first used to copy this value to the stack top, then utof is used to convert this to an SFP value. This result is left in place and can be accessed using control-fnewtime for the duration of the control function.  

          For comparison, here is the equivalent code from the PIC assembly language implementation of this same controller, written in HLOE "user" code: 
           banksel TMR1H
           movfw TMR1H
           PUSH
           FAR_CALL control,utof  
          CONTROL_NEWTIME_VARF macro
           movlw -.1
           PUSH
           FAR_CALL control,parm
           movlw -.2
           PUSH
           FAR_CALL control,parm
          endm
          
          This code is identical in purpose to the Frost code shown immediately before it. A value is left atop the main stack, and a macro name for it is created. Above, though, much of what is abstracted away by the Frost language remains evident. In particular, we see explicit PUSH operations and bank selection operations that are not required in Frost, and generally more verbose, clumsier syntax. 
          The control function continues by allocating and initializing some more automatic storage:
            macro control-raw-et is parmf(-4) end macro
             sfp-sub
              ( control-fnewtime )
              ( control-ftime ) 
          This latest value holds the elapsed time between the last iteration of the PID loop and this latest iteration. This value, control-raw-et, is "raw" in that no test for validity has been performed; it is entirely possible that the timer counter could have rolled over between the two iterations, resulting in an invalid, negative result. The code that detects and deals with this eventuality is next in thecontrol function:  
            macro control-ticked is .parm[-5] end macro
             if @ticked, 1 0 > ticked, 0,
          
          Above, an if structure is used to initialize control-ticked. This is the first case in which initialization is done in this way. The if simply leaves the correct initialization value atop the stack, regardless of which of its clauses executes. 

          Functionally, control-ticked is a copy of ticked, which is a Frost static variable declared elsewhere using the var keyword. Static variables, as noted, are fundamentally unsafe (non-reentrant). At the same time, they are the only way to effect communication between the main task of a Frost program and its event handlers (which is an inherently tricky thing to do anyway).  So, use of these variables must be carefully structured.  

          The function of ticked is to tell control whether the timer counter rolled over between the prior PID loop iteration and the current loop iteration. To be precise, control must be able to determine whether the counter rolled over between the time when the prior set of time measurements was taken, and the time when the current set was taken. If the counter rolled over, control-raw-et will be an invalid, negative value, and should not be used. 

          From the perspective of controlticked is volatile. If it is tested and found to be zero, there is no guarantee that it will be zero for the next line of Frost code. This is why a copy of it must be made. Various operations in the remainder of control will be predicated on whether or not ticked was 1 at the key point in time. Making a copy allows these operations to behave consistently. 

          In examining the code above, one might ask why control-ticked is not initialized thus: 
           macro control-ticked is .parm[-5] end macro 
            @ticked    /Not the actual implementation in Bot.frost 
          This is not done because, in addition to initializing control-ticked, this part of the control function must also reset ticked to zero if it was one, in anticipation of the next timer event. So, an ifwould be necessary in any case, and using a single if (and a single @) to control both the initialization of control-ticked and the resetting of ticked seemed wise. 

          The resetting operation is therefore embedded into the clause of the if that executes when ticked is one, right after the code that loads the correct value for control-ticked to the top of the stack. (The resetting itself is stack-neutral, so it does not disrupt the initialization of control-ticked). 

          The function continues with some more initializations:  
            macro control-et is parmf(-7) end macro 
             if control-ticked, event-time, control-raw-et, 
          Here, control-et is initialized. This will be set to control-raw-et (the result of a prior subtraction) or, in cases where the ISR has run since the last iteration of control, to constant value event-time. The latter value, event-time, was measured by the author in MPLAB. In cases where the ISR has executed since the last iteration, real ET will be much longer than otherwise, and this must be factored into the PIC calculations3.  Next, the control function proceeds as shown below: 
            macro control-resample is parmf(-9) end macro 
             if control-ticked, .addf[ 1.0 .samplef[4]], control-sample,  
          Here, position is resampled if ISR execution has been detected. This is done to ensure that the position value used really is post-ISR. Even when we know ticked was determined to be 1, we do not know if the ISR ran before or after the position sample was loaded into control-sample. Interrupts are always enabled, preemption can occur at any time, and any interleaving of events is possible.  Because of the relatively long execution time of the ISR (which does I/O), it is important that the position sample used by taken after the ISR. Otherwise, this value will be stale. 

          Now, control can proceed with the final calculation of the position delta, which is key to the PID algorithm. For the reasons just mentioned, this calculation uses control-resample, not control-sample
             macro control-diff is parmf(-11) end macro
              sfp-sub(control-prev)(control-resample)     
          A similar calculation is then used to calculate system error for this PID iteration:  
            macro control-err is parmf(-13) end macro
             sfp-sub(control-setp)(control-resample)   
          Then, we arrive at an initialization that is more complex than its predecessors: 
            macro control-sigma is parmf(-15) end macro
             .conform-i[
              .addf[ 
                control-I  
                .mulf[  
                      control-err
                      control-et 
                ] 
              ]
             ]         
          The function of the code above is to add the error, multiplied by elapsed time, into the running error total used by the PID algorithm. Function conform-i is a fairly simple function that returns its parameter, unless this parameter is outside of a range, in which case an extremum is returned.  If allowed to grow infinitely large, the integral term might reach the point where adding the instant error to it with each loop iteration would have no effect, due to issues associated with the floating point representation. Integral windup is also prevented by the action of conform-i

          Now, the PID loop implementation must finish calculating the differential term D. At this point we have calculated only the raw position difference control-diff. Recall from the "Time Measurement" section above that this must be divided by the elapsed time. Then, each of the three terms P, I, and D must get multiplied by its associated constant KPKI, or KD. The results must be summed together, and this sum passed to an analog output function. All of these operations except the first are handled by helper function usrpwm; the calculation of D is embedded in control's call to usrpwm, shown below: 
          .usrpwm[                                       
             /D term
             if .iszerof[control-diff], 0.0, .divf[ control-diff  control-et ], 
          
             /I term
             control-sigma
          
             /P term
             control-err
            ]
          
          Note that the division performed above is augmented by a check to ensure that the numerator is not zero. The PIC SFP implementation used here requires this check, due to some limitations affecting calculations that would yield a result that is less than the minimum representable SFP number. Details are given in Minimalist Floating Point Type

          After that, all that remains for the control function to do is to construct its appropriate return values atop the stack. By design, control constructs the parameters for the next call to longf, whose signature looks like this:
          macro longf-ftime is parmf(6) end macro
          macro longf-last is parmf(4) end macro
          macro longf-runi is parmf(2) end macro
          macro longf-setp is parmf(0) end macro
          d .longf *  is 
          From the top of the signature shown above down, we are passing elapsed time, prior position, the running error total, and the current setpoint. The construction of these values is handled by the next snippet of code shown below, which concludes the control function:
            if control-ticked,  14.0, control-fnewtime,
            control-resample
            control-sigma 
            control-set    

          The Event Handler  

          The event handler in "bot.frost" deals with I/O. It therefore represents a timer's "tick" event, occurring at the correct Scrapnet rate (slightly less than 6 hertz). The part of "bot.frost" that declares the event handler, along with some associated static variables, is shown below: 
          var setf,setg,ticked
          
          macro event-body is
           1>ticked
           when= $0 0  
             .graphy[15 @setf 5 19 ]
             .printch[8] /Backspace
             .printch[8]
             .printch['S'] 
             .graphy[15 @setg 6 19 ] 
             .printch['P'] 
           end=
           when= $0 1
            .graphx[15 @setf 60 15 ]  /15-position graph at 60,15
            .printch['S'].printch['e'].printch['t']
            .graphx[15 @setg 60 16 ]
            .printch['P'].printch['o'].printch['s']
           end=
          end macro
          
          event 1 is /Timer 1 tick
           event-body(netid)
          ;  
          Looking at the very bottom of this code fragment, one sees that event 1 consists entirely of an invocation of macro event-body. Use of a conditional macro instead of a run time if saves space in the object code, and also saves time, since no run time processor number check will take place.

          As we saw with some of the timing setup code already discussed (involving macro maybe-stagger), this macro invocation is contingent upon constant netid, which is a processor number residing near the top of "bot.frost". The developer must edit this number at the appropriate points in his or her building / Flash programming process. The when= blocks that make use of netid are very similar to those used for maybe-stagger

          At the top of the fragment shown above, we have the declaration of static variables setfsetg, and ticked. The third of these was discussed at length in the last section. As for setf and setg, like any other concurrently accessed static variable, these create potential issues. Here, setf and setg are written only by the main task, and are only read by the event handler. This one-way communication setup prevents any race conditions or timing issues. 

          The actual data held in these variables consists of the position and setpoint. These are expressed in the 0-15 scale actually used by the bar graphs. Importantly, this prevents the event handler from referencing any floating point operations, which helps reduce the object code size (see "Kernel Memory Allocation" and "Function Families" further below for information about this).   

          The real work done by the event handler consists mostly of ANSI GUI rendering. Recall that the first parameter to graphx / graphy is the maximum graph position. Then, the current graph position is passed, followed by the X and Y coordinates for the graph's let edge. In addition to bar graphs for setpoint and position in both dimensions, the word "Set" and the abbreviation "Pos" must be rendered (or, for the vertical bar graph, "S" and "P"). After the vertical bar graphs are drawn, character 8 (backspace) is sent twice to position the cursor where the "S" should be. Regardless of CPU number, tickedmust be set to 1. 

          While event 1 is by its nature tied to timer 1, code is necessary to actually set up and start timer 1. In the robotics proof-of-concept, this code is in main-setup. It consists mostly of PIC register manipulation using setbit and clearbit

          Finally, a word about event naming is necessary. On the CPUs actually targeted here, event 1 is associated with timer 1, event 0 with timer 0, and so on. (Some devices have a timer 2 as well). The event named event a is associated with change events wired up around the PORTA register. The event named event b is associated with PORTB, and so on. 

          The Frost language definition (explored below) allows for an event name to be either an alphanumeric identifier or a non-zero integer, which leaves the door open for supporting all sorts of things. The actual implementation provided assigns no significance to all of these potential event names, though, beyond the few event names just discussed. 

          Compiler Architecture   

          Ultimately, the Frost developer's source code input is preprocessed into a pure Frost code base that respects a formally specified context-free grammar. This grammar encompasses the Frost language as described in the article thus far, except for macros and other preprocessor directives like stack is, insert, and calls.

          The style of the compiler C++ code is mostly procedural; I did not introduce my own classes, for example, and the only really inheritance-based library I used was "iostream.h." Template-based programming is heavily employed in the compiler implementation, both in the use of STL and in the use of a template-based ad hoc lexer used for some ancillary tasks not offloaded to Flex. 

          This preprocessed source code is processed by a GNU Flex lexer, and compiled by a GNU Bison-based compiler. These tools are GNU implementations of the classic Lex and YACC tools present in most UNIX-style OS installations. In the input files for these compiler-generation tools, formal definitions of the Frost language, in its pure, preprocessed form, are apparent.

          The input file for the GNU Flex lexer is "frost.l". This file defines the tokens of the Frost language using the Flex regular expression syntax. The input file for the GNU Bison compiler-generator is "frost.ypp", and the basic language of this file is Bison's version of Backus-Naur Form

          While certain C++ language features (e.g. STL and std::string) are heavily used in the compiler implementation, the Flex output is plain C (file "lex.yy.c"). I made this decision because the use of C++ Flex-generated lexers was uncommon, maybe even experimental, when I was writing my compiler. Some scaffolding code to move strings between C and C++ formats is present, but it ends up being fairly unobtrusive.

          The compilation process itself occurs during the parsing process. No overall data structure is built to represent the program; rather, the code necessary to emit the proper PIC assembly language runs as the corresponding tokens are parsed. A compilerstate type, which is used as a singleton, helps facilitate this. This struct is declared in file "state.h". This somewhat simplistic single-pass compilation design results in fast throughput, and also helps limit resource utilization.  

          The compiler code was written quickly, but with legibility in mind. It has not, however, been optimized for speed, or to conserve resources. Some aspects of the compiler, in particular the use ofstd::string, could be implemented in considerably more optimal fashion. Nevertheless, the compiler seems to perform well. On the main development system I use, to build "bot.frost" takes about 10 seconds, and about half of this time is attributable to MPASM.  

          Compiler Build Process  

          The build process for "frost.exe" starts with a UNIX-based Flex/Bison build script ("frostcompiler/fmak") that yields some auto-generated C++ and C source code files. The author ran this step in a Cygwin BASH shell session, with the working directory set to the "frostcompiler" folder, but any BASH-like shell capable of running Flex / Bison should work. This step is followed by a MinGW-based C/C++ build step that incorporates  these generated files together with some non-Bison / non-Flex C++ source code files related chiefly to the Frost preprocessor. This step is executed by running Windows batch file "winfinalmake.bat" directly from Explorer. 

          Tokenization  

          The Flex input file "frost.l" uses regular expressions to tokenize the user's incoming Frost source code. The format of this file is such that C code is isolated within curly brace pairs, and the remainder of the file outside the curly braces contains regular expressions.

          In some cases, a particular regular expression will result in the return of a token name selected by the author. These are capitalized in the Frost compiler code base. They break the Frost source code into its most basic constituent parts: identifiers, tokens like *;, and >, and so on. A trivial example from "frost.l" is shown below: 
           [\;] {printf(yytext); return SEMITOK;}    
          This basically says that when a semicolon is found in the source, it should be printed to stdout, and assigned token SEMITOK. Here is a more complex example: 
          [e][v][e][n][t][ \t\n\r]+ {printf(yytext); return EVENTTOK;}  
          The syntax above means that when event is found in the source, followed by at least one whitespace character, then token EVENTTOK should be assigned. Again, the conforming portion of the source code is printed to stdout

          The sequence below defines the rules for a Frost identifier: 
          [a-zA-Z][a-zA-Z0-9\-]*    {printf(yytext); yylval=strdup(yytext); return IDENTTOK;} 
          In summary, the first character must be a letter, of either case. After that, letters, numbers, and the dash are permissible. 

          Below is a complete list of the tokens defines in "frost.l":
          • SEMITOK - the semicolon (;
          • DEETOK - the letter "d" (in function definitions)  
          • TTETOK  - the letter "t" (in table definitions) 
          • ISTOK - the word "is" (in function definitions)  
          • IFTOK - the word "if" (in conditionals)   
          • EVENTTOK - the word "event" (in event handler definitions)    
          • DOTTOK - the dot character (.)  
          • LPARENTOK - the left parenthesis ((
          • RPARENTOK - the right parenthesis ())  
          • IDENTTOK - an identifier, such as indexloop-index, or i 
          • NATNUMTOK - a non-negative integer, e.g. for an event signature, or for the stack 
          • NUMBERTOK - other numeric literals, i.e. a real number or a negative integer 
          • ATTOK - the "at" sign (@)  
          • ASSIGNTOK  - the assignment character (>
          • COMMATOK - the comma (,) 
          • STARTOK - the asterisk (*
          • QUOTTOK  - one or more characters between single quotes
          Finally, in analyzing "frost.l", realize that whitespace characters are all assigned to a single rule which just discards them. However, some of the definitions incorporate whitespace characters. EVENTTOK, for example, incorporates one or more trailing whitespace characters. This allows for other uses of the word "event" within another identifier, e.g. event-body (a real example from "bot.frost").

          Context-Free Grammar  

          If one takes the portion of "frost.ypp" that lies between Bison's %% designators, and removes the C++ code present between curly brace pair, the result is a BNF grammar for the Frost language, in its pure, preprocessed form. This grammar, shown in its entirety below,   
           prog: vents unity defs ;
           
           vents: | vents ventit ;
           
           ventit: EVENTTOK proc ISTOK unity SEMITOK ;
          
           proc: NATNUMTOK | IDENTTOK ; 
           
           defs: | defs defit ;
          
           defit: DEETOK DOTTOK IDENTTOK signature ISTOK unity SEMITOK | 
           TTETOK DOTTOK IDENTTOK tcollect SEMITOK ;
           
           tcollect: | tcollect tabit ; 
           
           numbv: NUMBERTOK| NATNUMTOK ; 
           
           tabit: numbv | QUOTTOK ;
           
           signature: | STARTOK | NATNUMTOK NATNUMTOK ; 
           
           unity: | unity unit ;
           
           unit: fcall | QUOTTOK | numbv | ATTOK IDENTTOK | 
           ASSIGNTOK IDENTTOK | conditi | IDENTTOK ; 
          
           conditi: IFTOK unity COMMATOK unity COMMATOK unity COMMATOK ; 
           
           fcall: DOTTOK IDENTTOK LPARENTOK unity RPARENTOK ;
          
          This should look familiar to anyone who has learned the basics of Frost. Taking it from the top, prog represents the whole program. It has an events section (vents), a main task (unity; this name stands in for any executable block throughout the BNF), and a section where functions and tables are defined (defs). The vents section, in turn, can evaluate to nothing, or to one or more ventit sections. Each of these is an event handler, which starts with EVENTTOK, ("event" plus whitespace, as we saw earlier) then has a numeric or identifier-based name, then "is", then executable code, then a semicolon. The remainder of the BNF proceeds in similar fashion, using a few other tokens from "frost.l", such as STARTOK (*), ASSIGNTOK (>), and ATTOK (@).

          Above, definitions like those of unityventit, and defit, exist purely to allow for the existence of zero or more instances of a counterpart (unitvents, or defs, respectively). Consider, for example, this rule: 
          defs: | defs defit ;   
          This means that defs, the definitions section of the Frost program, can either consist of nothing (from the left side of the pipe character), or of another instance of defs followed by an instance of defit, which is a single function declaration.

          In this way, the entire section can evaluate to nothing (which is a defs), or to a single function definition (which is a defit, and would have to be preceded by an empty defs), or to two function definitions (in which case the preceding defs is not empty). In this way, defs can equate to any number of function definitions, even zero. Similarly, unit is used in conjunction with unity to allow any number of executable statements (chiefly function calls) to be strung together, and ventit is used together with vents to allow for zero or more event handlers in a program. 

          The backbone of the Frost code is unit. This can evaluate to any of the basic structures of the Frost language; the rest of it (event handlers, functions, the main task, etc.) is mostly scaffolding aroundunit.  The definition of unit is shown below:  
          unit: fcall | QUOTTOK | numbv | ATTOK identv | ASSIGNTOK identv | conditi | identv ;    
          Taking each possibility in order, we have: 
          •  fcall - a function call, which will start with a dot 
          •  QUOTTOK - one or more characters between single quotes, for emission onto the main stack
          •  numbv - any numeric literal, integer or real
          •  ATTOK followed by an identifier - the @ operator (static dereference) 
          •  ASSIGNTOK followed by an identifier - the > operator (static assignment)
          •  conditi - a conditional (if) structure
          •  identv - an identifier; in this context, this is an MPASM bit or register name 
          The definition of signature is another particularly revealing one:

          signature: | STARTOK | NATNUMTOK NATNUMTOK ; 
          Taken from left to right, the options are
          • Nothing; a function signature can be empty 
          • The asterisk; parameters are used, requiring a base pointer; but no signature should be enforced
          • Two natural numbers, the parameter byte count and the return byte count 

          Directives 

          The BNF grammar given in the last section does not complete describe all of the code structures that can be present in a valid Frost source code file. Macros are the most obviously missing thing. In addition, there are preprocesser directives that are outside the Bison / GNU grammar given above. These are mostly parsed by code in file "nonbison.cpp" and they are listed below: 
          • stack is sets the size of the main stack, e.g. stack is 10 assigns a 10-byte size 
          • calls instructs Frost.exe to incorporate a HLOE function, e,g, calls divf,eq
          • altstack is sets the size of the second stack, e.g. altstack is 10
          • var declares a static variable, e.g. var setf, setg  
          The default main stack size is 80 bytes, and the default second stack size is 20 bytes. Program "bot.frost" uses an 80-byte second stack, since this was the size used (and extensively tested) in Scalable Processor Arrays for Cybernetic Control

          Minimalist Lexer Template 

          Most of the lexing and parsing associated with Frost features that reside outside the context-free grammar are handled by way of a minimalist lexer template in file "lillexer.h". This declares a single function, lillex(), whose signature is shown below: 
          template<typename parm_type,typename language_type> 
           void lillex
           (
            std::string &cbuffer,
            parm_type &parm, 
            language_type language,
            void (*whitespace)(std::string &)=0
           )    
          From top to bottom, the function parameters above are 1) the input source code, 2) an instance of type parm_type, which is specific to each grammar for which lillex() is applied and is likely astruct type, 3) language, a key-value map of token definitions to handler function pointers, and 4) an optional pointer to a whitespace-removal function.  The next snippet of code shows one example of the application of lillex()
           postprocessordata ppo;
           map<string, bool (*)(postprocessordata &p) > language;
           language[""]=nopush; 
           language["PUSH "]=foundpush; 
           language["PUSH\t"]=foundpush; 
           language["PUSH\n"]=foundpush; 
           ofstream outcode("target.asm");
           ppo.file=&outcode;
           ppo.buffer=incode;
           lillex(ppo.buffer,ppo,language); 
           outcode.close();  
          The function of the code shown above is to search for situations where a PUSH (assembly language stack macro) in the object code output being optimized is immediately followed by a POP. Such push/pop pairs end up leaving the system in an unaltered state, and are therefore removed from the object code in the step shown above. 

          This last bit of code begins by creating an instance of postprocessordata, which will be passed into all of the handler functions set up for the lillex() function in the language parameter. Then, language itself is created; as promised, it maps string token definitions to handler function pointers. These handler functions accept an instance of the postprocessordata type, and return a booleanvalue (true is returned when parsing is completed).  The handler function set up in the statement language[""] runs if no other token is matched at any point in the code, i.e. it is a default case. This is just a convention of the minimalist lexer template. 

          The map held in language is then set up by the code shown above. Note that the key strings are not regular expressions; they must be matched exactly in order for the handler function to get called. That is why three different definitions of PUSH are necessary above, even though they all point to foundpush().

          The lillex() template is a very loose framework and the I/O associated with parsing must mostly be set up by the code that is using "lillex.h". Above, the stream instantiated using variable outcode is written to by the handler functions set up in language. As a result, this stream is included in the postprocessordata instance that gets passed into each handler. This is just one of many ways to uselillex(), though. It is not necessary to write output to a file, or even, strictly speaking, for the parameter type to be a struct

          FAR_CALL Optimization 

          Another optimization is effected using lillex() as well. Each function call is bracketed by bank selection code. The highly segmented nature of the PICs dictates this. Macro FAR_CALL is a part of the HLOE infrastructure (in "kernel.inc") that facilitates this: 
          FAR_CALL macro caller_os,func_os 
           pagesel func_os
           call func_os
           pagesel caller_os 
           endm    
          The first pagesel adjusts the currently active code storage page to the value expected by the function. The second pagesel returns to the caller's active code storage page. But in cases where consecutive FAR_CALLs occur (which is very frequent in Frost object code), this second pagesel is superfluous. The value it writes into the page register is immediately overwritten and never used. So, "kernel.inc" also supplies HALF_FAR_CALL:
          ;Call destination residing within a different code page... does not restore the caller's
          ; code page after return, which may be OK (e.g. if the next operation is another 'Far'
          ; function call). 
          HALF_FAR_CALL macro func_os 
           pagesel func_os        
           call func_os
           endm 
          This second macro enables faster, briefer code like that shown below, which is taken from the object code generated from "bot.frost": 
           HALF_FAR_CALL clearbit 
           HALF_FAR_CALL clearbit
           HALF_FAR_CALL setbit
           HALF_FAR_CALL setbit
           HALF_FAR_CALL clearbit
           FAR_CALL main_setup,clearbit   
          Ultimately, Frost object code resembles this sequence whenever multiple calls occur in sequence. However, this is done using a two-step process. The initial pass made by Bison results in simple sequences of FAR_CALLs, and lillex() is used to optimize this result.  

          Kernel Memory Allocation  

          The primary function of "frost.exe" is, of course, to compile Frost code. However, part of the overall Frost build process is to incorporate the necessary assembly language code for the necessary HLOE "kernel" functions. This actually involves a parsing / transformation process of its own; HLOE "kernel" code is not written in ".asm" files, but in modified assembly language ".hloe" files that are slightly different. These end up getting translated into Microchip's assembly language, and incorporated into the object code monolith "target.asm". 

          The existence of ".hloe" files relates to the existence of an automated system to allow safe, concurrent (main task / event handler) access to static variables from the HLOE "kernel" functions. HLOE "kernel" code, by design, is procedural in nature, and achieves its goal of being small and fast by eschewing the abstractions that characterize "Frost" code.

          At the same time, Frost and HLOE set high standards with respect to safety for concurrency, and the key role of the ".hloe" to assembly language translation process is to abstract away many of the concurrency issues associated with the use of static allocation.  

          Function Families 

          Furthermore, the kernel's static allocation architecture contains features designed to facilitate the sharing of static memory locations between multiple functions, using different names.  

          This system of sharing relies on the creation of a series of function families, each of which shares a single set of static memory locations. Functions within a family must not call other functions in this same family, which would end up reusing the same static locations. Management of these issues is the responsibility of the developer of HLOE "kernel" code; the benefits of doing so are efficiency and correctness. 

          One such family includes the graphy and graphx "kernel" functions. It has the name aart. The two members of this function family make use of the static declarations shown below: 
          ansiadt udata
          aart00 RES .1
          aart01 RES .1
          aart02 RES .1 
          The names aart00aart01, and aart02, of course, are not ideal for actual development. So, before each function that participates in one of these static allocation families, one will see a variable definition section based on the #define directive. The beginning of the graphy function, including the variable definition section, is shown below: 
          ansiaff CODE
          #define flgg3 aart00
          #define vert aart01
          #define cont aart02
          graphy:
           ; Function body...
          This system of statics, as mentioned, relies on the fact that, for example, graphx does not call into graphy, even indirectly through a third function. This allows each function instance in the family to have unfettered access to a shared static data store, from call to return, while still allowing for this data store to be efficiently shared with other code.

          Of course, at any point in time a HLOE "kernel" function can be interrupted by the event handler, and the event handler will potentially make calls into the function family. This situation is handled by ensuring that function call instances that run during the execution of the interrupt handler use their own set of static memory locations.

          Such second sets of memory locations, though, are only used for function families called by both the main task and the ISR (interrupt service routine, i.e. an event handler). Function families that do not get called from both of these portions of the code do not require such protection.  

          The most basic HLOE functions, including single-byte operations like muldivu, and setbit, use the group of statics shown below this paragraph. These locations are termed the BLSS, for "Bottom-Level Static Storage". The BLSS family of functions represents the core HLOE library, and its members are called extensively by both the main task and the ISR. In writing other "kernel" functions, the availability of the BLSS functions is assumed. 
          ukernl udata 
          
          hllblss00 res 1    
          #ifdef HLLMULTITASK
          #ifdef HLLGUARDhllblss
          hllblss00isr res 1      
          #endif
          #endif
          
          hllblss01 res 1    
          #ifdef HLLMULTITASK
          #ifdef HLLGUARDhllblss
          hllblss01isr res 1      
          #endif
          #endif
          
          hllblss02 res 1    
          #ifdef HLLMULTITASK
          #ifdef HLLGUARDhllblss
          hllblss02isr res 1      
          #endif
          #endif
          
          Above, note that the actual declaration of the second set of variables is contingent on a function family-specific compile-time constant. Here, that constant is HLLGUARDhllblss.  In the previous example, from the ANSI libraries, the constant would be HLLGUARDansiart. This system of constants allows Frost.exe to selectively disable variable-cloning, e.g. for variable families that are not called by both the event handlers and the main task. 

          The other kernel functions are free to assume that the BLSS can be safely called from both the ISR or the main task, subject only to inherent hardware limitations. This is evident in the last set of declarations shown. Consider what happens, for example, when HLLMULTITASK is defined, as it is here, indicating that interrupts are in use. In this case, not just three static locations (hllblss00,hllblss01, and hllblss02), but six, are allocated. The three main task locations just listed are augmented by ISR-specific locations named hllblss00isrhllblss01isr, and hllblss02isr.

          Each of these resides one byte after its main task analog, and this layout is relied on in the implementation of the functions that use the BLSS data store. Specifically, these functions operate on one set of static locations if called from the main task, and the other if called from the ISR, ensuring that the promised level of safety is provided by the BLSS functions. 

          One example BLSS function implementation is shown below. This is function clearbit. In addition to the #define directives associated with shared statics, some key decision logic, based around variable in_isr, is evident: 
          ;Tool-generated code
          
          #define margp2 hllblss00
          
          clearbit:  
          
          #ifdef HLLMULTITASK
          #ifdef HLLGUARDhllblss 
           movf in_isr,f  
           btfsc STATUS,Z 
           goto clearbit0
          
           ; Function body (main task copy); Not shown here
          
          #undefine margp2
          #define margp2 hllblss00+1
          
          clearbit0:
          
           ; Function body (ISR copy); Not shown here
          
          #endif
          #endif
          
          Here, note that two copies of the actual function body exist in parallel. These bodies are replaced by three-line comments in the fragment shown above. These body sections are identical to each other, except that label names must be different. The way in which #define is used in the code shown above ensures that the ISR reads and writes static location hllblss00+1 while the main task uses hllblss00. In both cases, the clearbit code refers to this location as margp2, which was a name judged meaningful (at least, more so than hllblss00) during the development of clearbit

          Most importantly, the reader should note that the scaffolding and duplication shown in the last code sample is automatically generated; the existence of such mechanical-looking structures is the main reason for the existence of ".hloe" files distinct from (otherwise-similar) ".asm" files.  The next code sample shown below is from a ".hloe" file. In fact, this is the entire content of "sfpcopyf.hloe" (home of the copyf routine):  
           GLOBAL copyf
          sfp_copy CODE 
          HLTHUNK sfploc,copyf,mym,mye
           POP
           banksel mye ;Save arg
           movwf mye
           POP
           movwf mym
           PUSH
           movfw mye
           PUSH
           movfw mym
           PUSH 
           movfw mye
           PUSH
           return
           HLEND  
          It is the HLTHUNK keyword and its HLEND terminator that make this different from Microchip's assembly language. HLTHUNK is followed by a comma-delimited list. The first item in the list is the variable family name, and the second is the function name. Then, there is a list of static variable names. If a function does not use statics, it should be coded in ordinary assembly language, without HLTHUNK/HLEND

          In the last code fragment, the copyf function relies on static locations mym and mye. These are allocated from a pool of static memory locations associated with the name sfploc

          The implementation code between HLTHUNK and HLEND will be cloned, if necessary, as will the storage locations that underlie mym and mye.  

          The beginning of the automatically-generated ".asm" file that results from "sfpcopyf.hloe" is shown below:
          ;Tool-generated code
          
          sfp_copy CODE 
          
          #define mye sfploc00
          #define mym sfploc01
          
          copyf:  
          
          #ifdef HLLMULTITASK
          #ifdef HLLGUARDsfploc 
           movf in_isr,f  
           btfsc STATUS,Z
           goto copyf0
          
           ;Etcetera... 
          Note that the actual declarations of sfploc00sfploc00isr, etc. is not automated. This is handled in the ".hloe" files. without the aid of automation, e.g.
          sfpu UDATA
           GLOBAL sfploc00,sfploc01,sfploc02,sfploc03,sfploc04,sfploc05,sfploc06,sfploc07,sfploc08
           GLOBAL sfploc09,sfploc10,sfploc11,sfploc12,sfploc13,sfploc14,sfploc15,sfploc16,sfploc17 
           GLOBAL sfpaux00,sfpaux01,sfpaux02,sfpaux03
           
          sfploc00 res .1
          sfpaux00 res .1
          #ifdef HLLMULTITASK   
          #ifdef HLLGUARDsfploc
          sfploc00isr res .1
          sfpaux00isr res .1
          #endif
          #endif
          sfploc01 res .1
          sfpaux01 res .1
          #ifdef HLLMULTITASK
          #ifdef HLLGUARDsfploc
          sfploc01isr res .1
          sfpaux01isr res .1
          #endif
          #endif
           ;Etcetera... 

          Kernel Modularity 

          The last fragment of code shown is from file "sfp.hloe". The memory locations defined belong to the sfploc function family (which consists of all the operators and conversion functions in the SFP library). All of these functions thus depend on "sfp.hloe", i.e. "sfp.hloe" must be parsed and built into "target.asm" if these functions are to compile and work correctly.  

          One feature of HLOE is that the dependency tree implied in the last paragraph is built into the names of the ".hloe" files. The SFP multiplication operator is in file "sfpmulf.hloe", for example. The Frost build system relies on this fact. File "sfpmulf.hloe" by convention depends on file "sfp.hloe". File "ansiartgraphx.hloe" holds the graphx function and relies on "ansiart.hloe", and does "ansiartgraphy.hloe", and so on.

          Files are generally single-function, for reasons of modularity. If working with a more powerful target platform, it might be practical to just put the whole SFP library into a modular unit; even in C, "stdio.h" is basically monolithic. Not calling strstr, for example, does not necessarily prevent strstr-related costs from being incurred.  Here, functions are designed to be freestanding units as often as possible. 

          Kernel Portability / Extension  

          Unlike the Frost libraries supplied in the "frostcompiler" folder, none of the HLOE files provided are processor-specific or board-specific.  In fact, processor-specific and board-specific HLOE files are not supported by the compiler.

          In many cases, Frost is useful as a systems programming language, and the processor / board - specific code should be moved up to the Frost level. In other cases, generally applicable HLOE files make use of processor / board - specific .INC files. 

          In particular, if you are writing your own HLOE files and need processor / board - specific assembly language sections, "kernel.inc" (and its more specific versions) is the appropriate location for these, where they should take the form of macros. If you add your code to the bottom of the file, then it can employ useful, preexisting architecure-specific constructs like POP.

          Frost.exe Command Line 

          The Frost compiler executable accepts several command line arguments. These can be placed anywhere in the argument list:   

          NOBOUNDS : Omit stack overflow checks; normally, stack overflow is detected and results in the programming entering an infinite loop, during which the exclamation point character is emitted repeatedly to the serial output (if so configured).  
          CANONIZE : Convert Frost code to standardized format in compiler output. 
          BOARD : Set board (see "Configuration Management). 
          PROC : Set processor (see "Configuration Management).
          The build script from "frostbotproject" uses several of these options:
          %frost%frost.exe file bot.frost proc 16f690 board robot nobounds 

          Comparison with FORTH    

          The similarity between Frost and the FORTH programming language was mentioned briefly in Minimalist Floating Point Type. Some specific commonalities are listed below:


          • Use of main and auxiliary stack; both languages have a main stack with which most operations work, and a second, system stack that is also accessible using library calls. Both languages avoid infix notation and instead use stack-based postfix syntax.   

          • All-encompassing runtime environment; both languages basically oversee the entire target system, as opposed to having programs run in some external context like an OS.  
          • Type system; both languages are very flexible in this regard, e.g. they basically omit type checks. 

          • Target domain: both languages are designed for low-level / mission critical applications. FORTH is know for its application in such things as device drivers and space vehicle firmware. The predictability and comprehensibility of Frost has been amply discussed here and in Scalable Processor Arrays for Cybernetic Control

          However, there are also some key differences between Frost and FORTH. Unfortunately, FORTH has a reputation for being a bit difficult to maintain. Many of the differences between Frost and FORTH represent attempts to rectify this issue.

          In particular, the author hopes that unadorned stack manipulation is less prominent in Frost than it was in FORTH. The functional programming perspective is an addition in Frost that should redirect development style away from the raw stack and to the function calls that it facilitates. Of course, all of the direct stack manipulations are still present in Frost; but the overall perspective of the developer is intended to be one of functions passing each other parameters, not one built around a main stack.

          Finally, FORTH is a more mature and full-featured language than Frost. FORTH offers a richer set of data types than Frost can. It has been modified and extended in response to real-world feedback; people have attempted things like object-oriented FORTH, for example.

          More practically, a variety of schemes for FORTH concurrency exist outside the official language standard. The approach to concurrency used by Frost, in contrast, is to provide formally defined support for a lower-level, event-driven model of concurrency. 

          Future    

          An IDE would make Frost a more viable alternative to C or assembly language. There are also some features that are arguably missing from the language proper . One thing that is lacking is support for more sophisticated data structures. The reliance on the stack, and the lack at this time of anything like a reference into a heap, is an obstacle for many applications.

          How big an obstacle this turns out to be depends on the nature of the system being developed. The lack of a heap did not seem problematic to the author during the development of the robotics proof-of-concept provided with the article, and it seems likely that Frost is already fairly well-suited to control loop-based applications like the one in "bot.frost".  

          In addition, it must be admitted that the Frost language lacks a few things that would make it friendlier for developers already accustomed to other high-level languages. The absence of dedicated looping structures, like while or for, does not limit Frost in any fundamental way, but it is definitely a potential point of confusion.

          Similarly, infix notation is something most developers likely expect; it would be mere syntactic sugar, but it would make Frost more accommodating. The absence of these elements of syntactic sugar mostly reflect time limitations, and their impact on a single developer trying to bring an expansive project to some form of completion.  

          As with any of my articles, the future of Frost will be dictated by the readers of this site. If there are future additions, corrections, or changes, these will be made in response to the deficiencies and possibilities brought up in reader commentary.  

          License


          This article, along with any associated source code and files, is licensed under The GNU General Public License (GPLv3)

          References       

          • [1] Michaelson, G. An Introduction to Functional Programming through Lambda Calculus. Addison-Wesley, Wokingham UK: 1989.    
          • [2] D'Souza, S.; "Application Note 556: Implementing a Table Read". Microchip Technology, Inc., Chandler, AZ, USA: 2002.    

          Footnotes     

          1. The latest version of the language, C11, offers some new native features for dealing with concurrency.     

          2. Here, the term "statement" is used for code that evaluates to actual executable object code, and "directive" is used for everything else. In the first category are such things as function calls and literal values. In the second are macro declarations, calls directives, and configuration settings like stack is.   

          3. It might also be possible to correct the invalid, negative elapsed time result, e.g. by adding 256 to it.  in practice, the ISR's execution time is a long, relatively unchanging value that dominates the calculation of the real elapsed time. So, the use of a constant is a fast and simple approach that does not compromise accuracy.  

          No comments:

          Post a Comment