Overlay Code with Parallax Propeller and GCC

Written April 7th, 2016 by

Programs that use overlay code were very popular in the era of early home computer systems when the amount of memory was not much, and the ability to load portions of code only when necessary allowed to make programs much larger than the available memory.

With microcontrollers we are in a similar situation with sometimes very complex programs and a rather limited memory, just think for example to the management of the SD card file system or the internet access libraries, that can severely limit the memory available for the program itself, especially if they are to be used simultaneously.

The Parallax Propeller is a microcontroller with 32K of internal RAM memory and uses an external EEPROM to load the code to run at power up. Since only the first 32K of the EEPROM are used for the program it is possible to dedicate the exceeding space of larger memories for data storage. Fortunately with the GCC compiler and the standard tools it is also possible to store portions of the program code to be loaded when necessary.

The program

Build as an example a small program that writes two sentences on the serial port using two functions stored in two separate sources, then we will manage them through overlay.

The two source files might look like the following:

	void overlay_function1()
	    uart_print("Written by overlay_function1 !\r\n");

	void overlay_function2()
	    uart_print("Written by overlay_function2 !\r\n");

The program could simply initialize the serial port of the microcontroller, write a header to indicate that it is working and call the two functions above. We also add the source to manage the EEPROM using the I2C bus on the standard pins.

During the exposure I will refer the source code without carrying it in completely to not make the article too long. The attached package contains the complete sample files that can be used as reference.

The linker script

To use the overlays we instruct the linker on which object files are the code and where to place them in the microcontroller memory. To do this we must write a custom script.

We start from the standard script that can be found in the propeller-elf/lib/ldscripts directory with the name propeller.x. Copy the file in your working directory and rename it as propeller_ovl.ld. Open a text editor and load the file.

At the beginning of the script we find the MEMORY section that defines the memory areas in which to place the code and data. We need to define a new area in which to place the overlay code so add a line, like the one shown, just below the line that starts with hub:

  hub     : ORIGIN = 0, LENGTH = 32K
  ovl     : ORIGIN = 28K, LENGTH = 4K
  cog     : ORIGIN = 0, LENGTH = 1984 /* 496*4 */

This line defines a memory area called ovl that begins at location 28762 (28K) at the end of the internal ram with size 4096 (4K) bytes. All object files defined as overlays will be loaded in this area.

Because the area is located in the microcontroller’s ram memory, we must therefore reduce the memory available for the normal program code, so change the hub line with a length of 32K-4K=28K. The first lines of the script will be like these:

  hub     : ORIGIN = 0, LENGTH = 28K
  ovl     : ORIGIN = 28K, LENGTH = 4K
  cog     : ORIGIN = 0, LENGTH = 1984 /* 496*4 */

At this point we must tell the linker which object files make up the overlay code, so let’s move down the script until we get to the SECTIONS section. The first lines of this section in the standard script should be similar to the following:

  /* if we are not relocating (-r flag given) then discard the boot and bootpasm sections; otherwise keep them */
  /* the initial spin boot code, if any */
   .boot : { KEEP(*(.boot)) } >hub
   .bootpasm : { KEEP(*(.bootpasm)) } >bootpasm AT>hub

So we are going to add an OVERLAY section immediately after the opening brace so that it is read before any other instruction:

  /* overlays */
    .ovly0  { overlay0.o(.text .data) }
    .ovly1  { overlay1.o(.text .data) }
  } >ovl AT>drivers

  /* if we are not relocating (-r flag given) then discard the boot and bootpasm sections; otherwise keep them */
  /* the initial spin boot code, if any */
   .boot : { KEEP(*(.boot)) } >hub
   .bootpasm : { KEEP(*(.bootpasm)) } >bootpasm AT>hub

The OVERLAY : NOCROSSREFS line defines the beginning of the section informing the linker that we are defining the overlay code and that each overlay can not call the code in other overlays. In the lines between braces we are going to insert the list of objects files containing the overlay code and data, one per line. In the example above we have two objects files overlay0.o and overlay1.o.

The last line with the closing brace defines the memory area in which the code is executed, the ovl area defined at the beginning of the script, and where each object is physically stored, in the drivers area which corresponds to the EEPROM memory area above 32K.

At this point the script may already be used as the linker automatically produces the required symbols to load the objects, however to further simplyfying things we add a table that can be used to load the code with a simple index.

Go down a bit more in the script until the .data section and modify it as follows:

  .data	  :
    . = ALIGN(4);
    __ovly_table = .; 
        LONG(ABSOLUTE(ADDR(.ovly0))); LONG(SIZEOF(.ovly0)); LONG(LOADADDR(.ovly0));
        LONG(ABSOLUTE(ADDR(.ovly1))); LONG(SIZEOF(.ovly1)); LONG(LOADADDR(.ovly1));
  }  >hub AT>hub

The added rows defines a 2×3 long table called _ovly_table that we can use directly in C with a definition like the following:

extern uint32_t _ovly_table[2][3];

The first element of each row contains the address of the internal ram memory to load the code to, the second element contains the size in bytes of the code and the third element the address where it is stored on the EEPROM.

The last step is to redefine the stack pointer since with the standard script it is placed at the address 0x8000 at the end of the ram memory, in the same area that will be occupied by the overlay code (remember that the stack expands downward).

The last line of the script contains the stack pointer definition, so we are going to change it so it points to the same address as the beginning of the overlay area at address 28*1024=28672 (7000 hex):

  /* default initial stack pointer */
  PROVIDE(__stack_end = 0x7000) ;

In this way there are no interferences between the overlay code and the stack.

The script is now complete and we can use it with the linker by adding the -T directive on the command line. In a typical makefile we will have a line like this:

CFLAGS := -Os -Wall
CXXFLAGS := -Os -Wall
LDFLAGS := -s -T propeller_ovl.ld -Wl,-Map=$(NAME).map -fno-exceptions

The -T directive tells the linker to use the propeller_ovl.ld file as the customized linker script.

Overlay load

Now we have a program that compiles correctly, but when it runs it doesn’t show the output we are expecting because the two functions have not yet been loaded into memory. To load the code from the EEPROM we add the definition of the array generated by the linker and a function that loads the actual code:

extern uint32_t _ovly_table[2][3];

void eeprom_load_overlay(int n)
    uart_print("--- loading overlay ");
    uart_print(" from ");
    uart_print_number(_ovly_table[n][2], 16, 0);
    uart_print(" to ");
    uart_print_number(_ovly_table[n][0], 16, 0);
    uart_print(" size ");

    eeprom_read(HIGH_EEPROM_OFFSET(_ovly_table[n][2]), (uint8_t *)_ovly_table[n][0], _ovly_table[n][1]);

In the program’s main function then add the call to the overlay load function just before calling the actual functions:

    uart_print("\r\n*** OVERLAY DEMO ***\r\n\r\n");



Program upload

To upload the program to the microcontroller it is necessary to use the propeller-load program that comes with the gcc toolchain. The program does not require any particular instruction as it is capable of detecting that a portion of the program must be loaded to the upper part of the EEPROM because of the use of the drivers memory area. Make sure to have at least a 64K EEPROM installed:

marco@bridge:~/parallax/overlay$ propeller-load -t -p /dev/ttyACM0 -r demo.elf 
Propeller Version 1 on /dev/ttyACM0
Loading the serial helper to hub memory
10392 bytes sent                  
Verifying RAM ... OK
Loading cache driver 'eeprom_cache.dat'
1540 bytes sent                  
Writing cog images to eeprom
104 bytes sent                  
Loading demo.elf to hub memory
4792 bytes sent                  
Verifying RAM ... OK
[ Entering terminal mode. Type ESC or Control-C to exit. ]


--- loading overlay 0 from C0000000 to 7000 size 52
Written by overlay_function1 !
--- loading overlay 1 from C0000034 to 7000 size 52
Written by overlay_function2 !

The sample output shows the data of the overlays loaded from the EEPROM.


Using programs with overlay code opens to many possibilities. In the attached example we used simple functions but nothing prevents you to have much more complex code, just beware of the limitations that we have now. The memory used by the overlays can not be used by the main program so complex overlay code occupying several K-bytes automatically reduces the available space. The code from an overlay can not call the code from another overlay, this is because it would be necessary to implement a call tracking system to load the correct code greatly increasing the complexity. Loading the code always takes some time so it is advisable to group functions toghether to minimize loads. Even with some limitations we now have the possibility to write programs greater than the 32K of available memory.