Exploring Retro68 GCC-based SDK for 68K Macintosh computers

UPDATE: The 68k development environment with Retro68 together with CodeLite and pce/macplus emulator running on Ubuntu is now ready for download as a VirtualBox image. See my latest post for details.

Being a fan of vintage Apple computers, I have always been looking for a modern way to build apps for 68k Mac directly from laptop without having to resort to ancient tools like Macintosh Programmer’s Workshop or Think C. Luckily my dream is realized when I came across Retro68, a GCC-based 68k cross-compiler that seems to be feature-rich and easy to use. I spent a few weeks exploring Retro68 various features and this article will share my findings.

What is Retro68 all about?

Developed by Wolfgang Thaller, Retro68 allows you to compile your existing 68k C code, with only minor modifications using GCC from your modern OS and run the generated binary files on your 68k Mac. After compiling the code, Retro68 will create a MacBinary image (.bin), a disk image (.dsk) as well as a Mac application package with resource fork (.APPL) containing the compiled executable which can then be copied over to your favorite 68k machine for testing.

Retro68 is written in C and can be compiled for various platforms such as Linux, Mac OS and even Windows (with the help of cygwin), although Mac-compatible APPL application packages will only be generated if Retro68 is running on a Mac from a HFS/HFS+ compatible volume. The following sections will describe the steps needed to compile Retro68 on Ubuntu 16, my favourite Linux distro.

Compiling Retro68 from source

First perform a git clone of the Retro68 github repository. Next, copy the CInludes and RIncludes folders from your Macintosh Programmer’s Workshop (MPW) installation to the main Retro68 folder. If you do not have a copy of MPW, you can download the header files for MPW v3.5 here. After that, install the packages required by Retro68 using the following:

sudo apt-get install cmake libgmp-dev libmpfr-dev libmpc-dev libboost-all-dev bison autoconf automake texinfo

Verify that all packages have been installed successfully. In particular, the autoconf package is critical. If it is missing, compiling Retro68 will fail towards the end of the build process (which may take between 30-60 mins even on a fast computer). The output may also display some cryptic error messages that could confuse you for a while.

Next, create a folder named Retro68-build in the parent directory where the Retro68 SDK is downloaded. Also inside the parent directory, create a start_build.sh script with the following content:

rm -rf Retro68-build
cd Retro68-build
sh ../Retro68/build-toolchain.sh

Take note that this bash script will remove any previous build output before starting the build process. Now execute the script and be prepared to wait between 30 to 60 minutes for the build to complete. If the process is successful, there should be no error messages at the end and you will find a few sample applications under Retro68-build/build-target/Samples:

1. Dialog: a simple application showing a Dialog with various buttons
2. HelloWorld: a console-based Hello World application
3. Raytracer: demonstrate the use of various graphics funtions

In my experience, the Raytracer application may fail to build due to some missing header files. As this problem does not affect the functionality of Retro68, you can work around the issue by replacing the content of the Raytracer folder with the HelloWorld application and editing CMakeLists.txt to keep the same target name, e.g. ‘Raytracer':

add_application(Raytracer
	hello.c
	CONSOLE
   )

UPDATE: This issue has been fixed in the latest release of Retro68 (dated 12 April 2017) and the RayTracer sample application should build properly.

With this modification, Retro68 should finish building with no issues. You can now try the sample apps by copying the MacBinary files (.bin) file to your favourite machine, and extract them to show the original executable files.  The following is the screenshot of the sample Dialog application running on pce/macplus, a 68K Macintosh emulator:

The following is the HelloWorld sample console application, also running on pce/macplus. The app was built using libRetroConsole, a library by Retro68 that supports basic console functionality. There was no support for developing console application in the original Macintosh design as apps were supposed to be GUI-based.

Retro68 also generates a 1.44MB HFS disk image (.DSK) that can be written to a 1.44MB floppy disk (for example by using dd) to facilitate copying of the compiled application. On older Macs that do not support 1.44MB floppies, you can use the MacBinary output (.bin) for file transfer instead.

Code structure

Despite being based on GCC, a modern 32-bit compiler, C codes meant for vintage 68k development tools such as MPW or Think C should compile just fine under Retro68, with some modifications. Take a look at the Dialog sample application, which contains just 3 files:

• dialog.r: the text resource file
• dialog.c: the main application file
• CMakeLists.txt: build rules to compile the application

The CMakeLists.txt file contains just the following

cmake_minimum_required(VERSION 2.8)

add_application(Dialog
	dialog.c
	dialog.r
	)

Adding more .c files to the application requires modifying the make file to include these new files. Obviously it is not necessary to include the header files (.h) in the make file, however.

To build the application using the CMakeLists.txt file, run the following bash script:

rm -rf build
mkdir build
cd build
cmake .. -DCMAKE_TOOLCHAIN_FILE=./Retro68/Retro68-build/toolchain/m68k-apple-macos/cmake/retro68.toolchain.cmake
make

The output will then be created in the build folder.

For those who are new to 68k programming, the Dialog.r file is a text resource file which defines various GUI elements using a C-like syntax:

resource 'DLOG' (128) {
	{ 50, 100, 240, 420 },
	dBoxProc,
	visible,
	noGoAway,
	0,
	128,
	"",
	centerMainScreen
};

Retro68 uses Rez, an implementation of Apple’s Rez resource compiler which reads the .r files and compiles them to binary resource files whose names look like ‘dialog.r.rsrc’. These resource files will then be bundled together with the binary output from GCC to create the final Mac application.

With this design, most of the sample codes from MPW or even Think C should compile fine under Retro68 after changing CMakeLists.txt to include all required source files. Take note of the changes in various method names between MPW 3.1 and MPW 3.5, which may result in the need to modify parts of the code. For example,  DisposHandle is renamed DisposeHandle and AddResMenu is renamed AppendResMenu. Use the following code snippet to assist you with the changes:

#define ResetAlrtStage ResetAlertStage
#define SetDItem SetDialogItem
#define GetDItem GetDialogItem
#define GetItem GetMenuItemText
#define DisposHandle DisposeHandle
#define AddResMenu AppendResMenu
#define DisposPtr DisposePtr
#define ResrvMem ReserveMem

The text resource format has also been changed, resulting in cryptic error messages (for example, incorrect number of parameters) when using an older resource file. To fix this error, check the various resource header files in the RIncludes folder as well the sample applications from MPW 3.5 and find out the expected format.

Code segmentation

The original MPW compiler used 16-bit offsets and made uses of #pragma segment to organize the codes into different segments, the names of which need to be configured into the linker phase, to be loaded/unloaded dynamically at run-time as and when the respective code is needed. This results in complicated code and causes the generated application to contain multiple segments under its CODE resources, each of which smaller than 32KB. The following is the CODE resource of Eudora 1.3.1 when displayed under ResEdit:

Retro68, on the other hand, uses GCC which has 32-bit absolute addresses and does not need to stick faithfully to that part of the 68K Mac architecture. It simply places most of the code into a single segment and uses some custom code to make all those addresses point to the right places once the code is loaded. The Retro68 code responsible for this can be found in libretro/relocate.c.

The following is the CODE resource of the Dialog application compiled using Retro68:

There are only two segments, and most of the code can simply be put in CODE 1. The MPW architecture of segmenting via #pragma segment is no longer needed, and will simply be ignored by GCC.

On a side note, if your CODE resource has an extra segment with ID = 256, like in the following screenshot, your 68k Mac OS installation is most likely infected with the nVIR virus and you should use a virus scanner software such as Mcafee Virus Scan 101 to clean up the infection. In my case, the virus could have come from one of my downloads on MacGUI. Applications created using Retro68 should only have two code segments having 0 and 1 as their IDs.

Here comes a trap for new 68K programmers. I was happily using Retro68 to build my application, adding more and more code, when suddenly the following error message appeared immediately after launching the application:

The error message read, ‘The application has unexpectedly quit (out of application memory)”, under System 6.0.8 and System 7.5.5. It changed to, “The application has unexpectedly quit, because an error of type 15 occurred”, under System 7.6.1:

Error of type 15 is “Segment Loader Error” according to this, which perhaps indicated the same thing, out of memory while loading the application. Clearly memory was not an issue, so what had gone wrong?

The root cause was simply some hard-coded resource values which declared the memory requirements of the application. These values are used by Finder to allocate the amount of memory required by the application. In my case, my Sample.r text resource file had a section with the following code:

resource 'SIZE' (-1) {
...
kPrefSize * 1024,
kMinSize * 1024
}

where kPrefSize and kMinSize were declared in the header file as follows:

#define kMinSize 23
#define kPrefSize 35

These values mean that the application needs a minimum of 23KB, and prefers at least 35KB of memory to work properly. When more codes are added without these values being changed, the app will no longer work properly because Finder won’t allocate enough memory for it. Setting the above values to at least 100KB each fixed the issues for me.

Macintosh 512K support

After testing the generated binaries on my Macintosh SE running System 6.0.8, I tried to run the compiled app on my Macintosh 512K with Mac OS 3.2 (Finder 5.3) and received the message “A system error has occurred” with ID = 10 indicating that the app attempted to access a non-existent ROM routine or something like that. After some debugging, it seemed that the error happened even if the app only contained a single return; in the main() method, so the problem must have been somewhere inside the bundled codes that were called even before execution reached the main() method. But where exactly?

After some troubleshooting with the author, who was very helpful and provided me with a lot of good information, the issue seemed to be with method Retro68Relocate() in relocate.c. In particular, there are calls made to StripAddress and SysEnvirons, which are not supported on older ROMs:

#define RETRO68_GET_DISPLACEMENT_STRIP(DISPLACEMENT) \
	_RETRO68_GET_DISPLACEMENT(DISPLACEMENT, StripAddress)
...
RETRO68_GET_DISPLACEMENT_STRIP(displacement);
...
SysEnvRec env;
env.processor = 0;
SysEnvirons(0, &env);
if(env.processor >= env68040)
{
    FlushCodeCache();
}

Method StripAddress is used to convert GCC 32-bit offsets to their 24-bit equivalents whereas method SysEnvirons is used to check if the the code cache needs to be flushed on a 68040. None of these 2 methods are supported on the 64K ROM used by the Macintosh 512K. Being new to 68K programming, it took me almost a week to find a solution, that is to read the word value at the ROM85 memory position (0x028E) to see whether the ROM is 64K or 128K, and only call these two methods if the ROM is 128K. On 64K ROM, the following can be used as an alternative to StripAddress, by taking the last 24 bit of the 32-bit address:

#define StripAddress(x) ((Ptr) ((unsigned long)(x) & 0x00FFFFFF))

During the debugging process, in order to recompile only libretro with the modified files without having to compile the entire Retro68 SDK, I used the following script:

make -C ./Retro68/Retro68-build/build-target
rm ./Retro68/Retro68-build/toolchain/m68k-apple-macos/lib/libretrocrt.a
cp ./Retro68/Retro68-build/build-target/libretro/libretrocrt.a /Retro68/Retro68-build/toolchain/m68k-apple-macos/lib

The script forces a rebuild of the sample applications, which will then rebuild libretro and other related libraries. After that, the output file libretrocrt.a is copied to the correct folder expected by the m68k-apple-macos toolchain. This is needed otherwise the build will continue to use the old libretrocrt.a library file for building.

I submitted the above code change to the author, who integrated it to the latest release of Retro68 (dated 12 April 2017). With this, apps compiled using Retro68 should work fine on the Macintosh 512K (or even the Macintosh 128K), provided that they do not make any other function calls not compatible with old ROMs.

An issue which remains is that console apps built using Retro68 will not work on the 512K. The compiled app size is bigger than the 800K floppy disk (almost 912K due to the added library size), and even with the HD20, it will fail to run (not enough memory). According to the author, the culprit is libstdc++ huge implementation of locales, which get pulled in all the time in new versions of GCC, even though those codes are never really used. Unfortunately, fixing this problem seems to be non-trivial for now.

MPW object file compatibilities 

In one of my experiments, I received a linker error the moment I started to use the SerReset function (found in Serial.h) to initialize the serial port. A linker error also happens with OpenDeskAcc used in one of the MPW sample applications. This means that these functions are declared in the header files but not in the object files used during the link process, resulting in the linker errors. This is when I checked the original MPW disks and saw a lot of object files in the libraries folder, among which the Interface.o file contained the SerReset function which I was using. These libraries also contained a lot of other methods commonly used by other applications. Clearly Retro68 did not use any of these libraries. So how could most of the sample applications compile just fine?

I contacted the author and the answer provided was that he cannot reuse any of the MPW .o files because the format is different from GCC standards. Instead, he simply rewrites some of these methods in libretro/glue.c. For example, the OpenDriver method is rewritten using PBOpenSync with the following code:

pascal OSErr OpenDriver(ConstStr255Param name, short *drvrRefNum)
{
	ParamBlockRec pb;
	OSErr err;
	memset(&pb, 0, sizeof(pb));

	pb.ioParam.ioNamePtr = (StringPtr)name;

	err = PBOpenSync(&pb);
	*drvrRefNum = pb.ioParam.ioRefNum;
	return err;
}

Some other functions such as StripAddress do not need to be rewritten in glue.c, since they invoke the ROM functions directly:

EXTERN_API(Ptr) StripAddress(void * theAddress)                               ONEWORDINLINE(0xA055);

Unfortunately, with this implementation, many important API methods (e.g. File Manager, Device Manager or Quickdraw calls that are not declared as ONEWORDINLINE or TWOWORDINLINE) will not work unless glue.c is modified to include those methods, or the MPW .o library files are converted to a format accepted by GCC and included in the linker phase. For now, attempting to use any of the unsupported methods will result in a linker error. This is a known issue of Retro68, with no solutions yet.

Conclusion

Despite the limitations, I still like Retro68 as I can use it on my laptop running Ubuntu 16 to compile 68k Mac apps. Also, Retro68 together with CodeLite and pce/macplus, after some custom configurations, make a very good 68k development environment:

With just a few basic bash scripts to compile the project using Retro68 and run it in pce/macplus, I can comfortably edit my 68k codes using CodeLite. Even auto-completion is supported as well! For those who are interested, I will be preparing a VirtualBox Ubuntu image with Retro68, CodeLite, PCE/macplus and other necessary components allowing you to develop 68k apps the modern way. Stay tuned!

See also

PCE/macplus, the ultimate 68K Macintosh emulator