What follows is an example of a feature about the work of Professor Patrick H. Stakem.  This was published in issue 37 of Linux
User and Developer Magazine in the early part of 2004.  Back issues should be available from the LUD web site.  Click on the link for that.
There are slight differences from the printed version due to the fact that there is more space to publish here.

Key Links  http://FlightLinux.gsfc.nasa.gov      http://ipinspace.gsfc.nasa.gov/    Beowulf     http://www.scyld.com/  
 International Space Station                              http://www.sheflug.co.uk/featuresoft.htm

Linux is finding substantial use in space vehicles, as Richard Ibbotson
found out when he spoke to NASA's Professor Patrick H. Stakem
Eight Miles High

Richard "How did Linux and open source software become involved with space flight and exploration " ?

Pat  " In several ways. Scientists, familiar with the Unix environment from University days, quickly and seamlessly migrated to Linux. Engineers and programmers, taking advantage of free downloads, adopted Linux on an informal basis. The availability of application software such as OpenOffice.org provided a commonality with existing infrastructure, and the superiority of the Apache web server was realised. In many cases, the use of freely downloaded software helped to overcome a hurdle that would not have otherwise been done, due to budgetary restrictions. The availability of the source code was a major benefit to beleaguered embedded systems programmers, faced with the daunting tasks of developing and integrating custom device drivers in a real-time environment.  Initially, Linux was viewed as, and proved to be a security risk, as a large number of inexperienced end-users became their own sys admins. This was countered by a new series of procedures and mandatory training courses."

Richard  "What was the purpose of the FlighLinux project when it finally started " ?

Pat  "The FlightLinux project had the goal of providing a satellite flight demonstration of the Linux software. The proof-of-concept was done in conjunction with the on-orbit UoSat-12 mission, from Surrey Space Technology, Ltd (UK). This work was conducted with funding by the NASA Earth Science Technology Office (ESTO) Advanced Information Systems Technology (AIST) Program."

Richard "Did the project have to fit in with some sort of commercial project "?

Pat  "As spacecraft onboard computer hardware evolves to adapting existing commercial designs, the logical next step is to adapt commercial, off-the-shelf software, such as the Linux operating system supported by the GNU tool set. Given Linux, many avenues and opportunities become available. Web serving and file transfer become standard features. Onboard networks and an onboard file system become easy. Java is trivial to implement. Commonality with ground environments allows rapid migration of algorithms from ground-based to the flight system, and tapping into the world-wide expertise of Linux developments provides a large pool of talent. Full source for the operating system and drivers is available on day one of the project. We use Linux here in a large sense. Many of the BSD distributions would be equally applicable."

Richard  "Did it take long to get the project together" ?

Pat "An initial build of the FlightLinux software was running in March 2001. The hardware breadboard was available April 2001. The initial software load was approximately 400,000 bytes in size. The nature of the UoSat-12 memory architecture at boot time limited the load size to less than 512 Kilobytes. After the ROM-based loader completes, 4 megabytes of RAM memory is available. Initially, the project was going to use NASA's WIRE spacecraft for the on-orbit demonstration, but this did not prove feasible upon closer examination of the existing WIRE software architecture. The UoSat used the Intel 80386EX processor, with RAM, ROM, and asynchronous and synchronous serial interfaces, CAN bus, and an ethernet LAN connection."

Richard  "Did you need to make any adjustments for the use of standard off-the-shelf Linux software" ?

Pat  "Extensive customization of the SETUP routine, written in assembly language, was required. This routine in its original form relies on BIOS (Basic Input/Output System) calls to discover and configure hardware. In the UoSat configuration, there is no BIOS function, so these sections were replaced with the appropriate code. Sections of SSTL code were added to configure the unique hardware of the UoSat computer. The SETUP routine then configures the processor for entry to Protected Mode and invokes the decompression routine for the kernel. The SETUP routine is approximately 750 bytes in length and represents the custom portion of the code. This product is usually referred to as the Board Support Package. The remaining code is standard Linux software. We also modified the standard Linux SETUP routine to be table-driven. FlightLinux was implemented in an incremental manner. The initial software build does a "Hello, World" aliveness indication via the asynchronous test port and allows login. The synchronous serial ports allow communication in the flight configuration. The bulk memory device driver uses the 32-megabyte modules of extended memory as a file system. The breadboard has a single 32-megabyte module, and there are four modules in the flight configuration. The CAN bus drivers and the network interface were added later. "

Richard  "How did you find that changing from the traditional software model to an Open Source business model" ?

Pat  "The FlightLinux Project explored new issues in the use of "free software" and open-source code, in a mission critical application. Open-source code, as an alternative to proprietary software has advantages and disadvantages. The chief advantage is the availability of the source code, with which a small, competent programming team can develop and debug applications, even those with tricky timing relationships. The Open-Source code available today for Linux supports international and ad hoc standards. The use of a standards-based architecture has been shown to facilitate functional integration. It is a misconception that "free software" is necessarily available for little or no cost. The "free" part refers to the freedom to modify the source code. A disadvantage of developing with Open Source may be the perception that freely downloadable source code might not be mature or trustworthy. Countering this argument is the growing experience that the Open-Source offerings are as good as, and sometimes better than the equivalent commercial products. What is needed, however, is a strong configuration control mechanism. For the FlightLinux product, the FlightLinux Team assumed the responsibility of maintaining and certifying the official version of the build."

Richard  "What sort of hardware did you examine when you did your Flight Linux investigation" ?

Pat "Various microprocessor architectures have been and are being adapted from commercial products for space flight use. For all of the primary architectural candidates we identified, Linux was available in COTS form. The primary hardware for flight computers in the near term are derived from the Motorola PowerPC family (RHPPC, RAD6000, RAD750), the SPARC family (EH32), the MIPS family (Mongoose, RH32), the Intel architecture (space flight versions of 80386, 80486, Pentium, Pentium-II, Pentium-III), and the Intel ARM architecture. "

Richard  "Any other outstanding issues" ?

Pat  "We thought we might try POSIX which is an IEEE standard for a Portable Operating System based Unix. The use of a POSIX-compliant operating system and applications has many benefits for flight software. Among these benefits are: 1) software library reuse between missions and 2) software commonality between ground and flight platforms. For compliant code, the function calls, arguments, and resultant functionality are the same from one operating system to another. Source code does not have to be rewritten to port to another environment. Linux variants are mostly, but not completely, POSIX-compliant."

Richard  "What advantages are there in using Linux and similar software as opposed to the one off more conventional approaches" ?

Pat  "The advantages of Linux are numerous, but the requirements for spacecraft flight software are unique and non-forgiving. Traditional spacecraft onboard software has evolved from being monolithic - without a separable operating system - to using a custom operation system developed from scratch, to using a commercial embedded operating system such as VRTX or VxWorks. None of these approaches has proven ideal. In many cases, the problems involved in the spacecraft environment require access to the source code to debug. This becomes an issue with commercial vendors. Cost is also an issue. When source code is needed for a proprietary operating system, if the manufacturer chooses to release it at all, it is under a very restrictive non-disclosure agreement, and at additional cost. The Linux source is freely available to the team at the beginning of the effort. The use of the FlightLinux operating system simplifies several previously difficult areas in spacecraft onboard software. For example, the FlightLinux system imposes a file system on onboard data storage resources. In the best case, Earth-based support personnel and experimenters may network-mount onboard storage resources to their local file systems. The FlightLinux system both provides a path to migrate applications onboard and enforces a commonality between ground-based and space-based resources. FlightLinux also enables the implementation of the Java Virtual Machine, allowing for the up-link of Java applets to the spacecraft. Linux is not by nature or design a real-time operating system. Spacecraft embedded flight software needs a real-time environment in most cases. However, there are variations of real time, specified by upper limits on interrupt response time and interrupt latency. We can generally collect these into hard real-time and soft real-time categories. Examples of hard real-time requirements are those of attitude control, spacecraft clock maintenance, and telemetry formatting. Examples of soft real-time requirements include thermal control, data logging, and bulk memory scrubbing. Unix and Linux were not designed as real-time operating systems, but do support multi-tasking. Modifications or extensions to support and enforce process prioritisation are necessary to apply Linux to the embedded real-time control world. "

Richard  "Are there any disadvantages or issues with the use of Linux software on a space platform of the sort that is mentioned above"  ?

Pat  " Problems originate from the fact that the traditional Unix or Linux kernel is a monolithic entity that governs process prioritisation. Interrupt drivers and the kernel itself do not participate in the prioritisation scheme. The kernel typically has large stretches of non-preemptible code. This is necessarily in the design so that data structures can be modified in an atomic fashion. In a Linux kernel, all interrupt handlers run at a higher priority than the highest-priority task. In the Unix view, the kernel is the top level and most important task. In the real-time control world, this is not necessarily true. One approach to correcting this is to implement a threaded execution approach for the kernel and the interrupt handlers. The question arises as to how much the Linux kernel can be modified and still be referred to as a Linux kernel. Another approach is to treat the kernel itself as a scheduled task, under a Real Time Task Manager that manages process prioritisation and takes over control of interrupts. This has been referred to as kernel cohabitation. Many real-time schedulers for Linux are available for download. These include a Rate Monotonic Scheduler, which treats tasks with a shorter period as tasks with a higher priority, and an Earliest Deadline First (EDF) scheduler. Other approaches are also possible."

Richard  "What about hard drives ?  Surely the normal rotating hard drives that we use in day-to-day terrestrial circumstances will cause problems in space ? Anyone who has studied some basic physics will understand that."

Pat "Spacecraft onboard computers do not usually employ rotating magnetic memory for secondary storage. Initially, magnetic tape was used, but now the practice to use large arrays of bulk Dynamic Random Access Memory (DRAM), with various error detection and correction hardware and/or software applied. DRAM memory, usually treated as a sequential access device, is used to hold telemetry data during periods when ground contact is not available. Bulk memory is susceptible to errors on read and write, especially in the space environment, and needs multi-layer protection such as triple-modular redundancy (TMR), horizontal and vertical Cyclic Redundancy Codes (CRC), Error Correcting Codes (ECC), and scrubbing. Scrubbing can be done by hardware or software in the background. Although we usually think of bulk memory as a secondary storage device with sequential access, it may be implemented as random access memory within the computer's address space. This is the case with UoSat-12. The onboard computer on the UoSat-12 spacecraft has 128 megabytes of DRAM bulk memory, divided into four banks of 32 megabytes each, mapped through a window at the upper end of the processor's address space. This is the specific device driver that the QSS team developed and uses as a model for future development of similar modules. The current software of the UoSat-12 onboard computer treats this bulk memory as paged random access memory and applied a scrubbing algorithm to counter environmentally induced errors."

Pat "Given that the Linux operating system is onboard the spacecraft, support for a spacecraft LAN becomes relatively easy. Having the spacecraft operate as an Internet node is simple. This has been demonstrated on several recent missions. Interface between spacecraft components is traditionally provided by point-to-point connections, or a master/slave bus architecture. The use of a LAN onboard is not yet common. This is partially due to the lack of space-qualified hardware components, a situation which is rapidly being resolved."

Richard  "Linux can be used for some high quality clustering. Can you do that with Flight Linux" ?

Pat  "Having a Linux-based system enables a plethora of software that runs under Linux. One such package is the Beowulf software which implements a cluster of Linux machines. This may use several machines co-located in one satellite, or multiple satellites in a constellation linking their computational resources via inter-satellite communications. An approach to increase computational speed by the use of a multiprocessor system, dubbed a "Flight Beowulf", involves the case where an original problem - application - is split into parts or processes - that are acted upon by separate processors simultaneously. Ideally, given 'n' processors, the increased computational speed would be almost 'n' times that of a single processor. At any given point in the technology maturity curve, we can apply this technique to multiply the onboard processing power available. Of course, we must be observant of the impacts to other mission parameters such as power, weight, heat production, and size. This approach has been implemented and validated numerous times in a ground based environment. This lays the cornerstone for space-based cluster computing with off-the-shelf radiation-hardened processors by replacing the commercial processors in the engineering test unit with the equivalent rad-hard versions. Such a flight computer has the raw processing power capable of managing a complex scenario of sophisticated event detection and data mining algorithms for multiple instruments in a real-time, on-board environment."

Richard  "So, what you are saying is that you could put hundreds of small robot spacecraft upstairs and they would all work together as a single supercomputer ?  You could do large-scale exploration with all kinds of scientific instruments working together on space platforms.

Richard  "All sounds a bit too good to be true. When is your project due for completion ? Will there be another project of a similar nature" ?

Pat "Our FlightLinux Project concluded on June 30, 2002. We did not have sufficient time to complete the planned flight demonstration. We explored many of the issues in the use of a Linux -based operating system for spacecraft onboard computers, and found no insurmountable obstacles, but many distinct advantages. Many different applications with balloon-borne payloads, sounding rockets, and spacecraft are now considering the advantages of the Linux system."