This library is designed to simplify the startup and use of multiprocessors. It defines a common interface to multiprocessor machines that is fairly platform independent.
Combined with the spin-locks provided in libkern, it is possible to implement a complete symmetric multiprocessor (SMP) based system using the OSKit code.
There is currently one machine-dependent interface, smp_apic_ack for the x86.
Currently, SMP support is only provided for Intel x86 systems conforming to the Intel Multiprocessor Specification.
Systems which fully comply to the Intel MultiProcessing Specification (IMPS) should be supported. Since some of the code is based on Linux 2.0, some features (such as dual I/O APICs) are not fully supported. The APIC (Advanced Programmable Interrupt Controller) is not yet used for general interrupt delivery. Instead, all hardware interrupts are sent to the BootStrap Processor (BSP).
If a machine works with Linux 2.0 it should work with the OSKit; however, testing has been limited to a few dual-processor machines.
The SMP code must be compiled with a compiler that supports .code16 for full functionality. The smp library will compile without it, but it will only support a single processor.
Inter-processor interrupts (IPIs) are implemented. These are currently the only interrupts received by the Application Processors (APs). IPIs allow the client OS to implement TLB-shoot-down and reschedule requests.
It is important to note that if more than one processor wishes to run in “user mode,” that the per-processor data structures in libkern (such as base_tss, base_idt, and base_gdt) will have to be made per-processor.
The OSKit code has not been tested with more than two processors. Success (and failure) reports for systems with three or more processors would be appreciated.
smp_apic_ack mentions a potential pitfall with Intel x86 SMPs. If more than one processor tries to send an IPI to a target processor, or if a processor sends multiple IPIs without waiting for them to be processed, IPIs can get lost. It is up to the programmer to deal with this limitation.
The SMP library assumes that the base environment is usable. It starts up the Application Processors on the kernel support library’s “base” data structures. It is possible (in fact required in many cases) to reload per-processor copies.
The following are symbols from the kernel support library required by the SMP library:
The LMM library is used to allocate pages of memory below 1MB. This requires the symbols:
These minimal C library symbols are pulled in by the SMP support code:
This library provides SMP-safe implementations for:
This function does the initial setup for the SMP support. It should be called before any other SMP library routines are used. It identifies the processors and gets them ready and waiting in a busy-loop for a “go” from the boot processor.
Note that success does not necessarily mean the system has multiple processors. Rather, failure indicates that the machine does not support multiple processors. smp_get_num_cpus should be used to determine the number of CPUs present.
Don’t call this more than once. . . yet.
It returns 0 on success (SMP-capable system is found). E_SMP_NO_CONFIG is returned on non-IMPS-compliant x86 machines.
This function returns a unique (per-processor) integer representing the current processor. Note that the numbers are not guaranteed to be sequential or starting from 0, although that may be a common case.
On the x86, these numbers correspond to the processor’s APIC ID, which is set by the hardware. However, these are to be treated as logical processor numbers since the smp library may do a transformation in the future.
The processor’s ID.
Given a number first, it returns the first processor ID such that the ID is greater than or equal to that number.
In order to be assured of finding all the CPUs, the initial call should be made with an argument of 0 and subsequent calls should be made with one more than the previously returned value.
This is designed to be used as an iterator function for the client OS to determine which processor numbers are present.
Returns E_SMP_NO_PROC if there are no more processors, otherwise the ID of the next processor.
#include <oskit/smp.h>
void smp_start_cpu(int processor_id, void (*func)(void *data), void *data, void *stack_ptr);
This releases the specified processor to start running a function with the specified stack.
Results are undefined if:
smp_find_cur_cpu can be used to prevent calling smp_start_cpu on yourself. This function must be called for each processor started up by smp_init; if the processor is not used, then func should execute the halt instruction immediately.
It is up to the user to verify that the processor is started up correctly.
This returns the number of processors that exist.
The number of processors that have been found. In a non-SMP-capable system, this will always return one.
This function is a hook provided by the host OS to allow the SMP library to request physical memory be mapped into its virtual address space. This is called by smp_init_paging.
Note that this could be implemented using osenv_mem_map_phys.
The virtual address where the physical pages are mapped. Returns zero if unable to map the memory.
This routine is called by the OS when it is ready to turn on paging. This call causes the SMP library to make call-backs to the OS to map the regions that are SMP-specific. On Intel x86 processors, this means the APICS.
Zero on success, non-zero on failure.
This causes the target processor to run its interrupt handler for the IPI vector, if the appropriate entry of smp_message_pass_enable has been set to non-zero by that processor. A processor should only modify its own smp_message_pass_enable entry after it is ready to start receiving IPIs.
This call offers very limited functionality. The expectation is that the OS writer will implement the desired functionality on top of this primitive.
smp_message_pass_enable[CPUID]
This array contains an entry for each processor. If a processor is ready to start receiving inter-processor interrupts, it should set smp_message_pass_enable[smp_find_cur_cpu()] to non-zero. This is used internally by the SMP library to prevent interrupts from being delivered before the processor has set up enough state to receive them.
This routine ACKs the local APIC. The APIC must be ACKed before returning from the IPI handler. Due to limitations in the APIC design, IPIs can be lost if sent too closely together, as the APIC only handles two outstanding requests.