diff options
Diffstat (limited to 'open_issues/smp.mdwn')
| -rw-r--r-- | open_issues/smp.mdwn | 269 |
1 files changed, 37 insertions, 232 deletions
diff --git a/open_issues/smp.mdwn b/open_issues/smp.mdwn index 6496f388..ce2c2ce2 100644 --- a/open_issues/smp.mdwn +++ b/open_issues/smp.mdwn @@ -38,250 +38,55 @@ https://en.wikipedia.org/wiki/File:SMP_-_Symmetric_Multiprocessor_System.svg --> # Current Status +As of September 2024, the SMP support is implemented for i386 with +working internet, because it boots with only one cpu in the default +processor set. The slave processors are temporarily disabled until +they can be safely used per task. We are unable to turn on the full +set at boot time due to race bugs. Debian Hurd provides SMP enabled +GNU Mach kernels. -The GNU Mach source code includes many special cases for multiprocessor, -controlled by #if NCPUS > 1 macro. +# How to test the current SMP support -But this support is very limited: +The easiest way to test the SMP support, is via the [[qemu +guide|https://www.gnu.org/software/hurd/hurd/running/qemu.html]]. +Once you have the Hurd running you can build an SMP enabled GNU Mach. -- GNU Mach don't detect CPUs in runtime: The number of CPUs must be hardcoded in -compilation time. The number of cpus is set in `mach_ncpus` configuration -variable, set to 1 by default, in configfrag.ac file. This variable will -generate `NCPUS` macro, which is used by gnumach to control the special cases -for multiprocessor. If `NCPUS` > 1, then gnumach will enable multiprocessor -support, with the number of cpus set by the user in mach_ncpus -variable. Otherwise, SMP will be disabled. + $ git clone git://git.savannah.gnu.org/hurd/gnumach.git + $ cd gnumach + $ autoreconf -i + $ mkdir build + $ cd build + $ ../configure --enable-ncpus=4 --enable-kdb + $ make gnumach.gz + $ su + # mv /boot/gnumach-1.8-486.gz /boot/gnumach-1.8-486.gz.bak + # cp gnumach.gz /boot/gnumach-1.8-486.gz -- The special cases to multicore in gnumach source code have never been tested, - so these can contain many errors. Furthermore, these special case are - incomplete: many functions, such as `cpu_number()` or `intel_startCPU()` aren't - written. - -- GNU Mach doesn't initialize the processor with the proper options for - multiprocessing. For this reason, the current support is only multithread and - not real multiprocessor support. +You may optionally update `/boot/grub/grub.cfg` change `hd0` to `wd0` +and add `console=com0` -- Many drivers included in Hurd aren't thread-safe, and these could crash in a - SMP environment. So, it's necessary to isolate this drivers, to avoid - concurrency problems. + /boot/gnumach-1.8-486.gz root=part:2:device:wd0 console=com0 +update `/etc/fstab` and update `wd0` instead of `hd0`. -### Solution + /dev/wd0s2 / ext2 defaults 0 1 + /dev/wd0s1 none swap sw 0 0 -To solve this, we need to implement some routines to detect the number of -processors, assign an identifier to each processor, and configure the lapic and -IPI support. These routines must be executed during Mach boot. +You can shutdown via `/sbin/poweroff`. -> "Really, all the support you want to get from the hardware is just getting the -> number of processors, initializing them, and support for interprocessor -> interrupts (IPI) for signaling." - Samuel Thibault -> [link](https://lists.gnu.org/archive/html/bug-hurd/2018-08/msg00071.html) +start qemu with `-smp 4` and add `-nographic` if you want to use `com0`. -> "The process scheduler probably already has the support. What is missing is -the hardware driver for SMP: enumeration and initialization." - Samuel Thibault -[link](https://lists.gnu.org/archive/html/bug-hurd/2018-08/msg00083.html) + $ qemu-system-i386 -M q35,accel=kvm -smp 4 -m 2G -net \ + user,hostfwd=tcp::2223-:22 -net nic -hda debian-hurd-VERSION.img \ + -nographic -The current necessary functions are `cpu_number()` (in kern/cpu_number.h) and -`intel_startCPU()`. Another non-critical function, is `cpu_control()` -[*Reference*](https://www.gnu.org/software/hurd/gnumach-doc/Processor-Control.html#Processor-Control) +To test smp: -Other interesting files are `pmap.c` and `sched_prim.c` We also -have to build an isolated environment to execute the non-thread-safe drivers. - -> "Yes, this is a real concern. For the Linux drivers, the long-term goal is to -> move them to userland anyway. For Mach drivers, quite often they are not -> performance-sensitive, so big locks would be enough." - Samuel Thibault -> [link](https://lists.gnu.org/archive/html/bug-hurd/2018-08/msg00073.html) - -### Task list - -1. Implement a routine to detect and identify the processors - - This routine must check the number of processors, initialize the lapic of BSP - (the master processor), and assign a kernel ID for each processor. This kernel - ID does not have to be equal to the APIC ID. The relation kernel/APIC can be - settled with an array, where the kernel ID is the index, and the APIC contains - the data. GNU Mach can derive the list of processors from memory, reading from - ACPI table, or from MP table. However, MP table is deprecated in most modern - CPUs, so it is preferable to use ACPI table for this. - - The tasks to do for this are: - - - Detect the number of processors - - - Create a array indexed by kernel ID, which sets a relation with APIC ID. - - - Initialize the lapic of BSP - - - Initialize IOAPIC - - This routine could be called from `i386at_init()` - (i386/i386at/model_dep.c). This function will call the functions which - initialize the lapic and the ioapic. - - **NOTE**: This routine must be executed before `intel_startCPU()` or other routines. - - - **How to find APIC table** - - To find APIC table, we can read - RSDT table [RSDT reference](http://www.uefi.org/sites/default/files/resources/ACPI%206_2_A_Sept29.pdf#G10.1358180). - To get the address of RSDT, we need to read RDSP table. We can get the - RSDP table by this [RDSP - reference](http://www.uefi.org/sites/default/files/resources/ACPI%206_2_A_Sept29.pdf#G10.1357698) - Once we have the RSDT table, we need to read *Entry* field, and search the pointer to - the APIC table in the array referenced in this field. - - We can find an example about reading ACPI table in X15 OS: - [Reference](https://github.com/richardbraun/x15/blob/0c0e2a02a42a8161e1b8dc1e1943fe5057ecb3a3/arch/x86/machine/acpi.c#L576) - - - We need to initialize the `machine_slot` of each processor (currently only initializes cpu0). - The `machine_slot` has this structure. [Reference](https://github.com/AlmuHS/GNUMach_SMP/blob/0d490ef21c156907f3f26a6cdc00842f462a877a/include/mach/machine.h#L68): - - > `struct machine_slot { /*boolean_t*/` <br/> - > `integer_t is_cpu;` <br/> - > `/* is there a cpu in this slot? */ ` <br/> - > `cpu_type_t cpu_type; /* type of cpu */` <br/> - > `cpu_subtype_t cpu_subtype; /* subtype of cpu */` <br/> - > `/*boolean_t*/ integer_t running; /* is cpu running */` <br/> - > `integer_t cpu_ticks[CPU_STATE_MAX]; integer_t` <br/> - > `clock_freq; /* clock interrupt frequency */ };` <br/> - - - We can find an example of initialization in this [link:](https://github.com/AlmuHS/GNUMach_SMP/blob/0d490ef21c156907f3f26a6cdc00842f462a877a/i386/i386at/model_dep.c#L612) - - This modification also involve the redefinition of `NCPUS`, which must be set - to the maximum **possible** number of processors. We can do this by modifying - `configfrag.ac`, with this: - - > `# Multiprocessor support is still broken.` <br/> - > `AH_TEMPLATE([MULTIPROCESSOR], [set things up for a uniprocessor])` <br/> - > `mach_ncpus=2` <br/> - > `AC_DEFINE_UNQUOTED([NCPUS], [$mach_ncpus], [number of CPUs])` </br> - > `[if [$mach_ncpus` > `-gt 1 ]; then]` <br/> - > `AC_DEFINE([MULTIPROCESSOR], [1], [set things up for a` > `multiprocessor])` - > `AC_DEFINE_UNQUOTED([NCPUS], [256], [number of CPUs]) ` <br/> - > `[fi]` <br/> - - - Interesting files and functions - `machine.c` - [Link](https://github.com/AlmuHS/GNUMach_SMP/blob/master/kern/machine.c) - - `c_boot_entry()` - [Link](https://github.com/AlmuHS/GNUMach_SMP/blob/0d490ef21c156907f3f26a6cdc00842f462a877a/i386/i386at/model_dep.c#L529) - - - Example, in X15 OS: - [Link](https://github.com/richardbraun/x15/blob/d6d90a3276a09da65690b019e985392bf77b53b0/arch/x86/machine/cpu.c#L1114) - - 1.1. Implement a `cpu_number()` function. - - - This function must return the kernel ID of the processor which is executing the function. To - get this, we have to read the local apic memory space, which will show the - lapic of the current CPU. Reading the lapic, we can get its APIC ID. Once - we have the APIC ID of the current CPU, the function will search in the - Kernel/APIC array until it finds the same APIC ID. Then it will return the - index (Kernel ID) of this position. - -2. Implement a routine to initialize the processors - - - This routine will initialize the lapic of each processor and other structures - needed to run the kernel. We can find an example of lapic initialization - here - [reference](https://github.com/mit-pdos/xv6-public/blob/b818915f793cd20c5d1e24f668534a9d690f3cc8/lapic.c#L55) - Also, we can get more information in Chapter 8.4 and 8.11 of Intel - Developer Guide, - Volume 3. [link](https://software.intel.com/sites/default/files/managed/a4/60/325384-sdm-vol-3abcd.pdf) - -3. Implement `intel_startCPU()` - - - This function will initialize the descriptor tables of the processor specified by the - parameter, and launch the startup IPI to this processor. This function will be - executed during the boot of the kernel (process 0). The task to do in this function - are: - - - Initialize the processor descriptor tables - - Launch Startup IPI to this processor - We have a current implementation of `intel_startCPU()` in this - [link](https://github.com/AlmuHS/GNUMach_SMP/blob/smp/i386/i386/mp_desc.c). - This implementation is based in XNU's `intel_startCPU()` - [function](https://github.com/nneonneo/osx-10.9-opensource/blob/f5a0b24e4d98574462c8f5e5dfcf0d37ef7e0764/xnu-2422.1.72/osfmk/i386/mp.c#L423) - - We can find explainations about how to raise an IPI in this pages: - [*Reference 1*](https://www.cs.usfca.edu/~cruse/cs630f08/lesson22.ppt), - [*Reference 2*](https://www.cheesecake.org/sac/smp.html), [*Reference - 3*](http://www.dis.uniroma1.it/pub/quaglia/AOSV-traps-interrupts.pdf) We can - get information about how to raise an IPI in Intel Developer Guide, Volume 3, - Chapter 10.6 - -4. Implement another routine to start the processors - - - This routine calls to `processor_start()` for each processor, which will start the - processor using this sequence of calls: [`processor_start(processor_t - processor)`](https://github.com/AlmuHS/GNUMach_SMP/blob/5d527f532dfba9f2da54555d5fbe585dd458579b/kern/processor.c#L447) - -> - [`cpu_start(processor->slot_num)`](https://github.com/AlmuHS/GNUMach_SMP/blob/5d527f532dfba9f2da54555d5fbe585dd458579b/i386/i386/mp_desc.c#L335) - -> - [`intel_startCPU(cpu)`](https://github.com/AlmuHS/GNUMach_SMP/blob/5d527f532dfba9f2da54555d5fbe585dd458579b/i386/i386/mp_desc.c#L180) - - These articles shows some annotations about how to do the AP Startup: - - - [Reference1](https://wiki.osdev.org/Symmetric_Multiprocessing#AP_startup), - - [Reference2](https://stackoverflow.com/a/16368043/7077301) (...) - - After implement IPI support, It's recommended reimplement `machine_idle()`, - `machine_relax ()`, `halt_cpu()` and `halt_all_cpus()` using IPI. - - [reference](https://github.com/AlmuHS/GNUMach_SMP/blob/0d490ef21c156907f3f26a6cdc00842f462a877a/i386/i386at/model_dep.c#L201) - - Also in `ast_check.c`, we have to implement both functions, using - IPI - [Reference](https://github.com/AlmuHS/GNUMach_SMP/blob/master/i386/i386/ast_check.c) - - - This functions must force the processors to check if there are any AST - signal, and we ought to keep in the mind the following irc chat: - - -> <AlmuHS> what is the use of AST in gnumach? <br/> -> <AlmuHS> this file what do? <br/> -> https://github.com/AlmuHS/GNUMach_SMP/blob/master/i386/i386/ast_check.c <br/> -> <youpi> I don't know <br/> -> <youpi> but look at what calls that <br/> -> <youpi> see e.g. the call in thread.c <br/> -> <AlmuHS> This <br/> -> function is called during the sequence of cpu_up(), in machine.c <br/> -> <AlmuHS> but only if NCPUS > 1 <br/> -> <youpi> it seems like it's to trigger an AST check on another <br/> -> processor <br/> -> <youpi> i.e. a processor tells another to run ast_check <br/> -> <youpi> (see the comment in thread.c) <br/> -> <AlmuHS> <br/> -> https://github.com/AlmuHS/GNUMach_SMP/blob/master/kern/machine.c <br/> -> <youpi> well, the initialization part is not necessarily what's -> important to <br/> -> think about at first <br/> -> <youpi> i.e. until you know what you'll have <br/> -> to do during execution, you don't know what you'll need to intialize at <br/> -> initialization <br/> -> <youpi> you might even not need to initialize anything <br/> -> <AlmuHS> then, this is the reason because all functions <br/> -> in ast_check.c are empty <br/> -> <youpi> cause_ast_check being empty is really probably a TODO <br/> -> <AlmuHS> but I'm not clear what I need to write in this functions <br/> -> <youpi> what the comment said: make another processor run ast_check() <br/> -> <youpi> which probably means raising an inter-processor interrupt <br/> -> <youpi> (aka IPI) <br/> -> <youpi> to get ast_check() called by the other processor <br/> -> <AlmuHS> then, this funcions must raise an IPI in the processor? <br/> -> <youpi> that's the idea <br/> -> <youpi> the IPI probably needs some setup <br/> - -We can use [XV6 source - code](https://pdos.csail.mit.edu/6.828/2018/xv6.html). as model to implements - the function and routines. Some interesting files are - [`lapic.c`](https://github.com/mit-pdos/xv6-public/blob/master/lapic.c), - [`proc.c`](https://github.com/mit-pdos/xv6-public/blob/master/proc.c) and - [`main.c`](https://github.com/mit-pdos/xv6-public/blob/master/main.c) + $ sudo /path/to/smp /bin/bash + $ stress -c 7 +smp.c source can be found +[[here|https://lists.gnu.org/archive/html/bug-hurd/2024-02/msg00088.html]]. ## References - [Comments about the project bug-hurd @@ -289,7 +94,7 @@ maillist](https://lists.gnu.org/archive/html/bug-hurd/2018-08/msg00048.html) - [Initial thread in bug-hurd maillist](https://lists.gnu.org/archive/html/bug-hurd/2018-06/msg00048.html) - [SMP in GNU/Hurd FAQ](https://www.gnu.org/software/hurd/faq/smp.html) -- [GNU Mach git repository](http://git.savannah.gnu.org/cgit/hurd/gnumach.git) +- [GNU Mach git repository](https://git.savannah.gnu.org/cgit/hurd/gnumach.git) - [GNU Mach reference manual](https://www.gnu.org/software/hurd/gnumach-doc/) - [**MultiProcessor Specification**](https://pdos.csail.mit.edu/6.828/2011/readings/ia32/MPspec.pdf) - [**ACPI Specification**](http://www.uefi.org/sites/default/files/resources/ACPI%206_2_A_Sept29.pdf) |