From 2910b7c5b1d55bc304344b584a25ea571a9075fb Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Thu, 24 May 2012 23:08:09 +0200 Subject: Prepare toolchain/logs/master branch. --- .../multithreading/erlang-style_parallelism.mdwn | 201 --------------------- 1 file changed, 201 deletions(-) delete mode 100644 open_issues/multithreading/erlang-style_parallelism.mdwn (limited to 'open_issues/multithreading') diff --git a/open_issues/multithreading/erlang-style_parallelism.mdwn b/open_issues/multithreading/erlang-style_parallelism.mdwn deleted file mode 100644 index 75539848..00000000 --- a/open_issues/multithreading/erlang-style_parallelism.mdwn +++ /dev/null @@ -1,201 +0,0 @@ -[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] - -[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable -id="license" text="Permission is granted to copy, distribute and/or modify this -document under the terms of the GNU Free Documentation License, Version 1.2 or -any later version published by the Free Software Foundation; with no Invariant -Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled [[GNU Free Documentation -License|/fdl]]."]]"""]] - -[[!tag open_issue_hurd]] - -IRC, #hurd, 2010-10-05 - - antrik: Erlang-style parallelism might actually be interesting - for Hurd translators. - There are certain similarities between Erlang's message boxes - and Mach ports. - The problem is that all languages that implement the Erlang - actor model are VM-based. - sdschulze: I guess that's because most systems don't offer this - kind of message passing functionality out of the box... perhaps on Hurd - it would be possible to implement an Erlang-like language natively? - That would be quite attractive -- having the same API for - in-process parallelism and IPC. - But I don't see why Erlang needs a VM... It could also be - implemented in a library. - [...] - BTW, Scala doesn't require a VM by design. Its Erlang - implementation is a binary-compatible abstraction to Java. - [...] - My point was that Erlang employs some ideas that might be - usable in the Hurd libraries. - concerning multithreading stuff - Unfortunately, it will not contribute to readability if done in - C. - perhaps it's worth a look :-) - Actually, a Mach port is pretty close to an Erlang actor. - Currently, your I/O callbacks have to block when they're - waiting for something. - What they should do is save the Mach port and respond as soon - as they can. - So there should be a return status for "call me later, when I - tell you to" in the callbacks. - Then the translator associates the Mach port with the summary - of the request in some data structure. - As soon as the data is there, it tells the callback function to - appear again and fulfills the request. - That's -- very roughly -- my idea. - Actually, this eliminates the need for multithreading - completely. - sdschulze: not sure whether you are talking about RPC level or - libc level here... - It should be transparent to libc. - If the client does a read() that cannot be answered immediatly, - it blocks. - The difference is that there is no corresponding blocking - thread in the translator. - ah, so you are talking about the server side only - yes - you mean the callback functions provided by the translator - implementation should return ASAP, and then the dispatcher would call - them again somehow - allowing the server to be single-threaded, if desired - exactly - like: call_again (mach_port); - but if the functions give up control, how does the dispatcher know - when they are ready to be activated again? or does it just poll? - The translator knows this. - hm... - well, we are talking about the internal design of the translator, - right? - I'm not saying it's impossible... but it's a bit tricky - essentially, the callbacks would have to tell the dispatcher, - "call me again when there is an incoming message on this port" - Say we have a filesystem translator. - (or rather, it probably should actually call a *different* - callback when this happens) - The client does a "read(...)". - => A callback is called in the translator. - let's call it disfs_S_io_read() ;-) - err... diskfs - The callback returns: SPECIAL_CALL_ME_LATER. - yes, exactly that :) - But before, it saves the position to be read in its internal - data structure. - (a sorted tree, whatever) - The main loop steps through the data structure, doing a read() - on the underlying translator (might be the disk partition). - "Ah, gotcha, this is what the client with Mach port number 1234 - wanted! Call his callback again!" - Then we're back in diskfs_S_io_read() and supply the data. - so you want to move part of the handling into the main loop? while - I'm not fundamentally opposed to that, I'm not sure whether the - dispatcher/callback approach used by MIG makes much sense at all in this - case... - my point is that this probably can be generalised. blocking - operations (I/O or other) usually wait for a reply message on a port -- - in this case the port for the underlying store - so the main loop would just need to wait for a reply message on - the port, without really knowing what it means - on what port? - so disfs_S_io_read() would send a request message to the store; - then it would return to the dispatcher, informing it to call - diskfs_S_io_read_finish() or something like that when there is a message - on the reply port - main loop would add the reply port to the listening port bucket - and as soon as the store provides the reply message, the - dispatcher would then call diskfs_S_io_read_finish() with the reply - message - yes - this might actually be doable without changes to MIG, and with - fairly small changes to libports... though libdiskfs etc. would probably - need major rewrites - What made me think about it is that Mach port communication - doesn't block per se. - all this is however ignoring the problem I mentioned yesterdays: - we need to handle page faults as well... - It's MIG and POSIX that block. - What about page faults? - when the translator has some data mapped, instead of doing - explicit I/O, blocking can occur on normal memory access - antrik: Well, I've only been talking about the server side so - far. - sdschulze: this *is* the server side - sdschulze: a filesystem translator can map the underlying store - for example - (in fact that's what the ext2 translator does... which is why we - had this 2G partition limit) - antrik: Ah, OK, so in other words, there are requests that it - can answer immediatly and others that it can't? - that's not the issue. the issue is the the ext2 translator doesn't - issue explicit blocking io_read() operations on the underlying - store. instead, it just copies some of it's own address space from or to - the client; and if the page is not in physical memory, blocking occurs - during the copy - so essentially we would need a way to return control to the - dispatcher when a page fault occurs - antrik: Ah, so MIG will find the translator unresponsive? (and - then do what?) - sdschulze: again, this is not really a MIG thing. the main loop is - *not* in MIG -- it's provided by the tranlator, usually through libports - OK, but as Mach IPC is asynchronous, a temporarily unresponsive - translator won't cause any severe harm? - antrik: "Easy" solution: use a defined number of worker - threads. - sdschulze: well, for most translators it doesn't do any harm if - they block. but if we want to accept that, there is no point in doing - this continuation stuff at all -- we could just use a single-threaded - implementation :-) - Hard solution: do use explicit I/O and invent a - read_no_pagefault() call. - not sure what you mean exactly. what I would consider is something - like an exception handler around the copy code - so if an exception occurs during the copy, control is returned to - the dispatcher; and once the pager informs us that the memory is - available, the copy is restarted. but this is not exacly simple... - antrik: Ah, right. If the read() blocks, you haven't gained - anything over blocking callbacks. - * sdschulze adopted an ML coding style for his C coding... - antrik: Regarding it on the Mach level, all you want to do is - some communication on some ports. - antrik: Only Unix's blocking I/O makes you want to use threads. - Unless you have a multicore CPU, there's no good reason why you - would *ever* want multithreading. - (except poor software design) - antrik: Is there a reason why not to use io_read? - sdschulze: I totally agree about multithreading... - as for not using io_read(): some things are easier and/or more - efficient with mapping - the Mach VM is really the most central part of Mach, and it's - greatest innovation... - antrik: If you used explicit I/O, it would at least shift the - problem somewhere else... - sure... but that's a workaround, not a solution - I'm not sure how to deal with page faults then -- I know too - little about the Hurd's internal design. - Non-blocking io_read only works if we address the client side, - too, BTW. - which would be quite ugly in C IMHO - announce_read (what, to, read, when_ready_callback); - sdschulze: POSIX knows non-blocking I/O - never checked how it works though - Yes, but I doubt it does what we want. - anyways, it's not too hard to do non-blocking io_read(). the - problem is that then you have to use MIG stubs directly, not the libc - function - And you somehow need to get the answer. - resp. get to know when it's ready - the Hurd actually comes with a io_request.defs and io_reply.defs - by default. you just need to use them. - oh, ok - (instead of the usual io.defs, which does a blocking send/receive - in one step) - I'd be interested how this works in Linux... - what exactly? - simultaneous requests on one FS - ah, you mean the internal threading model of Linux? no idea - if it uses threading at all - youpi probably knows... and some others might as well - Callbacks are still ugly... -- cgit v1.2.3