[[!meta copyright="Copyright © 2011, 2012, 2013, 2014 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled [[GNU Free Documentation License|/fdl]]."]]"""]] [[!tag open_issue_mig]] [[!toc]] # 32-Bit vs. 64-Bit Interfaces ## IRC, freenode, #hurd, 2011-10-16 i guess it wouldn't be too hard to have a special mach kernel for 64 bits processors, but 32 bits userland only well, it means tinkering with mig like old sparc systems :p to build the 32bit interface, not the 64bit one ah yes hm i'm not sure mig would assume a 32 bits kernel, like now and you'll have all kinds of discrepancies in vm_size_t & such yes the 64 bits type should be completely internal types* but it would be far less work than changing all the userspace bits for 64 bit (ofc we'll do that some day but in the meanwhile ..) yes and it'd boost userland addrespace to 4GiB yes leaving time for a 64bit userland :) ## IRC, freenode, #hurd, 2011-11-14 also, what's the best way to deal with types such as type cache_info_t = struct[23] of integer_t; whereas cache_info_t contains longs, which are obviously not integer-wide on 64-bits processors ? you mean, to port mach to 64bit? no, to make the RPC declaration portable just in case :) refine integer_t into something more precise such as size_t, off_t, etc. i can't use a single line then struct cache_info contains ints, vm_size_t, longs should i just use the maximum size it can get ? or declare two sizes depending on the word size ? well, I'd say three youpi: three ? the ints, the vm_size_ts, and the longs youpi: i don't get it youpi: how would i write it in mig language ? I don't know the mig language me neither :) but I'd say don't lie i just see struct[23] of smething the original zone_info struct includes both integer_t and vm_size_t, and declares it as type zone_info_t = struct[9] of integer_t; in its mig defs file i don't have a good example to reuse which is lying yes which is why i was wondering if mach architects themselves actually solved that problem :) "There is no way to specify the fields of a C structure to MIG. The size and type-desc are just used to give the size of the structure. " well, this sucks :/ well, i'll do what the rest of the code seems to do, and let it rot until a viable solution is available braunr: we discussed the problem of expressing structs with MIG in the libburn thread (which I still need to follow up on... [sigh]) ## IRC, freenode, #hurd, 2012-12-12 In context of [[microkernel/mach/gnumach/memory_management]]. Or with a 64-bit one? ;-P tschwinge: i think we all had that idea in mind :) tschwinge: patches welcome :P tschwinge: sure, please help us settle down with the mig stuff what was blocking me was just deciding how to do it hum, what's blocking x86_64, except time to work on it ? deciding the mig types & such things i.e. the RPC ABI ok easy answer: keep it the same sorry, let me rephrase decide what ABI is supposed to be on a 64bit system, so as to know which way to rewrite the types of the kernel MIG part to support 64/32 conversion can't this be done in two steps ? well, it'd mean revamping the whole kernel twice as the types at stake are referenced in the whole RPC code the first step i imagine would simply imply having an x86_64 kernel for 32-bits userspace, without any type change (unless restricting to 32-bits when a type is automatically enlarged on 64-bits) it's not so simple the RPC code is tricky and there are alignments things that RPC code uses which become different when build with a 64bit compiler there are also things like int[N] for io_stat_struct and so on i see making the code wrong for 32 thus having to change the types pinotree: yes (doesn't mig support structs, or it is too clumsy to be used in practice?) pinotree: what's the problem with that (i explcitely said changing int to e.g. int32_t) that won't fly for some of the calls e.g. getting a thread state pinotree: no it doesn't support struct braunr: that some types in struct stat are long, for instance pinotree: same thing with longs youpi: why wouldn't it ? that wouldn't work on a 64bit system so we can't make it int32_t in the interface definition i understand the alignment issues and that the mig code adjusts the generated code, but not the content of what is transfered well of course i'm talking about the first step here which targets a 32-bits userspace only ok, so we agree the second step would have to revamp the whole RPC code again i imagine the first to be less costly well, actually no you're right, the mig stuff would be easy on the application side, but more complicated on the kernel side, since it would really mean dealing with 64-bits values there (unless we keep a 3/1 split instead of giving the full 4g to applications) See also [[microkernel/mach/gnumach/memory_management]]. (I don't see what that changes) if the kernel still runs with 32-bits addresses, everything it recevies from or sends through mig can be stored with the user side 32-bits types err, ok, but what's the point of the 64bit kernel then ? :) and it simply uses 64-bits addresses to deal with physical memory ok that could even be a 3.5/0.5 split then but the memory model forces us to run either at the low 2g or the highest ones but linux has 3/1, so we don't need that otherwise we need an mcmodel=medium we could do with mcmodel=medium though, for a time hm actually no, it would require mcmodel=large hum, that's stupid, we can make the kernel run at -2g, and use 3g up to the sign extension hole for the kernel map ## IRC, freenode, #hurd, 2013-12-03 I believe the main issue is redoing the RPCs in 64bit, i.e. the Mach/Hurd interface mach has always been 64-bits capable the problem is both mach and the hurd it's at the system interface (the .defs of the RPCs) azeem: ah, actually that's why you also say but i consider it to be a hurd problem the hurd itself is defined as being a set of interfaces and servers implementing them, i wouldn't exclude the interfaces that's what* # Structured Data ## IRC, freenode, #hurd, 2013-06-25 is there a nice way to get structured data through mig that I haven't found yet? say an array of string triples no :/ but you shouldn't need that my use case is getting info about fs translators from init to procfs [[hurd/translator/mtab]], [[hurd/translator/mtab/discussion]]. should I go for an iterator like interface instead? depends how many do you need ? you could go for a variable sized array too have a look at what already exists records, maybe 10-15, depends on many fs translators are running a variable sized array is ok if the size isn't too big (and when i say too big, i mean hundreds of MiB) an iterator is ok too if there aren't too many items you may want to combine both (i think that's what proc does) be aware that the maximum size of a message is limited to 512 MiB yeah I saw the array[] of stuff stuff, but array[] of string_t does not work, I guess b/c string_t is also an array how would I send an array of variable length strings? i'm not sure you can or maybe out of line somehow I expected mig to serialize arbitrary data structures, maybe it's to old for that? yeah, I read about uot of line, but that seems overkill it is old yes and not very user friendly in the end let me check we could stuff json into mig... see proc_getallpids for example we could get rid of low level serialization altogether :p hah, exactly what I was looking at (which is what i'll do in x15) type pidarray_t = array[] of pid_t; but that is trivial b/c its array[] of pid_t and always have the server writing guide near you yes well, make one big string and an array of lengths :p thought about that and said to myself, there must be a better way that I haven't found yet or one big string filled with real null-terminated c strings that you keep parsing until you ate all input bytes i'm almost certain there isn't type string_t = c_string[1024]; /* XXX */ yes even that isn't really variable sized you think anyone would object to me putting a json encoder in /hurd/init? it is probably better than me at serializing stuff... try with mig anyway the less dependencies we have for core stuff, the simpler it is but i agree, mig is painful would it be too hacky if I abused the argz functions? they do exactly what I'd need ## IRC, freenode, #hurd, 2013-06-26 there is https://code.google.com/p/protobuf-c/ and it has a rpc mechanism and I believe one could plug arbitrary transports easily please don't think about it we really don't want to add another layer of serialization it's better to completely redesign mach ipc anyway and there is a project for that :p ive seen x15 just food for thought i've studied google protocol buffers and fyi, no, it wouldn't be easy to plug arbitrary transports on top of mach there is a lot of knowledge about mach ports in mig [[hurd/translator/mtab]], [[hurd/translator/mtab/discussion]]. but again I face the challenge of serializing a arbitrary sized list of arbitrary sized strings yes list of ports is easier ;) but I think its worthwile so what about abusing argz* for this? you think it's too bad a hack? no since it's in glibc awesome :) but i don't remember the details well and i'm not sure the way you use it is safe yeah, I might have got the details wrong, I hadn't had the chance to test it ;) about this dynamic size problem a "simple" varying size array should do you can easily put all your strings in there seperated by 0? yes that's exactly what the argz stuff does you'll get the size of the array anyway, and consume it until there is no byte left good but be careful with this too since translators can be run by users, they somtimes can't be trusted and even a translator running as root may behave badly so careful with parsing noted