diff options
Diffstat (limited to 'hurd/translator/tmpfs')
| -rw-r--r-- | hurd/translator/tmpfs/notes_bing.mdwn | 101 | ||||
| -rw-r--r-- | hurd/translator/tmpfs/notes_various.mdwn | 218 | ||||
| -rw-r--r-- | hurd/translator/tmpfs/tmpfs_vs_defpager.mdwn | 73 |
3 files changed, 392 insertions, 0 deletions
diff --git a/hurd/translator/tmpfs/notes_bing.mdwn b/hurd/translator/tmpfs/notes_bing.mdwn new file mode 100644 index 00000000..fa3eeac2 --- /dev/null +++ b/hurd/translator/tmpfs/notes_bing.mdwn @@ -0,0 +1,101 @@ +[[!meta copyright="Copyright © 2007, 2008, 2009 Free Software Foundation, +Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled +[[GNU Free Documentation License|/fdl]]."]]"""]] + +[[!tag open_issue_hurd]] + +1. to run tmpfs as a regular user, /servers/default-pager must be executable by + that user. by default it seems to be set to read/write. + + $ sudo chmod ugo+x /servers/default-pager + + Then I get this error: + + tmpfs: /build/mbanck/hurd-20060825/build-tree/hurd/tmpfs/dir.c:62: diskfs_get_directs: Assertion `__builtin_offsetof (struct tmpfs_dirent, name) >= __builtin_offsetof (struct dirent, d_name)' failed. + +2. i rearranged the struct tempfs_dirent in tmpfs.h to line up with the struct + dirent. now the assert passes at line 62 of dir.c passes. + + struct tmpfs_dirent + { + struct tmpfs_dirent *next; + struct disknode *dn; + + char padding[3]; + uint8_t namelen; + char name[0]; + }; + + now ls works on an empty directory. + you can touch files, and run `ls' on them. + mkdir, rmdir works too. + fsysopts works + df works + +3. creating a symlink fails. + + old patch to get symlinks working: + + http://www.mail-archive.com/bug-hurd@gnu.org/msg11844.html + --- node.c.orig 2005-07-24 09:56:39.000000000 -0400 + +++ node.c 2005-07-24 09:55:46.000000000 -0400 + @@ -330,6 +330,7 @@ + create_symlink_hook (struct node *np, const char *target) + { + assert (np->dn->u.lnk == 0); + + np->dn_stat.st_size = strlen (target); + if (np->dn_stat.st_size > 0) + { + const size_t size = np->dn_stat.st_size + 1; + @@ -337,6 +338,7 @@ + if (np->dn->u.lnk == 0) + return ENOSPC; + memcpy (np->dn->u.lnk, target, size); + + np->dn->type = DT_LNK; + adjust_used (size); + recompute_blocks (np); + } + @@ -380,8 +382,6 @@ + error_t + diskfs_truncate (struct node *np, off_t size) + { + - if (np->allocsize <= size) + - return 0; + + if (np->dn->type == DT_LNK) + { + @@ -392,6 +392,9 @@ + return 0; + } + + + if (np->allocsize <= size) + + return 0; + + + assert (np->dn->type == DT_REG); + + if (default_pager == MACH_PORT_NULL) + + now symlinks work. + +4. can't write data to a file + + +--- + +miscellaneous notes: + +`diskfs_disk_name` could be `NULL`, but it is `"swap"` + +using `default_pager_object_set_size (np->dn->u.reg.memobj, size);` to truncate and grow. + +why are our blocks 512? shouldn't it something else? or at least settable? +or does [[libdiskfs]] demand this? + +`diskfs_get_filemap_pager_struct (struct node *np)` returns null. +shouldn't it return `default_pager`? diff --git a/hurd/translator/tmpfs/notes_various.mdwn b/hurd/translator/tmpfs/notes_various.mdwn new file mode 100644 index 00000000..5e4e991c --- /dev/null +++ b/hurd/translator/tmpfs/notes_various.mdwn @@ -0,0 +1,218 @@ +[[!meta copyright="Copyright © 2005, 2006, 2007, 2008, 2009 Free Software +Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled +[[GNU Free Documentation License|/fdl]]."]]"""]] + +[[!tag open_issue_hurd]] + + <antrik> hde: what's the status on tmpfs? + <hde> Broke + <hde> k0ro traced the errors like the assert show above to a pager problem. + See the pager cannot handle request from multiple ports and tmpfs sends + request using two differ ports, so to fix it the pager needs to be hacked + to support multiple requests. + <hde> You can enable debugging in the pager by changing a line from dprintf + to ddprintf I can tell you how if you want. + <antrik> and changing tmpfs to use a single port isn't possible?... + <hde> antrik, I am not sure. + <hde> IIRC k0ro was saying it cannot be changed and I cannot recall his + reasons why. + <sdschulze> antrik: Doing it the quick&dirty way, I'd just use an N-ary + tree for representing the directory structure and mmap one new page (or + more) for each file. + <hde> sdschulze, What are you talking about? + <sdschulze> hde: about how I would implement tmpfs + <hde> O + <azeem> sdschulze: you don't need to reimplement it, just fix it :) + <sdschulze> azeem: Well, it seems a bit more difficult than I considered. + <sdschulze> I had assumed it was implemented the way I described. + <hde> O and the assert above gets triggered if you don't have a + default-pager setup on /servers/default-pager + <hde> the dir.c:62 assert that is. + <azeem> hde: you sure? I think I have one + <hde> I am almost sure. + <azeem> mbanck@beethoven:~$ showtrans /servers/default-pager + <azeem> /hurd/proxy-defpager + <azeem> isn't that enough? + <hde> It is suppose to be. + <hde> Try it as root + <hde> I was experiecing alot of bugs as a normal user, but according to + marcus it is suppose to work as root, but I was getting alot of hangs. + <azeem> hde: same issue, sudo doesn't work + <hde> sucky, well then there are alot of bugs. =) + <azeem> eh, no + <azeem> I still get the dir.c assert + <sdschulze> me too + <sdschulze> Without it, I already get an error message trying to set tmpfs + as an active translator. + +--- + + <hde> I think I found the colprit. + <hde> default_pager_object_set_size --> This is were tmpfs is hanging. + <hde> mmm Hangs on the message to the default-pager. + +--- + + <hde> Well it looks like tmpfs is sending a message to the default-pager, + the default-pager then receives the message and, checks the seqno. I + checked the mig gen code and noticed that the seqno is the reply port, it + this does not check out then the default pager is put into a what it + seems infinte condition_wait hoping to get the correct seqno. + <hde> Now I am figuring out how to fix it, and debugging some more. + +--- + + <marco_g> hde: Still working on tmpfs? + <hde> Yea + <marco_g> Did you fix a lot already? + <hde> No, just trying to narrow down the reason why we cannot write file + greater then 4.5K. + <marco_g> ahh + <marco_g> What did you figure out so far? + <hde> I used the quick marcus fix for the reading assert. + <marco_g> reading assert? + <hde> Yea you know ls asserted. + <marco_g> oh? :) + <hde> Because, the offsets changed in sturct dirent in libc. + <hde> They added 64 bit checks. + <hde> So marcus suggested a while ago on bug-hurd to just add some padding + arrays to the struct tmpfs_dirent. + <hde> And low and behold it works. + <marco_g> Oh, that fix. + <hde> Yup + <hde> marco_g, I have figured out that tmpfs sends a message to the + default-pager, the default-pager does receive the message, but then + checks the seqno(The reply port) and if it is not the same as the + default-pagers structure->seqno then she waits hoping to get the correct + one. Unfortantly it puts the pager into a infinite lock and never come + out of it. + <marco_g> hde: That sucks... + <marco_g> But at least you know what the problem is. + <hde> marco_g, Yea, now I am figuring out how to fix it. + <hde> Which requires more debugging lol. + <hde> There is also another bug, default_pager_object_set_size in + <hde> mach-defpager does never return when called and makes tmpfs hang. I + <hde> will have a closer look at this later this week. + +--- + + <hde> Cool, now that I have two pagers running, hopefully I will have less + system crashes. + <marcus> running more than one pager sounds like trouble to me, but maybe + hde means something different than I think + <hde> Well the other pager is only for tmpfs to use. + <hde> So I can debug the pager without messing with the entire system. + <hde> marcus, I am trying ti figure out why diskfs_object_set_size waits + forever. This way when the pager becomes locked forever I can turn it + off and restart it. When I was doing this with only one mach-defpager + running the system would crash. + <marcus> hde: how were you able to start two default pagers?? + <hde> Well you most likely will not think my way of doing it was correct, + and I am also not sure if it is lol. I made my hacked version not stop + working if one is alreay started. + +--- + + <hde> See, the default-pager has a function called + default_pager_object_set_size this sets the size for a memory object, + well it checks the seqno for each object if it is wrong it goes into a + condition_wait, and waits for another thread to give it a correct seqno, + well this never happens. + <hde> Thus, you get a hung tmpfs and default-pager. + <hde> pager_memcpy (pager=0x0, memobj=33, offset=4096, other=0x20740, + size=0x129df54, prot=3) at pager-memcpy.c:43 + <hde> bddebian, See the problem? + <bddebian> pager=0x0? + <hde> Yup + <hde> Now wtf is the deal, I must debug. + <hde> -- Function: struct pager * diskfs_get_filemap_pager_struct + <hde> (struct node *NP) + <hde> Return a `struct pager *' that refers to the pager returned by + <hde> diskfs_get_filemap for locked node NP, suitable for use as an + <hde> argument to `pager_memcpy'. + <hde> That is failing. + <hde> If it is not one thing it is another. + <bddebian> All of Mach fails ;-) + <hde> It is alot of work to make a test program that uses libdiskfs. + +--- + + <bing> to run tmpfs as a regular user, /servers/default-pager must be + executable by that user. by default it seems to be set to read/write. + <bing> $ sudo chmod ugo+x /servers/default-pager + <bing> you can see the O_EXEC in tmpfs.c + <bing> maybe this is just a debian packaging problem + <bing> it's probably a fix to native-install i'd guess + +--- + + <bing> tmpfs is failing on default_pager_object_create with -308, which + means server died + <bing> i'm running it as a regular user, so it gets it's pager from + /servers/default-pager + <bing> and showtrans /servers/default-pager shows /hurd/proxy-defpager + <bing> so i'm guessing that's the server that died + +--- + + <bing> this is about /hurd/tmpfs + <bing> a filesystem in memory + <bing> such that each file is it's own memory object + <andar> what does that mean exactly? it differs from a "ramdisk"? + <bing> instead of the whole fs being a memory object + <andar> it only allocates memory as needed? + <bing> each file is it's own + <bing> andar: yeah + <bing> it's not ext2 or anything + <andar> yea + <bing> it's tmpfs :-) + <bing> first off, echo "this" > that + <bing> fails + <bing> with a hang + <bing> on default_pager_object_create + <andar> so writing to the memory object fails + <bing> well, it's on the create + <andar> ah + <bing> and it returns -308 + <bing> which is server died + <bing> in mig-speak + <bing> but if i run it as root + <bing> things behave differently + <bing> it gets passed the create + <bing> but then i don't know what + <bing> i want to make it work for the regular user + <bing> it doesn't work as root either, it hangs elsewhere + <andar> but it at least creates the memory object + <bing> that's the braindump + <bing> but it's great for symlinks! + <andar> do you know if it creates it? + <bing> i could do stowfs in it + +--- + + <antrik> bing: k0ro (I think) analized the tmpfs problem some two years ago + or so, remember?... + <antrik> it turns out that it broke due to some change in other stuff + (glibc I think) + <antrik> problem was something like getting RPCs to same port from two + different sources or so + <antrik> and the fix to that is non-trivial + <antrik> I don't remember in what situations it broke exactly, maybe when + writing larger files? + <bing> antrik: yeah i never understood the explanation + <bing> antrik: right now it doesn't write any files + <bing> the change in glibc was to struct dirent + <antrik> seems something more broke in the meantime :-( + <antrik> ah, right... but I the main problem was some other change + <antrik> (or maybe it never really worked, not sure anymore) + +--- + +[[!GNU_Savannah_bug 26751]] diff --git a/hurd/translator/tmpfs/tmpfs_vs_defpager.mdwn b/hurd/translator/tmpfs/tmpfs_vs_defpager.mdwn new file mode 100644 index 00000000..ef041a23 --- /dev/null +++ b/hurd/translator/tmpfs/tmpfs_vs_defpager.mdwn @@ -0,0 +1,73 @@ +[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +\#hurd, freenode, 2010 + + <slpz> humm... why does tmpfs try to use the default pager? that's a bad idea, and probably will never work correctly... + * slpz is thinking about old issues + <slpz> tmpfs should create its own pagers, just like ext2fs, storeio... + <slpz> slopez@slp-hurd:~$ settrans -a tmp /hurd/tmpfs 10M + <slpz> slopez@slp-hurd:~$ echo "foo" > tmp/bar + <slpz> slopez@slp-hurd:~$ cat tmp/bar + <slpz> foo + <slpz> slopez@slp-hurd:~$ + <slpz> :-) + <pochu> slpz: woo you fixed it? + <slpz> pochu: well, it's WIP, but reading/writing works... + <slpz> I've replaced the use of default pager for the standard pager creation mechanism + <antrik> slpz: err... how is it supposed to use swap space if not using the default pager? + <antrik> slpz: or do you mean that it should act as a proxy, just allocating anonymous memory (backed by the default pager) itself? + <youpi> antrik: the kernel uses the default pager if the application pager isn't responsive enough + <slpz> antrik: it will just create memory objects and provide zerofilled pages when requested by the kernel (after a page fault) + <antrik> youpi: that makes sense I guess... but how is that relevant to the question at hand?... + <slpz> antrik: memory objects will contain the data by themselves + <slpz> antrik: as youpi said, when memory is scarce, GNU Mach will start paging out data from memory objects to the default pager + <slpz> antrik: that's the way in which pages will get into swap space + <slpz> (if needed) + <youpi> the thing being that the tmpfs pager has a chance to select pages he doesn't care any more about + <antrik> slpz: well, the point is that instead of writing the pages to a backing store, tmpfs will just keep them in anonymous memory, and let the default pager write them out when there is pressure, right? + <antrik> youpi: no idea what you are talking about. apparently I still don't really understand this stuff :-( + <youpi> ah, but tmpfs doesn't have pages he doesn't care about, does it? + <slpz> antrik: yes, but the term "anonymous memory" could be a bit confusing. + <slpz> antrik: in GNU Mach, anonymous memory is backed by a memory object without a pager. In tmpfs, nodes will be allocated in memory objects, and the pager for those memory objects will be tmpfs itself + <antrik> slpz: hm... I thought anynymous memory is backed by memory objects created from the default pager? + <antrik> yes, I understand that tmpfs is supposed to be the pager for the objects it provides. they are obviously not anonymoust -- they have inodes in the tmpfs name space + <antrik> but my understanding so far was that when Mach returns pages to the pager, they end up in anonymous memory allocated to the pager process; and then this pager is responsible for writing them back to the actual backing store + <antrik> am I totally off there?... + <antrik> (i.e. in my understanding the returned pages do not reside in the actual memory object the pager provides, but in an anonymous memory object) + <slpz> antrik: you're right. The trick here is, when does Mach return the pages? + <slpz> antrik: if we set the attribute "can_persist" in a memory object, Mach will keep it until object cache is full or memory is scarce + <slpz> or we change the attributes so it can no longer persist, of course + <slpz> without a backing store, if Mach starts sending us pages to be written, we're in trouble + <slpz> so we must do something about it. One option, could be creating another pager and copying the contents between objects. + <antrik> another pager? not sure what you mean + <antrik> BTW, you didn't really say why we can't use the default pager for tmpfs objects :-) + <slpz> well, there're two problems when using the default pager as backing store for translators + <slpz> 1) Mach relies on it to do swapping tasks, so meddling with it is not a good idea + <slpz> 2) There're problems with seqnos when trying to work with the default pager from tasks other the kernel itself + <slpz> (probably, the latter could be fixed) + <slpz> antrik: pager's terminology is a bit confusing. One can also say creating another memory object (though the function in libpager is "pager_create") + <antrik> not sure why "meddling" with it would be a problem... + <antrik> and yeah, I was vaguely aware that there is some seqno problem with tmpfs... though so far I didn't really understand what it was about :-) + <antrik> makes sense now + <antrik> anyways, AIUI now you are trying to come up with a mechanism where the default pager is not used for tmpfs objects directly, but without making it inefficient? + <antrik> slpz: still don't understand what you mean by creating another memory object/pager... + <antrik> (and yeat, the terminology is pretty mixed up even in Mach itself) + <slpz> antrik: I meant creating another pager, in terms of calling again to libpager's pager_create + <antrik> slpz: well, I understand what "create another pager" means... I just don't understand what this other pager would be, when you would create it, and what for... + <slpz> antrik: oh, ok, sorry + <slpz> antrik: creating another pager it's just a trick to avoid losing information when Mach's objects cache is full, and it decides to purge one of our objects + <slpz> anyway, IMHO object caching mechanism is obsolete and should be replaced + <slpz> I'm writting a comment to bug #28730 which says something about this + <slpz> antrik: just one more thing :-) + <slpz> if you look at the code, for most time of their lives, anonymous memory objects don't have a pager + <slpz> not even the default one + <slpz> only the pageout thread, when the system is running really low on memory, gives them a reference to the default pager by calling vm_object_pager_create + <slpz> this is not really important, but worth noting ;-) |