1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
|
[[!meta copyright="Copyright © 2007, 2008, 2010, 2011, 2012, 2013 Free Software
Foundation, Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
id="license" text="Permission is granted to copy, distribute and/or modify this
document under the terms of the GNU Free Documentation License, Version 1.2 or
any later version published by the Free Software Foundation; with no Invariant
Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
is included in the section entitled [[GNU Free Documentation
License|/fdl]]."]]"""]]
# Implementation
* [[filetype]] option
* [[Hurd-specific_extensions]]
* [[Page_cache]]
* [[metadata_caching]]
* [[internal_allocator]]
## Large Stores
The `ext2fs` translator from the upstream Hurd code base can only handle file
systems with sizes of less than roughly 2 GiB.
[[!tag open_issue_hurd]]
### Ognyan's Work
* Ognyan Kulev, [[*Supporting Large ext2 File Systems in the
Hurd*|ogi-fosdem2005.mgp]], 2005, at FOSDEM
* Ognyan Kulev, [[large_stores]]
* <http://kerneltrap.org/node/4429>
Ognyan's patch lifts this limitation (and is being used in the
[[Debian_GNU/Hurd_distribution|running/debian]]), but it introduces another
incompatibility: `ext2fs` then only supports block sizes of 4096 bytes.
Smaller block sizes are commonly automatically selected by `mke2fs` when using
small backend stores, like floppy devices.
#### IRC, freenode, #hurd, 2012-06-30
<braunr> at least having the same api in the debian package and the git
source would be great (in reference to the large store patch ofc)
<youpi> braunr: the api part could be merged perhaps
<youpi> it's very small apparently
<antrik> braunr: the large store patch is a sad story. when it was first
submitted, one of the maintainers raised some concerns. the other didn't
share these (don't remember who is who), but the concerned one never
followed up with details. so it has been in limbo ever since. tschwinge
once promised to take it up, but didn't get around to it so far. plus,
the original author himself mentioned once that he didn't consider it
finished...
<youpi> antrik: it's clearly not finished
<youpi> there are XXXs here and there
<braunr> it's called an RC1 and RC2 is mentioned in the release notes
<antrik> youpi: well, that doesn't stop most other projects from commiting
stuff... including most emphatically the original Hurd code :-)
<youpi> what do you refer to my "that" ? :)
<braunr> "XXX"
<youpi> right
<youpi> at the time it made sense to delay applying
<youpi> but I guess by nowadays standard we should just as well commit it
<youpi> it works enough for Debian, already
<youpi> there is just one bug I nkow about
<youpi> the apt database file keeps haveing the wrong size, fixed by e2fsck
<pinotree> youpi: remember that patch should be fixed in the offset
declaration in diskfs.h
<youpi> I don't remember about that
<youpi> did we fix it in the debian package?
<pinotree> nope
<youpi> you had issues when fixing it, didn't you?
<youpi> (I don't remember where I can find the details about this)
<pinotree> i changed it, recompiled hurd and installed it, started a perl
rebuild and when running one of the two lfs tests it hard locked the vm
after ext2fs was taking 100% cpu for a bit
<pinotree> i don't exclude i could have done something stupid on my side
though
<youpi> or there could just be actual issues, uncovered here
<youpi> which can be quite probable
##### IRC, freenode, #hurd, 2013-03-19
<braunr> youpi: i'm back on polishing the large store patch
<braunr> did you remember seeing something else than the bzero/memset
out-of-scope changes ?
<braunr> (i mean, readily noticeable)
<youpi> I don't remember
<braunr> ok
<braunr> the original code already assumes mmap succeeds
<braunr> is it ok to consider the patch can do the same ?
<youpi> I'd say so
<braunr> ok
<braunr> youpi: actually, it looks like a good part of the patch isn't
related to large stores
<braunr> for example, in ext2fs/inode.c, there are calls to
dino_ref/dino_deref
<youpi> hum
<braunr> i'm not sure at all these have anything to do with handling large
stores
<youpi> but dino_ref is introduced by this patch, isn't it?
<braunr> it replaces dino
<youpi> yes, it replaces it because the dino() approach can't work beyond
2GiB
<braunr> i see
<braunr> youpi: i'd like to replace the recursive call to
disk_cache_block_ref with a goto, is that fine with you ?
<youpi> looks ok to me
<youpi> better than relying on tail recursion
<braunr> that's the idea :)
#### [[libpager]] API change
##### IRC, freenode, #hurd, 2013-03-04
<braunr> youpi: i don't remember exactly your answer when i asked about
considering the ext2 large store patch for merging
<youpi> there's just an API change that it introduces
<youpi> but otherwise I'd say we should just do it
<braunr> ok
<youpi> I've just checked the API change again
<youpi> it's simply adding a notify_on_evict parameter
<youpi> and a pager_notify_evict callback
<braunr> yes
<youpi> I'd say we mostly need to polish this
<youpi> ah, there is the same parameter on diskfs_start_disk_pager
##### IRC, freenode, #hurd, 2013-04-23
<braunr> and i'm working again on the ext2fs large store patch
<braunr> i finished separating the libpager interface change from the rest,
as Thomas suggested, so a new version should be ready soon
#### `EXT2FS_DEBUG`
##### IRC, freenode, #hurd, 2013-03-04
<braunr> youpi: do we want EXT2FS_DEBUG defined upstream ?
<youpi> I don't really have an opinion on this
<youpi> stuffing it in the large store patch is not good of course
<youpi> I wonder whether we want it by default.
<braunr> it is currently defined by the patch
<braunr> (in the debian package i mean)
<youpi> I've just seen that yes
<braunr> i won't include it upstream, and if we decide to keep this
behaviour, we can add a patch just for that
<braunr> or a define
<braunr> err
<braunr> a configure option
<youpi> ok
## Sync Interval
[[!tag open_issue_hurd]]
### IRC, freenode, #hurd, 2012-10-08
<braunr> btw, how about we increase our ext2 sync interval to 30 seconds,
like others do ?
<braunr> not really because others do it that way, but because it severely
breaks performance on the hurd
<braunr> and 30 seems like a reasonable amount (better than 5 at least)
That would be a nice improvement, but only after writeback throttling is implemented.
## Stripped vs. Unstripped `ext2fs.static`
[[!tag open_issue_hurd]]
### IRC, freenode, #hurd, 2013-09-17
<teythoon> I always had some trouble with dropping a rebuild ext2fs.static
into my test system and I never figured out why
<teythoon> I just followed a hunch and stripped the binary, and all of the
sudden it works
<teythoon> any ideas why?
<tschwinge> teythoon: I quick search found me:
<https://savannah.gnu.org/bugs/?8497> and
<http://news.gmane.org/find-root.php?message_id=%3c4090243E.2040605%40comcast.net%3e>.
<teythoon> tschwinge: ugh, thanks for the pointers ;)
<tschwinge> teythoon: They won't help too much I fear. Anyway, good
intuition (or whatever) ;-) that you found this out.
<tschwinge> teythoon: Not exactly related to stripped/unstripped per se
(that is, debug information), but in the past we've had an issue about
relro (see binutils/glibc, <http://www.airs.com/blog/archives/189>),
where a variable (that erroneously happend to be in such a read-only
section, if I remember correct) was tried to be modified -- which worked
"sometimes": depending on where exactly it was located in the binary
(which shifted around a page
<tschwinge> boundary by stripped/unstripped), it'd segfault or not. Burnt
several days on that before Samuel (IIRC) eventually figured it out.
<teythoon> tschwinge: well, thanks anyway ;)
## Increased Memory Consumption
### IRC, freenode, #hurd, 2013-09-18
<braunr> ext2fs is using a ginormous amount of memory on darnassus since i
last updated the hurd package :/
<braunr> i wonder if my ext2fs large store patches rework have introduced a
regression
<braunr> the order of magnitude here is around 1.5G virtual space :/
<braunr> it used to take up to 3 times less before that
<braunr> looks like my patches didn't make it into the latest hurd package
<braunr> teythoon: looks like there definitely is a new leak in ext2fs
<teythoon> :/
<braunr> memory only
<braunr> the number of ports looks stable relative to file system usage
<teythoon> braunr: I tested my patches on my development machine, it's up
for 14 days (yay libvirt :) and never encountered problems like this
<braunr> i've been building glibc to reach that state
<teythoon> hm, that's a heavy load indeed
<teythoon> could be the file name tracking stuff, I tried to make sure that
everything is freed, but I might have missed something
<braunr> teythoon: simply running htop run shows a slight, regular increase
in physical memory usage in ext2fs
<pinotree> old procfs stikes again? :)
<teythoon> braunr: I see that as well... curious...
<braunr> 16:46 < teythoon> could be the file name tracking stuff, I tried
to make sure that everything is freed, but I might have missed something
<braunr> how knows, maybe completely unrelated
<teythoon> the tracking patch isn't that big, I've gone over it twice today
and it still seems reasonable to me
<braunr> hm
# Documentation
* <http://e2fsprogs.sourceforge.net/ext2.html>
* <http://www.nongnu.org/ext2-doc/>
|
