Failed file system during testing

[dots89]

Hi there,

We are encountering a file system error during a long testing.
The problem is related to the fact that at the next attempt to write data and call 'ImapNodeBranch' function
value 'ulImapNode' is equal to 3840. This value is passed to 'RedBitGet' function and then 'ulBit >> 3U' (480) expression is larger than the 'abEntries' imap bitmap size. We have an array index out of bounds...
Maybe it's a bug and we need to add an index checking to the bitmap.c module.

Thanks!

[danielrlewis]

@dots89 I might know what's going on here. Based on the details you provided, I deduce that your block size (REDCONF_BLOCK_SIZE) is 512, since in that configuration the metaroot has a 480 byte bitmap. How large is your filesystem volume? Is it greater than 7.27 GB, by chance?

While looking into this, I discovered a bug: we aren't enforcing the maximum volume sizes which are inherent to the smaller block sizes. As a result, when the maximum is exceeded, it's possible to have an out-of-bounds access in the bitmap array, which would look exactly like what you describe.

For some background, I'll quote from the Reliance Edge Developer's Guide, specifically the "By the Numbers" chapter, which says the following:

Maximum volume size is mainly dependent on block size: larger block sizes allow larger volumes. Reliance Edge uses a free space bitmap, and for most common block sizes the limiting factor is the number of blocks which can be addressed by this bitmap. However, at very large block sizes (16 KB and up), the limiting factor is the number of blocks which can be addressed by 32-bit block numbers.

If using the POSIX-like API, the maximum volume size is as follows for each of the legal block sizes:

Block Size	Maximum Volume Size
128	84.0 MB (88,080,384 bytes)
256	840 MB (880,803,840 bytes)
512	~7.27 GB (7,801,405,440 bytes)
1024	~61.0 GB (65,531,805,696 bytes)
2048	~500 GB (536,938,020,864 bytes)
4096	~3.95 TB (4,346,641,121,280 bytes)
8192	~31.8 TB (34,978,482,094,080 bytes)
16384	~64.0 TB (70,368,744,161,280 bytes)
32768	~128 TB (140,737,488,322,560 bytes)
65536	~256 TB (281,474,976,645,120 bytes)

To elaborate on this a bit, block sizes less than 16384 bytes have a maximum volume size which is constrained by the implementation of the free space bitmap. The free space bitmap is split into block-sized "imap" nodes; and the bitmap in the "metaroot" block must be large enough so that it has a bit for each imap node. For example, with a block size of 512 bytes, the metaroot's bitmap is 480 bytes in size, large enough for 480 * 8 = 3840 imap nodes, and each imap node holds (512 - 16) * 8 = 3968 bits. If you do the math on that, 3840 imap nodes * 3968 bits per imap node * 512 bytes per block = ~7.27 GB maximum volume size.

However, there's a bug in that these maximums are poorly enforced. The Reliance Edge Configuration Utility (redconfig.exe) will, when static volume sizes are used, enforce the per-block size maximum. However, that doesn't help when sector count auto-detection is used, nor does it help when the redconf.[ch] files have been edited by hand. The driver ought to detect when the volume size exceeds the maximum size supported by the configured block size: if such a condition is detected, mount should return an error. Because we are not doing that, we try to use the volume despite the bitmap being too small, and eventually (when the allocation pointer exceeds the maximum volume size) we "walk off" the end of the bitmap. I can easily reproduce this. Obviously, a buffer overrun is not the ideal failure mode; it would be far better to return an error from mount, and we should add code to accomplish that.

However,—assuming I have correctly identified the root cause of the problem you are seeing (i.e., your volume is larger than ~7.27 GB),—you don't need to wait for us to fix anything. Since a block size of 512 bytes is too small for your volume size, you will either need to increase the block size or decrease the volume size. The former option (increasing the block size) is probably the most straightforward: you can use the table quoted above as a guide for how large the block size needs to be. However, larger block sizes do increase the amount of RAM the filesystem will need; if that's an issue, you could consider the latter option (decreasing the volume size), either by partitioning the storage device into multiple volumes, or by leaving part of it unused, such that each Reliance Edge partition is less than 7.27 GB in size.

[dots89]

Thanks a lot for your detail description! Yes, we use 16GB sd card and auto detect sector size with number of sectors. I will try to increase 'REDCONF_BLOCK_SIZE' .

[danielrlewis]

Commit f7111ee added code to detect when the volume size exceeds the maximum size supported by the block size, so that we fail gracefully in this situation.