in reply to Re^6: Adding cols to 3d arrays - syntax
in thread Adding cols to 3d arrays - syntax

Interesting re the 'session number' - in this scenario since there are duplicates, I assume that implies the dupes of the suspected session number can be considered a single transaction so can be restored in any order. Something to try tomorrow or perhaps next week due to weekend time constraints.

I recently tried a different chips bank sequence a week or so ago which has been successful and brought me to this point and the bitmap although still interesting did not 100% reflect that. I just rearranged it to properly reflect it and have uploaded it to some random file sharing site if you would like a peek.

http://www.filedropper.com/ppmfile2rearr_1

The lines which represent the LBA blocks are only 1 pixel wide so you need to zoom in to 100% to see them. They are the 32 vertical black lines. The thick horizontal wavy black band towards the bottom is all 0xFF in contrast to the black LBA lines which although close to 0xFF are not (they range from 0 to 16M or so). Being 8 bit the shades don't show this so I might see if I can write a 16 bit PPM tomorrow. Each pixel is a 16k page (or block depending on your terminology). The 8 bands of more even grey before each pair of black bands are interesting and will need more examination but I need to call it a night today so that will be tomorrow or monday.

Now I can see it more clearly there is a definite pattern going on there. It very much does look like 32 'heads of logs' with such order in it shared by all 32 vertical bands. Although I don't 100% know the relation between addressing and chips in the SSD there are 32 NANDs so with 32 of these bands it certainly looks like each chip is a vertical band. Towards the right of the image there is some other facets which look like partly written erase blocks at first glance.

One thing that confuses me is why the black area exists. Why would the drive bother to write 0x00 to these areas if erase resets to 0xFF - unless the OSes view of the drive is inverted perhaps. If so that would make perfect sense, with erased blocks at the head of each chip / log.

As for are there any valid LBA pages, it will take me some time to check. I might be able to do this tomorrow but depends on what the family want to do, monday otherwise.

Very pleased with getting this far so thanks for the ideas and assistance!

Pete

Replies are listed 'Best First'.
Re^8: Adding cols to 3d arrays - syntax
by jcb (Parson) on Sep 22, 2019 at 05:10 UTC

    While inversion is common in flash interfaces, you have also seen large blocks of 0xFF, so I doubt that is going on, unless some pages are inverted and others not. My guess would be that the drive writes all bits clear before erasing as part of wear-leveling for some reason?

      Yes I guess thats the only likely reason. Perhaps checking for bad blocks - the datasheet says blocks will start showing errors as time goes on. The chips only recognise errors in cells that are written to 0 according to the datasheet (understandably) so writing a page of 0s checks cell integrity before the reuse of each block (this will happen transparently to the controller chip).

      The datasheet states that 'the number of consecutive partial page programming operation within the same page without an intervening erase operation must not exceed 1 time for the page' which I take to mean the page can only be programmed once, but whether this is imposed by the chips themselves it does not say. If you don't care about the integrity of the data you could write 0s to the whole block over the previous data and save some cell transitions (meaning lifespan I would guess).

      The smoother grey sections of each column before the dark sections is curious and I need to investigate it further, but quick checks just now show data that looks fairly normal (not addressing, quite random). Must be something though.

      Re the log structure of a log structure disk, initially I was thinking this would be far too inefficient a way to use the disk because blocks that didn't need to be copy-written because they had not changed would require moving as much as the rest of the disk. But is it the case that such a compromise is indeed made with this method? It seems to prioritise wear levelling over total life span. A more complicated system could surely do a better job in terms of total write cycles per block needed (I imagine without having thought about it too deeply). Would a log structure really be this simple, a pure ring buffer (per each of the 32 chips)? Its certainly an absolute simple as anything way to achieve 100% perfect wear levelling. But the cost seems very high.

      Perhaps I can accurately determine the exact point the 'new data' starts for each column (they are not even) and write a map file to reflect this and see if the file system is in better shape as a simple exercise.

        No, if the datasheet says the block must be erased after one page write, issuing another page write before an erase is a good way to destroy some flash cells. A double write does not "save some cell transitions" but rather gives a good chance that that cell will not erase properly. This rules out any kind of incremental writes on this device, unless the firmware is really badly written. So "bank 32" is probably not (simple) validity flags.

        Those "smoother grey sections" might be the validity/log-state data. Presumably the controller maintains tables in RAM and flushes them to the NAND array at shutdown, possibly with some kind of checkpointing scheme ("bank 32"? the "extra LPNs"?) embedded in the map pages with normal writes. That this is not exactly robust against power failure does not rule out its use in this drive — we already know that the drive is not robust against power failure!

        Recopying live blocks as the "rewrite zone" approaches is what the early log-structured filesystems did, if I understand correctly. Trading a theoretical total life span for better wear leveling is not as absurd as it may sound — the SSD is dead as soon as it loses the last "extra" block anyway, no matter how many write cycles may remain on other blocks that are storing live data, since it can no longer provide its stated capacity and has no way to tell the host that the total space is dwindling, nor can most PC-ish (including Macs) filesystems handle storage devices that slowly dwindle away as SSDs do.