Eeprom collision

Technically ‘The begining’ is still in the future.

I’m still a bit confused.
If the write fails the block that failed to write is copied to another block?

I’m not sure why you’d need to copy the block in the first place.

I remember the pair who did Harambe’s Revenge (@MorganLapis and… can’t remember) saying something about using the whole of EEPROM for something, but I can’t remember the details.

1 Like

When writing to flash only 0 bits are programmed/flashed/burned and ‘1’ bits are left alone. You can overwrite an already programmed value as long as the changes are only 1 bits that change to 0 bits. an programmed 0 bit can’t be changed back to a 1 bit. The only way change 0 bits to 1 bits is to erase the complete 4K block.

A write fail is trying to write a value that has a bit set where the value in flash has a bit cleared. Example:

  • write 0x7F to xFF is no problem
  • write 0x80 to x7F will fail (write result will be 0x00)

This behaviour is the same for EEPROM but is handled internally by the MCU (and an erase is done in bytes instead of blocks)

If a single byte would change in a 4K save block (with bits set that where previously cleared). The old unchanged data before and after that byte must be copied to an erased block and then the new data can be merged

Ah, I forgot that part.

However, what I was going to suggest might still make sense,
but it depends whether or not you need to keep the number of erase cycles down and whether or not the amount of space needed would be an issue.

I was going to suggest maintaining two save blocks with a ‘save index’ as a header and alternating the blocks, which is what Pokemon did in generation 3:

https://bulbapedia.bulbagarden.net/wiki/Save_data_structure_in_Generation_III#Game_save_A.2C_Game_save_B

This way you end up erasing the old block every time and if the save is interrupted then the newer block is still intact.

It uses more flash space overall, but it’s a fairly robust system.

Basically:

  • Identify which of the blocks (i.e. the multiple of 4KB blocks) is older by reading the ‘save index’ at the start of both
  • Overwrite the older block with the new data, then increment the ‘save index’ and write that
  • If everything worked fine, the block just written is the ‘newer’ block and the block that was previously ‘newer’ will now be ‘older’ and thus will be overwritten next time
  • If saving was interrupted, the ‘newer’ block is still intact and can be loaded from, so people will only lose the progress that couldn’t be saved

And of course, this way you don’t need to worry about a write failing because you end up erasing a whole block, and the wear will be leveled between the two blocks because they alternate.

But like I say, it depends on whether having two blocks is too much of an additional cost.

It’s a shame the FX chip doesn’t behave that way.

Presumably using a chip that does that would be significantly more expensive, or harder to fit on the board.

1 Like

With over 100000 erase/program cycles, I don’t think we have to worry about that.

I have the alternate write/erase block in mind. But didn’t think about using an save index counter. Good idea.

there still need to be a partial update system as we just can’t store 4K of save data in ram / update in one go.

There are no flash chips with byte level erase features. It would be too time consuming to erase that way

Sometimes useless trivia about historical games comes in handy.

Anything not currently loaded into the game will be present in the ‘newer’ save block, so you should be able to just copy data from the ‘newer’ save block to the ‘older’/‘current’ save block.

As you say, it’s impossible to load an entire 4KB into RAM, so any save data that’s being modified will be smaller than that.

The hard part is specifying where in the 4KB block the data needs to be saved to.

If a game’s save data were only ever contiguous then it wouldn’t be a problem, you could just do (pseudocode):

uint8_t oldPageIndex;
uint8_t newPageIndex;
uint32_t saveIndex = findPageIndices(&oldPageIndex, &newPageIndex);

auto currentPageIndex = oldPageIndex
erasePage(currentPageIndex);
// saveDataIndex is the first address after the 'save index'

// Copy data from 'new' page to 'old' page
for(uint16_t index = saveDataIndex; index < startOfSaveBlock; ++index)
	write(oldPageIndex, index, read(newPageIndex, index));

// Write save data to 'old' page
for(uint16_t index = 0, writeIndex = startOfSaveBlock; index < saveDataSize; ++index)
	write(oldPageIndex, writeIndex, userData[index]);
	
// Finish copying data from 'new' page to 'old' page
for(uint16_t index = endOfSaveBlock; index < endOfPage; ++index)
	write(oldPageIndex, index, read(newPageIndex, index));
	
writeSaveIndex(oldPageIndex, saveIndex + 1);

But if save data doesn’t have to be contiguous then it becomes difficult.

But not so time consuming that EEPROM can’t do it?

I’ve been informed @Mr.Blinky is going and doing this all by hand for existing games on the “Goldcart”, from there we will have a table of available spaces left over after that.

1 Like

That’s impressive, @Mr.Blinky! :clap:

1 Like

My Shattered Lands engine uses 52 bytes of storage shared between each entry in the series. The games use the first 3 bytes to indicate the game ID. First 2 bytes is the game ID (SL for Shattered Lands) and the third byte for the game number in the series (1, 2, etc). So the planned 5 games in the series will share the same 52 bytes of EEPROM. Currently the engine just uses EEPROM_STORAGE_SPACE_START but can be modified to include an offset from there.

The save format allows for additional games/series to be made using the same engine and just change the 2 bytes for the game ID.

1 Like

What games are being included on the ‘Gold Cart’?

1 Like

To clarify with the help of community user @mameise I’m making a list of EPROM usage in programs based on @erieds repo and scanning the source files. With a complete overview of EEPROM collissions. It can be better decided what to do to limit the collisions.

1 Like

This is great. Could you put this up as a Google Doc spreadsheet? This would let active developers fix problems proactively. Would you also consider letting community contribute?.. (though I understand why you might want to curate this). :slight_smile:

2 Likes

A few years ago, I offered to set up a public wiki in this forum to maintain a record of EEPROM use. I got almost no responses or interest for it.

1 Like

His repo does not include my games, yet. :thinking: Also, it might be a good idea to just scam through this forum for other games which may not have been added.

Yes of course i searched for other games as well. Yours are also there. I am also not finished on adding and still searching. :wink:

They are too good to be left out :slight_smile:

Going to make a post about the “Goldcart” soon, @mameise has a github repo of it, so adding your game should be as simple as doing a pull request, it also will allow you to change the .png image that will show in the FX bootloader.

Standby…

2 Likes

Good Idea. I was already thinking about how to publish it as I doubt a single post here can contain all the data.

Too bad it didn’t work out back then. Would have saved a lot of time. But I’m gonna make the list.

From the list of 80-ish programs that use EEPROM I’ve checked 13 sofar and there are several collisions of which 3 overwrite Arduboy2 system settings and bootloader menu setting(I’m gonna claim the last EEPROM location for bootloader menu now :slight_smile: )

Don’t worry mine aren’t either. but they’re on our working list :wink:

for those of you who made programs using EEPROM it will be appreciated if you could post your EEPROM usage on this thread.

1 Like

@Mr.Blinky Let me know what you’ve got I’ll host the google spreadsheet.

I strongly suggest you use the first location, EEPROM address 0, for this, as we’ve been discussing. People are already aware (or should be or will be) that the first 16 bytes of EEPROM are for system use and off limits for sketches. I’ve already made changes for the next release of the Arduboy2 library giving you full control of address 0 (and it’s not being used now anyway).

I can think of two reasons for not using the last EEPROM address.

  1. It hasn’t been off limits for Arduboy sketches in the past so some may be using it legitimately. You won’t be able to check/modify all the sketches out there. You won’t even be aware of many. As I indicated above, no Arduboy sketch has “legally” been able to use address 0 since the first 16 bytes were reserved by the original Arduboy library in a commit on May 29, 2015, which is before the first Kickstarter shipments in February 2016.

  2. Using the last address could make things a bit difficult for porting to a new architecture with more EEPROM. Your last address might end up in the middle of the new EEPROM space. The first address is always going to be the first address, so porting would likely be less painful.

Edit: 3. You would have to document the fact that the last address is now off limits in a way that the word gets out. Address 0 is already documented as reserved in both the Arduboy and Aruduboy2 libraries.

1 Like

I think this is how you strongly suggest something, by the way.

and after you opposed to using one of the unused system flag bits. I modified the bootloader in homemade package 1.2.8 accordingly. However seeing several games corrupting the first locations I’m concidering moving the location to the end to reduce the risk of corrupting the boot menu flag. It would be annoying when suddenly the menu doesn’t show up after switching power on when it should or appear when it’s been disabled by the user.

I’'m still working on making the EEPROM usage list. When that’s complete I’ll get a better picture of what to do. If it remains with just a few games I’ll relocate their EEPROM addresses and I’ll stick with location 0 and revoke my claim. If the isue can’t be solved that easily and I can’t find another solution then I’ll probably use the last EEPROM location.