Eeprom collision

Development Project Falcon
#1

So this comes up from time to time, but will now be a bigger problem because we are including all the games together.

Some of the games EEPROM save data overlaps or collides with different games.

I noticed there is some discussion of doing a hashing system. Something that I wanted to do which is how it’s handled on gamebuino is just have a large table lookup where developers can basically “reserve” a memory address.

I think this is something that will have to be retroactively modified on any games that are included with the system. We (I) will have to make sure the eeprom works together on all included games.

3 Likes
Communicating EEPROM usage / collisions
#2

I thought the point of having save data in the flash was to solve the eeprom collision by swapping it into eeprom when the game is loaded.

#3

Oh, does the eeprom get swapped out by the bootloader also?

#4

Well, I thought it did… otherwise, what’s the save data for? :stuck_out_tongue:

2 Likes
#5

That was my intention. But I’ve put that idea on hold.

An alternative for EEPROM?

FX games can get an exclusive save ‘slot’ Besides the advantage of having more space, there’s no collision.

I think the EEPROM collisions could be ommited/minimized if there was a list of used EEPROM locations that could be queried easily. But since there isn’t one, most tend to pick some random location.

Edit:
when there’s an overview. Games that collide and have their source avialable could be adapted.

2 Likes
#6

Yes, I actually use a hashing system in Minesweeper.
I wrote it purposely to test the plausibility of such a system.

I’ve tested it, and if you overwrite any part of the data block it uses, including the hash and the data length,
it recognises that its data has been corrupted, and informs the user that ‘no save slot was found’.

Thanks to the data length it’s also backwards and forwards compatible if you increase the save size in later releases,
because the game calculates the hash correctly and then (for an older game) doesn’t touch the newer data.

As long as you never change the meaning of any part of the data, it will stay compatible.

It’s a pretty cool system.

Some of the implementing code isn’t compliant with the C++ standard though,
so I need to fix that.

I know a very good way to fix it though.

If you make it the user’s job to hand the function a buffer then the user can pass in the screen buffer instead of allocating a new one on the stack,
which gives the function an instant 1024 bytes to work with,
which is the size of EEPROM itself.

(I really wish I’d thought of that when I first created it.)

I started designing something like this once I think,
but my notes will be buried under a pile of paper somewhere.

We could build one, but it would be a lot of effort.

2 Likes
Communicating EEPROM usage / collisions
#7

Gamebuino just put a google spreadsheet that was open to edits.

I hate to say it but it may end up being a wiki post on the forum here.

I can do most of the heavy lifting, the first job would be to catalog all the games that use save data and where it is currently allocated to.

1 Like
#8

How did it handle clashes and address exhaustion?

I’m pretty sure we’ve got enough games using EEPROM that there will be clashes.

Now that the FX chip is coming,
I think the best option is to just make it easier to back up EEPROM,
either onto the FX chip or to computer files (or both).

#9

Yeah but backing up and rewriting the eeprom each time doesn’t make sense if in reality there are only 2 or 3 games that conflict with each other.

My argument is this: If we catalog all the games and we still have more than 50% of the eeprom left then we just have a public table to reference for people to play nice together. I’m all for having a backup and save utility within the bootloader for the eeprom but I don’t think it should be a default.

#10

Precisely, that’s why it should be optional.

Perhaps, but maybe that should be a job for another time?
I think the FX is already going to be consuming a big chunk of time and resources.

#11

Its gonna be my job to do that.

1 Like
#12

Maybe ask for help in the public section for volunteers to create an EEPROM list?

1 Like
#13

Yeah I’ll be making a public wiki thread at some point but I don’t expect most devs to chime in. Lots of old games where people don’t visit any more.

That will be the second part of the challenge making sure we have the source code to edit on the games not just the hex file for those.

1 Like
#14

Late to the party and maybe will say something stupid but if an external SPI flash is used to store game data and code why can’t it be used to store (multiple?) game saved state?

#15

It will be in the future.

The discussion was about EEPROM collisions by existing games. The idea is to preload Arduboy FX with 100 or more games and some of them may use the same EEPROM locations

1 Like
#16

Maybe it should be there from the beginning? It does solve the EEPROM collision problem for good and would provide a common mechanism for all ‘FX enabled’ games moving forward.

#17

It hasn’t been fully decided. I’m going to (try to) audit all of the existing games that use EEPROM, and see what we are dealing with here.

I don’t think that many games save at all, and I think most of them have on their own politely used different memory spaces.

Although, if you look at just microcity alone, it has the capacity to save the whole eeprom pretty much. But that is also a game where I would love to see the FX library integrated to move that to the external memory by default.

@dxb may have a good point, it might just be a good practice from the beginning as it is a sufficient catch all. How difficult is it to do do you think @Mr.Blinky? I think it’s lower priority now and I’d still want to look at how many games actually use the eeprom.

#18

I avoided using EEPROM on many occasions so i wasn’t overwriting someones save data. It would be incredible if we could save data to the FX chip. I think all Neo Retro games combine use 21 bytes of EEPROM.

#19

Need to work out my idea a bit more. The basic idea is to use (a multiple of) 4K block with a (multiple of) 4K backup block. The block just gets overwritten but on a write fail ( set 0 bits back to 1) the block gets copied to the backup block and the original block gets erased and becomes the backup block.

In addidtion to that when save data fits in one or more pages (256 bytes) the pages could be copied into the same 4K block multiple times to save on erase speed

There needs to be some housekeeping system to keep track of the actual data.

1 Like
#20

I guess my concern this is a lot of writing and backing up when most games don’t even use it. And the ones that do, barely do, only a few bytes for high scores or something. And just because we would otherwise be blind to what memory space we are using (without profiling, and recompiling like I had suggested) we are forced to assume the entire eeprom space is up for grabs.

Ok I think here is my new idea:

Try to refactor and catalog all games that are writing to eeprom so they play nice together. So that backing up the eeprom each time does not need to happen in nearly all cases.

In games that unavoidably use too much eeprom to move the save spots around “microcity” (is there any other game that actually uses a lot of eeprom?), then either it should be rewritten to use FX memory OR there is a utility on the flash cart, to back it up. A separate program the user can run to manage the eeprom banks.