Communicating EEPROM usage / collisions

To this effect, at least the awareness raised by the multiple posts has been enough so that naturally people seem to have spread their games out.

I mean, at least we aren’t dealing with a nightmare situation where everyone has started at address 0. (which I think is reserved? aren’t people supposted to start at 16?)

A tool that also allows users to submit their game data would be great, but without building some kind of usermodel it is kind of a pain to do.

Fancy way might be to allow a community post where you could write it out and then an api could pull it.

But probably it will need to be actively managed and people can email when it needs an update…

Or it could just be a public csv and allow open edits?

The user-permitted addresses start at 16 because the first 16 bytes (0-15) are reserved, but users aren’t supposed to start there because of the wear levelling issue - if everyone starts at 16, the first block of memory would wear out faster than all the rest.

The problem with making it a single thread is that threads lock after 500 posts.

A better way would be to pull all threads from a specific category (i.e. #games and #games:demos ), or if a thread tagging system were enabled there could be a specific tag that registers a thread for cataloguing.

It would probably be easier to do via GitHub because GitHub has ‘webhooks’ for when repos are updated, but not everyone uses GitHub.

If it were hosted on GitHub then that might work, especially since that would potentially allow the option of having a server auto-update using webhooks.

I’d say avoid CSV for something that people are going to be editing though, because it’s positional rather than named, which makes spotting mistakes and validating the information much more awkward.

Either JSON, XML or something like recfiles would be best, but anything with key-value pairs would work.

Sure I mean we could build an entire framework in sql and develop a react server… battery backup… AI powered… triangulated space satellites. Orbital confirmation network. Sure only take 15 minutes crank that out.

Let’s see how the game jam goes, if it’s a big hit I’ll put a bounty on making a solution to the EEPROM.

I thought the idea was to make it easier to understand. If you ask x out of n users about what EEPROM is and what a savegame is, I’m pretty sure the savegame is better understood.

maybe I misunderstood something. I thought the idea was to document collisions for people to understand. But you’re talking about managin (future) EEPROM ?

I had some ideas for a simple dynamic EEPROM system but the semi-official-eeprom-list showed it couldn’t prevent collisions. Such a system could still reduce EEPROM wear but I don’t really see that as a problem.

Yes, each time you change a game that uses EEPROM, the EEPROM would be restored. But I don’t think it would be really a issue as EEPROM can be rewritten 10 times more than a program can be flashed.

A more wear reducing method that I recently thought up is to patch a game with EEPROM emulation so the FX chip is used instead of EEPROM. This would only work for games stored on FX chip though.

The advantage over the backup/restore method is less EEPROM wear and the bootloader doesn’t need to be updated.

I’ve looked a bit more into the possability of patching games for EEPROM emulation and found out the EEPROM library uses the avr libc eeprom functions and those where created from assembly source files meaning the code will be the same and can be located easily for patching. a EEPROM emulator can also be appended to the game code as there is additional PROGMEM space available because of the Cathy3K bootloader and if there wouldn’t be any space in a worst case scenario. Well then at least the number of collisions with other games has been reduced :slight_smile:

I’m pretty confident a EEPROM emulator is feasible. Maybe something to work on during the upcomming jam…? :wink:

BTW @MLXXXp I recommend to replace EEPROM.read with eeprom_read_byte and EEPROM.update with eeprom_update_byte in the Arduboy2 library. These changes can save 18 bytes.

Now you’re talking :money_mouth_face:

1 Like

That would depend on how the allocations happen.

Most allocation systems start at the earliest available memory and work their way forward, and in this case that would lead to earlier blocks being written to more often than later blocks.

To avoid that there would have to be some way of ensuring that allocations spread the wear across the whole address range rather than bunching up around particular areas (or at least in a way that minimises reusing the same areas continually).

That’s also why a file allocation table style system is bad - the table is going to be overwritten more than most other areas.

It would depend on how people use them.

I was under the impression most people are more likely to play the same game long enough for end up with multiple highscores/saves before switching, but I can only really speculate on what people do.

Yes, you mentioned. In terms of doing that via a library it would be a good idea, but I’m not sure how well patching a binary after the fact would work.

Overwriting the eeprom writing instructions with jumps/calls to the new FX-manipulating code probably isn’t too difficult, but surely you’d have to make sure the replacement is equivalent, and you’d have to find an area of progmem that is currently unused?
(That’s probably easier if you’ve got the .elf file than just the .hex file.)

That only solves things for FX units though, not the earlier units.

Surely for earlier units, that could be a problem because it would be attempting to overwrite progmem in use by the bootloader?

I presume you mean internally?
If so, yes, that will be a saving
I stumbled upon this quite a while back and forgot about it - I did the same thing to one of my games at some point.
(I think I mentioned it to Filmote at the time, but I can’t remember if I mentioned it to MLXXXp.)

Arduino’s EEPROM library is quite overengineered for how most people use it (EERef and EEPtr are overkill), and it’s a bit too self-reliant to play well with the optimiser.

All they really needed was a template function that calls eeprom_read_byte/eeprom_read_block and eeprom_update_byte/eeprom_update_block with the appropriate pointer cast and sizeof(T) - i.e. equivalents of get and put with simpler implementations.

EEPROM emulation (and EEPROM backup/restore also) wouldn’t work for the older units as there is no extra memory available. the Patching process would be done by the flash/cart builder tool. so patched games will only be part of a flash image.

Yes. based on an empty sketch using arduboy2::begin();

yes, the line

EERef &update( uint8_t in )          { return  in != *this ? *this = in : *this; }

effectively uses eeprom_read_byte and eeprom_write_byte where they should have made use of the existing eeprom_update_byte function.

1 Like

In that case there’s no issue. I was merely concerned that we might end up with .hex files floating around that would cause problems if someone tried to load them onto an old unit.

Then yes.
I believe the same applies to most games too.

I was envisaging something much simpler that doesn’t involve any patching. Why can’t you simply copy the entire EEPROM to SD when you flash the next game?

That would require a new bootloader with backup and restore feature.

Ah, I see.

What do you guys fancy?

  • I want EEPROM emulation and patch games using the flash/cart builder tool
  • I want EEPROM backup/restore feature in a new bootloader

0 voters

Before I vote …

How do we get all FX devices to use the new bootloader? I assume you need to flash that onto the machine. Would it be fair to say that if we updated the bootloader, people using the original bootloader will not be affected - they just will not have the EEPROM backup / restore feature?

Will the patching approach be consistent across all homemade devices? I assume it will be but maybe the chip being used comes into play?

probably a sketch to update the bootloader.

I asume you mean the original Arduboy FX bootloader?

correct.

yes but if the FX chip is addressed differently the emulation code will be slightly different.

In order for the bootloader to backup/restore, the flashbuilder tool still needs to add the eeprom start and end info though. each game requires a 4K EEPROM block (EEPROM emulation requires 8K per game or 4K and a shared 4K block)

Yes, sorry that was ambiguous.

Is backing the full EEPROM not an option?

backing up full 1K isn’t a problem but without the start and end addresses the full 1K must be restored as well. I don’t think that’s a good idea.

Sorry if some of my questions are simple …

Why not? Are you just concerned about wear?

OK … just thinking about solutions that don’t require us to catalog all the existing games.

Its not just the backlog that needs to be considered, its future games as well. The average first time developer will smash out a game with little understanding of what all this ‘EEPROM management stuff’ is and someone else (probably one of the contributors to this thread) will have to do the heavy lifting.

So if you need to have the start / end address anyway to do a backup / restore function in the bootloader then it sounds to me like the patching option is a better solution. This assumes its doesn’t need the start / end addresses.

100% of the voters agree with me!

I’d like to reduce wear if possible. But thinking about it more if PROGMEM wears out after 10K flashes. So a full backup and restore on each flash leaves 90K EEPROM rewrites. I guess that’s not so bad.

Nope not required.

Exactly. The reality is that no one swaps games that quickly … or do they?

Should I change my vote?

That’s not up to me to decide.


one more thing to consider:

The downside of bluntly backing up and restoring EEPROM for every game is that each game requires an additional 4K flash. for the patching method only the games that use EEPROM will require 4K (with 4K shared block for all games) or 8K

1 Like

EEPROM writes fail after a million writes, don’t worry about it, seriously.

The only thing you need to be concerned about, is that you don’t have a game that has some kind of bug that’s continuously writing to the EEPROM while it’s being played because then if someone was playing the game a lot (several hours a day) for many months then eventually you could hit this limit.

But as far as like save games and correctly formed use of EEPROM, it will be fine.

There are many other factors to consider when trying to do this, protecting the EEPROM should not be one of them.

To that fact, I mean I’m willing to warranty an Arduboy against any accidental flash damage. As long as it isn’t damaged intentionally by the aforementioned methods.