On another front.
I’ve made a changes to bootloader so it uses pin 2 (PORTD2 / SDA) for flash chip select and loaded/flashed games successfully. Next thing is EEPROM backup
Technically it would be equivalent to doing:
uint16_t getDataAddressDebugMode()
{
// Read from progmem
}
uint16_t getDataAddressReleaseMode()
{
// Read from interrupt vector
}
But the advantage of the template is that you can set a constexpr variable to feed in as the template parameter.
Hopefully more like:
reinterpret_cast<uint8_t *>(&progDataPage)[0] = pgm_read_byte(&cartDataVector[1]);
reinterpret_cast<uint8_t *>(&progDataPage)[1] = pgm_read_byte(&cartDataVector[0]);
Or, if it were little endian…
*reinterpret_cast<uint16_t *>(&progDataPage) = pgm_read_word(cartDataVector);
How much memory space is having a library function for the application to just walk the extended memory, reading the headers itself and find its own save file.
I mean, I know we are trying to retain the information the bootloader already had gathered but, it’s a lot of abstraction for saving… how many bytes?
2 Likes
Looks like this a dead end. In the crt the __vectors table is defined as a table of hard JMP intstructions.
The actual interrupt handler is defined as weak and will end up to a jump to __bad_interrupt if the ISR is not defined somewhere else.
Actually I see no nice way of tricking this into:
reti
uint16_t some_address
1 Like
I think this would be a better option if it can be done, but it presents some challenges of its own.
Firstly, how does a game identify its slot?
Secondly, how to make sure that the progmem has some never-changing value that matches a never-changing value in the slot?
(Or at least, if the value changes in one place, it has to change in the other too.)
Thats interesting
How would you define cartDataVector to point to (progmem) address 0x0014 ?
Couple of hundred bytes maybe. It’s a waste of space and doesn’t resolve the issue
Thanks for looking into this
1 Like
const uint8_t * cartDataVector = reinterpret_cast<const uint8_t *>(0x0014);
Unfortunately reinterpret_cast can’t be constexpr (it’s allowed in GCC with -fpermissive, but is illegal in regular C++),
but I’m pretty sure the compiler would attempt this conversion at compile time since it’s just interpreting a specific address as a pointer, so there should be no ovehead.
Ultimately though, if you wanted you could do this instead:
*reinterpret_cast<uint8_t *>(&progDataPage)[0] = pgm_read_byte(0x0015);
*reinterpret_cast<uint8_t *>(&progDataPage)[1] = pgm_read_byte(0x0014);
Because pgm_read_byte is defined as:
#define pgm_read_byte_near(address_short) __LPM((uint16_t)(address_short))
And __LPM is actually a macro that generates assembly code for the LPM instruction.
But I’d advise at least doing:
constexpr uint16_t cartDataVectorAddressLow = 0x0014;
constexpr uint16_t cartDataVectorAddressHigh = 0x0015;
*reinterpret_cast<uint8_t *>(&progDataPage)[0] = pgm_read_byte(cartDataVectorAddressHigh);
*reinterpret_cast<uint8_t *>(&progDataPage)[1] = pgm_read_byte(cartDataVectorAddressLow);
So it’s clear what the numbers mean.
I’m forever telling FManga off for writing stuff like:
*reinterpret_cast< volatile uint32_t *>(0xA0002284).
1 Like
So I’ve been trying to follow the thread (there are a lot of posts!) but I am really starting to think that patching the game to have a pointer to the correct flash address just seems overly complicated and potentially error prone.
Why don’t we just have a simple custom file system, keeping the slot / linked list layout, and just leave space for a file name, e.g. 8.3 characters? Then for example the game can then just ask the library ‘get me the slot named MICROCTY.DAT’ and the library function will just walk the linked list looking for the matching slot. The advantage of this is that I can develop and upload the sketch with the Arduino IDE and I don’t need to do any weird patching magic between development / release versions. It would ‘just work’. If the extended data needs updating, its just a case of using the PC manager to upload the new .DAT file
Potentially making the file system generic enough means that when you distribute your .arduboy game you just need a game.hex and a game.dat file in the .arduboy zip file, and then the PC manager just needs to create slots for game.hex, game.dat (and potentially an extra slot for EEPROM, e.g. game.sav)
2 Likes
I like the idea of the getting the PC loader to fuse .hex, .dat and .sav files into a .slot file to be uploaded.
Essentially the slot system is kind of like a file system, but a less generic one.
What you’re suggesting about ‘finding the file’ is basically what @bateske was saying about finding the slot:
And my reply:
Why doesn’t it? What would make the following difficult to achieve?
uint24_t slotAddress = findSlotAddress("MICROCITY");
uint16_t dataAddress = getDataAddress(slotAddress);
uint16_t saveAddress = getSaveAddress(slotAddress);
1 Like
Having a generic file system opens up some interesting options though. If we split into game, data and save files into different slots:
- A game could easily create multiple save slots, e.g. for MicroCity I can just allocate a new file slot for a new save file, maybe all the saves are named CITYx.CTY etc. Up to the player how many cities they want to create (limited by how much free storage is available)
- Making extra levels could just be an extra data pack file. Maybe even have level editors within a game?
- Games could share saves or data, for example importing a character from one game into another
Each file slot could just contain a rather simple header, e.g.
7 byte ARDUBOY magic string
11 byte 8.3 file name
1 byte usage flag (i.e. is this a game, data or save? Used by the loader could look at this to see if it can be flashed)
2 byte previous slot address
2 byte next slot address
Then each slot that contains a game file could have its own header, e.g.
- Application name
- Author
- Version
- Group id
etc
The usage flag in the file slot header wouldn’t technically be necessary if the purpose is implicit from the file name extension, but maybe could help simplify the loader code.
1 Like
This is already possible with the slot system,
but the number of saves would have to be fixed size.
The code for a level editor would eat a large chunk of progmem.
Remember, the FX chip doesn’t magically grant you more progmem, it just allows you to move images and lookup tables into the FX chip.
I mentioned this earlier. It’s possible with the slot system, but only if games can organise a way to identify each other.
You don’t need both a file extension and a usage flag, they serve the same purpose.
You could just have the usage byte and let the PC loader interpret the usage byte as a particular extension.
The biggest problem with this approach is that the functions required to do all this navigation and file juggling are going to eat a lot of progmem.
1 Like
It was just an example of other use cases that could be available with the extra hardware. The level editor could even be a separate program to the game, so another case where you might want to share data between programs
Yes I definitely think we should at least have a sensible way of doing this. Your suggestion earlier to have something like findSlotAddress([string]) would allow for cross game reading of data
Agreed, we could have extension or usage flag but both wouldn’t be needed
The linker will only include the functions that you actually use so games that simply just need to read some extended data wouldn’t have to pay the progmem cost for e.g. file creation routines.
1 Like
Thanks @Pharap for the detailed examples
[/quote]
constexpr uint16_t cartDataVectorAddressLow = 0x0014;
constexpr uint16_t cartDataVectorAddressHigh = 0x0015;
reinterpret_cast<uint8_t *>(&progDataPage)[0] = pgm_read_byte(cartDataVectorAddressHigh);
reinterpret_cast<uint8_t *>(&progDataPage)[1] = pgm_read_byte(cartDataVectorAddressLow);
would be my choice (had to loose the * before the reinterpret)
It’s not about difficulty but about about not wasting precious progmem.
You still have to specify something (filename in your case) Why search for the file (and waste space for a search function) when the manager ca n display the page addresses also?
The only benefit I see in there is that you do not need to change the filename vs changing removing the page assignments after development. but is it worth spending more progmem on it?
Each section would require it’s own header. waisting storage a save file and EEPROM wouldt taKE 4K extra.
To limit that a file table would be required (and increase flash wear)
The discussion started about why I used a fixed area for development.
So I could use cart(); init for both Release and Development
vs
cartInit(); //Release
cartInit(progDataPage, saveDataPage); //Development
My goal is to save as much progmem as I can.
2 Likes
Sorry, I forgot the [0] was outside the bracket.
I don’t often mess around with pointers, I prefer to use references.
Perhaps it could be broken down a step further:
constexpr uint16_t cartDataVectorAddressLow = 0x0014;
constexpr uint16_t cartDataVectorAddressHigh = 0x0015;
uint8_t * progData = reinterpret_cast<uint8_t *>(&progDataPage);
progData[0] = pgm_read_byte(cartDataVectorAddressHigh);
progData[1] = pgm_read_byte(cartDataVectorAddressLow);
Or if the endianness matches:
constexpr uint16_t cartDataVectorAddress = 0x0014;
uint16_t * progData = reinterpret_cast<uint16_t *>(&progDataPage);
*progData = pgm_read_word(cartDataVectorAddress);
I think it’s worth investigating how much progmem it uses,
just so we can adequately balance up the pros and cons.
At the moment most of us are essentially flying blind and running off word-of-mouth for information, so it makes it hard to come to any conclusions about anything.
Because at the moment it seems that might be easier than coming to a decent arrangement for passing the addresses around.
I’m still doubtful that it’s going to be a lot of code.
It’s essentially just following a linked list.
If it were operating on a regular linked list, you’d be looking at something like this:
const Node * findNode(const char * target)
{
const Node * node = getFirstNode();
while(node != nullptr)
if(strcmp(node->name, target) == 0)
break;
return node;
}
This is a valid point. Most games will just want to find their own data and get on with things.
C code can deceiving.
assiging progDataPage with the above code takes 24 bytes already.
I’ll have a look at filename support.
1 Like
Although I think it is admirable (and important!) to make things as lean as possible and not waste precious progmem, I think it is also important that things are designed in a way that is easily approachable for beginners.
When I was envisioning a generic file system, this was because I was imagining that users could just drag & drop their .dat extension file using the PC manager and be able to access it in their Arduino sketch by file name. It means that the Arduino IDE can be used in the same way as you would usually with the Arduboy.
I guess what it comes down to is do we expect only advanced users to extend their game with the extra flash storage or do we want it to be as accessible as possible?
1 Like
C++
It’s possible that it could be reduced, but ultimately that’s not too bad.
The FX chip will be taking some of the load off of progmem if people won’t have to store their images in progmem anymore, so it’s alright if that comes at a progmem cost - hopefully it will balance out and still be a net gain.
Both alternatives would have a relatively easy to use interface for the programmer,
all this difficulty that we’re discussing is the back-end - the stuff that only the library writers and hardcore developers will care about.
What’s really at stake here are the flexibility and maintainability of the system.
3 Likes
We can try both methods out here, ultimately I’m most concerned about the user experience. I think its damn amazing how good you guys are at coding, the sheer mass of macros and memory defines here in this thread makes my head spin. (Sounds like @jhhoward too)
So the great thing with the dev kit is we can try a few things out, but I’m the decider. I’m really glad everyone here is putting in so much thought and effort into this part because this kind of discourse is what makes Arduboy really great.
And I think I’ve voiced my preference a couple times in this thread of doing something header/lookup based within the application. If we can pass memory locations intelligently and elegantly, then I’m open to that but at this point it seems like a lot of hoops to jump through.
2 Likes
That’s also what I have in mind.
I had some more thought about using a filename during my train ride. There’s a problem when hardcoding the name in the sketch (for the search) If two developers decide to name their program brick game only the first in flash memory could be located. 
A good PC manager will be most important for that. For those who want to write their own program, figuring out how how to get access to the programs data and save sections will be peanuts compared to learning the C++ language 
The first precaution would be to make the PC loader check and complain when uploading.
(The way I imagine it, the PC loader should be storing a copy of the entire FX chip in memory so it can easily do checks like that.)
This would also mean that the PC loader could protect games while they’re in the release phase (because of the SHA256 hash), so the only real damage potention would come from flashing via the Arduino IDE in development, where the checks can’t be run.
This much is true.
It’s not a very kind language if you’re a beginner,
but very few other languages would be suitable for such a memory constrained environment.