[Discuss] Arduboy Game Format (.arduboy) Guide

@MLXXXp @delegatevoid @crait

What if we add an EEPROM field ? This might be helpfull if you create an EEPROM Backup/restore TOOL. I know some games use a function to just look for an empty spot in EEPROM, but most games would just use a fixed place off addresses.

If the .Arduboy holds the range of addresses, you could backup the bytes and stick a game name to the backup.

"EEPROM": {
  "start" : "A number between 16 and 1023 and lower than the end Address, holding the first EEPROM address used",
  "end" : "A number between 16 and 1023 and higher or equal to the start Address, holding the last EEPROM address used"
}

I remember some addresses are used by the Arduoby library for sound and maybe other stuff, does anyone know at what address backup can start ? (obviously not zero) => found it and changed it

Why not zero? I don’t claim to know much at all about eeprom, or what arduboy uses by default, but surely backing up the entire eeprom, and restoring the entire eeprom would leave the arduboy in the same state as if you never performed the task. So no matter what parts of the eeprom a game uses, it’d be backed up correctly.

@spinal Nope, you would backup per game. But the first amount of bytes are taken by the arduboy config, you should not touch those. Just copying the complete EEPROM doesn’t give you much more value, if you could backup every games backup separately, you can choose which one to backup/restore.

Besides, we are talking about per game metadata !

@MLXXXp okay found it, 0-15 are reserved:

http://community.arduboy.com/t/rodger-dodger-and-endless-and-non-endless-dodging-game/2177/2?u=jo3ri

1 Like

@JO3RI @MLXXXp @crait if that’s something you guys use, then why not, it doesn’t hurt adding it.
It’s outside of my domain, so I can’t decide on that.

1 Like

It would also be good if the loaders provide a feature or tool that scans all the .arduboy files in their repository and uses this EEPROM information to report any overlaps discovered. A summary could be displayed, even if no conflicts were found, for the user’s benefit and for developers looking for an empty slot.

EEPROM information, if present, should also be displayed by loaders (along with the other information and images) in a given .arduboy file’s listing, again for the user’s benefit.

What should a game put in the info.json EEPROM fields if it does this?

Maybe we should add an optional boolean EEPROM field that indicates that the sketch does this. I can’t think of a good name for it. Maybe “various”? If “various” is set to true then the “start” and “end” fields must be set to cover the entire possible range that the sketch may use. If my above proposed scanning tool were implemented, it could flag sketches that do this and use it as a consideration in overlap calculations.

So long as your loading tool can deal with that, that’s fine. The real issue is which of them do you have to have for your chosen loader to work. If that’s different for different loaders (which seems to be the case now), then you’re going to get arduboy files that work on some loaders and not others. Without interoperability, there’s not much reason to have a standard.

I looked at it from the user point of view, so lets try it as a game author. Most of the required fields fall out of just developing the game. A title, author, and description should be artifacts of the process. A version you really ought to have if you’re going to distribute it, so you know what people have.

That leaves a screenshot and a banner. A screenshot might well be part of the development process. Then again, it might not. Requiring it might require extra work by the developer, so it ought to be optional.

The banner doesn’t come up during game development. From the description in the guide, it’s clearly part of the marketing campaign. If you didn’t do one of those, you don’t have a banner. It really ought to be optional.

Frankly, I no longer care. I’ll add default banner (the classic white cow eating marshmallows in a snowstorm) and screenshot images to arduweb so game authors can treat them as optional. I think it would be better for everyone if they were actually optional: Developers wouldn’t have to waste time deciding what to do about them, I wouldn’t have to add them to the zip file, end users wouldn’t have to download them, and loader authors could do something that suited their loader better than displaying whatever images authors who don’t want to provide them come up with.

1 Like

I don’t intend to develop a game loader, so don’t put much weight on my opinion, but I agree that “banner” and “screenshots” should be optional. All the other currently required fields are fine.

@mwm, If it’s decided they remain required:
Rather than arduweb defaulting to blank images, you could create a PNG banner containing the text from “title” in a standard font. For “screenshots”, use an image that says something like “No screenshot available” in a way that wouldn’t be confused with an actual screenshot.

Actually, the problems that I can think up for EEPROM addresses…

First of all, if we have multiple binaries, it might be best to put the EEPROM usage in each binaries object since each one could possibly use a separate location to save. The RPG I’m making has separate chapters that uses a different location for each game, which would be how you can carry over your save data from the previous game. Not only that, but each game could have multiple save locations for different things. Also, with different binaries for different systems, saving data may be handled differently, so we would need to think about a better naming convention for it other than EEPROM. As for storing the info into the .json, we would also need to figure out a way to meaningfully incorporate the data. Would it be a simple range of addresses? A list of addresses? A list of ranges?

I also considered that (but also remembered your concerns about things getting too complex :wink:). In a situation such as yours, even though each chapter uses different and possibly multiple locations, overall I would hope that all these locations reside in a single contiguous block. Therefore, one start and end address should be sufficient. Also, doing this would mean a nested object under “binaries” and you objected to nested objects, when I suggested it to allow common “binaries” titles.

If a different system saves data differently, we would add a new field specifically named and tailored for that new way. Overall (not just for storage), if for specific cases things get too complicated having multiple binaries in the same .arduboy file, then you would separate them out into multiple .arduboy files, similar to the way it would have previously been done for the production Arduboy and DevKit when there was only a single “device” field.

@mwm @crait I understand your concern. I had hoped to make it required as an extra for the users, but okay let’s not put the bargin on the shoulders of the “game providers”

“required”: [
“title”,
“version”,
“author”,
“binaries”,
]

If that’s different for different loaders (which seems to be the case now)

If a loader is pre-released and isn’t capable of “reading” old and new versions … well …

I agree with @MLXXXp on EEPROM

1 Like

I’ve updated, fixed and formated the “official” schema to include changes discussed here and elsewhere.

  • Fixes to the “schemaVersion” field.
  • Made “banner” and “screenshots” fields not required.
  • Changed the “EEPROM” field to lower case “eeprom”.
  • Proper formatting and validation of “eeprom” fields.
  • Added “variable” field to “eeprom”, used to indicate that the area used is not a fixed location.
  • Made “buttons” and “screenshots” require at least one entry.
  • Rearranged the fields in an order (which I feel) is desirable for a form that is built from the schema.

You can test out an example form created from it by going to the following URL:

http://jeremydorn.com/json-editor/?schema=N4IgJAzgxgFgpgWwIYgFwhgF0wB1QenwCsIB7AOwFpp5kA6UgJwHN8ATRpAM00oAYALPhqIkAYhAAaEJgCeOOGhCkARkThRMUmQEtMAG0Xod5LqTokKAAhNR9AVzZw2N8laRW6SRm3srSslZcOobaTtCMOjiYOhRKAMIUmHDkmBBWpFxWmPCuZhZkbsGGrnaOzq7unt6+/oHFcJJW9hAVKoHI5EjMcIzu5C76pEhOfTiMpMycCOmYpM04QyPVPn4BQSFwcLPzSG4Agqt1VsxICCbMVhCyEMkIdNrjpAqMMdtooCLIAGq9ELHkD4yeRGEAmZI9RjaBD2fQxRZwADyXDQAEZpOdyDoYQg0dInFwkLCtKh0boDKCAMqwURWABufwBYW2UEi0SZ6AAKrkvh4GYx/tYAO4wHSwbK5Ez5SxuKCkBCLHTbKxCvQwOhWbk6dK8mzpfmCtyokAAX2kMQpQLkCiUt0i5GY2gtoS5elC+JZbJicXQ+2ybrgQSYVjg/NkOQuVjlqSQJgqqojbgj6S8R3WDQeZpA4VZUW9gNQoGtoLtFydAaUABFPXmmR6IrWfSA/TmvQCMlkzH1Q71w6KHVGkrHyPG1SYJdqVrV05tM9IiTkmFaQbbMPbHeaK777IuofXc+ym9zAwuYMHMhLA6nHMcM1YALLEqIlU9MdJQPZWFSBroICrjnJAwgM4gLXC45xAA0OSLFd0FLB1y0tdBfgFOtsxrQ8CxAY8rEYOBDCQVp6UZawL2TKc1nqWdTXxJBkmXG04LAhD53IWRkTQABtUAu2QElszoxQzR4pg+KUNhBMoGI/1NABdTckJAStBP6FxpMUfc2yPXIJOSVSrAACmefMkH0ABKf0/w7XD8LgQjGkqABJSlESsAAOAA2PhUSDRg+I1ABROhmA1AAmbyAHZ+FCyhQoAZgyPpwtRKK+Bi+LOVC0LSQi1ABAAVhokAenIPCGJLZiNxAFJ7FxVBOJAAAFewAC8WvdEBKTPUhkj3ZscEVD9820asEFIbR9k0NDDg/JxtEa/Q6N43ptAAJUagBxNakCgMtpEpHAmDSbR721KAQHk8kXRADaUjKzTGywnDi2s04rODVNpyowwNUczArBhW4v0DCgQayQCg30IZVQdVArGatrDCaLrSB63omn2Ab9DFOiASaUbSAxqaKAxxhZochaltE9GrHWjamlWnaLmRw7XggJpTugCCcD8bGIHgKFC2BRiQHgqrnVBZqVD5gXmQbTClBwl5CgMiALODKZSHsHBKlgbwemsnnpe1ftLghz7KI2H6ip0JwUCF4tV3XRDrscu25YPYauVyZWKFViyRXmVk7OSNTJTt4H+zDq8aktjMirlOaHdg0XKpd0FEicKwnAZIYXg9rSnp0ns896f3rPN2Pb02AByHUtbJwNE7gCDvBJGCRbF9OlEOGJbgLx7FZ9v4/bVlUzyjPDBJcPZAjboUmAAa0qSu02+luirIewBnKp29quqktYGQcFVIVo+tbQfvcDX3yHLwPJ5DioIcBsV9J6gWriPth0gA3ILertbLM9hGD6F3kxZ20heJ0SUCAnQ3d0AAFVVoABlfIAyYIGKUolcZxAegra+VhkFoIvB4IUcAVBWBwN0QMXZLwUUAQ5EUYoYBRk/LYBwWcxp4TyPMYMIpBL8noa+Poqp+bbAglvRuiDQHgNTpAkA0D+JwIQZ1BuUAm6kCzsQgeBDsK5GIdZDw/w9Iil6IGCGUiNGDiznQ8iACZwlEnEgOksZFrSw3lmUQIQ5FdygTg/i3iwEKWugFZAIR3BsA4NsdIdCIDa1ZlofBXtmxuCCZE6JEBZgwDomwtw35mitDUkHIcmhhE7jPIwcudD4D6BwE0eJA0jqJQwTw7BflcHkAgnKBUexoLCwqgopRsDIiqMSL0rE1gnhTDOEQ1BuiUk4UMXQjwPTqGTLcNM6YEpcmMG3ukawHhGrxBaR/Xo2dQxikDMw8UyBAgFJaBUYMeEACO9gdB4RcO0CcKYq6OIcvE8UhEqjnx0KZbInByAQC4OcrZZwIIqD2COQWHdBn7wlkoAAQki85DRfwaXQvLRZuQ8UgQrpKZABs5jA0KRUFZX4cV9DYNqRYSBZBtECEsUYVCJgzJmBqLU6RAb/QKUcgAchtfAmL7yNXwAAKU2vgDajkABiVtAwRT4HwHAAAPFUttAzfNRAATlCrqqwopmDqk1CSzYAMWgiqweQMoTgXB/xjmvdVkjg4pH5j1CAcjvCcFkKo6keFfVnmOskjk+jaGbHxekPYLhL6YQOVkCIWwoWRp2DS5lEBWXsq+Zy4Y3K4X8qsGGzNfq0j2qBgUxqEqpUyvlYq5VaqGjpFRKFNy5rVRZ2+R5AQ5rLWsKTV+RaUBl5juYXpCg+hZACvgERDttbHWlE4f+JM/8/nrweBiEwf1EABtJNIPQR65GqHUJoUNPqs09XVQsmNOFSVWTHRLIxVxb3VseBMF4bxj1Fi3KiveLED6KwrNG7SFiAwtOMZU/6Kb8zkqAl+yNEEX1GGAxA9FW4QAqvjSBR9UH1X4uQ5+8Nd7MCZizK895nyuKKIIzJWSJoswqBMN4JUAGBlKCDWy1RKCS1IA8eqgNkGi5xsMAm/SiGARpvoSu6MmBhyRghmszYLhm7ZHmA8xYJbn67AOGvRd2xJPKmFTSjh5Q3Vbo9V9L1D4nwIlE3kyzzqN3FJ5aQOkBqrhwGoZwPSUEKDvnlDgDT6CIZPF8MTIz9m325GrHSAA0noJoSb8DBmpU8HzXDME7LcKDPz/0LyGFzuzKMOToh/CaMFqF+qcjZx0FwGFeFUhNZa2KYksgmhwEwFAPdIBMSHpmHiMEdxuOO3QJejQSSwO+k9exrojB15EYkyRkCTRZOHOPsplgfWLk+esfSrlwmSjx2kE8P9XGrRAZ49h0DGLXSWnE0PaDFIWl+teBcol7YyK2qAVApjmH7vyJw4pfDUnCOvcIRhsjp2RMNCKjnK5vi07SBqnVBqhwvojVDKlrQl0ntKUuRotbb3sggmsjjy2ORclqa0YGHJ6RvwpBPuFwwLguyLsnBZgpcA1TnJp8cYMSWCfUekLRj5zgGMYeZEdxQLG2M7jmFCwNjBg2qMxSr6wLQaHk/QNWX7IWyM7XzNk3JdlxQqB14VvoNvsDWFaG8lI1iXi8RzUpiY+h6FvTgPgJAA1BvDYm2Ns9o3k4ixm9ekJoJteO7cGb9s7vqZsAN82L8tuWkO9V35l3zqTy7dyEn6wegqG9A94Nq7vR/1Ai96QMBke0WPdw/H3PwYc9O7gPnsnMP9GTg/Pkk8VwLglE724UjgLR3pBrvsGuLSa6IJwDXJo3CLE5MKyOTPCerj2CBTPv0wZMXz+DDXSslBGojBX7B1pgYnDKYi87+wrugJ7+n1YWfVgADUVhj8L8xePHakQLVOFg6DXK3LFmjgosTpNEhinn5DLn3i2BhEhv9ieLFuXowMtEWvQvmhoM1kqF8lnh3lnk/i/rvvvh/vhjwuQoHhQCfn0DXPeN5k3DkpwJoL0OAaaDRt3nRjLvVCAPXsEiACXoCErhjv5hMHVFhsoGoLNqogFAFI1KtIiPeIUvrn3jhIocoaoe4FPLSmwFXr+jXjdkLK4pEGdiDlNiAP4A3nZICLHkoIJkNO2JOOYWCh4uno5OmgdtSjXGuM/vPmXiYMykNMqHTv9BDNoSoWod4HZAYfqlDPSN4AulYCqsGCYLcHsBok0JYovH1rcmyn5t4OKCskXmzo8lUFwOGnofEc1jYAhn1r0JiOETkv9J+HER4MzuQP4TypclrBAPOsDGzqZEMGEYYTaqBG3PpCkC4K4g4OZg6jSq0JEfMFZlnBDCkDEDwodFkjoCJp0Tsv9HKLCF8oGI8oYZvHtu3KDuCHAJCNCE5oYBxCekNiYNiLVGiB5BiEgDqh8XVKiGlHFI4egJSNcRknhFkunn6OQLVN+KIqKOKHgbtMEK0XGgKP9NEboSMJkukBcb9P9Hzoan1uQmzqiB5PpICfFPpIYFkgVi0rweCtSpscfDiZCRABBLMXIncQ8RiE8UiCiK8ZiP8V8T8X8TiGiECSCSAAFKyVEuydCVYLCQgPCYAUiQoCiVxvQotEDFibEfKTEgYQSaujSt+JgKSUaBSWOlSQlGOlME/H0HTnbiGG8kyfMJYuCWyTEhLiAFLvRgIVka8NoFyeIb6bwdLmngIRitIHVtoCItoEtpxu8CxkAA&value=N4IgzgxgFgpgtgQwGowE5gJYHsB2IBcAjADQgAuGZANjASCKQCYySoYAOFudDICArmShZUPUgDc0mbvnpMEZWrN4BzGDlRKQABX4AvPTV7t+AIyoYwsUctIZmCMSAhZmThKjJOwWfjkZO/KhU3r6oEDAAqsFO8AgYIbbOWHDsCDjYeEmm6Tho3hCa6lZYZGAEANqgFNRavABmCeoIcHUAvgC6pKYYOB4YLJXVlEZJjTR9rU7M4hgRdACCqIxmWACeIJ3dgmS45fhVyThkqFiJcnwQXFn0WyAwMOyncASg4v0I5kr1CFRgMKQwGQPF58AAGUjqALgtptIAA==&theme=bootstrap2&iconlib=fontawesome4&object_layout=normal&show_errors=interaction&required_by_default&no_additional_properties

With Firefox at least, I find it easiest to double or triple click on the URL (to select it all) and then right-click and select Open Link (whatever).

If everyone agrees that it is correct, the guide should be updated to match this schema (including making sure that the example info.json has all the fields and validates properly against the schema, which it currently doesn’t).

No one has responded to my previous post, so I assume either everyone is now satisfied with the schema, and .arduboy format in general, or is tired of discussing it and has taken a “whatever” attitude, or is busy with other things. I don’t want to spend much more time on it myself, but in the interest of making the schema more explicit and self-documenting:

With JSON a null string is valid, so even though some strings, such as “title”, are set as “required”, something like "title: "" would be valid. I’m sure that our intent in making some strings “required” was that the author provides at least one character in them.

The JSON schema specification includes “minLength” and “maxLength” keywords to constrain the size of strings. I’m thinking of adding "minLength": 1 to all the “required” string fields: “title”, “version”, “author”, button “control” and “action”, and the “filename” fields.

However, I have two questions:

  1. Should even the optional string fields, if included, be required to provide at least one character?

    It depends on how a loader parses and presents optional string fields. If they’re always presented as an empty title, even if not included in info.json, then it doesn’t matter. If the loader checks for a null string and treats it the same as if the field were not included in info.json, then again it doesn’t matter. If the loader expects that if the field is present it should contain something that can be displayed, then it helps to try and assure that.

  2. Do we want to restrict some/all strings to a maximum length? (in which case “maxLength” could be added to specify this.)

    I’m sure that loaders wouldn’t allow an “infinite” size string, and things like “title” probably have a limited display area reserved for proper formatting and presentation. Things like filenames could be limited by the O/S itself (255 is a typical maximum filename length).

So do any of the loaders presently have any of the restrictions above, or are there any thoughts on what they should be?

FYI the optional strings (other than filenames) are: “description”, “publisher”, “idea”, “code”, “art”, “sound”, and “title” for binaries and screenshots.

I would say that all strings should have a minLength of 1. If the string is required, allowing it to be empty sort of defeats the point. For optional strings, I can’t see where an empty string lets you do something that leaving it off won’t do, meaning it’s liable to be a mistake that we should let the user know about (is there some way to annotate that things as warnings instead of violations?). So those should have at least one character unless someone can come up with such a use.

I could have sworn I saw that a schemaVersion had been added, but all I can find is one mention of it here (of course, that the site subverts the browser search facility means I don’t really trust the search).

Did this get rejected, so that the now “old” versions arduboy files I’m currently building are simply broken?

Edit: I tried to tweak arduweb to deal with this in the suggested manner (i.e. fields being optional) by building hexes from binaries (and vice versa) and outputting them both, but this doesn’t work for screenshots. The previous version of screenshots was a list of strings, the new one is a list of dictionaries.

Just a very minor possible reason:
It could make it a bit easier for an author who was creating a JSON file by editing a blank template that might be made available. The author would be able to skip optional strings by leaving them null, instead of needing to delete the entire objects. Of course, ideally a program or form would be made available to authors, instead of a raw template. However, I think your “it’s liable to be a mistake” argument holds more weight towards the desirability of requiring at least one character.

I don’t think so. Keep in mind that the schema is just something that can be used by some kind of validator (or form generator) program to assure that a corresponding JSON file is correct. That validator can choose to do whatever it wants with the information provided in the schema.

Also, unless a loader program actually uses the schema to validate info.json, it can’t automatically trust info.json to be correct. The loader program must itself assure that the data provided is compliant with what it expects.

This is why I’d like to also provide maximum string length limits. Even if it’s just enforced manually by the author browsing the schema, it will prevent problems like “How was I to know that the loader would reject my title because it’s 700 characters long? You didn’t specify a limit!”. We should agree upon and document the maximum allowed length of all string fields.

Yes, “schemaVersion” has been added. The best way to determine what’s in the current info.json schema, is by looking at it.

Note that even though the info.json has changed in non backwards compatible ways since its inception, the current version is still considered to be Version 1 and including a “schemaVersion” field is optional. So yes, “old” versions are broken without a straightforward way of determining this.

This is because there currently exists .arduboy files with either old or new info.json files that both don’t contain the “schemaVersion” field. We don’t want to have to again go back and add “schemaVersion” to new format files. So, we’re already in a situation where there’s no “schemaVersion” field to determine if it’s the old or new format.

We’re currently in a chaotic situation where an info.json file with no “schemaVerson” field is considered to be Verson 1. The old non-compatible versions are considered to be beta.

A loader can choose to either reject an info.json that isn’t in the current format, or support the older beta format as well as the new V1 format, by checking and handling the differing field types within in the incompatible fields (“binaries” and “screenshots”).

In the future, if info.json becomes non backwards compatible, “schemaVersion” would be changed to require a value of 2 and be made a “required” field.

Sorry I’m busy with my dayjob mostly :slight_smile:

Last time I looked into json schema, the projects were pretty much dead. But you can do a lot more than validation & form generation with a good schema - look into the things people are doing with XML schema, like schema-aware editors and automatic test generation. Be nice to have something like the RNG compact format to, but that’s off topic.

If we use maximum length, it ought to be long. Loaders potentially have to deal with a hex file, so 64K doesn’t seem excessive.

BTW, thanks for doing the schema. Using a validator on generation means I can be sure my output is correct, and using it on input means my code gets simpler as I can assume a valid document everywhere else, and the user gets consistent, hopefully high quality error messages on invalid input! That there are people who don’t even create them for XML - which was born with schemas - still boggles me.

I would say making it optional is a mistake. Sure, for maximum backwards compatibility, you have to check the template explicitly if it doesn’t validate, but that’s true anyway, since not all “old” formats will validate now. If you use a validator (recommended) and it fails, you can either reject that file with an error message, or give it special handling. If you’re going to do the special handling, dealing with both isn’t a lot harder. If you’re not, then you now reject a few more files.

On the other hand, making schemaVersion optional means that the number of files without a schemaVersion is potentially much larger. Better to bite the bullet now while the problem is small and people are actively engaged in working on things.