Ah OK, so the reason I haven’t found any .arduboy packages with multiple binaries (or indeed any with Mini binaries) is because there just… aren’t any yet?
I propose "ArduboyMini" as the device identifier in .arduboy packages then. This would mean the device identifiers would be:
The game would ‘break’, but the bootloader would be fine, so at worst we’re going to end up with a few people saying “it’s not working” later down the line, and even then only for the handful of games that are FX-enabled, which is still a very small proportion.
As long as the developers for those (very few) games provide a mini-compatible .hex some time in the near future, that should at least tide things over for a while.
Despite how many people have actually produced .arduboy files for their games, in practice they’ve hardly ever been used.
There have only ever been three tools that actually used them, two of which are now defunct and inaccessible (their download pages are 404ing), and one that now only exists in source form in the Team ARG Museum.
Overwhelmingly people use either .hex files or source code. A large part of that is because Eried’s repo only accepted .hex files (and only .hex files compatible with the official Arduboy, not clones or the devkit), and because the emulator embedded in the forum only ran on uploaded .hex files (again, only .hexes for official units, not clones or devkits).
There has also always been a lack of ‘proper’ tools for creating .arduboy files, so I also expect there’s probably quite a few .arduboy files that somehow don’t follow the spec and have never actually been ‘validated’. (I don’t know about other people, but I always made mine ‘by hand’ with a text editor and some kind of file zipper like 7Zip. I would expect other people did more or less the same thing though.)
In other words, .arduboy has been a bit of a mess for years. This is nothing new.
If there’s a call for reexamining the format and maybe changing how it works, I’m all for it, and it’s probably better to do something like that sooner rather than later.
Though I think it would be better if it were properly discussed with a group of people, particularly those who are going to have to be dealing with it, as that should help to foresee and avoid potential problems, and hopefully satisfy enough people’s needs and wants that the format shouldn’t need to change again for quite some time.
I.e.
^ this.
Ahem…
Ah, thank you for an example. I was having a hard time finding one.
Then I would like to change that. I would like to have a relatively robust (and standardized) .arduboy package creator in my toolset, and as such, I would like to formalize some of these things. I find that leaving the floor fully open tends to lead the conversation nowhere, so instead I will propose the changes I’d like and then there can be discussion and changes. As such:
.arduboy files. This means flashdata and flashsave fields in each binary, and all the usual fields from schemaVersion: 3. I’ll show an example latertitleimage field to the binary sections, to indicate to a flashcart builder which image should be used as the title image when “compiling” the flashcart slot for this particular binary. This field is optional, and I suggest that an implicit “title.png” be used for flashcart creation if none is specified. Again, examples laterbinaries array:
flashdata or flashsave is set when the device is Arduboy, this is an error and package readers can say as suchflashdata and flashsave, but it isn’t necessaryflashdata and flashsaveSo, an example might be:
{
"schemaVersion": 3,
"title": "Game thing",
/* ... */
"binaries": [
{
"title": "Game thing - Arduboy",
"filename": "GameThing_arduboy.hex",
"device": "Arduboy",
"titleimage": "titleimage-special.png"
},
/* Just an example, you wouldn't normally have both fx and non-fx binaries */
{
"title": "Game thing - ArduboyFX",
"filename": "GameThing_arduboyfx.hex",
"device": "ArduboyFX",
"flashdata": "gamething_fx.bin"
/* No titleimage means implicitly use title.png, if it exists */
},
{
"title": "Game thing - ArduboyMini",
"filename": "GameThing_arduboymini.hex",
"device": "ArduboyMini",
"flashdata": "gamething_mini.bin"
}
]
}
Here’s my first thought of how I’d like to incorporate homemade configurations:
Arduboy-Board-BootloaderSize-Display[-FlashSelect]
Then “Arduboy”, “ArduboyFX”, and “ArduboyMini” would be aliases for this long form, for example:
| Alias | Long Form |
|---|---|
| Arduboy | Arduboy-leonardo-4k-ssd1306 |
| ArduboyFX | Arduboy-leonardo-3k-ssd1306-d1 |
| ArduboyMini | Arduboy-leonardo-3k-ssd1306-d2 |
Ah, that’s a great idea! Can you explain more about “FlashSelect” and what the values there would be?
And all those configuration options are great, they can really cut down on incompatible software. I know the bootloader size can be used to limit how large the binary can be, but I’m wondering:
“FlashSelect” means which hardware CPU pin is connected to the chip select of the flash chip. I took inspiration from the homemade package – looks like D1, D2, and E2 are used. Arduboy and Arduboy FX use D1, while the Mini uses D2.
Questions for @Mr.Blinky I think. I think at least the board can be identified via USB.
By Arduboy do you mean an Arduboy with the FX modchip? Or is this pin used even without the modchip? Sorry if that’s a stupid question.
So, let’s say some of this information is not able to be queried from the device. Are those 4 bits of data just used for humans to manually inspect and determine if the binary is right for them?
My bad. Arduboy doesn’t use a flash select, you’re right. This suggests to me a modification of my original proposal – edited.
EDIT:
If people adopt a standard like this for distributing games, it could greatly improve tooling.
Example: suppose you want to add a new game to the flashcart of your Arduboy Mini. You open up the Toolset and download the cart off of the device. Maybe at this time the Toolset asks you what kind of device it’s working with and you select “Arduboy Mini” (or even better, if it’s possible to ask the bootloader directly what kind of device/screen/etc it is, like you suggested earlier). Then you add the .arduboy file for the new game to the flashcart. The Toolset verifies that the .arduboy file contains a binary for the type of Arduboy you have and adds the appropriate hex/bin data.
Next time just make a thread in the ‘help & information’ section, usually someone comes up trumps.
(Though if you did mention your problem earlier in the thread, I must have completely skimmed over it, otherwise I would have sent you that link earlier.)
I’m probably going to sound really pedantic and nitpicky because most of my objections are about details and wording…
By sole complaint here is naming consistency.
schemaVersion and sourceUrl already camel case, so I think any newly added field identifiers should also be camel case for the sake of consistency. Thus flashData and flashSave rather than flashdata and flashsave.
That aside, those fields make sense.
Personally I think cartImage would make more sense because ‘title image’ is more ambiguous. (I can imagine a user being stumped and thinking it’s asking for a picture of the titlescreen.)
Though if it is going to be ‘title image’ then it should at least be titleImage for the reason mentioned above. Consistency is the most important thing.
Similarly, I think something like cart.png or cartImage.png would be less ambiguous.
I’m perfectly happy with all the identifiers, they follow the established pattern.
The only thing I’d suggest is that perhaps an ArduboyFX/Mini ought to be required to have at least one of flashData or flashSave since not having either doesn’t really make sense.
Unless there’s some scenario where not having either actually would make sense?
(Or perhaps such a requirement isn’t representable via a JSON schema? This is the only time I’ve ever had to deal with one so I haven’t got around to investigating what their features and limitations are yet.)
Personally I’d be perfectly happy with ignoring unofficial configurations entirely. At the moment the consensus is that anyone who has the skill to build their own ‘homebrew’ device ought to be able to work out how to compile games from source.
That said, I think @brow1067’s suggested solution is appealing. Though it might mean that either nonstandard devices can’t be covered by the schema, or the schema is going to have to have a ridiculously long list of valid options.
I’m not sure anyone ever actually used the schema, but I still think it would be a good idea to try to produce an updated one for the sake of having a kind of ‘authority’ on the situation.
(Besides which, at the moment the old schema is the only thing I’ve got to tell me what the original format was actually supposed to be, so at the moment I’m very much in favour of having a schema as handy documentation if nothing else.)
Two suggestions:
kb rather than k for the bootloader size suffix, purely so that anyone accidentally running into the string out-of-context will be more likely to realise that it’s a size and not a part number.One thing I’d like to look into revising is how the contributors to the project are documented. I’ve only just started thinking about it so I don’t exactly have all the details worked out, but I’ve given it some brief thought…
At the moment there’s several fixed roles that contributors can take (e.g. ‘publisher’, ‘idea’, ‘code’, ‘sound’, ‘art’), each of which take a single string (even when there might be more than one person contributing to that particular aspect of the game).
At a minimum I’d like to see that single string replaced with an array of objects that include information about each of the contributors, including the option of providing URLs for the contributor’s forum account, GitHub, et cetera, and possibly an option for providing the person’s ‘real name’ and other pseudonyms alongside the name they’re actually being credited under. Such information would all be optional, but providing it would allow tools to display the information and perhaps provide hyperlinks.
I’m also wondering if instead of a fixed selection of roles, it might be better to just have a single credits list and allow people to write whatever roles they want into some kind of ‘role’ field. (There could still be a small set that are ‘recognised’ roles in case a loader/editor wanted to do something special like display a little logo next to the role name or group people into categories after-the-fact.) Much as I don’t want to overcomplicate things, I think it would be nice to give people a bit more flexibility.
That aside, I think the genre field might need revising. At the moment there’s a limited number of fixed genres, and I think that either ought to be expanded or people ought to be allowed to put anything as a genre (and again, maybe have a small number of ‘recognised’ genres so that loaders/editors can react specially).
This probably isn’t the sort of stuff you care about at the moment, and it doesn’t matter if this gets kicked down the line, I just thought I’d put the idea forward while I thought of it in case anyone else has any opinion on the matter.
Unfortunately, many packages are already using flashdata and flashsave lowercase, that is something I have actually checked. I think it would be unwise to change it now, or to support multiple naming schemes. Case insensitive JSON is an entirely different thing.
Ironically I chose ‘title’ to be less ambiguous because most of the tooling around the carts uses the term “title” by itself to refer to the image. Though I suppose end users wouldn’t know that. In that case, I’d say just image is good enough for the binaries. If you do want it to be the full cartimage, I think it should be lowercased, since it would at least have “local” consistency with existing fields (again, flashsave and flashdata are already in active use). I’m fine with cart.png as the default image name.
Not necessarily! The mini has I2C while the FX does not; simply having that capability would make your Mini game incompatible with the FX, and thus those fields would not be required. Considering that, I figure there may be FX-specific features that won’t work on the mini (do they use different screens? Maybe the 4 color mode doesn’t work on mini? etc)
HIGHLY AGREE! I would be happy to simply remove all the arbitrary contributor-related fields (except for ‘author’, as that’s required and is written to the cart) and add a contributors section with arbitrary titles. Example:
{
"contributors" : {
"Mr.Blinky" : "Everything",
"ArtDude" : "Art + Emotional Support"
/* etc */
}
}
EDIT: I see you wanted to maybe provide more information; they can be objects if you want (with like github urls and such)
Perhaps it might find more use in the future, but at the moment manual inspection is all we really need.
I don’t know whether there is some way to get a device to admit its hardware spec over serial, (at the moment I’m presuming not,) but what I do know is that if there isn’t then people are going to be coming to the forum with issues and it’s going to be up to the forum users to diagnose and resolve the problem, so anything that can help us to do that with fewer headaches is going to be a big bonus.
Also, people who use homemade devices are already expected to know things like their screen type, processor type, and pin configuration, so expecting them to deal with specification strings really isn’t a big ask.
There will probably be people turning up with out-of-spec knock-offs too, but we shouldn’t worry too much about those.
How many is ‘many’?
(I’m expecting it to be less than 20, but I’m prepared to be surprised.)
If it’s not too many then it might be worth fixing this while it’s still only a minor issue.
The fact that exists disturbs me.
I don’t think even half the developers know that.
We really need some sort of FAQ and/or official glossary.
I definitely think cartImage would be better.
Especially if the default is going to be cart.png.
I would presume not, but I’ll accept for now that there might one day be cause to not have either flashData and flashSave entries.
Again, if fixing the stuff that’s already in the wild wouldn’t be too much hassle I think it would be better to enforce consistency now while the option is there.
If it is going to be too much hassle then I’ll concede, though again I think cartimage would trump image. The more obvious the names are, the fewer confused questions we’ll get later down the line.
Fair enough.
As far as I’m aware it’s the same screen and the same CPU.
The pin assignments are probably different, but nobody’s going to be accessing the pins without opening the case.
There was that secondary processor that the FX used to have but I think it was removed for the commercial units, or it only came with the ‘upgrade it yourself’ versions or something. (I can’t remember the details.) Nobody ever did anything with it, either due to lack of knowledge or impracticality.
It might be an option to make that optional and then have loaders build an author string from the list of contributors.
Though that might be too much hassle. I’m mostly thinking of user convinience rather than programmer convinience.
I suppose that could just be an optional feature of a generator instead of the responsibility of a loader. (‘Loader’ = thing that reads an .arduboy file, ‘generator’ = thing that creates an .arduboy file.)
Even with that functionality author wouldn’t be redundant because it might be a team name (e.g. Team ARG, Press Play on Tape).
Yeah, I was thinking something more like:
"contributors":
[
{
"username": "Filmote",
"name": "Simon Holmes",
"aliases": [ "That guy that makes all the games" ]
"contributions": [ "Code" ],
"urls":
[
{ "site": "GitHub", "url": "https://github.com/filmote" },
{ "site": "Arduboy Forum", "url": "https://community.arduboy.com/u/filmote" }
]
},
{
"username": "Vampirics",
"name": "Stephane C",
"aliases": [ "That guy who makes all the art for all the games" ]
"contributions": [ "Art" ],
"urls":
[
{ "site": "GitHub", "url": "https://github.com/vampirics" },
{ "site": "Arduboy Forum", "url": "https://community.arduboy.com/u/vampirics" }
]
}
]
So they effectively end up becoming mini user profiles.
(I was going to add nationality and spoken languages too, but I stopped myself. I should have been a data entry clerk.)
For ease of use, ‘generator’ software could save user details so that later on you only need to check off some names in a list and it’ll inject the full profile.
Now I think about it, there is actually another semi-official Arduboy version: the prototype ‘Project Falcon’ FX units. But there were so few of them and they were handed out to people who were technically-minded so it’s probably not worth considering them. Just thought I’d mention that they exist.
I’m fine with cartimage or cartImage if you find it better than image, I don’t have a strong opinion.
I tend to find even “supposed small scale” fixes to end up being tremendously awful. But I will leave that up to the community if they want to change the naming convention there. Consider that it’s not just the packages that need to be updated (they aren’t all from the same people, btw), it’s also getting all tooling that’s currently using this (such as @brow1067’s debugger) to change what they accept. Is it worth it just for naming consistency, when it’s just natural for these kinds of things to occur over a long period of time? I don’t know. But I’d rather not put everything on hold for an indeterminate amount of time while we try to wrangle everyone into changing their existing files and in all the locations (the cart builder website stores some .arduboy files IIRC), especially with Arduboy Minis getting close to going out to people (or so I assume).
I’m against this for reasons too long to explain (unless you want me to). I’d rather have a field required for several things (cart building and beyond) to just be required and not computed. It’s too important. I don’t care what users put there, it just needs to be there.
Hm. I feel like this may be worse than what we have now in the arduboy format. I know that having a limited set of fields sucks but at least users can understand it and are more willing to fill them out. But if you put up this many roadblocks to setting contributors, chances are they just won’t bother. I would say ANY indication of contribution is better than none. How often will people (contributor or not) REALLY care about their true name vs username vs aliases. If they do, you can get that on their github or other URL. It feels like preening, which can be done in one of your URLs if you care, but not in the data format. Furthermore, URLs are self describing, so adding additional descriptors is an unnecessary annoyance IMO. If a parsing program REALLY cares about “what kind of” url something is, they can parse the URL extremely easily and determine it. Neither github.com nor arduboy.com will be changing their domains anytime soon. I would rather see something like this:
"contributors" : {
"Filmote" : {
"contributions" : [ "Code" ],
"urls" : [
"https://github.com/filmote",
"https://community.arduboy.com/u/filmote"
]
},
"Vampirics" : { /* etc */ }
}
This is easier to read, more to the point, and easier to write. This unfortunately IS something I feel strongly on, having had a lot of experience with data storage / retention / formatting / documentation / etc. I will push against this unless there’s a really good reason to have all this data right in the format. I’m all for a list of urls; that can act as a relational database for your additional user data in this case. I’m sorry to be harsh, this is just one of those things I’ve seen turn into a nightmare too many times to want it to happen again.
I’ve heard about those, thanks for reminding me.
Yes I’m quoting myself. I get the feeling this may be a point of contention, so I would like to explain further by giving an example of where this leads (these are not my opinions, just examples):
(I could keep going)
Your proposed contributor format is already longer than the binary data. This by itself should be cause for pause. Now consider what people would want or need from a ‘mini user profile’
No, but I’m expecting there’s probably no more than ten people that have ever produced them, including myself.
So far I count three known sources.
However one of those, Team ARG, might not actually be an issue because I think all their .arduboy files were nuked at the same time the GitHub was.
Like I say, they’ve never really been very popular because the tooling has never really been there, and there was never a coordinated central repository like there was for .hex files. Most of the ones that exist have been created in case something ever actually materialised.
Though I don’t know if Filmote’s been collecting them on the cart website or not.
I think that’s probably four tools at most, not counting the no longer maintained Team ARG uploader.
.arduboy files. I’m not sure if they do or not.Having had to deal with both C’s standard library and PHP in the past, I’m inclined to say ‘yes’, though that’s under my existing assumption that the number of existing .arduboy files is in the low tens.
We don’t have to.
We could have a transition period during which both forms are acceptable and then later down the line remove the old style when it seems safe enough to do so (or loaders can continue to accept the old style as a precaution and just produce a warning about it).
Some have already arrived, some probably won’t arrive for quite a while yet. International shipping varies wildly, and I think the devices are probably send out in waves anyway.
Either way, I think people are still going to be using bare .hex files for a while. It’s going to take better tooling (think GUI .arduboy file generators) and gradual adoption for .arduboys to actually become a thing that people do regularly.
I don’t mind if it has to be required.
Like I say it could be the generator’s job to spit it out.
Though I would still like to know why it’s so important.
Most of it, if not all of it, would have been optional, just as it was in the original.
(In the original only the schemaVersion, title, author, version, and binaries were mandatory.)
The point would have been that it would be there for loaders and generators to do what they want with. They’d be free to display it or ignore it.
Honestly, I think even with better tooling it’s going to be a bit of an uphill struggle to get more people using the format anyway.
I think some of the more consistent contributors will adopt it, but unless there’s a really good, easy-to-use GUI form the drive-by contributors are probably going to stick to .hex files.
It’s hard to beat “press ‘Export compiled binary’; drag-and-drop file onto forum” for user friendliness. (That’s one of the reasons I was thinking about generators having some presaved settings if those settings are used frequently.)
(Come to think of it, it’s probably going to be a while before .arduboy files will trigger an in-forum emulator window.)
I somewhat think that it would make more sense for contributors to just be an array:
"contributors" :
[
{
"name": "Filmote",
"contributions": [ "Code" ],
"urls":
[
"https://github.com/filmote",
"https://community.arduboy.com/u/filmote"
]
},
{
"name": "Vampirics",
"contributions": [ "Art" ],
"urls":
[
"https://github.com/vampirics",
"https://community.arduboy.com/u/vampirics" }
]
}
]
But I don’t strongly care. If there’s a reason for using the user names as field names then fair enough.
That aside I don’t really mind if the other stuff is dropped. It was just an idea.
Nein, mein Deutch ist nicht so gut. (も、日本語が上手じゃない。)
I’m suddenly reminded that the ‘loderunner scenario’ is something that might need to be accounted for. Historically there have been a few games that essentially had the same game split up into multiple ‘chapters’. Granted that should be less common now the FX exists, but there may be some cases where someone wants to do the same thing so people with non-FX units can experience the same game through ‘chapters’.
It’s probably not a big thing, but it may need some consideration at some point.
This is a good point. Though usually (from experience) code like this tends to last forever, which means we would have to agree on supporting both formats basically indefinitely, and all tools would be required to do so.
Edit: Another issue is that it complicates parsing a lot. What if both are supplied, flashdata and flashData? Just more cases to handle, and again in all programs. It is idealistic to assume code like this will “go away”, it almost never does, it instead becomes the standard
Literally working on that right now (and it already exists in the last few releases), that’s part of the reason for this discussion. I sometimes feel like very few people have actually looked at the screenshots or opened the program… or read the description
One reason is that I’m using it as part of the key to identify a game (along with the title). If that field is computed, it has a higher likelihood of changing, and worse, changing without the user knowing, since it’s a computed field and not in the data. This is bad if it’s used as a key. Generally speaking, the author should almost never change, as such it’s a good candidate for a key and a bad candidate for a computed field.
While optional fields are good, like actually good, and I am usually for them, in this case I’m not, simply because of the nature of the data. If we were talking like… the binaries, I’d be all for as many optional fields as you want, especially if they help describe the data. But when we get into personal descriptors, the list is basically infinite, and supporting what the community wants generally ends up with a massive amalgamation of fields you now have to support, and sometimes it can get “political” (hence why I included pronouns; some people think that’s stupid and are loud about it. others think it is required and are also loud about it). While you don’t HAVE to support optional fields like that, there’s pressure to do so, especially because again the nature of the fields (personalized data). Hence why I’d rather just go as minimal as possible and not do any of that.
I like using the names as object keys because it allows a shorthand that I figure people will use:
"contributors" : {
"Filmote" : {},
"Vampirics" : {}
}
I think that should be accepted, as at least they are listed.
Dang I should’ve picked a less known language, I personally don’t speak other languages (wish I did).
For proper testing:
Nope they all use an atmega32u4 and use the Arduino VID and PIDs
A utility that allowes you to examine or manipulate FX data? like the Fx activator is already one (although only an FX version excists atm)
Maybe time for a proper .Arduboy format thread ? so info like that can be easily found?
It’s only used on the modchips that turn a regular Arduboy into an Arduboy FX
Nobody but me. The Arduboy FX activator does something with it. It gives the ModChip owners to cool option to boot always into the menu or boot the last flashed game automatically and change this choise whenever they like. Also if there is an updated bootloader they can easily update it and in case the bootloader gets corrupted for some reason they can restore the bootloader by a long press of the reset button.
dubbed the Arduboy FX devKit. These use D2 for flash chip select. The original design (was changed D1 for easier access for the modchips)
When theres a username field there must be a password field too 
the commandline uploader.py does, the gui version does not.
Thank you! Hm… OK so what I’m seeing is “there’s no reliable way to determine it”, since anyone could have a different bootloader (I get that it’s unlikely).
So one last thing @Mr.Blinky, you mentioned FX::enable() compiled to a single bit set/clear instruction. Is this instruction different between Arduboy FX and Arduboy Mini? If so, is there someplace I could find the exact bytes these compile to? If not, is there some other FX instruction which is different between Arduboy FX and Arduboy Mini, and what are the byte representations (or where can I find it)?
Brilliant! This probably covers all cases except for maybe “alternate wiring” pro micros.
I think it’s really unlikely. I’ve never heard of a homemade Arduboy not using either Caterina or Cathy3k. I think the process will be:
Ah I’m a bit confused, I’m sorry. How does the JEDEC ID distinguish between Caterina and Cathy3k? I’ve only parsed a little of the data out of it (the bits used in the scripts I took from Mr.Blinky) so IDK what else is in there.
As for other tests, is the bootloader slightly different on an Arduboy Mini? I don’t see any indication on that page as such, or is this just for determining custom hardware?
There are two versions of the bootloader one that boots into the menu and the other to boots with the last flashed game (power up + down for menu) other then that you have:
*classic catarina bootloader (different VID PIDs for leonardo/micro)
*sparkfun (used different VID PIDS)
These all use a hardware version of 1.0(without the dot)
If you just want to know about the flash chip select being used, you could just scan for the FX:enable and FX:disable (cbi / sbi) instructions (need to look those up but too tired atm)
But you asked for the display too so I gave the reliable version 