Hey, first post here.
Got my Arduboy a week ago. Can’t believe I didn’t know it existed before. Well, now I do and very happy about that. Thank you for this tiny amazing thing.
Anyway, been writing a script for branching dialog-based game and started to work on a sprite, sprite animation, world map (scenes) editor. It’s rough on the edges at the moment but works including export to arduboy.
It is based on Firebase stack which is free to use to until app hits quite a decent load.
I was wondering if anyone is interested in using this tool and if so, do you prefer to deploy it to your own Firebase project (linked to your Google account, easy to do) or have it available online? I’m ok to open my deployment instance for anyone to use but that might limit any needed customisations for the editor on per-game basis, not sure, not much experience with game development
Near future plans are to add custom fields to all of the objects, add collision/whatnot layers, code cleanup, most likely improve export example with way more compact progmem arrays and cetainly branching dialog editor.
I have never heard of anybody on this site using Firebase … I have had a look at the exported code in your repo but are not sure what you do there to push it to ‘native’ Arduboy code. Is there a compiler that does that?
I am interested to see how well the code compiles down as one of the biggest challenges in writing for the Arduboy is to get everything to fit in the limited memory you have available. If this process is smart you will have no problems - if it creates ‘lazy’ or ‘verbose’ code, you may run out of memory very quickly.
Come to think of it I don’t even know if I still have one.
If I do it hasn’t been touched in eons.
(I tend to avoid Google whenever I can.)
This is very true.
One thing that’s already ringing alarm bells for me is snprintf.
Anything using printf-style formatting is highly likely to be slower than necessary and will probably use more memory than necessary too.
There’s some other stuff that looks concerning,
but I’m having a hard time imagining what a generated example will look like from the code templates.
Otherwise good to see proper C++-style casting, but could use more constexpr.
check out existing dev scene and existing demo exporter in /player
1: URL for config.js is then https://europe-west2-the-tiny.cloudfunctions.net/export-world
2: Existing exporter and dev scene:
# install node.js from https://nodejs.org (LTS version is fine)
npm run fetch # this will fetch and cache json
npm run gen # this will build c++ from cached json
Exported C++ is located at player/src/generated and generation templates are at player/tiny/generate
Overall, this is my first C++ code I ever written, so you can think of player exporter templates as just a bad example of code generation, but it should give some ideas how to generate proper code from json.
Hehee, oh yes snprintf – was just way too lazy to render “Scene #1” properly.
Oh, thanks! I assumed arduboy.print doesn’t move cursor and didn’t want to mess with xy measurements, found that stdio is available with vsnprintf, so wrapped that instead.
Well, I’ve written quite a lot of plain C along with Objective-C. That placement new was something I was looking for to instantiate scenes over each other in the same memory block. Was figuring out if I go with plain c or try to pick up some basics of C++. Did some reading, absolutely but will see how far I’ll get without help from you guys
For future reference: Print::print calls Print::write (which is virtual),
and the documentation for Arduboy2::write states:
The single ASCII character specified is written to the the screen buffer at the current text cursor. The text cursor is then moved to the next character position in the screen buffer. This new cursor position will depend on the current text size and possibly the current wrap mode.
(And of course you could have just asked. :P)
Honestly the entire printf family is somewhat bulky and slow,
mainly because it was designed in an era where people were still figuring out how to build good APIs and good compilers.
It’s not usually much of a problem on desktop,
but on an embedded system it’s a big concern.
The main reason is because it has to parse the format string,
and because of that it has to include all of the possible format string decoding logic,
even if you only ever use one type in the variadic arguments,
purely because it can’t actually know what the string says until runtime, even for a string literal.
Technically the compiler could examine the string and eliminate some branches,
but I doubt many compilers bother with that,
the most they usually stretch to is analysing the format string and making sure the argument types match.
That makes more sense.
I’m not entirely sure it’s needed, but like I say it’s hard to tell what the template code will eventually translate to.
Are there any pre-translated examples available?
(By the way, a global placement new should be available as part of the Arduino library in the next release.)
Aside from which a C-like API might reduce your potential number of adopters.
Most Arduboy programmers are used to doing arduboy.someFunction,
so they’d probably expect to be able to do tiny.someFunction or something similar.
(Ironically Tiny::someFunction() might hit a bit of resistance despite it being a somewhat better option for stateless functions. Some people have an irrational fear of the scope resolution operator. Mainly because a lot of Arduino code neglects to use namespaces.)
My overall idea is to have this tool (webapp and export helpers) available and people generate own C++ from json editor spits out. At least for now as I’m quite far from knowing C++ enough to build something everyone could use. For exampe, I’ll most likely just create a scene layer for collision detection, have that layer’s identifier collisions or something and export that particular layer differently comparing to grid layers with sprites.
Any pointers would be really appreciated
I’m far from happy with my current export approach, especially sizeof summing of total bytes needed to store largest scene in memory.
Looked into compiling with avr-gcc instead of avr-g++, turns out PlatformIO does that for .c files. But again, I’ll better learn C++ instead of extern “C” wrapping all the arduino, arduboy API I’ll be using.
Then of course there’s no need for layer1Node2Size because you can just use sizeof directly.
You can then either copy the whole object using memcpy_P,
or copy individual parts by doing e.g. pgm_read_byte(&layer1Node2.x).
That would at least partially eliminate the need for the placement new.
Come to think of it, at the moment I don’t think your code is actually properly reading from progmem.
I.e. your placement news might not actually be doing what they’re supposed to be doing.
There might be some other ways to reduce the need for directly manipulating uint8_t *s,
but I’d need more time to pick through the code.
For another thing I think maybe you should replace all the if(index == 0), if(index == 1) with a switch and use each case to delegate to another function.
If a function is only used once the compiler is likely to inline it,
and I think the compiler is more likely to create a jump table (or perform some similar optimisation) if you use a switch.
It might still be able to do it from the ifs, but I think it would have more work to do so it might give up before it gets to that point.
Aside from being a huge amount of effort , doing that might actually consume more progmem than necessary depending on how good the compiler is at optimising.
That part is handled by ejs templates which should almost certainly be per-game asset (in game repo, separate from Tiny) but absolutely. most likely I’ll just switch to camelcase.
Thank you for structs in PROGMEM with constexpr! Haven’t seen constexpr used like this.
Regarding placement new and PROGMEM, I still believe I do both of those things properly (ignoring the mess in the code). Let me try to illustrate what I’m exactly trying to accomplish. Without tiny, without code generation and other stuff which at this point is not that relevant:
“Definitions” for nodes, layers and scenes. That would also include sprites, loops, most likely other data structures which reside in PROGMEM. Now, thanks to you, those are structs.
Some kind of way to calculate total size of bytes needed for largest scene (I mean for class instances, not definition structs) in the game. In that test – sum of basic sizeofs (with missing sizeof for Layer * number of layers)
Classes – Scene, Layer, Node (omitting SpriteNode: public Node for now). Those are instantiated using placement new in a single byte blob which is shared between Scenes as the game has only one active scene at the time. When scene are switched, old one is just left behind and over that a new one is created. Instances takes care of object graph relationships – like scene has 0…n layers and so on, also instances do load data from definitions using pgm_read_* when needed. definitions are private to each instance.
Instantiate function which is supposed to be also generated which builds up instances and provides definitions to instances.
So far so good. This (actual generated code for around 30 nodes) works just fine, even switches between two scenes without noticable delay (I was worried that instantiate would take more time) but all that sizeof madness and ptr += sizeof(WhateverWasJustInstantiated) is not something I would like to keep there.
My aim at the moment is to keep definitions (PROGMEM) and class instances as two separate things. classes get pointers to definitions, but public api from this part are classes only.
Started to look into template magic,… well that will take some time.
And as always, @Pharap thank you for your tips. Also will move away from ifs.
It probably doesn’t really matter if they’re const or constexpr in most cases,
but I tend to prefer constexpr when I want to indicate that something is supposed to be truly ‘immutable’ rather than just ‘read only’.
There are a few cases where having constexpr instead of const actually matters because it allows you to move some of the processing to compile time.
Unfortunately in C++11 there are limitations to what you can do at compile time.
They relaxed the rules in C++14 so it becomes even more useful from C++14 onwards.
(The Arduino IDE is set to compile everything as C++11.)
Hrm, I may have misread the code.
I thought you were trying to use placement new to copy data from progmem at one point,
but it might be that you were copying a pointer to progmem.
Just so you know: technically being able to do .width = 128, .height = 64 in the struct initialiser is non-standard behaviour.
It will be standard in C++20 (see aggregate initialisation (3) and (4)), but currently it’s non-standard.
On the one hand I think something like this will work,
but on the other hand I can’t shake the feeling that there’s possibly an easier way.
If I’m reading this right then after instatiate, storage will look like this:
So the pgm_read_ macros will be hidden behind member functions on Scene, Layer and Node?
Simply copying data around is surprisingly cheap.
AVR chips are pretty fast despite being 8-bit.
At the moment the only solution I can think of off the top of my head (and normally I probably wouldn’t consider this, but since you’re auto-generating this data anyway) is to create a struct that precisely mimics the intended data structure.
I didn’t know that, been using it for years, I believe even some apple docs where using this for CGRect and CGFrame struct merging, calculations. Good to know.
Was looking into turning init & sizeof plus for storage area into some kind of builder, seems to be possible using templates but that’s only init part, oh well.
Yes, I think so. Basically whatever I need in storage, one class instance after another. It’s not important how those instances are ordered in storage because stage will have array of layer pointers, each layer will have array of node pointers (omitted from test.h).
My intention is to:
instantiate active scene
run scene->update() which invokes update for each layer, which invokes update for each node
run `scene->render() which does the same
Absolutely, that’s really not that important at the moment. Those definitions and wrapper classes can be anything. Sprites, Scene entities, Keyframes whatnot.
Tiny editor has scenes and each scene can have 0…n layers which can have 0…n nodes. Layers are basically a way to render parallax for example (or define collisions/triggers/characters/… but in that case those won’t be generated as visual layers in C++ code)
Thank you for a nice example on how to use variadic templates! Seems like union is the simplest way to go for now.
Minesweeper! That’s a one clean codebase. Will dive in!
Yes. I’m going to consider those as an internal detail for Scene classes.
This is something I was also considering but the thing is, Layer, Scene, Node, all those classes can be subclasses (say RainLayer: public Layer with some custom logic on how to render nodes (most likely sprites) for that layer) so it won’t make it much easier.
I believe so. For example node which renders sprite animation will keep track of current frame index and so on.
Oh, that’s so wicked! I’ll check it out. Already learned something new – static_assert.
That might not be that surprising if you’re used to C rather than C++.
It is actually standard C, just not standard C++. (I have no clue about objective C, I’ve never used it.)
It didn’t end up in C++ originally because it was a C99 addition,
and by the time C++11 came around there were more important things on the agenda.
(Also this is what I mean about surprises - C and C++ have more differences than most people realise.)
Hrm… It should work.
At the very least it would encapsulate ptr,
and should hide some of the less pleasant details.
Ok, that makes sense.
Presumably if there were only one layer the system would ideally optimise so that the scene holds the nodes directly instead?
It’s certainly a good example of how to extract individual parts from a variadic list of values,
but really it only really scratches the surface of what’s possible.
A bit of rambling about templates...
Pesonally I find that the best way to figure out what’s possible is actually to learn a functional programming language (e.g. Haskell) that operates on ‘cons lists’ as a core data structure and has pattern matching.
(Unfortunately not everyone has the time or the spare sanity to learn a functional language. :P)
Variadic templates can be thought of as being like cons lists and template specialisation can be thought of as a form of pattern matching, and from there you can build a lot of complex compile-time magic.
constexpr functions will eventually take over from a lot of that,
but not quite everything,
and when you don’t have access to C++14 onwards then variadic templates remain your best bet when constexpr functions won’t cut it.
It makes a nice change to talk about this stuff.
Most people hear ‘templates’ and run a mile. :P
I’ll warn you that the templated GameStateMachine was a bit of an experiment.
I was trying to find a good way to prevent the states from being dependent on each other as well as producing something reusable.
I’m still not completely sure whether I succeeded or not, but it seemed to work at least.
Also virtual functions seem to consume quite a bit of progmem, so think twice before using them.
(Presumably the progmem use is mainly due to the vtable, but it seems to be larger than what the vtable alone should consume, so there might be more to it. I never got round to investigating properly.)
At any rate, hearing even one person call my code ‘clean’ justifies the effort I put in to try to keep it clean.
(The actual gameplay state could be a bit cleaner, but I’ve never got round to going back to fix it, mainly because I’ve always thought it would only be for my own benefit.)
That’s also probably going to make calculating the size of storage a bit more difficult, because you need to know what the largest subclass will be, which probably means yet more unions.
Ok, in that case they can only be partially stored in progmem.
Just to warn you, it seems I decided to declare each struct on a single line that day for some reason.
(I’ll make an issue to remind me to fix that eventually.)
Speaking of new things, make sure to put enum classes high on your ‘to learn’ list.
Type safety is a wonderful thing.
(I probably don’t need to tell you this, but the all-caps enumerators used in the article are a bad idea - stick to pascal case like you currently are.)
Yeah, I’ll play around with that after I’ll figure out how exactly I want to add dialog editing to the editor and then what to do with them in the game.
Yes, I suppose so. I added support for custom properties in editor for scene, layer, node which can be used to set custom className for each of them in code generation stage. I suppose Scene and Layer base clases could be just Container then it doesn’t matter if node parent is scene or layer and so on.
I’ll start to work on actual game instead of just this player app which only renders scenes to figure out those details.
Watched few CppCon sessions about “modern” C++ and template metaprogramming. And I thought XSLT is madness. Pattern matching with SFINAE will take some time for me
But good, now I have some ideas what can be done while not exactly how.
That’s a pitty. I was going for virtual update and draw for everything.
Will I be ok with an array of function pointers in layer to call child nodes draw?
Absolutely, thank you for your effort. It helps quite a lot.
In the end I went with summing all sizeofs for each scene. And then using union to find out the largest scene. Not that bad.
Absolutely! For example editor has optional identifier properties for scenes, layers, nodes. I’ll look into turning those into enums so that I can lookup some particular say node using that.
Now I have simplified, cleaned up export, added custom props, my next step seems to be add support for custom class names in export and try to build an actual 1st game scene in separate repo.
Sorry it’s taken me so long to reply to this,
I had it open in a tab to remind myself to reply, then I lost track of the tab.
I don’t actually watch that many videos, I tend to prefer reading articles,
but one C++ talk I do particularly like is Bjarne Stroustrup’s Going Native 2012 Keynote.
‘Pattern matching’ and SFINAE are separate concepts (though SFINAE relies on ‘pattern matching’ to work).
(Technically ‘pattern matching’ is actually called ‘template specialisation’, but the way it works can be likened to ‘pattern matching’ from functional programming languages and I have seen a handful of people call it ‘pattern matching’ informally.)
I did attempt to write an explanation of what SFINAE is a while back,
but it’s only really an introduction with a few examples of some of the more advanced stuff,
it still needs a few more advanced examples and some better explanations of the more advanced techniques.
You may or may not find it useful for understanding SFINAE:
It’s somewhat harder without access to std::enable_if or std::void_t,
but those are easy enough to recreate.
If you have anything specific in mind I can probably help get it working.
Without meaning to sound arrogant, there probably aren’t that many people here who understand template metaprogramming because it’s quite specialist knowledge.
I’m not sure how expensive an array of function pointers would be since it’s not something I use often.
I would assume it would be at least slightly cheaper than using virtual functions depending on how it’s implemented,
though there might be some cheaper (albeit not quite as nice) alternatives that might be worth considering.
E.g. switch-ing on an enumeration (enum class).
Hey! Really not a problem I appreciate any input whenever you have some time to spare.
Been busy myself with consulting work lately and did some reorganization of editor (updated screenshot on the top). Now sprite editing and scenes are in one screen. Makes it way easier to preview sprite frames and loops in actual scenes while drawing. And needed this to also add dialog editing there, which I haven’t yet figured out, not sure how exactly I’m going to implement that. Started to write script but nothing is ready for testing yet. The perks of new medium. I’ve never written anything for games.
Thank you! It is really useful. If you’re going to extend this article, a section about using might be good addition. Had no idea using can be, well, used in this context – as a type alias member or however this is called.
Thank you! I will certainly ping you with questions about templates. Soon enough I’m going to try to build overall architecture for this game I’m slowly been working on.
May I ask, how you got involved with this voodoo priest wizardry of C++?
Also those switch statements most likely can be generated. For time being then I’ll just keep virtual and will refactor later on.
I’m hesitent to include it because I think explaining language features/syntax would be a slippery slope.
If I explain using do I then also have to explain typename and const and & (references)?
Do I have to explain member initialiser lists too?
I’d rather say “explaining language syntax is outside the scope of the article”. For now at least.
Besides which, I don’t think it’s too hard to find an explanation for using by doing an internet search,
particularly because it’s a keyword.
The committee decided that C’s typedef is a bit of a stumbling block for people learning the language because the way it defines a type alias is backwards compared to how most other things in the language are defined (i.e. typedef expression alias;) so they introduced a new use for using that mimics variable definition (i.e. using alias = expression;).
One case where using behaves differently to typedef is with function pointers.
When you use typedef to alias a function pointer you have to put the alias in the middle of the expression.
E.g. typedef void (*Func)();
But with using, the alias remains on the left hand side, so you end up wit using Func = void (*)();, which personally I think is much more sensible.
And of course using supports ‘template type aliases’ which typedef doesn’t.
Personally speaking I never use typedef any more because using type aliases make much more sense and feel more natural.
Honestly I’m not entirely sure.
I think I mainly started learning about them because it just seemed like a natural progression after understanding the more basic/more common uses of templates.
I learned C# before C++, during which I learnt about C#'s generics,
so I started learning about C++'s templates because I saw them as the C++ equivalent.
(Technically templates sort of are the equivalent - it’s the same concept, but the implementation is drastically different.)
At first I didn’t realise quite how different they are and how much more they are capable of,
but I gradually learnt about the differences between the two systems.
So I think probably I started looking into template metaprogramming purely because I was exploring just how different the two systems were.
Either that or I’d heard the rumours about the crazy things people could do with template metaprogramming, so I decided to learn how to do it simply to find out just how crazy it was.
In fairness this wouldn’t really be much of a concern in desktop C++ programming where RAM is plenty and virtual calls are relatively cheap.
But on an embedded system sometimes you have to sacrifice nice language features to fit everything in ROM.
By the way, when people criticise C++ for doing ‘too much behind their back’,
8 times out of 10* virtual and the vtables it generates is what they’re talking about.
That kind of complaint only makes sense in the context of C vs C++ though,
* The other 2 times are templates, but that’s a case of ‘your mileage may vary’.
Some compilers are better at inlining templates and preventing unnecessary code generation than others,
and most modern compilers are a lot better at it than the compilers of 10-20 years ago.
There’s a lot of arguments and advice like that based on experience with outdated technology.
For example, the reason for(;;) became a thing in the first place is because there was a certain C compiler in the early days of C that wasn’t smart enough to optimise while(true) into a proper infinite loop, so for(;;) would generate a few less CPU instructions.
Thankfully modern compilers aren’t that dumb.
Oh yes, absolutely.
Tried again to build simple prototype in plain c (avr-gcc and extern "C" for the bridge). While it works, it’s not worth it purely because C++ makes it way easier to put everything together. Including simple things like function overloading ThingiePrint() vs ThingiePrintString(const char *); ThingiePrintUInt8(const uint...
Btw, randomly stumbled across alternative to the switch-case using labels which is even faster:
That’s also one of the reasons I decided to learn Haskell (because it looked crazy).
In both cases they had a big impact on my programming outlook and showed me a new way of looking at things, so learning crazy things seems to be worthwhile. :P
That’s more or less the conclusion I would have expected.
Arduino might have started out in C, but the library is deeply reliant on C++ now,
so bridging it to C is bound to be difficult.
(Though it’s a shame they aren’t using more C++-style practices within their code (e.g. replacing macros with template functions), the library could really benefit from it.)
Yeah, that’s one thing that’s always bothered me about C.
Particularly because every other C-like language I’ve ever used has it.
(Lua and Haskell don’t, but they both have good reasons for not having it.)
I do somewhat understand why they left it out - it makes linking external pre-compiled libraries harder because of name mangling,
but at the same time function overloading is such a useful and convinient tool that it’s worth the price I think.
Generally speaking I don’t condone treating C++ like ‘C with a few extras’ because there are different mindsets behind what constitutes ‘good code’ in each language,
but I do think that someone using C++ in a C-ish way is probably still better off than someone using pure C because even some of the ‘little’ extras like function overloading and type safety (e.g. void * not being implicitly convertible to any pointer type as it is in C) can make a big difference to usability and code quality.
I had to check to see whether this is even legal.
Apparently it’s a compiler extension specific to GCC,
including the compiler-specific && unary operator.
This reminds me of elm (elm-lang.org) which is functional programming language built using Haskell
I’m working on Tiny and getting ready to work on a game for Arduboy because it looks crazy. And very interesting for me.
Never built a game, already had 2 iterations for editor, looks like I’m going to rebuild it one more time – structure is not right. Need to make database structure more abstract – just a node tree instead of having sprites and scenes with nested layers, nodes. Looked into adding dialogs, triggers and two more separate collections models doesn’t make any sense.
Hehee, yeah, most likely I’ll be in this category for a long time
Sorry, forgot to add most important part – this makes any performance difference if code jumps from one label to another directly. Say label1 code finishes and does goto* labels[another]. switch-case goes back to jump table, this goto thingie can just jump directly. According to the post I still unfortunately can’t find, haven’t tried to investigate.
Oh yes, I’m staying away from int.
Thanks! Had no idea there is least8, fast8 variants. Will give it a read.