Arduino Compilation Problem? Randomly Stopped Working? 😫

crait · September 28, 2023, 2:46pm

Whenever I try to compile via CLI, I now get this error.

In file included from C:\Users\Admin\AppData\Local\Arduino15\packages\arduino\hardware\avr\1.8.6\cores\arduino\USBAPI.h:44:0,
                 from C:\Users\Admin\AppData\Local\Arduino15\packages\arduino\hardware\avr\1.8.6\cores\arduino\USBCore.cpp:20:
C:\Users\Admin\AppData\Local\Arduino15\packages\arduino\hardware\avr\1.8.6\cores\arduino\USBCore.cpp:75:29: error: 'USB_VID' was not declared in this scope
  D_DEVICE(0xEF,0x02,0x01,64,USB_VID,USB_PID,0x100,IMANUFACTURER,IPRODUCT,ISERIAL,1);
                             ^
C:\Users\Admin\AppData\Local\Arduino15\packages\arduino\hardware\avr\1.8.6\cores\arduino\USBCore.h:270:61: note: in definition of macro 'D_DEVICE'
  { 18, 1, USB_VERSION, _class,_subClass,_proto,_packetSize0,_vid,_pid,_version,_im,_ip,_is,_configs }
                                                             ^~~~
C:\Users\Admin\AppData\Local\Arduino15\packages\arduino\hardware\avr\1.8.6\cores\arduino\USBCore.cpp:75:29: note: suggested alternative: 'USB_Recv'
  D_DEVICE(0xEF,0x02,0x01,64,USB_VID,USB_PID,0x100,IMANUFACTURER,IPRODUCT,ISERIAL,1);
                             ^
C:\Users\Admin\AppData\Local\Arduino15\packages\arduino\hardware\avr\1.8.6\cores\arduino\USBCore.h:270:61: note: in definition of macro 'D_DEVICE'
  { 18, 1, USB_VERSION, _class,_subClass,_proto,_packetSize0,_vid,_pid,_version,_im,_ip,_is,_configs }
                                                             ^~~~
C:\Users\Admin\AppData\Local\Arduino15\packages\arduino\hardware\avr\1.8.6\cores\arduino\USBCore.cpp:75:37: error: 'USB_PID' was not declared in this scope
  D_DEVICE(0xEF,0x02,0x01,64,USB_VID,USB_PID,0x100,IMANUFACTURER,IPRODUCT,ISERIAL,1);
                                     ^
C:\Users\Admin\AppData\Local\Arduino15\packages\arduino\hardware\avr\1.8.6\cores\arduino\USBCore.h:270:66: note: in definition of macro 'D_DEVICE'
  { 18, 1, USB_VERSION, _class,_subClass,_proto,_packetSize0,_vid,_pid,_version,_im,_ip,_is,_configs }
                                                                  ^~~~
C:\Users\Admin\AppData\Local\Arduino15\packages\arduino\hardware\avr\1.8.6\cores\arduino\USBCore.cpp:75:37: note: suggested alternative: 'USB_Recv'
  D_DEVICE(0xEF,0x02,0x01,64,USB_VID,USB_PID,0x100,IMANUFACTURER,IPRODUCT,ISERIAL,1);
                                     ^
C:\Users\Admin\AppData\Local\Arduino15\packages\arduino\hardware\avr\1.8.6\cores\arduino\USBCore.h:270:66: note: in definition of macro 'D_DEVICE'
  { 18, 1, USB_VERSION, _class,_subClass,_proto,_packetSize0,_vid,_pid,_version,_im,_ip,_is,_configs }
                                                                  ^~~~

Error during build: exit status 1

I haven’t compiled this project in a few weeks, so now I am at a loss of words for what might be the cause…

When I try to compile, I am using this:

arduino-cli compile -b arduboy:avr:arduboy --output-dir "../Compiled/Arduboy/" --build-property="build.extra_flags=-DSYSTEM_ARDUBOY"

This is super worrisome… I can’t remember updating any boards, the IDE, etc. So, I tried on my other computer, and now I have the same issue. I even reverted my branch to one I know was compiling, correctly, and still get the error. I also tried re-installing the Arduino IDE, Arduino CLI, different Arduboy board versions from the Library Manager in the Arduino IDE… Anyone have any ideas for this?

Compiling with the actual Arduino IDE seems to have a completely different problem that I am trying to solve, now, because I think it cannot find the other files that are supposed to be included in the project, like the Arduino CLI seemingly can. So, I’m kind of stuck.

Freezingsnail · September 28, 2023, 3:16pm

what is your env like?

❯ arduino-cli core list
ID                   Installed Latest Name                    
arduboy:avr          1.1.0     1.1.0  Arduboy                 
arduboy-homemade:avr 1.3.9     1.3.9  Arduboy homemade package
arduino:avr          1.8.6     1.8.6  Arduino AVR Boards

I’m using this rn on a debian WSL install

arduino-cli compile --fqbn "arduboy-homemade:avr:arduboy" --optimize-for-debug  --output-dir .

Never seen that --build-property flag before
I’ve never used the CLI from windows

I think it cannot find the other files that are supposed to be included in the project

everything needs to be in root or /src iirc
and it depends on the version too
its super clunk
it used to be anything for a short period but then it got changed to just src.
I forget why exactly

crait · September 28, 2023, 3:29pm

Okay, here’s my output for the environment info:

ID          Installed Latest Name
arduboy:avr 1.1.0     1.1.0  Arduboy
arduino:avr 1.8.6     1.8.6  Arduino AVR Boards

I use it to define a preprocessor macro, SYSTEM_ARDUBOY, in order to do a few things in my code. I eventually want a few other systems defined so I can cross-compile my game for other hardware, like with SYSTEM_PC or SYSTEM_NX. I’ve never had an issue with it, but removing it and using #define SYSTEM_ARDUBOY in main.cpp has caused a FLOOD of other errors when I try compiling. Seriously, like thousands of errors. Seems like basic stuff, like PROGMEM and uint8 are not even defined anywhere.

Just for kicks, I used…

typedef unsigned char uint8_t;
typedef signed char int8_t;

… and the errors still were popping up. I think I need to like, define that in every single file or something and I think that’s a waste of time. Ain’t nobody got time for that. I’m wondering why something so inherit to the Arduino library is not being found at all.

Freezingsnail · September 28, 2023, 3:39pm

#include <stdint.h> should handle the uint8_t issue
do you have #include <avr/pgmspace.h> ?
I’ve had issues with the linker not being able to find the <> libs before

Is there a repo I can clone?

Let me look at my git commit history, I was passing a flag for debug but that might of been when I was using cmake.

Yea I never was passing that debug flag to the CLI, that’s something I’ve been meaning to do actually so I don’t need to delete all these debug macros thrown about

crait · September 28, 2023, 3:44pm

I’ve never had to include stdint.h or avr/pgmspace.h, though. I jump back to previous branches from like 2 months ago when it was compiling and now it’s not. In fact, I’m including the Arduboy2 library, which includes Arduino, which includes pgmspace.

I’m sorry, but it’s private, for now.

I’ve had sooo many issues with Visual Studio Code doing this, but never the actual linker!!

Freezingsnail · September 28, 2023, 3:56pm

Yea I gave up on getting coc-clangd to find them lol, switched back to code
I think I’ve always used the avr import, its in my repos from years ago.

maybe try reinstalling the arduboy2 lib through the cli if you didn’t. IIRC the cli and the ide don’t use the same dir for the libs / share installed libs.

arduino-cli config init --additional-urls https://arduboy.github.io/board-support/package_arduboy_index.json
arduino-cli core update-index
arduino-cli lib install Arduboy2

what’s in the directories category under arduino-cli config dump ?
(also why is this dump and not list, the cli is not consistent at all lol)

Maybe try wsl

I think that’s all I have without being able to play around with it myself, unfortunately any compile issue I’ve had with arduino has been a pita since its not like normal gcc and c files and the arduino forum hasn’t been helpful either

crait · September 28, 2023, 4:36pm

Thank you so much for helping me! It is really appreciated!!

Okay, I tried this and the output was Already installed Arduboy2@6.0.0

Lmao. Here’s what it outputs:

oard_manager:
  additional_urls:
  - https://arduboy.github.io/board-support/package_arduboy_index.json
build_cache:
  compilations_before_purge: 10
  ttl: 720h0m0s
daemon:
  port: "50051"
directories:
  data: C:\Users\Admin\AppData\Local\Arduino15
  downloads: C:\Users\Admin\AppData\Local\Arduino15\staging
  user: C:\Users\Admin\Documents\Arduino
library:
  enable_unsafe_install: false
logging:
  file: ""
  format: text
  level: info
metrics:
  addr: :9090
  enabled: true
output:
  no_color: false
sketch:
  always_export_binaries: false
updater:
  enable_notification: true

I think I can get you access to the repo, if you really wanna help dive into the issue. Just shoot me a DM. Lmao

acedent · September 28, 2023, 5:43pm

@crait - sorry for your pain!
Summoning @Pharap who seems very knowledgable on these inner workings…

Pharap · September 29, 2023, 1:09am

I’ve been summoned! So now I’d better deliver…

As long as all your .h and .cpp files are in the src folder, it ought to be able to find them.

I couldn’t really diagnose that without at least an error message and ideally a proper investigation.

If/when I get a moment I’ll grab a copy of the repo and see if I can work out anything that might be causing the issue. I don’t have Arduino CLI installed, but getting it compiling on my setup might shed some light on the situation.

I’ve done some research.

Essentally what happens when compiling via the Arduino IDE is that the values of USB_VID and USB_PID are detected from the settings provided in boards.txt and fed to the compiler with the -D flag. (Along with various other macro definitions such as -DARDUBOY_10, -DARDUINO_ARCH_AVR.)

I presume Arduino CLI is supposed to be able to do the same, but I’m not certain as I’ve never used it.

You might potentially be able to solve the issue manually by adding -DUSB_VID=0x2341 and -DUSB_PID=0x8036 to your command line.

However, that’s more of a bodge than a proper solution, and it might upset things for people who want to compile the game to run on a homemade Arduboy, so I’d very much advise against using that as a long-term solution.

The ideal solution would be to find out why Arduino CLI isn’t producing those defines.

One other option might be to try using Mr Blinky’s homemade package. The boards URL for that is https://raw.githubusercontent.com/MrBlinky/Arduboy-homemade-package/master/package_arduboy_homemade_index.json.

You could also check for ARDUBOY_10, which is what the custom Arduboy board is supposed to define. (Mr Blinky’s homemade package also defines that for compatibility.)

For those who don’t have the Arduboy board installed, ARDUINO_ARCH_AVR ought to work too. Or at least it would be enough to detect that you’re not dealing with desktop or whatever NX is supposed to be.

Yes, it would do.

Remember what I told you about translation units?

Every .cpp is compiled in a separate translation unit as a blank slate. Therefore if you only define SYSTEM_ARDUBOY in main.cpp it’s never going to show up in any of the other .cpp files because they don’t see anything defined in main.cpp.

If you’re going down that route then you’d ideally want some kind of header (.h) file that’s included by every relevant source (.cpp) file.

Something like:

// Platform.h
#if defined(ARDUBOY_10) || defined(ARDUINO_ARCH_AVR)
	#if !defined(SYSTEM_ARDUBOY)
		#define SYSTEM_ARDUBOY
	#endif
#elif defined(_WIN32) || defined(_WIN64) || defined(__GNUC__) || defined(__clang__)
	#if !defined(SYSTEM_PC)
		#define SYSTEM_PC
	#endif
#else
	#error "Could not detect target platform"
#endif

(I’ll leave you to work out how to detect whatever NX is.)

And perhaps also have that same header deal with any platform-specific behaviour you might need to deal with.

E.g.

#if defined(SYSTEM_ARDUBOY)
#include <stdint.h>
#include <avr/pgmspace.h>
#include <Arduboy2.h>
#endif

#if defined(SYSTEM_PC)
#define PROGMEM
#include <stdint.h>
#endif

It’s true that Arduboy.h will indirectly include those, though it’s still a good idea not to rely on header files indirectly including other files and to manually include all the headers that you actually need.

Also it’s worth using <stdint.h> and <avr/pgmspace.h> in cases where you need uint8_t and PROGMEM but nothing from <Arduboy2.h>, e.g. in .cpp files that define images, in part because then that file isn’t tied to an unnecessary dependency and in part because having fewer files and symbols to process should make the code compile marginally quicker. (Though on a modern computer you’re probably talking mere milliseconds.)

Don’t even get me started on it…

It caused me quite a headache at the time.

Eventually I solved the issue by just working almost entirely inside the src directory as much as possible and just making the .ino as tiny as possible.

Pretty much all my games have an .ino that consists of just:

#include "Game.h"

// Licence text goes here

Game game;

void setup()
{
	game.setup();
}

void loop()
{
	game.loop();
}

I later found out that you can actually leave the .ino blank and have your setup and loop functions in a .cpp file inside src, so if I ever do get around to doing anything in future I may end up doing that.

Pharap · September 29, 2023, 1:42am

I tried compiling and I got the error:

Game.ino:473:60: error: macro "SPRITE_WORLD" requires 2 arguments, but only 1 given
SPRITE_WORLD((uint8_t)Z(test_world[2 + y * width + x])),

I can kind of guess what’s gone wrong here, but I don’t know enough about your framework to know how to fix it.

After incorporating the ARDUBOY_10 detection that I mentioned earlier into system.h there’s a few more errors such as dbmFlip not being declared.

I get the feeling if these errors were cleared up the code would actually be compiling though. I’m not getting any VID/PID related errors at least, which more or less confirms there’s probably an issue with your CLI setup.

Freezingsnail · September 29, 2023, 1:50am

I compiled it with:

arduino-cli compile -b arduboy-homemade:avr:arduboy --output-dir "../Compiled/Arduboy/" --build-property="build.extra_flags=-DSYSTEM_ARDUBOY"

oh and I had to rename Game to game lol. Major clunk
I didn’t run it for what it’s worth either.
the normal arduboy board wasn’t happy about the FX lib, it’s prob an environmental issue and not code

crait · September 29, 2023, 1:55am

This is because you tried to compile without SYSTEM_ARDUBOY defined because that’ll switch the SPRITE_WORLD() macro to a single parameter.

Pharap · September 29, 2023, 2:29am

You’ll definitely need the #if defined(ARDUBOY_10) || defined(ARDUINO_ARCH_AVR) detection in system.h then, otherwise people using the Arduino IDE won’t be able to compile from source.

I take it that’s Mr Blinky’s homemade package?

If so, that’s interesting that it worked when the other board didn’t.

Were you getting the same vid/pid not defined error when you tried it with the ‘normal’ package, or some other issue?

I forget what exactly but FX::begin is supposed to do something to halt the code if there’s no FX chip connected, so that’s entirely expected.

Freezingsnail · September 29, 2023, 2:34am

No I meant it failed to compile because the lib wasn’t found. This was on osx though, I’ll try wsl tomorrow.

I can’t remember if I ever got the cli to work with the fx lib. Pretty sure I switched to the homemade package to get the cli working.

Pharap · September 29, 2023, 3:06am

It seems the ArduboyFX library doesn’t actually show up in the Arduino IDE library manager.
I’m presuming it isn’t properly registered with the library system for some reason.

I just discovered that the issue with dbmFlip is that it was only added 2 months ago and my copy of the library was 4 months old.

After manually updating the library, the code compiles for me in VSCodium and the ArduinoIDE.

There are some concerning warnings though:

const MenuDefinition * being implicitly converted to MenuDefinition * in a few places
- Which could be fixed by changing show_menu and next_menu to accept const MenuDefinition *s instead of MenuDefinition *s
A possible truncation issue in bundle.h where 0xffffff is too large for an int.
- This could probably be fixed by changing the 0xffffff to 0xFFFFFFul.
There’s a few functions whose optional arguments are being specified in the wrong place. (E.g. draw_bitmap, find_progress_point, draw_bar)
- The optional argument needs to be specified in the .h, not the .cpp because it’s the caller that needs to know what default value to provide, not the function itself. (Again this is to do with how translation units work.)

Most of those would actually be errors if it weren’t for Arduino using the awful -fpermissive flag.

The remaining warnings after that are just about unused variables and parameters, so those likely aren’t anything worth worrying about.

Anyway, the remaining issue of the vid/pid macro definitely lies with either Arduino CLI or the Arduboy package.

If @crait can also compile on the homemade package then that would suggest that it’s the ‘normal’ Arduboy package that’s the problem and that would have to be fixed by the host of that package once we know what the issue is.

(Based on what I saw in the boards.txt on the homemade package I do have a theory as to what the problem might be, but I’d have to talk to Mr Blinky about it since he’s most likely to be able to confirm whether that might be the issue.)

crait · September 29, 2023, 6:33am

Dang, I really appreciate the suggestions and help from everyone. I’m really hoping this helps people who happen to have the issue in the future. What I ended up doing was manually uninstalling Arduino CLI and the IDE, then finding behind any preferences and boards/libraries/etc, then deleting them all, then doing a fresh install. I thought that I had done this, before, but I guess I needed to completely uninstall everything Arduino-related on my PC. I’m not entirely sure what caused it to be fixed, but I was able to successfully compile my program.

I have no way to verify this, but I do believe this was the issue. I’m not sure how they were messed up, but I figured it was better to just reinstall everything than trying to poke around to find the cause of this issue.

filmote · September 29, 2023, 6:38am

Good to see you have heaps of memory left! Both PROGMEM and RAM, you lucky devil.

Dark & Under II
Sketch uses 28746 bytes (96%) of program storage space. Maximum is 29696 bytes.
Global variables use 1970 bytes (76%) of dynamic memory, leaving 590 bytes for local variables. Maximum is 2560 bytes.

crait · September 29, 2023, 6:44am

Actually, I’m about to get even more memory back when I finish putting the menus onto the FX. After that, none of my static data (maps, music, sprites, creature data tables, attack data, strings, etc.) will be on the device, so I can start adding loads of content without having to worry about running out of memory. I am also going to put a very simple scripting engine in, as well, to make the objects in the world scriptable. Those scripts will also be in the FX chip, which will let me work around the problem of not be able to put compiled code onto the FX.

Really looking forward to showing off how I was able to achieve this in the dev log. Byte Monsters - A monster-taming RPG [Dev Log]

filmote · September 29, 2023, 6:45am

Interesting, I have 29K of PROGMEM you only have 28K. Are you using the optimized core / Cathy3K bootloader?

crait · September 29, 2023, 6:48am

No, I don’t think so… Maybe I should switch at some point, but I am using Arduboy2Base. Does that include it?