E-Paper Matrix Game

Imagine a little e paper iot widget in a durable case.

Yeah, you can’t play any fast paced games, but you can play slow ones, but more importantly it’s about sharing information. This can be the perfect little controller for ANY smart system.

Think of it like like a logitech smart remote, but more like a key fob, and its truly universal.

I think this is the answer. It’s a different product, it’s not meant for fast games. Fast games could use OLED…

Could theoretically be two forks? Could the code be written for two libraries, whether you have the OLED version or the EPaper version?

3 Likes

Ping @tonym128

Dude it’s a wicked spotify player isn’t it? Gotta put a nice DAC on there.

Without evidence to the contrary, yes it’s theoretically possible assuming:

  • Writing to an E-Paper screen is just a matter of copying a memory buffer across
  • The memory layout of the frame buffer is similar

If the memory layout is different then you’d have to decide whether to break games that do their own drawing or possibly burn cycles doing a memory reformat on-the-fly.

But E-Paper generally has a slow refresh rate,
so it’s only going to be good for slower games anyway.

Hi,
Thanks so much for the interest. Hopefully you will still be interested after the video :stuck_out_tongue:

Quick note on the E-Paper, I am getting roughly 300ms for the partial screen updates (these lead to some ghosting). For the full screen update, they take a full 3s, but it is crystal clear and doesn’t fade once the power is removed.

I put together a quick video of the library in action with a ESP32 and OLED and ESP32 and EPaper display.

I run the same code base on both versions, just a define at the start brings in the right libraries and setup.

You can check out the code here

Special shout out to @ESPboy, whose fork of a fork of a fork made this all possible :slight_smile:

You can read about the original badge here

Some interesting tidbits.

I was getting 60fps on the original OLED version without any scaling. Once I started scaling the screen my framerate dropped into the low 20’s. I found out after a lot of trial and effort this was the SPI limit even with pushing direct screen writes and some optimisations. I threaded the code and got it back up to the mid 20’s, but most importantly unlocked the game logic rate. I did the same thing for the input and got another mediocre speed boost.

Once I had the Badge up and running, I needed a way to control the EPaper Badge, it has a single button on it, the original idea was to recode the game to play with one button (short and long press), but that was going to be more effort than I wanted, so I grabbed a PS3 controller I had lying around and hooked it up via BT and that’s my input now, worked great with the OLED badge too :stuck_out_tongue:

On the code base, it’s a bit of a hacky hack with a lot of defines to block out code for certain versions. There is a small possibility this would run on an ESP8266 still, the code should all still be there, I tried to have a light touch and block in my code. There’s about 4 distinct versions in there now.

The ESP32 with OLED using TFT_eSPI and Touch Buttons
The ESP32 with OLED using TFT_eSPI and PS3 Gamepad
The ESP32 with EPaper using TFT_eSPI and PS3 Gamepad
The ESP32 with EPaper using TFT_eSPI and One Button (not doing detection other than single click)

I was meant to block off the input threading and display threading with defines as well, but I think if I went back to it, I would probably restart the code and get my input and display seperated properly into their own backing implementations.

At some point I lost the ability to run 60fps on the unscaled version, mostly due to the front and back buffer copying for the threading on the display, I was trying to eek a bit more fps out.

Feel free to grab the code and run with it, or fork and PR back.

If you’re looking at the code and wondering about the display changes. There’s a font and back buffer which mimics the original arduboy buffer, but all the pixels are listed in order. For the display threading I only lock on the buffer copy.

For the display onto the OLED, I made a colour from the boolean array at runtime.
The EPaper works almost the same, but with the different calls for the libraries.

For the input threading, that works similarly I populate a byte in the thread and only lock on updates to it.
Then whenever the game logic needs the input, it gets the last known values after locking them.

So not my major forte, but I think it should avoid too much waiting on locks.

Have fun and happy hacking.

3 Likes

LOL you’ve got it paired up with a bluetooth controller too I hadn’t seen that. Oh man the esp32 is it.

Oh gosh that is still really good! I mean, yeah you aren’t gonna get squario to work very well on there, but a lot of puzzle type games. I think making this into a tamagotchi style enclosure would be nice.

Ah jeeze, I just realized it’s an MMORPG tamagotchi. Yes!

Possibly unrelated other than the fact it’s me, but super stoked with the ESP32 TV Out PS 3 Controller Arduboy too :slight_smile:

I had to hack it onto a branch though and gut a lot of code.

5 Likes

Haha wow! Well that is awesome I’m sure it was quite a bit of work, but is it really true that it isn’t enough processing power or memory?

I’ve done TV out on the arduino before, but… is it because of the resolution?

Are you double buffering i.e. you have what should be drawn on the OLED in a buffer and then interpolating that to another buffer for the TV?

What is the hardware for this is it a resistor ladder?

Nice work!

You know it’s a real shame that there isn’t an uncompressed low to medium res digital format so you could like bit bang into an hdmi connector somehow…

probably this way?

Are you double buffering i.e. you have what should be drawn on the OLED in a buffer and then interpolating that to another buffer for the TV?

I used Bitluni’s code for the composite output, it’s using a RCA cable for it (just the yellow one) into the TV AV Port.

Bitluni’s code was really portable and well written, with the original memory issues, I decided to throw my code away and just use his double buffer, which I populate in the normal Arduboy display thread using the Arduboy sBuffer.

What is the hardware for this is it a resistor ladder?
No additional hardware, you connect the ESP pin 25 to the Core and Ground to the outer.

Here’s the original code.

From the difficulty side
I had the ROM space for it, that was fine, but for the Ram it uses a 320x200x256 colour buffer for front and back to swap between, 64kb each, the bluetooth starting up uses quite a bit of ram, so I had to minimize the ram used. I changed the code to only use a boolean display buffer instead of a uint8_t and defined white and black based on that. That sorted out the memory issue.

Then I had an issue with the timing of the generation of the TV signal, that wasn’t working because the Bluetooth runs on Core0 and you cannot change it. Your app code runs on Core1, the TV out code needs a full core to run at it’s proper syncing speed. Eventually I came up with the idea to change the application to Core 0 with the Bluetooth and then gave Core 1 over to pushing pixels to the screen.

It seems to be running fine, but I think I messed up the timing a tiny bit with the lower resolution or the bool screen buffer, so there is a tiny shimmer on the screen.

I also had to remove all the buffering and threading I did in my code, which is why I ended up branching again.

I almost gave up at a few points, but happily I didn’t and got it working :slight_smile:

So no hardware other than the ESP32 hooked up to the TV and the PS3 controller.

It was for the gunning for the idea of minimalism just a chip and an off the shelf ps3 controller and you’re gaming.

I imagine you could potentially drive the screen and the tv at the same time, but might start hitting some speed issues if you try to run scaling, might run into memory issues if you need to buffer too much… it might be fun to have it work on the screen and the tv at the same time…

It was a fun project and my kids are quite happy to get a chance to play the games on the TV now.

2 Likes

Ahhh it uses a full color buffer, there is your problem :slight_smile:

Ah and needing full speed to generate the frequency you need.

So I guess it has a built in DAC?

Yeah it has two 8-bit DAC, so that helps a lot :slight_smile:

1 Like

I think this is so cool, it shows off the advantages of why having extra processing power other than just running the game at a faster speed.

It gives you the extra power to run peripherals and interfaces!

There has got to be a way to clean it up by working with just black and white? I’ve seen really good TV out from Arduino before, looked just as good as an old Atari.

1 Like

On the other hand, the attraction of retro consoles or neo retro-style consoles or fantasy consoles is in creativity through limitations.
Increasing the power of the platform destructs this magic.
For example, thinking about the balance of power and interest led me to the decision not to use extremely powerful ESP32 in my project. And there are a lot of gadgets with ESP32 like odroid-go, pocketSprite, ESPlay and much more like this… with nes, snes, gb, gbc, genesis emulators working fine on it. It can be easily made almost everything without any limitation problems. So what’s cool, what motivation to use this pseudo retro in comparison with Unity or similar modern game engines and game systems?
So the power/limitations balance is the point. And the answer to this question is not simple )

1 Like

Lol the same argument again. I guess Arduboy doesn’t deserve to continue any more.

zx spectrum has been finished 28 years ago by his father - Sir Clive Sinclair. but speccy’s heritage and creative communities lives and will live forever probably, and even reborns these days )

So worked on it a bit more, fixed up the TV out and got the screen to stop flickering and got a few more games working, Catacombs, Dark and Under, Shadow Runner, Circuit Boy.

The games all need a few minor changes to setup the Core for gamelogic and the just added WiFi and Over the Air Updates.

So I would say it’s using a lot of the ESP32’s power to get some valued added features going.

Additional features from the extra processing power :slight_smile: results in

  • Bluetooth Controller Support
  • WiFi up and running.
  • Over the air fimrware updates.
  • Built in TV out

The WiFi (and maybe Bluetooth) would allow for multiplayer.

I’m going to see about getting the screen and buttons going again, as I think that would be amazing to have working as well. I may start a new project for that rather than have this thing which I think is pretty awesome sitting on a branch :stuck_out_tongue:

2 Likes

When you are doing the TV out, is it also able to draw to oled at the same time, or are you directing the bits only at the tv out library? Oh, wait, I mean… the… e-paper? I guess the only display that is working is the tv?

1 Like

If you are feeling ambitious there is a co-op multiplayer version of Catacombs of the Damned that you could port to using WiFi/Bluetooth:

3 Likes

Don’t have dual screen working, but I think it’s plausible, maybe some sort of runtime key button press, or a firmware layer under the Arduboy framework to switch between TV Out , Screen and EPaper, possibly just do both at the same time always.

With the OTA updates, you might need to do something like that anyway, when it detects the OTA update being sent, shut down the BT stack and switch off the screen.

I don’t like how I have had to modify the loop and setup in the game’s ino files, so going to take a think on a better approach but not sure how to get the game code loop() to run on the alternate core without it. I think if the Arduboy framework was the controller of the loop() call, then I could control launching the game loop in a serperate thread.

If I was going to do this I would look into

  • Refactor the code from scratch again to break out screen display into seperate headers that can be chained to support multiple outputs and runtime switching.
  • Build in some sort of under Arduboy framework for choosing runtime options instead of ifdefs
  • Refactor the code for input to support mutliple input mechanisms and potentially add multiple controllers.
  • When a second+ controller is added / found, allow it to be exposed as an additional controller or be used primary.
  • Try find a way to only modify the framework for the support of the TV out code and not the game files.
  • See if there’s a way to store multiple games, but because you can’t load code into memory from flash, the only option would possibly be a VM type framework… or a way to only flash the diff of code, since > 1.3mb is Wifi and BT code.
  • Bring back Audio and get it working over the composite out.
  • Build in a simple shared gamestate multiplayer for wifi to bring in another player.

One thing to mention about the ESP32 is that it’s probably easily blown the budget for a coin cell, especially with the TV out or the LCD screen, but could possibly get away with a small lipo. The EPaper TTGO T5 linked earlier has a charger and batter port built in too. so it’s pretty much an all in one solution.

In all honesty it’s probably more work than I was thinking of investing time in for my single unit hackery, but if you were going to build out a product, I think it’s all doable.