Screen Mirroring Guide / How To Screenshot Your Game

You can use digitalRead on a pin that has been written to to obtain its state.

I’ve been finding the screen mirroring function very useful for playing Circuit Dude. Instead of staring at a tiny screen, bring it up much larger on the PC screen and play it from there :slight_smile:

3 Likes

What if that pin was set using setRGBled() and thus is in PWM mode?

Yeah it won’t work for PWM but you can always store the PWM value sin a variable and then send those over serial.

In order for this to work, @crait would have to modify his manager and @davidperrenoud would have to modify his Chrome extension. Both would have to expect additional data, in a standardised format, for each individual LED in the RGB LED indicating:

  • whether that particular LED is being controlled digitally or via PWM
  • if it’s controlled digitally, whether it is on or off. Otherwise, its analog PWM value.

It would then be up to each sketch that uses mirroring to decide how to maintain this data, and transmit it along with the screen image data. Even if a sketch didn’t use the RGB LED, “dummy” (all off) data would have to be transmitted.

Have got screen mirroring working on a Raspberry Pi, writing pixels directly to the Linux frame buffer! Have also added code to switch between different colour palettes using the Pi’s keyboard…

The code isn’t pretty, but it works. Patched together from snippets of different Raspberry Pi examples - needs wiringPi installed for the serial and may have some other dependencies too, can’t remember, anyway here it is:

http://pastebin.com/Tvpiv4G5
http://pastebin.com/31dhZdDN

3 Likes

Superb. Nice job. I must try this with my Pi.

2 Likes

thanks for this guide, ill be adding that into every game i make (unless its too big but i didnt check how much this added, id assume not much?)

also, i had problems reflashing new games and couldt figure out why, then it dawned on me i had the screen mirror open, as soon as i closed that, reflashed just fine

this SHOULD HAVE BEEN OBVIOUS but well im not captain obvious sometimes

1 Like

Am I the only one who experience hangs when closing the Serial Monitor window?

Win10, running with admin privileges.

Screen Mirroring works, and I can close the window no problems.
Serial Monitor works, but closing the window hangs the application.

Can you post a diagram that maps bytes to pixels? I’m trying to do my own implementation.

122aae02ad9e5ec21319b177889378bb2f73c03a

1 Like

Thanks, I got it working with Processing (.org) which is a Java-based graphical tool.

screenshot 3

import processing.serial.*;

Serial myPort;
byte[] inBuffer;
color background;
color foreground;

void setup(){
	size(128, 64);   
	printArray(Serial.list());
	myPort = new Serial(this, Serial.list()[1], 9600);
	background = color(0);
	foreground = color(255);
	inBuffer = new byte[1024];
	frameRate(60);
}

void draw(){  
	while (myPort.available() > 0) {
		inBuffer = myPort.readBytes();
		myPort.readBytes(inBuffer);
		background(background);
		
		if (inBuffer != null) { 
			for(int i=0; i<1024; i++){
				int b = i/128;
				if(inBuffer[i] != 0){
					for(int j=0; j<8; j++){
						set(i%128,j+b*8,((inBuffer[i] >> j & 1 ) == 1)?foreground:background);              
					}
				}
			}                       
		}
	}
}
3 Likes

Here’s some updated code that includes scaling:

You can change the foreground and background color as you wish.

The Arduboy doesn’t actually need a screen connected to play the games. My DIY Arduboy has a removable screen. The screen can be removed at any time and it continues working.

import processing.serial.*;

Serial myPort;
byte[] inBuffer;
color background;
color foreground;
float scale = 8;

void setup(){
  size(1024, 512);   
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[1], 9600);
  background = color(0,0,0);
  foreground = color(0,255,0);
  inBuffer = new byte[1024];  
  frameRate(60);
  surface.setResizable(true);
  noStroke();
}

void draw(){    
  scale = round(width/128.0);  
  while (myPort.available() > 0) {    
    inBuffer = myPort.readBytes();
    myPort.readBytes(inBuffer);
    background(background);
    if (inBuffer != null) {        
        for(int i=0; i< inBuffer.length; i++){
          int b = i/128;
          if(inBuffer[i] != 0){
            for(int j=0; j<8; j++){
              if((inBuffer[i] >> j & 1 ) == 1){                
                fill(foreground);                
              }
              else{
                fill(background);
              }               
              rect((i%128)*scale,scale*(j+b*8),scale,scale);              
            }
          }
         }                       
    }
  }
  println(frameRate);
}
1 Like

Here’s a version that lets you play vertically oriented games.

screenshot 1

import processing.serial.*;

Serial myPort;
byte[] inBuffer;
color background;
color foreground;
float scale = 4;

void setup(){
  size(512, 1024);   
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[1], 9600);
  background = color(204,204,204);
  foreground = color(153,0,255);
  inBuffer = new byte[1024];  
  frameRate(60);
  surface.setResizable(true);
  noStroke();
}

void draw(){
  scale = round(height/128.0);
  while (myPort.available() > 0) {    
    inBuffer = myPort.readBytes();
    myPort.readBytes(inBuffer);
    background(background);
    if (inBuffer != null) {        
        for(int i=0; i< inBuffer.length; i++){          
          if(inBuffer[i] != 0){
            for(int j=0; j<8; j++){
              if((inBuffer[i] >> j & 1 ) == 1){                             
                fill(foreground);
              }
              else{
                fill(background);
              }
              float x = (i%128)*scale;
              float y = scale*(j+floor(i/128)*8); 
              rect(y,height-scale-x,scale,scale);
            }
          }
         }                       
    }    
  }
}
1 Like

Hi Jesse,

Thanks for sharing the code. I was wondering how do you sync the serial data and the beginning of the frame. I get a distorted image :sweat_smile:

1 Like

The code just works for me. It’s sending 1024 bytes in order from 0 to 1023. As long as no bytes get lost in transmission it should be aligned. Maybe you can try another computer?

Think about the 1024 bytes that are sent as being a single frame that is transmitted. These frames are basically sent over as multiple bursts of data. If you just read the first 1024 bytes you receive and assume that’s a frame, then you’ll most likely be off by a little, reading some of a previous frame and some of the next frame. This will make everything look all distorted.

If you can, depending on the environment, detect the beginning or end of a burst, you should be able to just throw out the data you have for the previous frame.

I tested it for a couple days for a bunch of different games and didn’t notice any sync issues. That sounds like a good way to fix the issue crait. I’ll think about it and post a solution.

Curious: has Anyone attempted this with SPI? :slight_smile:

Well… not exactly.

But I’ve used a logic analizer to record SPI clock and data while developing my fast 18 cycles per SPI byte display routine. With the analyzer I could export the data and convert them to screenshots though.

I had an idea for an OLED display to TV OUT. which should be doable but it is one of many projects on my project heap. Would be awesome to hook Arduboy to a video projector :smiley:

Haven’t thought about other ways yet.