Screen Mirroring Guide / How To Screenshot Your Game


(Holmes) #1

What Is Screen Mirroring?

Screen mirroring is displaying the Arduboy’s screen on your computer’s screen. You can do this to take a screenshot or video of your Arduboy game!

How Do I Do It?

Some games include the functionality already coded into them, such as Circuit Dude. Just turn on screen mirroring and connect your Arduboy to your computer using the USB cable.You’d also need a program that can display the screen on your computer.

• For PC, you can use the latest version of the Arduboy Manager that I made
• For other platforms, you can use this Chrome extension by @davidperrenoud

Video?

You can check out a video here:

What If I Want To Add This Feature To A Game?

If you want to add screen mirroring to your own game, it’s really easy!

In the setup() section, you should set the baud rate and open up a serial connection with the Serial.begin() function.


void setup() {
  arduboy.begin();
  //...
  Serial.begin(9600);
}

To actually send over what’s printed on the screen, at the bottom of the loop() section, we need to output the contents of the screen buffer over the serial connection with the Serial.write() function when we display the buffer to the screen.

loop() {
  //...
  Serial.write(arduboy.getBuffer(), 128 * 64 / 8);
  arduboy.display();
}

Arduboy Manager 2.0 Released!
Arduboy Utility : Android Application to upload, backup, and so on
Screen Grabs for Vast LIB
(spinal) #3

Just for fun, how about adding the RGB LED values to the screen mirroring also? I’m sure an extra 3 bytes wont hurt the speed.


(Scott) #4

The setRGBled() function, which is a common way of controlling the RGB LED, is write only. Where would the mirror function in the Arduboy get the current RGB values to send them to the PC?

The RGB LED can also be controlled digitally, using digitalWriteRGB(), which is also write only. So, there’s the same issue.

Finally, it’s difficult to determine whether the RGB LED is currently being controlled using PWM via setRGBled() or digitally using digitalWriteRGB().


(spinal) #5

Fair enough, just a thought. :slight_smile:


(Mike McRoberts) #6

You can use digitalRead on a pin that has been written to to obtain its state.


(Mike McRoberts) #7

I’ve been finding the screen mirroring function very useful for playing Circuit Dude. Instead of staring at a tiny screen, bring it up much larger on the PC screen and play it from there :slight_smile:


(Scott) #8

What if that pin was set using setRGBled() and thus is in PWM mode?


(Mike McRoberts) #9

Yeah it won’t work for PWM but you can always store the PWM value sin a variable and then send those over serial.


(Scott) #10

In order for this to work, @crait would have to modify his manager and @davidperrenoud would have to modify his Chrome extension. Both would have to expect additional data, in a standardised format, for each individual LED in the RGB LED indicating:

  • whether that particular LED is being controlled digitally or via PWM
  • if it’s controlled digitally, whether it is on or off. Otherwise, its analog PWM value.

It would then be up to each sketch that uses mirroring to decide how to maintain this data, and transmit it along with the screen image data. Even if a sketch didn’t use the RGB LED, “dummy” (all off) data would have to be transmitted.


#11

Have got screen mirroring working on a Raspberry Pi, writing pixels directly to the Linux frame buffer! Have also added code to switch between different colour palettes using the Pi’s keyboard…

The code isn’t pretty, but it works. Patched together from snippets of different Raspberry Pi examples - needs wiringPi installed for the serial and may have some other dependencies too, can’t remember, anyway here it is:

http://pastebin.com/Tvpiv4G5
http://pastebin.com/31dhZdDN


Just an Ardu-tease
(Mike McRoberts) #12

Superb. Nice job. I must try this with my Pi.


(curly) #14

thanks for this guide, ill be adding that into every game i make (unless its too big but i didnt check how much this added, id assume not much?)

also, i had problems reflashing new games and couldt figure out why, then it dawned on me i had the screen mirror open, as soon as i closed that, reflashed just fine

this SHOULD HAVE BEEN OBVIOUS but well im not captain obvious sometimes


(Ulf Benjaminsson) #15

Am I the only one who experience hangs when closing the Serial Monitor window?

Win10, running with admin privileges.

Screen Mirroring works, and I can close the window no problems.
Serial Monitor works, but closing the window hangs the application.


(Jesse Campbell) #16

Can you post a diagram that maps bytes to pixels? I’m trying to do my own implementation.


(Pharap) #17

122aae02ad9e5ec21319b177889378bb2f73c03a


(Jesse Campbell) #18

Thanks, I got it working with Processing (.org) which is a Java-based graphical tool.

screenshot 3

import processing.serial.*;

Serial myPort;
byte[] inBuffer;
color background;
color foreground;

void setup(){
	size(128, 64);   
	printArray(Serial.list());
	myPort = new Serial(this, Serial.list()[1], 9600);
	background = color(0);
	foreground = color(255);
	inBuffer = new byte[1024];
	frameRate(60);
}

void draw(){  
	while (myPort.available() > 0) {
		inBuffer = myPort.readBytes();
		myPort.readBytes(inBuffer);
		background(background);
		
		if (inBuffer != null) { 
			for(int i=0; i<1024; i++){
				int b = i/128;
				if(inBuffer[i] != 0){
					for(int j=0; j<8; j++){
						set(i%128,j+b*8,((inBuffer[i] >> j & 1 ) == 1)?foreground:background);              
					}
				}
			}                       
		}
	}
}

(Jesse Campbell) #19

Here’s some updated code that includes scaling:

You can change the foreground and background color as you wish.

The Arduboy doesn’t actually need a screen connected to play the games. My DIY Arduboy has a removable screen. The screen can be removed at any time and it continues working.

import processing.serial.*;

Serial myPort;
byte[] inBuffer;
color background;
color foreground;
float scale = 8;

void setup(){
  size(1024, 512);   
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[1], 9600);
  background = color(0,0,0);
  foreground = color(0,255,0);
  inBuffer = new byte[1024];  
  frameRate(60);
  surface.setResizable(true);
  noStroke();
}

void draw(){    
  scale = round(width/128.0);  
  while (myPort.available() > 0) {    
    inBuffer = myPort.readBytes();
    myPort.readBytes(inBuffer);
    background(background);
    if (inBuffer != null) {        
        for(int i=0; i< inBuffer.length; i++){
          int b = i/128;
          if(inBuffer[i] != 0){
            for(int j=0; j<8; j++){
              if((inBuffer[i] >> j & 1 ) == 1){                
                fill(foreground);                
              }
              else{
                fill(background);
              }               
              rect((i%128)*scale,scale*(j+b*8),scale,scale);              
            }
          }
         }                       
    }
  }
  println(frameRate);
}

(Jesse Campbell) #20

Here’s a version that lets you play vertically oriented games.

import processing.serial.*;

Serial myPort;
byte[] inBuffer;
color background;
color foreground;
float scale = 4;

void setup(){
  size(512, 1024);   
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[1], 9600);
  background = color(204,204,204);
  foreground = color(153,0,255);
  inBuffer = new byte[1024];  
  frameRate(60);
  surface.setResizable(true);
  noStroke();
}

void draw(){
  scale = round(height/128.0);
  while (myPort.available() > 0) {    
    inBuffer = myPort.readBytes();
    myPort.readBytes(inBuffer);
    background(background);
    if (inBuffer != null) {        
        for(int i=0; i< inBuffer.length; i++){          
          if(inBuffer[i] != 0){
            for(int j=0; j<8; j++){
              if((inBuffer[i] >> j & 1 ) == 1){                             
                fill(foreground);
              }
              else{
                fill(background);
              }
              float x = (i%128)*scale;
              float y = scale*(j+floor(i/128)*8); 
              rect(y,height-scale-x,scale,scale);
            }
          }
         }                       
    }    
  }
}

(birdman) #21

Hi Jesse,

Thanks for sharing the code. I was wondering how do you sync the serial data and the beginning of the frame. I get a distorted image :sweat_smile:


(Jesse Campbell) #22

The code just works for me. It’s sending 1024 bytes in order from 0 to 1023. As long as no bytes get lost in transmission it should be aligned. Maybe you can try another computer?