Sound "mirroring" to a PC

Would it be possible to get the current audio tone being played and pass that over Serial to a screen mirroring program?

What exactly would you expect the result to be? An image of a sound wave?

No, the purpose would be play the same sound through computer speakers during screen mirroring.

Not easily. You would have to modify the sound library, or whatever the sketch uses to generate sound, to send information over serial indicating what and when something is playing. The computer, or whatever is receiving the sound data, would have to interpret it to play the equivalent sound.

You would also have to devise the format for how to represent the sound data, and also a protocol for multiplexing screen info and sound.

Right. I think it would be easier to just solder a headphone jack in parallel of the speaker.

That presents it’s own problems. The Arduboy speaker is wired across two output pins. If you were to wire these pins to a jack and connect it to the line input of a computer, you could end up shorting one of the pins to ground if you plugged the Arduboy into the computer via USB at the same time.

Also, the output to the speaker can go to 10V peak to peak. An audio “line” level input is usually specified for around 1V peak to peak.

For both of the above reasons, you would probably want to add some sort of Arduboy to line level signal conditioning circuitry.

That’s good to know. I made an Arduboy clone back in 2017 and would update that design.

I know some audio jacks have a built-in toggle for switching between speaker and headphones. A resistor in series with the headphone pin could lower the sound to a safe voltage. Not sure if I’d put the volume knob in series with the headphone pin.

The easiest way I can think of for doing this is to assign one ‘code’ (i.e. a byte with a specific numerical value) to each operation (i.e. each overload of the ArduboyTones function that’s being used, plus one for transmitting a copy of the screen buffer), and assign each operation either a fixed number of parameters (and thus a fixed size of bytes to be transmitted) or a means of reporting how many bytes will be transmitted for a variable number of parameters.

E.g. 0x01 could mean ‘mirror screen buffer’, after which the receiver would expect 1024 bytes (a full frame); 0x02 could correspond to tone(uint16_t freq, uint16_t dur = 0), meaning it would be followed by two 2-byte parameters; 0x03 could be noTone() and thus have no parameters… et cetera.

(Effectively a kind of remote call system.)

It would be a lot of work, the sound might not sound quite right and the protocol’s probably a bit flimsy, but it’s doable for someone who knows enough about sound, rendering and serial transmission to write a program capable of receiving, interpreting and handling the data properly.

1 Like

It would be neat if there was an audio equivalent of this:
Serial.write(arduboy.getBuffer(), WIDTH * HEIGHT / 8);

Getting the sound that’s currently playing for each frame, and sending it over serial.

The video frames and audio output are often not related or synchronised with each other. Most sound libraries are interrupt driven, generating sound in the “background” while screen updates are done by the “foreground” code. (An exception to this is the Arduboy2 library’s BeepPin functions, where the standard way of timing tone durations is by frame counts.)

As has been said, it would have to be done on a per tone basis, which would require it to be done by the library or whatever is generating tone. And you have to consider things like the ArduboyPlaytune library, which can be playing two notes from a two part score at once.

Things get even more complicated for ATMlib and Ardvoice, which generate PWM analogue waveforms.

1 Like