How exactly do pointers work?

This inquiry was actually sparked from @filmote 's Steve the jumping dinosaur game. In the functions getImageWidth() and getImageHeight() in Images.h, (I’m going to refer to just getImageWidth()) it has a pointer called image. The Steve struct has another pointer called image. Later on, Steve’s image pointer becomes a pointer (I think) for all the Steve images, but is then used as a pointer for getImageWidth()(???).

Here’s what I currently know about pointers:
Each thing has an address-which should not be confused with a value. So in the following:

string arduboy = "The Arduboy is so awesome"
string *ptr = arduboy;

What’s happening here (I think) is ptr is a pointer, which is assigned the address of the string arduboy. (or maybe I’m confusing this with the address of operator (&)).

For something like this:

struct Arduboy
{
    uint8_t *pointer;
};

Arduboy arduboy;

uint8_t returnValue(uint8_t *pointer)
{
    return (pointer);
}

uint8_t getValue()
{
    returnValue(arduboy.pointer);
}

So this is along the lines of what @filmote did. If I’m correct, the returnValue() pointer is a pointer to itself (???) and arduboy.pointer is a pointer to the returnValue() pointer (???).

I’d appreciate a more general response about pointers, but as @pharap pointed out several times, context is always a great inclusion.

Every value that is stored has a “start” point where it is stored in memory. The type definition establishes the length of the value, in the case of the string it actually uses a helper function to encode the length of the string within the value that is actually stored (I don’t know how this is done).

The pointer is simply a memory address to where this value “starts”. It’s like the address to a house and the value is the people that live there.

This is my bad explanation but you’re on the right track other resources that teach about pointers are helpful.

1 Like

The first thing you have to understand to properly understand pointers is RAM.

Random access memory, like many other types of computer memory, is byte addressable. That means that every byte of memory has an associated address that can be used to uniquely identify that byte of memory, and in turn the hardware is designed in such a way that indivual bytes can be read or written when the specific byte’s address is provided.

A memory address is represented with a pattern of bits in binary in such a way that memory addresses are effectively just ordinary integers.

In other words, you can effectively think of RAM as being like a giant indexable array of bytes.

(Or in the Arduboy’s case, not that giant - just 2560 bytes, or 2.5KB.)

So, what does this have to do with pointers?

Essentially pointers are memory addresses. They are indices into that giant array of bytes.

(That also means they are secretly integers, or arguably aren’t integers but may be treated like integers and/or share certain properties that integers have.)

The main reason they are called ‘pointers’ in a programming context is because in a programming context pointers aren’t just an address, they are also associated with the type of the object they ‘point’ to. A uint8_t * points to a uint8_t, a Point * is a pointer to a Point object, et cetera.


With that introduction out of the way, it’s time to address your points:

Images for use with Arduboy’s Sprites format are stored as arrays of uint8_t, and the Sprites functions expect a pointer to the first byte of one of those arrays.

The first two bytes of a Sprites-format image array are the width and height of the image, which is why getImageWidth and getImageHeight are able to extract the image’s dimensions - they merely read the uint8_ts in the first two addresses of the image array.

The remaining uint8_ts contain the actual image data, and may contain more than one image, known as a ‘frame’ in Sprites’ terminology.

The Sprites library knows how many bytes a frame occupies because it calculates the size of the frame (in bytes) by using the width and height of the image.

It doesn’t know precisely how many frames an image holds, it trusts the programmer to never attempt to access a frame that isn’t there, much like the compiler trusts the programmer to never try to go outside the bounds of an array.

(There’s also one more thing going on with images, which is that images aren’t actually stored in RAM, they’re stored in progmem, which is why pgm_read_byte is needed to read them, but that’s a story for later. Best get ordinary RAM pointers out of the way first…)

Technically an address is a value, but it is not the value of the object that the pointer points to, it’s the value of the pointer itself.

This is incorrect, it should be:

string arduboy = "The Arduboy is so awesome"
string *ptr = &arduboy;

The address-of operator (unary &) ‘takes’ the address of an object, an action which produces a pointer that points to the first byte of the object whose address has been ‘taken’. In this case that’s a string *.

As an aside, there is no string type in Arduboy code unless you define one. Arduino provides a String type (with a capital S), but you shouldn’t really use it in Arduboy code if you can help it because String is generally too costly for the Arduboy’s resource constraints.

This also wouldn’t compile (or at least it shouldn’t compile on a sane compiler, but Arduino disables some of the safety precautions so it probably does compile and does something horrendous).

The problem here is that you’re trying to implicitly convert a pointer to a uint8_t, not read the value it points to.

To read the value a pointer points to you have to use the derefence operator (unary *), i.e. *pointer.

The problem with this one is that you’re missing the actual return.

I think what you were aiming for is something more like this:

struct Demo
{
	uint8_t * pointer;
	
	uint8_t readPointer()
	{
		return *pointer;
	}

	void writePointer(uint8_t value)
	{
		*pointer = value;
	}
};

The expression *pointer reads the value of the object to which a pointer points.

The statement *pointer = value assigns the value of value to the object to which the pointer points.


Pointers aren’t always guaranteed to point to valid objects, which is one of the reasons they’re considered dangerous.

It’s perfectly possible to create a pointer that contains an invalid address, i.e. an address that doesn’t actually exist in the hardware’s memory. For example, the Arduboy only has 2560 bytes, so if you were to create a pointer with a value of e.g. 4095 and try to dereference it then the hardware wouldn’t be able to comply with that request and would do whatever it does when it encounters such an error.

On desktop the OS would catch the problem, display an error message, and probably terminate the program. On Arduboy it would probably just cause the Arduboy to reset.

However, there’s one invalid pointer value that’s actually useful: nullptr. nullptr is an invalid pointer that, rather than being accidentally invalid, is purposely used to signal the fact that the pointer is invalid. In reality there are lots of invalid pointer values, but nullptr’s job is to act as a way to inform other programmers that the pointer’s is intentionally invalid. It’s essentially used to mean “this pointer isn’t pointing to anything in particular at the moment”, which is useful in various different circumstances.

For example, a linked list - a list formed by several different ‘list nodes’ that each point to the next node in the chain - must have an end, and often that end is signalled by having the last node’s pointer be nullptr, to signal that “there are no more items, this is the end of the list”.


One other problem that can arise from pointers is when a pointer points to an object that becomes invalid.

Most notably, local variables (as you may recall) only exist for the duration of the scope that they’re in exists, e.g. for as long as the function in which they’re declared is running. As soon as their scope ends, e.g. the function they’re declared in returns, they cease to exist.

Thus if a pointer points to a local variable and the function that local variable is in returns, that pointer now points to invalid memory which is likely to be overwritten by something else and cannot be relied upon.


There’s a lot more that could be said about pointers, but I have to stop at some point, otherwise people start complaining.

Hopefully that’s enough information to help you understand more or less what pointers are and some of their uses and issues.

I will say that generally you should avoid using pointers if you can, though that’s not always possible or practical. It’s certainly harder on an embedded system than it would be on desktop.

(E.g. Sprites would have to use pointers internally even if its public-facing functions didn’t, as would anything that touches progmem on Arduboy.)

Hopefully you’ll have some follow up questions so I can fill in some of the blanks that I’ve had to leave this time around.


Recommended learning resources:

1 Like

Oh. I was reading some examples from some other websites and I guess that’s why I used a string.

Oh, yep. I forgot how a pointer technically isn’t a normal variable. (I think I forgot to add pgm_read_byte after the return (that’s what @filmote did).

What I was going for was like a flexible function which you can use any pointer instead. Sort of like adding a pointer to the parameter of return, then later being able to use any image pointer to point(?) to that parameter pointer and get the value from that image(?)

Guess I did confuse it with the address of operator :sweat_smile:. I’ve seen stuff before (like some of the examples above) where there aren’t any address of operators, so is it necessary when using pointers?

I think the biggest question I have though, is why/when should you use pointers? By now, I understand pointers, I just don’t see in what context they should be used in.

It is a normal variable. It has a value, you can reassign it, you can do anything you can do with any other variable.

But a pointer is a pointer, not the thing it points to. pointer is a pointer. *pointer is the thing the pointer points to. You have to have that distinction otherwise you can’t copy or reassign the pointer itself.

Yes, but that’s expected. You won’t have seen pgm_read_byte in any ordinary C++ tutorials because it’s specific to the kind of CPU the Arduboy uses.

I’m not sure quite what you mean by this.

You seem to be getting your terminology muddled.

return doesn’t “have” a parameter. Functions have parameters, return statements may be used to return the values of those parameters, but the return statements do not ‘have’ a parameter.

And now you’ve just lost me.

Get what value from an image? The width? The height?
Individual bytes of the frame data?

It depends on what you’re trying to assign to what…

Presume there to be a variable uint8_t object = 8;.

  • The type of the expression object is uint8_t.
  • The type of the expression &object is uint8_t *.

When assigning a variable (any variable) the type of the expression on the right hand side of the assignment operator (=) must either match or be convertable to the type of the variable on the left hand side of the assignment operator.

Thus:

// Attempt to assign 'uint8_t' to 'uint8_t *'.
// Types do not match and are not implicitly convertible,
// thus this is a compile-time error.
uint8_t * pointerA = object;

// Attempt to assign 'uint8_t *' to 'uint8_t *'.
// Types are an exact match. There is no error.
uint8_t * pointerB = &object;

// Attempt to assign 'uint8_t *' to 'uint8_t *'.
// Types are an exact match. There is no error.
uint8_t * pointerC = pointerB;

// Attempt to assign 'uint8_t *' to 'uint8_t * *'.
// Types do not match and are not implicitly convertible,
// thus this is a compile-time error.
uint8_t * * pointerD = pointerB;

// Attempt to assign 'uint8_t * *' to 'uint8_t * *'.
// Types are an exact match. There is no error.
uint8_t * * pointerE = &pointerB;

Whenever you look at any expression (e.g. a variable; a literal value such as an integer literal, character literal, or string literal; a function call; an operator being applied to one or more other expressions), it pays to always think about the type of the expression.

When used properly, types guard against runtime bugs by detecting inconsistencies and nonsensical behaviour at compile time.

Most of the time you shouldn’t. You should only really use them when you have no other choice.

Those times are:

  • When you need to refer to a specific object (e.g. a variable) in such a way that you can refer to a different object later on, or specifically refer to no object.
    • If you want to refer to an object and you don’t need to reassign the reference, it’s better to use references rather than pointers.
  • When you need to do something involving hardware addresses.
    • On desktop this is relatively uncommon due to most hardware features being hidden behind common libraries.
    • On embedded systems this is often necessary unless the hardware details are sufficiently well hidden by utility libraries.
    • This is relevant for Arduboy because pointers are necessary for reading progmem (via pgm_read_byte & co).
  • When you need to use dynamic memory allocation.
    • On Arduboy you won’t. It’s too expensive.
    • On desktop, most of the time dynamic memory allocation is hidden behind other data types like ‘smart pointers’ or lists.
  • When you need to use a library that requires a pointer.
    • Whether the library needs to use a pointer is another matter. I’ve seen a lot of libraries use pointers when they could use something safer instead, like references. But if you can’t change the library or aren’t willing to spend the effort then you must make do.

Dealing with progmem is the most likely reason you would need a pointer in an Arduboy game. If you’re making the most of the hardware you’re likely to need to store some game data (e.g. maps, text) in progmem, and you’ll need pointers to read that data back.


By the way, the Wikipedia article about pointers is a lot better than it used to be when I was first starting out:

(Note the sheer number of sections in the contents.)

The C++-specific section is here:

It’s even got this really nice diagram:

(It’s a tiny bit inaccurate, but it gets the idea across.)


I’ll leave it there for now to give you chance to take that in.

I’m greatly anticipating the questions “What’s a reference?”, “Why are references better than pointers?”, “What is progmem?” and “How do I use progmem?”, since those are the natural follow-ons from here.

(Also maybe “What’s the difference between a value and an object?”, which I wouldn’t have been able to answer before because you wouldn’t have known what addressable memory is.)

I think I meant one function where you can get the height/width of any image, through probably a pointer for that image?

That’s just the side effect of learning pointers :wink:.

What’s a reference? Why are references better than pointers? What is progmem (let me guess, is it the program memory/storage and the thing that has all the hex number address)? How do I use progmem?. Also, one more: what’s the difference between progmem, flash memory, and RAM? You’ve pointed me in the direction (:wink:) to use the F (I can’t remember if it’s called a literal or a macro) to save the text to the progmem(?), rather than the flash memory(?).

That’s what the getImageWidth and getImageHeight in the dino demo do. They work with any image as long as they follow the Sprites format.

It would be impossible to make anything more general because the data format dictates the specifics, thus a different format would require a different strategy and thus a different function.

A reference is like a pointer in that it can reference another object, but:

  • You don’t have to explicitly dereference the reference, that happens automatically.
  • A reference only ever refers to one object, the reference cannot change to refer to another object after it has been created.
  • A reference cannot be null.

If you had a function like so:

void doSomething(int * pointer)
{
	*pointer = 5;
}

And someone wrote doSomething(nullptr) then the function causes a problem (e.g. error message, crash, whatever) at runtime because it’s illegal to dereference a nullptr and it’s not known that the pointer is null until runtime.

But if you had a function like so:

void doSomething(int & reference)
{
	reference = 5;
}

Then it’s impossible to do doSomething(nullptr) because nullptr is a pointer, not a reference, and there is no null reference. The compiler forces the caller to provide a valid object. (With some caveats.)

Here’s some demo programs to show you the general behaviour of a reference:

void example()
{
	int variable = 5;
	int * pointer = &variable;
	int & reference = variable;

	// 'variable' is 5
	arduboy.println(variable);

	*pointer = 10;

	// 'variable' is 10
	arduboy.println(variable);

	// '*pointer' is 10
	arduboy.println(*pointer);

	reference = 15;

	// 'variable' is 15
	arduboy.println(variable);

	// 'reference' is 15
	arduboy.println(reference);
}
// "Pass by value".
// 'value' holds a copy of the value
// that was given to the function as an argument.
void passByValue(int value)
{
	value = 5;
}

// "Pass by reference".
// 'reference' is a reference to the variable
// that was given to the function as an argument.
void passByReference(int & reference)
{
	reference = 5;
}

void demonstrate()
{
	int variable = 10;
	
	// 'variable' is 10
	arduboy.println(variable);
	
	passByValue(variable);
	
	// 'variable' is still 10
	arduboy.println(variable);
	
	passByReference(variable);
	
	// 'variable' is has become 5
	arduboy.println(variable);
}


void doSomething(int * pointer)
{
	*pointer = 5;
}
void example()
{
	int variable = 5;
	int & reference = variable;
	
	// Take the address of the variable
	int * pointerA = &variable;
	
	// Take the address of the reference
	int * pointerB = &reference;

	// The pointers are equal.
	// I.e. they hold the same address,
	// the address of variable.
	arduboy.println(pointerA == pointerB ? F("true") : F("false"));
}
void example()
{
	// Compiler Error:
	// An integer literal is not an object, it is a value,
	// which means it doesn't have an address,
	// and therefore it's impossible to make a reference to it.
	int & reference = 10;

	// For the record, the same applies to pointers:
	int * pointer = &10;
}
  • Not having to manually dereference is convinient.
  • The fact they can’t be null automatically rules out lots of potential bugs.
  • The fact they can’t be reassigned encourages better use of scoping.

Progmem is flash. Sort of.

The Arduboy has three types of memory:

  • RAM
  • Flash*
  • EEPROM

* The Arduboy’s flash memory is commonly termed ‘progmem’ (‘program memory’) because it’s where the program’s code and read-only data is kept.

These can all be categorised with two properties:

  • Volatility - Whether or not the data stays after the power has been cut.
    • Volatile memory loses its data when the power is cut.
    • Non-volatile memory retains its data without being powered.
  • Writability - Whether or not the memory can be written to while a user-mode Arduboy program is running.
    • Writable memory can be written to when a user-mode program is running.
    • Non-writable memory cannot be written to when a user-mode program is running.

They can be categorised as thus:

  • RAM is volatile and writable.
  • Flash is non-volatile and non-writable.
  • EEPROM is non-volatile and writable.

RAM is where nearly all mutable variables are kept - local and global.

Flash (progmem) is where all the machine code instructions and some read-only data is kept. It can only be written to while the Arduboy’s processor is in bootloader mode, which is how it’s possible to upload new programs despite the memory being read-only while a program is running.

EEPROM is what you use to save and load save data.

As a special mention, the FX chip contains flash memory, but because it’s off-board memory it can’t contain executable machine code and it can be overwritten when a user-mode program is running.


Now, here’s the catch…

Each of these types of memory are addressable, but on AVR (the architecture the Arduboy uses) pointer dereferencing can only read and write RAM for technical reasons. (I can explain those technical reasons if you want. The short answer is “different address spaces”.)

To read from progmem a special machine code instruction is needed, hence why pgm_read_byte exists. It’s actually a macro that embeds assembly code into the program that will produce the correct flash-reading instruction(s). There’s several similar functions for different basic data types (e.g. pgm_read_float, pgm_read_ptr, memcpy_P) that also effectively generate or otherwise use those special progmem-reading machine code instructions to read progmem.

EEPROM is a similar affair, though the interface Arduino exposes uses integer addresses rather than pointers because of the way it’s intended to be used.


Given F("hello"), the "hello" is a string literal, and the F is a macro.

The F macro is actually defined as thus:

I won’t go into what that actually does at the moment because it’s relatively advanced and relies on a bit of hackery (and frankly is a bit risky and poorly thought out).

As mentioned above, progmem is flash.

The F macro places the string entirely in flash/progmem, as opposed to RAM, which is where it would normally end up.

The thing is, although a string literal normally ends up in RAM, it also (usually) occupies progmem.

What would normally happen is that the actual characters would be kept in progmem and then when the string literal is needed by the program there would be some code to allocate a small buffer in RAM (just as large as is necessary) and the byes would be copied from progmem into RAM.

By using the F macro the copy in progmem exists as it normally would, but the data isn’t copied into a RAM buffer, the pointer returned points directly to the version in progmem and the code that uses it is expected to use pgm_read_byte instead of trying to dereference it. That’s much more efficient, but also more work for the programmer.

This only applies to the AVR processor architecture though. On a desktop everything would end up in RAM. (Ignoring some caveats that I won’t go into now.)

There’s nothing particularly special about displaying addresses in hexadecimal.

All addresses can be interpreted as integers, and hexadecimal is just another way to display integers.

Using hexadecimal is just convention because often there’s several thousand addresses to deal with and hexadecimal is more regular and compact. It also fits the size of the pointers better. E.g. the Arduboy’s pointers are 16-bits in size (as opposed to the 32-bit or 64-bit pointers most desktops, laptops or phones have) which means you only need 4 hexadecimal digits to represent all the valid pointer addresses (and some invalid ones - invalid because the Arduboy only has 32KB of progmem, 2.5KB of RAM and 1KB of EEPROM).


That’s quite an infodump, but hopefully that clears up a lot and you’re starting to see the bigger picture.

(Fundamentally a computer is just a machine that copies data around, does arithmetic, makes a few branching decisions, and possibly sends and receives a few peripheral signals.)

1 Like

Thanks! I have one final question: why do you need a reference to a variable? For example:

int variable = 10;
int & reference = variable;

int getValue()
{
    return (reference)
}

Why not just do

int getValue()
{
    return (variable)
}

(Assuming this is what you might use a reference for)

Would one of the potential applications actually be for private variables?

You don’t necessarily, it depends on what you’re trying to do.

You only need a reference if you want to be able to modify the object being referenced. Otherwise you end up with a copy rather than the original.

Try examining and possibly running the examples I posted above.

Particularly this:

// "Pass by value".
// 'value' holds a copy of the value
// that was given to the function as an argument.
void passByValue(int value)
{
	value = 5;
}

// "Pass by reference".
// 'reference' is a reference to the variable
// that was given to the function as an argument.
void passByReference(int & reference)
{
	reference = 5;
}

void demonstrate()
{
	int variable = 10;
	
	// 'variable' is 10
	arduboy.println(variable);
	
	passByValue(variable);
	
	// 'variable' is still 10
	arduboy.println(variable);
	
	passByReference(variable);
	
	// 'variable' is has become 5
	arduboy.println(variable);
}


void doSomething(int * pointer)
{
	*pointer = 5;
}

In that case it wouldn’t really make a difference because you’re returning an int, which by definition causes the value to be copied. But consider this:

class Example
{
private:
	int variable;
	
public:
	int & getVariableReference()
	{
		return variable;
	}

	int getVariableValue()
	{
		return variable;
	}
};

void test()
{
	Example example;
	
	// Success: Returns a reference,
	// which can be used to assign
	// to the private variable.
	example.getVariableReference() = 5;
	
	// Failure: Cannot assign to
	// a non-reference value.
	example.getVariableValue() = 5;
}

They can be used to return references to private variables, though that’s not a particularly interesting use.

There’s some uses that I can’t demonstrate because it would require other knowledge that you don’t have yet.

One you might be interested in is to get a reference to an object that results from a more complex expression, such as array indexing:

Enemy enemies[10];

void doSomethingWithEnemies()
{
	for(uint8_t index = 0; index < 10; ++index)
	{
		Enemy & enemy = enemies[index];
		
		// From here on out you can
		// use 'enemy' in place of 'enemies[index]',
		// which makes the code more readable.
	}
}

If you did Enemy enemy = enemies[index]; instead then enemy would be a copy of the object in enemies[index], which means any attempts to modify enemy would only modify enemy and not the object at enemies[index].

Taking a reference instead (i.e. Enemy & enemy = enemies[index];) means that any modification made to enemy will directly affect the object at enemies[index] because they refer to the same object.

Then of course you can use a ranged for loop:

Enemy enemies[10];

void doSomethingWithEnemies()
{
	for(Enemy & enemy : enemies)
	{
		// From here on out you can
		// use 'enemy' in place of 'enemies[index]',
		// which makes the code more readable.
	}
}

The same rules apply. It’s only by using a reference that you can modify the actual object in the array. Using Enemy enemy : enemies would give you a copy of the data instead.

Which brings me to one last point: sometimes a reference is more efficient if an object happens to be expensive to copy. References are cheap and lightweight, typically the same size as a pointer (which isn’t a coincidence). Ultimately it depends on the datatype, but because of that fact it’s not uncommon to see references used heavily to avoid copying larger objects, even in desktop code.

1 Like