If you would use recarchables you could use 3 and could get away with not using a voltage regulator. With normal batteries you’d need 4 cells with a big display. Reason for this is that you need an higher nominal voltage as normal batteries have a high internal resistance causing the voltage to drop considerably when high currents are drawn from them. (big OLED display).
linear voltage regulators are the most simplest. They reduce the voltage simply by waisting the excess power into heat.
Besides boost (step up) (DC-DC) converters you also have step down (DC-DC) converters these are more efficient then linair voltage regulators so batteries would last longer.
If you like the simplicity of a linair voltage regulator (single part) and the efficiency of a DC-DC converter but don’t want all those parts. Then there are some nice DC-DC converters in a single package. Have a look here