I’m frequently asked how to select the correct propeller for a given power system + airframe. This was a very generalised response I gave on one occasion which gives a good simplified explanation of the problem:
The ratio between diameter and pitch should be determined by the drag of your plane at its desired cruise speed (or whatever speed you want to optimise for). You want the smallest diameter and the largest pitch you can get, while still keeping the prop efficient, which typically is around the point where the props pitch speed (prop rpm*pitch) is 1.25 * the planes airspeed.
If your prop diameter to pitch ratio is any larger you'll be spinning more blade area through the air than you need to, any lower and the prop will be ‘stalling’ in the air and falling off its efficiency curve. However, you may need a larger diameter to pitch ratio than this to ensure that you have enough thrust for takeoff, for example.
Once you have worked out the diameter to pitch ratio you need, then you need to use the biggest propeller you can find with that ratio, that won't cause the tips of the props to come with 10% of the speed of sound, and fits the rest of your airframe constraints, the main one being that as the size of the prop increases the RPM of the motor will need to decrease, requiring a larger motor to output the same power.
All of this is a balancing act between efficiency, performance, payload capacity, practicality etc and lots of information and time is required to optimise all of these factors
Actually using this information to inform propeller choice is still quite dificult however…
A good starting point is making the plane approximately the weight you'll want to be flying it at, fly it with an airspeed sensor, RPM sensor and an APC propeller, then fly at your desired speed, take that data and look at the APC propeller performance data charts for your prop, and from that work out approximately how much thrust (=drag) your propeller was putting out at the RPM and airspeed you recorded. From that you can go through their performance data for their other props and find a propeller that would be a good match, be practical, and require a motor that you can obtain, and fits in your weight budget