You really need to find out how many RPMs you're supposed to be turning.
Until you do,its all a guessing game.
If I had to guess,I'd say you'll need to be up near 4000-4500 or so.
If thats the case something is wrong,other than the prop. Maybe the tach isn't working right? Maybe the boat is waterlogged and weighs way more than it should?
For every inch of pitch you gain/lose your RPMS change by about 200.
On a well running boat a change from a 17" pitch to a 15 would increase the RPMS by about 400. When going to more pitch,such as a 17" to a 19",you'd lose about 400.
If you have a 17" pitch, and can only turn 3000 RPMS,you'll never find the right prop. To gain that many RPMS you'd have to go to a 12" pitch and that just aint right on a boat that big.