Well, my experience is apparently 180 degrees from y'alls!
I used the 4lb balls with the Scotty 1060's for several years and they are the perfect size for a manual downrigger. They can also be bought in the "sinker" dept. instead of the downrigger dept.- a LOT cheaper!
I've fished them at 120 ft successfully- yes there is blowback-no more than 10fts difference at 120 ft. They are tedious to bring up from that depth but not the least bit of struggle with the weight- just turn the handle. I can actually bring the manual up faster than the electrics I now have can do. Of course, not near as conveniently.
20-50ft is utterly simple to fish with these and very effective. I now have them rigged to set right on the seats of the 14ft'er bungeed in place- I wouldn't go fishing without them. In fact I trolled with the electric motor in a lake on Saturday- fishing 18-22 ft. down on the downriggers with silver flatfish- hammered the trout!(21 fish) My buddy stubbornly fished on the surface- he caught 4!
I do have electrics now on my 16ft boat- it's quite the luxury- but I am never bummed about having to use the manuals in the small boat- they are awesome!
I have tried the 8 and 10lb balls on the manuals- I agree that I would NOT be using a manual for long if I HAD to use those weights! It's just not practical to try and lift those.
I would also advise getting the 2ft./turn spool like the model 1060 has-you will not like the little 1ft/turn spool- THAT is too much work, with any weight. 2ft per turn can get you up from 50ft. in 10 seconds.

Jim