Try this (I know, it will seem like it takes "forever") fire 1 shot, wait 5 min.
Then fire another. Repeat this 4-5 times and see IF your spaced out shots shoot to the SAME point (close enough for your satisfaction).
Then fire your 5-10 shots at the interval you were before.
This is just a "check" to see if your shots are really walking or something else is causing them to move in between shots.
My thinking is that if it's heat related, you'll see it "walk" when you fire them in rapid succession but NOT when there's a long time interval in between.
Other things, like something lose, a scope mount, or faulty scope, or Action Screws, especially on a bigger gun with some recoil could cause this.
But on a .17 rimfire, that's a little bit more of a "Hmm"?