It's only ever happened to one team, Milwaukee in 2007 I think. So fingers crossed! It is unlikely.
As the famous Sir Alex Ferguson once said, "It's getting tickly now, squeaky bum time I call it"
It's hard for #3 to move back to #6, because that means all the top 3 picks went to teams outside of the top 3. While it's likely somebody gets jumped, it's rarer that everyone in the top 3 gets jumped. Moving back 3 spaces regardless sucks for your team. Moving back 1 space sucks.
Some teams that moved back the max 3 spaces:
'09, Kings and Wizards finished 1st and 2nd worst, drafted 4th and 5th.
'07, Grizzlies, Celtcis, Bucks finished 1st, 2nd, and 3rd worst, ended up drafting 4th, 5th, and 6th.
'06 Blazers with the worst record drafted 4th, but amazingly ended up with the best player in the draft, LaMarcus Aldridge by trading #4 (Tyrus Thomas) and Viktor Khryapa. Then traded just #7 (Randy Foye) for #6 to get Brandon Roy. There were only 2 perennial All-Stars/All-NBA players in the top 20, and Portland got both of them despite the worst case scenario in the lottery and by giving up basically nothing to move up. Kevin Pritchard should have won Exec of the Year!