See, because if Bean and Fuzzy are also in the loop, they still have free will. It's just everything around them that would be repeating. Potato understands his fourth dimensional temporal dynamics...
Although I don't know why he assumes they're in it with him...