Xbox Has a Laggy Media Streaming Problem, but Microsoft Is Working On It
Restarting is almost always the solution.
Xbox is looking into some reports of media apps slowing down after they've been used for a long time. A few users are experiencing this problem, which shows up as slow performance in apps after they've been streaming or watching media for a while.