For selected window have (set-)window-point always operate on buffer's point.