Maybe this is not caused by the object cache but by nginx microcaching?
More rarely changing pages are held in cache for some requests, although they may need rerendering.
Pages that are individualized or often-changing (as WooCommerce cart, check-out, account pages) should be excluded from microcaching:
Do you have the same issue when you disable the object cache?