Sometimes it's surprising to see the limits of user testing.
There are times when I've done user tests with users at different stages of a design from paper prototyping to high fidelity. One thing I've learned is that these tests are great for helping you to challenge your biases and see things from your user's perspective.
However, they seem to be less useful for filling in the blanks on smaller but critical details. The types of details that only an expert can discern and point out long before a user complains about it once the product has gone live.
Sometimes a good ol' discussion with your fellow team of designers and walking through several rounds of design reviews will take an experience/interface to the finish line.