He completely forgot about the hospital and that she's concerned about his memories, but he knows it's important to her. Probably more important than his desire to go home and heal her...and then hide out there for a while.

They can hide out more after the hospital visit. "Oh, right... Right, we should head there first..." He gives her hand a little squeeze and starts towards the door.