"...What if you just didn't tell anyone about those other adventures at the time?" He's going somewhere with this, he promises.

"...If...if you're worried about breaking someone's memories...then hearing that you didn't say anything in the future would guarantee you make sure not to say it- right? What if the entire time...it's been 'you', you were just keeping a secret to keep them safe until the time was right."