Ah, yes, That all makes sense.

I thought initially when they (I forget who) said they all had some experience with some of the stones, they'd send Clint for the Mind Stone given his rather intimate association with it. But in the end, apart from Thor going to Asgard and Nebula into space for the purple one, it seems to have come down to a bit of a lottery - I mean, all Bruce has in common with the Time Stone is the colour.