Per the FAQ on Keane's website, Keane's drummer Richard Hughes, stated the following:
"We've been asked whether "Somewhere Only We Know" is about a specific place, and Tim has been saying that, for him, or us as individuals, it might be about a geographical space, or a feeling; it can mean something individual to each person, and they can interpret it to a memory of theirs... It's perhaps more of a theme rather than a specific message... Feelings that may be universal, without necessarily being totally specific to us, or a place, or a time..."
With the nostalgic sentiment and the overall tone of the song, I think Keane is attempting to express a Portuguese term known as 'saudade', which does not have a direct English translation but roughly means "that which we remember because it is gone."
@xaratherus The word would be nostalgia
"Saudade" is more like a wistful or melancholic longing for something you can never reach, which may not even exist in the first place -- like an imagined past that never was, or an alternate present or future that never will be; it's the shape of "something missing" that can never be filled.