I think "That's all" refers not only to the indifference promoted in the song, but also the song itself, i.e. the structure ("That's all of the song"). It's very simple, especially for Morrissey. Many of his verses tend to be short, and when he repeats a whole verse he tends to change it very minimally (see: "On the Streets I Ran"). The indifference introduced by the lyrics extends to the song's structure, giving "That's all" I dual role.
I think "That's all" refers not only to the indifference promoted in the song, but also the song itself, i.e. the structure ("That's all of the song"). It's very simple, especially for Morrissey. Many of his verses tend to be short, and when he repeats a whole verse he tends to change it very minimally (see: "On the Streets I Ran"). The indifference introduced by the lyrics extends to the song's structure, giving "That's all" I dual role.