In informal settings, most former presidents are referred to as ‘President [whatever]’. It’s only in formal gatherings where it’s considered appropriate. Most view it as a courtesy. Though, it should be noted, all of one ex-president takes offense to being called “Former President”; and he’s the same dumbass that most everyone outside of his circles no longer allow that courtesy.