Brutalism is terribly misunderstood if you see it as an attempt to look ugly.
It is an
attempt to look monumental.
It's really not all that different from the look of old fortresses meant to impress; simple, bulky, and large. Similar, for that matter, to the Pyramids.
They play with the apparent scale in order to convey a certain kind of grandeur. Not the fussy rococo kind of grandeur; it's more just the sheer scale of them that imparts a sense of inevitability or permanence, scale on the magnitude of geological features. It's supposed to be a bit overwhelming and a bit humbling. Plain and impressive and in a way very organic once you realize the geological inspiration.
Given the era in which brutalism emerged, it's almost even touchy-feely-earth-mothery (...if you're used to looking at volcanos or salt plains or sheer cliffs or endless dunes, when you think of nature. You'd miss the point entirely if you spend all your days looking at snowflakes and orchids and quaint mountain creeks in moss-filled forests.)
A masterpiece of scale and function, shown at
http://www.andreas-praefcke.de/carthalia/world/cdn_ottawa_national.htm.
And great for public buildings. In the case of the National Arts Centre in particular, it pays tribute to the geographical "pingo & polygon" landforms of Canada's north.
I tend to agree with how alienating it can be to live in a brutalist building, but then who wants to live in a monument? The pyramids, a fortress, all rather stern places for daily life.