I think there are a number of reasons for this. The change really started in the beginning of the twentieth century, with the Modernists. Especially with the World Wars, they came to see the world as a difficult, absurd, and incomprehensible. So, the thinking was, why shouldn't their poetry reflect that? Poems stopped being written for a general audience, and so the general audience stopped listening. Contemporary poets have inherited this perception.
First of all, though, poetry is inherently more difficult to understand than narrative. It's metaphor and symbol-laden, the language is more compact than straight narrative and poems are often about things that are difficult, if not impossible, to articulate otherwise. Poems need to be read slowly, and I think when a modern reader sees something relatively short in length, he/she assumes it needs to be read quickly. That's not the case. Most readers also come to the page thinking they're going to get a narrative, when sometimes poetry is more about sound and wordplay, the same way that not all song lyrics "make sense."
Second, some poets really don't care about writing for the other 92% of the population. They're perfectly content with that. I had a poem published (in a reputable mag) that I don't expect anyone to "get" in the same way they would a narrative. But it was fun to write, and fun to read in the way that it plays with language, and I make no apologies.
That being said, there are many contemporary poets whose work is very accessible. Billy Collins and Tony Hoagland come immediately to mind. Steve Scafidi's "To Whoever Set My Truck on Fire" (http://fishousepoems.org/archives/steve_scafidi/to_whoever_set_my_truck_on_fire.shtml) and Nick Lantz's "The Year We Blew Up the Whale--Florence, Oregon" (http://howapoemhappens.blogspot.com/2010/09/nick-lantz.html) are great examples of unpretentious contemporary poetry.
As an English teacher, I can assure you that we teach tons of poetry in school. Unfortunately, most of it what's taught was published before my student's parents were born, and they have trouble connecting with it. I make it a point to use contemporary poetry in the classroom, and wish more teachers would.
But finally, I don't think it's fair to expect poetry to stay the same. "The Highwayman" was published in 1906, "The Raven" in 1845. They're great poems, but to criticize modern poetry for not sounding like them is kind of like expecting modern pop to sound like ragtime.