As far as I can tell, it's a marketing gimmick that's hardwired in. Once you buy/subscribe, it sits there staring at you, tempting you to buy more. But, hey, B&N is in the business of making money.
Right now, the way around it is to just choose to view Books.
How are you deleting these items?
To answer your question:
As long as I have downloaded a subscription or single issue purchase of a magazine to my library I can delete it by going to manage subscriptions, under the image of whatever magazine I subscribed to, and then clicking the delete option.
What then happens (or so it looks to me) is that my library retains for me the right to "re-download" the particular magazine issue sometime in the future. I believe this is why the cover/icon remains and I have (as an experiment) "re-downloaded" an old issue of a magazine simply by clicking that cover/icon. I have also then "re-deleted" it but of course the cover/icon still remains.
I know in the end this is merely a housekeeping problem but I guess it bothers me because I can easily add and delete books from my library (I do it all the time) but the mags they just keep going on and on.