It's probably a question of what you have it set to show you.
If you have "all" vs "downloads" on the left, of course it will show even if you have removed it from the device, because "all" shows what's also in your Amazon Library.
As to putting it into a collection, if you don't want to see it on the home page, you have to have it set to "collections" somehow. Note, however, that there are two sort/show options on the right side. If you choose "collections" in the left one, that's all you'll see. It will not show ANY uncollected books.
If, like me, you want to pull a book out of a collection when you're reading it and put it back when you're done, you want to set that one to "all items" and then set the right one to "collection". This way, when in list view, the first thing you see is always your collections; behind those you'll see any uncollected books or other content on your device.
I believe it works similarly in Grid View, but I don't use that because the collection icons are boring.
Also note that if you're using the new 'home screen' view, the first page will always show one large and two small thumbnails representing the most recent things opened or delivered to the device. The list of books to the right and thumbnails on the bottom change -- sometimes showing your Wishlist items, sometimes things you have already in your library but not on your device. The bottom suggestions sometimes come from your friends on GoodReads, your Wishlist or Library, and also sometimes based on one of the three thumbnails. Regardless, you can get to the list or grid views by swiping to the left.