Is there a way to see only the music I have downloaded in the iPhone app?

When I click "my music" I get a huge stream of music that includes everything I've listened to. I just want to see what's on my phone, and I want it organized in some coherent fashion, like by artist, then album, then song. Right now I can only choose one of those categories, and it's insanely annyoing. 

For the record, I'm only on Spotify because there's not an alternative. I was with Rdio (RIP) since 2011, but it's gone now. So far Spotify's app is driving me crazy, but I'm a captive subsriber...alas. 

Is there any hope that the app will make some sense soon for those of us that want to use it as a music library ala itunes?

Thanks,

KC