Hey there @vidario1975! Welcome to the community, and thanks for sharing your question.
The library does work a little bit differently on Spotify, but hopefully you'll get the hang of it pretty quickly. Yes, you do need to follow an artist for them to appear in the artists section of your library. This section basically acts like a bookmark to your favorite artists' main pages, which can be a little tricky to get used to at first since it doesn't take you right to the albums/tracks you've saved. When you get to their page, look towards the top (right above the list of popular tracks), and you should see a link labeled "liked songs," which you can then tap to get to the content you've saved.
If you'd like to just view individual albums, you would use the "albums" filter in your library, and you can sort by "creator" if you'd like your list of albums sorted alphabetically by artist, or feel free to use any of the other sorting options that are to your liking. Alternatively, you can use the search bar at the top of the library, and type in the first few letters of an artist's name, and this should help you find any albums, playlists, etc., you've saved that are by that artist or include the artist's name, such as a playlist title.
Finally, if you'd like to focus on specific songs, you might have a look at your "liked songs" playlist, which contains every individual track you've saved; please note this will not include tracks from albums where you've only hearted the album itself and not specific tracks from that album. If you'll drag down, you should see a search box at the top that says "find in liked songs," and you could search for songs from particular artists in this way as well. There are a variety of ways to sort this playlist too.
Please note that albums and songs will show up in their respective categories of the library, even if you don't follow the artist, so it's not necessary to follow every artist whose songs you've added in other ways unless you specifically want that artist in your artists section.
All in all, Spotify does offer a few different ways to find and interact with your favorite content. The structure is definitely a little different than you might be expecting, but I hope this helps you out, and you enjoy listening to your music!
dinomightSpotify Star
Help others find this answer and click "Accept as Solution".
If you appreciate my answer, maybe give me a Like.
Note: I'm not a Spotify employee.