Taking playlist randomisation into my own hands – using Javascript to improve Spotify’s Broken Shuffle

Have you ever noticed that when you use Spotify’s shuffle feature, you always end up hearing the same songs over and over again? It can be incredibly frustrating to hear the same music all the time, especially when you’re trying to discover new tunes. This is a problem many Spotify users face and it’s something that needs to be addressed.

Spotify is my go-to music streaming platform. I rely on it for all my listening needs, be it at work, on my commute, or when exercising. Even while reading a book to wind down at night. I use it to match the mood and activity. It’s great that they have so many curated playlist options to choose from!

Streamings services can be incredibly frustrating due to their tendency to select more popular songs for their shuffle feature, leaving users with a fairly limited selection of tunes that can become repetitive quickly. This is something I encounter on a daily basis, as I listen to music for at least four hours every day and find that the same old songs keep playing over and over again.

Advertisement Begins

Advertisement End

The Problem Many People Face

There are some 1800 songs in my library, some of which I haven’t heard in months because the algorithm thinks they’re not a good fit for my current listening session. The size of my Spotify library provides enough music to provide 30 days of non-repeating listening (assuming 60 tracks a day which roughly matches 3 hours of daily listening).

There exists a service called Playlist Machinery which is a “web app that lets you build complex playlists by assembling simple components.”. This worked for a while however, the scheduling capability is limited to 100 repeats which means I’d have to log on every 100 days and set it up anew. This is obviously not a problem for the average user, however, I’m a bit lazy in the sense that I would never get around to actually logging on and resetting the schedule. Resulting in the whole service ceasing to work. (totally my fault and a non-issue for most, and this is why I included the solution in this blog post.)

Besides… as a home automation enthusiast and software engineer, the idea of being tied to a platform and its annoying limitation bothered me, because I knew I could rebuild the service relatively easily.

Design Goals

My plan is to leverage the NodeRED platform to implement a series of custom nodes that provide similar functionality to the aforementioned service. The following functional requirements have been identified:

• FR1: The playlist should be randomised with a random number algorithm (traditional shuffling)
• FR2: Artists to be separated in the playlist to avoid repeated songs from the same artist in a short period of time
• FR3: Shuffling must occur nightly to create a new Daily Mix
• FR4: The algorithm should maintain a cache of recently used songs to avoid repeating the same stuff while unplayed songs are available in the library.
• FR5: Once configured, automation should schedule automatically and run at the desired time of day

Playlist Shuffle != Mathematical Shuffle

Not only are computer shuffling algorithms never truly random, but the idea of shuffling in a music playlist context is also biased by peoples listening preferences. We don’t really want a random shuffle when it comes to music because it makes the listening experience less enjoyable. A computer algorithm is designed to follow a specific set of rules, which means we have to design an algorithm to meet the requirements listed above.

For example, with a bit of luck in a dice game, it is entirely possible to roll the same number 4 times in a row. Were a shuffling algorithm to generate a playlist in the same manner, we would find many artists repeated in a row.

Ironically, listening to the same artist for 20 minutes straight creates the impression of “improper” shuffling because our human brains can quickly identify artist clusters that the algorithm should have been “random” enough to avoid. It’s not a problem of randomness, but rather a gap in understanding what we really want to achieve. We have to extend the shuffling algorithm to give it the concept of a song-to-artist relationship and how to handle it.

Enter FR2 Artist Separation, where we move away from a mathematically correct shuffle and closer toward a playlist shuffle (what we really mean when we say “shuffle” in the context of music playlists).

Affiliate Content Start

Infrared Remote for Roku TV, 2 Pack

(210)

$9.99 (as of May 18, 2024 11:04 GMT +08:00 - More infoProduct prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on [relevant Amazon Site(s), as applicable] at the time of purchase will apply to the purchase of this product.)

Btootos Wireless Earbuds Bluetooth 5.3 Headphones Bass Stereo Ear Buds with Noise Cancelling Mic LED Display in Ear Earphones IP7 Waterproof 36H Playtime for Laptop Pad Phones Sport Workout Dark Black

(724)

$29.99 (as of May 18, 2024 11:04 GMT +08:00 - More infoProduct prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on [relevant Amazon Site(s), as applicable] at the time of purchase will apply to the purchase of this product.)

2 Pack-USB C Headphones for iPhone 15 Type C Earbuds Wired Earphones with Mic & Remote Control Noise Cancelling for iPhone 15 pro, iPad Pro, Galaxy S23/S22/S21/S20/Ultra Note 10/20, Pixel 7/6/6a/5/4

(1411)

$9.98 (as of May 18, 2024 11:04 GMT +08:00 - More infoProduct prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on [relevant Amazon Site(s), as applicable] at the time of purchase will apply to the purchase of this product.)

Affiliate Content End

Artist Separation Algorithm

A very interesting algorithmic challenge presented itself when it came to separating artists in a list of songs. The following section explains how the algorithm works. I wasn’t able to find much on this kind of algorithm on the internet (possibly because the subject is too niche or implementation in most cases is not that glamourous as to warrant writing a blog post about it).

Another way of framing the problem could be finding an algorithm for spatially separating elements in a list by some property. If you recognise this problem from other domains, please let me know in the comments and I’ll update the heading to give it a more generic name.

Develop an algorithm to spatially separate list elements by some distance based on the value of an object property

General description of “Artist Separation” in algorithmic terms

The algorithm receives an array of Spotify track records representing our input list. At this point, the list is randomly shuffled but clusters of artists may well appear in this list as we discussed before.

We define a ARTIST_SEPARATION distance of 4 songs, meaning in a subset of any 4 consecutive songs, the artists should be unique. This can be adjusted based on preference.

Advertisement Begins

Advertisement End

The main mechanism for separating artists is the affectionately named artistMap containing one record for each artist. Note that songs created by multiple artists are treated separately to ensure FR2 is satisfied for song collaborations.

We loop over the input list and begin populating the artistMap. New records are initialised with a numeric value that exceeds the ARTIST_SEPARATION value. This ensures that songs are added to the playlist later.

After initialisation, we begin processing the input and start separating artists. We pop a new track from the list and check if all the artists for that track have a corresponding value artistMap that is less than 4 (the minimum distance between artists as defined in ARTIST_SEPARATION.) See the function called trackArtistWasRepeated which encapsulated this logic and stores the return value in wasRepeated.

There are two scenarios we have to handle:

• Scenario 1: Track is not repeated and we can safely add it to the playlist
• Scenario 2: Track was played recently and we have to ignore it for now.

If all artists for the track match this condition, we can safely add the song to the output playlist.

At this point, we must reset the artistMap records to 0 to ensure songs by the same artist/s are ignored for 4 iterations until we have a sufficiently large artist gap. This has been implemented in resetTrackARtists.

Kitchen Multi-Timer Pro

Now you’re cooking

Multi Timer Pro is your ultimate meal prep companion, keeping track of multiple cooking times and making adjustments on the fly. Give it a try today and become a better home cook!

The end of each iteration involved bumping all values in artistMap up by 1. (See increase function).

A little more complexity is required to implement Scenario 2. We created a buffer to put aside tracks that need to be ignored temporarily. (see if(hasrepeated) statement).

1. Later, we maintain this buffer using a sequence of steps (given the buffer is non-empty):
2. Peek at the first element using Array.prototype.shift() (First in Last out Queue)
3. Check if we are safe to add this track to the playlist using the aforementioned trackArtistWasRepeated.
4. If we are safe, simply add it to the output playlist.
5. If the track’s artist was played recently, add it back to the buffer using Array.prototype.push().

Note: Using shift() and push() in this manner effectively rotates the array. This ensures we maintain the input order to the best of our ability while satisfying FR2.

We continue this process until we run out of songs in the input list. It is possible that there are some songs left in the buffer at the very end. This is because we cannot add them to playlist without breaking the artist separation distance. This is particularly prone to happen for short playlists with lots of repeating artists.

Artist Separation Source Code

The algorithm is shown below and implemented in JavaScript as a NodeRed custom node.

module.exports = function (RED) {

    function ShuffleNode(config) {
        RED.nodes.createNode(this, config);

        this.on('input', function (msg) {
            const ARTIST_SEPARATION = 4;

            const increase = (artistMapParam) => {
                Object.keys(artistMapParam).forEach(a => artistMapParam[a] = ++artistMapParam[a])
            }
            
            const printTrack = (track) => {
                return track.track.name + " - " +
                       track.track.artists.map(a => a.name);
            }
            
            let buffer = [];
            let artistMap = {};
            // Reverese input because we want to use pop()
            // rather than shift() for performance reasons.
            let inputList = msg.payload.reverse();
            let outputList = []
            
            const resetTrackARtists = (track) => {
                track.track.artists.forEach(artist => {
                    artistMap[artist.id] = 0;
                });
            }
            
            // Initialise artistMap
            inputList.map(track => {
                track.track.artists.map(artist => {
                    if (!Object.keys(artistMap).includes(artist.id)) {
                        artistMap[artist.id] = ARTIST_SEPARATION + 1;
                    }
                    
                })
            
            })

            const wasRepeated = (a) => {
                return artistMap[a.id] <= ARTIST_SEPARATION;
            }
            
            const trackArtistWasRepeated = (t) => {
                return t.track.artists.some(wasRepeated);
            }

            while (inputList.length > 0) {
                let currentTrack;
                let hasRepeated;
                do {
                    currentTrack = inputList.pop();
                    //console.log("Current track: ", printTrack(currentTrack));
                    
                    hasRepeated = trackArtistWasRepeated(currentTrack)
                    
                    if (hasRepeated) { // track was played recently - skip this one and move on to next
                        //console.log("\t\tArtist repeated. Adding to buffer.Skipping to next.")
                        buffer.push(currentTrack)
                        continue;
                    }
                                        
                    //console.log("\t\tAdding track to playlist.")
                    outputList.push(currentTrack);
            
                    increase(artistMap); // increase all by 1
                    resetTrackARtists(currentTrack);
                    
                    if (buffer.length > 0) {
                        
                        let peekBufferTrack = buffer.shift()
                        //console.log("\t\tMaintaining Buffer - Item: ", printTrack(peekBufferTrack))
                        if (!trackArtistWasRepeated(peekBufferTrack)) {
                            //console.log("\t\t\tAdding track to inputList (from buffer):", printTrack(peekBufferTrack))
                            inputList.push(peekBufferTrack)
                        } else {
                            //console.log("\t\t\tTrack not ready. Returning element to buffer.")
                            buffer.push(peekBufferTrack); // rotates the buffer
                        }
                            
                    }
                } while (hasRepeated && inputList.length > 0);
            
                
            }
            
            msg.payload = outputList;
            this.send(msg);
        });
       
    }
    RED.nodes.registerType("artist-separation", ShuffleNode);

};

You can view the Gihub Repository from the project page

NodeRED Spotify Automation

Set of NodeRED custom nodes for loading Spotify library items such as saved tracks and playlists, caching local copies, shuffling, sampling, filtering, artist separation and saving playlists back to Spotify.

The building blocks in NodeRED

	Node Name	Description
	shuffle	Shuffles the input in payload
	first	Filters the first N messages in payload
	playtlist – save	Saves Spotify tracks in payload to the specified playlist
	sample	Samples a specified number of inputs from payload
	append – context	Saves payload to context, appending to existing elements if available.
	use – context	Loads a list of tracks from global context
	filter – tracks	Filters tracks in payload against a list of tracks saved in named context
	artist – separation	Uses my custom algorithm to separate artists by some distance. Fails fast
	spotify	Spotify API node included in this library for convenience. Can be used to download Saved Tracks and paylist tracks.

Step sequence to generate final Daily Real Shuffle playlist

1. GetTracks: Download My Saved Tracks using spotify node and store them in context mySavedTracks using the append - context node.
2. Saved Tracks: Use the use -context node to load the cached list of 1800 songs in my library from Node Red context.
3. Filter out recentlyShuffled: Remove songs that have been recently included in the shuffle playlist by filtering out tracks in context list recentlyShuffled (more on this context list later)
4. Filter out myTopTracks: Similarly, filter out my top tracks to avoid including tracks that I listen to regularly (outside of the Real Daily Shuffle Playlist)
5. If running out of songs: I included a condition here to reset the recentlyShuffled list if the set of available songs is insufficient to create a decent playlist (of about 60 songs). In this case, the recentlyShuffled context is reset to an empty list.
6. Sample 60: In the happy case (we have not run out of songs), we simply randomly sample 60 tracks from the available tracks in payload.
7. Shuffle: Randomly shuffle those tracks
8. Separate Artists: Separate artists to ensure each song in the sequence is by a unique artist. We don’t want to listen to 4 Lady Gaga & Tony Bennett songs in a row (or maybe we do).
9. Daily Real Shuffle: Once this is done, we have our final list of songs that is ready to be saved into our Daily Real Shuffle playlist using the custom playlist - save node.
10. Add to song buffer: Note that we are using the custom node append - context here to add the same songs to our recentlyShuffled list, such that they will not be included in future playlists – until we run out songs.

This NodeRed flow creates completely unique playlists for me every morning at 2 am, meaning for about 30 days I can listen to uninterrupted music without repeating tracks. It is am amazing piece of code and I am looking forward everyday to listen to this go-to playlist. It is so easy to just open Spotify and hit play on “Daily Real Shuffle”.

Featured Content Start

Products for sale on my Etsy Store

Additional card sets are available on my store already. I am planning to create new flashcards in the future to expand into other areas of music theory and piano practice. Check my store for all available learning materials

• Rhythm training (200 flashcards!) – Etsy Product Link
• Major scales – Etsy Product Link
• Minor scales – Etsy Product Link
• Arpeggio drills – Coming Soon
• Cadences – Coming Soon
• Chord progression & improvisation – Coming Soon
• Jazz & Blues Scales – Coming Soon

Get 10% off your order by using the following link: Use code BLOGVISITOR10

Featured Content End

Advertisement Begins

Advertisement End

Conclusion

Spotify’s shuffle feature keeps playing the same songs over and over again and I got tired of it. In this post, I show how to implement a custom shuffling algorithm in Node RED to generate a custom Daily Real Shuffle playlist that updates everyday with songs in your library that you haven’t heard in a while!