On Repeat
Are artists trotting out the same old set lists gig after gig?
Have you ever considered how much thought a band gives to the songs they play in their live sets? Perhaps you’ve noticed there are a couple of big favourites to get the crowd going, before a lull for new material — or maybe even the dreaded acoustic bit — before the real belters come out for the finale and encore.
But how much does their “set list” vary for each venue they play on a tour? As their new material dries up and they rely on the old favourites, do they stick to the same old formula, or do they mix it up night after night to stop themselves getting in a rut? And what about those myths — have the Grateful Dead really never played the same set twice? Was Billy Joel right when he said he parted company with Elton John because “I got tired of doing the same show over and over again”? And do REM really never play Shiny Happy People live?
Thankfully setlist.fm have been crowd-sourcing set lists stretching back decades, so we have some data we can use to start answering these sorts of questions. We took a look at over 200 of the most popular artists on setlist.fm (~180,000 sets) and analysed how their sets have changed over time, mathematically comparing each set with the previous one to determine a “similarity” measure between consecutive gigs.
Our Consecutive Set Similarity (CSS) measure scores sets as a 1 if they are identical to the previous set, and 0 if they are totally different. Gigs that reverse the set list order of the previous gig would score -1.
[Incidentally it appears it’s very rare for bands to reverse their set lists — the only incident we could find was this anecdotal set list from the Pixies which was apparently a reverse of the previous nights set, with the band walking off after the encore (the first song).]
Remember, as we discuss this CSS measure, if it averages close to 1 for an artist, it doesn’t necessarily mean all a bands set lists are similar, but rather that a lot of their set lists were similar to the previous gigs set list.
Let us look at the bands that score highly by their average CSS measure first. These are the bands that are consistently playing the same sets night after night. Iron Maiden score highest, having an average CSS measure of 0.94 — about 80% of their 2,188 gigs recorded on setlist.fm were identical to their previous set.
The bottom of the list gives us the bands that are continually refreshing their set list, adding new songs and refreshing the order. Umprey’s McGee weren’t familiar to us but they certainly meet their Wikipedia description ”…the band’s approach shares many elements with groups like Phish and the Grateful Dead such as varying set lists, improvisation, playing two sets per night…”.
We also see the Grateful Dead appearing second to bottom, perhaps as expected given the myth that they have never played the same set twice.
Let’s tackle that Grateful Dead myth first then. Using data pulled from setlist.fm we’ve found no fewer than five instances where the band played the same set list. While some of these might be due to data collection issues (some are on consecutive days in the same location) it seems likely we can count this myth BUSTED: the gig they played on 8th July 1970 at Mississippi River Festival 1970 matches the one played on 16th January 1970 at Springer’s Inn, Portland, OR, USA.
While we’re tackling myths then let’s look at that Billy Joel one, was he right to say that he “got tired of doing the same show over and over again” with Elton John? If we compare the gigs the two played together (the blue highlight below) to two individuals (in orange) we can see Billy certainly had a point — the two score very highly on our set similarity measure (the vertical axis).
The plot below shows the average CSS measure over an artists life time by their total number of sets on setlist.fm. The size of their circle shows how long the average set is.
So it’s fair to say from the data that Billy has a point — so perhaps the myth is PLAUSIBLE.
Finally, what about REM — do they really never play Shiny, Happy, People live? Well it turns out, at least according to setlist.fm that that myth is VERIFIED. The only two gigs listed with that song were only two tracks long and for TV studios — so we excluded them from our analysis.
What have we learnt about individual artists from our analysis? Well let’s just pick out a couple….
Bob Dylan is the most prolific artist in our analysis — and as you can see from the above chart his gigs started to get a bit more repetitive between the 60's and 80’s. However, as he ramped up the number of live shows, he kept them interesting — maybe for himself as much as the punters. From 2013 onwards though, he’s more or less been playing the same gig.
Over the same period The Rolling Stones haven’t been nearly as prolific, offering up slightly more than a third of Dylan’s live performances. Their “Licks” World Tour in 2002 marked the zenith of their diversity on the road, a big departure from the uniformity of their 1982 “Tattoo You” European Tour (they scored a massive 0.97 in that year).
We can visually compare the set lists on tours to see what this difference looks like. The Tattoo You tour had a very fixed set list as the data indicates:
The difference to the Lick’s Tour being immediately obvious:
You can explore different artists in the interactive career tracker here.
Taking this comparison between artists further we can group them into groups; based on the diversity of their gigs, the number of sets they’ve played and the length of their average set.
The bands from our analysis in each “cluster” are listed below. We’ve provided rough descriptions to aid understanding the differences between each one:
Cluster 1 — Artists with relatively short gigs, which have little variety between sets
Cluster 2 — Artists who’ve played a large number of gigs but do little to vary the set between each
Cluster 3 — Artists who play longer gigs, who’ve played a decent number of gigs of their lifetime with an average amount of diversity between each gig
Cluster 4 — Artists who’ve played a relatively low number of shows compared to others we analysed (<800), they have short sets and keep things reasonably diverse
Cluster 5 — Artists who’ve played a huge number of gigs and like to mix things up between gigs
Cluster 6 — Artists who have a diverse set list from gig to gig but have played a relatively low number of gigs
Cluster 7 — Artists who play long sets, with little diversity, who have played a low number of gigs
In conclusion it’s clear that there’s a big difference between the effort artists put into a tour, or set of gigs. With a few exceptions those bands who have been around longest and played the most gigs are typically those that churn out the same set time after time, perhaps because of the effort it requires to constantly mix up a set and keep the audience happy.
We’ll end on this thought: if you’ve ever been to a gig and wondered if the band get bored playing a set they’d done many times before, then spare a thought for The National during their performance at the MOMA PS1 VW Dome. The set list is here, and is shown in part below. The band played for 6 hours as part of an art installation with the artist Ragnar Kjartansson, playing the song Sorrow 108 times in a set that provided six encores, one an hour!
Methodology
The set lists were downloaded from setlist.fm using Alteryx to work through the top artists for each letter of the alphabet — while this wasn’t the most scientific method of finding artists it led to a diverse list, which was our aim. Each page was pulled in before the individual set lists were extracted into a data table.
Once the set list data was download we used a Kendall Tau correlation, again built in Alteryx, to compare each list to the next. This blog post from Ritesh Agrawal helping with the methodology. Any songs which didn’t match between lists were added to the end of each set, a common method of dealing with this “dis-jointedness” problem.
Analysis was then performed in Tableau, iterations of visualisations were performed over several subjects before finding the subjects we featured in the article.
Acknowledgements Thanks to setlist.fm for the data, and also to Louis Archer for sub-editing.
PS. if you’ve made it this far then we hope you enjoyed the article, let us know by giving us a “clap” (or several).
