Now and Then
Looking back at 100 albums of Now
This year Now That’s What I Call Music celebrates the huge milestone of 100 albums. Since 1983 these “mix-tape” albums has been a staple of British music culture and the formula behind them still feels remarkably simple — yet it works: take some of the chart toppers from the last few months, add a few tracks that people may have missed, and then spice it up with something alternative or different at the end.
In reality, the “best” chart music depends more on artists giving their permission for the song to be included on a Now album. Robbie Williams seemingly can’t say no and has had the most Now tracks with an unbelievable 30 tracks, Madonna meanwhile has never agreed.
Depending on how old you are will determine how much you care for this music industry milestone. For some it’s a nostalgic reminder of when music was better — for them music got worse after Britpop died. For others it remains an easy way to keep in touch with the current musical bangers. For others, namely the younger generation it’s hard to fathom the concept; how in the age of Spotify and Apple Music, digital downloads and millions of curated playlists, can a CD compilation album of current music have a market?
And yet it has a market. Now 100 will be the biggest selling album of the year. The Now TWICM series has sold over 120 million albums since 1983.
For the Databeats team the Now albums have memories of being the go-to present of aunts and uncles. Rob remembers getting ‘Now 4’ for Christmas (now worth £650 on vinyl!) and enjoying Dancing with Tears in my Eyes by Ultravox too much. For Chris it’s a Sony Walkman for a Birthday, a double cassette of Now 13 and ‘A Little Respect’ by Erasure.
This is a data-driven article though, and so we shouldn’t be wallowing in nostalgia…100 albums means there’s a lot of data available. Using Spotify’s amazing API (Now That’s What I Call Data!) we can delve into the audio features of tracks to learn about their danceability, energy, bpm, and much more and thus explore the history of the Now series.
As always with our Databeats articles the starting point of this article was a discussion that needed some data: while Now albums always seemed commercial and popular, as a teenager they also seemed to be a bit eclectic. For example, The Smiths What Difference Does It Make was featured on Now 2, while today’s Now albums seem to feature more pop records. So the question we mulled over was, is this still true? Has the Now album changed over the years? Given it reflects, or perhaps informs, the tastes of a certain target audience, then can we use Now as a proxy to infer changes in music preferences across the decades? And if so what can we learn by doing so?
Are recent Now albums really peddling a rather bland, homogenised musical landscape? And if so how much blame should Ed Sheeran take for this?
A quick note about the data. While the data from Spotify is amazing, not many of the Now albums are available on Spotify. So we’ve had to use user playlists to get the data. Some tracks are missing from the playlists, some songs are not available on Spotify. As there was no album data, we then had to join the data back to an official track listing from Now. To do this Chris did a fancy fuzzy join in Alteryx, explained below. No data is perfect, this is certainly less than perfect.
Let’s start with a look at the variables and how they’ve changed over the years…
Pop Music. Everyone Talking about Pop Music.
As we see below pop has become the dominant genre to the Now album over the decades, typified by Taylor Swift’s move from country to pop in 2014. But calling it just pop is doing pop a disservice, as it’s no longer just the preserve of just 3 minute songs and a catchy tune.
Pop is diverse, complex and has a vast array of sub-genres. Take Now 100 with Ed Sheran’s Sing and Spice Girls Wannabe. Both pop, according to Spotify, but hardly the same.
Rock is stuck with pensioners doing greatest hit tours rather than making contemporary records suitable for Now albums. Specific genres like Britpop come and quickly go, but the musical world always seems to need boy bands….
Don’t Worry, Be Happy?
Valence describes the musical positiveness conveyed by a track. The higher the value, out of 100, the more positive the mood of the song.
Now Albums have always been advertised as being happy and shiny. The data shows the 80's were the high point of Now happiness, but since then it has been a gradual decline. Many recent albums have starting dropping below 50, tipping them into being negative — the Now 100 spike being an obvious outlier because of the inclusion of ‘classic’ tracks.
Music characteristics can be assigned to what is known as the human emotional model of affect. (‘A circumplex model of affect,’ Russell 1980) This model arranges 28 human emotions according to the two dimensions, arousal and pleasure, in circular order. This model allows us to classify music as shown in the quadrant below.
If we add in the Energy of the track we can look at tracks which are negative and energetic — i.e. angry, and those which are positive but gentle — calming. On the whole we see Now albums have become less happy, and in these angry times, have become even slightly angry.
We need to be careful though, averaging by album hides a lot of the diversity of tracks within each album. For example, there are a lot of angry tracks, the angriest being Stamp! by Healy And Amos. There are fewer sad ones, the saddest being, Say Something by A Great Big World & Christina Aguilera, and a lot less Calm tracks like Broken Stones by Paul Weller.
Start Me Up
Popularity is measured out of a 100 — with the higher the number, the higher the popularity. Lots of ‘Now’ tracks have zero popularity. Some deservedly so, Garry Glitter’s Dance Me Up, but some are more puzzling, for example The Rolling Stones Undercover (Of The Night).
The reason for this is likely to be the fault of our data gathering. Spotify has multiple instances of each song across many compilation albums and “Best of” albums, etc (Undercover features on the album of the same name as well as Jump Back — The Best of). Each version has its own separate stream and so each gains a different popularity on Spotify. i.e. popularity doesn’t measure the popularity of the track overall, just the popularity of that instance of the track. We can’t guarantee which instance we pick in our matching and so in using popularity we have to assume such data issues average out on aggregation across the data. We won’t analyse individual tracks popularity.
The main (surely only?) task for the producers of the Now Albums is to decide the songs they add and their running order. The blueprint is simple. Put the bangers up front at the start of CD1 and CD2, and leave a few for the end so people will listen all the way through. While we can’t track this across years we can assume the formula remains unchanged.
Cum on Feel the Noize
As most parents have probably noticed music is getting louder, this is something that started from at least Elvis Presley and Buddy Holly in the 1950s. (See — ‘Pop music is louder, less acoustic and more energetic than in the 1950s’ Guardian) Now albums are keeping with that trend by also getting louder, although a little more quieter recently than at its peak.
Time after time
Since Now started the average song length has shortened by a minute from a peak of 4:31 to 3:24, perhaps signifying the move away from longer rock anthems. There are lots of theories as to why and we don’t want to repeat them here, but many focus around the attention spans of modern generations artists “gaming” the Spotify streaming system, either killing the modern intro to get around 30 second demo limits or adding more, shorter, songs to albums to push them higher in the charts.
The Beat Goes On
Previous analysis of hits on the American Billboard charts has shown that 120 BPM is the optimum tempo. So it’s no surprise that a hit based album should hover around that mark. In comparison Dubstep is mostly 80–90, Hip Hop is around 80–115 BPM and Drum and Bass averages a BPM of 160–180.
I Wanna Dance with Somebody
Unsurprisingly, Now albums are full of energy and danceability. Danceabilty has hovered around the same mark for the entire period. There has been more of a change with Energy, and occasionally energy trumps danceabilty in a way that is not completely clear.
Both measures have small fluctuations. One reason is that research, admittedly in Germany, has shown how during warmer months, and assuming there is some in a British summer, the charts are filled with happier, active and energetic songs with a higher danceability. (see ‘Summer hot, Winter not!’ Helmholz et al 2017)
Back to our preface though, are albums and songs more homogeneous now?
To analyse this we need to look at the spread of the variables we’ve looked at so far across each album. If tracks were becoming more similar and less eclectic as the Now series progresses we’d expect to see that spread, i.e. the standard deviation, reduce.
The charts above show the standard deviation of the standardised versions of each variable (so they’re comparable), if the line goes down the tracks are getting more similar (less eclectic) with respect to that variable. Likewise if the line goes up then there’s more of a spread.
What’s perhaps interesting to note is that while BPM and Energy show a slight trend towards getting more varied as time goes by it’s the spikes in the mid 90’s that stand out — showing just how eclectic that period was compared to now.
The other variables are all seeing a trend towards tracks becoming more similar as time progresses. Certainly they’re all settling on much more similar lengths and Loudness. Less pronounced is the trend towards more similar Dancability and Valence.
So are our tracks becoming more homogeneous? Beyond a conformity to a similar length of song, there doesn’t seem to be enough evidence to say for definite they are.
Conclusion
So what have we learnt? In the Now series — which represents perhaps the best barometer of “mainstream” musical culture we have — then Pop is more prevalent than ever, songs are getting shorter and louder and even a bit angrier. Furthermore, despite our preconceptions, we’re not really seeing the series becoming less eclectic.
This presents somewhat of a dichotomy though; we’ve already noted that more tracks are labelled as pop than ever before and yet the albums themselves are as eclectic as ever (at least from a data perspective). So what we’re really seeing is a widening of the pop genre, artists like Florence and the Machine and Twenty One Pilots are considered now pop whereas previous decades would see bands like these firmly in other genres.
So it’s clear that pop is bigger and more eclectic than ever, which suits the record labels as they can continue to define and redefine the genre and push “different” artists such as Ed Sheeran on a mainstream audience without seeing them labelled as “indie or alternative”.
Is this good for music? Do genre labels really matter — particular for mainstream music? Perhaps we’ll leave these questions for more expert commentators.
Non-music listeners might argue music isn’t what it used to be, and they are right it isn’t — it’s better. It’s more diverse and more fun (at least if you escape the Now album). Although when it comes to Ed Sheeran, the Databeats team draw the line.
Methodology
Producing a Databeats article isn’t easy, to produce the 2000 words and handful of charts in an article takes a lot of effort that is often hidden behind the scenes. In this section we talk through some of those hidden details — if you’re not interested, and we don’t blame you, then move on but if you’re keen to see behind the scenes then read on.
The initial idea for the article led to a trawl of data sources on Google, hunting for the Now track listing was relatively straightforward as we quickly found https://www.nowmusic.com/ — listing every single Now album.
After Webscraping the albums using Octoparse (much easier than Parsehub) we then had to match them to the actual Spotify data via the API. The API work was harder, especially for two non-coders, R and python scripts were tried but eventually we resorted to using webtools. We used this playlist by GeordieLee, alongside sortyourmusic.playlistmachinery.com (for the data points) and later organizeyourmusic.playlistmachinery.com (for the Genre) — the latter only displaying the top 400 tracks and so we had to use Google WebDev tools to find the underlying json and extract it before converting in Alteryx.
The next challenge was that, sadly the tracks on the playlist didn’t match to the Now listings on nowmusic.com (mainly due to subtle differences in the artist and title spellings) and so we had to perform some fuzzy matching to attach the Now album and track order. Here we used Alteryx to compare the track names and titles to the track list (after stripping the words such as 9" version, etc at the end of tracks). Alteryx compares the words and does a brute match, then examines the number of changes to get from one title to another in order to produce a “match score”. We filtered these at a suitable threshold and then examined manually in some cases to check for false positive matches.
From there, as usual, we produced a number of Tableau charts, exploring the data and finding stories. It was at a late stage we realised we needed Genre and so revisited the data via organizeyourmusic.playlistmachinery.com to add to the story.
Thanks for reading — we’d love to hear your comments below.
