Spotify approved a patent to record user speech: Do you need to worry?

I’m sure many of you are familiar with this phenomenon: you’re with your friends, talking about the latest episode of RuPaul’s Drag Race, and three hours later you’re met with an ad for the new Australian season of the show. Whether or not the conspiracy theories that our smartphones are always listening and watching us are true, it’s nothing shy of creepy that “coincidences” like these keep happening. The last thing we want is for our music streaming services to surveil us, but Spotify might be the conductor of this train.

Editor’s note: This article was updated on August 12, 2022, to update formatting and provide the most recent information about this topic.

What is Spotify’s patent all about?

A Google Pixel 3 on a comforter next to the Sony WF-1000XM3 true wireless earbuds and charging case.

The new technology, if implemented, would collect and store recordings of users’ conversations.

In February 2018, Spotify submitted an “Identification of taste attributes from an audio signal” patent. Plainly, this technology may analyze Spotify users’ speech and background noise to assess the speaker’s age, accent, gender, and even mood to recommend music. It also claims to be able to detect the speaker’s physical environment (bus, train, or coffee shop), and approximate the number of people in the surrounding area. This patent was then approved on January 12, 2021.

This technology would use your device’s microphone to record and analyze audio to determine user preferences and make music recommendations. It would work in tandem with Spotify’s existing technology, which recommends music based on a user’s listening history and their friends’ streaming activity. Spotify made its case and stated that the alternative method of identifying a user’s music preferences, which requires the user manually selecting them, is too tedious and that the technology would relieve users of this burden.

When we published this article in May 2021, Spotify maintained that this technology has not been implemented, nor is there evidence that it has. As of August 12, 2022, there have not been any updates from Spotify about this patent or technology.

How could Spotify’s patent affect you?

The Twilight blue Amazon Echo 4th gen with the LED light glowing blue on a white desk.

Many people avoid smart speakers out of concern that they are always listening.

This approved patent has outraged Spotify users and artists globally: no one wants a powerful organization to spy on and manipulate them. On April 2, 2021, digital rights organization Access Now issued a letter to Spotify outlining a series of concerns about the patent. On May 4, 2021, a coalition of musicians and human rights groups issued a similar letter under the web domain Stop Spotify Surveillance. The concerns outlined in these letters include the implications of emotional manipulation, privacy violations, racial and gender discrimination, personal data insecurity, and worsening of inequalities in the music industry.

While the letter isn’t exactly long, some of the points need unpacking.

Emotional manipulation

If you’ve seen the Netflix documentary The Social Dilemma, you may be aware that tech giants like Facebook design their platforms to be as seductive and addictive as possible. The more time a user spends on an ad-based app, the more money the company generates. The documentary also reports that manipulative techniques that keep people online directly correlate to higher rates of mental illness in those users. If Spotify applies its technology to directly monitor a user’s emotional state and adjusts their experience with the app, it could grant the corporation a lot of power over that user’s well-being.

Access Now’s letter argues that Spotify’s technology, if implemented, could create a similar effect. It alleges that if the technology determines that a user is feeling depressed, and its objective is to keep said listeners on Spotify’s platform for as long as possible, Spotify might feed the user music with the effect of keeping them feeling depressed.

Privacy violations

Two women sitting on an outdoor couch. One woman is playing guitar.

Nobody wants their conversations recorded when they’re hanging out in their bedroom listening to music or gathering with friends. If the surveilling technology is always enabled while an app with this feature is in use, even your most private conversations would no longer be private. In addition, it would be impossible to obtain consent to be monitored from everybody within earshot of a device playing Spotify, especially if you’re listening to Spotify through your headphones in a café or on the bus, or using it to stream music at a house party.

Racial and gender discrimination

Spotify’s patent states that the technology can determine a user’s gender based on their speaking voice, which isn’t exactly true (or inherently useful to do). This statement assumes vocal pitches indicate one’s gender, but no system that does this will be perfect. As the Access Now letter points out, using voice data to identify a person’s gender would prevent the ability to self-identify.

You can’t assume one’s race or gender based on their voice.

Spotify also claims the technology will be able to determine a user’s accent which, as the Stop Spotify Surveillance letter points out, could have unintended racist implications since it may base assumptions about a person on the way they speak.

Personal data insecurity

Okay, so maybe you’re not worried about Spotify catching wind of your conversations, but if the company collects and stores this data, it could become a target for criminal data breaches. The Stop Spotify Surveillance letter also points out that even government authorities might request access to this data (or just buy it), in which case Spotify may have little control over whether or not it stays private. It’s unsettling to think that third parties could collect a conversation you once had in private, but it’s a very real risk.

Exacerbation of inequalities in the music industry

Marginalized artists already struggle to gain traction within the music industry due to socioeconomic disparities, and this surveillance won’t help. As the Stop Spotify Surveillance letter mentions, streaming services that promote already-popular music only worsen these inequalities because their algorithms try to keep users on the service and eventually get them to pay for it.

Do you need to worry about Spotify?

A picture of the Etymotic ER2SE wired earbuds being worn by a woman.

Nobody wants to worry about a private company listening in on their conversations.

On April 15, 2021, Spotify issued a response to the Access Now letter that states that it hasn’t rolled out any of these patented features, and it doesn’t plan to. According to the statement, Spotify constantly develops new technology for its platform, but these technologies are not always used. We reached out to Spotify to further inquire about its patent and if there is any intent to use this technology in the future. In response, a representative directed us back to the Access Now response letter.

In fairness to Spotify, this is very common among tech companies. There are so many complicated challenges to solve that sometimes what makes it to the patent office ends up getting dropped like a hot potato before it goes too far. It’s just the nature of the game.

However, Spotify’s response to Access Now did not make any promises to abstain from the technology forever. The Stop Spotify Surveillance coalition letter asks Spotify to publicly commit to never “use, sell, license, or monetize” the technology outlined in the patent for any reason. This letter has been signed by 180 organizations and individuals, who requested a public response from Spotify by May 18, 2021. Fight for the Future, another advocacy group, also published a blog post confirming that Spotify had not replied to its letter (sent May 4, 2021).

If Spotify implements this technology, what can you turn to?

A photo of a vinyl record with a label and producer listed on the front; essential information for those looking how to find new music.

If the idea of being surveilled scares you, it might be time to switch back to analog.

The unfortunate truth is that modern tech companies that don’t step on your privacy in some way are few and far between. To access most media services, you have to agree to a policy that strips you of at least some privacy. One reason streaming services are so affordable is that the companies often sell your personal data to advertisers.

If maintaining your privacy is important to you, the most obvious alternative to using a streaming service like Spotify is to not use a streaming service at all. Old-school CD players and turntables aren’t going to collect your personal data, though they don’t offer the same conveniences as modern technology. If you do switch back to analog, it may comfort your soul to know that musicians who barely get paid by streaming services will financially benefit a lot more if you purchase their music directly.

However, before you throw your phone away, remember that even if you choose not to use a service like Spotify, chances are most of the people around you are using it. If Spotify implements voice-surveillance technology, it might be nearly impossible to get away from it.

Frequently asked questions about Spotify’s voice recognition patent

While Spotify has not issued clear statement that this feature doesn’t use the voice recognition technology, there is information on Spotify’s website about how Only You works, and it doesn’t seem to include the voice recognition technology. According to the blog post, Only You simply summarizes data from your listening history on the streaming service based on songs and artists you have saved.

Leave a Reply