SALT LAKE CITY — Yes, Alexa is listening to you — and the smart speaker-turned-personal assistant isn’t the only one.

Amazon’s Alexa is one of the many now prevalent digital assistants meant to make your life easier and answer simple questions about your upcoming calendar appointments, or set timers and alarms, and even change the temperature or turn on your wi-fi connected smart home appliances.

Much of the ability of these digital assistants come from the vast computing power that Amazon has to run speech recognition software to change your voice into 1’s and 0’s that Alexa can understand. But to anyone who’s familiar with trying to ask the device a simple question knows, sometimes it’s incredibly frustrating.

“Somewhere along the way someone has to annotate that data that’s coming in,” Matt Day from Bloomberg News said.

Designated employees go in and see how much of what you said to the machine was heard correctly. They transcribe and annotate other snippets from conversations and questions to make sure that Alexa is understanding them properly, then feed that information back into the system. The goal: to keep teaching Alexa to be better at answering even the simplest questions, like what the weather is going to be like tomorrow.

“People are saying billions of things into Alexa [Amazon] says, and it’s going to pick up accidentally a ton of stuff including some disturbing things or some funny things,” Day said.

Day said those who work for Amazon across the globe have heard things like children crying, poor singing in the shower, and what, at times, may have been a sexual assault.

This process isn’t something that is unheard of, though. In the tech world, this form of processing is broadly known as ‘supervised learning’ when a human is involved, and when it is paired with other machine learning is known as ‘semi-supervised learning’.

There are thousands of these employees hired to listen and transcribe recordings captured by Amazon’s line of Echo speakers. Bloomberg reports those employees are transcribing up to 1,000 audio clips during their 9-hour shifts every day.

According to Bloomberg, several people that had reportedly worked for the internet giant said that the recordings are associated with the customer’s first name, device serial number, and an account number.

Amazon told KSLNewsradio that they take the security of their customers’ personal information seriously and only annotate “an extremely small sample of Alexa voice recordings in order [to] improve the customer experience.”

The statement continued: “Employees do not have direct access to information that can identify the person or account as part of this workflow. All information is treated with high confidentiality and we use multi-factor authentication to restrict access, service encryption and audits of our control environment to protect it.”

Amazon said that the employees are there to help train the speech recognition so that Alexa better understands your requests.

“We have strict technical and operational safeguards and have a zero tolerance policy for the abuse of our system,” the Amazon statement continued.

Echo devices use an on-device keyword spotting to detect a, “wake word,” and an Amazon spokesperson said that this is the default.

“Echo devices are designed to detect only your chosen wake word (Alexa, Amazon, Computer or Echo). The device detects the wake word by identifying acoustic patterns that match the wake word. No audio is stored or sent to the cloud unless the device detects the wake word (or Alexa is activated by pressing a button).” They continued.

You can tell if your device is streaming that information back to the cloud to get your answer when the light ring on the top of the Echo glows blue.

 

Follow @walk_colb