The story behind LEGO Voice Instructions

Matthew Shifrin, the fan behind LEGO Audio & Braille Building Instructions, explains how the LEGO Foundation is taking on his passion project

“Do I look like the kind of person who builds LEGO sets in his spare time?” asks Matthew Shifrin. “Probably not.” The reason he opens with that question is because he is blind. He is presenting at the LEGO Fan Media Days 2019 in Billund, Denmark.

Matthew’s friend Lilya took the time to transcribe instructions for Matthew, so that he could build sets on his own. The independence of being able to build a LEGO set without assistance was very important to the avid fan.

“I had a friend, Lilya, who would write down all the building steps for me so that I could upload them into a system that allowed me to read the building steps on a Braille reader through my fingers. She learned Braille to engage with me and support my LEGO passion, and then spent countless hours translating LEGO instructions into Braille,” Matthew explains.

“Blind people explore the world by touching things. So if I wanted to figure out how the Statue of Liberty I would have to climb it. If I tried to climb it I would get arrested. But if I built that same Statue of Liberty out of LEGO pieces then I am able to understand how it is shaped,” he said. “I realised that blind kids deserve to learn and know these things.”

Lilya and Matthew created a website, titled LEGO for the Blind, and shared the instructions for every set they had built. She would write out the text based instructions, he would build the set and see how the instructions worked. When requests started coming in for specific set instructions, it was more than the pair could manage.

“When we named elements we were hoping to get as many new LEGO builders into LEGO. So we thought okay if someone sees these elements for the first time what is the most intuitive name you can give them?” Matthew says, explaining why the element names in the instructions differ from the official element names.

At this stage, Matthew reached out to the LEGO Foundation to share his experience and pitch that the company produces instructions for blind people. “Lilya died two years ago and I said to myself I will not let this project rest until LEGO took it up,” he says.

Now, the LEGO Foundation is supporting an official scheme to produce voice instructions. Tech Innovation Director Olav Gjerlufsen explains that by using the code behind the LEGO set instructions, automation could be used to translate the illustrated instructions into text based instructions if a language were devised that could describe the pieces appropriately.

Working with a company in Vienna, Megan Shellenbarger is working on using machine learning to translate the LXFML data that it being translated into voice instructions. As well as describing how to build the set, the voice instructions explain what the builder is putting together and what it looks like when complete.

While the project is currently in a pilot phase, with four LEGO sets to get voice instructions initially, the aim is for every single LEGO set to get voice instructions eventually. It is an ambitious aim and the company plans to evaluate the four pilot instructions in January 2020, then determining what the next phase of this worthwhile initiative is.

To continue to support the work of Brick Fanatics, please buy your LEGO sets from LEGO.com and Amazon using our affiliate links.

Graham

Graham is the Editor of BrickFanatics.com, with plenty of experience working on LEGO related projects. He has contributed to various websites and publications on topics including niche hobbies, the toy industry and education. If you would like to get involved with Brick Fanatics, as a builder, writer or photographer – then please contact Graham at [email protected]

Graham

Leave a Reply