2nd Place in Alexa Day Competition
Time
Feb. 2019 - Apr. 2019 (2 months)
Tools
Tortu / Lucid Chart / Figma / iMovie / Alexa Dev Tools
Team
Melissa Shi / Ally Liu / Qian Wang / Paopao Ge / Sijia Li / Menghan Zhang / Mengqing Jiang / Wei Wang
Role
UX Lead / Researcher
Contribution
Lead the design of the conversation flow.
Handover and communicate the design to the development team and help them form the technical solution.
Followed the research lead and was part of most of the contextual interviews.
00
Summary
Problem
How might we help the visually impaired apply makeup more easily and independently?
Currently, there are 285 million people in the world who are visually impaired, with two-thirds of whom are female. However, it’s not easy for them to apply makeup on their own. In the current makeup industry, the visually impaired are often ignored and being under-served. However, we believe, “Everyone has the right to wear makeup, be confident and beautiful.”
Solution
An Alexa Skill that provides guidance applying makeup and checks the makeup.
With the help of 4 girls who have different kinds of visual impairment, we learned that the most challenging step of all is to check if things are well done. So we designed a voice user interface (VUI) as an Alexa Skill focusing on checking makeup and providing guidance and tips.
Key Features
1 / Check if the makeup is done well through computer vision.
Users can ask Beauty+ to check their current makeup at any time. Beauty+ will take a picture and use computer vision technologies to check if the product is blended well or if there is any unexpected color (e.g., not fully covered acne, mascara stains, eyebrow mistakes).
2 / Tailored guidance and tips
Beauty+ can guide users step by step, and provide tailored tips when needed. We gathered 121 tips, and many of them are especially designed for people with no or limited vision.
Other Features
Concept Video & Prototype Demo
Market Value
There are 285 million visually impaired people around the world, and this is a huge unexplored market. What's more, although designed for the visually impaired, many of our features are actually applicable to the general crowd. This enabled Beauty Plus to enter the larger market (the makeup industry has a 48.3-billion-dollar market value, and it's robustly growing at a speed of 5.5% per year).
48.3 B
Cosmetic Market Value
5.5%
Yearly Growth
85%
Women wear Makeup
01
Research
Competitor Analysis
The makeup industry has been ignoring people with visual impairments.

We researched current technologies in the makeup industry but sadly didn’t find any that are designed for people with visual impairments. In fact, none of the products we found support accessible features as simple as auditory feedback. We listed their main features to benchmark our future design.

Since video blog is a mainstream way to learn about makeup, we also researched makeup videos on Youtube. We found many makeup tutorials made by blind people, but didn’t see any intended to teach blind people.

existing competitors
Role-Play
We tried putting on makeup without vision, and realized tips about the applying the product are extremely important.

To better empathize with people with visual impairments and prepare for future user interviews, we did role-play makeup sessions, which means we put on makeup with eyes closed. Each of us did two role-plays, one by ourselves, one following a youtube tutorial.

Controlling the amount of product and the position of the tools is unexpectedly hard.

Using bone structure as landmark to describe things is better.

Tips for applyng makeup with eyes closed is different and crucial.

Contextual Interviews
We interviewed the visually impaired and realize checking is the true challenge, tips are also important, position guidance is not needed.

We interviewed three individuals with different types of visual impairments (including blindness), and observed them doing their daily makeup routine or ask them to follow a tutorial if they don’t usually use makeup. See below our key insights:

1. Despite experience level, all interviewees desire feedback on how well they did.

2. Interviewees have special tips and tools based on their vision condition.

3. Interviewees can control tools and apply them at the right position easily.

4. Interviewees tends to stay with the same makeup style to guarantee the best quality.

5. Checking the makeup during the day is hard, that’s why I don’t wear makeup/salient color.

Background Research
We did more research to build a set of tips tailored to the target users.

We looked through more than 100 makeup videos and articles and compiled a list of proper tips. We organized them into four categories: “For first-time users,” “How to apply,” “How to fix,” and “How to self-check.”

02
Ideate
Personas

We summarized two primary personas. One is an experienced makeup user who needs help mainly for the final check; the other is a novice user who needs more guidance and tips.

Walking the wall & Ideate

For ideation, we used “walking the wall” method. Design ideas were then plotted on a cost/value matrix. See below our final decisions.

See below the selected 6 features:

03
Design
Technical Experiments
Facing this unfamiliar platform, we prepared ourselves by building a simple app on it.

With no experience designing an Alexa Skill, how can we quickly learn about the capabilities and limitations of this platform? I first looked into design guidelines and development documentations, and then designed and built a simple Alexa Skill. This hands-on experience helped me get to know the framework, useful resources, and best practices to design for it. I was also able to experiment the boundaries of its capabilities.

Whiteboarding
With the selected features, we sketched out the conversation flow on the whiteboard.
Dialogue Flow
Finally, we designed the conversation flow and wrote scripts together.

For prototyping the conversational UI, we started with a whiteboard, markers, and post-its, and then digitized them as a flow of intents with Tortu and Lucid Chart. We broke down the flow into several parts and designed anchors to link them. After finalizing the flow, we wrote example utterances with slots for each intent.

04
Iterate
Wizard of Oz
Takeaway: be succinct in auditory responses.

We want to be polite, we want to be helpful, and we want to guarantee discoverability of key features. However, these are not reasons for having overly verbose responses. Through “Wizard of Oz” testing, we made our responses more concise, and the dialogue flows simpler. See below some tips we concluded:

  • VUI shouldn’t offer more than two options at a time.
  • Customize response based on user needs.
  • Pause as progressive disclosure.
  • Reply “Let me know ...” rather than “Do you want ...?” to simplify the flow.
  • Don’t ask, just do.

See below two examples:

Technical Prototyping
Due to technical constraints, we refined each character to generate the right tone for Alexa.

Testing in the simulator reveals more detailed problems, even problems with the punctuations we used, because Alexa can only pronounce two punctuations (“,” and “.”). So question marks, dashes, and apostrophes needed to be replaced while remaining similar tones.

Alexa simulation window
05
Final Design
Technical Solution

See below our technical solution framework and the computer vision solution.

Prototype Demo

Due to time and resouces constraints, we couldn't implement all our designs. See our final prototype showcasing several core functions.

Concept Video

We made a pitch at Alexa Day competition with a concept video and won 2nd awards 🏆! See below our concept video

View Design Summary ↑
06
Next Steps

As a first step towards accessible makeup for the visually impaired, our solution is still far from perfect. Future steps for this project can be:

  • Test with more users who have visual impairments to refine current design and scripts.
  • Design a personality for the skill to make it more adorable and memorable.
  • Identify and design for higher-level user needs, e.g. emotional needs.
  • Explore potential ways to incorporate or tackle more high-value-high-cost features.
  • Expand our target user group and design for the general crowd.