Project Challenge:

To design, enhance and scaffold the experience of a speech recognition-driven, learn-to-read educational subscription program for kids ages 5-8.

Project User:

For uptake, this project targeted parents who were looking to provide an engaging way for their children to learn to read. For the children themselves, we sought to create an enjoyable and interactive way to learn to read that could provide dynamic support while not distracting from the curriculum.

Project Role:

I was the Lead Experience Designer on this project - an initiative co-supported by LeVar Burton of Reading Rainbow fame. This included research, design, prototyping, playtesting, project coordination, management of UI, VFX and an external dev team. Marketing and packaging support, including copywriting and collateral work, was included among all of these things as well.

The Challenge:

This was a one of a kind project that I was truly proud to be a part of, but it was also incredibly complex. Reading Adventure is a live-service experience that combines tangible books that can be seen by a tablet’s camera by using Osmo’s patented ‘reflector’ technology that follows along with a child’s progress when a physical Osmo book is placed in front of the device. The game can then listen using speech recognition and follow the child’s reading page by page, correcting mistakes and offering support when required. Furthermore, the game uses a ‘crystal wand’ that a child can also use to point at words and images in the books that allow for various interactions or requests for how to pronounce a word.

From a product perspective there are a number of challenges that would need to be overcome for a live-service offering of this type that was seeking to marry both hardware and software, not the least of which were marketing and messaging, each of which I took part in shaping.

But from a purely experience perspective, this is a game that has a high level of user contingency that would need to be solved. At its core, the game needed to always keep a simultaneous eye on where the book is, if the book current in front of the device is correct, if it’s positioned correctly based on the game’s need, what page a user is on, what the child is saying and what they may be pointing at with their wand…at any given time. And should anything be out of place - the game needs to elegantly bring the child back into the experience with simple requests that need to remain easy to follow. This is all on top of the curricular logic of the game itself, how and when it decides to offer help, how and when it moves a player up or down a difficulty level and so forth.

This was a tricky project, and it needed to work with 72 books out of the gate - not to mention be designed and documented clearly enough for production hand-off, all of which would only be for the commencement of a years-long live-service offering, after which service feature design would continue. Those features may be covered in a subsequent case study.

Research & Discovery:

Our market research on this project, driven by a full team of qualitative and quantitative researchers, was incredibly exhaustive in terms of exploring and then establishing the parameters of what we would need to accomplish for our target user base. These key product targets would ultimately include:

• Targeting new readers aged 5-7 with a learn-to-read service that used speech recognition to support and correct reading.

• Offering a subscription curriculum to our users by way of monthly books that would escalate in complexity, each offering new spelling patterns, sight words and comprehension skills to learn.

• A set of multiple starting points for our users that would allow the most appropriate on-ramp for our families.

• Intended to work with camera recognition of not only physical Osmo books, but also a crystal wand that could be used for interactive phonics games.

• An underlying system that could monitor and guide the child’s progress so that a parent did not have to.

It should also be noted that this project’s owners wanted to create a companion app that would facilitate a parent’s selection of the appropriate point in our program and provide a first exposure of the main reading product to the child. I would also end up designing this companion app, but this will be discussed further in a later case study.

Proposed Solution:

The product in an of itself was comprised of a great number of play design, curriculum design and engineering solutions. But on the user experience side specifically, with all the moving parts of this project, we would need guidelines for Reading Adventure to work in an organized and expected fashion. I took to immediately researching and then documenting the most optimal way that we could scaffold interactions in the game product and laid these rules out for all stakeholders on the project.

Given that we were working with several satellite teams for content on this initiative, the documentation would be necessary to ensure that we were consistent in our designs and always knew how to teach children its interactive components and where and how to place instructional elements.

This documentation needed to cover a plethora of user situations throughout the game that would involve reading, book placement, wand placement, touch-screen interactions and speech-related interactions. This included ‘zoning’ that content for different device orientations and sizes. I wrote and iterated on this 24 page document, accompanied by Figma examples of a number of the key interactions, for distribution throughout the team.

Using these principles I set out in creating branching state flows for every part of the game, as well as every sub-feature therein, and labled each one accordingly with the relevant VO, VFX and SFX required to carry out a sequence. These flows would be handed off to engineering and technical artists to break them down by timeline and put them into production.

As a companion for a number of documents would also be information architecture diagrams of varying complexity for handling inputs, books and guiding users through curriculum.

Speaking of prototypes, I would also need to create prototypes for early stages of the product’s interactive portions such that we could test how fun certain interactions were and how we would incorporate curriculum into them. We ran these prototypes on zoom with children for 1:1 playtests and simulated automated speech recognition through a secondary operator that would trigger specific states in the prototype.

This prototyping step was critical in not only validating experiential concepts, but even more so, validating that our curricular assessment was working correctly.

Once we’d tested concepts, fleshed out our flow and brought together the elaborate tapestry of tech that would make this product function from speech recognition to camera vision detection, we began deploying our iterative builds, sending them out to test users through alpha and beta test phases that would be accompanied by a survey to be filled out by parents that probed them about questions regarding user engagement and learning outcomes.

There is a lot more that can be said regarding the work that I put into holistically helping define the experience in an end-to-end fashion through everything including the purchase flow, web wireframes, packaging material wireframing, copywriting for the app store, supporting materials, along with email and retention-related outreach…and this is not to mention the completely separate companion app for this project that I also designed in order to provide parents with the optimal means of finding a starting place for their child in our program, the Osmo Reading Level Finder (see right).

As a product designer with a focus on experience, I always look at the full spectrum of the experience by our users from start to finish, and this project was no different.

A hardware/software hybrid project of this complexity was an incredible joy to work on and I look forward to seeing it flourish and evolve as its live service carries onward into the future.

Reading Adventure was launched in September of 2022.

Evaluation and Validation:

The Reading Adventure product has already seen recognition with a 2022 NAPPA (National Parenting Product) award, a 2024 KidScreen Award and numerous parent testimonials. Reading Adventure achieved $500,000 in sales the first 3 months, 2,000 units sold before the launch date (pre-sale), with 85% subscriber retention after 30 days..