NPC 5/17/13 Featured Presentation: Virtual Occupational Therapy

Below is an edited transcript of the May 17th, 2013 NonProfit Commons in Second Life meeting, featuring Andrew Hughes of Designing Digitally, Inc.

You can view the full transcript at: https://docs.google.com/document/d/19PLNXWnfoi3b7C-D05neJCONP98HvBZVjhhJlrZxx8E/ and photos at: https://plus.google.com/events/gallery/crk8ercovt8052kc4gu6klmtfcs

Today for our featured presentation we are happy to feature Andrew Hughes of Designing Digitally who will present about the company’s recent project using Kinect motion tracking and immersive training simulation for stroke victims that they have been working on for the National Institute of Health.

About Andrew Hughes:

Andrew Hughes founded Designing Digitally, Inc. which specializes in E-Learning, Training Simulations, Serious Games, and Virtual Immersive Learning. Andrew has extensive experience in education as a professor at both the University of Cincinnati and at the Art Institute of Ohio – Cincinnati. Currently Andrew is the president of Designing Digitally, Inc. and also is a curriculum evaluator for ACICS the private college accreditation board. The majority of Andrew’s experience has been in the development of enterprise learning solutions for Government and Fortune 500 clients.

Andrew also was a consultant for the Ohio Board of Regents and the U.S. Department of Education for the Office of Innovation where he helped to develop ground-breaking learning spaces for the K-12 sector. Having successfully taken on responsibilities in instructional design, project management, sales, and leading his own team, Andrew has propelled Designing Digitally, Inc. to be an award winning virtual immersion and E-Learning company.

Let’s welcome Andrew up, please take a seat and start whenever you are ready.

Buffy Beale: Cheering madly!

Kali Pizzaro: Hey Andrew

Jen (jenelle.levenque): Welcome to NPC Andrew

Andrew Hughes: Hello everyone! nice to see you all! I am the President of Designing Digitally, Inc. We’re located between Cincinnati and Dayton Ohio We’ve been doing development work since 2001, and went INC’d in 2006. We do a very wide range of work. Mostly it’s for online learning, in forms of both 2d and 3d interactive learning.

What I am here to talk about today is our effort we created with a SBIR grant from the National Institute of Health. We partnered with a healthcare agency called Barron Associates to create a new product called VOTA, Virtual Occupation Therapy Assistant

We are using the Microsoft Kinect and the Unity3D gaming engine to create dynamic AI bots that help you re-learn how to use your arms. It’s for stoke victims, and people that have MS, etc. What this software is doing is using AI technology to help guide you in learning how to do everyday things again. Such as putting the groceries away. We are tracking the six axis’s that you have with your arms. We are using 2 Kinects together to get very high detailed tracking and the system is built into kiosks that are now going to be deployed in hospitals, and PT locations throughout the nation.

Tarquin Evermore: Oh how cool alternative use of Kinect!

Buffy Beale: it really is fantastic

Andrew Hughes: This SBIR grant was for Phase 1, which was to build the system and now we’re awaiting to see if Phase 2 (full deployment) will be funded so we can roll this out throughout the USA. We have built a content management system that sends all data about your progress, and even records your movements for you to see online and also for the your therapist to review to see how your movement has improved throughout time of using this tool. The AI bots actually are not static, so they change the difficulty, and even change the scenarios based on how well you are doing so it learns from you while you are learning! And increases the difficulty based on your past performance. 🙂

Rhiannon Chatnoir: for those that might not know, SBIR is short for Small Business Innovation Research – a series of government grants to encourage small biz development/projects

Andrew Hughes: Thanks Rhiannon!

So with that said here is an overview (we can only show just this small video but take into consideration that the actual development is much more than this, it’s just a small teaser for people until w know if we get phase 2 of the SBIR. https://www.youtube.com/watch?v=Asy0Kkaff0s

Tarquin Evermore: This sounds awesome and exciting. It probably has a nifty side effect, I am sure many stroke sufferers suffer from depression for thinking they are ‘less than’ an able bodied person, and having to rely on others. It sounds so liberating to get that freedom back ;3

Andrew Hughes: So this is one of many projects we’re working on. Yes, the biggest thing is we wanted them to feel accomplished by doing thing they did everyday with little effort. So our goal was to teach them to cook in a safe way and do things around their home they had no problem doing in the past. This is just one of the scenarios out of many by using 2 Kinects together we can get a better sense of the 3d space.

Sum Anachen (szemanzoltan): It is awesome!

Tarquin Evermore: And how else but virtually? if they make a mistake or slip, they don’t have to worry about burning the food, or worse, themselves!

Andrew Hughes: right Tarquin! 🙂

OK, In regards to our efforts, this is something we’ve been working on for awhile now. Unity3D obviously has become a very popular gaming engine. What we’re doing is trying to push it’s efforts far beyond what it can do to produce learning experiences that are engaging, educational, and yet entertaining so the learning is fun and it sticks

Rhiannon Chatnoir: to view the movie it will show on the big NPC screen behind us, just click play

Azwaldo Villota: Please, could you also post video URL here in chat?

Andrew Hughes: https://www.youtube.com/watch?v=Asy0Kkaff0s

Kali Pizzaro: engages

Rhiannon Chatnoir: you can also view it on youtube http://youtu.be/Asy0Kkaff0s

Glitteractica Cookie: watching on YT now

Andrew Hughes: We used the Kinect to build a serious game called Air Marshaller serious game where we used the Kinect to teach you how to do the air marshal signals

Sister (sister.abeyante): May I make a constructive request? Can your company please caption the youTube video, as the “auto captioning” does not do a very good job and the content is important to the deaf /HOH as well as the hearing. thanks!

Andrew Hughes: sure!

Rhiannon Chatnoir: so the video mentions that your real world Occupational therapist, so do you anticipate this to be a facilitated experience with a OT and patient

Andrew Hughes: Yes there is a OT. It can be either on site with you or online where they can see your progress in real time.

Tarquin Evermore: might be a good idea. I often have trouble when trying to listen to some of the videos a bhuddist monk puts up, because I can’t hear it.

Andrew Hughes: Another good example of using the Kinect for fun purposes is that Kinect serious game we built in Unity3D https://www.youtube.com/watch?v=Ef_vAih13pU

Tarquin Evermore: I love this idea, when you turn therapy into a game, it seems less like theropy and ‘boring’

Andrew Hughes: Glad you guys like it. We wanted to do activities that people take for granted each day, such as the daily chores we find annoying. They become very hard when you don’t have control over your limbs. So we wanted to focus on accomplishments of everyday activities.

Jen (jenelle.levenque): My dad was very frustrated after his stroke

Wisdomseeker (lissena): and playing adds to consistency of practice, I suspect–less frustrating?

Andrew Hughes: agreed!

Wisdomseeker (lissena): music is being used with stroke patients in Canada, I know

Tarquin Evermore: They put Wii Fits in old age homes and such to get older people to move. And since it was a game, they didn’t notice it was theropy so they wanted to do it instead of “aww gee…do I haveta?” and since WIi Fit was low impact, it was perfect for the elderly.

Sister (sister.abeyante): Question: Will the program accommodate individual differences in physical ability i.e. a person with no arms needs to learn to do these same tasks, but does not have limbs to do it. Can it adapt/be adapted to something like that?

Andrew Hughes: It’s funny. if we can change the learning culture from “I have to” to “I want to” then you will see excellent results.

Rhiannon Chatnoir: and pairing the motor movement of Kinect with the game-like process of going through the daily routine works on both a physical and cognitive level I would think

Tarquin Evermore: Maybe different “games” for different physical types? Like maybe a ‘tour de france‘ type game for those without arms to learn how to use their legs again?

Andrew Hughes: That’s a great idea

Tarquin Evermore: And didgital ‘rewards’ for completing the task? Heck we all know why folks play MMOs, thats part of why it is addicting. The whole ‘reward’ thing.

Sum Anachen (szemanzoltan): Well MMO gives you a community feeling thats why it works so well

Buffy Beale: Question: Are there any other countries interested in your program for their hospitals?

Rhiannon Chatnoir: so does that correlate in the VOTA with the points that get earned throughout .. they are earning them via activities and how accurately done? is this mainly to keep progress and work to improve their overall score?

Andrew Hughes: Yes digital rewards are for completing tasks. You are ranked based on people = to you rather than just seeing a top 10, that way you can strive to be a little better than the person above you

Buffy Beale: nice idea

Sister (sister.abeyante): Question: Is this system only going to be available in a hospital or clinic environment, or do you anticipate making it into a portable “kit” that could be set up on someone’s home?

Tarquin Evermore: I am betting Japan would be all over this awesome. Heck they have an adorible robitic baby sea lion. All white and fluff and awww <3! it reacts just like the real deal. They use them in nursing homes.

Andrew Hughes: from there we are not sure about other hospitals in countries because we’re doing this as a SBIR so it’s not otu to market yet. We are actually making Kiosks that can be portable.

Rhiannon Chatnoir: Ozma, the relative scoring reminds me some of the math learning game you led at Arizone State Univ

Tarquin Evermore: Speaking of math, I am remembering Number Munchers very fondly. Still LD as heck, but that is one of the few math games I remember fondly XD

Andrew Hughes: Did you guys have any other questions regarding what we’ve built?

Rhiannon Chatnoir: you have done development of projects both in virtual spaces like SL, etc and then these free standing experiences in Unity or otherwise, what wisdom or differences in process do you find

Azwaldo Villota: What are the main advantages to developing with Unity? Is it due to the connectivity with Kinect?

Sister (sister.abeyante): I do. Ever the advocate for equal access— will the product be accesible to the deaf.hard of hearing? Captions rather than just audio?

Andrew Hughes: The reason why we did it in Unity3d is:

1. Level of detail / fidelity

2. Flexibility of the platform

3. Ability to create dynamic experiences

4. Kinect integration (we ended up having to custom code this with DDL’s and so while we thought it would not be hard to use Unity3D with the Kinect we found that we had to do a ton of R&D to get it to work properly

It can be deployed as an app, and we have it autoboot that exe file when the computer boots up, so that way it’s acting as it’s own OS

Sister (sister.abeyante): and I also have a related question to the one on accessibility for the deaf/Hard of hearing… will it be available to people who do not speak/understand English?

Rhiannon Chatnoir: so being able to be freestanding experience if need be

Andrew Hughes: yep! It can be. Right now it was setup of English only based on the SBIR requirement. You are correct Rhiannon.

Sister (sister.abeyante): And the accessibility to the deaf/HOH? Was that also an SBIR requirement?

Azwaldo Villota: Wow. Great info, much appreciation for you sharing here today, Andrew.

Andrew Hughes: Sister – Right now it is not accessible but it will be for Phase 2 if we are awarded the grant for phase 2. The Phase 1 is the prototype effort, phase 2 is the funding to do revisions / additions / mass produce.

Sister (sister.abeyante) nods… so the prototype wasn’t, but the mass production will be- that’s great.

Rhiannon Chatnoir: the first phase grant is more of a proof of concept, working prototype. if you do not get phase two grant support, what will be next step if any for this project

Tarquin Evermore: with this current economy, how about are you going to get this funding?

Andrew Hughes: Rhiannon – Good question about funding. The partering company is still going to be working with us to deploy it if we get funded or not. The partnering company is known for getting Phase 2 passed, so we shall see. Tarquin – It’s in review now and we will find out in September if we got Phase 2.

Frans Charming: 🙂

Rhiannon Chatnoir: not sure how much this gets impacted Tarquin, http://www.sbir.gov/ for deeper info on the program

Sister (sister.abeyante) wonders if, from the user’s/consumer’s end of things, this will be covered by insurance? Or will the user have to pay out of pocket to use this?

Frans Charming: Hopefully the sequester has no impact on the phase 2 funding.

Andrew Hughes: Sister – This side of it has not been discussed yet

Wisdomseeker (lissena): perhaps I missed this? how did you measure results of your program?

Sister (sister.abeyante) nods and reflects that, without it being covered by insurance, many or most folks most needing it will not be able to use it due to cost to the consumer.

Andrew Hughes: Wisdomseeker – There are multiple analytical pieces

Dancers Yao: great Q Sister…this has been an ongoing problem for seniors…to afford any of the new tech services

Rhiannon Chatnoir: any other questions or comments for Andrew

Buffy Beale: Just to say well done, it looks like this will be part of the future Andrew 🙂

Andrew Hughes: Thanks Buffy

Dancers Yao: really great to see you appreciate using a dynamical approach to your work….thanks

Ozma Malibu: Wonderful. I want to spread the word.

Rhiannon Chatnoir: yes, and good luck on phase two funding.. I have been part of an SBIR grant process too .. know they are a lot of work and waiting

Andrew Hughes: Thanks everyone!

Gentle Heron: Very cool Andrew. I’ll be looking for the next Phase of development from you.

Azwaldo Villota claps while clapping

Jen (jenelle.levenque): Regarding the payment question, if they are in therapy, then the specific tools used shouldn’t be an issue

Wisdomseeker (lissena): yes…just wondered what you were look inspiring!

Tarquin Evermore: BRB reconnecting :3

Wisdomseeker (lissena): sorry for mashup

Wisdomseeker (lissena): Inspiring!

Rhiannon Chatnoir: let’s thank Andrew for being here today and you can find more at

Buffy Beale: cheering!!

Frans Charming applauds

Scarlett Sismondi: thanks!

MusE Starsmith: ♫♫ APPLAUSE APPLAUSE ♫♫

MusE Starsmith: Applauds!

Jen (jenelle.levenque): Yay Andrew and company

Jen (jenelle.levenque): ********APPPLLLAAAUUUSSSEEE********

Andrew Hughes: Thanks everyone! 🙂 We are working hard to be creative!

Zadark Xavorin applauds

Sum Anachen (szemanzoltan) applauds

Buffy Beale: thanks Andrew

Andrew Hughes: thanks everyone for the time today!

Written by: Rhiannon Chatnoir