Deskucchi is one of the most exciting mixed-reality apps that I tried on the Meta Quest 2. This app uses ChatGPT and Google Cloud TTS APIs, alongside Quest Passthrough, hand-tracking and VRoid Hub—to being AI virtual characters to life and interact with them both physically and with voice.
See it in action!
The virtual characters can be occluded ( hidden behind real objects) based on the object setup in-app and that of the Quest options.
You can also bring your own VRoid 3D virtual characters using VRoid API.
Aside from that, you can also define the personality of the AI avatar using a prompt. This uses ChatGPT API to define the character’s interactions with you. You can say for example, “Be the evilest person alive, respond to me like the evilest demo”, or “Be the nicest person, fall in love with me”.
This will lead the character to interact with you based on that initial definition.
Deskucchi supports the Meta Quest 2 and Quest Pro hand-tracking. There aren’t lots of options, but you can poke and even grab the characters but it’s a bit glitchy. There are also some mini-games and gesture-based dancing you can do with the virtual character.
I, for example, spawned an anime girl and made her fall in love with me, an then made her pure evil. She was smart of course, as behind the scenes it uses ChatGPT 3.5.
How to use it?
Now, although Deskucchi is a free-to-download app on the Oculus Store–you’ll need a subscription for both ChatGPT and Google Cloud. I actually create a new Google Cloud account, so I got the free trial which gives you $300 for free to use during the trial.
You can’t use the free ChatGPT account, you need to have your credit card added and subscribe to ChatGPT as those API calls cost money. I tried without and it didn’t work, just so you know. Maybe you want to try it out first, but for me, it worked immediately when I added my Credit Card to Chat GPT. BTW, you can put a limit on the spending like $10 max monthly if you are worried about using a 3rd party software and getting high bills. Basically, you shouldn’t but just in case.
Once you are in the app, you need to type the API keys. I recommend using SideQuset to copy-paste it into the Quest’s browser input field and then copy it from there and paste it into the Deskucchi app.
Follow this guide to see how you can do it on Microsoft Windows PC.
Summary
Deskucchi makes use of different APIs to make a unique mixed-reality app that stands out from the rest. It’s not easy to setup and you need a subscription to ChatGPT and Google Cloud to make it work. So although it’s not accessible for some although the app itself is free, once you set it up it gives you a glimpse into the future. There are ways to experience a similar thing using Roblox if I remember correctly.
In the near future, there will be apps that will implement these features and much more. Things will get much easier with future VR and Mixed Reality headsets that already have depth-scanning and built-in integration with the headset’s own APIs to make it more accessible and easier for devs to create amazing mixed reality/augmented reality apps that use Artificial Intelligence and machine-learning based algorithms to create amazing apps like this and way more.
I personally can’t wait to see apps similar to that and more running on the Apple Vision Pro, that would be totally sick!
Keep in mind that Quest 2 has black and white passthrough only, and it will be even more immersive on the Meta Quest Pro and on the upcoming Meta Quest 3.
SUBSCRIBE!
I hope you enjoy this article. Don’t forget to SUBSCRIBE to my VR channel here. Thank you.