Hanashi - Immersive AI Storytelling

Hanashi (話 - "story" in Japanese) is an immersive storytelling game where you become the hero of an AI-generated narrative. Every choice you make shapes a unique story brought to life through fully AI-generated scenarios, voice narration, dynamic sound effects, and adaptive soundtracks.

Vision

The goal was to create a super immersive experience that combines cutting-edge AI technology with rich audio-visual storytelling. Every element of the game—from the narrative to the soundscape—is dynamically generated in real-time, creating a truly unique experience for each playthrough.

Key Features

🎭 AI-Generated Narratives

Stories dynamically created using Anthropic's Claude Sonnet 4, adapting to your choices in real-time. Each playthrough generates a completely unique narrative path.

🎤 Immersive Voice Narration

ElevenLabs text-to-speech with character-level timestamps for synchronized subtitles. The narrator brings the story to life with expressive, natural-sounding voices.

🔊 Dynamic Sound Design

AI-generated sound effects and ambient soundscapes that match the story setting. Every action has its own unique sound effect, and ambient soundscapes create the perfect atmosphere.

🎵 Adaptive Soundtrack

Mood-based music that evolves with the narrative (calm, tense, danger, mystery, triumph). The soundtrack seamlessly transitions as the story's emotional tone shifts.

🎨 3D Visual Experience

Beautiful 3D scene rendered with React Three Fiber, featuring bloom effects, ordered dithering, and smooth camera movements. The visual experience complements the immersive audio.

🎮 Interactive Storytelling

Make bold choices that significantly impact the narrative direction. Your decisions matter and create branching storylines that feel meaningful.

⚡ Real-time Generation

All content generated on-demand with intelligent caching for performance. Stories, voices, sounds, and music are all generated in real-time while maintaining smooth gameplay.

Technical Architecture

Story Generation Flow

1. User Action → Player selects an action choice
2. Parallel Generation:
   • Anthropic Claude generates narrative text, action choices, and mood
   • ElevenLabs generates action sound effect
3. Sequential Generation:
   • ElevenLabs generates narrator voice with timestamps
   • If mood changed, generate new mood music
4. Rendering: Story displayed with synchronized audio, subtitles, and visual effects

Audio System

The game features a sophisticated multi-layer audio system:

• Background Music: Fades out when story begins
• Soundstage: Ambient loop matching the story setting (generated once per story)
• Action Sounds: Short sound effects for each action (generated per action)
• Mood Music: Progressive background music that crossfades when mood changes
• Narrator Voice: Delayed playback to let action sounds play first

Caching Strategy

• LLM Responses: Cached by message history hash
• TTS Audio: Cached by text + voice ID
• Sound Effects: Cached by prompt in metadata
• Mood Music: Cached by setting + mood combination

Tech Stack

AI & Audio Generation

• Anthropic Claude Sonnet 4 - Story generation and narrative AI
• ElevenLabs - Text-to-speech, sound effects, and music generation
  • Voice narration with alignment timestamps
  • Action sound effects
  • Ambient soundscapes (soundstage)
  • Mood-based background music

Frontend

• Next.js 16 - React framework with App Router
• React Three Fiber - 3D rendering with Three.js
• @react-three/postprocessing - Visual effects (bloom, tone mapping, dithering)
• Zustand - State management
• Tailwind CSS - Styling
• Vercel Blob Storage - Audio file caching

Visual Features

• Bloom Effects: Glowing post-processing for atmospheric visuals
• Ordered Dithering: Custom shader for stylized rendering
• Smooth Camera: Dynamic camera that follows story progression
• 3D Text: Story history rendered as 3D text in the scene
• Character Animation: Animated character model with cursor following

Development

Built for Project ElevenLabs - showcasing the future of AI-powered interactive storytelling. The project demonstrates how multiple AI services can work together to create a cohesive, immersive experience.

Debug Mode

Add ?debug to the URL to access Leva controls for adjusting lighting parameters, bloom effects, character animations, and scene elements.

Credits

Created by:

• Cody Adams
• Jordan Legg

Built for Project ElevenLabs - showcasing the future of AI-powered interactive storytelling.

---

Experience stories like never before. Every choice matters. Every story is unique.