04 — AI Collaborative Canvas

SketchWiz

A real-time, AI-enhanced collaborative whiteboard. Draw diagrams, solve math and physics equations, and brainstorm together in persistent rooms powered by Socket.IO and Google Gemini.

Turborepo Next.js Node.js + Express Socket.IO Google Gemini Prisma PostgreSQL

Live site Demo GitHub ↗

The challenge

Collaboration that
thinks alongside you.

Problem

Traditional whiteboard tools like Miro are powerful but act simply as dumb canvases—they have no understanding of what is actually being drawn. If a student writes out a complex physics equation, the software can't help them solve it.

From a systems perspective, syncing freehand drawing across multiple clients concurrently is highly prone to race conditions. If handled poorly, strokes get lost, cursors lag, and server resources are exhausted by high-frequency socket broadcasts.

Solution

SketchWiz bridges the gap by piping canvas snapshots directly into Google Gemini's vision models. It can read handwritten math/physics equations and provide step-by-step solutions, or take a rough sketch and suggest professional enhancements.

The architecture utilizes a Turborepo monorepo. A Node.js/Express server handles real-time stroke broadcasting via Socket.IO, while Prisma continuously persists the room's canvas state to a PostgreSQL database so sessions survive browser refreshes.

<50ms stroke sync latency

∞ persistent rooms

Gemini math & vision engine

System design

Architecture diagram

A Turborepo monorepo decoupling the Next.js client from the Node.js WebSocket server, with Postgres handling persistent room data.

Next.js Client A canvas draw events

⟷

Socket.IO Hub real-time sync

⟷

Next.js Client B canvas draw events

↓

Express.js Server Node.js backend

⟶

Prisma ORM typesafe queries

⟶

PostgreSQL persistent room storage

↓

Canvas Snapshot base64 png payload

⟶

Google Gemini API math · physics · vision

⟶

AI Overlay live canvas solution

Engineering

Technical deep-dive

Real-Time Sync via Socket.IO

To enable seamless multi-user drawing, the Express server acts as a WebSocket hub. The infinite canvas emits specific events (`draw`, `erase`, `cursorMove`) which are broadcast strictly to clients connected to that specific Room ID. This isolates traffic and ensures high-performance syncing.

socket.to(roomId).emit('draw-event', payload);
// Broadcasts only to specific active room

AI Math & Sketch Engine

The core intelligence of SketchWiz relies on multimodal reasoning. When a user asks for help, the frontend captures the current canvas bounds as a Base64 image and pipes it directly to the Google Gemini API. Gemini reads the handwritten math/physics equations and streams back a formatted, step-by-step solution.

const response = await gemini.generateContent([
{ inlineData: { data: base64Canvas, mimeType: "image/png" } },
"Solve this physics problem step by step."
]);

Persistent Canvas Storage

Rooms in SketchWiz are not ephemeral. To ensure users don't lose data on refresh, the Express backend uses Prisma to interact with a PostgreSQL database. Canvas drawing paths are serialized and continuously upserted into the database using an intelligent debouncing mechanism to prevent database bottlenecking.

await prisma.room.upsert({
where: { name: roomId },
update: { canvasState: stateJSON }
});

Turborepo Architecture

The entire ecosystem is structured as a Monorepo using Turborepo. This allows the Next.js frontend (`apps/web`) and the Express backend (`apps/server`) to seamlessly share TypeScript definitions and UI configurations (`packages/config`), ensuring strict end-to-end type safety between the client and server.

// packages/config/types.ts
export type CanvasStroke = { x: number, y: number, color: string };

SketchWiz

Collaboration thatthinks alongside you.

Architecture diagram

Screenshots

Technical deep-dive

Collaboration that
thinks alongside you.