From Blender to ESP32: The Intellar Engine Animation Pipeline

store: https://intellar.square.site/

source code  :  https://github.com/intellar/Intellar-Engine-Animation

youtube demo:   

https://youtube.com/shorts/1qVl1EnIAbE

https://www.youtube.com/shorts/eF7kYPP8HNs

Redefining 2D Embedded Animation

Why Traditional Frame-by-Frame Fails on the ESP32 In a classic animation pipeline, playing a 2D animation requires loading an image sequence frame by frame. At a resolution of 240×320 pixels in RGB565 format, a single uncompressed image consumes 153.6 KB. A smooth, fluid animation would instantly exhaust an ESP32's available Flash and RAM, making complex interactive characters impossible.

The Solution: Real-Time Skeletal Animation The Intellar Engine transforms embedded 2D rendering by replacing heavy image sequences with dynamic cut-out skeletal animation.

• Cut-Out Sprite Parts: The character is split into individual, tightly packed image planes (e.g., head, ears, torso, paws).

• Hierarchical Rigging: A virtual skeletal armature drives the character. Joints are visually aligned, and moving a parent bone naturally cascades down to its children.

• Massive Memory Savings: Instead of storing hundreds of full-screen frames, the engine stores only the unique texture fragments and a highly compact list of matrix transformations for the keyframes.

• The Result: A fully rigged character, complete with a diverse suite of complex animations, fits entirely into a single, self-contained 60 KB binary footprint.

The Core Innovation: A Bidirectional Blender ↔ Python Pipeline

The true heart of this project is not just the rendering engine, but the complete bidirectional workflow between Blender and Python. Rather than a simple one-way export button, this pipeline provides true synergy:

• Importing & Rigging: Character parts (head, ears, torso, etc.) are imported into Blender as image planes. The Python system automatically generates and instantiates the base armature (the rig), allowing you to visually align the joints to your artwork.

• Visual Animation: Bones are animated directly within Blender's familiar and powerful interface using standard keyframes.

• The Python Round-Trip: Animations are not exported as a black box. They are translated into readable Python data structures. This means you can seamlessly pass animations back and forth between the 3D software and your code environment.

External Editing & AI Integration

Because the animation data lives externally as Python code, it is completely decoupled from the 3D software.

You can modify animations directly in the code—adjusting timings, tweaking motion curves, or altering keyframes—and then re-import those changes into Blender to visually preview and refine them. This architectural flexibility opens incredible possibilities for Artificial Intelligence: an AI agent can assist you in generating new animation sequences, smoothing transitions, or procedurally creating motion variations directly within the Python files, which you can then visually validate in Blender.

The Foundation: Powered by the Intellar Engine

This entire animation pipeline doesn't exist in a vacuum—it is built directly on top of the Intellar Engine. As the core hardware and software platform of the Intellar ecosystem, the Engine serves as the robust foundation for all our embedded creations. It provides the unified architecture—from the ESP32-S3 microcontrollers and custom PCBs to the display drivers—that gives this animation system the power and seamless hardware integration it needs to run flawlessly.

Deployment & Preview: The .iska Format

Once the animation process is perfected within the Blender/Python ecosystem, the .iska format comes into play. It is not the core of the animation process, but rather the final deployment artifact.

It is an ultra-lightweight, highly optimized custom binary container. Its primary role is to allow you to preview the exact final rendering on your desktop (via iska_view.py) before permanently embedding it into the ESP32's flash memory.

• Zero External Dependencies: No PNG decoders, JSON parsers, or Python interpreters are needed on the ESP32. The C++ engine loads and reads the binary directly into an RGB565 framebuffer.

• All-in-One Structure: Embeds the skeleton hierarchy, RGB565 texture blocks, and animation keyframes into a unified payload.

The C++17 Engine: High Performance & Hardware Optimization

The core runtime (src/Iska/) is written in C++17 without dependencies to extract maximum performance from the ESP32-S3.

• Microsecond Execution: A full frame render completes in just ~0.10 ms on a single core. The real bottleneck is the physical SPI bus (~31 ms per frame for 240x320 at 40 MHz).

• Affine Blitter: Uses nearest-neighbor sampling to preserve crisp, sharp pixel-art edges without filtering blur.

• Transparent Keying: Magenta is treated as a transparency key, skipping invisible pixels on the fly with zero alpha-blending overhead.

• No Scale Inheritance: Bone scale is not inherited by child bones. This allows for expressive Squash & Stretch effects on specific parts without uncontrollably deforming the rest of the body.

A Fully Reproducible Pipeline

From initial artwork to screen display, the entire chain is fully automated, deterministic, and verifiable.

• Visual Rigging: Joint placement and draw order (Z-axis) are handled visually in Blender.

• Computed Assembly: Scale, feet line, pivots, and RGB565 packing are entirely computed by Python tooling (rig_build.py, iska_pack.py).

• Bit-for-Bit Verification: The tooling enforces a strict check where the C++ engine rendering is compared against a Python simulation. If a single byte differs across 94 comparison frames, the build fails.

On the Robot (ESP32-S3)

The generated .iska asset is embedded directly into the ESP32-S3 firmware. Because there is no filesystem step, the animation can never drift out of sync with the code.

• Footprint: Measured at just 6.2% RAM and 11.3% Flash (asset included).

• Direct Drive: The panel is driven directly with Arduino SPI—no heavy third-party graphics libraries required.

Quick Start: From Blender to Engine

The entire pipeline can be executed from the repository root using standard Python (3.10+, pillow). Blender is only required for visual rigging and optional keyframing.

1. Artwork & Planes Import each character part into Blender as a plane (Images as Planes). The object name must match the bone name exactly (e.g., torso, head). Position each plane on the Z-axis to define its draw order (larger = in front).

2. Scaffold the Rig Generate the starting skeleton seeds and animation template using the built-in scaffolding tool: python tools/new_character.py <id> --parts torso,head,armL,armR,footL,footR

3. Seed and Align the Skeleton Push the seeds into Blender to generate the armature, then manually align the joints (bone heads) to act as the exact pivot points for each part: python tools/open_blender.py --id <id> planes --script blender_make_rig.py ...

4. Build the Asset Compute the runtime rig (converting Blender data into screen pixels) and generate the .iska file: python tools/build_asset.py --id <id> --rig-from-blender

5. Animate & Preview Animations can be coded purely in Python (tools/anims_<id>.py) or keyed visually in Blender and baked back into JSON. Validate and view the result in real-time on your desktop: python tools/iska_view.py assets/<id>.iska

Repository Overview

Source code: github.com/intellar/Intellar-Engine-Animation

src/Iska/ - the C++17 runtime engine (zero dependencies)

IskaLoader - reads the .iska binary into memory

IskaPlayer - resolves the bone hierarchy and interpolates the keyframes

IskaRender - the nearest-neighbour affine blitter with the magenta transparency key

tools/ - the Python toolchain (the Blender <-> Python <-> .iska pipeline)

build_asset.py - runs the whole chain in one command, with the byte-for-byte verification

new_character.py - scaffolds a new character (rig seeds + animation template)

open_blender.py - launches Blender with the right scene and script

blender_make_rig.py - creates/refreshes the armature in a .blend

blender_bind.py - binds the planes to the bones and enforces the engine's rules

blender_export.py - exports planes + bones from Blender to scene.json

blender_import_anims.py / blender_bake_anims.py - the Blender <-> JSON round trip

rig_build.py - computes the runtime rig in screen pixels

iska_pack.py - packs PNG -> RGB565 + poses into the .iska file

iska_view.py / iska_preview.py - desktop preview of the exact engine render

iska_clip_check.py - the 94-frame C++ vs Python pixel-identical check

demo/main.cpp - desktop player (SDL2 window + headless frame capture)

firmware/esp32/ - the ESP32-S3 firmware

main.cpp - the runtime loop

display565 - the Arduino SPI driver for the 240x320 panel

lcd_config.h - pinout and orientation

platformio.ini - the PlatformIO build

docs/FORMAT.md - the .iska binary format specification

assets/rabbit3.iska - the shipped asset (the rabbit demo, ~60 KB)