Source code: https://github.com/intellar/ImpulsePlay
To clone the repository:
git clone https://github.com/intellar/ImpulsePlay
Updates:
2026-06: physics engine extracted into a dedicated module, with collision regression tests
2025-06: match-3 ball game, explosive balls, score and high score saved in LittleFS
2025-05: impulse-based physics engine with squash & stretch deformation
2025-04: animated eyes driven by IMU, with laser target mode
This project is a portable interactive companion built around an ESP32-S3, a color touch screen, and a BMI160 IMU. Tilt the device and balls roll with gravity. Shake it and inertia kicks in. Touch the screen to grab, throw, or delete balls. In another mode, a pair of cartoon eyes reacts to movement, blinks, and squashes against the edges of the screen.
The goal is not just a demo of sensors on a display. It is a small object you can pick up, move, and play with — a tactile companion that feels alive because physics, touch, and motion are tightly coupled.
The firmware runs on an ESP32-S3 Super Mini with a 240 × 320 ILI9341 touch display and a BMI160 6-axis IMU. It is developed in PlatformIO with the Arduino framework, using TFT_eSPI for rendering and LittleFS for persistent high-score storage.
The current build targets an ESP32-S3 Super Mini (4 MB flash, 2 MB PSRAM) wired to a rectangular ILI9341 SPI display with an XPT2046 resistive touch controller.
Main components:
ESP32-S3 Super Mini (N4R2)
ILI9341 TFT display, 240 × 320 pixels, SPI + touch
BMI160 IMU on I2C
LiPo battery with voltage divider on ADC
ESP32-S3 GPIO
Display pin
11 MOSI
12 SCLK
13 MISO
9 DC
8 RST
5 CS
4 Backlight (BL)
1 Touch CS
The display uses the HSPI port at 80 MHz. Color order is configured as BGR with byte swapping to match common ILI9341 modules.
ESP32-S3 GPIO
BMI160
6 SDA
7 SCL
The BMI160 provides accelerometer and gyroscope data. A complementary quaternion filter separates slow tilt (gravity) from fast motion (inertia).
ESP32-S3 GPIO
Function
2 Battery voltage via resistor divider (×2)
Battery percentage is estimated between 3.20 V (empty) and 3.79 V (full) and shown in the top bar alongside the frame rate.
This wiring can be reproduced on a generic ESP32-S3 dev board with a compatible ILI9341 touch module. The pin assignments are defined in Firmware/platformio.ini via TFT_eSPI build flags.
The firmware lives in the Firmware/ folder. Open it as a PlatformIO project.
Main libraries:
TFT_eSPI — display and touch rendering
BMI160_SensorAPI — IMU driver
ArduinoJson — included for future extensions
Main software blocks:
The project supports two application modes:
MODE_BALL — physics sandbox and match-3 game (default at startup)
MODE_EYES — animated eyes reacting to IMU data
Within ball mode, two game modes are available from the menu:
Game mode — score, color matching, explosive balls, game over timer
Simulation mode — physics only, no scoring or game over
The firmware follows a simple loop: read touch input, process IMU data, then render the active mode.
Touch handling runs every frame with priority order: menu clicks first, then the gear icon, then game gestures (grab, drag, throw, delete zone).
The IMU module uses a complementary filter (98% gyro, 2% accelerometer) to maintain orientation as a quaternion. From that, it produces:
a gravity vector used by the ball physics
gyro rates used by the eye deformation system
This separation is important: tilting the device slowly changes gravity direction, while shaking it adds inertia without confusing the two effects.
Ball simulation uses a fixed-timestep impulse-based 2D engine running at 100 Hz with 2 substeps per frame. This keeps collisions stable even when the display frame rate varies.
Constants (defined in physics_engine.h):
Ball radius: 35 px
Restitution (bounce): 0.7
Air friction: 0.99
Gravity force: 1800
Inertia sensitivity: 5.0
Each frame, the engine:
Accumulates real elapsed time (capped at 0.1 s to avoid the “spiral of death”)
Filters the IMU gravity vector (EMA α = 0.3)
Computes linear acceleration as gravity_vec − filtered_gravity
Applies gravity + inertia forces to each active ball
Integrates position and velocity
Resolves wall collisions and ball–ball collisions
Updates explosion particles
gravity_vec + filtered gravityapply forces + integrateclamp + bounceresolve_collision impulsesquash_v updated on impactBMI160physics_update_fixedBall stateScreen edges
Ball–ball collisions follow a classic impulse resolution (similar to simplified Box2D):
Positional correction (80% of overlap) to prevent balls sinking into each other
Impulse along the collision normal if objects are approaching
Visual squash state updated from impact strength
The physics module was extracted from ball_animation.cpp into physics_engine.cpp, with a standalone collision test in Firmware/test/physics_engine_collision_test.cpp.
The project uses two complementary deformation systems — one for balls, one for eyes.
Collisions do not deform the physics shape — balls remain circles of fixed radius. Instead, each ball carries a visual state:
squash_nx, squash_ny — impact direction
squash_v — impact strength
On render, the ball is drawn as a rotated ellipse:
compression up to 30% along the impact normal
stretch along the perpendicular axis to preserve approximate 2D area
fast decay (×0.9 per substep) for a short, punchy effect
Because TFT_eSPI has no native rotated filled ellipse, the firmware implements draw_rotated_ellipse() using scanline rasterization — one square root per horizontal line, O(height) per ball.
Eye deformation is parametric and proactive, driven by gyroscope and tilt rather than collision impulses:
fast rotation squints the leading eye and widens the trailing one
edge overflow against a virtual wall compresses width and increases height
idle animations add blinking, look-around, and subtle wobble
a “dizzy” state triggers after rapid yaw rotation
All deformation values pass through SmoothedValue EMA filters to avoid IMU jitter.
In Game mode, the screen becomes a physics-based match-3 puzzle.
Gameplay:
Up to 12 balls of different colors can exist on screen
Touch an empty area to spawn a new ball; touch a ball to grab it
Drag and release quickly to throw the ball with gesture velocity
Drag a ball to the top-left corner (50 × 50 px) to delete it without explosion
When 3 or more same-color balls touch (within 2.1× radius), they pop with a particle explosion
Larger groups score more: 100 base + 100 per extra ball
About 1 in 10 new balls is an explosive bomb (white)
Tap a bomb to arm it; after 2 seconds it detonates, destroying nearby balls (+50 pts each)
A 15-second patience timer resets after each match — if it expires, game over
High score is saved to /highscore.txt on LittleFS
In Simulation mode, the same physics runs without scoring, bombs, or game over. Useful for testing motion and collisions.
The in-game menu (gear icon, top-right) lets you switch between Game and Simulation modes and reset the high score.
Eyes mode renders a pair of cartoon eyes on a dedicated sprite band. The eyes:
follow device tilt
squash and stretch with gyroscopic motion
blink and look around when idle
enter a dizzy state after fast rotation
A secondary laser target mode can be activated for a charging-and-firing animation tied to IMU orientation.
Mode switching between BALL and EYES is implemented in code (current_mode in main.cpp). The on-screen toggle button is currently commented out in the touch handler but can be re-enabled easily.
Touch gestures are handled in display_handler.cpp:
Gesture / Action
Tap empty space / Create or grab nearest ball (40 px radius)
Drag / Move ball with finger
Quick release (< 100 ms history) / Throw with computed velocity
Release in top-left 50×50 zone / Delete ball silently
Tap explosive ball / Arm bomb fuse
Tap gear icon / Open / close menu
Tap during game over / Restart game
A 200 ms debounce prevents accidental double-taps after menu interactions.
The ball mode uses full-screen double buffering via a TFT_eSprite canvas pushed each frame. This eliminates flicker on the SPI display.
The top bar shows:
battery voltage and percentage
current FPS
Typical performance on ESP32-S3 is around 20–25 FPS with multiple balls, particles, and UI. The main rendering cost comes from rotated ellipses and particle drawing. The physics engine itself is lightweight compared to rendering.
The project is built with PlatformIO. From the repository root:
pio run -d Firmware
pio run -d Firmware -t upload
Or in one step:
pio run -d Firmware -t upload
To specify the USB port:
pio run -d Firmware -t upload --upload-port COMx
Serial monitor (115200 baud):
pio device monitor -b 115200 -d Firmware
PlatformIO environment: esp32-s3-supermini-n4r2 (see Firmware/platformio.ini).
On first upload, LittleFS is used automatically for high-score storage. No separate filesystem upload is required unless you add assets under data/.
Companion_portable_ESP32/
└── Firmware/
├── platformio.ini # Board, pins, libraries
├── include/
│ ├── physics_engine.h # Ball/Particle structs, collision API
│ ├── ball_animation.h
│ ├── eyes_animation.h
│ ├── display_handler.h
│ ├── imu_handler.h
│ └── ...
├── src/
│ ├── main.cpp # Main loop
│ ├── physics_engine.cpp # Fixed-timestep simulation
│ ├── ball_animation.cpp # Game + rendering
│ ├── eyes_animation.cpp # Eye deformation
│ └── ...
└── test/
└── physics_engine_collision_test.cpp
Many ESP32 demos show sensor values as numbers on a screen. This project goes further: the IMU directly drives a playable physics world. Tilt becomes gravity. Shake becomes inertia. Touch becomes interaction. The result feels like a small companion object rather than a sensor dashboard.
The same hardware stack could live inside a handheld toy, a desk gadget, or a robot face prototype. The eyes mode shares DNA with other Intellar animated-eye projects such as the Animated eye OLED demo, while the ball game explores real-time physics and game logic on a microcontroller.
For a more integrated hardware platform with FPC satellite boards, see the Intellar Engine — the companion firmware could be adapted to that ecosystem in a future revision.
Working today:
ILI9341 touch display on ESP32-S3 Super Mini
BMI160 gravity and inertia for ball physics
Impulse-based collision engine with squash & stretch rendering
Match-3 game with bombs, score, and persistent high score
Simulation mode for physics-only play
Animated eyes with IMU-driven deformation
Battery monitoring and FPS overlay
Extracted physics module with collision tests
Known limitations:
Frame rate drops with many balls and particles (rendering bound)
BALL / EYES mode switch is in code but the on-screen button is disabled
No wireless control yet (unlike the Intellar Engine Bluetooth dashboard)
High score writes to flash on every match (could be debounced)
Planned improvements:
Re-enable mode toggle button on screen
Pre-rendered ball sprites to reduce ellipse cost
Optional Bluetooth control for remote play
Port to Intellar Engine satellite display board
GitHub: https://github.com/intellar/ImpulsePlay
Intellar store: https://intellar.square.site/
Related project — Animated eye OLED: https://www.intellar.ca/blog/animated-eye-oled
Related project — Intellar Engine: https://www.intellar.ca/blog/intellar-engine-satellite-boards
This project is not possible without your support: