PIXY
An interactive campaign website featuring an AR-based lip filter to promote PIXY's new lip vinyl collection.
Role
Year
Deliverable
- Interactive Campaign Website
- AR Lip Filter Feature
Website
Tech Stack
Core Challenge
- Overcoming the high friction and massive drop-off rates of traditional app-store dependencies by delivering a fully browser-based virtual trial.
- Executing computationally heavy, real-time 3D facial landmark tracking and AR rendering natively inside mobile browsers without causing UI freezes or overheating mid-range devices.
Key Impact
- Successfully delivered an immersive, zero-friction web AR 'Fitting Lip Room' that allowed instant virtual product try-ons directly from social media links.
- Achieved exceptionally high user dwell times and campaign engagement by seamlessly merging a nostalgic retro-desktop UI with cutting-edge web computer vision.
Lessons Learned
- Web-based computer vision is fully production-ready when coupled with aggressive lifecycle optimizations—isolating heavy tracking loops prevents thread bottlenecks.
- Gamifying the consumer discovery flow with a personalized personality quiz before the AR experience significantly improves conversion and intent-to-purchase.
The Friction of App-Store Dependencies
When launching the new "Hyperlast Glazed Lip Vinyl" collection, PIXY wanted to move beyond passive digital marketing and offer consumers an interactive way to discover their perfect shade. Traditional virtual try-on experiences typically require downloading a dedicated mobile application. However, for a fast-moving retail campaign, forcing a consumer to navigate to an app store creates immense friction, leading to catastrophic drop-off rates.
The goal was clear: architect a high-fidelity, zero-download Augmented Reality (AR) "Fitting Lip Room" running natively inside standard mobile web browsers. Consumers needed to click a link on social media, immediately grant camera permissions, and see the lip vinyl realistically rendered on their face in real-time.
The campaign landing page featuring an immersive retro-desktop interface designed to gamify the consumer journey.
Engineering Real-Time Computer Vision in the Browser
To bypass native app constraints, I designed a performance-first architecture leveraging web-based machine learning and hardware-accelerated rendering.
- Precision Tracking via MediaPipe FaceMesh: I implemented Google’s MediaPipe FaceMesh to map 468 3D facial landmarks in real-time. By isolating the specific matrix arrays governing the inner and outer lip boundaries, we constructed a highly responsive dynamic mesh. This programmatic mask flawlessly deforms and moves alongside the user's natural expressions, ensuring the lipstick texture remains perfectly mapped whether they smile, speak, or pout.
- Thread Isolation & Frame-Rate Optimization: Web AR is notoriously heavy on mobile CPUs. To prevent UI freezing and maintain a smooth frame rate on mid-range smartphones, I optimized the Next.js rendering loop. By decoupling the machine learning inference pipeline from the main UI thread, the application preserves critical device resources, allowing the camera feed and the interactive retro interface to remain highly fluid.
- Gamified Discovery Pipeline: Beyond the AR technical layer, we structured the user journey around an interactive retro personality quiz. This kept the onboarding experience engaging while gathering consumer preferences, directly feeding tailored shade recommendations into the final checkout stage.
The end-to-end user discovery flow: from a gamified personality quiz to real-time browser-based AR try-ons and personalized shade results.
Bridging Nostalgia with Modern Technology
The PIXY campaign beautifully proved that complex computer vision tasks no longer require native applications to be highly effective. By wrapping cutting-edge machine learning inside a playful, nostalgic 90s desktop aesthetic, we completely removed consumer barrier-to-entry.
The campaign delivered a massive surge in average session duration compared to static product pages. More importantly, it established a new benchmark for web-based experiential commerce, demonstrating how smart architecture can make advanced technology deeply accessible, lightweight, and incredibly fun.
Behind the scenes: Gathering real-time user feedback and observing consumer interaction during the live campaign launch.