Any tips or suggestions from folks who've tried similar approaches?
I would suggest a decomposition approach building specification or requirements documents for components one at a time with multiple passes using whatever output you have from figma as context.
Opus, o3(o5) and gemini pro are good at this. I instructed the model to build a comprehensive specification.md for the components (eg UI, data transform, data export) one at a time and instructed the model to ask me questions where anything is unclear or ambiguous. Once you have the component spec documents, you can go straight into implementation if you feel good about it or have a model create requirements.md (specify must, must not verbiage) -> architecture.md (specify complete architecture document but without any code) -> implementation.md (specify a complete stand alone implementation document for the programmer to use that contains all code and instructions) using the specification documents as context then give it to any frontier model to implement. Add as many screenshots as you have to represent UI components.
This has worked well for me when the documentation I have is insufficent.