A recent development in the field of software design and development involves tldraw's new feature, "Make It Real." This tool, incorporating OpenAI's GPT-4V API, offers the ability to convert vector drawings into functioning software code, specifically using Tailwind CSS and JavaScript.
Technical Overview of "Make It Real"
"Make It Real" operates by interpreting vector-based sketches and translating them into web code. The underlying technology is OpenAI's GPT-4V, a variant of the GPT-4 model designed to process visual inputs. The system works by generating a base64 encoded PNG of the drawn components and then passing that to GPT-4 Vision. The model then interprets these visuals and generates corresponding HTML, Tailwind CSS, and JavaScript code. The whole magic lies within a single prompt:
You are an expert web developer who specializes in tailwind css.
A user will provide you with a low-fidelity wireframe of an application.
You will return a single html file that uses HTML, tailwind css, and JavaScript to create a high fidelity website.
Include any extra CSS and JavaScript in the html file.
If you have any images, load them from Unsplash or use solid colored retangles.
The user will provide you with notes in blue or red text, arrows, or drawings.
The user may also include images of other websites as style references. Transfer the styles as best as you can, matching fonts / colors / layouts.
They may also provide you with the html of a previous design that they want you to iterate from.
Carry out any changes they request from you.
In the wireframe, the previous design's html will appear as a white rectangle.
Use creative license to make the application more fleshed out.
Use JavaScript modules and unkpkg to import any necessary dependencies.Respond ONLY with the contents of the html file.
The utilization of "Make It Real" requires an API key from OpenAI, which presents potential security concerns, if you run it in the online demo. However, you can also run it locally on your machine.
Practical Use Cases and User Experiences
Users have applied "Make It Real" in diverse scenarios, ranging from creating interactive sliders and games to developing web interfaces. For instance, demonstrations include the creation of a tic-tac-toe game and interactive UI elements. User feedback, like Ashe Oro’s account of iterating multiple builds affordably, indicates the tool's practicality and cost-effectiveness. Here are some other examples:
let's go https://t.co/4dHJiLCegg pic.twitter.com/4y9VMiTN0h
— tldraw (@tldraw) November 15, 2023
state charts + wireframes + annotations pic.twitter.com/vBciMZ49yf
— tldraw (@tldraw) November 16, 2023
The Development Background of tldraw
Steve Ruiz, based in London, developed tldraw. Since its launch in 2021, the project has received significant funding and is supported by GitHub sponsors. The integration of the GPT-4V API, a version of OpenAI's model capable of interpreting visual images, was a key development in enabling the "Make it Real" feature.
Potential Impact on Software Development
"Make It Real" introduces a novel method of software development, where visual designs can be swiftly transformed into functional prototypes. This approach suggests a shift in traditional software development processes, highlighting the increasing role of AI in rapid prototyping and design-to-code conversion.
Conclusion
The introduction of "Make It Real" by tldraw marks a noteworthy development in the integration of AI in software design and development. It showcases the potential for AI to streamline and revolutionize traditional processes, although it also raises considerations regarding security and accessibility. As the technology evolves, it may significantly alter the landscape of software development and design.
Unlock the Future of Business with AI
Dive into our immersive workshops and equip your team with the tools and knowledge to lead in the AI era.
