Beginner’s Guide: Building a Generative UI Chat App with AI Coding Assistants (AER & Cursor)

Introduction

Welcome to the exciting world of generative AI and its application in UI development! In this tutorial, we’ll walk through the process of building a simple generative UI client-server application. This project will enable you to prompt specific UI components through a chat interface. We’ll be using a combination of cutting-edge AI coding tools: AER and Cursor.

Why This Tutorial?

Generative UI is an emerging pattern that allows the dynamic generation of UI elements based on user input. This is a powerful technique that enhances user experience and simplifies UI development. By the end of this tutorial, you’ll learn how to:

• Set up a basic client-server application with a chat interface
• Use AI coding assistants (AER and Cursor) to generate code
• Implement dynamic UI components based on user input
• Understand the workflow of using AI agents for coding tasks

Estimated Time: 1.5 - 2 hours

Prerequisites:

• Basic knowledge of JavaScript
• Familiarity with Node.js and npm
• Basic understanding of Vue.js
• Familiarity with git and the command line

1. Setting Up the Project Plan

Before we dive into code, it’s crucial to have a plan. Our plan consists of three main sections:

1. High-Level Overview: The broad steps to achieve our goal.
2. Team: Our set of AI agents and their responsibilities.
3. Task Breakdown: Specific tasks with high-level objectives and detailed AI coding prompts.

High-Level Overview

1. Set up client (Vue.js) and server (Express.js) applications.
2. Create a chat interface on the client side.
3. Implement a server endpoint that responds with generative UI components based on user input.
4. Implement dynamic UI components using Vue.js.
5. Collect documentation and load into our context
6. Refine and enhance the application with more features

Our AI Team

• AER Client: Manages the client-side (Vue.js) code using Architect Mode.
• AER Server: Manages the server-side (Express.js) code using Architect Mode.
• Anthropic Document Editor Agent: Handles documentation processing and file edits.
• Anthropic Bash Agent: Executes terminal commands for database setup and other system-level tasks.

Task Breakdown

Our tasks are broken down into setup and individual coding prompts. Each task contains a high-level objective and a low-level detailed prompt for the AI agent.

2. Initial Setup and Context Management

2.1 Initializing AI Agents

We will use two AER instances (one for client and one for server), and then two more anthropic powered agents one for editing files and other for running bash scripts:

• Client Editor (AER): This is for our front end components.
• Server Editor (AER): This agent will work on our server side code.
• Document Editor: This agent will be used for cleaning up documentation.
• Bash Agent: This is used for running CLI commands.

We will start by adding our server and client code to the respective AER instances by using add directory commands like add server/ and add source/. This is to make sure that the agents understand the context of the codebase, also this makes files editable inside the agent.

2.2 Setting Read-Only Context

To prevent accidental edits in areas outside the current focus, we set read-only contexts. The following will be added as read-only:

• Server files as read-only for the client agent with readon server/
• Client files as read-only for the server agent readon source/
• The genui component as read-only for server agent readon genui.ts

2.3 Documentation Collection

We will use the Anthropic’s tool-use capabilities to gather documentation for our project. We will specifically be gathering the following:

• Tool use documentation from Sonnet
• Vue.js dynamic components documentation

We’re using a custom script collect to scrape documentation from URLs and store it in our local AI docs directory. The raw markdown files will first be saved as tool-use-raw.md and dynamic-components-raw.md in AI docs. These files are then cleaned up using our file editor agent which generates cleaned up files tool-use.md and dynamic-components.md respectively. This ensures we have a clean, readable format that AI agents can understand.

Here’s how we use the Bash agent and file editor agent:

# Example Bash command to collect documentation
collect <URL> > AI_docs/tool-use-raw.md
 
# Example prompt to clean up the documentation
Read tool-use-raw.md and then create tool-use.md with examples and docs specifically around tool use

The file editor agent will use the content from tool-use-raw.md to create a new, cleaner file named tool-use.md. This process is also repeated for the Vue.js dynamic component documentation using different file names.

After this, we commit our changes using git add . and git commit -m "Add documentation" so that our AER instances can access these readon and doc files.

3. Implementing the Generative UI

3.1 Creating the Tool Endpoint

Our first step is to create a new tool endpoint in our server application. This endpoint will be responsible for receiving the user’s prompt and returning the appropriate UI component type using Sonet.

• Action: Update server to create a new tool endpoint.
• Prompt: Create a new endpoint /tool that takes a tool parameter and uses Sonet tool calling to return a component type (text, star rating, color picker, contact form)

// Example server-side code (using Express.js)
 
app.post('/tool', async (req, res) => {
    const { tool } = req.body;
    const response = await getToolResponse(tool);
    res.json(response);
});
 
async function getToolResponse(prompt: string) {
    const response = await openai.chat.completions.create({
      model: 'claude-3-sonnet-20240229',
      messages: [
          {role: "user", content: prompt},
      ],
        tools: [
            {
                type: "function",
                function: {
                    name: "getComponentType",
                    description: "Gets the component type to use for the prompt",
                    parameters: {
                      type: "object",
                      properties: {
                          componentType: {
                            type: "string",
                            enum: ["text", "starRating", "colorPicker", "contactForm"]
                          }
                      },
                      required: ["componentType"]
                  }
              }
          ],
        tool_choice: {
            type: "function",
            function: {
                name: "getComponentType"
            }
        }
    });
    return response.choices[0].message
  }
 
 

• Explanation: The server now has a new endpoint /tool that is called with tool parameter. This parameter is passed into the getToolResponse which calls the AI model with the tool definition. The tool will then return the componentType which will be used to generate UI components.
• Code Change: The code adds a new /tool route that handles POST requests and passes the user prompt to a function that uses the OpenAI API to get a component type. The response is sent back as JSON.
• Note: AER’s architect mode is used, where AER will draft the changes and then pass it to an editor model to apply the changes. This enhances the accuracy of the coding. Also, any linting issues (such as missing imports) are auto-fixed by AER.

3.2 Integrating the Tool Call in the Client

Next, we update the client-side code to call this /tool endpoint after the enter key is pressed, which will then trigger UI generation.

• Action: Update client to use /tool endpoint.
• Prompt: Modify the client to call /tool after enter key is pressed, passing the user’s prompt.

// Example client-side code (using Vue.js)
<script setup lang="ts">
import { ref } from "vue";
import axios from "axios";
 
const messages = ref([]);
const newMessage = ref("");
 
const sendMessage = async () => {
  try {
    const response = await axios.post("/tool", { tool: newMessage.value });
    messages.value.push({
      id: generateId(),
      created_at: Date.now(),
      type: "user",
      text: newMessage.value,
    });
    messages.value.push({
      id: generateId(),
      created_at: Date.now(),
      type: "bot",
      text: null,
      componentType:
        response.data.tool_calls[0].function.arguments.componentType,
      componentProps: response.data.tool_calls[0].function.arguments,
    });
    newMessage.value = "";
  } catch (error) {
    console.error("Error sending message:", error);
  }
};
 
function generateId() {
  return (
    Math.random().toString(36).substring(2, 15) +
    Math.random().toString(36).substring(2, 15)
  );
}
</script>
 
<template>
  <div class="chat-container">
    <div class="chat-messages">
      <div
        v-for="message in messages"
        :key="message.id"
        class="message"
        :class="{ 'user-message': message.type === 'user' }"
      >
        <div v-if="message.type === 'user'">{{ message.text }}</div>
        <div v-else>
          <component
            :is="message.componentType"
            v-if="message.componentType"
            v-bind="message.componentProps"
          >
          </component>
          <div v-else>
            {{ message.text }}
          </div>
        </div>
      </div>
    </div>
    <div class="chat-input">
      <input
        type="text"
        v-model="newMessage"
        @keyup.enter="sendMessage"
        placeholder="Type your message..."
      />
    </div>
  </div>
</template>

• Explanation: The client now sends the user message to the /tool endpoint. The response contains the componentType and any additional props, which we’ll use to create the correct UI components dynamically.
• Code Change: The code modifies the sendMessage method to call /tool and store the response. The data returned includes the componentType, which will be used later to render the appropriate component.
• Note: Initially, we had to correct the response format and adjust the code to parse the response from the tool.

3.3 Enhancing the Message Data

Now, we’ll update the client-side message data structure to store more information for each message, such as ID, timestamp, component type, and more.

• Action: Update the message type on the client side.
• Prompt: Enhance the messages type to include id, created_at, type, text, response, and submitted. Create a new ID generation method.

// Example of a new client side interface
interface Message {
  id: string;
  created_at: number;
  type: "user" | "bot";
  text: string | null;
  response?: string;
  submitted?: any;
  componentType?: string;
  componentProps?: any;
}

• Explanation: The Message type is updated with additional fields to store more information about each message, including an ID, timestamp, and other properties.
• Code Change: The code adds new properties to the Message type in the client-side code and generates a new id when creating a message.
• Note: This change provides more structure and control over each chat message

4. Dynamic UI Components

4.1 Generating UI Components

We’ll now instruct our AI coding assistant to create individual UI components based on the component types we received from our /tool endpoint.

• Action: Generate new UI components.
• Prompt: Create the source components genui-text, genui-star, genui-contact, and genui-picker and update the client to use the dynamic components in the response instead of raw text.

// Example of a dynamic component inside the template
<component
  :is="message.componentType"
  v-if="message.componentType"
  v-bind="message.componentProps"
/>

• Explanation: The AI coding assistant will now create four new components in our source folder based on the provided prompts. These components include a text input, a star rating, a color picker, and a contact form. The top level ui will use a dynamic component, rendering the right component based on the componentType.
• Code Change: Four new component files are generated with all the necessary styling and the logic for the component. Additionally, the root level vue component has a dynamic component that shows the correct component based on the returned value from the server.
• Note: It is crucial that we provided a clean and well-formatted documentation to our AI agent on how to use dynamic components.

4.2 Correcting Component Response

After a quick test run, we realize that we need to change how the props get passed into the component. This is corrected using the following:

• Action: Ensure that the component props get set correctly on the client side
• Prompt: Update the client to parse and set the component type and props correctly based on the tool’s output

messages.value.push({
  id: generateId(),
  created_at: Date.now(),
  type: "bot",
  text: null,
  componentType: response.data.tool_calls[0].function.arguments.componentType,
  componentProps: response.data.tool_calls[0].function.arguments,
});

• Explanation: The client now pulls both the componentType and the componentProps from the tool’s output.
• Code Change: We modify the code block that handles the bot’s response to assign both the componentType and the componentProps using the output from the tool call.

5. Enhancing the UI and Data Submission

5.1 Removing Submit Button

For the text component, we want to remove the submit button to make it act more like a chat message.

• Action: Remove the submit button from the genui-text component.

• Prompt: Remove submit from genui-text

• Explanation: We remove the submit button from the text component so that it looks like a normal text chat.

• Code Change: The genui-text component is updated to remove the submit button and logic.

5.2 Handling Dynamic Submissions

We need to update how the app handles submissions for our dynamic components. When the user submits information we need to save it and update the UI.

• Action: Update the client side to handle submissions of all the components
• Prompt: Update the client to save all the responses into submitted and update the UI with the state.

// Example component submit method
const submitValue = (value: any) => {
  const tempBotMessage = messages.value[messages.value.length - 1];
  tempBotMessage.submitted = value;
};

• Explanation: This change allows users to submit the values from the UI components and updates the UI accordingly.
• Code Change: Each component now calls the parent submitValue function which will update the state in the client app.

5.3 Hiding Response Submission text

We want to ensure that our UI does not show the submit text unless a value has been submitted. This is done with a simple update.

• Action: Hide the submit text on the UI unless there is a value.

• Prompt: The submitted response should not be shown unless there is a value

• Explanation: This will conditionally render the submission text only after an actual value has been set

• Code Change: This modifies the template for the bot response to conditionally render the response text.

6. Database Setup and Persistence (Bash Agent)

6.1 Setting up a SQLite Database

To store our chat messages, we’ll use a SQLite database. We’ll use the bash agent to set up the database and a new table.

• Action: Use the bash agent to set up a SQLite database
• Prompt: Create a new SQLite database at server/app.db with a table named messages that matches the message structure in the client-side code, with columns for ID, created_at, type, text, response and submitted values.

# Example command for bash agent
sqlite3 server/app.db "
CREATE TABLE messages (
    id TEXT PRIMARY KEY,
    created_at INTEGER,
    type TEXT,
    text TEXT,
    response TEXT,
    submitted TEXT
);
"

• Explanation: This prompt instructs the bash agent to create the necessary SQL database in our server directory.
• Code Change: The bash agent executes the given SQL command to create a new database and table, which is then validated to ensure the table was created with the correct schema.

Conclusion and Next Steps

In this tutorial, we’ve successfully built a simple generative UI chat application using AI coding tools like AER and Cursor. We’ve covered:

• Setting up a client-server application
• Using AI agents for code generation
• Implementing dynamic UI components
• Integrating documentation and data storage

Next Steps

To further enhance this application, you can:

• Implement load and save commands to persist the data.
• Add more advanced UI components.
• Improve the user interface and experience.
• Utilize more advanced architect patterns to handle code generation.

This project demonstrates the potential of AI coding assistants to significantly speed up and enhance the development process. We encourage you to continue experimenting and exploring new possibilities with these tools.

Thank you for following along, and we look forward to seeing what you build next!