When brainstorming ideas for a fun project to showcase feature flags, this coffee shop fantasy kept nudging me: "Pick me! Build me!". So, I decided to indulge that thought and create Brew Haven, a dummy coffee shop app that explores the power of feature flags.

This project was originally created as part of a dev challenge to showcase the power of feature flags in an engaging way.

Project Overview

As you might have guessed, I built a playground to explore the power of feature flags, packaged as a coffee shop app. The app features customizable menus, seasonal items, and dynamic A/B testing for promotions using the DevCycle SDK. It also leverages the DevCycle Management APIs to power an admin panel, giving shop managers full control over these features in real time.

Demo

The following video gives a short walkthrough of the app.

You can try out the live app here: Brew Haven

Note: The admin page uses a dummy password auth. Use admin@123 passowrd to view the admin panel and play around with the available feature flags.

Technologies Used

To bring Brew Haven to life, I used the following:

• React with Vite for a fast and modular frontend.

• Shadcn UI for styling and components.

• Netlify Functions for secure serverless DevCycle Management API calls.

• DevCycle SDK for feature flag integration on the client side.

What is DevCycle?

DevCycle is a feature management platform that helps developers experiment with new features, run A/B tests, and gradually roll out updates without downtime. It uses feature flags—switches in your code that let you turn specific features on or off in real time.

With DevCycle, you can:

• Deploy features safely using controlled rollouts.

• Customize user experiences through A/B testing.

• Quickly respond to user feedback without redeploying your code.

DevCycle integrates seamlessly into your development process, making feature management simple and efficient. In Brew Haven, I used it to power dynamic menu customization, promotions, and more.

How Does DevCycle Work?

DevCycle is a powerful tool designed to simplify the management of feature flags and provide advanced functionality for tailoring your software’s behavior. Here’s an overview of how it works:

1. Feature Flags and Feature Types
   At the heart of DevCycle are features, which can have one or more toggles (or flags). Features are categorized into four types, tailored for specific use cases:

   • Release: Designed for separating code deployment from feature release. This enables merging incomplete or in-progress code into production without affecting end users until it’s toggled on.

   • Ops: Helps ensure system safety during feature rollouts, with built-in mechanisms for gradual rollouts or emergency kill switches.

   • Experiment: Ideal for A/B or multivariate testing. It lets you distribute users across variations and track outcomes to make data-driven decisions.

   • Permission: Gating features based on user attributes, such as subscription plans or roles, allowing granular control over feature access.

2. Variables and Variations
   Each feature flag can have associated variables, representing configurable values. For example, a flag for a promotion might include variables like discountPercentage or minimumOrderValue.
   Variations are pre-defined combinations of variable values, which dictate how the feature behaves. For instance:

   • Variation A might offer a 10% discount.

   • Variation B might provide a 15% discount with a higher order value threshold.

3. Targeting Rules and Rollouts
   DevCycle allows you to define rules that control which users see specific variations. These rules can be based on user properties (like location or plan type) or random distribution for A/B testing. Features can also be rolled out gradually, allowing you to monitor performance and ensure stability before scaling.

This structured approach ensures safe experimentation, seamless feature rollouts, and precise customization—all with minimal risk to your users’ experience.

Feature Flags in Brew Haven

Brew Haven uses the following feature flags to make the app dynamic and customizable:

1. Coffee Menu: Feature flags control menu customization, seasonal items, and order personalization.

2. Payment & Ordering: Enable or disable online payments, loyalty points, and live order tracking with simple toggles.

3. A/B Testing Promotions: Run experiments on discounts and promotional offers to optimize customer engagement.

The first two are release type feature flags, while the last one is an experiment type feature. These flags not only made development faster but also showcased the versatility of DevCycle.

Integrating DevCycle

The source code of the app is open source, and is available on GitHub. We’ll briefly look at how to integrate and use DevCycle in a React App.

Client Side Usage of DevCycle SDK

After adding the DevCycle React SDK dependency, we first initialize the DevCycleProvider in the App.tsx file as shown below:

// src/App.tsx

import { withDevCycleProvider } from "@devcycle/react-client-sdk";
// ...

function App() {
  return (
    <ThemeProvider defaultTheme="system" storageKey="coffee-shop-ui-theme">
      <Router>
        <Layout>
          <Routes>
            <Route path="/" element={<Home />} />
            <Route path="/menu" element={<Menu />} />
            <Route path="/checkout" element={<Checkout />} />
            <Route path="/orders" element={<Orders />} />
            <Route
              path="/admin"
              element={
                <ProtectedRoute>
                  <Admin />
                </ProtectedRoute>
              }
            />
          </Routes>
        </Layout>
      </Router>
      <Toaster />
    </ThemeProvider>
  );
}

export default withDevCycleProvider({
  sdkKey: import.meta.env.VITE_DEVCYCLE_CLIENT_SDK_KEY,
  options: {
    logLevel: "debug",
  },
})(App);

And then we create a useFeatureFlags hook to get the various feature flags variables associated with the app as shown below:

// src/hooks/use-feature-flags.ts

import { useVariableValue } from "@devcycle/react-client-sdk";
import { featureKeys } from "@/lib/consts";

export function useFeatureFlags() {
  const showNutritionInfo = useVariableValue(
    featureKeys.SHOW_NUTRITION_INFO,
    false,
  );
  const enableOnlinePayment = useVariableValue(
    featureKeys.ENABLE_ONLINE_PAYMENT,
    false,
  );
  const showPromotionalBanner = useVariableValue(
    featureKeys.SHOW_PROMOTIONAL_BANNER,
    "",
  );

  // etc.

  return {
    showNutritionInfo,
    enableOnlinePayment,
    showPromotionalBanner,
    // ...
  };
}

This hook is used in the app to toggle various code sections on/off to change the UI.

Interacting with DevCycle Management APIs

For securely calling the Management APIs, we use Netlify serverless functions so that the DevCycle API's client_id and client_secret are not exposed to the client.

Before we can interact with the DevCycle APIs, we need to generate an auth token using the DevCycle APIs client id and secret. The below code shows how to generate the auth token:

// netlify/functions/feature-flags.mts

interface AuthToken {
  access_token: string;
  expires_in: number;
  token_type: string;
}

let tokenCache: { token: string; expiresAt: number } | null = null;

async function getAuthToken() {
  if (tokenCache && tokenCache.expiresAt > Date.now()) {
    return tokenCache.token;
  }

  try {
    const response = await fetch("https://auth.devcycle.com/oauth/token", {
      method: "POST",
      headers: {
        "Content-Type": "application/x-www-form-urlencoded",
      },
      body: new URLSearchParams({
        grant_type: "client_credentials",
        audience: "https://api.devcycle.com/",
        client_id: process.env.DEVCYCLE_API_CLIENT_ID!,
        client_secret: process.env.DEVCYCLE_API_CLIENT_SECRET!,
      }),
    });

    if (!response.ok) {
      throw new Error(`Auth failed: ${response.status}`);
    }

    const data: AuthToken = await response.json();

    tokenCache = {
      token: data.access_token,
      expiresAt: Date.now() + data.expires_in * 1000 - 5 * 60 * 1000,
    };

    return data.access_token;
  } catch (error) {
    console.error("Failed to get auth token:", error);
    throw error;
  }
}

Now we can fetch the available feature flags, and their config (to get the currently active variation) using the below code:

// netlify/functions/feature-flags.mts

interface Distribution {
  _variation: string;
  percentage: number;
}

interface FeatureConfig {
  _feature: string;
  _environment: string;
  status: string;
  targets: {
    _id: string;
    name: string;
    distribution: Distribution[];
    audience: {
      name: string;
      filters: {
        operator: "and" | "or";
        filters: { type: string }[];
      };
    };
  }[];
}

async function getFeatures(
  featuresUrl: string,
  headers: Record<string, string>,
) {
  const featuresResponse = await fetch(featuresUrl, { headers });

  if (!featuresResponse.ok) {
    throw new Error(`Failed to fetch flags: ${featuresResponse.status}`);
  }

  const featuresData = await featuresResponse.json();

  // Fetch the config for each of the feature flags
  // We're only fetching the configs for the development env
  const configPromises = featuresData.map(async (feature: any) => {
    const configResponse = await fetch(
      `${featuresUrl}/${feature._id}/configurations?environment=development`,
      { headers },
    );

    if (!configResponse.ok) {
      console.error(`Failed to fetch config for feature ${feature._id}`);
      return null;
    }

    const configs: FeatureConfig[] = await configResponse.json();

    return {
      ...feature,
      targets: configs[0].targets,
      status: configs[0].status,
    };
  });

  const featuresWithConfigs = await Promise.all(configPromises);
  return featuresWithConfigs.filter((f) => f !== null);
}

To update the feature flags config (changing the variation, or toggle the flag altogether), we can use the below function:

async function updateFeature(
  featuresUrl: string,
  featureId: string,
  headers: Record<string, string>,
  update: any,
) {
  const response = await fetch(
    `${featuresUrl}/${featureId}/configurations?environment=development`,
    {
      method: "PATCH",
      headers,
      body: JSON.stringify(update),
    },
  );

  if (!response.ok) {
    throw new Error(`Failed to update feature: ${response.status}`);
  }

  return await response.json();
}

Finally, here is the Netlify function that uses the above functions to serve the client requests:

export default async (req: Request) => {
  try {
    const { method } = req;
    const token = await getAuthToken();
    const featuresBaseUrl = `https://api.devcycle.com/v1/projects/${process.env.DEVCYCLE_PROJECT_ID}/features`;
    const headers = {
      Authorization: `Bearer ${token}`,
    };

    if (method === "GET") {
      const features = await getFeatures(featuresBaseUrl, headers);

      return Response.json({ features });
    }

    if (method === "PATCH") {
      const body = await req.json();
      const { featureId, update } = body;

      const data = await updateFeature(
        featuresBaseUrl,
        featureId,
        {
          ...headers,
          "Content-Type": "application/json",
        },
        update,
      );

      return Response.json({ data });
    }

    return new Response(JSON.stringify({ error: "Method not allowed" }), {
      status: 405,
      headers: {
        "Content-Type": "application/json",
      },
    });
  } catch (error) {
    console.error("Error:", error);
    return new Response(
      JSON.stringify({
        error: error instanceof Error ? error.message : "Internal server error",
      }),
      {
        status: 500,
        headers: {
          "Content-Type": "application/json",
        },
      },
    );
  }
};

This Netlify function is called by the client in the following way:

// src/lib/api.ts

export async function getFeatureFlags() {
  try {
    const response = await fetch("/.netlify/functions/feature-flags");
    if (!response.ok) {
      throw new Error("Failed to fetch flags");
    }

    const data = await response.json();
    return { success: true, data: data.features as Feature[] };
  } catch (error) {
    return {
      success: false,
      error: error instanceof Error ? error.message : "Unknown error",
    };
  }
}

// and, so on...

The above code snippets capture how the DevCycle SDK and its Management APIs are used within the app. You can go through the shared source code to view the complete implementation in more detail.

Wrapping Up

Brew Haven isn’t just a coffee shop app — it’s a showcase of how feature flags can make your projects more dynamic, responsive, and fun.

Feature flags allow you to:

• Do gradual rollouts.

• Reduce deployment risks.

• Enable rapid experimentation.

• Personalize user experiences.

Next time you’re sipping coffee and dreaming big, think about how feature flags can brew innovation into your projects. ☕

Thank you for sticking with me until the end! I hope you’ve picked up some new concepts along the way. I’d love to hear what you learned or any thoughts you have in the comments section. Your feedback is not only valuable to me, but to the entire developer community exploring this exciting field.

Until next time.

Keep adding the bits, and soon you'll have a lot of bytes to share with the world.

Something about the simplicity of voice notes combined with the technical challenge intrigued me. Could I take this concept further? Could I build a modern, AI-powered voice notes app using web technologies? That urge to build led me here, to this blog post, where I’ll walk you through building Vhisper, a voice notes app with AI transcription and post-processing, built with the Nuxt ecosystem and powered by Cloudflare.

And before you say it, I must make a confession: “Hi! My name is Rajeev, and I am addicted to talking/chatting.”.

Project Overview

Now that the formalities are done, let’s focus on what we’ll be building in this project. The goal is to create Vhisper, a web-based voice notes application with the following core features:

• Recording Voice Notes: Users can record voice notes directly in the browser.

• AI-Powered Transcription: Each recording is processed via Cloudflare Workers AI, converting speech to text.

• Post-Processing with Custom Prompts: Users can customize how transcriptions are refined using an AI-driven post-processing step.

• Seamless Data Management (CRUD): Notes and audio files are efficiently stored using Cloudflare’s D1 database and R2 storage.

To give you a better sense of what we’re aiming for, here’s a quick demo showcasing Vhisper’s main features:

You can experience it live here: https://vhisper.nuxt.dev

By the end of this guide, you’ll know exactly how to build and deploy this voice notes app using Nuxt, NuxtHub and Cloudflare services—a stack that combines innovation with developer-first simplicity. Ready to build it? Let’s get started!

Project Setup

Before setting up the project let’s review the technologies used to build this app:

1. Nuxt: Vue.js framework for the application foundation

2. Nuxt UI (v3): For creating a polished and professional frontend

3. Drizzle: Database ORM

4. Zod: For client/server side data validation

5. NuxtHub: Backend (database, storage, AI etc.), deployment and administration platform for Nuxt

6. Cloudflare: Powers NuxtHub to provide various services

Prerequisites

To follow along, apart from basic necessities like Node.js, npm, and some Nuxt knowledge, you’ll need:

1. A Cloudflare account to use Workers AI and deploy your project. If you don’t have one, you can set it up here.

2. A NuxtHub Admin Account for managing apps via the NuxtHub dashboard. Sign up here.

Project Init

We’ll start with the NuxtHub starter template. Run the following command to create and navigate to your new project directory:

# Create project and change into the project dir
npx nuxthub init voice-notes && cd $_

If you plan to use pnpm as your package manager, add a .npmrc file at the root of your project with this line to hoist dependencies:

# .npmrc
shamefully-hoist=true

Now, install the dependencies:

1. Nuxt modules:

    pnpm add @nuxt/ui@next
   

2. Drizzle and related tools:

    pnpm add drizzle-orm drizzle-zod @vueuse/core
   

3. Icon packs:

    pnpm add @iconify-json/lucide @iconify-json/simple-icons
   

4. Dev dependencies:

    pnpm add -D drizzle-kit
   

Update your nuxt.config.ts file as follows:

export default defineNuxtConfig({
  modules: ["@nuxthub/core", "@nuxt/eslint", "nuxt-auth-utils", "@nuxt/ui"],

  devtools: { enabled: true },

  runtimeConfig: {
    public: {
      helloText: "Hello from the Edge 👋",
    },
  },

  future: { compatibilityVersion: 4 },
  compatibilityDate: "2024-07-30",

  hub: {
    ai: true,
    blob: true,
    database: true,
  },

  css: ["~/assets/css/main.css"],

  eslint: {
    config: {
      stylistic: false,
    },
  },
});

We’ve made the following changes to the Nuxt config file:

1. Updated the Nuxt modules used in the app

2. Enabled required NuxtHub features

3. And, added the main.css file path.

Create the main.css file in the app/assets/css folder with this content:

@import "tailwindcss";
@import "@nuxt/ui";

Testing the Setup

Run the development server:

pnpm dev

Visit http://localhost:3000 in your browser. If everything is set up correctly, you’ll see the message: “Hello from the Edge 👋” with a refresh button.

Building the Basic Backend

With the project setup complete, let’s dive into building the backend. We’ll begin by creating API endpoints to handle core functionalities, followed by configuring the database and integrating validation.

But before jumping to code, let’s understand how you’ll interact with various Cloudflare offerings. If you’ve been attentive, you should know the answer, NuxrHub, but what is NuxtHub?

What is NuxtHub?

NuxtHub is a developer-friendly interface built on top of Cloudflare’s robust services. It simplifies the process of creating, binding, and managing services for your project, offering a seamless development experience (DX).

You started with a NuxtHub template, so the project comes preconfigured with the @nuxthub/core module. During the setup, you also enabled the required Cloudflare services: AI, Database, and Blob. The NuxtHub core module exposes these services through interfaces prefixed with hub. For example, hubAI is used for AI features, hubBlob for object storage, and so on.

Time is ripe now to work on the first API endpoint.

/api/transcribe Endpoint

Create a new file named transcribe.post.ts inside the server/api directory, and add the following code to it:

// server/api/transcribe.post.ts 
export default defineEventHandler(async (event) => {
  const form = await readFormData(event);
  const blob = form.get("audio") as Blob;
  if (!blob) {
    throw createError({
      statusCode: 400,
      message: "Missing audio blob to transcribe",
    });
  }

  ensureBlob(blob, { maxSize: "8MB", types: ["audio"] });

  try {
    const response = await hubAI().run("@cf/openai/whisper", {
      audio: [...new Uint8Array(await blob.arrayBuffer())],
    });

    return response.text;
  } catch (err) {
    console.error("Error transcribing audio:", err);
    throw createError({
      statusCode: 500,
      message: "Failed to transcribe audio. Please try again.",
    });
  }
});

The above code does the following:

1. Parses incoming form data to extract the audio as a Blob

2. Verifies that it’s an audio blob and is less than 8MB in size using a @nuxthub/core utility function ensureBlob

3. Passes on the array buffer to the Whisper model through hubAI for transcription

4. Returns the transcribed text to the client

Before you can use Workers AI in development, you’ll need to link it to your Cloudflare project. As we’re using NuxtHub as the interface, running the following command will create/link a new or existing NuxtHub project with this project.

npx nuxthub link

/api/upload Endpoint

Next, create an endpoint to upload the audio recordings to the R2 storage. Create a new file upload.put.ts in your /server/api folder and add the following code to it:

// server/api/upload.put.ts
export default defineEventHandler(async (event) => {
  return hubBlob().handleUpload(event, {
    formKey: "files",
    multiple: true,
    ensure: {
      maxSize: "8MB",
      types: ["audio"],
    },
    put: {
      addRandomSuffix: true,
      prefix: "recordings",
    },
  });
});

The above code uses another utility method from the NuxtHub core module to upload the incoming audio files to R2. handleUpload does the following:

1. Looks for the files key in the incoming form data to extract blob data

2. Supports multiple files per event

3. Ensures that the files are audio and under 8MB in size

4. And, finally uploads them to your R2 bucket inside recordings folder while also adding a random suffix to the final names

5. Returns a promise to the client that resolves once all the files are uploaded

Now we just need /notes endpoints to create & fetch notes entries before the basic backend is done. But to do that we need to create the needed tables. Let’s tackle this in next section.

Defining the notes Table Schema

As we will use drizzle to manage and interact with the database, we need to configure it first. Create a new file drizzle.config.ts in the project root, and add the following to it:

// drizzle.config.ts
import { defineConfig } from 'drizzle-kit';

export default defineConfig({
  dialect: 'sqlite',
  schema: './server/database/schema.ts',
  out: './server/database/migrations',
});

The config above mentions where the database schema is located, and where should the database migrations be generated. The database dialect is set to sqlite as that is what Cloudflare’s D1 database supports.

Next, create a new file schema.ts in the server/database folder, and add the following to it:

// server/database/schema.ts
import crypto from "node:crypto";
import { sql } from "drizzle-orm";
import { sqliteTable, text } from "drizzle-orm/sqlite-core";

export const notes = sqliteTable("notes", {
  id: text("id")
    .primaryKey()
    .$defaultFn(() => "nt_" + crypto.randomBytes(12).toString("hex")),
  text: text("text").notNull(),
  createdAt: text("created_at")
    .notNull()
    .default(sql`(CURRENT_TIMESTAMP)`),
  updatedAt: text("updated_at")
    .notNull()
    .default(sql`(CURRENT_TIMESTAMP)`)
    .$onUpdate(() => sql`(CURRENT_TIMESTAMP)`),
  audioUrls: text("audio_urls", { mode: "json" }).$type<string[]>(),
});

The notes table schema is straightforward. It includes the note text and optional audio recording URLs stored as a JSON string array.

Finally, create a new file drizzle.ts in the server/utils folder, and add the following to it:

// server/utils/drizzle.ts
import { drizzle } from "drizzle-orm/d1";
import * as schema from "../database/schema";

export { sql, eq, and, or, desc } from "drizzle-orm";

export const tables = schema;

export function useDrizzle() {
  return drizzle(hubDatabase(), { schema });
}

Here we hook up hubDatabase with the tables schema through drizzle and export the server composable useDrizzle along with the needed operators.

Now we are ready to create the /api/notes endpoints which we will be doing in the next section.

/api/notes Endpoints

Create two new files index.post.ts and index.get.ts in the server/api/notes folder and add the respective codes to them as shown below.

index.post.ts

// server/api/notes/index.post.ts
import { noteSchema } from "#shared/schemas/note.schema";

export default defineEventHandler(async (event) => {
  const { user } = await requireUserSession(event);

  const { text, audioUrls } = await readValidatedBody(event, noteSchema.parse);

  try {
    await useDrizzle()
      .insert(tables.notes)
      .values({
        text,
        audioUrls: audioUrls ? audioUrls.map((url) => `/audio/${url}`) : null,
      });

    return setResponseStatus(event, 201);
  } catch (err) {
    console.error("Error creating note:", err);
    throw createError({
      statusCode: 500,
      message: "Failed to create note. Please try again.",
    });
  }
});

The above code reads the validated event body, and creates a new note entry in the database using the drizzle composable we created earlier. We will get to the validation part in a bit.

index.get.ts

// server/api/notes/index.get.ts
export default defineEventHandler(async (event) => {
  try {
    const notes = await useDrizzle()
      .select()
      .from(tables.notes)
      .orderBy(desc(tables.notes.updatedAt));

    return notes;
  } catch (err) {
    console.error("Error retrieving note:", err);
    throw createError({
      statusCode: 500,
      message: "Failed to get notes. Please try again.",
    });
  }
});

Here we fetch the notes entries from the table in descending order of updatedAt field.

Incoming data validation

As mentioned in the beginning, we’ll use Zod for data validation. Here is the relevant code from index.post.ts that validates the incoming client data.

const { text, audioUrls } = await readValidatedBody(event, noteSchema.parse);

Create a new file note.schema.ts in the shared/schemas folder in the project root directory with the following content:

// shared/schemas/note.schema.ts
import { createInsertSchema, createSelectSchema } from "drizzle-zod";
import { z } from "zod";
import { notes } from "~~/server/database/schema";

export const noteSchema = createInsertSchema(notes, {
  text: (schema) =>
    schema.text
      .min(3, "Note must be at least 3 characters long")
      .max(5000, "Note cannot exceed 5000 characters"),
  audioUrls: z.string().array().optional(),
}).pick({
  text: true,
  audioUrls: true,
});

export const noteSelectSchema = createSelectSchema(notes, {
  audioUrls: z.string().array().optional(),
});

The above code uses the drizzle-zod plugin to create the zod schema needed for validation (The above validation error messages are more suitable for the client side. Feel free to adapt these validation rules to suit your specific project requirements.).

Creating DB Migrations

With the table schema and API endpoints defined, the final step is to create and apply database migrations to bring everything together. Add the following command to your package.json's scripts:

// ..
"scripts": {
  // ..
  "db:generate": "drizzle-kit generate"
}
// ..

Next, run pnpm run db:generate to create the database migrations. These migrations are auto applied by NuxtHub when you run or deploy your project. You can test it by running pnpm dev and checking the Nuxt Dev Tools as shown below (this is a local sqlite database that is used in the dev mode).

We are done with the basic backend of the project. In the next section, we will code the frontend components and pages to complete the whole thing,

Creating the Basic Frontend

We’ll start with the most important feature first: recording the user voice, and then we’ll move on to creating the needed components and pages.

useMediaRecorder Composable

Let’s create a composable to handle the media recording functionality. Create a new file useMediaRecorder.ts in your app/composables folder and add the following code to it:

// app/composables/useMediaRecorder.ts
interface MediaRecorderState {
  isRecording: boolean;
  recordingDuration: number;
  audioData: Uint8Array | null;
  updateTrigger: number;
}

const getSupportedMimeType = () => {
  const types = [
    "audio/mp4",
    "audio/mp4;codecs=mp4a",
    "audio/mpeg",
    "audio/webm;codecs=opus",
    "audio/webm",
  ];

  return (
    types.find((type) => MediaRecorder.isTypeSupported(type)) || "audio/webm"
  );
};

export function useMediaRecorder() {
  const state = ref<MediaRecorderState>({
    isRecording: false,
    recordingDuration: 0,
    audioData: null,
    updateTrigger: 0,
  });

  let mediaRecorder: MediaRecorder | null = null;
  let audioContext: AudioContext | null = null;
  let analyser: AnalyserNode | null = null;
  let animationFrame: number | null = null;
  let audioChunks: Blob[] | undefined = undefined;

  const updateAudioData = () => {
    if (!analyser || !state.value.isRecording || !state.value.audioData) {
      if (animationFrame) {
        cancelAnimationFrame(animationFrame);
        animationFrame = null;
      }

      return;
    }

    analyser.getByteTimeDomainData(state.value.audioData);
    state.value.updateTrigger += 1;
    animationFrame = requestAnimationFrame(updateAudioData);
  };

  const startRecording = async () => {
    try {
      const stream = await navigator.mediaDevices.getUserMedia({ audio: true });

      audioContext = new AudioContext();
      analyser = audioContext.createAnalyser();

      const source = audioContext.createMediaStreamSource(stream);
      source.connect(analyser);

      const options = {
        mimeType: getSupportedMimeType(),
        audioBitsPerSecond: 64000,
      };

      mediaRecorder = new MediaRecorder(stream, options);
      audioChunks = [];

      mediaRecorder.ondataavailable = (e: BlobEvent) => {
        audioChunks?.push(e.data);
        state.value.recordingDuration += 1;
      };

      state.value.audioData = new Uint8Array(analyser.frequencyBinCount);
      state.value.isRecording = true;
      state.value.recordingDuration = 0;
      state.value.updateTrigger = 0;
      mediaRecorder.start(1000);

      updateAudioData();
    } catch (err) {
      console.error("Error accessing microphone:", err);
      throw err;
    }
  };

  const stopRecording = async () => {
    return await new Promise<Blob>((resolve) => {
      if (mediaRecorder && state.value.isRecording) {
        const mimeType = mediaRecorder.mimeType;
        mediaRecorder.onstop = () => {
          const blob = new Blob(audioChunks, { type: mimeType });
          audioChunks = undefined;

          state.value.recordingDuration = 0;
          state.value.updateTrigger = 0;
          state.value.audioData = null;

          resolve(blob);
        };

        state.value.isRecording = false;
        mediaRecorder.stop();
        mediaRecorder.stream.getTracks().forEach((track) => track.stop());

        if (animationFrame) {
          cancelAnimationFrame(animationFrame);
          animationFrame = null;
        }

        audioContext?.close();
        audioContext = null;
      }
    });
  };

  onUnmounted(() => {
    stopRecording();
  });

  return {
    state: readonly(state),
    startRecording,
    stopRecording,
  };
}

The above code does the following:

1. Exposes recording start/stop functionality along with the current recording readonly state

2. Captures user’s voice using the MediaRecorder API when startRecording function is invoked. The MediaRecorder API is a simple and efficient way to handle media capture in modern browsers, making it ideal for our use case.

3. Captures audio visualization data using AudioContext and AnalyserNode and updates it in real-time using animation frames

4. Cleans up resources and returns the captured audio as a Blob when stopRecording is called or if the component unmounts

NoteEditorModal Component

Next, create a new file NoteEditorModal.vue in the app/components folder and add the following code to it:

<!-- app/components/NoteEditorModal.vue -->
<template>
  <UModal
    fullscreen
    :close="{
      disabled: isSaving || noteRecorder?.isBusy,
    }"
    :prevent-close="isSaving || noteRecorder?.isBusy"
    title="Create Note"
    :ui="{
      body: 'flex-1 w-full max-w-7xl mx-auto flex flex-col md:flex-row gap-4 sm:gap-6 overflow-hidden',
    }"
  >
    <template #body>
      <UCard class="flex-1 flex flex-col" :ui="{ body: 'flex-1' }">
        <template #header>
          <h3 class="h-8 font-medium text-gray-600 dark:text-gray-300">
            Note transcript
          </h3>
        </template>

        <UTextarea
          v-model="noteText"
          placeholder="Type your note here, or use voice recording..."
          size="lg"
          :disabled="isSaving || noteRecorder?.isBusy"
          :ui="{ root: 'w-full h-full', base: ['h-full resize-none'] }"
        />
      </UCard>

      <NoteRecorder
        ref="recorder"
        class="md:h-full md:flex md:flex-col md:w-96 shrink-0 order-first md:order-none"
        @transcription="handleTranscription"
      />
    </template>

    <template #footer>
      <UButton
        icon="i-lucide-undo-2"
        color="neutral"
        variant="outline"
        :disabled="isSaving"
        @click="resetNote"
      >
        Reset
      </UButton>

      <UButton
        icon="i-lucide-cloud-upload"
        :disabled="!noteText.trim() || noteRecorder?.isBusy || isSaving"
        :loading="isSaving"
        @click="saveNote"
      >
        Save Note
      </UButton>
    </template>
  </UModal>
</template>

<script setup lang="ts">
import { NoteRecorder } from "#components";

const props = defineProps<{ onNewNote: () => void }>();

type NoteRecorderType = InstanceType<typeof NoteRecorder>;
const noteRecorder = useTemplateRef<NoteRecorderType>("recorder");
const resetNote = () => {
  noteText.value = "";
  noteRecorder.value?.resetRecordings();
};

const noteText = ref("");
const handleTranscription = (text: string) => {
  noteText.value += noteText.value ? "\n\n" : "";
  noteText.value += text ?? "";
};

const modal = useModal();
const isSaving = ref(false);
const saveNote = async () => {
  const text = noteText.value.trim();
  if (!text) return;

  isSaving.value = true;

  const audioUrls = await noteRecorder.value?.uploadRecordings();

  try {
    await $fetch("/api/notes", {
      method: "POST",
      body: { text, audioUrls },
    });

    useToast().add({
      title: "Note Saved",
      description: "Your note was saved successfully.",
      color: "success",
    });

    if (props.onNewNote) {
      props.onNewNote();
    }

    modal.close();
  } catch (err) {
    console.error("Error saving note:", err);
    useToast().add({
      title: "Save Failed",
      description: "Failed to save the note.",
      color: "error",
    });
  }

  isSaving.value = false;
};
</script>

The above modal component does the following:

1. Displays a textarea for allowing a manual note entry

2. The modal integrates the NoteRecorder component for voice recordings and manages the data flow between the recordings and the textarea for user notes.

3. Whenever a new recording is created, it captures the emitted event from the note recorder component, and appends the transcription text to the textarea content

4. When the user clicks the save note button, its first uploads all recordings (if any) by calling the note recorder’s uploadRecordings method, and then save the note by calling the notes API endpoint created earlier.

5. The save note button first uploads all recordings (if any) asynchronously by calling the uploadRecordings method, then sends the note data to the /api/notes endpoint. Upon success, it notifies the parent by executing the callback passed by it, and then closes the modal.

NoteRecorder Component

Create a new file NoteRecorder.vue in the app/components folder and add the following content to it:

<!-- app/components/NoteRecorder.vue --> 
<template>
  <UCard
    :ui="{
      body: 'max-h-36 md:max-h-none md:flex-1 overflow-y-auto',
    }"
  >
    <template #header>
      <h3 class="font-medium text-gray-600 dark:text-gray-300">Recordings</h3>

      <div class="flex items-center gap-x-2">
        <template v-if="state.isRecording">
          <div class="w-2 h-2 rounded-full bg-red-500 animate-pulse" />
          <span class="mr-2 text-sm">
            {{ formatDuration(state.recordingDuration) }}
          </span>
        </template>

        <UButton
          :icon="state.isRecording ? 'i-lucide-circle-stop' : 'i-lucide-mic'"
          :color="state.isRecording ? 'error' : 'primary'"
          :loading="isTranscribing"
          @click="toggleRecording"
        />
      </div>
    </template>

    <AudioVisualizer
      v-if="state.isRecording"
      class="w-full h-14 p-2 bg-gray-50 dark:bg-gray-800 rounded-lg mb-2"
      :audio-data="state.audioData"
      :data-update-trigger="state.updateTrigger"
    />

    <div
      v-else-if="isTranscribing"
      class="flex items-center justify-center h-14 gap-x-3 p-2 bg-gray-50 dark:bg-gray-800 rounded-lg mb-2 text-gray-500 dark:text-gray-400"
    >
      <UIcon name="i-lucide-refresh-cw" size="size-6" class="animate-spin" />
      Transcribing...
    </div>

    <div class="space-y-2">
      <div
        v-for="recording in recordings"
        :key="recording.id"
        class="flex items-center gap-x-3 p-2 bg-gray-50 dark:bg-gray-800 rounded-lg"
      >
        <audio :src="recording.url" controls class="w-full h-10" />

        <UButton
          icon="i-lucide-trash-2"
          color="error"
          variant="ghost"
          size="sm"
          @click="removeRecording(recording)"
        />
      </div>
    </div>

    <div
      v-if="!recordings.length && !state.isRecording && !isTranscribing"
      class="h-full flex flex-col items-center justify-center text-gray-500 dark:text-gray-400"
    >
      <p>No recordings...!</p>
      <p class="text-sm mt-1">Tap the mic icon to create one.</p>
    </div>
  </UCard>
</template>

<script setup lang="ts">
const emit = defineEmits<{ transcription: [text: string] }>();

const { state, startRecording, stopRecording } = useMediaRecorder();
const toggleRecording = () => {
  if (state.value.isRecording) {
    handleRecordingStop();
  } else {
    handleRecordingStart();
  }
};

const handleRecordingStart = async () => {
  try {
    await startRecording();
  } catch (err) {
    console.error("Error accessing microphone:", err);
    useToast().add({
      title: "Error",
      description: "Could not access microphone. Please check permissions.",
      color: "error",
    });
  }
};

const { recordings, addRecording, removeRecording, resetRecordings } =
  useRecordings();

const handleRecordingStop = async () => {
  let blob: Blob | undefined;

  try {
    blob = await stopRecording();
  } catch (err) {
    console.error("Error stopping recording:", err);
    useToast().add({
      title: "Error",
      description: "Failed to record audio. Please try again.",
      color: "error",
    });
  }

  if (blob) {
    try {
      const transcription = await transcribeAudio(blob);

      if (transcription) {
        emit("transcription", transcription);

        addRecording({
          url: URL.createObjectURL(blob),
          blob,
          id: `${Date.now()}`,
        });
      }
    } catch (err) {
      console.error("Error transcribing audio:", err);
      useToast().add({
        title: "Error",
        description: "Failed to transcribe audio. Please try again.",
        color: "error",
      });
    }
  }
};

const isTranscribing = ref(false);
const transcribeAudio = async (blob: Blob) => {
  try {
    isTranscribing.value = true;
    const formData = new FormData();
    formData.append("audio", blob);

    return await $fetch("/api/transcribe", {
      method: "POST",
      body: formData,
    });
  } finally {
    isTranscribing.value = false;
  }
};

const uploadRecordings = async () => {
  if (!recordings.value.length) return;

  const formData = new FormData();
  recordings.value.forEach((recording) => {
    if (recording.blob) {
      formData.append(
        "files",
        recording.blob,
        `${recording.id}.${recording.blob.type.split("/")[1]}`,
      );
    }
  });

  try {
    const result = await $fetch("/api/upload", {
      method: "PUT",
      body: formData,
    });

    return result.map((obj) => obj.pathname);
  } catch (error) {
    console.error("Failed to upload audio recordings", error);
  }
};

const isBusy = computed(() => state.value.isRecording || isTranscribing.value);

defineExpose({ uploadRecordings, resetRecordings, isBusy });

const formatDuration = (seconds: number) => {
  const mins = Math.floor(seconds / 60);
  const secs = seconds % 60;
  return `${mins}:${secs.toString().padStart(2, "0")}`;
};
</script>

This component does the following:

1. Allows recording the user’s voice with the help of useMediaRecorder composable created earlier. It also integrates the AudioVisualizer component to enhance the user experience by providing real-time audio feedback during recordings.

2. On a new recording, sends the recorded blob for transcription to the transcribe API endpoint, and emits the transcription text on success

3. Displays all recordings as audio elements for users perusal (using URL.createObjectURL(blob)). It utilizes the useRecordings composable to manage the recordings

4. Uploads the final recordings to R2 (the local disk in dev mode) using the /api/upload endpoint, and returns the pathnames of these recordings to the caller (the NoteEditorModal component)

AudioVisualizer Component

This component uses an HTML canvas element to represent the audio waveform along a horizontal line. The canvas element is used for its flexibility and efficiency in rendering real-time visualizations, making it suitable for audio waveforms.

The visualization dynamically adjusts based on the amplitude of the captured audio, providing a real-time feedback loop for the user during recording. To do that, it watches the updateTrigger state variable exposed by useMediaRecorder to redraw the canvas on audio data changes.

Create a new file AudioVisualizer.vue in the app/components folder and add the following code to it:

<!-- app/components/AudioVisualizer.vue -->
<template>
  <canvas ref="canvas" width="640" height="100" />
</template>

<script setup lang="ts">
const props = defineProps<{
  audioData: Uint8Array | null;
  dataUpdateTrigger: number;
}>();

let width = 0;
let height = 0;
const audioCanvas = useTemplateRef<HTMLCanvasElement>("canvas");
const canvasCtx = ref<CanvasRenderingContext2D | null>(null);

onMounted(() => {
  if (audioCanvas.value) {
    canvasCtx.value = audioCanvas.value.getContext("2d");
    width = audioCanvas.value.width;
    height = audioCanvas.value.height;
  }
});

const drawCanvas = () => {
  if (!canvasCtx.value || !props.audioData) {
    return;
  }

  const data = props.audioData;
  const ctx = canvasCtx.value;
  const sliceWidth = width / data.length;

  ctx.clearRect(0, 0, width, height);
  ctx.lineWidth = 2;
  ctx.strokeStyle = "rgb(221, 72, 49)";
  ctx.beginPath();

  let x = 0;
  for (let i = 0; i < data.length; i++) {
    const v = (data[i] ?? 0) / 128.0;
    const y = (v * height) / 2;

    if (i === 0) {
      ctx.moveTo(x, y);
    } else {
      ctx.lineTo(x, y);
    }

    x += sliceWidth;
  }

  ctx.lineTo(width, height / 2);
  ctx.stroke();
};

watch(
  () => props.dataUpdateTrigger,
  () => {
    drawCanvas();
  },
  { immediate: true },
);
</script>

useRecordings Composable

The NoteRecorder component uses the useRecordings composable to manage the list of recordings, and to clear any used resources. Create a new file useRecordings.ts in the app/composables folder and add the following code to it:

// app/composables/useRecordings.ts
export const useRecordings = () => {
  const recordings = ref<Recording[]>([]);

  const cleanupResource = (recording: Recording) => {
    if (recording.blob) {
      URL.revokeObjectURL(recording.url);
    }
  };

  const cleanupResources = () => {
    recordings.value.forEach((recording) => {
      cleanupResource(recording);
    });
  };

  const addRecording = (recording: Recording) => {
    recordings.value.unshift(recording);
  };

  const removeRecording = (recording: Recording) => {
    recordings.value = recordings.value.filter((r) => r.id !== recording.id);
    cleanupResource(recording);
  };

  const resetRecordings = () => {
    cleanupResources();

    recordings.value = [];
  };

  onUnmounted(cleanupResources);

  return {
    recordings,
    addRecording,
    removeRecording,
    resetRecordings,
  };
};

You can define the Recording type definition in the shared/types/index.ts file. This allows for auto import of type definitions in both client & server sides (The intended purpose of the shared folder is for sharing common types & utils between the app & server). Also, while you’re at it, you can also define the Note type.

// shared/types/index.ts
import type { z } from "zod";
import type { noteSelectSchema } from "#shared/schemas/note.schema";

export type Recording = {
  url: string;
  blob?: Blob;
  id: string;
};

export type Note = z.output<typeof noteSelectSchema>;

Creating the Home Page

Now that we have all the pieces ready for the basic app, it is time to put everything together in a page. Delete the content of the home page (app/pages/index.vue), and put the following content to it:

<!-- app/pages/index.vue -->
<template>
  <UContainer class="h-screen flex justify-center items-center">
    <UCard
      class="w-full max-h-full overflow-hidden max-w-4xl mx-auto"
      :ui="{ body: 'h-[calc(100vh-4rem)] overflow-y-auto' }"
    >
      <template #header>
        <span class="font-bold text-xl md:text-2xl">Voice Notes</span>
        <UButton icon="i-lucide-plus" @click="showNoteModal">
          New Note
        </UButton>
      </template>

      <div v-if="notes?.length" class="space-y-4">
        <NoteCard v-for="note in notes" :key="note.id" :note="note" />
      </div>
      <div
        v-else
        class="my-12 text-center text-gray-500 dark:text-gray-400 space-y-2"
      >
        <h2 class="text-2xl md:text-3xl">No notes created</h2>
        <p>Get started by creating your first note</p>
      </div>
    </UCard>
  </UContainer>
</template>

<script setup lang="ts">
import { LazyNoteEditorModal } from "#components";

const { data: notes, refresh } = await useFetch("/api/notes");

const modal = useModal();
const showNoteModal = () => {
  modal.open(LazyNoteEditorModal, {
    onNewNote: refresh,
  });
};

watch(modal.isOpen, (newState) => {
  if (!newState) {
    modal.reset();
  }
});
</script>

On this page we’re doing the following:

1. Fetch the list of existing notes from the database and display them using the NoteCard component

2. Shows a new note button which when clicked opens the NoteEditorModal. On successful note creation the refresh function is called to refetch the notes

3. The modal state is reset on closure to ensure a clean slate for the next note creation

The cards and modals headers/footers used in the app follow a global style that is defined in the app config file. Centralizing styles in the app configuration ensures consistent theming and reduces redundancy across components.

Create a new file app.config.ts inside the app folder, and add the following to it:

// app/app.config.ts
export default defineAppConfig({
  ui: {
    card: {
      slots: {
        header: "flex items-center justify-between gap-3 flex-wrap",
      },
    },
    modal: {
      slots: {
        footer: "justify-end gap-x-3",
      },
    },
  },
});

You’ll also need to wrap your NuxtPage component with the UApp component for the modals and toast notifications to work as shown below:

<!-- app/app.vue -->
<template>
  <NuxtRouteAnnouncer />
  <NuxtLoadingIndicator />
  <UApp>
    <NuxtPage />
  </UApp>
</template>

NoteCard component

This component displays the note text and the attached audio recordings of a note. The note text is clamped to 3 lines with a show more/less button to show/hide rest of the text. Text clamping ensures that the UI remains clean and uncluttered, while the show more/less button gives users full control over note visibility.

Create a new file NoteCard.vue in the app/components folder, and add the following code to it:

<template>
  <UCard class="hover:shadow-lg transition-shadow">
    <div class="flex-1">
      <p
        ref="text"
        :class="['whitespace-pre-wrap', !showFullText && 'line-clamp-3']"
      >
        {{ note.text }}
      </p>
      <UButton
        v-if="shouldShowExpandBtn"
        variant="link"
        :padded="false"
        @click="showFullText = !showFullText"
      >
        {{ showFullText ? "Show less" : "Show more" }}
      </UButton>
    </div>

    <div
      v-if="note.audioUrls && note.audioUrls.length > 0"
      class="mt-4 flex gap-x-2 overflow-x-auto"
    >
      <audio
        v-for="url in note.audioUrls"
        :key="url"
        :src="url"
        controls
        class="w-60 shrink-0 h-10"
      />
    </div>

    <p
      class="flex items-center text-sm text-gray-500 dark:text-gray-400 gap-x-2 mt-6"
    >
      <UIcon name="i-lucide-clock" size="size-4" />
      <span>
        {{
          note.updatedAt && note.updatedAt !== note.createdAt
            ? `Updated ${updated}`
            : `Created ${created}`
        }}
      </span>
    </p>
  </UCard>
</template>

<script setup lang="ts">
import { useTimeAgo } from "@vueuse/core";

const props = defineProps<{ note: Note }>();

const createdAt = computed(() => props.note.createdAt + "Z");
const updatedAt = computed(() => props.note.updatedAt + "Z");

const created = useTimeAgo(createdAt);
const updated = useTimeAgo(updatedAt);

const showFullText = ref(false);

const shouldShowExpandBtn = ref(false);
const noteText = useTemplateRef<HTMLParagraphElement>("text");
const checkTextExpansion = () => {
  nextTick(() => {
    if (noteText.value) {
      shouldShowExpandBtn.value =
        noteText.value.scrollHeight > noteText.value.clientHeight;
    }
  });
};

onMounted(checkTextExpansion);

watch(() => props.note.text, checkTextExpansion);
</script>

And we are done here. Try running the application and create some notes. You should be able to create notes, add multiple recordings to the same note etc. Everything should be working now, or is it?

Try playing the audio recordings of the saved notes, are these playable?

Serving the Audio Recordings

We can’t play the audio recordings because these are saved in R2 (local disk in dev mode), and nowhere we are serving these files. It is time to fix that.

If you look at the /api/notes code, we save the audio urls/pathnames with an audio prefix

await useDrizzle()
  .insert(tables.notes)
  .values({
    text,
    audioUrls: audioUrls ? audioUrls.map((url) => `/audio/${url}`) : null,
  });

The reason to do so was to serve all audio recordings through an /audio path. Create a new file […pathname].get.ts in the server/routes/audio folder and add the following to it:

export default defineEventHandler(async (event) => {
  const { pathname } = getRouterParams(event);

  return hubBlob().serve(event, pathname);
});

What we’ve done above is to catch all requests to the /audio path (by using the wildcard […pathname] in the filename), and serve the requested recording from the storage using hubBlob.

With this, the frontend is complete, and all functionalities should now work seamlessly.

Further Enhancements

What you’ve created here is a basic version of the application—with all must-have features—that you saw in the beginning of the article. You can further refine the app and take it closer to the demo by:

What you’ve created here is a solid foundation for the application, complete with the core features introduced earlier. To further enhance the app and bring it closer to the full demo version, consider implementing the following features:

1. Adding a settings page to save post processing settings

2. Handle post processing in the /transcribe api route

3. Allowing edit/delete of saved notes

4. Experimenting with additional features that fit your use case or user needs.

If you get stuck while implementing these features, do not hesitate to look at the application source code. The complete source code of the final application is shared at the end of the article.

Deploying the Application

You can deploy the application using either the NuxtHub admin dashboard or through the NuxtHub CLI.

Deploy via NuxtHub Admin

• Push your code to a GitHub repository.

• Link the repository with NuxtHub.

• Deploy from the Admin console.

Learn more about NuxtHub Git integration

Deploy via NuxtHub CLI

npx nuxthub deploy

Learn more about CLI deployment

Source Code

You can find the source code of Vhisper application on GitHub. The source code includes all the features discussed in this article, along with additional configurations and optimizations shown in the demo.

Conclusion

Congratulations! You've built a powerful application that records and transcribes audio, stores recordings, and manages notes with an intuitive interface. Along the way you’ve touched upon various aspects of Nuxt, NuxtHub and Cloudflare services. As you continue to refine and expand Vhisper, consider exploring additional features and optimizations to further enhance its functionality and user experience. Keep experimenting and innovating, and let this project be a stepping stone to even more ambitious endeavors.

Thank you for sticking with me until the end! I hope you’ve picked up some new concepts along the way. I’d love to hear what you learned or any thoughts you have in the comments section. Your feedback is not only valuable to me, but to the entire developer community exploring this exciting field.

Until next time!

Keep adding the bits and soon you'll have a lot of bytes to share with the world.

This time, my enthusiasm has led me to tackle a challenge many developers face often: searching GitHub efficiently. Let me introduce Chat GitHub, a tool where the complexity of code repositories meets the simplicity of conversation.

Why Chat GitHub?

So, why a chat interface for GitHub search? Well, GitHub search, as powerful as it is with all its filters, can’t beat the simplicity of natural language queries. Let’s be real—it won’t answer simple questions like, “When did I make my first commit?” (What filters would you even use to find this information with the current GitHub Search?). GitHub certainly knows (reminds me of “I Know What You Did Last Summer”), but it just won’t answer you directly.

Natural language makes the experience frictionless. Instead of crafting complex queries, you can just ask GitHub, like you would a colleague.

When Sébastien Chopin mentioned the idea to me, I was onboard immediately (also because I was itching to build something…).

How it Works?

At its core, Chat GitHub leverages the power of OpenAI's language models to interpret your natural language queries and translate them into GitHub's search syntax. Here's a simplified breakdown of the process:

1. You enter a query in plain English

2. The AI model interprets your request

3. It then selects the appropriate search endpoint, and generates the necessary GitHub API query parameters

4. We send a request to GitHub's API with these details

5. The results are fetched and presented to you in a clean, easy-to-digest chat interface

Here's a sneak peek of what the chat interface looks like:

You can try it out live here: https://chat-github.nuxt.dev

We’ll explore the key aspects of this process in the sections to follow. Ready? Let’s get started.

Project Setup

This project follows a similar setup to my last one Hub Chat (GitHub link), and I’ve reused several components with some slight modifications. I won't bore you by repeating the same details, but if you’re new here, feel free to follow along with both posts (previous post) for a more complete picture.

Tech Stack

Here’s a quick breakdown of the tech stack:

• Nuxt 3: For the overall framework and routing.

• OpenAI APIs: To handle the natural language processing.

• GitHub API: To fetch the data you’re looking for—remember? Only it knows what you did last summer.

• NuxtUI: To make sure the UI is smooth and responsive.

• Nuxt-Auth-Utils: For user authentication and handling GitHub login (Only authenticated users can start a chat).

• Nuxt MDC: For parsing and displaying the markdown responses

• NuxtHub: For deployment, database, and caching (all powered by Cloudflare).

Prerequisites

• GitHub account: You'll need this to generate a GITHUB_TOKEN for API queries, and to create an OAuth App for authentication.

• OpenAI account: For creating an OpenAI API key, so the app can process your queries.

• Optional: If you're looking to deploy the project yourself, you'll need Cloudflare and a NuxtHub account.

Setting up the project

Follow the setup process from my previous article. Just remember to add the nuxt-auth-utils, @octokit/rest and OpenAI dependencies.

# Add the nuxt-auth-utils module
npx nuxi module add auth-utils

# Add the Octokit Rest & OpenAI libraries
pnpm add @octokit/rest openai

Once the dependencies are set up, you should be able to run the project with pnpm dev and see the default app UI at localhost:3000.

Configs and Environment Variables

At this point, you can enable the hub database and cache in the nuxt.config.ts file for later use, as well as create the necessary API tokens and keys to place in the .env file.

Enabling database and cache in nuxt.config.ts:

hub: {
  cache: true,
  database: true,
},

Next, generate the following tokens and keys, and store them in your .env file (located in the root of your project):

• GitHub Token: Create a GitHub token (no special scope required) for making API calls.

• OpenAI API Key: Generate a key from your OpenAI dashboard.

• GitHub OAuth App: Create a GitHub OAuth app and get its CLIENT_ID and CLIENT_SECRET. This will be used to authenticate users (only authenticated users can start a chat). For more information on how to create a GitHub OAuth app, refer to the GitHub documentation.

Your .env should contain the following entries:

NUXT_SESSION_PASSWORD=at_least_32_chars_string
NUXT_OAUTH_GITHUB_CLIENT_ID=github_oauth_client_id
NUXT_OAUTH_GITHUB_CLIENT_SECRET=github_oauth_client_secret
NUXT_GITHUB_TOKEN=your_personal_access_token
OPENAI_API_KEY=your_openai_api_key

Note: NUXT_SESSION_PASSWORD will automatically be created by nuxt-auth-utils in development if you haven’t set it manually.

And that’s it for the setup—phew!

In the next section, we’ll explore the core of the chat process: making sense of the user query, converting it to a GitHub API call amd generating a final response.

From User Query to GitHub API Call

Now, let’s break down how Chat GitHub processes your query, identifies the necessary actions, and makes the appropriate GitHub API call. This involves three main steps: interpreting the user query, calling the necessary tools (the GitHub API), and generating the final response.

1. Interpreting the User Query

The first challenge is understanding what the user is asking for. To do this, the system relies on OpenAI’s language models to parse natural language inputs. It takes into account several factors:

• Intent Recognition: What is the user trying to achieve? Are they searching for commits, issues, repositories, or user profiles?

• Parameter Extraction: Once the intent is clear, the model extracts necessary parameters like repo name, user, dates, and other filters. These details are critical for constructing a meaningful API call.

• Tool Selection: Based on the query, the AI determines if a GitHub API tool needs to be invoked (for example, searching commits or issues).

To guide the AI, I created a detailed system prompt to provide context. Here’s a portion of that prompt:

const systemPropmt = `You are a concise assistant who helps \
users find information on GitHub. Use the supplied tools to \
find information when asked.

Available endpoints and key parameters:

// ...

2. issues (Also searches PRs):
  - Sort options: comments, reactions, reactions-+1, ...
  - Query qualifiers: type, is, state, author, assignee, ...
  - use "type" or "is" qualifier to search issues or PRs (type:issue/type:pr)

// ...

When using searchGithub function:
1. Choose the appropriate search endpoint.
2. Formulate a concise query (q) as per the user's request.
3. Add any relevant sort or order parameters if needed.
4. Always use appropriate per_page value to limit the number of results.

// ...

Examples: 
// ...

2. Find the total number of repositories of a user
arguments: 
{
  "endpoint": "repositories",
  "q": "user:<user_login>",
  "per_page": 1
}

Summarize final response concisely using markdown when appropriate \
(for all links add {target="_blank"} at the end). Do not include \
images, commit SHA or hashes etc. in your summary.`

Note: I restricted the AI to only the most important GitHub search endpoints /search/commits, /search/issues, /search/repositories and /search/users.

In addition to the system prompt, we create tools definitions that lists the types of tools, their names, and their specific parameters (in this case I only create one function tool, searchGithub).

const tools: OpenAI.ChatCompletionTool[] = [
  {
    type: 'function',
    function: {
      name: 'searchGithub',
      description:
        'Searches GitHub for information using the GitHub API. Call this if you need to find information on GitHub.',
      parameters: {
        type: 'object',
        properties: {
          endpoint: {
            type: 'string',
            description: `The specific search endpoint to use. One of ['commits', 'issues', 'repositories', 'users']`,
          },
          q: {
            type: 'string',
            description: 'the search query using applicable qualifiers',
          },
          sort: {
            type: 'string',
            description: 'The sort field (optional, depends on the endpoint)',
          },
          order: {
            type: 'string',
            description: 'The sort order (optional, asc or desc)',
          },
          per_page: {
            type: 'string',
            description:
              'Number of results to fetch per page (max 25)',
          },
        },
        required: ['endpoint', 'q', 'per_page'],
        additionalProperties: false,
      },
    },
  },
];

A couple of key design decisions we made:

• Complimentary System Prompt & Tool Definition: The system prompt gives context, while the tool definition ensures the API queries are correctly structured.

• Result Limit: While GitHub allows more items per response, we restrict results to 25 to keep things manageable.

• No Pagination: We decided to skip pagination, keeping things simple and avoiding unnecessary complexity.

Now, the AI is ready to handle the user query and transform it into a structured format that the system can use. Here is the relevant code:

let _openai: OpenAI;
function useOpenAI() {
  if (!_openai) {
    _openai = new OpenAI({
      apiKey: process.env.OPENAI_API_KEY,
    });
  }

  return _openai;
}

export const handleMessageWithOpenAI = async (
  event: H3Event,
  messages: OpenAI.Chat.ChatCompletionMessageParam[]
) => {
  const openai = useOpenAI();
  const response = await openai.chat.completions.create({
    model: MODEL, // used gpt-4o
    messages, // contains system prompt and the complete chat history
    tools, // defined above
  });

  const responseMessage = response.choices[0].message;
  const toolCalls = responseMessage.tool_calls;

  if (toolCalls) {
    messages.push(responseMessage);

    for (const toolCall of toolCalls) {
      const functionName = toolCall.function.name;
      if (functionName === 'searchGithub') {
        const functionArgs = JSON.parse(toolCall.function.arguments);
        const functionResponse = await searchGithub(
          functionArgs.endpoint,
          {
            q: functionArgs.q,
            sort: functionArgs.sort,
            order: functionArgs.order,
            per_page: functionArgs.per_page,
          }
        );

        // ..
      }
    }

    // ...
  }

  return responseMessage.content;
}

2. Using Tool Calls for GitHub Search

Once the AI determines the need for a GitHub API call, it generates a tool_call with the necessary parameters. Here’s how we handle this with the searchGitHub function:

export type SearchParams = {
  endpoint: string;
  q: string;
  order?: string;
  sort?: string;
  per_page: string;
};

const allowedEndpoints = {
  commits: 'GET /search/commits',
  issues: 'GET /search/issues',
  repositories: 'GET /search/repositories',
  users: 'GET /search/users',
} as const;

type EndpointType = keyof typeof allowedEndpoints;

let _octokit: Octokit;

function useOctokit() {
  if (!_octokit) {
    _octokit = new Octokit({
      auth: process.env.NUXT_GITHUB_TOKEN,
    });
  }

  return _octokit;
}

export const searchGithub = async (
  endpoint: string,
  params: Omit<SearchParams, 'endpoint'>
) => {
  if (!endpoint || !allowedEndpoints[endpoint as EndpointType]) {
    throw createError({
      statusCode: 404,
      message: 'Endpoint not supported',
    });
  }

  const octokit = useOctokit();
  const endpointToUse = allowedEndpoints[endpoint as EndpointType];

  try {
    const response = await octokit.request(endpointToUse as string, params);

    return response.data;
  } catch (error) {
    console.error(error);
    throw createError({
      statusCode: 500,
      message: 'Error searching GitHub',
    });
  }
};

The function is a straightforward GitHub API search using the @octokit/rest library. The endpoint and parameters are passed from the tool_call, and we make the request to GitHub’s API, fetching the appropriate data. You can learn more about the GitHub search APIs from the official GitHub Docs.

3. Generating the Final Response

Once the GitHub API returns its results, the final step is to package them into a user-friendly response. The AI processes the raw data—whether commits, issues, repos, or users—and generates readable summaries. Here’s the relevant code snippet:

if (tool_calls) {
  // ...

  for (const toolCall of toolCalls) {
    // ...

    messages.push({
      tool_call_id: toolCall.id,
      role: 'tool',
      content: JSON.stringify(functionResponse),
    });
  }

  const finalResponse = await openai.chat.completions.create({
    model: MODEL,
    messages: messages,
  });

  return finalResponse.choices[0].message.content;
}

// ..

In the code above, you can see how we take the API response and push it to the messages array, preparing it for the AI to format into a concise response that’s easy for the user to understand.

Managing GitHub API Rate Limits

If you’ve used the GitHub API (or any third-party API), you’ll know that most of them come with rate limits. So, how can we minimize GitHub API calls? The answer is simple: we avoid making duplicate calls by caching every GitHub response. If a user requests the same information that another user (or even themselves) asked for earlier, we pull the data from the cache instead of making another API call.

Now, the question is: how do we implement this in Nuxt? It’s actually quite easy, thanks to Nitro’s Cached Functions (Nitro is an open source framework to build web servers which Nuxt uses internally). Let’s take a look at the revised searchGithub function below:

export const searchGithub = defineCachedFunction(
  async (
    event: H3Event,
    endpoint: string,
    params: Omit<SearchParams, 'endpoint'>
  ) => {
    if (!endpoint || !allowedEndpoints[endpoint as EndpointType]) {
      throw createError({
        statusCode: 404,
        message: 'Endpoint not supported',
      });
    }

    const octokit = useOctokit();
    const endpointToUse = allowedEndpoints[endpoint as EndpointType];

    try {
      const response = await octokit.request(endpointToUse as string, params);

      return response.data;
    } catch (error) {
      console.error(error);
      throw createError({
        statusCode: 500,
        message: 'Error searching GitHub',
      });
    }
  },
  {
    maxAge: 60 * 60, // 1 hour
    group: 'github',
    name: 'search',
    getKey: (
      event: H3Event,
      endpoint: string,
      params: Omit<SearchParams, 'endpoint'>
    ) => {
      const q = params.q.trim().toLowerCase().split(' ');

      const mainQuery = q.filter((term) => !term.includes(':')).join(' ');
      const qualifiers = q.filter((term) => term.includes(':')).sort();

      const finalQuery = [...(mainQuery ? [mainQuery] : []), ...qualifiers]
        .join(' ')
        .replace(/[\s:]/g, '_');

      let key = endpoint + '_q_' + finalQuery;

      if (params.per_page) {
        key += '_per_page_' + params.per_page;
      }

      if (params.order) {
        key += '_order_' + params.order;
      }

      if (params.sort) {
        key += '_sort_' + params.sort;
      }

      return key;
    },
  }
);

We’ve modified our earlier function to use cachedFunction, and added H3Event (from the /chat API endpoint call) as the first parameter—this is needed because the app is deployed on the edge with Cloudflare (more details here). The most important part here is how we create a unique cache key. Here’s a breakdown:

1. Query Qualifiers: First, we break down the q parameter into its qualifier pairs (e.g., for finding the first PR of a GitHub user like ra-jeev—yes, that’s me—the q value would be author:ra-jeev type:pr). Sometimes, it may contain the direct query string too, and the code accounts for this.

2. Sorting Qualifiers: Next, we sort the qualifiers alphabetically. This is because the AI could create the same query as type:pr author:ra-jeev. We could handle this in the system prompt, but why over-complicate things for the AI?

3. Creating the Cache Key: Finally, we combine all the qualifiers and other parameters into a single string, separated by underscores.

We set the cache duration to 1 hour, as seen in the maxAge setting, which means all searchGitHub responses are stored for that time. To use cache in NuxtHub production we’d already enabled cache: true in our nuxt.config.ts.

Now that we’ve tackled rate limiting, it’s time to shift our focus to response streaming. In the next section, we’ll explore how to implement streaming for a more seamless and efficient user experience.

Adding AI Response Streaming

Enabling AI response streaming is usually straightforward: you pass a parameter when making the API call, and the AI returns the response as a stream. In our Hub Chat project, for example, we handled the stream chunks directly client-side, ensuring that responses trickled in smoothly for the user.

But in the case of Chat GitHub, there’s an additional complexity—tool_call responses. So, we have two approaches to manage:

1. Enable streaming only for the final response generation, but this limits us when there’s no tool_call, wasting an opportunity to stream from the start.

2. Enable stream response for both the OpenAI calls, but this requires us to manage a stream more carefully on the server side (as we need to handle the tool_call, if any, there itself).

I took the seemingly complex path and went ahead with the second approach—”Two roads diverged in a wood, and I—I took the one less traveled by, And that has made all the difference.”

Enable Streaming in OpenAI Calls

Here’s how the OpenAI API call was modified to enable streaming:

export const handleMessageWithOpenAI = async function* (
  event: H3Event,
  messages: OpenAI.ChatCompletionMessageParam[],
) {
  const openai = useOpenAI();
  const responseStream = await openai.chat.completions.create({
    model: MODEL,
    messages,
    tools,
    stream: true, // enable streaming
  });

  const currentToolCalls: OpenAI.ChatCompletionMessageToolCall[] = [];

  for await (const chunk of responseStream) {
    const choice = chunk.choices[0];

    // if it is normal text chunk just yield it
    if (choice.delta.content) {
      yield choice.delta.content;
    }

    // if the delta contains tool_calls, then collect all chunks
    if (choice.delta.tool_calls) {
      for (const toolCall of choice.delta.tool_calls) {
        if (toolCall.index !== undefined) {
          if (!currentToolCalls[toolCall.index]) {
            currentToolCalls[toolCall.index] = {
              id: toolCall.id || '',
              type: 'function' as const,
              function: {
                name: toolCall.function?.name || '',
                arguments: '',
              },
            };
          }

          if (toolCall.function?.arguments) {
            currentToolCalls[toolCall.index].function.arguments +=
              toolCall.function.arguments;
          }
        }
      }
    }

    // once it has returned all chunks with tool_calls, we will
    // get the final finish_reason as 'tool_calls'. We can call
    // the mentioned tool now
    if (choice.finish_reason === 'tool_calls') {
      messages.push({
        role: 'assistant',
        tool_calls: currentToolCalls,
      });

      for (const toolCall of currentToolCalls) {
        if (toolCall.function.name === 'searchGithub') {
          try {
            const functionArgs = JSON.parse(toolCall.function.arguments);
            const toolResult = await searchGithub(
              event,
              functionArgs.endpoint,
              {
                q: functionArgs.q,
                sort: functionArgs.sort,
                order: functionArgs.order,
                per_page: functionArgs.per_page,
              }
            );

            messages.push({
              role: 'tool',
              tool_call_id: toolCall.id,
              content: JSON.stringify(toolResult),
            });
          } catch (error) {
            console.error('Error parsing tool call arguments:', error);
            throw error;
          }
        }
      }

      try {
        const finalResponse = await openai.chat.completions.create({
          model: MODEL,
          messages,
          stream: true,
        });

        for await (const chunk of finalResponse) {
          if (chunk.choices[0].delta.content) {
            yield chunk.choices[0].delta.content;
          }
        }
      } catch (error) {
        console.error(
          'Error generating final response or saving user query :',
          error
        );

        throw error;
      }
    }
  }
};

The revised handleMessageWithOpenAI function works like this:

• Converted it to an AsyncGenerator: This allows the function to yield data chunks progressively as they are received.

• Added stream: true to both OpenAI API calls: This tells OpenAI to stream the response back to us.

• Yielding Response Chunks: For each chunk of text that we get from the stream, we simply yield it to the caller.

• Handling tool_calls: Since streaming is enabled, the tool_call information also arrives in chunks. We collect these chunks until the OpenAI API signals the completion of this part (finish_reason === 'tool_calls'), and then invoke the searchGitHub function using the parameters provided by the tool_call.

• Final Response: After the GitHub search is done, we yield the response in chunks in the same way.

Converting to a ReadableStream

To complete the process, the chunks from handleMessageWithOpenAI are converted into a ReadableStream format, which is then returned to the client (not shown here). Here’s how that works:

export const asyncGeneratorToStream = (
  asyncGenerator: AsyncGenerator<string, void, unknown>
) => {
  let cancelled = false;
  const encoder = new TextEncoder();
  const stream = new ReadableStream({
    async start(controller) {
      try {
        for await (const value of asyncGenerator) {
          if (cancelled) {
            break;
          }

          controller.enqueue(
            encoder.encode(`data: ${JSON.stringify({ response: value })}\n\n`)
          );
        }

        // Send done to signal end of stream
        controller.enqueue(encoder.encode(`data: [DONE]\n\n`));

        controller.close();
      } catch (err) {
        console.log('Error in stream:', err);

        /* eslint-disable @stylistic/operator-linebreak */
        const errorMessage =
          err instanceof Error
            ? err.message
            : 'An error occurred in the stream';
        /* eslint-enable @stylistic/operator-linebreak */

        controller.enqueue(
          encoder.encode(
            `event: error\ndata: ${JSON.stringify({
              message: errorMessage,
            })}\n\n`
          )
        );

        controller.close();
      }
    },
    cancel(reason) {
      console.log('Client closed connection. Reason:', reason);
      cancelled = true;
    },
  });

  return stream;
};

This code transforms the AsyncGenerator we created earlier into a ReadableStream. Here’s a breakdown of what’s happening:

• AsyncGenerator: We pass the AsyncGenerator from handleMessageWithOpenAI as an argument to this function.

• Creating a ReadableStream: Inside the start method of the ReadableStream, we wait for chunks from the AsyncGenerator.

• Formatting Chunks: For each text chunk received, we format it according to the Server-Sent Events (SSE) convention (You can read more about SSE in my previous post). Each chunk is wrapped in this format: data: ${JSON.stringify({ response: value })}\n\n.

• Encoding the Stream: Using TextEncoder, we encode the chunks before sending them to the client. This encoding step is crucial, especially in production environments, as it ensures that the client correctly receives the stream in real-time.

• Error Handling: If an error occurs (as we throw errors from the AsyncGenerator), we send an event: error message to the client, signaling that something went wrong, and then close the stream to terminate the connection cleanly.

And then this stream is returned to the client from the API endpoint

// inside /api/chat event handler
return asyncGeneratorToStream(
  handleMessageWithOpenAI(event, llmMessages)
);

Handling the Stream on the Client-Side

To handle the streamed response, we’ll use a refactored version of the useChat composable, initially created for the Hub Chat project. This time, we’ll extend it to handle error events. Here's the updated code:

export function useChat(apiBase: string, body: Record<string, unknown>) {
  async function* chat(): AsyncGenerator<string, void, unknown> {
    try {
      const response = await $fetch(apiBase, {
        method: 'POST',
        body,
        responseType: 'stream',
      });

      let buffer = '';
      const reader = (response as ReadableStream)
        .pipeThrough(new TextDecoderStream())
        .getReader();

      while (true) {
        const { value, done } = await reader.read();

        if (done) {
          if (buffer.trim()) {
            console.warn('Stream ended with unparsed data:', buffer);
          }

          return;
        }

        buffer += value;
        const messages = buffer.split('\n\n');
        buffer = messages.pop() || '';

        for (const message of messages) {
          const lines = message.split('\n');
          let event = '';
          let data = '';

          for (const line of lines) {
            if (line.startsWith('event:')) {
              event = line.slice('event:'.length).trim();
            } else if (line.startsWith('data:')) {
              data = line.slice('data:'.length).trim();
            }
          }

          if (event === 'error') {
            const parsedError = JSON.parse(data);
            console.error('Stream error:', parsedError);

            throw new Error(
              parsedError.message ?? 'Failed to generate response'
            );
          } else if (data) {
            if (data === '[DONE]') return;

            try {
              const jsonData = JSON.parse(data);
              if (jsonData.response) {
                yield jsonData.response;
              }
            } catch (parseError) {
              console.warn('Error parsing JSON:', parseError);
            }
          }
        }
      }
    } catch (error) {
      console.error('Error sending message:', error);
      throw error;
    }
  }

  return chat;
}

Here’s a breakdown of what the code does:

• We use Nuxt’s $fetch to make a POST request to the API endpoint, passing the responseType: 'stream'. This tells the client to expect a streaming response.

• Once we receive the ReadableStream, we create a streamReader for it. This allows us to process the chunks one at a time as they arrive. We also pass the chunks through a TextDecoder to convert the raw bytes into readable text.

• The stream is in Server-Sent Events (SSE) format, so we parse and handle the event and data parts appropriately. Each data chunk is parsed and returned to the client component for rendering.

• The code also listens for and handles any error events that may occur, ensuring a smoother user experience by gracefully handling stream interruptions or API errors.

And this concludes the road less traveled that we took earlier. In the next section, we’ll cover how we can authenticate our users.

User Authentication with GitHub OAuth

Now that our backend is ready to handle client requests, how do we restrict access to authenticated users? And how do we provide context to the AI, like answering a query such as, “When did I make my first ever commit?” To control who can access the backend, we use authentication. And to give the AI context about the user, we rely on GitHub OAuth for authentication.

We’re leveraging nuxt-auth-utils, which simplifies integrating GitHub OAuth into our Nuxt app. This allows us to authenticate users with their GitHub accounts and manage sessions effortlessly.

Server-Side Implementation

On the server side, we need to create a route that handles the GitHub access token when the user logs in. Make sure to create a GitHub OAuth app and add the CLIENT_ID and CLIENT_SECRET to the .env as mentioned in the setup section.

To implement this, create a github.get.ts file in the route/auth folder of the server directory with the following content:

export default oauthGitHubEventHandler({
  async onSuccess(event, { user }) {
    await setUserSession(event, {
      user: {
        id: user.id,
        login: user.login,
        name: user.name,
        avatarUrl: user.avatar_url,
        htmlUrl: user.html_url,
        publicRepos: user.public_repos,
      },
    });

    return sendRedirect(event, '/chat');
  },
  // Optional, will return a json error and 401 status code by default
  onError(event, error) {
    console.error('GitHub OAuth error:', error);
    return sendRedirect(event, '/');
  },
});

The event handler name is important and should be oauthGitHubEventHandler (more details can be found here). On successful login, we call the setUserSession utility function to store the user details in an HTTP cookie and redirect them to the chat page.

For our API routes, we can then call the requireUserSession utility to ensure only authenticated users can make requests. Below is the full /api/chat endpoint handler:

export default defineEventHandler(async (event) => {
  const userSession = await requireUserSession(event);

  const { messages } = await readBody(event);
  if (!messages) {
    throw createError({
      statusCode: 400,
      message: 'User messages are required',
    });
  }

  const llmMessages = [
    {
      role: 'system',
      content: getSystemPrompt(userSession.user.login),
    },
    ...messages,
  ];

  return asyncGeneratorToStream(
    handleMessageWithOpenAI(event, llmMessages, userSession.user.login)
  );
});

As you can see, we retrieve the currently logged-in GitHub user’s details and pass the login info into the system prompt. This gives OpenAI the context it needs to answer queries like, “When did I make my first commit?” The getSystemPrompt utility helps with that:

export const getSystemPrompt = (loggedInUserName: string) =>
  systemPrompt +
  `\n\nNote: The currently logged in github user is "${loggedInUserName}".`;

Client-Side Handling

On the client side, we use the built-in AuthState component from nuxt-auth-utils to manage authentication flows, like logging in and checking if a user is signed in.

Here’s how to handle sign-in:

<AuthState v-slot="{ loggedIn }">
  <UButton
    v-if="loggedIn"
    size="lg"
    trailing-icon="i-heroicons-arrow-right-16-solid"
    to="/chat"
  >
    Go to Chat
  </UButton>

  <div v-else class="flex flex-col items-center justify-center gap-y-2">
    <UButton
      size="lg"
      icon="i-simple-icons-github"
      to="/auth/github"
      external
    >
      Sign in with GitHub
    </UButton>
    <p class="text-sm text-gray-600 dark:text-gray-300 text-center">
      Start Chatting Now!
    </p>
  </div>
</AuthState>

Notice the external attribute on the Sign-In button. This attribute is essential—it tells the framework to treat the GitHub authentication as an external process. Without it, the framework will try to redirect you to the /auth/github route on the client side, causing errors (It did get me for sure).

This completes the server-side and client-side setup for user authentication. You can go through the shared GitHub repo to see how we also restrict navigation on the client side by using an auth middleware, ensuring that only authenticated users can access the chat page.

Bonus: Enhancing Engagement

As the project neared completion, I kept thinking about how to make it more engaging. I wanted to show what users were querying about and who they were querying for. However, I didn’t want to save every type of query—especially those like “When did I make my first commit?”—as they were both pointless and numerous. Instead, I decided to save only the queries made about other users, repositories, and other entities, excluding self-referential queries.

Database Setup

To achieve this, we needed a database, which is why we enabled database: true in our nuxt.config.ts file. This setting binds Cloudflare’s D1 database in production and uses its platform proxy during development.

First, we create the necessary database tables using the hubDatabase server composable:

await hubDatabase().exec(
  `CREATE TABLE IF NOT EXISTS queries (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    text TEXT NOT NULL,
    response TEXT NOT NULL,
    github_request TEXT NOT NULL,
    github_response TEXT NOT NULL,
    queried_at DATETIME DEFAULT CURRENT_TIMESTAMP
  );
`.replace(/\n/g, '')
);

// ... other tables and indexes

Next, we utilize utility functions to save the queries. The saveUserQuery function is invoked only if a tool call was made from the handleMessageWithOpenAI function we discussed earlier.

export type ToolCallDetails = {
  // eslint-disable-next-line @typescript-eslint/no-explicit-any
  response: any;
  request: SearchParams;
};

export type UserQuery = {
  userMessage: string;
  toolCalls: ToolCallDetails[];
  assistantReply: string;
};

const getAvatarUrl = (toolCall: ToolCallDetails) => {
  let avatarUrl;
  const responseItem = toolCall.response.items[0];

  if (responseItem) {
    if (responseItem.author) {
      avatarUrl = responseItem.author.avatar_url;
    } else if (responseItem.user) {
      avatarUrl = responseItem.user.avatar_url;
    } else if (responseItem.owner) {
      avatarUrl = responseItem.owner.avatar_url;
    } else if (responseItem.avatar_url) {
      avatarUrl = responseItem.avatar_url;
    }
  }

  return avatarUrl;
};

const shouldSaveUserQuery = (
  toolCall: ToolCallDetails,
  loggedInUser: string
) => {
  const responseItem = toolCall.response.items[0];
  if (
    responseItem &&
    ((responseItem.author && responseItem.author.login === loggedInUser) ||
      (responseItem.user && responseItem.user.login === loggedInUser) ||
      (responseItem.owner && responseItem.owner.login === loggedInUser) ||
      responseItem.login === loggedInUser)
  ) {
    return false;
  }

  return true;
};

export const saveUserQuery = async (
  loggedInUser: string,
  userQuery: UserQuery
) => {
  const toolCall = userQuery.toolCalls[0];
  const matchedUser = toolCall.request.q.match(/(?:author:|user:)(\S+)/);
  if (matchedUser) {
    const queriedUser = matchedUser[1].toLowerCase();
    if (queriedUser !== loggedInUser) {
      const avatarUrl = getAvatarUrl(toolCall);

      await storeQuery(
        userQuery.userMessage,
        userQuery.assistantReply,
        toolCall,
        { login: queriedUser, avatarUrl }
      );
    }
  } else if (shouldSaveUserQuery(toolCall, loggedInUser)) {
    await storeQuery(userQuery.userMessage, userQuery.assistantReply, toolCall);
  }
};

It checks if the query involves the currently logged-in user. If it does, we simply return without logging anything; otherwise, we store the query details in the database using:

const storeQuery = async (
  queryText: string,
  assistantReply: string,
  toolCall: ToolCallDetails,
  queriedUser?: { login: string; avatarUrl?: string }
) => {
  try {
    const db = hubDatabase();

    const queryStmt = db
      .prepare(
        'INSERT INTO queries (text, response, github_request, github_response) VALUES (?1, ?2, ?3, ?4)'
      )
      .bind(
        queryText,
        assistantReply,
        JSON.stringify(toolCall.request),
        JSON.stringify(toolCall.response)
      );
    if (queriedUser) {
      const [batchRes1, batchRes2] = await db.batch([
        queryStmt,
        db
          .prepare(
            `INSERT INTO trending_users (username, search_count, last_searched, avatar_url)
              VALUES (?1, 1, CURRENT_TIMESTAMP, ?2)
              ON CONFLICT(username)
              DO UPDATE SET search_count = search_count + 1, last_searched = CURRENT_TIMESTAMP, avatar_url = COALESCE(?2, avatar_url)`
          )
          .bind(queriedUser.login, queriedUser.avatarUrl),
      ]);

      console.log('storeQuery: ', batchRes1, batchRes2);
    } else {
      const res = await queryStmt.run();

      console.log('storeQuery: ', res);
    }
  } catch (error) {
    console.error('Failed to store query: ', error);
  }
};

API Endpoints

We then create API endpoints to retrieve the trending users and recent queries, as shown below (note the use of endpoint caching):

export const getRecentQueries = async () => {
  const db = hubDatabase();
  console.log('getRecentQueries');
  const result = await db
    .prepare(
      'SELECT id, text, response, queried_at FROM queries ORDER BY queried_at DESC LIMIT ?'
    )
    .bind(10)
    .all<RecentQuery>();

  console.log('getRecentQueries: ', result);
  return result.results;
};

export default defineCachedEventHandler(
  async () => {
    const results = await getRecentQueries();

    return results;
  },
  {
    maxAge: 10 * 60, // 10 minutes
  }
);

Frontend Integration

This setup allows us to display the data in the frontend, providing users with insights into trending queries and recently searched users, as illustrated in the screenshot below.

You can go through the shared GitHub repo to see the whole implementation in detail.

And with that, now you can also know what you did last summer—phew!

Source Code

The project’s code is open source and can be checked here

Deployment

You can deploy the app using your NuxtHub admin panel or manually through the NuxtHub CLI. For more details on deploying an app through NuxtHub, refer to the official documentation.

The best part is that this project is now listed as a template on the NuxtHub Templates page. So, if you already have a NuxtHub account, you can deploy this project in one click using the button below (Just remember to add the necessary environment variables in the panel).

Further Enhancements

Currently, we rely on the AI's ability to generate GitHub API queries from natural language input. While we’ve provided it with details about various qualifiers, it can still struggle with more complex or intricate queries. I also experimented with tool-calling models from Cloudflare’s Workers AI and Groq API, and found that gpt-4o performed better for these tasks. Claude 3.5 Sonnet should definitely offer even better results, but it tends to be more expensive.

Here are a few alternative approaches we could explore to improve query accuracy and results:

• Fine-tune/train a tool-calling model specifically on the GitHub Search API documentation.

• Create embeddings from the GitHub Search documentation and store them in a vector database. Cloudflare offers a free tier for its vector database now, which we could leverage. When a user query is made, we could retrieve relevant information from the embeddings and include it in the system prompt.

If you have any other ideas for improving this project—or any feedback—please feel free to share them in the comments!

Conclusion

And there you have it—a GitHub search tool, powered by OpenAI, wrapped in a chat interface, and ready for action. We’ve gone through setting up the project, streaming responses, authenticating users, and even added a sprinkle of engagement to make things interesting.

While the project has plenty of room for improvements, it’s a solid foundation to build upon. And who knows—maybe with a few tweaks, it’ll be predicting your next GitHub repo before you even think of it :-).

May all your commits be bug-free!

Until next time!

Keep adding the bits and soon you'll have a lot of bytes to share with the world.

But beyond these practical reasons, there’s something even more satisfying: the joy of creation and learning. So, if you're ready to dive in, maybe you'll pick up some new concepts like Server-Sent Events, response streaming, markdown parsing, and, of course, deploying your own chat playground without any of the usual fuss. So, get comfy—however you like—and let's get started!

Project Overview

Alright, so what exactly are we going to create here? As you might have guessed already, we will be creating a chat interface to talk to different text generation models supported by Cloudflare Workers AI. Below is a brief list of capabilities we will build along the way:

• Ability to set different LLM params, like temperature, max tokens, system prompt, top_p, top_k etc while keeping some of these optional

• Ability to turn LLM response streaming on/off

• Handle streaming/non-streaming LLM responses on both, the server and the client side

• Parsing LLM responses for markdown and display it appropriately

• Auto-scrolling the chat container as the response is streamed from the LLM endpoint

• Adding the dark mode (this one is trivial but let's add it here for completeness)

This is how the interface will look when we are through this article:

You can try it out live here: https://hub-chat.nuxt.dev/

We will cover each of the tasks in detail in the following sections.

Project Setup

Now that we've set the stage for our LLM playground project, let's dive into the technologies we'll be using and get our development environment ready.

Technologies we'll use

1. Nuxt 3: Nuxt 3 is a powerful Vue.js framework that will serve as the foundation of our application.

2. Nuxt UI: A Nuxt module that will help us create a sleek and responsive interface.

3. NuxtHub: NuxtHub is a deployment and administration platform for Nuxt, powered by Cloudflare. We can use NuxtHub to access different Cloudflare offerings like D1 database, Workers KV, R2 Storage, Workers AI etc. In this project, we'll use it to access the LLMs as well as to deploy our project

4. Nuxt MDC: For parsing and displaying the chat messages

Prerequisites

Apart from the basic prerequisites like Node/Npm, Code Editors, and some VueJs/Nuxt knowledge, you'll need the following to follow along:

1. A Cloudflare account: To use the Workers AI models, as well as to deploy your project on Cloudflare Pages for free. If you don't have it already, you can set it up here.

2. A NuxtHub Admin Account: NuxtHub admin is a web based dashboard to manage NuxtHub apps. You can create your account here.

Setting up the project

We can either start with a Nuxt (or Nuxt UI) template and add NuxtHub on top of it, or we can use the NuxtHub starter template and add Nuxt UI to it. We'll take the second approach:

1. Create a new NuxtHub project

    # Init the project and install dependencies
    npx nuxthub init cf-playground
   
    # Change into the created dir
    cd cf-playground
   

2. Add the Nuxt UI module. The below command will install the @nuxt/ui dependency as well as add it as a module in your nuxt config file

    npx nuxi module add ui
   

3. Similarly, add the Nuxt MDC module

    npx nuxi module add mdc
   

Now we have setup everything that we need for this project. You can try running the project with pnpm dev (or an equivalent command if you're using a different package manager) and visit http://localhost:3000 in your browser. If you've dark mode enabled on your system then you'll notice that dark mode is already up and running in the project (We just need a way to toggle it).

With our development environment set up, we're ready to start building our LLM playground. In the next section, we'll begin implementing our user interface using NuxtUI components.

Creating the UI Components

First, we will create a component that will let us set numerical values for LLM settings using either a range slider or an input box. Let's call it RangeInput.

The RangeInput Component

We utilize a FormGroup component from NuxtUI to create this component. We put a Range slider in the default slot and an input in the hint slot to achieve the desired outcome. Here is the complete component code

<template>
  <UFormGroup :label="label" :ui="{ container: 'mt-2' }">
    <template #hint>
      <UInput
        v-model="model"
        class="w-[72px]"
        type="number"
        :min="min"
        :max="max"
        :step="step"
      />
    </template>
    <URange v-model="model" :min="min" :max="max" :step="step" size="sm" />
  </UFormGroup>
</template>

<script setup lang="ts">
const model = defineModel({ type: Number, default: undefined });

defineProps({
  label: {
    type: String,
    required: true,
  },
  min: {
    type: Number,
    default: undefined,
  },
  max: {
    type: Number,
    default: undefined,
  },
  step: {
    type: Number,
    default: undefined,
  },
});
</script>

Instead of taking the model value as a prop and then emitting the changes manually, we use the defineModel macro to achieve two way binding. If the initial model value is undefined the input box will be empty. This will help in making some of the LLM params optional.

The LLMSettings Component

LLMSettings component utilizes many RangeInputs that we created above as most of the settings are numerical values. Apart from RangeInputs we use a textarea for the system prompt, a toggle for enabling/disabling response streaming, a select menu for choosing the LLM model, and an accordion for hiding the optional params.

Here are the relevant parts of the component

<template>
  <div class="h-full flex flex-col overflow-hidden">
    <!-- Settings Header Code -->
    <UDivider />
    <div class="p-4 flex-1 space-y-6 overflow-y-auto">
      <UFormGroup label="Model">
        <USelectMenu
          v-model="llmParams.model"
          size="md"
          :options="models"
          value-attribute="id"
          option-attribute="name"
        />
      </UFormGroup>

      <RangeInput
        v-model="llmParams.temperature"
        label="Temperature"
        :min="0"
        :max="5"
        :step="0.1"
      />

      <RangeInput
        v-model="llmParams.maxTokens"
        label="Max Tokens"
        :min="1"
        :max="4096"
      />

      <UFormGroup label="System Prompt">
        <UTextarea
          v-model="llmParams.systemPrompt"
          :rows="3"
          :maxrows="8"
          autoresize
        />
      </UFormGroup>

      <div class="flex items-center justify-between">
        <span>Stream Response</span>
        <UToggle v-model="llmParams.stream" />
      </div>

      <UAccordion
        :items="accordionItems"
        color="white"
        variant="solid"
        size="md"
      >
        <template #item>
          <UCard :ui="{ body: { base: 'space-y-6', padding: 'p-4 sm:p-4' } }">
            <RangeInput
              v-model="llmParams.topP"
              label="Top P"
              :min="0"
              :max="2"
              :step="0.1"
            />

            <!-- Other optional params -->
          </UCard>
        </template>
      </UAccordion>
    </div>
  </div>
</template>

<script setup lang="ts">
type LlmParams = {
  model: string;
  temperature: number;
  maxTokens: number;
  topP?: number;
  topK?: number;
  frequencyPenalty?: number;
  presencePenalty?: number;
  systemPrompt: string;
  stream: boolean;
};

const llmParams = defineModel('llmParams', {
  type: Object as () => LlmParams,
  required: true,
});

defineEmits(['hideDrawer', 'reset']);

const accordionItems = [
  {
    label: 'Advanced Settings',
    defaultOpen: false,
  },
];

const models = [
  {
    name: 'deepseek-coder-6.7b-base-awq',
    id: '@hf/thebloke/deepseek-coder-6.7b-base-awq',
  },
  { 
    name: 'llama-3-8b-instruct', 
    id: '@cf/meta/llama-3-8b-instruct', 
  },
  // ...other models
]
</script>

The ChatPanel Component

This is the most important component of all as it handles the core chat functionality of the app. It consists of three parts:

Chat Header

It displays the app name/label apart from some global buttons for clearing chats, dark mode toggling and for showing the settings drawer on mobile devices

<template>
  <div class="flex items-center justify-between p-4">
    <div class="flex items-center gap-x-4">
      <h2 class="text-xl md:text-2xl text-primary font-bold">Hub Chat</h2>
      <UTooltip text="Clear chat">
        <UButton
          color="gray"
          icon="i-heroicons-trash"
          size="xs"
          :disabled="clearDisabled"
          @click="$emit('clear')"
        />
      </UTooltip>
    </div>
    <div class="flex items-center gap-x-4">
      <ColorMode />
      <UButton
        icon="i-heroicons-cog-6-tooth"
        color="gray"
        variant="ghost"
        class="md:hidden"
        @click="$emit('showDrawer')"
      />
    </div>
  </div>
</template>

<script setup lang="ts">
defineEmits(['clear', 'showDrawer']);

defineProps({
  clearDisabled: {
    type: Boolean,
    default: true,
  },
});
</script>

Chats Container

For parsing and displaying the chat messages. It also shows a message loading skeleton when streaming is off, as well as a NoChats placeholder when needed.

<div ref="chatContainer" class="flex-1 overflow-y-auto p-4 space-y-5">
  <div
    v-for="(message, index) in chatHistory"
    :key="`message-${index}`"
    class="flex items-start gap-x-4"
  >
    <div
      class="w-12 h-12 p-2 rounded-full"
      :class="`${
        message.role === 'user' ? 'bg-primary/20' : 'bg-blue-500/20'
      }`"
    >
      <UIcon
        :name="`${
          message.role === 'user'
            ? 'i-mdi-user'
            : 'i-heroicons-sparkles-solid'
        }`"
        class="w-8 h-8"
        :class="`${
          message.role === 'user' ? 'text-primary-400' : 'text-blue-400'
        }`"
      />
    </div>
    <div v-if="message.role === 'user'">
      {{ message.content }}
    </div>
    <AssistantMessage v-else :content="message.content" />
  </div>
  <ChatLoadingSkeleton v-if="loading === 'message'" />
  <NoChats v-if="chatHistory.length === 0" class="h-full" />
</div>

To keep the user informed of the progress, we show a message loading skeleton when streaming is off. To do this we utilize a loading variable which can have three states: idle, message & stream. So when a non-streaming request is made we set the ref to message and the loading skeleton is shown.

User Message Textbox

For entering user message. It shows up as a single line textarea that resizes automatically when needed.

<div class="flex items-start p-3.5 relative">
  <UTextarea
    v-model="userMessage"
    placeholder="How can I help you today?"
    class="w-full"
    :ui="{ padding: { xl: 'pr-11' } }"
    :rows="1"
    :maxrows="5"
    :disabled="loading !== 'idle'"
    autoresize
    size="xl"
    @keydown.enter.exact.prevent="sendMessage"
    @keydown.enter.shift.exact.prevent="userMessage += '\n'"
  />

  <UButton
    icon="i-heroicons-arrow-up-20-solid"
    class="absolute top-5 right-5"
    :disabled="loading !== 'idle'"
    @click="sendMessage"
  />
</div>

For allowing the user to send message on hitting enter, we use the keydown event listener with the needed modifiers (@keydown.enter.exact.prevent). Similarly, for adding a newline we use enter + shift keys.

Now that we have our UI components in place, let's shift our focus to the backend. We'll set up the AI integration and create an API endpoint to handle our chat requests. This will bridge the gap between our frontend interface and the AI model.

Setting up AI and API Endpoint

Integrating AI into our project is very simple thanks to NuxtHub. Let's look at our nuxt.config.ts file in the root dir of the project.

// https://nuxt.com/docs/api/configuration/nuxt-config
export default defineNuxtConfig({
  compatibilityDate: '2024-07-30',
  // https://nuxt.com/docs/getting-started/upgrade#testing-nuxt-4
  future: { compatibilityVersion: 4 },

  // https://nuxt.com/modules
  modules: ['@nuxthub/core', '@nuxt/eslint', '@nuxtjs/mdc', "@nuxt/ui"],

  // https://hub.nuxt.com/docs/getting-started/installation#options
  hub: {},

  // Env variables - https://nuxt.com/docs/getting-started/configuration#environment-variables-and-private-tokens
  runtimeConfig: {
    public: {
      // Can be overridden by NUXT_PUBLIC_HELLO_TEXT environment variable
      helloText: 'Hello from the Edge 👋',
    },
  },

  // https://eslint.nuxt.com
  eslint: {
    config: {
      stylistic: {
        quotes: 'single',
      },
    },
  },

  // https://devtools.nuxt.com
  devtools: { enabled: true },
});

To enable AI, we just need to add the ai: true flag under hub config options above. If needed, we can enable other NuxtHub Cloudflare integrations like D1 database (database: true), Workers KV (kv: true) etc. While we are here, we can remove runtimeConfig as we don't need it. (You can also remove/modify the eslint config as per your code editor settings).

If we want to use AI in dev mode (which we definitely do), we need to link this project to a Cloudflare project. This is needed as we will be talking directly to the Cloudflare APIs as no workers are running yet (and also because the AI usage will be linked to your Cloudflare account). This is a straight forward task, just run the following command

npx nuxthub link

Running the link command will make sure that we are logged into to our NuxtHub account, and will allow us to create a new NuxtHub project (backed by Cloudflare) or choose an existing one.

Creating Chat API Endpoint

Let's create a chat api endpoint. Create a new file chat.post.ts ("post" in the file name signifies that this endpoint will only accept HTTP POST requests) in the server/api directory and add the following code to it

export default defineEventHandler(async (event) => {
  const { messages, params } = await readBody(event);
  if (!messages || messages.length === 0 || !params) {
    throw createError({
      statusCode: 400,
      statusMessage: 'Missing messages or LLM params',
    });
  }

  const config = {
    max_tokens: params.maxTokens,
    temperature: params.temperature,
    top_p: params.topP,
    top_k: params.topK,
    frequency_penalty: params.frequencyPenalty,
    presence_penalty: params.presencePenalty,
    stream: params.stream,
  };

  const ai = hubAI();

  try {
    const result = await ai.run(params.model, {
      messages: params.systemPrompt
        ? [{ role: 'system', content: params.systemPrompt }, ...messages]
        : messages,
      ...config,
    });

    return params.stream
      ? sendStream(event, result as ReadableStream)
      : (
          result as {
            response: string;
          }
        ).response;
  } catch (error) {
    console.error(error);
    throw createError({
      statusCode: 500,
      statusMessage: 'Error processing request',
    });
  }
});

hubAI is a server composable that returns a Workers AI client for interacting with the LLMs. We pass the messages as well as the LLM params that we received from the frontend and run the requested model.

If you look at the above code carefully you'll notice that this endpoint supports both: streaming and non-streaming responses. To return a stream from the endpoint we just need to call the sendStream utility function.

We are through with our server side code. Let's look at how to handle the stream response on the client side in the next section.

Consuming Server Sent Events

Before diving into how to handle stream responses, let's understand why streaming is crucial for LLM interactions and how Server Sent Events (SSE) facilitate this process.

Why Stream LLM Responses with Server Sent Events?

Large Language Models (LLMs) can take considerable time to generate complete responses, especially for complex queries. Traditionally, web applications wait for the entire response before displaying anything to the user. However, this approach can lead to long waiting times and a less engaging user experience.

Server Sent Events (SSE) offer a solution to this problem.

Here's how SSE benefits LLM responses:

1. Immediate Feedback: As soon as the LLM starts generating content, it can be sent to the client and displayed, providing immediate feedback to the user.

2. Improved Perceived Performance: Users see content appearing progressively, giving the impression of a faster, more responsive system.

3. Real-time Interaction: The gradual appearance of text mimics human typing, creating a more natural and engaging conversational experience.

Technically, SSE works by establishing a unidirectional channel from the server to the client. The server sends text data encoded in UTF-8, separated by newline characters. This simple yet effective mechanism allows for real-time updates in web applications, making it ideal for streaming LLM responses.

Now that we understand the importance of streaming for LLM responses and how SSE enables this, let's look at how to implement this in our Nuxt 3 application.

Handling Server Sent Events with Nuxt 3 POST Requests

Since ours is a POST request, we need handle it differently. Nuxt 3 docs gives you an excellent starting point to do this. Reproducing the code from the docs

// Make a POST request to the SSE endpoint
const response = await $fetch<ReadableStream>('/chats/ask-ai', {
  method: 'POST',
  body: {
    query: "Hello AI, how are you?",
  },
  responseType: 'stream',
})

// Create a new ReadableStream from the response with TextDecoderStream to get the data as text
const reader = response.pipeThrough(new TextDecoderStream()).getReader()

// Read the chunk of data as we get it
while (true) {
  const { value, done } = await reader.read()

  if (done)
    break

  console.log('Received:', value)
}

We need to set the request responseType flag as stream, and set the type of $fetch response as ReadableStream. Then we create a stream reader while decoding the received chunks by piping the events through a TextDecoder.

Below is a glimpse of the events data received from our chat endpoint (sent by the LLM)

data: {"response":"Hello","p":"abcdefghijklmnopqrstuvwxyz0123456789abcdefghij"}

data: {"response":"!","p":"abcdefgh"}

data: [DONE]

These events may contain more than one data line per event, and some events may not contain a complete JSON object (rest of the object will arrive with the next event). To handle this stream we can create a composable with a streamResponse generator function as shown below

export function useChat() {
  async function* streamResponse(
    url: string,
    messages: ChatMessage[],
    llmParams: LlmParams
  ) {
    let buffer = '';

    try {
      const response = await $fetch<ReadableStream>(url, {
        method: 'POST',
        body: {
          messages,
          params: llmParams,
        },
        responseType: 'stream',
      });

      const reader = response.pipeThrough(new TextDecoderStream()).getReader();

      while (true) {
        const { value, done } = await reader.read();

        if (done) {
          if (buffer.trim()) {
            console.warn('Stream ended with unparsed data:', buffer);
          }

          return;
        }

        buffer += value;
        const lines = buffer.split('\n');
        buffer = lines.pop() || '';

        for (const line of lines) {
          if (line.startsWith('data: ')) {
            const data = line.slice('data: '.length).trim();
            if (data === '[DONE]') {
              return;
            }

            try {
              const jsonData = JSON.parse(data);
              if (jsonData.response) {
                yield jsonData.response;
              }
            } catch (parseError) {
              console.warn('Error parsing JSON:', parseError);
            }
          }
        }
      }
    } catch (error) {
      console.error('Error sending message:', error);

      throw error;
    }
  }

  // For handling non-streaming responses
  async function getResponse() {}

  return {
    getResponse,
    streamResponse,
  };
}

To handle non streaming responses we can also create a simple $fetch call wrapper function in the same composable (check out the code in the linked Github Repo).

Now that we understand how to consume Server Sent Events and handle streaming responses, let's put all the pieces together in our main chat interface. We'll integrate the components we've built, set up the chat API call, and use our useChat composable to manage the LLM responses.

Final Chat Interface Page

The below is the final code of the index page (our app has only one page). Here we use all the components that we created before, and call the chat api endpoint. Then we use the useChat composable to handle the LLM responses.

<template>
  <div class="h-screen flex flex-col md:flex-row">
    <USlideover
      v-model="isDrawerOpen"
      class="md:hidden"
      :ui="{ width: 'max-w-xs' }"
    >
      <LlmSettings
        v-model:llmParams="llmParams"
        @hide-drawer="isDrawerOpen = false"
        @reset="resetSettings"
      />
    </USlideover>

    <div class="hidden md:block md:w-1/3 lg:w-1/4">
      <LlmSettings v-model:llmParams="llmParams" @reset="resetSettings" />
    </div>

    <UDivider orientation="vertical" class="hidden md:block" />

    <div class="flex-grow md:w-2/3 lg:w-3/4">
      <ChatPanel
        :chat-history="chatHistory"
        :loading="loading"
        @clear="chatHistory = []"
        @message="sendMessage"
        @show-drawer="isDrawerOpen = true"
      />
    </div>
  </div>
</template>

<script setup lang="ts">
import type { ChatMessage, LlmParams, LoadingType } from '~~/types';

const isDrawerOpen = ref(false);

const defaultSettings: LlmParams = {
  model: '@cf/meta/llama-3.1-8b-instruct',
  temperature: 0.6,
  maxTokens: 512,
  systemPrompt: 'You are a helpful assistant.',
  stream: true,
};

const llmParams = reactive<LlmParams>({ ...defaultSettings });
const resetSettings = () => {
  Object.assign(llmParams, defaultSettings);
};

const { getResponse, streamResponse } = useChat();
const chatHistory = ref<ChatMessage[]>([]);
const loading = ref<LoadingType>('idle');
async function sendMessage(message: string) {
  chatHistory.value.push({ role: 'user', content: message });

  try {
    if (llmParams.stream) {
      loading.value = 'stream';
      const messageGenerator = streamResponse(
        '/api/chat',
        chatHistory.value,
        llmParams
      );

      let responseAdded = false;
      for await (const chunk of messageGenerator) {
        if (responseAdded) {
          // add the response to the current message
          chatHistory.value[chatHistory.value.length - 1]!.content += chunk;
        } else {
          // add a new message to the chat history
          chatHistory.value.push({
            role: 'assistant',
            content: chunk,
          });

          responseAdded = true;
        }
      }
    } else {
      loading.value = 'message';
      const response = await getResponse(
        '/api/chat',
        chatHistory.value,
        llmParams
      );

      chatHistory.value.push({ role: 'assistant', content: response });
    }
  } catch (error) {
    console.error('Error sending message:', error);
  } finally {
    loading.value = 'idle';
  }
}
</script>

This completes bulk of the coding. The only things remaining are:

1. Response parsing for markdown and display

2. Auto scrolling the chat container

Let's tackle these in the next section.

Polishing the Chat Interface

We will finish the remaining items in our task list now and call it a day. Stay with me, it won't take long now.

Using Nuxt MDC to Parse & Display Messages

If you look at the Chats Container code in one of the previous sections you'll notice a component AssistantMessage for displaying the response. Reproducing the relevant code here

<div v-if="message.role === 'user'">
  {{ message.content }}
</div>
<AssistantMessage v-else :content="message.content" />

We use the parseMarkdown utility function from the MDC module that we included earlier to parse the content, and then use the MDCRenderer component from the same module for displaying it. To take care of streaming response we add a watch for the message content and redo the parsing with useAsyncData composable.

<template>
  <MDCRenderer class="flex-1 prose dark:prose-invert" :body="ast?.body" />
</template>

<script setup lang="ts">
import { parseMarkdown } from '#imports';

const props = defineProps<{
  content: string;
}>();

const { data: ast, refresh } = await useAsyncData(useId(), () =>
  parseMarkdown(props.content)
);

watch(
  () => props.content,
  () => {
    refresh();
  }
);
</script>

The <MDC> component was causing overwriting of previous messages that were similar (the start of the message) to the latest message from the API endpoint. This happens because we don't have a unique key in our messages. Here is the relevant code from the <MDC> component.

const key = computed(() => hash(props.value))

const { data, refresh, error } = await useAsyncData(key.value, async () => {
  if (typeof props.value !== 'string') {
    return props.value
  }
  return await parseMarkdown(props.value, props.parserOptions)
})

As you can see, the key for useAsyncData is generated based on the content props value. So if the server streaming responses start with the same letters (e.g., Here's a joke..., Here's another...), the key will be same, and all similar previous responses from the server gets overwritten in the UI.

In the AssistantMessage component we are using useId composable to generate a unique id for each useAsyncData call, so the issue doesn't occur.

Using MutationObserver to Auto Scroll the Chats Container

In the ChatPanel component, the only scrolling part is the chats container. There can be other ways to observe the changes in the container but the simplest one I found is a MutationObserver.

In the case of streaming, we keep appending new content to the latest message that is being displayed. So to handle this effectively, we create a MutationObserver, give it a target to watch (the ChatsContainer), and some criteria to watch for (childList, subtree & characterData).

Here is the relevant code to do this

const chatContainer = ref<HTMLElement | null>(null);
let observer: MutationObserver | null = null;

onMounted(() => {
  if (chatContainer.value) {
    observer = new MutationObserver(() => {
      if (chatContainer.value) {
        chatContainer.value.scrollTop = chatContainer.value.scrollHeight;
      }
    });

    observer.observe(chatContainer.value, {
      childList: true,
      subtree: true,
      characterData: true,
    });
  }
});

onUnmounted(() => {
  if (observer) {
    observer.disconnect();
  }
});

When the ChatsPanel gets mounted we create a new MutationObserver which when based on the given criteria, we make the chats container scroll to its fullest.

Bonus: Handling the Dark Mode

As previously mentioned, the dark mode is already working; we just need a way to toggle it. We can also change the flavor of gray we want in the app. This can be done by adding an app.config.ts file in the app directory.

// app.config.ts
export default defineAppConfig({
  ui: {
    primary: 'orange',
    gray: 'slate',
  },
});

Add the following in your app.vue file for setting the required background colors.

<script setup lang="ts">
useHead({
  bodyAttrs: {
    class: 'bg-white dark:bg-gray-900',
  },
});
</script>

And add a new ColorMode component in your app/components directory.

<template>
  <ClientOnly>
    <UButton
      :icon="isDark ? 'i-heroicons-moon-20-solid' : 'i-heroicons-sun-20-solid'"
      color="gray"
      variant="ghost"
      aria-label="Theme"
      @click="isDark = !isDark"
    />

    <template #fallback>
      <div class="w-8 h-8" />
    </template>
  </ClientOnly>
</template>

<script setup lang="ts">
const colorMode = useColorMode();

const isDark = computed({
  get() {
    return colorMode.value === 'dark';
  },
  set() {
    colorMode.preference = colorMode.value === 'dark' ? 'light' : 'dark';
  },
});
</script>

Now you can use this color mode toggle button anywhere you like. In our app we've added it in the ChatHeader component.

Phew! And we have completed all the tasks we had set out to complete at the beginning of this article.

Deploying the App

You can deploy the project in multiple ways. I would recommend to do it through the NuxtHub Admin console. Push your code to a Github repository, link this repository with NuxtHub and then deploy from the Admin console.

But if you want to see it live right away then you can use the below command

npx nuxthub deploy

For more details on deployment you can visit the NuxtHub Docs.

Source Code

I only covered the most important parts of the source code here. You can visit the linked GIthub Repo for the complete source code. It should be self explanatory, but if you have question then please feel free to drop a comment below.

Conclusion

Congratulations! You've successfully built a feature-rich LLM playground using Nuxt 3, NuxtUI, and Cloudflare Workers AI through NuxtHub. We've covered a wide range of topics, including:

• Handling streaming responses using Server Sent Events

• Parsing and displaying markdown content in chat messages

• Implementing auto-scrolling for a better user experience etc.

You can take this project as the starting point and improve it further by:

• Adding the ability to talks to other types of LLMs, e.g. text-to-image, image-to-text, speech recognition etc.

• Implementing user authentication and session management

• Adding support for multiple conversation threads etc.

Thank you for staying till the end. I hope you learned some new concepts from this article. Do let me know your learnings in the comments section. Your feedback and experiences are valuable not just to me, but to the entire community of developers exploring this fascinating field.

Until next time!

Keep adding the bits and soon you'll have a lot of bytes to share with the world.

Traditionally, selecting a domain name has been a mix of creativity, gut feeling, and perhaps a quick poll among people in your circle of influence. The post-GPT era might have added another ingredient to the mix: consulting your trusted AI/LLM. But if you have talked to this advisor, it generally throws some names at you without revealing why it chose these names. And this is where "Name Insights" steps in, offering not just suggestions, but insights into the 'why' behind those domain name choices.

Introduction

I have been thinking about domain names (hoarding to be frank) for quite some time now. Maybe because I have bought more domains in the recent past than my usual appetite allows me to. And the first step to this whole process is: knowing the name that you want to buy.

After the usual availability checks, you wonder whether that weird-looking name is right for the purpose you're buying it for. Let's be honest, all the good ones are already taken (why does it feel like I'm not writing a tech article?) or the price is simply beyond your reach. Sometimes you zero down on more than one name, so which one should you go ahead with? These were the guiding questions behind building Name Insights.

Name Insights aims to address these challenges by providing detailed, contextual answers to help you make informed decisions about domain names.

App Features

Name Insights offers three powerful services to assist in your domain name decision-making process. Each service utilizes an AI-driven scoring system that evaluates domains based on six key aspects of what makes a good domain name. Let's explore each feature:

Name Insights & Scoring

This feature provides an unbiased view of what kind of app or service is best suited for a given domain name. It:

• Evaluates the domain on 6 different parameters, namely:

  1. Brand Impact: Memorability, brandability, uniqueness, emotional appeal.

  2. Usability: Length, spelling simplicity, pronunciation clarity, absence of numbers/hyphens.

  3. Relevance and SEO: Relevance to purpose, keyword inclusion, extension potential.

  4. Technical Considerations: TLD appropriateness, potential social media availability.

  5. Legal and Cultural Factors: Potential trademark risks, cultural/linguistic considerations.

  6. Market Potential: Ability to target desired audience, scalability for business growth.

• Provides an overall score based on weighted parameters.

• Offers a brief explanation for each score.

• Highlights pros and cons of the domain name.

Domain Names Comparison

This feature helps you choose between different options you might have. It:

• Compares two different domain names

• Uses the same scoring strategy as the insights feature

• Determines a "winning" name

• Provides a brief summary explaining the choice

Domain Name Ideas

This feature is most useful when you're starting from scratch. It:

• Builds upon the same domain name scoring strategy, and offers five distinct domain name suggestions based on your app/service idea

• Provides an overall score for each suggestion

• Breaks down different category scores

• Offers a brief summary of strengths and weaknesses for each name

App Demo/Screenshots

Home Page

Name insights loading animation

Name Insights Page (for hashnode.com domain)

Name comparison page

Comparison loading animation

Comparison results for fitnawake.com & fitandawake.com

Name ideas results page

Technical Details

Here is a brief overview of the app stack and approach

1. Nuxt3: Name Insights uses the full-stack capabilities of Nuxt3 as its backbone for a faster turnaround time. You can achieve the same results using another framework of your choice, but for me it made more sense because of where it is hosted (see point 2).

2. NuxtHub: The app is hosted on Cloudflare Pages using NuxtHub. NuxtHub offers a great DX for hosting serverless Nuxt apps, and for talking to various Cloudflare services. At the time of writing this article it supports Cloudflare features such as KV, D1, and R2.

3. NuxtUI: The app UI is built using NuxtUI, a collection of prebuilt components based on HeadlessUI & TailwindCSS.

4. Claude 3.5 Sonnet / Anthropic AI: The core features of the app are thanks to the Anthropic AI SDK, using the Claude 3.5 Sonnet model. The best in class reasoning capabilities of the model allows for a good overall experience of the app features.

At the heart of the app features is the domain scoring strategy. At present, the app relies on the advanced reasoning capabilities of the Claude 3.5 Sonnet model to grade a domain name. The key to the implementation lies in carefully crafted system prompts (which you call Prompt Engineering). These prompts instruct the AI on how to analyze domain names, what factors to consider, and how to present the results. E.g. the below is the core part of the system prompt for scoring a domain and generating insights:

const nameScorePrompt = `You are an AI assistant specialized in 
evaluating and scoring domain names. Analyze the given domain name 
as if you're seeing it for the very first time, without any prior 
knowledge of its actual use or purpose.

Evaluate the domain based on these categories and weights:
1. Brand Impact (30%)
2. Usability (20%)
3. Relevance and SEO (20%)
4. Technical Considerations (15%)
5. Legal and Cultural Factors (10%)
6. Market Potential (5%)

Provide a score out of 100 for each category, along with a brief, 
unbiased explanation. Calculate the weighted overall score out of 100 
(rounded to the nearest integer). Include concise lists of strengths 
and weaknesses of the domain name based solely on its characteristics, 
not its known use. 
`

const response = await this.anthropic.messages.create({
  model: "claude-3-5-sonnet-20240620",
  max_tokens: 1000,
  temperature: 0.4,
  system: systemPrompt,
  messages: [
    {
      role: "user",
      content: `Analyze the domain name: ${domainName}`,
    },
  ],
});

The prompt is intentionally detailed and specific to get the desired results from the LLM. We incorporate the required JSON structure into the prompt to receive the output in JSON format. This output is then parsed using the following function:

export function extractAndParseJson<T>(text: string): T {
  try {
    return JSON.parse(text) as T;
  } catch (error) {
    console.error("Error parsing JSON:", error);
  }

  const backtickPattern = /```(?:json)?\s*([\s\S]*?)\s*```/g;
  const matches = text.match(backtickPattern);

  if (matches) {
    for (const match of matches) {
      const content = match.replace(/```(?:json)?\s*|\s*```/g, "").trim();
      try {
        return JSON.parse(content) as T;
      } catch (error) {
        continue;
      }
    }
  }

  throw new Error("No valid JSON found in the text");
}

To try out different LLMs from other AI services, the backend implementation is kept generic using interfaces which every AI service needs to implement and extend.

Here is the AIService interface:

export interface AIService {
  getDomainScore(domainName: string): Promise<string>;
  compareDomains(firstDomain: string, secondDomain: string): Promise<string>;
  getDomainSuggestions(purpose: string): Promise<string>;
}

And the BaseAIService

export abstract class BaseAIService implements AIService {
  protected getSystemPrompt(promptType: SystemPromptType): string {
    return getSystemPrompt(promptType);
  }

  abstract getDomainScore(domainName: string): Promise<string>;

  abstract compareDomains(
    firstDomain: string,
    secondDomain: string
  ): Promise<string>;

  abstract getDomainSuggestions(purpose: string): Promise<string>;
}

This approach provides a common system prompt for all AI services while allowing individual services the flexibility to override and define their own custom system prompts if needed.

App & Repo Links

You can try out the app live at https://name-insights.nuxt.dev/

The complete app code can be found here:

Limitations

As the app relies entirely on an LLM for its core functionality, the output is not deterministic. This means that if you analyze the same domain name multiple times, you may receive slightly different scores. This variability is inherent to large language models, which can produce different outputs based on subtle differences in how they process the input each time.

However, in my testing, I've found that this limitation doesn't significantly impact the app's utility. The variance in scores tends to be low, and the overall insights remain consistent. Name Insights is an excellent sounding board for giving you a head start in your domain name search.

Further Enhancements

This is just the beginning. Below are some of the actions items that can further enhance the app outcome and usefulness:

1. Integrating other LLMs and utilize multiple models simultaneously for arriving at a score and generating insights

2. Adding real world data to the mix. This will make the scoring more trustworthy and accurate.

3. Adding real time domain name availability checks, and possibly the social media handles.

Conclusion

Selecting the perfect domain name is a critical step in establishing your online presence. With its three distinct features Name Insights aims to make the process easier at various stages of the domain name search journey. It helps you to:

• Gain objective insights into the strengths and weaknesses of potential domain names

• Make informed comparisons between different options

• Generate fresh ideas tailored to your specific needs

Irrespective of the slight variability in results, the overall consistency and depth of analysis provided by Name Insights make it an invaluable resource for smarter domain name decisions.

I hope you liked reading the article. Do try out the app and it would mean the world to me if you can share your feedback with me.

Until next time...!

Keep adding the bits and soon you'll have a lot of bytes to share with the world.

Introduction

While helping my son with his studies, I often struggle with creating questions to ask him to test his knowledge. It has more to do with the idea of reading the content myself first, which I find tedious. I can ask the questions at the chapter's end, but that doesn't feel comprehensive enough. In these situations, I often think, "If only someone could read the chapter for me, comprehend the content, and generate relevant questions along with the answer keys".

With the advancements in AI and the recent interest in its generative capabilities (ChatGPT, Bard, etc.), I wondered if such a solution is possible using AI. To keep the experiment short, I looked for a low-code solution to achieve this, and Firebase extensions seemed like the right choice.

What is generative AI?

As the name suggests, generative AI is a type of artificial intelligence that can create new content or data based on its learning from existing data. This new content or data can be similar to the existing data or be entirely original. It usually works by giving some content to the AI along with your instructions (generally called a prompt) on how to use that content, and the AI gives back an appropriate response.

Now that you know about generative AI, you should have a basic idea about the approach you'll need to take to get the desired results. Let's break it down into steps

1. Get the text content from the user for which they want to generate questions

2. Pass the content along with a suitable instruction set to the AI

3. Parse the response from the AI and present it to the user in a suitable format

Implementation

The implementation of the quiz generator is quite simple: use a textarea to get the text content from the user and pass it to the configured ChatGPT Firebase extension. You can configure the said extension as shown below.

Configuring the ChatGPT extension

Go to the Firebase extensions hub, search for chat in the search box, and click "Chatbot with ChatGPT" from the search results.

You can install this extension from the extension page in one of your Firebase projects. Click on the "Install in Firebase console" button, select your project from the new tab that opens up, and complete the steps for installing the extension into the project.

Click the Next button. This extension creates a cloud function called to generate a response, and we need to enable the Secrets Manager API to store our OpenAI API Key securely. Based on the current state of your project, you may need to enable other APIs (Cloud Functions, Artifact Registry, etc.). Click Enable and then click Next

Next, the extension needs to use Cloud Firestore (If you haven't created a Cloud Firestore yet, it will create one for you in datastore mode. For using it with Firebase, you'll need to change it back to the native mode from Google Cloud Console) and Secrets Manager to process the input text, the OpenAI API response, and the stored OpenAI API key. Click Next

Finally, we need to configure the extension. You need to configure the following

1. Add your OpenAI API Key (you can create it here). Make sure to click on the Create Secret button to create a secret in the Secrets Manager

2. Select the desired model (GPT 3.5/4)

3. Set the temperature to a low value (0.2-0.4). The temperature value (allowed range 0-2) dictates the randomness of the AI response; higher values make the output more random. I used a temperature of 0.3.

4. Set the cloud function location (ideally set it to the exact location where your Cloud Firestore is located)

5. You'll also need to configure the collection path and the field names where the input text and the API response will be stored. For this test, I used requests as the collection name and left the field names as it is.

We can leave the rest of the settings as it is and click on the Install extension button. Wait for the installation to complete; now we're ready to test it.

Testing the extension

To test the extension you need to create the collection that you configured in the last section (requests in my case) and add a prompt field to it.

The way this extension works is:

1. You add the input text and your instructions to the prompt field

2. The cloud function (generateAIResponse) created by the extension gets invoked automatically

3. The function sends your prompt to the OpenAI API and creates a status field (of type map) with its state key set to PROCESSING

4. On a successful response from the API call, the state is changed to COMPLETED and the API call output is stored in a new field named response

Prompt Structure

How do you structure your prompt? A good prompt should start by setting a context so that ChatGPT behaves accordingly. Then, you state your requirement, followed by the input text, and finally, you give a response cue to the AI.

To summarize, you can use the below prompt structure

Context + Instruction + Input Text + Response Cue

For the quiz generator, you can use the following initial prompt

“You're a teacher who needs to create quizzes to test your student's
knowledge. Using the given text content create several distinct MCQs for the
purpose.

The content: {{text_content}}

Your response:”

The above prompt generates a response in a format similar to the one below. Please note that the format may vary slightly for you.

1. question text
    a) Option 1
    b) Option 2
    ...
...

This is a very good start, but the API is not returning the answers to these questions. Also, it is difficult to use it as a quiz as the response lacks structure.

Structuring the response

For better control over the whole thing, you need a structured response, such as a JSON formatted output, from the ChatGPT API call. Tweak the instruction part of your prompt to the following:

Using the given text content creates several distinct MCQs for the purpose. 
Return your response as an RFC8259-compliant JSON array using the structure
[{"question": "the question", "choices": ["choice 1", "choice 2", ...],
"answer": "the correct choice"}, {...}]

Here I’ve intentionally removed any spaces from the JSON to save some tokens from the response. Rerunning the same test with the changed instruction gives us the desired structured result. Now, you can display the questions & the choices in any way you want using your favourite technology. As you have the answer key, you can also convert these questions into a quiz.

Using images as content

You can extend it further and use images that contain text (say, a snapshot of a book's pages) as the starting point of the quiz generator. The underlying approach of the solution remains the same; you're just adding a step at the beginning. Using another Firebase extension, you can extract the text from an image and use that as input like before.

Configuring the Cloud Vision extension

Go back to the Firebase extensions hub, search for vision, and click on "Extract Image Text with Cloud Vision AI" from the search results. Install the extension by following the steps you completed for the previous extension.

This extension enables Google's Cloud Vision API and creates a new cloud function named extractText. Apart from the Cloud Firestore User permission, it also needs the Storage Admin permission (the extension reads the images from Cloud Storage).

Finally, configure the extension as follows (you should keep your cloud function's location near your Cloud Storage/Firestore's location). You can leave the rest of the settings as it is and install the extension

Once the extension has finished installing, you can test it out by uploading an image to your cloud storage bucket under the snapshots folder. A few seconds after adding the image, you should see a new collection, extractedText with a single document containing the text that was extracted from the image.

For my test, I used the following screenshot

And this is the result of the Cloud Vision extension

You can use this text as input to the ChatGPT extension and generate questions as before.

Conclusion

Firebase extensions are powerful low-code options for many different use cases. As we saw in this article, leveraging the power of ChatGPT and Cloud Vision extensions could make the basic quiz generator functionality work without writing a single line of code.

Now, you can integrate Cloud Storage and Firestore into your front end to create a personalized and engaging quiz generator that generates relevant questions based on the given content.

I hope you liked reading the article. If you've questions or just want to say hi, just drop a message in the comments section.

-- Keep adding the bits, soon you'll have more bytes than you may need. :-)

Introduction

When I learnt about the Outerbase hackathon on Hashnode I was intrigued about it. I thought it was another database, "base" being the operative word. But I was only half right, or maybe half wrong, it depends on whom you're asking. Outerbase is an interface to your data (currently only residing in some relational databases), but it adds a lot of bells and whistles to make it interesting.

Some of the features include:

1. Commands: These are functions in the cloud (Lambda functions?) that can talk to your database. You can create a chain of nodes to handle different steps of the process (AWS Step functions? Of course, it is not there yet but maybe soon...)

2. EZQL: It enables you to ask questions to your database in plain text. No more making your own SQL queries.

3. Plugins: Your data tables are much more than a spreadsheet, Outerbase plugins enable you to visualize that. You can add different plugins for different types of data, and interact with it in a way that feels native.

The last feature is what caught my fancy, and I decided to focus on that for this hackathon. Here is a short video demo of the plugins that I created.

Preparing the base with Bun

Since I decided to focus on plugins, I wanted a way to quickly create the basic template on top of which I could build upon. One way was to just get the template from the Outerbase repo and copy-paste it to create more of it. But where is the fun in that? There is a saying, "Automate it, silly." (Even if it takes you days to build that automation). So that is the path I took.

Creating templates

You can create local templates (and maybe publish them later on) with Bun and then run a simple command to use that template. Your local templates should be present inside a .bun-create folder in the following paths

• $HOME/.bun-create/<name>: global templates

• <project root>/.bun-create/<name>: project-specific templates

<name> is the template/folder name you want to use. Drop your template files into the template folder and run the following command to use that template

# Notice the "./" in the beginning. Without that it looks 
# for the template on Github (bug). 
bun create ./<template-name> <destination>

There are two types of Outerbase plugins;

• Table plugins: these work on the whole database table, and

• Cell plugins: made for working with a cell but applied on a column so that they're available to all the cells of that column

An Outerbase plugin consists of at most three views; 1. The configuration view, 2. The data view, and 3. The data editor/dialog view. These views are created using Web Components (Custom HTML Elements). A basic Outerbase component can be represented as follows

const templateEditor = document.createElement("template");
templateEditor.innerHTML = `
<style>
  #container {
    max-width: 320px;
  }
</style>

<div id="container"></div>
`;

class OuterbasePluginCellEditor extends HTMLElement {
  static get observedAttributes() {
    return [...observed_attributes];
  }

  config = new OuterbasePluginConfig({});

  constructor() {
    super();

    this.shadow = this.attachShadow({ mode: "open" });
    this.shadow.appendChild(
      templateEditor.content.cloneNode(true)
    );
  }

  connectedCallback() {
    this.config = new OuterbasePluginConfig(
      decodeAttributeByName(this, "configuration")
    );

    this.config.cellValue = this.getAttribute("cellvalue");
    this.render();
  }

  render() {}
}

Before we can use this component we need to register this component with the window

window.customElements.define(
  "outerbase-plugin-cell-editor",
  OuterbasePluginCellEditor
);

The first plugin: Audio Player

If your data consists of audio urls, wouldn't it be cool to play the audio from the database view itself? This Outerbase cell plugin allows you to do exactly that.

All the magic of this plugin resides in its Editor view. We simply attach an HTML audio element to the view. Set the audio src and load it. And our audio link is ready to play. Below is the updated render method shown earlier.

render() {
  const srcUrl = this.getAttribute("cellvalue");
  if (srcUrl) {
    this.shadow.getElementById(
      "container"
    ).innerHTML = `<audio id="audio-player" controls />`;
    const player = this.shadow.getElementById("audio-player");
    player.src = srcUrl;
    player.load();
  }
}

The above code allows us to get this view (the audio player dialog)

The second plugin: Video Player

Since we have an audio player for our database now, it is only logical to do the same thing for videos. But this is not that simple. Most of the video links you encounter are YouTube (and maybe some Vimeo) video links, so I set out to make that work.

Now, the URLs that we see in the browser's address bar (watch URLs) for YouTube videos are different from when you want to embed the YouTube player in your website. Assuming the database will store the watch URLs, we need to create the embed URLs. The below function does exactly that using a regex.

getYouTubeEmbedUrl(url) {
  const youtubeRegex =
    /^.*(youtu.be\/|v\/|u\/\w\/|embed\/|watch\?v=|\&v=)([^#\&\?]*).*/;
  const match = url.match(youtubeRegex);
  if (match && match[2].length == 11) {
    return `https://youtube.com/embed/${match[2]}`;
  }
}

Now we can create an iframe, set its src as this embed URL and our player should be ready. We do all this in the editor/dialog view of the plugin.

this.shadow.getElementById("container").innerHTML = 
    `<iframe id="video-player" type="text/html" width="360" height="240" frameborder="0" />`;
const player = this.shadow.getElementById("video-player");
player.src = embedUrl;

The above code works fine locally but when deployed to the Outerbase console it fails to load the player. The reason is Outerbase plugins run inside a sandboxed iframe to keep the data secure. The YouTube player needs to use the browser cache to store its assets but the configured sandbox permissions don't allow it, and the player fails to appear.

But the same technique can be utilized for other video hosting platforms, Vimeo being one of them. The problem with Vimeo is that it doesn't have well structured consistent URLs. So we use its oEmbed APIs to fetch the correct URL.

async getVimeoEmbedUrl(url) {
  const res = await fetch(`https://vimeo.com/api/oembed.json?url=${url}`);

  const data = await res.json();
  return `https://player.vimeo.com/video/${data.video_id}?title=0&byline=0&dnt=1`;
}

And now we can use the same iframe player to play this video.

dnt parameter saved the day for this plugin, else all that work would have come to nought. Here is the view we get from the player

There is still one error present in the browser console for the Vimeo player, and that is for missing the presentation permission. This can be alleviated by sandbox="allow-presentation" on the parent iframe, or maybe there is some other Vimeo URL query param that can help with that. I haven't explored further as even with the error in the console, the video can be played (The first player load doesn't work, but no problem afterwards).

The third plugin: Markdown Editor

How cool it would be to create blog posts or write docs from the database view itself? This md-editor plugin is exactly what you need for all such cases.

Let's first take a peek at the plugins editor view

This plugin is different from the other plugins we've seen so far, as here we need to use third-party scripts and CSS to achieve the goal. How do we go about this, and which markdown editor to integrate? These are important questions to answer. Let's tackle these questions one by one

1. How to include third-party scripts and CSS: Even though we're using web components, these are still part of the DOM, so maybe we can create script and link tags dynamically to load these. An example would be as follows

    // JavaScript
    function loadCSS(url) {
      const link = document.createElement('link');
      link.rel = 'stylesheet';
      link.href = url;
      document.head.appendChild(link);
    }
   
    function loadJS(url) {
      const script = document.createElement('script');
      script.src = url;
      document.body.appendChild(script);
    }
   
    // Example usage:
    loadCSS('https://cdnjs.cloudflare.com/ajax/libs/bootstrap/4.6.0/css/bootstrap.min.css');
    loadJS('https://code.jquery.com/jquery-3.6.0.min.js');
   

2. Which editor to integrate: There are many markdown editors available, but we need something that can work with Web Components (in-browser). I also wanted to avoid any kind of bundling of the scripts with the plugin. This is because the Outerbase plugins can have only so much size, and markdown/WYSIWYG editors are bulky. So we need one which is available from a CDN. I looked at and tried QuillJs (with QuillJs Markdown module), SimpleMDE and ToastUI Editor, and the latter one seemed relatively maintained and worked on the first try so that is the one we integrate.

Using the functions listed just above, if you try to load the scripts and CSS in an Outerbase plugin you'll get the gotcha moment of this plugin; you're not allowed to load third-party CSS because of the CSP (content security policy). What do we do now? Without the stylesheets, it is pointless to integrate the editor. Maybe we bundle the CSS with the plugin (bundling the script is still a NO, due to its size)?

Bun as a package manager & bundler

Now we go back to reading about bun. After reading the docs it became clear that Bun doesn't support bundling the CSS at present. It simply copies the CSS files to the outdir and renames their references. There are two ways out of this:

1. Create a bun plugin to handle the CSS file bundling.

   1. Read the CSS files from node_modules

   2. Minify using some third-party CSS minifier (no native CSS loader) and,

   3. Inject as text into the final bundle

2. Simply get the minified CSS files from the CDN, and inject them as text into the plugin code (no bundling needed as the plugin code is very small)

So we pick the easy way out here and pick the second option. Run the bun init command to quickly create a package.json file (so that we can use bun APIs). Create a new file build.js for handling the tooling. This is the function which does what we need. Replacements is just an object where the keys are the identifiers we want to replace with the actual CSS styles.

const bundle = async (replacements) => {
  if (replacements) {
    const indexFile = Bun.file("index.js");
    let indexFileText = await indexFile.text();

    for (const key in replacements) {
      // Fetch the CSS file from the CDN using the URL
      const res = await fetch(replacements[key]);
      const fText = await res.text();

      indexFileText = indexFileText.replace(key, fText);
    }

    createOutput("out", indexFileText);
  } else {
    fs.cpSync("index.js", "out/index.js");
  }
};

const createOutput = (dir, fileText) => {
  const filePath = `${dir}/index.js`;
  const directoryPath = path.dirname(filePath);
  if (!fs.existsSync(directoryPath)) {
    fs.mkdirSync(directoryPath, { recursive: true });
  }

  fs.writeFileSync(filePath, fileText);
};

Since we're in the CLI realm now, added a little complexity to get the file names as CLI input (using commander). You can check the Github repo for the code.

Coding the plugin

First of all, we load the script (remember the CSS is already injected into the plugin, in the style tag of the cell editor's shadow dom, to be precise).

loadToastUiEditor() {
  const scriptSrc =
    "https://uicdn.toast.com/editor/latest/toastui-editor-all.min.js";
  // Optimization to not load the script again 
  // and again, as the editor is recreated every time
  if (document.scripts) {
    for (const script of document.scripts) {
      if (script.src === scriptSrc) {
        console.log("script already loaded, bail out");
        return;
      }
    }
  }

  const el = document.createElement("script");
  el.src = scriptSrc;

  el.onload = () => {
    this.render();
  };

  el.onerror = (event) => {
    console.log("failed to load the script", event);
  };

  // We're adding the script to the document and not 
  // the shadow dom, because the shadom dom is recreated
  // whenver the editor is opened
  document.head.appendChild(el);
}

As soon as the script is loaded we are ready to show our markdown editor

render() {
  try {
    const Editor = toastui.Editor;
    this.editor = new Editor({
      el: this.shadow.querySelector("#editor"),
      height: "420px",
      initialEditType: "markdown",
      initialValue: this.getAttribute("cellvalue"),
      previewStyle: "vertical",
      usageStatistics: false,
      theme: this.config.theme,
      events: { keydown: this.handleKeyDown },
    });

    this.setEditorPosition();
  } catch (error) {
    console.log("render error", error);
  }
}

Many things are going on here most of which are self-explanatory, I'll briefly touch upon the important points

1. We open the editor with whatever content the cell was holding

    initialValue: this.getAttribute("cellvalue")
   

2. Because of the way cell plugins have been designed, they pop up near the cell to which they're attached. For our plugin, we need a centred dialog. We achieve that through plain old DOM manipulation

    setEditorPosition() {
      const agPopUpChild = document.querySelector(".ag-popup-child");
      const container = this.shadow.getElementById("container");
   
      setTimeout(() => {
        agPopUpChild.style.left = `${
          (window.innerWidth - container.offsetWidth) / 2
        }px`;
        // agPopUpChild.style.top = `${
        //   (window.innerHeight - container.offsetHeight - 100) / 2
        // }px`; // -100 offset for outerbase top bars
   
        agPopUpChild.style.top = "0px"; // Just hardcode at 0px otherwise top border not visible
      }, 10);
    }
   

3. The theme is set to the editor using theme: this.config.theme doesn't work without some manipulation. At the moment we do not receive the theme metadata in cell plugins. The below code finds the theme from the DOM styles

    const agPopUp = document.querySelector(".ag-popup");
    const colorScheme = window.getComputedStyle(agPopUp)["color-scheme"];
    this.config.theme = colorScheme === "normal" ? "light" : "dark";
   

4. If we press enter in the markdown editor (to add a new line), the cell plugin editor closes itself (maybe there is a keydown event listener somewhere listening for Enter key events). We skirt through it by stopping such event propagation

    events: { keydown: this.handleKeyDown }, //Listen for keydown events from the md editor
   
    handleKeyDown(_, event) {
      if (event.key === "Enter") {
        event.stopPropagation();
      }
    }
   

Now we're ready to enjoy our writing with the md-editor. Once we're done, we can click the save button to close the editor and update the cell's content.

const saveBtn = this.shadow.getElementById("save-btn");
saveBtn.addEventListener("click", () => {
  const finalContent = this.editor.getMarkdown();
  triggerEvent_$PLUGIN_ID(this, {
    action: OuterbaseColumnEvent_$PLUGIN_ID.onStopEdit,
    value: finalContent,
  });
  triggerEvent_$PLUGIN_ID(this, {
    action: OuterbaseColumnEvent_$PLUGIN_ID.updateCell,
    value: finalContent,
  });
});

ChatGPT: The secret sauce of the plugin

This plugin comes integrated with ChatGPT to help with your writing with some preconfigured actions. Using it you can change the tone of your writing, get suggestions for headlines, generate a summary for your content, and more.

The below method prepares the UI for showing the tone selections.

prepareTonesSelections() {
  const toneSelect = this.shadow.getElementById("tone-select");
  this.tones.forEach((value) => {
    this.addSelectOption(toneSelect, value);
  });

  const toneBtn = this.shadow.getElementById("tone-btn");
  toneBtn.addEventListener("click", () => {
    const selectedIndex = toneSelect.selectedIndex;
    if (selectedIndex) {
      const selectedTone = toneSelect.options[selectedIndex].value;
      const prompt = `Make the following text better and rewrite it in a ${selectedTone.toLowerCase()} tone`;
      this.handleSelectionAction(prompt);
    }
  });
}

The action is handled in the below method where we go ahead only if some text is selected in the editor. We also show a "Thinking..." text as a loader and finally replace that with the result received from the API call.

async handleSelectionAction(prompt) {
  const [start, end] = this.editor.getSelection();
  const selectedText = this.editor.getSelectedText(start, end);
  if (selectedText) {
    this.editor.insertText(`${selectedText}\n\nThinking...`);
    const currLinePos = end[0] + 2;
    this.editor.setSelection(
      [currLinePos, 1],
      [currLinePos, "Thinking...".length + 1]
    );

    const generatedText = await this.talkToChatGPT(prompt, selectedText);
    if (generatedText) {
      this.editor.insertText(generatedText);
    }
  }
}

Below is the method which makes the API call to ChatGPT

async talkToChatGPT(instruction, text) {
  const res = await fetch("https://api.openai.com/v1/completions", {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      Authorization: `Bearer ${this.config.apiKey}`,
    },
    body: JSON.stringify({
      model: "gpt-3.5-turbo-instruct",
      prompt: `${instruction}: ${text}`,
      max_tokens: 2048,
      temperature: 0.3,
      n: 1,
    }),
  });

  const data = await res.json();
  return data.choices[0].text.trim();
}

Similarly, we handle the other commands using other appropriate prompts. You can check the GitHub repo for the complete source code of the plugin.

Current Limitations

1. The cell plugin editors are closed automatically as soon as users click somewhere outside. It would be great to have an option to close the editor when users explicitly click on a button

2. The changed data does not persist even though correct events are sent to the parent DOM. One workaround is to use the Outerbase commands to make a call to the database, but due to lack of time I haven't explored that

3. The toolbar items that open a pop-up/dialog in the markdown editor (Heading / Link / Image etc.) do not work. The dialogs get closed as soon as you click on them. This is an open issue with the ToastUI editor where this functionality doesn't work in Shadow Dom. There is a workaround to replace these buttons with a similar button that doesn't open a dialog. Again due to lack of time, this has not been explored.

Resources

The complete source code of the plugins and the templates can be found here

The demo video showing the plugins in action

Conclusion

When you explore a new thing many roadblocks will come. Some roadblocks will have a workaround, and some will be dead ends. Now it is up to you whether you quit on the first roadblock, or push ahead and try to find a way. This Outerbase hackathon was one such adventure for me, and I thoroughly enjoyed it.

Hope you liked reading the article. Do let me know your thoughts in the comments section.

Remember to keep adding the bits, soon you'll have more bytes than you'll ever need :-)

Until text time! Adios.

Tl;dr this article covers a complete rewrite of an existing app using a different technology stack. The article also covers the creation of a 1Password CLI Shell Plugin.

Introduction

To do the rewrite I wanted to use Nuxt3. The reasoning was simple, I've used Nuxt2 in the past, but haven't had the chance to look at Nuxt3 (and Vue3 for that matter). For the database and to process the important database events I decided to use Appwrite. Appwrite has inbuilt authentication but I wanted to try out Passage by 1Password, so this app uses that.

Appwrite provides a CLI, and 1Password also has a CLI but both are not integrated. So it seemed logical to first integrate the two and create the Appwrite shell plugin for the 1Password CLI.

Part 1: Creating a 1Password shell plugin

If you go over to the 1Password docs page which mentions how you can contribute and create your own shell plugin, you'll learn that the whole thing is implemented using Go.

I haven't tried going anywhere with Go😉. But the documentation looked straightforward forward so what is the harm in trying?

Setup the environment

I'm not going to rewrite what is already covered in detail in the docs link shared above. Just follow the link and install the needed dependencies. I had installed GNU Make using homebrew which installs it as gmake. So we'll need to replace make commands with gmake (unless you add a gnubin directory to the PATH as mentioned in the link).

Choosing a Provisioner

The first decision point comes when you need to choose a provisioner. As the docs page says, "Provisioners are in essence hooks that get executed before the executable is run by 1Password CLI, and after the executable exits in case any cleanup is needed.". So how a third-party CLI authenticates you needs to be configured here. This is not a one size fits all scenario. You need to go and read the said third-party CLI documentation to figure it out. But I had used the appwrite CLI, it needs Email & Password so this reading documentation advice is not for me.

1Password CLI supports the following provisioners

const (
    APIClientCredentials = sdk.CredentialName("API Client Credentials")
    APIKey               = sdk.CredentialName("API Key")
    APIToken             = sdk.CredentialName("API Token")
    AccessKey            = sdk.CredentialName("Access Key")
    AccessToken          = sdk.CredentialName("Access Token")
    AppPassword          = sdk.CredentialName("App Password")
    AppToken             = sdk.CredentialName("App Token")
    AuthToken            = sdk.CredentialName("Auth Token")
    CLIToken             = sdk.CredentialName("CLI Token")
    Credential           = sdk.CredentialName("Credential")
    Credentials          = sdk.CredentialName("Credentials")
    DatabaseCredentials  = sdk.CredentialName("Database Credentials")
    LoginDetails         = sdk.CredentialName("Login Details")
    PersonalAPIToken     = sdk.CredentialName("Personal API Token")
    PersonalAccessToken  = sdk.CredentialName("Personal Access Token")
    RegistryCredentials  = sdk.CredentialName("Registry Credentials")
    SecretKey            = sdk.CredentialName("Secret Key")
    UserLogin            = sdk.CredentialName("User Login")
)

Appwrite CLI uses email/password for authentication so, LoginDetails and UserLogin look like the two promising choices. So I picked one of them. The next step asks for an example credential which was confusing as it didn't say whether this example is for the login or the password. Anyway, after putting some random string there we get the template code generated. And we've got a new problem

Both of the above choices generate code with errors. These field names are not defined. And also only one fieldname is created.

Fields: []schema.CredentialField{
  {
    Name:                fieldname.Login,
    MarkdownDescription: "Login used to authenticate to Appwrite.",
    Secret:              true,
    Composition: &schema.ValueComposition{
      Length: 21,
      Charset: schema.Charset{
        Lowercase: true,
        Digits:    true,
      },
    },
  },
},

After defining the missing fieldsname, and adding a new field (password) in the array, tried validating, building and initing the plugin (using op plugin init appwrite). It asked for the login and password and created an entry in the 1Password app. The moment of truth is here

Run appwrite login. Wait for the email and password to be supplied by the 1Password App, and it never happens.

This is the moment I realized something is not right. Tried looking for a config file in the system and found one at ~/.appwrite/prefs.json. It doesn't store a password just a cookie. Essentially what the CLI does is, take the login credentials from you interactively, and then make an auth call and store the auth token in that file for future use. We could store the cookie in 1Password but then who takes care of refreshing the token?

Before giving up, this is where I got in touch with the 1Password team. Learnt that interactive login is not supported, and the following verbatim response "If your file supports provisioning the credentials in a file, you can use the SDK’s file provisioner when building the plugin.".

Interesting! File provisioner is mentioned in the docs but mostly you're going to miss reading it. Well, time to eat my words and go to appwrite CLI docs, and read it, of course. Found that there is a CI mode that works using the API Key. The only limitation is, it works with a single project at a time. And also the supported CLI commands depends on the permissions granted to the API Key.

appwrite client \
    --endpoint https://cloud.appwrite.io/v1 \
    --projectId [YOUR_PROJECT_ID] \
    --key YOUR_API_KEY

On running this command from the home dir of my system, it added the endpoint and the API key to the prefs.json mentioned earlier, but it also created a new appwrite.json file mentioning the project ID in the home dir (from where the command was executed). So the prefs.json doesn't contain the project id. If you've used appwrite then you'll know that the project folder also contains an appwrite.json file mentioning the project id. This hints that we only need to store the endpoint and the API key in 1Password, and then run the appwrite commands from the project directory itself (1Password supports tying up credentials to a particular dir).

After all this detective work, we are finally ready to implement the plugin.

Implementing the plugin

After deleting and running the gmake new-plugin command again, this time I selected API Key as the credential type. Modified the generate api_key.go file as shown below

Required Fields, Default Provisioner & Importer

Fields: []schema.CredentialField{
    {
        Name:                fieldname.APIKey,
        MarkdownDescription: "API Key used to authenticate to Appwrite.",
        Secret:              true,
        Composition: &schema.ValueComposition{
            Length: 256,
            Charset: schema.Charset{
                Lowercase: true,
                Digits:    true,
            },
        },
    },
    {
        Name:                fieldname.Endpoint,
        MarkdownDescription: "Appwrite server endpoint.",
        Secret:              false,
        Optional:            false,
    },
},
DefaultProvisioner: provision.TempFile(appwriteConfig, provision.AtFixedPath(ConfigPath())),
Importer: importer.TryAll(
    TryAppwriteConfigFile(),
)}

Note that we're creating the temp file at a fixed path provision.AtFixedPath(ConfigPath())). Where ConfigPath is as shown below

func ConfigPath() string {
    configDir, err := os.UserHomeDir()
    if err != nil {
        return "~/.appwrite/prefs.json"
    }

    return configDir + "/.appwrite/prefs.json"
}

appwriteConfig function

This function takes the credentials from 1Password and converts that into a JSON object so that a temp config file can be created for the appwrite CLI

func appwriteConfig(in sdk.ProvisionInput) ([]byte, error) {
    // Create config object from the incoming fields
    config := Config{
        APIKey:   in.ItemFields[fieldname.APIKey],
        Endpoint: in.ItemFields[fieldname.Endpoint],
    }

    // Covert the content to a JSON string
    contents, err := json.MarshalIndent(&config, "", "  ")
    if err != nil {
        return nil, err
    }

    // Convert the string to bytes, to be written to the temp file
    return []byte(contents), nil
}

// The config struct (notice the differnt json key names)
type Config struct {
    APIKey   string `json:"key"`
    Endpoint string `json:"endpoint"`
}

TryAppwriteConfigFile function

This function reads an existing prefs.json file at the config path and shows a prompt to the user that these credentials can be imported into 1Password when they run any of the appwrite commands requiring authentication.

func TryAppwriteConfigFile() sdk.Importer {
    return importer.TryFile(ConfigPath(), func(ctx context.Context, contents importer.FileContents, in sdk.ImportInput, out *sdk.ImportAttempt) {
        var config Config
        if err := contents.ToJSON(&config); err != nil {
            out.AddError(err)
            return
        }

        fmt.Println("Printing the imported file")
        fmt.Println(config)

        if config.APIKey == "" {
            return
        }

        if config.Endpoint == "" {
            return
        }

        out.AddCandidate(sdk.ImportCandidate{
            Fields: map[sdk.FieldName]string{
                fieldname.APIKey:   config.APIKey,
                fieldname.Endpoint: config.Endpoint,
            },
        })
    })
}

That's it. We're done. The only thing remaining is writing some tests, and configuring which appwrite commands don't require auth. The latter can be done in appwrite.go file as shown below

NeedsAuth: needsauth.IfAll(
    needsauth.NotForHelpOrVersion(),
    needsauth.NotWithoutArgs(),
    needsauth.NotWhenContainsArgs("client"),
    needsauth.NotWhenContainsArgs("login"),
    needsauth.NotWhenContainsArgs("logout"),
    needsauth.NotForExactArgs("deploy"),
    needsauth.NotForExactArgs("projects"),
    needsauth.NotForExactArgs("storage"),
    needsauth.NotForExactArgs("teams"),
    needsauth.NotForExactArgs("users"),
    needsauth.NotForExactArgs("account"),
    needsauth.NotForExactArgs("avatars"),
    needsauth.NotForExactArgs("functions"),
    needsauth.NotForExactArgs("databases"),
    needsauth.NotForExactArgs("health"),
    needsauth.NotForExactArgs("locale"),
),

The plugin tests can be modified in api_key_test.go file.

// Test whether our file provision is working
func TestAPIKeyProvisioner(t *testing.T) {
    plugintest.TestProvisioner(t, APIKey().DefaultProvisioner, map[string]plugintest.ProvisionCase{
        "temp file": {
            ItemFields: map[sdk.FieldName]string{
                fieldname.APIKey:   "zsaugacpwq6k54nnbdbmh1cys98u2a32qqkacma2ioxn1e2j6eyrk9urom0vzcvm6qbbm8s6l4xbm86n37foauiqba9tlcvohuoz87j7nwvpob5wr71k58i105fn39a10vj7ob84opwf1vrfat3m8konch7xxy2z2dh1ykohdbef7xgmvtn82lebe4mzmfzoylqy4jslrok11zbjtmd6xs84ukd7b1k9ofyuanvinmlhkgua32p5x0gqbexample",
                fieldname.Endpoint: "http://localhost/v1",
            },
            ExpectedOutput: sdk.ProvisionOutput{
                Files: map[string]sdk.OutputFile{
                    ConfigPath(): {
                        Contents: []byte(plugintest.LoadFixture(t, "import_prefs.json")),
                    },
                },
            },
        },
    })
}

// Test the importer
func TestAPIKeyImporter(t *testing.T) {
    plugintest.TestImporter(t, APIKey().Importer, map[string]plugintest.ImportCase{
        "Appwrite prefs file": {
            Files: map[string]string{
                ConfigPath(): plugintest.LoadFixture(t, "import_prefs.json"),
            },
            ExpectedCandidates: []sdk.ImportCandidate{
                {
                    Fields: map[sdk.FieldName]string{
                        fieldname.APIKey:   "zsaugacpwq6k54nnbdbmh1cys98u2a32qqkacma2ioxn1e2j6eyrk9urom0vzcvm6qbbm8s6l4xbm86n37foauiqba9tlcvohuoz87j7nwvpob5wr71k58i105fn39a10vj7ob84opwf1vrfat3m8konch7xxy2z2dh1ykohdbef7xgmvtn82lebe4mzmfzoylqy4jslrok11zbjtmd6xs84ukd7b1k9ofyuanvinmlhkgua32p5x0gqbexample",
                        fieldname.Endpoint: "http://localhost/v1",
                    },
                },
            },
        },
    })
}

To run the tests we can run gmake test or make test command.

And this gives us the green light we needed.

Part 2: Creating the app

We start with the basics here. Scaffold the app using the below command

npx nuxi@latest init <project-name>

Since we're using Passage for auth, install @passageidentity/passage-elements. Now here is a catch, to make use of the appwrite client SDK we need to create a session using email/password or through supported OAuth providers. Both of these are ruled out as passage auth doesn't need a password, and is also not a supported OAuth provider. I still installed the appwrite SDK as I used its locale services in my app flow.

yarn add @passageidentity/passage-elements appwrite

We definitely need the server SDKs of both of the above. @passageidentity/passage-node for verifying the authenticity of incoming client requests, and node-appwrite for interacting with the appwrite database.

yarn add node-appwrite @passageidentity/passage-node

For styling and readymade components, I decided to use NuxtLabs UI. Now we're all set to start writing the app.

Passage Auth

Passage provides readymade custom web components which handle the end-to-end auth for you. To use it in a Nuxt (or Vue) app, we need to register these components as custom elements in the Nuxt config file. If we do not do that then we'll get warnings in our browser console. Add the following inside the defineNuxtConfig input object in your nuxt.config.ts file.

vue: {
  compilerOptions: {
    isCustomElement: (tag) => tag.startsWith('passage-'),
  },
},

Now I tried to use the <passage-auth> component directly in a page template after adding the necessary import

import '@passageidentity/passage-elements/passage-auth';

But it doesn't work, you'll get the below error

Cannot use import statement outside a module

This happens because Nuxt is running in SSR mode and web components are not available server side. You can get more information on this passage documentation for NextJs here. I tried following the same approach outlined in the link, load the component client side in the onMounted hook

onMounted(()=>{
  require('@passageidentity/passage-elements/passage-auth');
});

But sadly this also is a no-go. Nuxt3 ships with Vite bundler by default, and using require is not supported in Vite. No issues, simply replace require with import there you might say, but we get typescript error "An import declaration can only be used at the top level of a namespace or module".

So what is the solution? To resolve this I created separate components which only contain passage elements, and load this new component only on the client side using the <ClientOnly> option provided by Nuxt.

So a SignUp component might look like this

<template>
  <passage-register :app-id="passageAppId" />
</template>

And then you use this component on a sign-up page as shown below

<template>
  <ClientOnly>
    <UContainer>
      <UCard class="max-w-md mx-auto mt-8 text-center">
        <h1 class="text-3xl font-medium">Sign up</h1>

        <sign-up />

        <div
          class="text-sm font-medium text-gray-500 dark:text-gray-300 text-center"
        >
          Already have an account?
          <UButton variant="link" :padded="false" to="/sign-in">
            Sign in
          </UButton>
        </div>
      </UCard>
    </UContainer>
  </ClientOnly>
</template>

After making the above adjustments it works all right.

Creating user & family accounts

Since we want to be able to invite family members (or create accounts for them ourselves) to the app, we need to roll out our own database schema to connect their accounts. But appwrite supports the creation of Teams and linking user accounts to them. To use this feature we will need to create appwrite users and then link those users to a team. How do we correlate the Passage users to the appwrite users?

Passage allows us to listen to new account creations or a fresh login, by attaching a callback (onSuccess) to passage elements. But there is a catch, we can't simply bind the callback to the pasasge element using the "@" syntax of vue. I needed to use the "." syntax to make it work. You can read more about this here.

This is my final SignUp component which uses the passage callback to make an API call to a Nuxt serverless API

<script setup lang="ts">
import '@passageidentity/passage-elements/passage-register';
import { authResult } from '@passageidentity/passage-elements';
const { getUser } = usePassageUser();

const {
  public: { passageAppId },
} = useRuntimeConfig();

const onRegistrationDone = async (authResult: authResult) => {
  try {
    const res = await $fetch('/api/users', {
      method: 'post',
    });

    console.log('got response from user create', res);
    await getUser(authResult.auth_token);
    navigateTo('/onboarding');
  } catch (error) {
    console.log('failed to create user in appwrite');
  }
};
</script>

<template>
  <passage-register :app-id="passageAppId" .onSuccess="onRegistrationDone" />
</template>

And this is the /api/users code. We receive the request from the client, verify its authenticity using the passage's node SDK, and then create a new user in appwrite using their node SDK.

import { UserObject } from '@passageidentity/passage-node';
import { protectRoute } from '../usePassage';
import { useAppwrite } from '../useAppwrite';

export default defineEventHandler(async (event) => {
  console.log('incoming post event for api/users/', event);

  await protectRoute(event);

  const user = event.context.auth.user as UserObject;
  console.log(`got some auth user:`, user);

  const { $users } = useAppwrite();

  // create an appwrite user with the same ID returned by Passage
  const res = await $users.create(
    user.id,
    user.email,
    undefined,
    undefined,
    user.user_metadata?.name as string
  );

  console.log('res of user create', res);

  return {
    status: 'ok',
  };
});

And this is the protectRoute function to verify the authenticity of the request

import { H3Event } from 'h3';
import Passage, { Metadata } from '@passageidentity/passage-node';

let _passage: Passage | null = null;

const getPassage = () => {
  if (!_passage) {
    const {
      passageApiKey,
      public: { passageAppId },
    } = useRuntimeConfig();

    const passageConfig = {
      appID: passageAppId,
      apiKey: passageApiKey,
    };

    _passage = new Passage(passageConfig);
  }

  return _passage;
};

export const protectRoute = async (event: H3Event) => {
  const passage = getPassage();

  try {
    const userId = await passage.authenticateRequest(event.node.req);

    if (userId) {
      console.log('request authenticated', userId);

      const user = await passage.user.get(userId);
      event.context.auth = { user };

      return;
    }
  } catch (error) {
    console.log('failed to authenticate request', error);
  }

  throw createError({
    statusCode: 401,
    message: 'Unauthorized',
  });
};

Handle adding family members

During the onboarding of a new user, the app asks them to create a family account and add family members to it. But to invite the family members they should have a passage account, right? That is the approach we've followed above, the appwrite user id is provided by Passage. To handle this, we first create Passage users using the passage-node SDK, get the user ids, create corresponding appwrite users, and then finally link these users to the created team/family.

The frontend function which adds the family members

const addMembers = async () => {
  loading.value = true;
  try {
    const res: any = await $fetch('/api/families', {
      method: 'post',
      body: {
        type: 'ADD_MEMBERS',
        familyId: metadata?.family_id,
        members: members.value,
        onboardStep: metadata?.onboard_step,
      },
    });

    console.log('response of add members', res);
    if (res.user && user.value) {
      user.value.user_metadata = res.user.user_metadata;
    }

    emit('membersAdded');
  } catch (error) {
    console.log('error is adding members', error);
  }

  loading.value = false;
};

The /api/families code

import { UserObject } from '@passageidentity/passage-node';
import { protectRoute, createUser, updateUser } from '../usePassage';
import { useAppwrite } from '../useAppwrite';

const addFamilyMembers = async (
  userId: string,
  urlOrigin: string,
  data: any
) => {
  if (!data.familyId || !data.members || !data.members.length) {
    throw createError({
      statusCode: 400,
      message: 'Missing familyId or members to add',
    });
  }

  const { $users, $teams } = useAppwrite();
  try {
    const userPromises = [];
    for (const member of data.members) {
      // Create Passage Users
      userPromises.push(
        createUser(member.email, {
          name: member.name,
          family_id: data.familyId,
          roles: member.role,
          onboard_step: 'done',
        })
      );
    }

    const users = await Promise.all(userPromises);
    console.log('created new users in passage: ', users);

    const appwriteUserPromises = [];
    for (const user of users) {
      appwriteUserPromises.push(
        $users.create(
          user.id,
          user.email,
          undefined,
          undefined,
          user.user_metadata?.name as string
        )
      );
    }

    const appwriteUsers = await Promise.all(appwriteUserPromises);

    console.log('created new users in appwrite: ', appwriteUsers);

    const memberPromises = [];
    for (const user of users) {
      memberPromises.push(
        $teams.createMembership(
          data.familyId,
          [user.user_metadata?.roles as string],
          `${urlOrigin}/join-team`,
          user.email,
          user.id,
          user.phone,
          user.user_metadata?.name as string
        )
      );
    }

    const memberships = await Promise.all(memberPromises);

    console.log('created memberships in appwrite', memberships);

    let updatedUser;
    if (data.onboardStep === 'family') {
      // updat passage user metadata
      updatedUser = await updateUser(userId, {
        onboard_step: 'jar',
      });
    }

    return {
      user: updatedUser,
    };
  } catch (error) {
    console.log('failed to add members', error);
    throw createError({
      statusCode: 500,
      message: 'Failed to add family members',
    });
  }
};

export default defineEventHandler(async (event) => {
  console.log('incoming post event for api/families/');

  await protectRoute(event);

  const body = await readBody(event);
  console.log('body', body);

  if (!body.type || !['CREATE', 'ADD_MEMBERS'].includes(body.type)) {
    throw createError({
      statusCode: 400,
      message: 'Missing or unsupported event type',
    });
  }

  const user = event.context.auth.user as UserObject;
  console.log(`got some auth user: ${user}`);

  const origin = getHeader(event, 'origin');

  let data;
  if (body.type === 'CREATE') {
    data = await createFamily(user.id, origin || '', body);
  } else {
    data = await addFamilyMembers(user.id, origin || '', body);
  }

  return {
    status: 'ok',
    ...data,
  };
});

These are the passage functions to create and update a user.

import Passage, { Metadata } from '@passageidentity/passage-node';

export const createUser = async (email: string, metadata: Metadata) => {
  const passage = getPassage();

  let userData = await passage.user.create({ email, user_metadata: metadata });
  console.log(userData);

  return userData;
};

export const updateUser = async (userId: string, data: Metadata) => {
  const passage = getPassage();

  let userData = await passage.user.update(userId, { user_metadata: data });
  console.log(userData);

  return userData;
};

As you can see, we're storing important user metadata inside the passage user object itself. Barring the name, none of the other fields are visible to the user. We use the familyId field from this metadata to add members to the same family as the calling user.

{
  name: member.name,
  family_id: data.familyId,
  roles: member.role,
  onboard_step: 'done',
}

Appwrite functions

The app also uses some appwrite functions to handle important database events. The shell plugin that we created in the first part gets verified for creating/deploying these functions. The database events that the app handles currently include

1. Jar created event: Whenever a new jar is created and it has auto credit enabled, then we update the jar object and set its nextMoneyAt field. This is the field that is queried by the cron job for auto-crediting the configured amount to the jar.

2. Transaction event: On every transaction (create or update), we update the corresponding jar balance. If the transaction was done by a child, it will be pending and the jar won't be updated. Once the said transaction is approved (transaction update event) by a family member (role: member), then only the jar gets updated.

3. Cron job: This job runs every day at midnight and auto credits the configured auto credit amount to the eligible jars.

App Screenshots

The rest of the app follows the same principle. All app data is fetched from Nuxt serverless APIs where every API call is authenticated by the passage-node SDK. Here are some of the screenshots from the app.

For styling the passage elements, I had to set the following CSS variables

passage-register,
passage-login {
  --passage-container-background-color: transparent;
  --passage-body-text-color: #ffffff;
  --passage-primary-color: #fbbf24;
  --passage-onprimary-color: #0f172a;
  --passage-hover-color: #f59e0b;
  --passage-button-font-weight: 500;
  --passage-container-max-width: 100%;
  --passage-container-padding: 30px 0 10px;
  --passage-button-width: 100%;
  --passage-control-border-color: #6b6b6b;
  --passage-otp-input-background-color: #434343;
}

Sign up screen

Sign in screen

Sign in code

App dashboard

Family screen

Transactions screen

User profile screen

This screen uses the <passage-profile> component provided by the passage's client SDK. We can change our profile update operation from here.

But it seems there is a bug in profile updation which the Passage team needs to look at. This app uses 3 other hidden metadata fields, and when you try to update the name it throws an error as shown below

The error message is clear enough, but the app doesn't have any control on this component. This can be fixed by the Passage team only.

Create jar

Add family members

Make transaction

Part 3: Automating the env file

As this app utilizes 1Password in a big way, I decided to make use of the 1Password CLI to handle the app env file as well. Below is a simple script that reads an env file and creates a 1Password item in the specified vault and replaces field values with the respective 1Password secrets.

#!/bin/sh

# Parse command-line arguments
while [ $# -gt 0 ]; do
  key="$1"

  case $key in
    --vault)
      vault="$2"
      shift # past argument
      shift # past value
      ;;
    --item-name)
      item_name="$2"
      shift # past argument
      shift # past value
      ;;
    --env-file)
      env_file="$2"
      shift # past argument
      shift # past value
      ;;
    *)
      # unknown option
      shift
      ;;
  esac
done

# Check if the --vault, --item-name, and --env-file parameters were provided
if [ -z "$vault" ] || [ -z "$item_name" ] || [ -z "$env_file" ]; then
  echo "Please provide --vault, --item-name, and --env-file parameters."
  exit 1
fi

# Check if the specified .env file exists
if [ ! -f "$env_file" ]; then
  echo "The specified .env file does not exist."
  exit 1
fi

# Check if the 1Password CLI is installed
if ! command -v op > /dev/null 2>&1; then
  echo "1Password CLI is not installed. Please install it before running this script."
  exit 1
fi

# Check if the item already exists in the vault
if op item get "$item_name" --vault "$vault" > /dev/null 2>&1; then
  echo "Item '$item_name' already exists in the '$vault' vault. Skipping creation."
  exit 0
fi

# Variable to store the fields string
fields=()
skipped_fields=()

# Read the .env file line by line
while IFS= read -r line || [ -n "$line" ]; do
  # Skip empty lines and comments
  if [ -z "$line" ] || [ "${line#"#"}" != "$line" ]; then
    continue
  fi

  # Split the line into key and value
  key="${line%%=*}"
  value="${line#*=}"

  # Check if the value is already a reference to a 1Password secret
  if [ "${value#op://}" != "$value" ]; then
    echo "Skipping creation of '$key' field as it is already a reference to a 1Password secret."
    skipped_fields+=($line)
    continue
  fi

  # Add the key-value pair to the fields array
  fields+=("$key=$value")
done < "$env_file"

# Create the item in 1Password using the op create command
op item create \
  --category Server \
  --title "$item_name" \
  --vault "$vault" \
  "${fields[@]}" \
  --tags "$item_name,env"

# Update the .env file with the secret references
for ((i = 0; i < ${#fields[@]}; i++)); do
  # Split the field into key and value
  field="${fields[i]}"
  IFS="=" read -r key value <<< "$field"

  fields[i]="$key=op://$vault/$item_name/$key"
done

final_fields=("${fields[@]}" "${skipped_fields[@]}")

# Overwrite the .env file with the updated key-value pairs
printf "%s\n" "${final_fields[@]}" > "$env_file"

echo "Item '$item_name' created successfully in the '$vault' vault."
echo "Updated the '$env_file' file with the secret references."

We need to pass the vault name, the item entry name and the env file path while executing the script. Now this env file can be uploaded to the repo along with the rest of the code.

./save-env.sh --vault AppVaults --item-name FamPro --env-file ./client/.env

APPWRITE_ENDPOINT=op://AppVaults/FamPro/APPWRITE_ENDPOINT
APPWRITE_PROJECT_ID=op://AppVaults/FamPro/APPWRITE_PROJECT_ID
APPWRITE_DATABASE_ID=op://AppVaults/FamPro/APPWRITE_DATABASE_ID
APPWRITE_JAR_COLLECTION_ID=op://AppVaults/FamPro/APPWRITE_JAR_COLLECTION_ID
APPWRITE_TRANSACTION_COLLECTION_ID=op://AppVaults/FamPro/APPWRITE_TRANSACTION_COLLECTION_ID
APPWRITE_API_KEY=op://AppVaults/FamPro/APPWRITE_API_KEY
PASSAGE_APP_ID=op://AppVaults/FamPro/PASSAGE_APP_ID
PASSAGE_API_KEY=op://AppVaults/FamPro/PASSAGE_API_KEY

To load the secrets back into the env file we can use the below command (with the correct file paths)

op inject -i .env_template -o .env

Supported features

The app supports the following features at the moment

1. Creating a family account.

2. Adding members to the family with a member or child role. Only a user with a member role can add other members.

3. Creating multiple jars per user (member or child). Again, only users with a member role can create jars.

4. Making ad-hoc transactions against a jar. A user with a member role can make transactions in any of the jars in the family. A child can only view and do transactions in their jars.

5. Transactions done by a user with a child role remain pending. These transactions need to be approved by a user with a member role before the jar balance can be updated.

Further enhancements

1. Improve the dashboard experience

2. Allow deletion of member accounts

3. Allow modification and deletion of transactions

App source code and link

The full source code of the app along with the 1Password CLI script can be found here

The code for the appwrite shell plugin can be checked in this PR.

You can play around with the app here: https://fam-pro.vercel.app/

Conclusion

Overall it was a great experience building with 1Password, Appwrite and Nuxt3. I did face some challenges in the process outlined above, but we could overcome those. 1Password offers a seamless auth experience and it would be interesting to see what features they add on top of the existing functionality. I also thank the Hashnode team for organizing this hackathon.

I hope you enjoyed reading the article. It would mean a lot to me if you can show appreciation by sharing your feedback. If you found any mistake then please let me know in the comments.

-- Keep adding the bits, soon you'll have more bytes than you may need. :-)

Introduction

I've always wondered how a VS Code extension works and what it takes to implement one. Recently this urge got the best of me and I decided to finally create a VS Code extension. But what to build? I like using Hashnode's AI editor for rephrasing texts and other writing assistance, so I thought why not implement a similar functionality for VS Code? Since writing assistance is most useful for a markdown file (which can also work as a blog post), I created a basic writing assistant for markdown (and text) files.

Here is a simple GIF showcasing how the extension works.

Ready to dive into how I implemented it? Let's get started.

Getting Started

Before we can start building the extension, we need to gather and prepare the necessary tools. In this case, the needed tools are node, git, yeoman and generator-code. For a newcomer like myself, this basic tutorial is perfect. I recommend going through it to learn the fundamentals.

Run yo code command and pick New Extension (js/ts) from the choices as shown below

I haven't selected the webpack option yet. We can always do so later on. This is my current directory structure.

Implementing the extension

We're mostly interested in the package.json and extension.ts files while building the extension. The important fields in the package.json file are

1. activationEvents: When should our extension be activated

2. main: The entry point of the extension code (the entry is in the out folder which gets generated when you debug or run the extension)

3. contributes: What does this extension contributes to the VS Code commands and settings

For my extension I wanted an experience similar to the Hashnode AI Editor, so adding commands to the VS Code command palette was not what I was after. What helped me here was the sample extensions directory on GitHub. Their code-actions sample was exactly what I had in mind (and it targets only the markdown files).

Target Markdown and Text files

To target specific types of files you need to change the activationEvents. Since we want to work only with Markdown and Text files at the moment, this is what my package.json says

// ...
"activationEvents": [
  "onLanguage:markdown",
  "onLanguage:plaintext"
],
"main": "./out/extension.js",
"contributes": {},
// ...

This extension will only get activated for the languages mentioned above.

Also, since we don't want any command palette commands, we can remove everything under the contributes key.

Showing the actions in the light bulb menu

There is one function called activate inside the extension.ts file which is of interest. This is where you need to write your code. Notice that I've removed all the unnecessary boilerplate code

// This method is called when your extension is activated
// Your extension is activated the very first time the 
// command is executed
export function activate(context: vscode.ExtensionContext) {}

There is another complementary function called deactivate which can be used if you need to do any kind of resource cleaning before the extension is deactivated.

If you run/debug the extension now, and create a new markdown file in the new VS Code window which pops up, you won't notice any difference as there is no palette command, and also there is no code in the activate function. Let's change that and add the following in the extension.ts file

class MyCodeActionProvider implements vscode.CodeActionProvider {

  provideCodeActions(
    document: vscode.TextDocument,
    range: vscode.Range | vscode.Selection,
    context: vscode.CodeActionContext,
    token: vscode.CancellationToken
  ): vscode.ProviderResult<(vscode.CodeAction | vscode.Command)[]> {
    console.log('inside the provideCodeActions method');
    throw new Error('Method not implemented.');
  }
}

export function activate(context: vscode.ExtensionContext) {
  const actionProvider = vscode.languages.registerCodeActionsProvider(
    ['markdown', 'plaintext'],
    new MyCodeActionProvider(),
    {
      providedCodeActionKinds: [
        vscode.CodeActionKind.RefactorRewrite,
        vscode.CodeActionKind.QuickFix,
      ],
    }
  );

  context.subscriptions.push(actionProvider);
}

What we're doing here is informing VS Code about the kind of code actions we're providing which include RefactorRewrite & QuickFix. The actual "commands/code actions" need to be provided by the provideCodeActions method of the MyCodeActionProvider class. If you debug the extension now you should see the below console log in your original project window's debug console.

inside the provideCodeActions method

We're making progress. Replace the provideCodeActions method's code with the following

provideCodeActions(
  document: vscode.TextDocument,
  range: vscode.Range | vscode.Selection,
  context: vscode.CodeActionContext,
  token: vscode.CancellationToken
): vscode.ProviderResult<(vscode.CodeAction | vscode.Command)[]> {
  // If there is nothing selected, we won't provide any action
  if (range.isEmpty) {
    return;
  }

  // supported actions and their kinds
  const actions = [
    {
      id: 'rephrase',
      title: 'Rephrase selected text',
      kind: vscode.CodeActionKind.QuickFix,
    },
    {
      id: 'headlines',
      title: 'Suggest headlines',
      kind: vscode.CodeActionKind.QuickFix,
    },
    {
      id: 'professional',
      title: 'Rewrite in professional tone',
      kind: vscode.CodeActionKind.RefactorRewrite,
    },
    {
      id: 'casual',
      title: 'Rewrite in casual tone',
      kind: vscode.CodeActionKind.RefactorRewrite,
    },
  ];

  const cActions = [];
  // prepare the code actions for the above actions
  for (const action of actions) {
    const cAction = new vscode.CodeAction(action.title, action.kind);
    cAction.command = {
      command: `my-shiny-extension.${action.id}`,
      title: action.title,
      arguments: [action.id],
    };

    cActions.push(cAction);
  }

  return cActions;
}

Debug/run the extension now and you should see the above actions in the bulb tooltip when you select some text in a markdown/text file.

Of course, nothing will happen if you click on any of these actions. This is because we've only created the actions, but haven't written any code to handle them.

Handling the Code Actions

To handle the actions we need to register these commands with the VS Code extension context. This can be done inside the activate function. Let's do a little bit of refactoring.

Move the actions out of the provideCodeActions method and make it a class property of MyCodeActionProvider

public static readonly actions = [
  {
    id: 'rephrase',
    title: 'Rephrase selected text',
    kind: vscode.CodeActionKind.QuickFix,
  },
  {
    id: 'headlines',
    title: 'Suggest headlines',
    kind: vscode.CodeActionKind.QuickFix,
  },
  {
    id: 'professional',
    title: 'Rewrite in professional tone',
    kind: vscode.CodeActionKind.RefactorRewrite,
  },
  {
    id: 'casual',
    title: 'Rewrite in casual tone',
    kind: vscode.CodeActionKind.RefactorRewrite,
  },
];

Change its reference inside the provideCodeActions method to

for (const action of MyCodeActionProvider.actions) {
  // ...
}

Add a new method handleAction which will handle the actions when a user clicks on them. The actionId argument will be passed by the caller. Remember we had passed arguments: [action.id] while returning the code actions from the providecodeActions method?

handleAction(actionId: string) {
  console.log(`handleAction for ${actionId}`);
}

Now change the activate function as shown below

export function activate(context: vscode.ExtensionContext) {
  const myActionProvider = new MyCodeActionProvider();
  const actionProvider = vscode.languages.registerCodeActionsProvider(
    ['markdown', 'plaintext'],
    myActionProvider,
    {
      providedCodeActionKinds: [
        vscode.CodeActionKind.RefactorRewrite,
        vscode.CodeActionKind.QuickFix,
      ],
    }
  );

  context.subscriptions.push(actionProvider);
  for (const action of MyCodeActionProvider.actions) {
    context.subscriptions.push(
      // use the same id which we used in the command field 
      // of the code actions
      vscode.commands.registerCommand(
        `my-shiny-extension.${action.id}`,
        (args) => myActionProvider.handleAction(args)
      )
    );
  }
}

For all the actions which we support, we're registering a corresponding command with the VS Code extension context. The command id that we use here must match the command id which we returned from the provideCodeActions method.

If we run/debug the extension now and click any of our actions from the light bulb menu we should see the corresponding console log in the debug console.

Integrating with the OpenAI APIs

Now the only thing remaining is: using the OpenAI APIs to make changes to any written text. Let's get it over with.

Add the OpenAI library to the codebase

yarn add openai

Import it into the extension.ts file

import { OpenAIApi, Configuration } from 'openai';

And replace the handleAction method's code with the following

async handleAction(actionId: string) {
  const editor = vscode.window.activeTextEditor;
  if (
    !editor ||
    editor.selection.isEmpty ||
    !['rephrase', 'headlines', 'professional', 'casual'].includes(actionId)
  ) {
    // return if no active editor, or no active selection 
    // or if unsupported actionId passed
    return;
  }

  // Create the OpenAI Service
  const openAiSvc = new OpenAIApi(
    new Configuration({
      apiKey: '<your_open_ai_api_key>',
    })
  );

  // Get the currently selected text
  const text = editor.document.getText(editor.selection);
  // current selection range
  let currRange = editor.selection;

  try {
    // Adding a filler/loading text before making the API call
    const fillerText = '\n\nThinking...';
    editor
      .edit((editBuilder) => {
        // insert the filler text after the current selection end
        editBuilder.insert(currRange.end, fillerText);
      })
      .then((success) => {
        if (success) {
          // Select the filler text now
          editor.selection = new vscode.Selection(
            editor.selection.end.line,
            0,
            editor.selection.end.line,
            editor.selection.end.character
          );

          // store this new selection range
          currRange = editor.selection;
        }
      });

    // Create the prompt prefix based on the action id
    let promptPrefix = '';
    switch (actionId) {
      case 'rephrase':
        promptPrefix =
          'Rephrase the following text and make the sentences more clear and readable';
        break;
      case 'headlines':
        promptPrefix = 'Suggest some short headlines for the following text';
        break;
      case 'professional':
        promptPrefix =
          'Make the following text better and rewrite it in a professional tone';
        break;
      case 'casual':
        promptPrefix =
          'Make the following text better and rewrite it in a casual tone';
        break;
    }

    // Make the OpenAI API Call using the desired model and configs
    /* eslint-disable @typescript-eslint/naming-convention */
    const response = await openAiSvc.createCompletion({
      model: 'text-davinci-003',
      prompt: `${promptPrefix}:\n\n${text}\n\n`,
      temperature: 0.3,
      max_tokens: 500,
      frequency_penalty: 0.0,
      presence_penalty: 0.0,
      n: 1,
    });
    /* eslint-enable @typescript-eslint/naming-convention */

    // We'd reuqested for only one result, use that
    let result = response.data.choices[0].text;
    editor
      .edit((editBuilder) => {
        if (result) {
          // replace the filler text with the actual result
          editBuilder.replace(
            new vscode.Range(currRange.start, currRange.end),
            result.trim()
          );
        }
      })
      .then((success) => {
        if (success) {
          // Select the resulting text (the text can be longer
          // and span over multiple lines, so we treat it 
          // appropriately to make a complete selection)
          editor.selection = new vscode.Selection(
            currRange.start.line,
            currRange.start.character,
            currRange.end.line,
            editor.document.lineAt(currRange.end.line).text.length
          );

          return;
        }
      });
  } catch (error) {
    console.error(error);
  }

  // In case of API error, show an error text instead
  editor.edit((editBuilder) => {
    editor.selection = new vscode.Selection(currRange.start, currRange.end);
    editBuilder.replace(editor.selection, 'Failed to process...');
  }):
}

And we're done with the code. If you run/debug the extension you should see appropriate text replacements for your text. Running it on a couple of my sentences gives me the following results

As always, you can play around with the prompts and get better consistent output from the OpenAI API.

Adding Extension Settings

You may have noticed that we've added the OpenAI API key directly in the code. We should move it to VS Code settings under our extension name. To do that we need to change the contributes key in the package.json file. While we're doing that we can also move the maxTokens property instead of hardcoding it to 500 in the code.

// ..
"contributes": {
  "configuration": {
    "title": "My Shiny Extension",
    "properties": {
      "myShinyExtension.openAiApiKey": {
        "type": "string",
        "default": "",
        "description": "Enter you OpenAI API Key here"
      },
      "myShinyExtension.maxTokens": {
        "type": "number",
        "default": 1200,
        "description": "Enter the maximum number of tokens to use for each OpenAI API call"
      }
    }
  }
},
// ..

Now these two entries will appear under "My Shiny Extension" in the VS Code settings. To read the values of these entries we can use the below code (put it just above the OpenAI service creation).

const configs = vscode.workspace.getConfiguration('myShinyExtension');
const openAIApiKey = configs.get<string>('openAiApiKey');
const maxTokens = configs.get<number>('maxTokens');
if (!openAIApiKey) {
  vscode.window.showErrorMessage(
    'Missing OpenAI API Key. Please add your key in VSCode settings to use this extension.'
  );

  return;
}

Do remember that we're creating the OpenAI service instance on each API call. This is not optimal and you should move it to the constructor and handle the error scenarios appropriately.

Conclusion

As is evident from the article, developing a VS Code extension can be easy and fun. If you're observant you can recreate an existing thing (Hashnode AI editor in this case) in your way someplace else, and learn a ton along the way.

To publish an extension we need to complete a few more steps and optionally bundle it using webpack or a suitable bundler. To learn more about the process you can visit these links: 1. publishing an extension, 2. Bundling an extension

This was just a sneak peek of the extension I created. If interested, you can look at the complete source code of the extension here.

If you want to try out the extension (Write Assist AI), you can get it from here.

I hope you liked reading the article. If you found any mistakes in the article, please let me know in the comments. Your suggestions and feedback are most welcome.

-- Keep adding the bits, soon you'll have more bytes than you may need. :-)

The AI hype refuses to die, more so after the release of Chat GPT and more recently the GPT4. I've been missing the AI action so far, so when MindsDB & Hashnode announced this hackathon and I saw the Twitter Bot implementation using the OpenAI APIs I knew that I wanted to build a bot for the hackathon.

But then the question arose, a bot for what purpose and for which application? Twitter was already done and dusted. I wanted my bot to be a wise and witty companion who is always available, so that helped me zero down on a Gmail bot. But the Gmail Integration was not yet implemented, and that was the second reason for choosing to build it. Practicing my Python skills, and the idea of contributing to a good project were the other important reasons.

Tl;dr this article is about creating a new application (Gmail) handler for MindsDB and then using that handler to create an email bot that replies to incoming emails interestingly and poetically.

Setting up the environment

Since the first task is to develop the Gmail handler, we must set up the environment for development. But before that, I needed to know how to contribute to this project, and how to install MindsDb for development. During the installation I faced only one issue related to libmagic which was not installed on my Mac by default, so had to install it using brew install libmagic.

The next step was to learn the basics of creating an app handler. This gave me a good overview of what I'm supposed to do for creating the Gmail handler.

Going through the relevant docs and following the mentioned steps is crucial if you want to contribute to any existing project.

Running the existing installation

Now it was time to get my hands dirty, and the first step in that direction was to use an existing app handler. But before that, I followed the "Predict Home Rental Prices" tutorial from the "Learning Hub" in the local MindsDB web console, and everything worked fine.

Next was the turn of the Twitter handler. I tried creating a tweets database using the below command in the local MindsDB browser console.

CREATE DATABASE my_twitter 
WITH 
    ENGINE = 'twitter',
    PARAMETERS = {
      "bearer_token": "twitter bearer token",
      "consumer_key": "twitter consumer key",
      "consumer_secret": "twitter consumer key secret",
      "access_token": "twitter access token",
      "access_token_secret": "twitter access token secret"
    };

At least it should error out saying invalid credentials, but instead, I got the below error

Can't connect to db: Handler 'twitter' can not be used

Well, that's a bummer. Why it is not working? This is where you start debugging and find out what is happening in the codebase. And how do we do that? I simply searched for the error string "Can't connect to db" in the codebase and found the issue to be related to the handler not getting imported. After some more investigation saw that the zsh console has this info message

Dependencies for the handler 'twitter' are not installed by default. If you want to use "twitter" please install "['tweepy']"

And there we have it. This gives us a clue that if we want to use any of the other handlers (except the basic ones) we need to install their dependencies manually.

pip install tweepy and restarting MindsDB was enough to get the error I was hoping for in the first place

Can't connect to db: Error connecting to Twitter api: 401 Unauthorized Unauthorized. Check bearer_token

Now we're all set for development. As the docs said to study the Twitter handler, I simply created a copy of the twitter_handler folder and renamed it to gmail_handler. Then replaced "Twitter" with "Gmail" in __init__.py and __about__.py files along with related method name changes in the gmail_handler file. Verified it by executing the same Twitter create database command but replacing the engine with "gmail", and it seemed to call our gmail_handler.

Implementing the Gmail Handler

Going by the steps mentioned in how to create an application handler we need to modify the below methods. Before we can read/write emails we need to authenticate the user, so the first targets were the connect and the check_connection methods.

Setting up a Google project for Gmail APIs

To use the Gmail APIs we need to set up a Google Cloud Project and a Google Account with Gmail enabled. We will also need to enable the Gmail API from the Google Cloud Console.

Then we need to create OAuth Client Ids for authenticating users, and possibly an Auth Consent Screen (if this is the first time we're setting up OAuth)

Setting up OAuth Client Id will give us a credentials file which we will need in our gmail_handler for connection. You can find more information on how to set up a Google project for the Gmail APIs here.

Initing the GmailHandler class

We take the connection arguments (which are passed with the CREATE DATABASE command) and store them for future use. We also register an "emails" table where we will store our data.

class GmailHandler(APIHandler):
    """A class for handling connections and interactions with the Gmail API.

    Attributes:
        credentials_file (str): The path to the Google Auth Credentials file for authentication
        and interacting with the Gmail API on behalf of the uesr.

        scopes (List[str], Optional): The scopes to use when authenticating with the Gmail API.
    """

    def __init__(self, name=None, **kwargs):
        super().__init__(name)

        self.connection_args = kwargs.get('connection_data', {})
        self.credentials_file = self.connection_args['credentials_file']
        self.scopes = self.connection_args.get('scopes', DEFAULT_SCOPES)
        self.token_file = None
        self.max_page_size = 500
        self.max_batch_size = 100
        self.service = None
        self.is_connected = False

        emails = EmailsTable(self)
        self._register_table('emails', emails)

Handling Google Authentication

Following the link in the previous section, and the MindsDB code requirements, we need to do the following

1. Replace the content of requirements.txt inside the gmail_handler folder with the following. Do remember to install these modules using the pip install command

    google-api-python-client
    google-auth-httplib2
    google-auth-oauthlib
   

2. The connect method. Here we use the credentials files created in the previous section for authenticating the user.

    def connect(self):
        """Authenticate with the Gmail API using the credentials file.
   
        Returns
        -------
        service: object
            The authenticated Gmail API service object.
        """
        if self.is_connected is True:
            return self.service
   
        self.service = self.create_connection()
   
        self.is_connected = True
        return self.service
   
    def create_connection(self):
        creds = None
        token_file = os.path.join(os.path.dirname(self.credentials_file), 'token.json')
   
        if os.path.isfile(token_file):
            creds = Credentials.from_authorized_user_file(token_file, self.scopes)
   
        if not creds or not creds.valid:
            if creds and creds.expired and creds.refresh_token:
                creds.refresh(Request())
            elif not os.path.isfile(self.credentials_file):
                raise Exception('Credentials must be a file path')
            else:
                flow = InstalledAppFlow.from_client_secrets_file(self.credentials_file, self.scopes)
                creds = flow.run_local_server(port=0, timeout_seconds=120)
   
        # Save the credentials for the next run
        with open(token_file, 'w') as token:
            token.write(creds.to_json())
   
        return build('gmail', 'v1', credentials=creds)
   

3. The check_connection method

    def check_connection(self) -> StatusResponse:
        """Check connection to the handler.
   
        Returns
        -------
        StatusResponse
            Status confirmation
        """
        response = StatusResponse(False)
   
        try:
            # Call the Gmail API
            service = self.connect()
   
            result = service.users().getProfile(userId='me').execute()
   
            if result and result.get('emailAddress', None) is not None:
                response.success = True
        except HttpError as error:
            response.error_message = f'Error connecting to Gmail api: {error}.'
            log.logger.error(response.error_message)
   
        if response.success is False and self.is_connected is True:
            self.is_connected = False
   
        return response
   

Now we're ready to run the create database command and authenticate the user (of course we've not decided on the columns of our table but we will come to that). Simply run

CREATE DATABASE mindsdb_gmail
WITH ENGINE = 'gmail',
PARAMETERS = {
  "credentials_file": "mindsdb/integrations/handlers/gmail_handler/credentials.json"
};

Fetching Emails from the Gmail API

The flow for fetching emails using MindsDB is like this: you execute an SQL SELECT query, and the select method of the APITable class gets called. There you parse the query params and finally call the Gmail API accordingly.

• The select method of the EmailsTable

class EmailsTable(APITable):
    """Implementation for the emails table for Gmail"""

    def select(self, query: ast.Select) -> Response:
        """Pulls emails from Gmail "users.messages.list" API

        Parameters
        ----------
        query : ast.Select
           Given SQL SELECT query

        Returns
        -------
        pd.DataFrame
            Email matching the query

        Raises
        ------
        NotImplementedError
            If the query contains an unsupported operation or condition
        """

        conditions = extract_comparison_conditions(query.where)

        params = {}
        for op, arg1, arg2 in conditions:

            if op == 'or':
                raise NotImplementedError(f'OR is not supported')

            if arg1 in ['query', 'label_ids', 'include_spam_trash']:
                if op == '=':
                    if arg1 == 'query':
                        params['q'] = arg2
                    elif arg1 == 'label_ids':
                        params['labelIds'] = arg2.split(',')
                    else:
                        params['includeSpamTrash'] = arg2
                else:
                    raise NotImplementedError(f'Unknown op: {op}')

            else:
                raise NotImplementedError(f'Unknown clause: {arg1}')

        if query.limit is not None:
            params['maxResults'] = query.limit.value

        result = self.handler.call_gmail_api(
            method_name='list_messages',
            params=params
        )

        # filter targets
        columns = []
        for target in query.targets:
            if isinstance(target, ast.Star):
                columns = []
                break
            elif isinstance(target, ast.Identifier):
                columns.append(target.parts[-1])
            else:
                raise NotImplementedError(f"Unknown query target {type(target)}")

        if len(columns) == 0:
            columns = self.get_columns()

        # columns to lower case
        columns = [name.lower() for name in columns]

        if len(result) == 0:
            result = pd.DataFrame([], columns=columns)
        else:
            # add absent columns
            for col in set(columns) & set(result.columns) ^ set(columns):
                result[col] = None

            # filter by columns
            result = result[columns]
        return result

• The get_columns method. These are the columns that our "EmailsTable" will have.

def get_columns(self):
    """Gets all columns to be returned in pandas DataFrame responses

    Returns
    -------
    List[str]
        List of columns
    """
    return [
        'id',
        'message_id',
        'thread_id',
        'label_ids',
        'from',
        'to',
        'date',
        'subject',
        'snippet',
        'body',
    ]

• The call_gmail_api method of the GmailHandler class. The way the Gmail messages API works is, first it returns a list of the messages that match your query criteria. These messages contain just the threadId & the messageId etc and not the full email. Then using the "messageIds" you fetch the full messages separately.

def call_gmail_api(self, method_name: str = None, params: dict = None):
    """Call Gmail API and map the data to pandas DataFrame
    Args:
        method_name (str): method name
        params (dict): query parameters
    Returns:
        DataFrame
    """
    service = self.connect()
    if method_name == 'list_messages':
        method = service.users().messages().list
    elif method_name == 'send_message':
        method = service.users().messages().send
    else:
        raise NotImplementedError(f'Unknown method_name: {method_name}')

    left = None
    count_results = None
    if 'maxResults' in params:
        count_results = params['maxResults']

    params['userId'] = 'me'

    data = []
    limit_exec_time = time.time() + 60

    while True:
        if time.time() > limit_exec_time:
            raise RuntimeError('Handler request timeout error')

        if count_results is not None:
            left = count_results - len(data)
            if left == 0:
                break
            elif left < 0:
                # got more results that we need
                data = data[:left]
                break

            if left > self.max_page_size:
                params['maxResults'] = self.max_page_size
            else:
                params['maxResults'] = left

        log.logger.debug(f'Calling Gmail API: {method_name} with params ({params})')

        resp = method(**params).execute()

        if 'messages' in resp:
            self._handle_list_messages_response(data, resp['messages'])
        elif isinstance(resp, dict):
            data.append(resp)

        if count_results is not None and 'nextPageToken' in resp:
            params['pageToken'] = resp['nextPageToken']
        else:
            break

    df = pd.DataFrame(data)

    return df

• Inner method _handle_list_messages_response and other related methods

# Handle the API response by downloading the full messages
# using a Batch Request.
def _handle_list_messages_response(self, data, messages):
    total_pages = len(messages) // self.max_batch_size
    for page in range(total_pages):
        self._get_messages(data, messages[page * self.max_batch_size:(page + 1) * self.max_batch_size])

    # Get the remaining messsages, if any
    if len(messages) % self.max_batch_size > 0:
        self._get_messages(data, messages[total_pages * self.max_batch_size:])

def _get_messages(self, data, messages):
    batch_req = self.service.new_batch_http_request(lambda id, response, exception: self._parse_message(data, response, exception))
    for message in messages:
        batch_req.add(self.service.users().messages().get(userId='me', id=message['id']))

    batch_req.execute()

# This method shows how to parse the full email returned 
# by the Gmail API
def _parse_message(self, data, message, exception):
    if exception:
        log.logger.error(f'Exception in getting full email: {exception}')
        return

    payload = message['payload']
    headers = payload.get("headers")
    parts = payload.get("parts")

    row = {
        'id': message['id'],
        'thread_id': message['threadId'],
        'label_ids': message.get('labelIds', []),
        'snippet': message.get('snippet', ''),
    }

    if headers:
        for header in headers:
            key = header['name'].lower()
            value = header['value']

            if key in ['from', 'to', 'subject', 'date']:
                row[key] = value
            elif key == 'message-id':
                row['message_id'] = value

    row['body'] = self._parse_parts(parts)

    data.append(row)

def _parse_parts(self, parts):
    if not parts:
        return

    body = ''
    for part in parts:
        if part['mimeType'] == 'text/plain':
            part_body = part.get('body', {}).get('data', '')
            body += urlsafe_b64decode(part_body).decode('utf-8')
        elif part['mimeType'] == 'multipart/alternative' or 'parts' in part:
            # Recursively iterate over nested parts to find the plain text body
            body += self._parse_parts(part['parts'])
        else:
            log.logger.debug(f"Unhandled mimeType: {part['mimeType']}")

    return body

The above is sufficient to fetch and store the emails of the authenticated user in the database. We can run an SQSL SELECT query like below to fetch the emails. The query parameter supports all the filter options that are available in the Gmail API

SELECT *
FROM mindsdb_gmail.emails
WHERE query = 'from:test@example.com OR search_text OR from:test@example1.com'
AND label_ids = "INBOX,UNREAD" 
LIMIT 20;

Sending Emails using the Gmail API

For sending emails through the Gmail API and MindsDB we need to use the SQL INSERT query. This in turn calls the insert method of the EmailsTable class, we created earlier.

def insert(self, query: ast.Insert):
    """Sends emails using the Gmail "users.messages.send" API

    Parameters
    ----------
    query : ast.Insert
        Given SQL INSERT query

    Raises
    ------
    ValueError
        If the query contains an unsupported condition
    """
    columns = [col.name for col in query.columns]

    if self.handler.connection_args.get('credentials_file', None) is None:
        raise ValueError(
            "Need the Google Auth Credentials file in order to write an email"
        )

    supported_columns = {"message_id", "thread_id", "to_email", "subject", "body"}
    if not set(columns).issubset(supported_columns):
        unsupported_columns = set(columns).difference(supported_columns)
        raise ValueError(
            "Unsupported columns for create email: "
            + ", ".join(unsupported_columns)
        )

    for row in query.values:
        params = dict(zip(columns, row))

        if not 'to_email' in params:
            raise ValueError('"to_email" parameter is required to send an email')

        message = EmailMessage()
        message['To'] = params['to_email']
        message['Subject'] = params['subject'] if 'subject' in params else ''

        content = params['body'] if 'body' in params else ''
        message.set_content(content)

        # If threadId is present then add References and In-Reply-To headers
        # so that proper threading can happen
        if 'thread_id' in params and 'message_id' in params:
            message['In-Reply-To'] = params['message_id']
            message['References'] = params['message_id']

        encoded_message = urlsafe_b64encode(message.as_bytes()).decode()

        message = {
            'raw': encoded_message
        }

        if 'thread_id' in params:
            message['threadId'] = params['thread_id']

        self.handler.call_gmail_api('send_message', {'body': message})

This method calls the same call_gmail_api we saw earlier to send an email. We can use an SQL INSERT query to send an email. The thread_id and message_id parameter values are only required if we're replying to an incoming email and want that our reply should form a thread with the original email. (The "subject" should exactly match the original subject line for it to work)

INSERT INTO mindsdb_gmail.emails (thread_id, message_id, to_email, subject, body)
VALUES ('187cbdd861350934d', '8e54ccfd-abd0-756b-a12e-f7bc95ebc75b@Spark', 'test@example2.com', 'Trying out MindsDB',
        'This seems awesome. You must try it out whenever you can.')

Creating the Gmail Bot

Now that we're unblocked and can fetch/send emails easily using our shiny new GmailHandler, we're ready to work on our Gmail bot.

Obtaining an OpenAI API key

Since we're developing this locally we do not have the luxury of using the inbuilt API key provided by MindsDB Cloud. We need to create an account on OpenAI and create an API key.

Training the Model using MindsDB

I do not have access to GPT4 APIs, so I'm using the gpt-3.5-turbo model itself. We're just telling GPT to respond to the email with a proper salutation and signature and to keep the email tone casual. This is done using the prompt_template parameter.

CREATE MODEL mindsdb.gpt_model
PREDICT response
USING
engine = 'openai',
max_tokens = 500,
api_key = '<your_api_key>', 
model_name = 'gpt-3.5-turbo',
prompt_template = 'From input message: {{input_text}}\
by from_user: {{from_email}}\
In less than 500 characters, write an email response to {{from_email}} in the following format:\
Start with proper salutation and respond with a short message in a casual tone, and sign the email with my name mindsdb';

Once the training is complete we're ready to see our bot in action. Run the following command

SELECT response
FROM mindsdb.gpt_model_email
WHERE from_email = "alice@example.com" 
AND input_text = "Hi there, I'm bored. Give me a puzzle to solve";

And we get the following response, seems quite all right, isn't it?

On asking for a new puzzle it says the following

Giving the bot a persona

Since our initial experiments seem to work fine, we can get a bit adventurous now. Let's give our bot a combined persona of Master Yoda from the Star Wars movies, and Edgar Allan Poe, the famous poet. We do this by changing the prompt_template in our earlier command

CREATE MODEL mindsdb.gpt_model_yodapoe
PREDICT response
USING
engine = 'openai',
max_tokens = 800,
api_key = '<your_api_key>', 
model_name = 'gpt-3.5-turbo', -- you can also use 'text-davinci-003' or 'gpt-3.5-turbo'
prompt_template = 'From input message: {{input_text}}\
by from_user: {{from_email}}\
In less than 500 characters, write an email response to {{from_email}} in the following format:\
<respond with a 4 line poem as if you were Edgar Allan Poe but you are also a wise elder like Master Yoda from the Star Wars movies. The wordings should be like Master Yoda but the format should be like Poe. Do not mention that you are Master Yoda or Edgar Allan Poe. Sign it with a made up quote similar to what Voltaire, Nietzsche etc would say. Do not explain or say anything else about the quote.>';

Train the model, and then run the same SELECT queries by changing the model name

SELECT response
FROM mindsdb.gpt_model_yodapoe
WHERE from_email = "alice@example.com" 
AND input_text = "Hi there, I'm bored. Give me a puzzle to solve";

This is what I get

On changing the input text to the following

SELECT response
FROM mindsdb.gpt_model_yodapoe
WHERE from_email = "alice@example.com" 
AND input_text = "Hi there, What's in a hackathon?";

we get the following result

Overall this seems to be working fine. We can always fine-tune the persona based on our taste. Now we can connect this response generation with the actual email sending and we may as well create a scheduled job to read emails at regular intervals and reply to them based on some predefined criteria.

Outcomes

When I started working on this feature, I had the following goals and their respective outcomes at the end

1. Create the Gmail handler: This in my opinion is done. There may be some bugs that I'll need to solve in due course

2. Contribute to the MindsDB project: I've already opened a PR for my changes and now I'm hoping that it gets merged into the codebase.

3. Create a Gmail Bot: We've all the ingredients in place, we just need to deploy it somewhere so that it is always available. I did try deploying the source on a droplet but I'm stuck with an error for which I've raised a GitHub issue.

4. Practice my Python skills: I think I've made good progress on this while working on the feature. Python is not my area of expertise, so I'm quite pumped that I was able to create a working integration in a short span of 7-8 days.

Conclusion

Overall it was quite fun and interesting to create a Gmail Bot and the needed Gmail Handler for MindsDB. During the process, I got to see the inner workings of the MindsDB codebase. I learnt to use the Gmail APIs and how emails are structured behind the scenes and how to parse it. It also allowed me to use the MindsDB integration with OpenAI, and now I too can say AI is eating the world :-).

Hope you liked reading the article. If you've noticed any error or issue anywhere please do let me know in the comments.

-- Keep adding the bits, soon you'll have more bytes than you may need.

Introduction

Recently I was looking to implement keyboard navigation for my daily puzzle game GoldRoad. I had a vague idea about adding key event listeners and using these key presses but I wanted to learn if there is something more to it. And I wasn't disappointed. This article tells the story of the rabbit hole I fell into.

To navigate a website using the keyboard, we typically use the Tab key (If you're on the Safari browser, and haven't changed any settings then you need to use the option + Tab keys). We keep pressing the Tab key and we're taken to different elements of the webpage.

If you're on a laptop or desktop right now, and if you haven't tried it already, you can experience it yourself by pressing the Tab key a couple of times. The elements that get focussed and in what order are decided by their "tabindex" attribute.

What is tabindex?

As the name suggests, tabindex is the index of tabbing and it is a global HTML attribute. It decides which elements on a page get focussed and in what order when keyboard navigation is done using the Tab key. It can take integer values: a lower positive value implies that the element will be reached (and focussed) ahead of others (including elements having tabindex of 0). Any negative value (usually -1 is used) means that the element can't be reached by pressing the tab key alone.

Some interactive HTML elements like buttons, inputs, selects, anchor tags etc get a default tabindex value of 0, and these are the elements that usually get focussed when we do keyboard navigation.

Run the below CodeSandBox to see it in action. The h2 element has a tabindex of 0, and since it is present before the grid of buttons, it gets focused first when you press the Tab key, and then the buttons get focused one after another in the order they were added to the DOM, and so on. Notice that the button with tabindex -1 doesn't get focussed.

Shortcomings of the Tab key

If you tried navigating the previous grid, you'll notice the below problems

1. To get to the 9th button, 10 key presses are needed (one for the h2 element, and then 9 more for the 9 buttons). And how do you go back and forth between these buttons? We can keep pressing the Tab key but that is not an optimal experience.

2. What if we wanted to start from the middle of the grid? We could give the middle button a positive tabindex value, but even though positive integers are valid values we should avoid using values greater than 0 (this is because it messes up the keyboard navigation for people using assistive technologies).

3. Similar to issue 1, if we've other focusable elements below the grid (e.g. the bottom p tag), then it will take a lot of key presses to reach there. How do we solve this?

Grid Navigation using the Arrow Keys

We can improve our grid navigation by using the arrow keys. This will solve the 1st issue, and maybe the 3rd issue partially, but to solve it effectively we need to consider the grid as a single entity in terms of the tab stops. So the first tab key takes us to the h2 element, the second one takes us to the grid, and the third tab press takes us to the paragraph element. And to navigate inside the grid we use the arrow keys with the help of the roving tabindex technique.

What is Roving tabindex?

To consider the grid as a single entity we need to assign a tabindex of -1 to all the inner buttons and give a tabindex="0" to that one button where the focus should land. Then we use EventListeners for tracking the key presses, and we keep roving this tabindex="0" and the corresponding focus to the appropriate button. At a time there will be only one button having a 0 tabindex.

This technique of managing focus inside a component using the tabindex attribute is called the roving tabindex technique. This also allows us to come back to the same element of the grid from where we tab away from it (this is a requirement for better accessibility).

Roving tabindex implementation

Enough talk. Let's implement the "roving tabindex" technique for the below grid. You can try navigating this grid with your arrow keys after pressing the Tab key once.

Create a Grid of Buttons

Create a new react project with the create-react-app command or use whichever CRA alternative you prefer. Then create a new file called GridNavigator.js inside your src folder. Add the following code to the file.

import { useRef } from "react";

export const GridNavigator = ({ rows, cols, start }) => {
  const currentIdx = useRef(start);

  return (
    <div>
      {Array.from(Array(rows), (_, row) => (
        <div key={`row-${row}`}>
          {Array.from(Array(cols), (_, col) => {
            const idx = `${row}${col}`;

            return (
              <button
                key={`col-${idx}`}
                tabIndex={currentIdx.current === idx ? "0" : "-1"}
              >
                {idx}
              </button>
            );
          })}
        </div>
      ))}
    </div>
  );
};

This component takes the number of rows & columns from its parent and creates a corresponding grid of buttons. It also sets the tabindex of each of the buttons to -1 (except the button having its idx === start).

To test this GridNavigator you can call it inside your App.js file as shown below

import { GridNavigator } from './GridNavigator';

import './App.css';

function App() {
  return (
    <div className="App">
      <GridNavigator rows={5} cols={5} start='24' />
    </div>
  );
}

export default App;

Setting the Accessibility Attributes

It is a good practice to assign proper roles to each grid item for better accessibility. And you should also assign proper ARIA attributes to describe the grid to the users who take the help of assistive technologies. Below are some of the attributes we'll be using

• role: Since our grid is made up of interactive elements we'll be using role="grid" for the outermost div, role="row" for each row, and role="gridcell" for the buttons. Other possible replacements for the grid role is the table or the treegrid roles.

• aria-label: This can be used to give the grid a proper caption.

• aria-rowcount & aria-colcount: For mentioning the rows & columns count of the grid. These attributes need to be used on the outermost div having the role\="grid".

• aria-rowindex & aria-colindex: On each button having the role of gridcell, we should mention its row & column index.

For more details about various aria attributes related to the grid role, you can visit this MDN link.

Make minor adjustments to the code from the last section as shown below

//... rest of the code
return (
  <div 
    role="grid"
    aria-label='A grid of buttons'
    aria-rowcount={rows} 
    aria-colcount={cols}
  >
    {Array.from(Array(rows), (_, row) => (
      <div key={`row-${row}`} role="row">
        {Array.from(Array(cols), (_, col) => {
          const idx = `${row}${col}`;

          return (
            <button
              key={`col-${idx}`}
              tabIndex={currentIdx.current === idx ? "0" : "-1"}
              role="gridcell"
              aria-rowindex={row}
              aria-colindex={col}
            >
              {idx}
            </button>
          );
        })}
      </div>
    ))}
  </div>
);
//... rest of the code

Adding the key event listener

Add a key-down event listener on the outermost div. Apart from the event listener, we will give every button a ref so that we can use it to focus the appropriate button on arrow key presses.

Make the following changes to the GridNavigator component

// import createRef
import { useRef, createRef } from 'react';

Inside the GridNavigator function, create a ref to store the references of all the grid buttons

const btnRefs = useRef({});

Add the key event listener on the div with role="grid". Also, create and add a ref to all the buttons

return (
    <div
      role='grid'
      aria-label='A grid of buttons'
      aria-rowcount={rows}
      aria-colcount={cols}
      onKeyDown={onKeyDown}
    >
      {Array.from(Array(rows), (_, row) => (
        <div key={`row-${row}`} role='row'>
          {Array.from(Array(cols), (_, col) => {
            const idx = `${row}${col}`;
            // If we don't have a ref for this button, create it
            if (!btnRefs.current[idx]) {
              btnRefs.current[idx] = createRef();
            }

            return (
              <button
                key={`col-${idx}`}
                ref={btnRefs.current[idx]}
                tabIndex={currentIdx.current === idx ? '0' : '-1'}
                role='gridcell'
                aria-rowindex={row}
                aria-colindex={col}
              >
                {idx}
              </button>
            );
          })}
        </div>
      ))}
    </div>
  );

Add the onKeyDown function along with the helper functions

// Parses the row & col value of the currently selected button
const parseRowCol = () => {
  const row = parseInt(currentIdx.current[0], 10);
  const col = parseInt(currentIdx.current[1], 10);
  return { row, col };
};

// Moves the focus & saves the id of the newly focused button
const handleFocus = (row, col) => {
  currentIdx.current = `${row}${col}`;
  const btnRef = btnRefs.current[currentIdx.current];
  btnRef.current.focus();
};

// Handles keyboard events
const onKeyDown = (event) => {
  const { row, col } = parseRowCol();

  switch (event.key) {
    case 'ArrowUp':
      if (row > 0) {
        handleFocus(row - 1, col);
      }

      break;
    case 'ArrowDown':
      if (row < rows - 1) {
        handleFocus(row + 1, col);
      }

      break;
    case 'ArrowLeft':
      // If we're on the leftmost col then move to the extreme right
      // col of the previous row, provided it's not the first row
      if (col > 0) {
        handleFocus(row, col - 1);
      } else if (row > 0) {
        handleFocus(row - 1, cols - 1);
      }

      break;
    case 'ArrowRight':
      // If we're on the rightmost col then move to the first 
      // col of the next row, provided it's not the last row
      if (col < cols - 1) {
        handleFocus(row, col + 1);
      } else if (row < rows - 1) {
        handleFocus(row + 1, 0);
      }

      break;
    default:
      return;
  }
};

Now try navigating the grid using your keyboard. To start the navigation you need to press the Tab key which will put the focus on the button having idx === start. Afterwards, you can use your arrow keys to navigate the grid. To come out of the grid press the Tab key once more. If you try to focus the grid once more, your focus should land on the exact button where you left it.

This implementation solves all the problems we set out to solve. But some of the end users may not know that they can use the arrow keys to navigate the grid, so we mustn't rely on keyboard navigation alone.

Conclusion

• Keyboard navigation is crucial for web accessibility and improves the user experience for those who depend on it.

• The roving tabindex method can enhance keyboard navigation for complicated interactive elements such as grids.

• Combining tabindex with correct accessibility attributes such as roles and ARIA attributes ensures that web applications are accessible to a broader audience, including those with disabilities.

Hope you enjoyed reading the article. If you found any mistake in the article please let me know in the comments so that I can fix it.

Cheers :-)

Introduction

Before going further, let's review the two terms, Caching and CDN, which are essential for a basic understanding of the topic.

What is Caching?

Caching is the process of storing copies of some of your data in temporary storage locations for faster retrieval. These temporary storage locations can be anywhere in between and including the original content storage (e.g. a database) and the client (e.g. your browser).

What is a CDN?

A CDN or Content Delivery Network is a network of geographically distributed servers that are used to serve content/data to end users faster.

These servers sit between us, the client, and the origin server, and can keep a cache of our content for the configured time. Also, since they are distributed across the world, the data is served from the edge that is closest to us. And because of this proximity, the round trip delay is much lower compared to if the request had to be served from the origin server (or simply the origin).

But how does the CDN get the data in the first place?

Initially, the CDN cache doesn't have any data, we call it a cold cache. When a request is made, it is routed to the edge closest to you. The edge location checks its cache, which is empty, gets the data from the origin and returns it to you, simultaneously storing it in its cache (this is called warming the cache) for future requests.

So one request is all it takes for the CDN to get the data?

Yes and no! The edge location closest to you gets the data when you make the first request. Now, any subsequent "similar" request to the same edge, either by you or someone else, is served from the cache. Since this cache has the relevant data for the request, it is called a warm or hot cache.

If a request is made to some other edge location by someone else then that server might not have any data stored yet, so the same process of data fetching and storing gets repeated there.

As you might have guessed, in this article we will be looking at storing our API responses at the CDN for faster retrieval.

Configuring API Caching with Firebase CDN

So how do we configure the firebase CDN? If you look at the firebase documentation, you won't find any separate subsection called CDN. Firebase CDN is covered under firebase hosting, you can read about it in the introduction here.

To use the CDN for API caching we need to connect our firebase function(s)/cloud run to firebase hosting. But does that mean we need to host our website on firebase hosting? Not necessarily.

Setup a Firebase Project

If you don't have a firebase project already, create one using either the command line or the firebase console.

To show the difference between a direct firebase function call and one through the CDN we'll be creating a simple project. I simply executed the firebase init command inside an empty directory and followed the prompts (selected the default choice for each). In this project, I've enabled only hosting and functions.

This is my current project directory. Everything has been generated by the init command

Next, head to the index.js file within the functions folder. I've created two simple functions inside it as shown below. We'll be using one of the functions directly and the other one through the CDN. On function calls, we just respond with preconfigured messages

const functions = require('firebase-functions');

exports.helloWorld = functions.https.onRequest((req, res) => {
  functions.logger.info(`helloWorld! Hostname: ${req.hostname}`);
  res.send({ message: 'Hello World!' });
});

exports.wonderfulWorld = functions.https.onRequest((req, res) => {
  functions.logger.info(`wonderfulWorld! Hostname: ${req.hostname}`);
  res.send({ message: 'What a Wonderful World!' });
});

If we try to call these functions from our index.html file we'll get CORS errors. So before deploying the functions let's install the cors package using the "npm i cors" command and wrap the two functions' bodies inside it.

const functions = require('firebase-functions');
// Require cors. For testing we're allowing it for all origins
const cors = require('cors')({origin: true})

exports.helloWorld = functions.https.onRequest((req, res) => {
  functions.logger.info(`helloWorld! Hostname: ${req.hostname}`);
  cors(req, res, () => {
    res.send({ message: 'Hello World!' });
  })
});

exports.wonderfulWorld = functions.https.onRequest((req, res) => {
  functions.logger.info(`wonderfulWorld! Hostname: ${req.hostname}`);
  cors(req, res, () => {
    res.send({ message: 'What a Wonderful World!' });
  })
});

Now we're ready to test these functions. Deploy the functions using the firebase deploy command

firebase deploy --only functions

Now head over to the index.html file inside the public folder and replace its content with the below code. I've just modified the file generated by firebase and added the two function calls in it.

Don't forget to replace <project_id> with your actual project id. You may also need to change the function region if you deployed to a region other than us-central1

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1" />
    <title>Welcome to Firebase Hosting</title>

    <style media="screen">
      body {
        background: #eceff1;
        color: rgba(0, 0, 0, 0.87);
        font-family: Roboto, Helvetica, Arial, sans-serif;
        margin: 0;
        padding: 0;
      }
      #message {
        background: white;
        max-width: 360px;
        margin: 100px auto 16px;
        padding: 32px 24px;
        border-radius: 3px;
      }
      #message h1 {
        font-size: 32px;
        color: #ffa100;
        font-weight: bold;
        margin: 0 0 16px;
      }
      #message p {
        line-height: 140%;
        font-size: 14px;
      }
      #message a {
        display: block;
        text-align: center;
        background: #039be5;
        text-transform: uppercase;
        text-decoration: none;
        color: white;
        padding: 16px;
        border-radius: 4px;
        cursor: pointer;
      }
      #message a:hover {
        background: #028bd5;
      }
      #message,
      #message a {
        box-shadow: 0 1px 3px rgba(0, 0, 0, 0.12), 0 1px 2px rgba(0, 0, 0, 0.24);
      }
      #cdn,
      #direct {
        background: #e1e1e1;
        padding: 4px 8px;
        border-radius: 4px;
        color: black;
      }
      @media (max-width: 600px) {
        body,
        #message {
          margin-top: 0;
          background: white;
          box-shadow: none;
        }
        body {
          border-top: 16px solid #ffa100;
        }
      }
    </style>
  </head>
  <body>
    <div id="message">
      <h1>Welcome</h1>
      <p>Click on the button below to make the API Calls...</p>
      <a onclick="makeCalls()">Test API Calls</a>
      <p>Response through CDN</p>
      <pre id="cdn">Waiting for the click&hellip;</pre>
      <p>Response through function</p>
      <pre id="direct">Waiting for the click&hellip;</pre>
    </div>

    <script>
      const directEl = document.getElementById('direct');
      const cdnEl = document.getElementById('cdn');

      async function makeCalls() {
        directEl.textContent = 'Loading...';
        cdnEl.textContent = 'Loading...';

        const promises = [
          fetch(
            'https://us-central1-<project_id>.cloudfunctions.net/helloWorld'
          ),
          fetch(
            'https://us-central1-<project_id>.cloudfunctions.net/wonderfulWorld'
          ),
        ];

        const [directCall, cdnCall] = await Promise.allSettled(promises);
        if (directCall.status === 'fulfilled' && directCall.value.ok) {
          const directCallData = await directCall.value.json();
          console.log('directCallData', directCallData);

          directEl.textContent = JSON.stringify(directCallData, null, 2);
        }

        if (cdnCall.status === 'fulfilled' && cdnCall.value.ok) {
          const cdnCallData = await cdnCall.value.json();
          console.log('cdnCallData', cdnCallData);

          cdnEl.textContent = JSON.stringify(cdnCallData, null, 2);
        }
      }
    </script>
  </body>
</html>

If you load this html file in your browser and click the button you should be able to see the configured messages. But the first output label is misleading, as we haven't configured any CDN yet. Let's change that in the next section.

Below is a screenshot of my Chrome dev console's network tab. As you can see both requests are taking similar times at this point

Connecting Firebase function & CDN

To connect one of the functions to the CDN, let's head over to the "firebase.json" file at the root of the project, and make the below changes

// Modify the hosting attribute of your "firebase.json"
"hosting": {
  //...other hosting settings

  // Add the "rewrites" attribute within "hosting"
  "rewrites": [
    {
      "source": "/api/wonderful", // Your api route
      "function": "wonderfulWorld", // Your function name
      "region": "us-central1" // The region where the function is deployed
    }
  ]
}

If you want to use Google Cloud Run instead of firebase cloud functions, you can connect the two using the below rewrite

"hosting": {
  "rewrites": [
    {
      "source": "/api/wonderful",
      "run": {
        "serviceId": "<cloud_run_service_id>",
        "region": "us-central1" // The region where cloud run is deployed
      }
    }
  ]
}

Next, head over to the index.html file and replace the "wonderfulWorld" function URL with the below URL. Replace <project_id> with your actual project id.

const promises = [
  fetch(
    'https://us-central1-<project_id>.cloudfunctions.net/helloWorld'
  ),
  fetch('https://<project_id>.web.app/api/wonderful'),
];

After making this change we need to deploy our changes to firebase hosting. We can simply execute the "firebase deploy" or "firebase deploy --only hosting" command to do it.

Now, if we reload our local index.html file, or, head over to the URL https://<project_id>.web.app and click on the button a couple of times, we should see results similar to as shown below.

Do note that the response size for the wonderful API call has increased from the earlier value of 13 Bytes. Also, as you can see, the first calls for both functions take significantly more time which is because of the cold start of the cloud function. Afterwards, the direct function call is consistently faster compared to the wonderful call routed through the CDN. This is because we've added one extra step to the trip and haven't added any cache yet.

You can click on these calls to see their details (specifically the Response headers)

The helloWorld call

The wonderful API call

As you can see, there are many extra headers present in this call compared to the previous one. Some of the interesting headers we can notice

• x-served-by: it mentions a cache (of course it is empty at this point)

• vary: This contains more header names as compared to the direct function call

• x-cache: With value MISS, which is correct because there is nothing in the cache so it missed serving from the cache

• x-cache-hits: With value 0. If it serves from the cache then this value would be a positive integer

• cache-control: With a value private for both calls. This is the header we'll be modifying for enabling cache. private means that the data is private and can only be stored in a private cache, say your browser.

Configuring cache-control header

Head over to the index.js file within the functions folder, and add the following line to the two functions just before the "res.send" line.

res.setHeader('cache-control', 'public, max-age=30, s-maxage=90');

Deploy your functions changes by executing "firebase deploy --only functions".

What we're doing here is:

• Setting the cache-control header to public. This allows the CDN to cache the data

• Setting max-age to 30. This means that the browser can store this response, and it will remain fresh and valid for 30 seconds from the time it was generated.

• Setting s-maxage to 90. This directive is similar to the max-age directive but applies only to the shared caches (the CDNs and proxies in between the origin and the client). So we're storing the response for a longer duration at the CDN. This may or may not be the case always and it can be removed if not needed.

So essentially what we've done is allow caching at the client as well as the intermediate steps in the journey. And also configured the time for which we can go on without asking the origin for fresh data.

Below is what I see in the network tab of my browser's dev console after making some requests

As you can see, the first requests for both functions are taking their usual timings. But the second requests which were made after ~10 seconds take only 4ms. And we also see that it has been served from the disk cache. That means the responses were cached locally and no new network request was made.

Below are the response headers for the first two wonderful calls. Notice the x-cache header with a value of MISS. If you check your dev console, you'll also see that for the second call, no request headers are present. This is because no network request was made for it.

What is interesting is the third batch of requests. The helloWorld function call takes its usual ~400-500ms range (because a fresh request was made to the firebase function) but the wonderful call takes only 69ms. Further clicking on the wonderful request gives me the below details

We see that the x-cache has a value of HIT now and x-cache-hits is 1. This request was served from the cache, and our firebase function was not called. We can also verify this by checking the function logs in the Google Cloud Console.

What we've achieved here is:

1. significantly lesser response time

2. a lesser number of firebase functions invocations

3. potentially lesser number of database calls (generally you would get the data from a database and not just return a static value which we're doing currently)

When and where to cache

When deciding to use a cache, the first question you should ask yourself is, "When and where (locally, CDN etc.) to do it"? There is no silver bullet here, and every case needs to be analyzed based on its pros and cons. A wrong decision may return stale and irrelevant data to your users.

In general, if the same data is needed for a lot of users then it is a good candidate to be cached at the CDN. For example, weather data, some meta information, a blog post etc. The time for which to store the data depends on the case at hand.

User profile data doesn't change that frequently, but it is useful for only one person so it can be stored locally (using private for cache-control) for a short duration. And so on.

Things to Keep in Mind with Firebase CDN

If you're convinced about using a cache for your API, below are some things that you should keep in mind

• In general, a firebase function can be configured for up to 9 minutes of request timeout. But when you connect your functions to firebase hosting, the request timeout value for these functions gets capped at 60 seconds.

• Only GET & HEAD requests can be cached at the CDN

• For Firebase the Cache keys' generation depends on the following factors. If any of these factors are different a different key gets generated and hence cache hit or miss happens accordingly

  • The hostname (<project_id>.web.app in the example project of this article)

  • The path (/api/wonderful)

  • The query string (we didn't use any query string)

  • The content of the request headers specified in the Vary header (by default Firebase CDN uses Origin, cookie, need-authorization, x-fh-requested-host and accept-encoding as can be seen from the screenshots above). Only the __session cookie (if present) is made part of the cache key.

• Sometimes you need to remove the API data cached at the CDN. This can be done by redeploying to firebase hosting (using firebase deploy --only hosting)

• Instead of setting the cache-control header individually within each function, we can add it to firebase.json itself. Though for more control over individual requests caching, adding the header within the function is better.

Limitations

While Firebase CDN is good for general use cases, if you want more from your CDN then you should look for a proper CDN service like Cloudflare, Google and so on. Firebase CDN doesn't provide DDoS protection, Rate Limiting etc out of the box. Also, data transfer out of hosting is free till 360MB/day, beyond that you get charged $0.15/GB.

Real Live Caching Example

For one of my recent projects, I applied the Firebase CDN cache for one of the endpoints. The project is a daily puzzle game based on React SPA, called GoldRoad where every player gets the same puzzle which gets refreshed at midnight GMT.

This is how I've configured the cache. The request gets stored at the CDN for the maximum time possible (including a buffer of 5 mins).

let cacheTime = 300; // 5 mins local cache time
let serverCacheTime;

// 1. game is the current puzzle object
// 2. nextGameAt is the dateTime when the new puzzle will be available
const nextGameAtInMs = new Date(game.nextGameAt).getTime();

// Server Cache time, minus the local cache time (some buffer)
serverCacheTime = parseInt((nextGameAtInMs - Date.now()) / 1000) - cacheTime;
if (serverCacheTime < 0) {
  serverCacheTime = 0;
}

if (serverCacheTime < cacheTime) {
  cacheTime = serverCacheTime;
}

response.set('Cache-Control', `public, max-age=${cacheTime}, s-maxage=${serverCacheTime}`
);

Many other directives can be used in the cache-control header. You should read more about these directives for advanced use cases.

Conclusion

In conclusion, using the firebase CDN can greatly enhance your API performance by caching frequently accessed data closer to the end user, thus reducing latency and improving response times. It also reduces firebase functions invocations, as well as database calls, thus reducing cost.

But before embarking on this journey, do analyze the pros and cons of adding a cache for your use case.

Hope you enjoyed reading the article. If you found any mistake in the article please let me know in the comments.

Cheers :-)

Further reading

We've only looked at a couple of directives for the cache-control header, for a complete overview please visit the MDN site

For understanding caching in detail you can visit the following links

HTTP caching on MDN

HTTP Cache on web.dev

"What is debugging?" is a typical question to start an article on the topic, but this is not that article. Instead what I want you to do here is, take a look at the cover image, and process it. Now, try to answer, why the person in the image is doing whatever he is doing. There can be multiple correct answers to this question, but if you answered along the lines of "to keep the crops healthy for a better yield" then you're not far from the zen of software development.

Debugging is not a chore, even though it feels like one at times, but is an important pillar on which any healthy and yielding software is built. You cannot love programming and not love (okay, "love" may be a strong word here, any value on the positive axis should do) debugging at the same time. This is akin to "Heisenberg's Certainty Principle" (is there one?) for Software development. The sooner you start loving the debugging process, the closer you're to achieving zen.

Why, and what, to debug?

Before we get to the "how", we need to understand the "why", and the "what". I'm sure by now you've some inklings on why we need to debug. Because we want our software to be healthy and useful for the end user. And we achieve that by making it error-free while giving the best experience.

The "what" is the next step of the "why". Whatever hinders our ability to give the best experience to the end user needs to be debugged and corrected. Some of these hindrances can be because of faulty code, some because of the UI/UX, some others may be because of the network, and so on.

The "what" may have left you confused. There is no clear answer in the previous paragraph. And you're right because it depends on the situation you're in. In my previous life, I used to work on Video Telephony (think Zoom calls for phones, but long back and with different standards and protocols). Now there was a phone already in the market which used to work fine with other similar phones. Then there was this phone in the testing phase which we were working on. And when you make a video call between these two, no video used to come on either of the devices. Of course, the video call between the two of our devices was fine as well. Now, whose fault is it? The only truth here is that the call should work, and we should debug every entity in between and including the two devices but starting with ours.

How to debug?

There are many techniques and approaches to debugging. What you need to understand is that the what, and the how, go hand in hand. Depending on the issue you're facing, one way of debugging might be better suited for the job at hand. And sometimes you may need to apply more than one technique to find out the root cause. I'm not going to go into the definitions and detailed explanations of these techniques, you can read about them with a simple Google Search (or maybe ask ChatGPT?).

The starting point of debugging is not using a debugger or adding some debug messages. Rather it is having at least a basic understanding of how something works. If you have that understanding, you can use any appropriate technique to debug. Let's go back to the video call example. On a high level, the way it works is:

1. The calling device sends signals/messages to the server that it wants to do a video call with the other device

2. The server informs the other device about the incoming call

3. The two devices do further signalling to arrive at a common medium of communication (the audio & the video codecs)

4. Finally, the media flows between the devices (with or without the server in the loop)

Now we can start eliminating the suspects based on the information we've at hand. Since the call gets established we can rule out the signalling issues. Next, we check whether the other device's transmitted video packets reach our device and whether our video packets leave the device or not. After verifying all these using network packet sniffers, the conclusion was that there is something wrong with the video packets themselves.

So after talking to my senior, I dumped these incoming/outgoing packets into separate files (like the debug messages), and tried playing these with a video player. Finally, the issue was traced to the other device not sending (and expecting) initial video config header bytes (if I remember correctly, mere 12-16 bytes). And this I could figure out only because I had the required knowledge of the config header standard. The final fix was us adding a check; if the call is with that device, don't send or expect the config bytes and hardcode it. Even though we were doing everything correctly we had to add the fix, because the other device was already live in the market and could not be changed.

So the point I'm trying to make here is that unless you have the needed knowledge it can be very challenging to fix an issue. That doesn't mean that you cannot gain this knowledge in parallel while fixing the issue, in most cases that is how you do it. So try to gain knowledge about how exactly your software works. This may require you to venture beyond your assigned work, but that is how you improve.

Also, many times it is beneficial to discuss the issue with your peers and seniors, as they may provide a clue based on their experiences. And the other point is, sometimes you need to add the fix at your end even if everything is correct there :-). So don't resent it.

What to take home from debugging?

The first thing you should do after any debugging session is, reflect on it. Think about why the issue happened, and how it was fixed. Now that you have that extra knowledge what you could have done better if you had this knowledge before the issue? In my case, I could have checked for the config header bytes in the packet sniffer logs instead of dumping them into files and then finding out.

The second thing is, actually apply whatever you learnt in your previous experiences. Later on, with a different device and codec a similar issue occurred where the initial video was blurry. From the packets' analysis itself, I could figure out that the issue was with the config header.

Every debugging session should make you a smarter and better developer.

Don't rule out anything

During one of my recent debugging sessions, I relearned the fact that one should not rule out anything while fixing an issue.

The recent issue

The issue was the "copy to clipboard" not working on the DuckDuckGo Android browser. The issue was reported by one of the players of my puzzle game GoldRoad. I use the Web Share API for sharing the user's daily stats, and as a fallback for sharing Clipboard API is used for copying the result to the clipboard.

const shareStats = async () => {
  const text = 'The text to share';

  if (window.navigator.share) {
    try {
      await window.navigator.share({
        text,
      });
    } catch (error) {}
  } else {
    await window.navigator.clipboard.writeText(text);
  }
};

If you look at the compatibility chart for the Clipboard API you'll see that it has wide availability.

Also, the "clipboard-write" permission of the Permissions API is granted automatically to pages when they are in the active tab. So that should allow me to use the writeText method of the Clipboard API on any browser.

But it didn't work despite such reassurances. Now how do you debug the issue considering it needs to be done on Android (the DuckDuckGo Mac client works fine)? Since console logs were of no use, I used the normal p / div tags to print the debugging messages in the browser window itself.

The debugging revealed the following

1. "Navigator.clipboard.writeText" throws NotAllowedError with a message saying Write permission denied

2. Since write permission was denied, so maybe somehow we could query and ask for the needed permission using the Permissions API? Alas! "Navigator.permissions" is not available on DuckDuckGo