Introduction

In the previous post, we implemented streaming text-to-speech with the Minimax API by manually parsing Server-Sent Events and handling the aggregated summary block at the end of the stream. While functional, that approach had two pain points: hand-rolled SSE parsing logic and the need to detect and discard the final summary chunk. Since then, three things have improved the situation significantly:

1. Minimax added stream_options.exclude_aggregated_audio to the API
2. The eventsource-parser library provides a robust, spec-compliant SSE parser
3. Most importantly, I published minimax-speech-ts, a TypeScript SDK that wraps the entire Minimax Speech API (Lol)

Excluding the Aggregated Audio

In the previous post, we had to handle a summary block at the end of the stream that contained the complete aggregated audio data. We checked for data.status === 1 to filter out the final chunk with status === 2. This was necessary because the API always sent the full concatenated audio as the last event, which we didn’t need when streaming.

Minimax has since added a stream_options.exclude_aggregated_audio parameter. When set to true, the final chunk no longer contains the complete audio data. This means we no longer need to filter it out ourselves:

const response = await $fetch<ReadableStream>(
  `https://api.minimaxi.chat/v1/t2a_v2?GroupId=${minimaxGroupId}`,
  {
    method: "POST",
    headers: {
      Authorization: `Bearer ${minimaxAPIKey}`,
    },
    responseType: "stream",
    body: {
      stream: true,
      stream_options: {
        exclude_aggregated_audio: true, // no more summary block
      },
      text: "your text",
      model: "speech-02-hd",
      voice_setting: {
        voice_id: "Chinese (Mandarin)_Warm_Bestie",
        speed: 0.95,
        pitch: -1,
        emotion: "neutral",
      },
      language_boost: "Chinese,Yue",
    },
  }
);

With this option, every chunk in the stream has status: 1 and contains only an audio segment, except for the final chunk with status: 2 that now has an empty audio field. This eliminates the need for the status-checking logic we previously had in processEventData.

Replacing Manual SSE Parsing with eventsource-parser

In the previous implementation, we manually split the stream on \n\n boundaries and stripped data: prefixes. While this worked for Minimax’s specific formatting, it was fragile and made assumptions about the event structure. The eventsource-parser library provides a spec-compliant SSE parser that handles all edge cases for us.

Using EventSourceParserStream

The library exposes an EventSourceParserStream – a TransformStream that takes decoded text and outputs parsed SSE events. This is a direct replacement for our custom TransformStream:

import { EventSourceParserStream } from "eventsource-parser/stream";

const eventStream = response
  .pipeThrough(new TextDecoderStream())
  .pipeThrough(new EventSourceParserStream());

Each event emitted by EventSourceParserStream has a data property containing the event payload (with the data: prefix already stripped), an event property for the event type, and an id property for the event ID. For Minimax’s API, we only need event.data.

Simplified TransformStream

With the SSE parsing handled by the library, our TransformStream now only needs to extract audio data from parsed events:

import { EventSourceParserStream } from "eventsource-parser/stream";

const audioTransform = new TransformStream({
  transform(event, controller) {
    try {
      const parsed = JSON.parse(event.data);

      if (parsed.base_resp?.status_code !== 0) {
        controller.error(parsed.base_resp?.status_msg || "Unknown API error");
        return;
      }

      if (parsed.data?.audio) {
        controller.enqueue(Buffer.from(parsed.data.audio, "hex"));
      }
    } catch (error) {
      // Skip malformed events
    }
  },
});

const audioStream = response
  .pipeThrough(new TextDecoderStream())
  .pipeThrough(new EventSourceParserStream())
  .pipeThrough(audioTransform);

return sendStream(event, audioStream);

Compare this with the previous implementation: the buffer management, \n\n splitting, data: prefix stripping, and flush logic are all gone. The EventSourceParserStream handles all of that correctly according to the SSE specification.

Why Build a New SDK?

Minimax does provide an official JavaScript SDK, but it’s an MCP (Model Context Protocol) server – designed for AI agent tool-calling, not for direct use in application code. There is no official Node.js client library for calling the Minimax Speech API directly. If you want to integrate Minimax TTS into a server application, you’re left writing raw HTTP calls yourself, which is what I was doing in the previous post. That’s the gap minimax-speech-ts fills: a proper typed client library for the HTTP API.

Using minimax-speech-ts

While the above improvements already simplify the integration, there’s still boilerplate involved: constructing the request, handling authentication, converting hex to buffers, and managing error codes. I built minimax-speech-ts to wrap all of this into a clean TypeScript SDK.

Installation

npm install minimax-speech-ts

Basic Usage

import { MiniMaxSpeech } from "minimax-speech-ts";

const client = new MiniMaxSpeech({
  apiKey: process.env.MINIMAX_API_KEY!,
  groupId: process.env.MINIMAX_GROUP_ID,
});

// Non-streaming: get complete audio as a Buffer
const result = await client.synthesize({
  text: "your text",
  model: "speech-02-hd",
  voiceSetting: {
    voiceId: "Chinese (Mandarin)_Warm_Bestie",
    speed: 0.95,
    pitch: -1,
    emotion: "neutral",
  },
  languageBoost: "Chinese,Yue",
});

await fs.promises.writeFile("output.mp3", result.audio);

Streaming Usage

The SDK returns a ReadableStream<Buffer> that handles all the SSE parsing, hex decoding, and error handling internally:

const stream = await client.synthesizeStream({
  text: "your text",
  voiceSetting: { voiceId: "Chinese (Mandarin)_Warm_Bestie" },
  streamOptions: { excludeAggregatedAudio: true },
});

// Use directly with sendStream in Nuxt/Nitro
return sendStream(event, stream);

The SDK uses a camelCase interface (excludeAggregatedAudio) and automatically converts to the snake_case wire format (exclude_aggregated_audio) expected by the API. It also provides typed error classes for different failure modes:

import {
  MiniMaxAuthError,
  MiniMaxRateLimitError,
  MiniMaxValidationError,
} from "minimax-speech-ts";

try {
  const stream = await client.synthesizeStream({ text: "hello" });
} catch (e) {
  if (e instanceof MiniMaxRateLimitError) {
    // Back off and retry
  } else if (e instanceof MiniMaxAuthError) {
    // Invalid API key
  }
}

Beyond basic TTS, the SDK also covers voice cloning, voice design, async synthesis for long-form content, and voice management – the full Minimax Speech HTTP API surface. WebSocket support is planned for a future release.

Building a TypeScript SDK with Claude Code

The minimax-speech-ts library was built entirely using Claude Code, Anthropic’s CLI for Claude. The result is a ~1500-line TypeScript library with 79 tests, a single runtime dependency (eventsource-parser), dual ESM/CJS output, CI/CD, and TypeDoc-generated API documentation – built in about a day.

I’ve been developing a mental framework for effective AI-assisted coding that I think of as three iterative steps: Context, Limit, and Progress. The development of this SDK is a good concrete example of how it plays out in practice.

Step 1: Context – Feed the AI the Right Information

The quality of AI-generated code is directly bounded by the context it has access to. My initial prompt to Claude Code referenced:

• The Minimax API documentation for the complete API spec
• The MiniMax-MCP-JS repository as a reference for how Minimax structures their APIs
• My existing blog post as context for the current “dumb” approach
• My other TypeScript libraries (epubcheck-ts, epub.ts) as style reference

The single most important factor was that Minimax provides their API documentation in markdown format (e.g. speech-t2a-http.md). Claude Code could fetch and read the full API specification directly, generating accurate type definitions, request/response handling, and validation logic without manual transcription. Pointing at my existing libraries meant it could match my preferences for project structure, tooling choices (tsup, vitest), and coding conventions without explicit configuration.

Without good context, the AI guesses. With good context, it builds.

Step 2: Limit – Set Boundaries and Constraints

Context alone produces code, but it doesn’t guarantee correct code. The limit step is about defining and enforcing constraints that keep the output within acceptable bounds.

Tests as limits: Tests were written alongside the implementation from the very first feature commit. This meant Claude Code could run the 79 tests after each change and catch regressions immediately, covering all API methods, error classification, snake_case mapping, and streaming edge cases. Tests are the most concrete form of limit – they define exactly what “correct” means.

Linting as limits: Adding ESLint with typescript-eslint in strict mode (including no-explicit-any) prevented the AI from taking shortcuts with loose types. TypeScript’s strict mode itself is a limit – it forces the generated code to handle nullability and type narrowing properly.

Validation as limits: Client-side parameter validation (emotion/model compatibility, WAV format restrictions, required field checks) encodes domain-specific constraints that the AI learned from the API docs. The declarative validate() helper using [condition, message] tuples made these constraints explicit and testable.

Step 3: Progress – Review, Fix, and Ship

With context and limits in place, the final step is iterating toward completion: expanding scope, catching remaining issues, and preparing for release.

Expanding scope: Starting from core synthesize() and synthesizeStream() methods, I asked Claude Code to expand to the full API surface – async synthesis, file upload, voice cloning, voice design, voice management. The limits from step 2 ensured each expansion didn’t break existing functionality.

/review for catching subtle bugs: Claude Code has a built-in /review command that performs a code review on the current codebase. Running /review after the main implementation caught several issues: a TypeScript overload ordering bug (where the general signature shadowed the specific outputFormat: 'url' overload), missing traceId propagation in error paths, and the need to URL-encode the groupId query parameter. These are exactly the kinds of subtle issues that slip through during development but get caught by a systematic review pass.

AGENTS.md and .github/copilot-instructions.md for ongoing progress: The library includes an AGENTS.md file that documents the architecture, key patterns, test patterns, and build commands. This file is symlinked as CLAUDE.md so that Claude Code automatically picks it up as project instructions, while other AI coding tools (GitHub Copilot, Cursor, etc.) can read it via AGENTS.md – the emerging convention for AI agent context files. The symlink trick avoids maintaining two copies of the same content. The repository also has a .github/copilot-instructions.md file, generated by GitHub Copilot agent itself based on the existing codebase and AGENTS.md. GitHub Copilot automatically incorporates these instructions when performing code reviews on pull requests, so its PR reviews are aware of the project’s specific patterns (e.g. “validate before fetch”, “camelCase public API with snake_case wire format”) rather than applying generic heuristics. These files feed back into step 1 – they become context for future AI-assisted development, creating a virtuous cycle.

Shipping: Finally, Claude Code generated the README, CI workflow (Node 18/20/22), TypeDoc configuration, license, and npm metadata. Throughout the process, it maintained consistency in patterns like the camelCase-to-snake_case conversion and the error hierarchy.

The Takeaway

The Context → Limit → Progress cycle isn’t a one-shot sequence – it’s iterative. Each round of progress reveals new context needs and new constraints to enforce. The entire library went from empty directory to published npm package in a day, but the quality came from deliberately feeding the right context, setting clear limits, and iterating through review. I’ll explore this framework in more depth in a future post.

Conclusion

The three improvements covered in this post each address a different layer of the integration:

• exclude_aggregated_audio eliminates the need to filter the summary block at the API layer
• eventsource-parser replaces manual SSE parsing with a spec-compliant library
• minimax-speech-ts wraps the entire API into a typed SDK with streaming, error handling, and camelCase ergonomics

If you’re integrating Minimax TTS into a Node.js application, using the SDK is the most straightforward path. If you need more control or are integrating with a different runtime, the combination of exclude_aggregated_audio and eventsource-parser provides a clean foundation.

Additional Resources

Related Articles

• Handling Minimax TTS API: Basic HTTP and Streaming (Previous post)
• Convert Google text to speech API result to HTTP streamed response
• Convert Azure text to speech API result to HTTP streamed response
• Convert Azure text to speech API result to a web ReadableStream
• Convert AWS Polly text to speech API result to HTTP streamed response

Links

• minimax-speech-ts on GitHub
• minimax-speech-ts on npm
• minimax-speech-ts API Reference
• eventsource-parser on GitHub
• Minimax TTS API Documentation
• Claude Code

By the way – guess what tool and process I used to write this blog post.

Introduction

Previously, we discussed the usage of the Azure Text-to-Speech API and how to implement real-time audio streaming by converting the API’s output into a stream using Node.js PassThrough. In this tutorial, we will focus on converting the API’s output into a web-compatible ReadableStream.

Why web (readable) stream?

While the Node.js stream module would work for most API use cases, using web streams provides some additional benefits:

• Web streams are the modern standard across all JavaScript environments. This means you can use the same stream handlers in server and client code. For example, TransformStream handlers can move between API and browser code without modification, allowing for greater flexibility when design needs it.

• Compatible with the Fetch API. The Fetch API natively returns a ReadableStream, so using web streams allows for seamless integration with the Fetch API and other web platform features.

• Unified interface across different TTS providers: By using web streams, we can create a consistent interface for different TTS providers’ SDK or API, since output formats accepted in JavaScript can be converted to a ReadableStream one way or another.

Pull or Push?

Azure supports PushAudioOutputStream and PullAudioOutputStream. Both can be used to create a web-compatible ReadableStream. In the previous article we used PushAudioOutputStream, but let’s revisit the technical differences between the two.

• PushAudioOutputStream: To create PushAudioOutputStream, we need to provide PushAudioOutputStreamCallback, which has to implement function write(dataBuffer: ArrayBuffer) and function close().

• PullAudioOutputStream: To create PullAudioOutputStream, we need to provide PullAudioOutputStreamCallback, which has to implement function read(dataBuffer: ArrayBuffer): number and function close().

For our use case, we need to mimic a fetch streamed response, which returns a web standard ReadableStream. Comparing the interface of ReadableStream, which requires start, pull, and cancel methods, we can see that PullAudioOutputStream is more aligned with the ReadableStream interface, with its read method corresponding to the pull method in ReadableStream.

Implementation

For implementation, first we’ll set up a speech synthesizer similar to our previous article. Note the only difference is that we will be setting up a PullAudioOutputStream here.

const outputFormat =
  sdk.SpeechSynthesisOutputFormat.Audio16Khz128KBitRateMonoMp3;
const pullStream = sdk.PullAudioOutputStream.create();
const audioConfig = sdk.AudioConfig.fromStreamOutput(pullStream);
const speechConfig = sdk.SpeechConfig.fromSubscription(
  azureSubscriptionKey,
  azureServiceRegion
);
speechConfig.speechSynthesisVoiceName = voiceName;
speechConfig.speechSynthesisOutputFormat = outputFormat;

const synthesizer = new sdk.SpeechSynthesizer(speechConfig, audioConfig);
function speakTextAsync(synthesizer, text) {
  return new Promise((resolve, reject) => {
    synthesizer.speakTextAsync(
      text,
      function (result) {
        synthesizer.close();
        resolve(result);
      },
      function (err) {
        synthesizer.close();
        reject(err);
      }
    );
  });
}
await speakTextAsync(synthesizer, text);

Then we’ll create a ReadableStream from the PullAudioOutputStream. We only need to implement pull and cancel. Since the read method requires an ArrayBuffer, we’ll create a new buffer for each read operation.

const readableStream = new ReadableStream({
  async pull(controller) {
    try {
      const buffer = new ArrayBuffer(1024);
      const bytesRead = await pullStream.read(buffer);

      if (bytesRead > 0) {
        const chunk = new Uint8Array(buffer, 0, bytesRead);
        controller.enqueue(chunk);
      } else {
        pullStream.close();
        controller.close();
      }
    } catch (error) {
      console.error("[Speech] Error reading from pull stream:", error);
      controller.error(error);
      pullStream.close();
    }
  },
  cancel() {
    pullStream.close();
  },
});

return readableStream;

One minor optimization we can make is to use the type and autoAllocateChunkSize options on the ReadableStream. This allows the stream to reuse buffers from existing allocations or allocate new ones as needed to minimize memory usage and improve performance.

const readableStream = new ReadableStream({
  type: "bytes",
  autoAllocateChunkSize: 1024,
  async pull(controller) {
    try {
      const byobRequest = controller.byobRequest;
      if (byobRequest?.view) {
        const buffer = byobRequest.view.buffer as ArrayBuffer;
        const bytesRead = await pullStream.read(buffer);

        if (bytesRead > 0) {
          byobRequest.respond(bytesRead);
        } else {
          pullStream.close();
          controller.close();
        }
      } else {
        // Fallback: allocate our own buffer
        const buffer = new ArrayBuffer(1024);
        const bytesRead = await pullStream.read(buffer);

        if (bytesRead > 0) {
          const chunk = new Uint8Array(buffer, 0, bytesRead);
          controller.enqueue(chunk);
        } else {
          pullStream.close();
          controller.close();
        }
      }
    } catch (error) {
      console.error("[Speech] Error reading from pull stream:", error);
      controller.error(error);
      pullStream.close();
    }
  },
  cancel() {
    pullStream.close();
  },
});

return readableStream;

Then we can proceed to read from the readableStream as needed.

Conclusion

In this blog post, we explored how to convert Azure Text-to-Speech output into a web-readable stream. By leveraging the PullAudioOutputStream and the ReadableStream API, we can create a flexible and efficient solution for handling audio data in web applications. This approach not only improves performance but also enhances maintainability by providing a clear separation of concerns between the TTS provider’s interface and the API response format.

Update: This post has a follow-up — Minimax TTS API Update: let’s vibe a TypeScript SDK — covering eventsource-parser for spec-compliant SSE parsing, the new exclude_aggregated_audio option, and the minimax-speech-ts TypeScript SDK.

Introduction

Continuing from our series of integrating TTS (Text-to-Speech) API services from Google, Azure and AWS, the latest attempt in my TTS exploration is Minimax. Minimax is a relatively new company and unlike previous attempts, it is not a full blown cloud service provider, but rather a specialized API service for AI, video and audio related tasks.

Minimax provides three ways to interact with their TTS API: WebSocket, HTTP and MCP. Since I’m working on a server application, I’ll focus on the HTTP API. For the HTTP API, Minimax offers 2 modes: streaming and non-streaming. In this post I’ll cover both approaches, with the goal of implementing a complete streaming API with low latency and high performance.

Integrating Minimax TTS API

Unfortunately, Minimax doesn’t provide an official SDK, so we’ll use the HTTP API directly. The API is well documented, and we can use any HTTP client to interact with it. In this example, I’ll use the $fetch function from Nuxt 3, but you can use any HTTP client of your choice.

The Non-streaming API

To begin, we’ll implement the basic TTS API integration using the non-streaming API. This means we wait for the audio to be completely generated, then download the entire audio as a buffer. The implementation is simple and straightforward:

const response = await $fetch(
  `https://api.minimaxi.chat/v1/t2a_v2?GroupId=${minimaxGroupId}`,
  {
    method: "POST",
    headers: {
      Authorization: `Bearer ${minimaxAPIKey}`,
    },
    body: {
      text: "your text",
      model: "speech-02-hd",
      voice_setting: {
        voice_id: "Chinese (Mandarin)_Warm_Bestie",
        speed: 0.95,
        pitch: -1,
        emotion: "neutral",
      },
      language_boost: "Chinese,Yue",
    },
  }
);
const audioHex = response.data.audio;
const audioBuffer = Buffer.from(audioHex, "hex");

Here we’re calling the speech-02-hd model with a preference for Chinese,Yue. This is required for my case because Mandarin Chinese and Cantonese text often look very similar and the target spoken language cannot be reliably determined just by text. High quality Cantonese voice support is one of the unique strength of Minimax. Note that even if the voice is set to Chinese (Mandarin)_Warm_Bestie, the audio can still be generated in Cantonese, and any English inside the text will still be pronounced as English. This behaviour is similar to the Azure multilingual voices and is very convenient when handling multilingual texts.

Streaming the Non-streaming API

Even if the non-streaming API is not designed for streaming, we can still convert the complete received response into a stream. This allow user to start playing the audio as soon as our API start sending the response, rather than waiting for the entire audio to be downloaded in browser.

// continue from the previous code snippet
const audioBuffer = Buffer.from(audioHex, "hex");
const stream = Readable.from(audioBuffer);
return sendStream(event, stream);

Another optimization we could do is to receive the TTS API response as a stream, and then pipe it to the HTTP response. However we would be still bounded by the fact that the TTS API would not start responding until the entire audio is generated. A much better approach is to use the streamed version of the API.

Implement Real streaming using the streamed API

According to the Minimax documentation, the streamed API has the following response format:

//end
{
    "data":{
        "audio":"hex audio_chunk1 + hex audio_chunk2 + hex audio_chunk3",
        "status":2
    },
     "extra_info":{
      ...
    },
    "trace_id":"01b8bf9bb7433cc75c18eee6cfa8fe21",
    "base_resp":{
        "status_code":0,
        "status_msg":""
    }
}
// thrid chunk
{
    "data":{
        "audio":"hex audio_chunk3",
        "status":1,
    },
    "trace_id":"01b8bf9bb7433cc75c18eee6cfa8fe21",
    "base_resp":{
        "status_code":0,
        "status_msg":""
    }
}
//second chunk
{
    "data":{
        "audio":"hex audio_chunk2",
        "status":1,
    },
    "trace_id":"01b8bf9bb7433cc75c18eee6cfa8fe21",
    "base_resp":{
        "status_code":0,
        "status_msg":""
    }
}
//first chunk
{
    "data":{
        "audio":"hex audio_chunk1",
        "status":1,
    },
    "trace_id":"01b8bf9bb7433cc75c18eee6cfa8fe21",
    "base_resp":{
        "status_code":0,
        "status_msg":""
    }
}

At first glance, this response format is not trivial to parse. Due to the nature of network streaming, we cannot assume that a response will always be received in a single JSON chunk. This means we need to keep reading the stream until we receive a complete JSON object. Since we might receive multiple JSON objects in a single response, the parentheses matching of { and } must be checked when storing the response in a buffer. We also need to trim away any parts from the next JSON object that might be included in the current buffer to ensure JSON.parse() doesn’t throw an error.

The last chunk of the response is a summary block, which can be ignored for audio streaming purposes. It would be nice if the API allowed us to opt out of receiving the summary block, but since that’s not possible, we’ll simply drop it.

data: {"data":{"audio":"(hex)...... \n\n
data: ......(hex)","status":1,},"trace_id":"01b8bf9bb7433cc75c18eee6cfa8fe21","base_resp":{"status_code":0,"status_msg":""}}\n\n

Fortunately, by investigating the actual formatting and behavior of the streamed API response, the server always chunks its response starting with data: and ending with \n\n. This clearly indicates that the streamed API response is implemented as Server-Sent Events (SSE). This means we can make assumptions about parsing the streamed response, or even use a library that handles SSE for us. In this post, I’ll handle the response parsing manually, checking for data: and \n to extract the audio chunks.

Let’s implement the streaming API in our Nuxt 3 server:

const response = await $fetch<ReadableStream>(
  `https://api.minimaxi.chat/v1/t2a_v2?GroupId=${minimaxGroupId}`,
  {
    method: "POST",
    headers: {
      Authorization: `Bearer ${minimaxAPIKey}`,
    },
    responseType: "stream", // set this to receive a stream in `response`
    body: {
      stream: true, // ask for streaming response
      text: "your text",
      model: "speech-02-hd",
      voice_setting: {
        voice_id: "Chinese (Mandarin)_Warm_Bestie",
        speed: 0.95,
        pitch: -1,
        emotion: "neutral",
      },
      language_boost: "Chinese,Yue",
    },
  }
);

To convert the received hex into a stream, we need to read the stream and parse the response as it comes in. We’ll use TransformStream since fetch returns a ReadableStream when responseType is set to stream. TransformStream allows us to convert the incoming stream into a format suitable for sending to the client in a streamed format.

First, we need to convert the ReadableStream (which contains uint8 array chunks) into a string format that we can parse. This is straightforward with the TextDecoderStream API.

const decodedStream = response.pipeThrough(new TextDecoderStream()); // Now the stream is in string format

Next, we’ll create the actual TransformStream logic. We need to implement start, transform, and flush methods. The main logic will be in the transform method. Since we know a complete SSE event is separated by \n\n, we can use this to determine if we need to fetch more data or if we already have a complete event to process. Note that this assumes no \n\n is present in the audio data itself, which is a safe assumption for audio data in hex format.

const processStream = new TransformStream({
  start() {
    buffer = "";
  },
  transform(chunk, controller) {
    try {
      buffer += chunk;
      let eventEndIndex = buffer.indexOf("\n\n");
      while (eventEndIndex !== -1) {
        const event = buffer.substring(0, eventEndIndex).trim();
        buffer = buffer.substring(eventEndIndex + 2); // '\n\n' is 2 characters long
        if (event) {
          try {
            const audioBuffer = processEventData(event);
            if (audioBuffer) {
              controller.enqueue(audioBuffer);
            }
          } catch (error) {
            controller.error(
              (error as Error).message || "TTS_PROCESSING_ERROR"
            );
            return;
          }
        }
        eventEndIndex = buffer.indexOf("\n\n");
      }
    } catch (error) {
      controller.error(
        "TTS_PROCESSING_ERROR: Failed to process text-to-speech data"
      );
      return;
    }
  },
  flush(controller) {
    if (!buffer.trim()) {
      return;
    }
    try {
      const audioBuffer = processEventData(buffer);
      if (audioBuffer) {
        controller.enqueue(audioBuffer);
      }
    } catch (error) {
      // no op
    }
  },
});

In processEventData, we parse the actual event data, removing the SSE data: prefix and \n\n suffix. We extract the hex audio data into a Buffer and return it. The buffer is then sent to the client using controller.enqueue().

We also check for data.status === 1 to ensure the summary block at the end of the stream is ignored.

function processEventData(eventData: string) {
  const dataMatch = eventData.match(/^data:\s*(.+)$/m);
  if (!dataMatch) return null;

  const jsonStr = dataMatch[1].trim();
  if (!jsonStr) return null;

  const parsed: TTSChunk = JSON.parse(jsonStr); // this might throw if the JSON is malformed

  if (parsed.base_resp?.status_code !== 0) {
    throw createError({
      name: "TTS_API_ERROR",
      message: parsed.base_resp?.status_msg || "Unknown API error",
    });
  }

  if (parsed.data.status === 1 && parsed.data.audio) {
    return Buffer.from(parsed.data.audio, "hex");
  }

  return null;
}

Finally, we can pipe the transform stream to sendStream and send a binary audio stream to the client:

// continue from the previous code snippet
// decodedStream is a ReadableStream of strings
// processStream is a TransformStream that processes the SSE events
decodedStream.pipeThrough(processStream);
return sendStream(event, processStream.readable);

This approach involves some native assumptions about event formatting, but it works reasonably well as an integration example and MVP for this task.

Conclusion

In this post, we explored how to integrate the Minimax TTS API using both non-streaming and streaming approaches. The non-streaming method provides a straightforward way to get audio data, suitable for short texts or applications where latency isn’t a concern. However, for applications requiring real-time audio playback, the streaming approach, although more complex, offers a significant improvement in user experience by allowing audio to be played as it’s generated.

The streaming implementation demonstrates how to handle Server-Sent Events manually and convert hex audio chunks into a streamable format. This approach provides low latency and improved perceived performance, making it ideal for interactive applications where immediate audio feedback is important.

Additional Resources

Related TTS Integration Articles

• Convert Google text to speech API result to HTTP streamed response
• Convert Azure text to speech API result to HTTP streamed response
• Convert AWS Polly text to speech API result to HTTP streamed response
• Convert OpenAI API stream to HTTP streamed response (General streaming concepts)

Documentation

• Minimax TTS API Documentation
• Server-Sent Events - MDN
• ReadableStream - MDN
• TransformStream - MDN
• TextDecoderStream - MDN

In previous articles, I’ve explored streaming implementations for Google’s Text-to-Speech API and Azure’s Text-to-Speech service. Continuing this series, let’s see how to implement streaming with AWS Polly, Amazon’s text-to-speech service.

Spoiler: It’s trivial.

AWS Polly Javascript SDK

The command we would be using is SynthesizeSpeechCommand from the AWS SDK for JavaScript v3. The response from the SynthesizeSpeechCommand already includes an AudioStream. AudioStream has methods transformToByteArray, transformToString, and transformToWebStream, which transform the audio stream into different formats. For HTTP streaming, we can just use transformToWebStream() to convert the AWS SDK audio stream into a web-compatible stream.

Example code

The following code convert a text input to an ogg audio stream using AWS Polly’s SynthesizeSpeechCommand.

import type {
  LanguageCode,
  VoiceId,
} from '@aws-sdk/client-polly'
import {
  PollyClient,
  SynthesizeSpeechCommand,
} from '@aws-sdk/client-polly'

const client = new PollyClient({
  region: awsRegion,
  credentials: {
    accessKeyId: awsAccessKeyId,
    secretAccessKey: awsAccessKeySecret,
  },
})

...

const command = new SynthesizeSpeechCommand({
  Text: text,
  OutputFormat: 'ogg_vorbis',
  VoiceId: 'Ruth' as VoiceId,
  LanguageCode: 'en-US' as LanguageCode,
  Engine: 'neural',
  TextType: 'text',
})
const response = await client.send(command)
if (!response.AudioStream) {
  throw createError({
    status: 500,
    message: 'SPEECH_SYNTHESIS_FAILED',
  })
}
const stream = response.AudioStream.transformToWebStream()
setHeader(event, 'content-type', 'audio/ogg; codecs=opus')
setHeader(event, 'cache-control', 'public, max-age=3600')
return sendStream(event, stream)

Conclusion

Since AWS SDK already provides a stream based response and useful helper methods to convert the audio stream, implementing HTTP streaming for AWS Polly is straightforward. This allows you to reduce the delay between the request and the audio playback, enhancing the overall user experience with real-time audio playback.

Background

In the year 2025, many of us are still using paper ballots and stamps to vote, still waiting for days for the tallying results, and even weeks for a final outcome if disputes trigger a recount. One would have thought technology would have solved this problem by now. However, e-voting remains a controversial topic and faces significant resistance. The reasons for opposition aren’t merely rooted in tradition or politics; many concerns relate to legitimate issues of security, integrity and privacy. Are we destined to count paper ballots for important decisions indefinitely?

A few years ago, I worked on a project implementing a secure e-voting system on blockchain. It was surprising to me that even in the web3 space—where trustless systems, public verifiability, zero knowledge and governance are all highly valued—e-voting remains a niche topic that has received limited attention and resources. In the end, I wasn’t able to complete a fully functional MVP. Here I would like to share some concepts and ideas I’ve learned, hoping they might benefit others interested in this field.

In this series of articles, we will explore the reasons why e-voting is not widely adopted, and how blockchain and zero-knowledge proof technologies can help address some of these challenges.

Why (not) e-voting?

E-voting systems come in many forms, but they can primarily be divided into two categories: online voting and electronic voting machines (EVMs). Online voting enables citizens to cast their ballots via the internet, while EVMs are physical devices used for on-site electronic voting.

In the following sections, we’ll discuss the advantages and disadvantages of e-voting. Some points apply to both categories, while others are specific to one form.

Advantages of e-voting

• Speed & Efficiency: E-voting can provide faster results, as the counting process is automated and can even be done in real-time.

• Reduce human error: Counting errors can be completely eliminated with e-voting, and recounting is never needed. Depending on the system design, errors in voter eligibility verification and even double voting can also be prevented.

• Accessibility: E-voting machines and online voting platforms can be designed with accessibility in mind, featuring audio transcription capabilities and adjustable fonts. In traditional voting, people with disabilities often must rely on poll workers for assistance.

• Cost-effective: E-voting reduces costs associated with printing, mailing, and labor for counting and verifying votes.

• Convenience: Remote voting allows citizens to vote from anywhere, which is especially beneficial for people overseas or with mobility challenges. Even for on-site voting, e-voting machines can reduce waiting times.

Disadvantages of e-voting

• Security: E-voting systems must operate on servers connected to the internet, making them vulnerable to hacking and cyber attacks. For example, a hacker could alter vote counts or manipulate election results.

• Integrity: E-voting websites and hardware can be susceptible to supply chain attacks and tampering. For instance, a hacker could modify the user interface of a voting machine to mislead voters into selecting the wrong candidate or even change the vote count after the election.

• Privacy: Ensuring votes are cast in private and that voter identities remain confidential is crucial. However, in the context of e-voting, it is challenging to guarantee that votes leave no digital fingerprints and are truly anonymous, as digital authentication methods are required to verify voter identities. Data breaches could expose voter identities and even the content of their votes.

• Technical issues: Software bugs and hardware glitches can lead to errors in the voting process, such as miscounted votes or system crashes. Additionally, internet connectivity problems or hardware failures could prevent voters from casting their ballots.

• Transparency: While paper ballots can be physically counted, audited, and observed by third parties, e-voting systems are often closed-source and proprietary, making it difficult to verify the integrity of such black-box systems. Even open-source or audited systems face challenges in ensuring that the deployed system remains un-tampered and un-compromised.

What would an ideal e-voting system look like?

An ideal e-voting system should preserve the advantages of e-voting while mitigating most of the disadvantages. However, it is often impossible to eliminate all drawbacks, and design trade-offs must be made based on specific use cases.

To design a practical e-voting system that would gain widespread acceptance, let’s take a step back and analyze the characteristics of an ideal voting system.

• Public verifiability: The system should allow third parties to verify the following information:

  • Number of eligible voters
  • The start, end, and duration of the voting period
  • The number of votes casted
  • The number of votes deemed invalid, if any
  • The tallying results

  This ensures that the entire voting process and results can be audited and verified by any third party, not just by a trusted authority.

• Individual verifiability: The system should allow voters to verify the following information. Whether this information can also be verified by third parties depends on design and implementation constraints:

  • If one is eligible to vote
  • If one’s vote is casted
  • If one’s vote is counted

  This ensures that voters can confirm their votes are included in the tally.

• Anonymity: The system should ensure the following information cannot be proven or obtained by anyone, including the voting authority:

  • One cannot know the option for a just-cast vote until tallying
  • Any voter’s vote cannot be linked to their identity
  • No voter can prove they voted for a specific candidate

  These concepts are often referred to as ballot secrecy and receipt freeness. This ensures voters cannot be coerced or bribed to vote for a specific candidate (vote buying) and that their votes remain truly private.

• High availability: The system should possess the following properties to ensure voters can cast their ballots during the voting period:

  • Capacity to handle the expected number of voters
  • Resistance to DDOS attacks
  • No single point of failure
  • Disaster recovery plan

How can blockchain and zero-knowledge proof help?

Looking at the above requirements, designing a system that satisfies all of them is highly challenging. How can one make something publicly verifiable while ensuring parts of it remain secret? How can secrecy be maintained in such a system if the secret holder might be incentivized to reveal their voting choice? Many existing systems—e-voting or otherwise—rely on trusting some form of authority to resolve these conflicts and act as the primary entity ensuring the system’s privacy and integrity. This reliance often becomes the source of fraud and disputes, as such authorities can be bribed, compromised, or even have their own incentives to commit fraud.

This is where blockchain and zero-knowledge proof come into play. Blockchain is designed as a publicly verifiable ledger that eliminates the need for trusted authorities. It is inherently highly available and tamper-proof. Zero-knowledge techniques, often used on top of blockchain, provide public verifiability of private information. These two technologies can be combined to create a system that is both publicly verifiable and private, while also ensuring the system’s integrity and availability.

In the next article, we will discuss some technical and mathematical concepts related to these technologies and how they can be used to design a practical next-generation e-voting system.

Background

Although I’m not a Google Cloud Platform or Google Analytics expert, I often encounter strange issues that neither ChatGPT nor StackOverflow can resolve. This is one of those cases.

We were working on an HTML widget designed to be embedded in partner websites as an <iframe>. Google Analytics 4 (GA4) tracking was added to the widget, but we noticed that the number of logged page view events was significantly lower than expected.

Initially, we suspected that ad-blockers were causing the problem. However, after comparing the GA4 data from the partner’s site, we ruled this out. The page views on the parent site were normal and much higher than those logged from the iframe.

It is worth mentioning that this issue is not about triggering iframe GA4 events from a cross-origin parent site, which can be resolved using postMessage.

Troubleshooting the Issue in Production

Initial Checks

To eliminate implementation issues, we first confirmed that GA4 events were being sent correctly when the widget was accessed directly (not embedded). The simplest way to do this was to open the widget’s src URL directly, and check the network tab in the browser’s developer console. By filtering network requests using the GA4’s tag ID, we could easily inspect outgoing events from gtag.

Here, we see two requests: one for loading the gtag JavaScript and another for sending the page_view event. This confirms that GA4 events are triggered correctly when the widget is opened directly.

Next, we checked what happens when the widget was embedded in an iframe. By inspecting the iframe element on the parent site, we observed the following:

The gtag.js script was loaded correctly, but the page_view event was not being sent!

Manually Triggering GA4 Events from the iFrame

To determine whether the issue was related to JavaScript, we attempted to manually trigger a page_view event. By default, gtag automatically sends a page_view event upon initialization. To test if the automated event was being blocked, we manually triggered it with the following code:

console.log(gtag); // Sanity check to ensure gtag is loaded
gtag("event", "page_view");

First, we tested this in the non-embedded version of the widget:

As expected, after a few seconds, a new network call is triggered, indicating that the page_view event was sent successfully. The few seconds delay is due to the default GA4 event batching.

Let’s try it in the embedded version. Note that we can select the iframe in the console to run the code within the iframe context.

Nothing happened! The event was not sent, indicating that something was blocking it.

Using the GA Debugger

To dig deeper, we used the Google Analytics Debugger Chrome extension, which logs all GA events to the console. After installing and enabling the extension, we refreshed the page.

For sanity checks, we ran it on the non-embedded version first:

The event was triggered successfully, and its details were logged in the console.

Next, we tested the embedded version:

The event details were missing, and we saw two warnings in the console:

• Unable to update session cookie.
• Unable to set cookie.

It became clear that these cookie-related warnings were preventing the event from being sent.

Cookies in Cross-Origin iFrames and SameSite

The root cause of the problem was Chromium’s changes to the SameSite cookie attribute. SameSite controls whether cookies should be sent in cross-site requests. In this case, since the iframe’s domain differed from the parent site’s domain, cookies were blocked by default.

To mitigate this issue, the SameSite=None attribute must be used to allow cross-origin cookies.

Fixing the Cookie Issue

To ensure proper cookie settings for gtag and GA4, the following configuration flags should be added during gtag initialization:

gtag("config", "GA_MEASUREMENT_ID", {
  cookie_flags: "SameSite=None; Secure",
  cookie_update: false,
});

Here’s the result after implementing these flags:

With this fix, the page_view events were successfully sent from the iframe.

Conclusion

While I did knew of the SameSite cookie changes, I didn’t expect them to affect simple GA4 page_view events from cross-origin iframes that don’t interact with their parent site. It would be helpful if GA4 provided better fallback mechanisms for events when cookies are blocked—or at the very least, documented this behavior more clearly.

This raises an interesting question: are advertising agencies that embed ads in iframes aware of this issue? I’m not sure. Hopefully, this post won’t degrade everyone’s privacy by revealing this workaround, but it might help others struggling with similar problems.

Background

In the previous post, we covered the initial steps to set up Vertex AI Search for Commerce (yes, they have changed their product name again, from Retail to Commerce). For some use cases, importing an existing product catalog and user data may be sufficient to build the data model.

However, if you lack enough existing data to train the model or need fresh data to confirm that your model is functioning as expected, you must feed user events to the Vertex AI Search service as they occur on your website. In this post, we will discuss how to integrate real-time event collection into your site.

Collecting Real-time Events

Google provides a documentation page specifically on how to record user events. Generally, there are three methods: Google Tag Manager (GTM), API, and JavaScript Pixel. You can find more details here.

A list of recommended user events and their payloads can be found here.

If you are already using GA4 events, some of these can automatically convert to retail user events, as detailed here.

However, some retail user event types do not have a direct equivalent in GA4, which means you will need to either ignore those event types or send them manually through another method.

The Easy Way - GTM & GA4

If you already have GTM set up on your website, this should be the simplest method. The GTM console includes templates like Variable - Ecommerce and Variable - Cloud Retail. Refer to the GTM documentation for straightforward implementation.

Unfortunately, we do not have GTM set up on our website due to complications with running it alongside Content Security Policy (CSP), so I will skip this part.

API - For Those with GA4 or Concerned About Ad Blockers

If you are familiar with implementing server-side events (like in Meta’s Conversions API), you can manually build your JSON payload and use the userEvents.write API to send events to the Vertex AI Search service. This ensures that events are sent even if the user has ad blockers or has disabled JavaScript, provided you have the appropriate server-side API related to the event.

Additionally, if you are already using Google Analytics 4 (GA4), you can use a prebuilt_rule called ga4_bq to send a GA4 event payload directly to the userEvents.write API. While this is mentioned in the documentation, I’m unsure in what scenarios this would be applicable since you cannot directly retrieve a GA4 event payload from gtm.js. Please inspire me if you have ideas!

JavaScript Pixel Tracking

For me, the most suitable method appears to be using the JavaScript Pixel. Similar to GA4 or other pixel tracking services, I would simply include a script and call functions when a user event occurs.

The documentation provides a complete JavaScript example:

var user_event = {
  eventType: "detail-page-view",
  visitorId: "visitor-id",
  userInfo: {
    userId: "user-id",
  },
  attributionToken: "attribution-token",
  experimentIds: "experiment-id",
  productDetails: [
    {
      product: { id: "123" },
    },
  ],
};

var _gre = _gre || [];
// Credentials for project.
_gre.push(["apiKey", "api-key"]);
_gre.push(["logEvent", user_event]);
_gre.push(["projectId", "project-id"]);
_gre.push(["locationId", "global"]);
_gre.push(["catalogId", "default_catalog"]);

(function () {
  var gre = document.createElement("script");
  gre.type = "text/javascript";
  gre.async = true;
  gre.src = "https://www.gstatic.com/retail/v2_event.js";
  var s = document.getElementsByTagName("script")[0];
  s.parentNode.insertBefore(gre, s);
})();

While this seems straightforward, I have some bad feeling looking at it. The example covers the code for configuration, event formatting, and script loading, but why are the script tags created after all the event formatting and function calls? What function from window._gre should I invoke after the script has loaded?

Retail Pixel - It Doesn’t Work as Expected

From the example code above, it seems that the retail pixel behaves like GTM or any other pixel tracking service. My expectations were:

A global variable under window is created as an array, in this case, _gre. The array is used to store configuration and function calls before the script loads. The actual script loads asynchronously. After the script loads, it should read the _gre array and execute any configuration or function calls stored within. Finally, the _gre should be replaced by a new object with the same name, implementing functions that allow interaction with the retail API. However, parts of this process are not shown in the example code, particularly what to do after the script is loaded. One might expect to either call a new function like _gre.logEvent, or continue using _gre.push to send user events via the pixel. Unfortunately, such functionality does not exist, and the example code appears to encompass the entire functionality of the retail pixel.

Configuration Values Are Not Stored in Pixel

Typically, one would expect configuration values like:

_gre.push(['apiKey', 'api-key']);
_gre.push(['projectId', 'project-id']);
_gre.push(['locationId', 'global']);
_gre.push(['catalogId', 'default_catalog']);

to be stored in the pixel object. Any subsequent calls to logEvent (or its _gre.push equivalent) would then be able to read these values. However, these values are not retained in the pixel object, meaning they must be provided each time a user event is sent.

_gre.push Is Not Hooked by the Script

User events are sent by invoking _gre.push(['logEvent', user_event]). This works if _gre.push is called before the pixel JavaScript is loaded. However, if it is called after the script has loaded, it will not function, as the _gre.push function is not hooked after the script is loaded. In GTM, pushing something into the data layer should trigger GTM events because it hooks into the .push function of data layer variables. Unfortunately, this is not the case for the retail pixel; you must manually trigger another function to send the user event after the script is loaded.

_gre.logEvent Is Not a Function

Continuing from the previous point, if .push does not work, what would the trigger function be? It appears that logEvent is a function name that should be called. However, it is not a function, and calling _gre.logEvent results in an error.

Simple Analysis of the Retail Pixel

To resolve these mysteries, one must examine the source code of the retail pixel.

It turns out the function is indeed called logEvent, but it is encapsulated within a new variable called cloud_retail. This cloud_retail.logEvent function takes the entire _gre as a parameter, and is only invoked once after the script has loaded!

To send a user event after the script is loaded, you must call _gre.push(['logEvent', user_event]) and then call cloud_retail.logEvent(_gre) after the script is loaded.

However, note that the _gre array is not cleared after the script loads, so if there are existing events in _gre before the script loads, they will also be sent if you call logEvent(_gre) again, resulting in duplicate events. For cloud_retail.logEvent(_gre) to function correctly, you must clear _gre after invoking it to prevent this.

Another issue arises: _gre or the pixel does not store configuration values like project ID or catalog ID, meaning these values must be passed again to _gre.

Thus, to log events using _gre and cloud_retail.logEvent naively, the code would look like this:

_gre = [];
_gre.push(["apiKey", "api-key"]);
_gre.push(["projectId", "project-id"]);
_gre.push(["locationId", "global"]);
_gre.push(["catalogId", "default_catalog"]);
_gre.push(["logEvent", user_event]);
cloud_retail.logEvent(_gre);

Hack, or “actual implementation”

A more practical solution would be to just forget _gre, store these config values in a separate object, and pass it directly to cloud_retail.logEvent every time an event is sent.

function logEvent(user_event) {
  // get api-key, project-id, global and default_catalo from some where
  cloud_retail.logEvent([
    ["apiKey", "api-key"],
    ["projectId", "project-id"],
    ["locationId", "global"],
    ["catalogId", "default_catalog"],
    ["logEvent", user_event],
  ]);
}

Consequently, I developed this class for my Nuxt project:

class GoogleRetailPixel {
  userId;
  visitorId;
  apiKey;
  projectId;

  constructor(apiKey, projectId) {
    this.apiKey = apiKey;
    this.projectId = projectId;
    window._gre = window._gre || [];
    window._gre.push(["apiKey", apiKey]);
    window._gre.push(["projectId", projectId]);
    window._gre.push(["locationId", "global"]);
    window._gre.push(["catalogId", "default_catalog"]);
  }

  setUserId(userId) {
    this.userId = userId;
  }

  setVisitorId(visitorId) {
    this.visitorId = visitorId;
  }

  logEvent(eventType, payload = {}, { attributionToken, experimentIds } = {}) {
    if (!this.visitorId) {
      return;
    }
    const event = {
      eventType,
      attributionToken,
      experimentIds,
      visitorId: this.visitorId,
      userInfo: this.userId
        ? {
            userId: this.userId,
          }
        : undefined,
      ...payload,
    };
    // HACK: cloud_retail does not replace _gre on init
    // cloud_retail only calls logEvent once on _gre and
    // it does not even clear _gre after that
    if (window.cloud_retail) {
      window.cloud_retail.logEvent([
        ["apiKey", this.apiKey],
        ["projectId", this.projectId],
        ["locationId", "global"],
        ["catalogId", "default_catalog"],
        ["logEvent", event],
      ]);
    } else {
      window._gre.push(["logEvent", event]);
    }
  }
}

With this implementation, I push events into _gre if the cloud_retail object is not yet loaded, and call cloud_retail.logEvent if it is. This prevents duplicate events from being sent while allowing the asynchronous loading of the retail pixel.

I have also wrapped this functionality as a Nuxt 2 library, just in case I need to use it in other projects.

Since the project requiring this is still running Nuxt 2, I haven’t written a Nuxt 3 version yet, but it should be straightforward and may happen if we migrate to Nuxt 3.

Confirming Event Transmission

To confirm that the events are being sent, you can check the User Events Integration page in the Vertex AI Search console, specifically within the Event Tab on the Data Page. This section displays real-time event counts from the specified date to the present. You can also monitor the percentage of unjoined events to ensure you are sending correct product IDs or to determine if the product catalog needs updating or importing.

Conclusion

It seems that using the retail pixel is not as straightforward as I initially thought. The pixel has limited functionality and does not appear as production-ready as other Google products or similar services named “pixel”. However, with a few tweaks and hacks, it can still function as expected.

Now that we have real-time events being sent to the Vertex AI Search service, I hope that my model requirements will be met, allowing me to start training my model. However, even if I meet the data requirements stated on the model training page, the training could still fail midway due to insufficient data.

As this issue is challenging for me to debug as a Google Cloud user, I will likely wait to see if more data collected over time resolves the problem or if I can find the time to contact someone from Google Cloud to investigate. Stay tuned for any new posts in case I have further findings.

Background

In one of our e-commerce project, a very useful feature we always wanted to have personalized item recommendations for users. Since we don’t have a dedicated data scientist, we don’t have the resource to home-bake our own model, and was looking for a suitable managed cloud service for this.

We looked into Amazon Personalize, which seems easy and promising, but unfortunately, we didn’t have time to set up a new data pipeline just for it, and couldn’t even try the setting up models.

Recently, I came across Google Cloud’s Vertex AI Search for Commerce, which seems to have seamless integration with Google Merchant Center and Google Analytics(GA4). This lowers integration costs, so I decided to give it a try. Turns out, it is not that easy.

Importing Historical Data

To train a recommendation model in Vertex AI Search, we need data, including products and user events. User events must include proper IDs for products so that the model can learn the relationship between products. Events that contain invalid product IDs are called unjoined events, and will be ignored by the model.

Importing Product Catalog

There are a few ways to import product catalogs into Vertex AI Search. Here, we will cover two of them. Note that there are different limitations for each import method.

Importing Product Catalog from Google Merchant Center

If you already have Google Merchant Center set up either for shopping ads or Google Ads, you can easily import the product catalog from there. This is the easiest way to import a product catalog, especially when you have product structured data already set up in your e-commerce site. Google Merchant Center will fetch all the products automatically from your website without any additional import procedure.

Sadly, the last time I used Google Merchant Center, all my products were disapproved since they were considered as unsupported shopping content. After a while, they were completely removed from the product list even when I don’t want shopping ad, and wouldn’t reappear somehow. So, I can’t use this method.

Importing Product Catalog via API

As a developer, importing via API is the most flexible way to import a product catalog. The product schema is defined as follows:

    const { data } = await axios.post('https://retail.googleapis.com/v2/projects/${your-project-number}/locations/global/catalogs/default_catalog/branches/0/products:import', {
      "inputConfig": {
        "productInlineSource": {
          "products": [
            %{your products}
          ],
        }
      }
    }, {
      headers: {
        'Authorization ': `Bearer $(gcloud auth print-access-token)`,
      },
    });

To get your project number, use the following command:

gcloud projects list \
--filter="$(gcloud config get-value project)" \
--format="value(PROJECT_NUMBER)"

However, there is one extra thing to add to make the API work. If you encounter the following error:

{
  "error": {
    "code": 403,
    "message": "Your application is authenticating by using local Application Default Credentials. The retail.googleapis.com API requires a quota project, which is not set by default. To learn how to set your quota project, see https://cloud.google.com/docs/authentication/adc-troubleshooting/user-creds .",
    "status": "PERMISSION_DENIED"
  }
}

Then you need to add the following header to your request:

  headers: {
    'x-goog-user-project': ${your-project-id},
    ...
  }

Importing User Events

Like the product catalog, there are a few ways to import user events into Vertex AI Search. We will only cover Google Analytics(GA4) data import here since it requires the least effort for sites already set up with GA4

Importing GA4 Data from BigQuery

Before we can import Google Analytics 4(GA4) events into Vertex AI Search, we need to have the data in BigQuery. Follow the guide to set up BigQuery export to GA4. Normally, GA4 events are exported daily to a dataset named analytics_123456789.events_20241116 where 123456789 is your GA4 property ID and 20241116 is the date partition of the export.

Once the GA4 data is in BigQuery, we can import the table using the Vertex AI Search console. However, the web UI console can only import one table at a time. Since GA4 exports are partitioned by date, it would be tedious to import them one by one.

One simple solution is to merge all the tables into one table and import them. However, this is only feasible if the size of historical data is small. The sample SQL is as follows:

CREATE TABLE `analytics_123456789.combined_events` AS
SELECT * FROM `analytics_123456789.event_*` WHERE _PARTITIONTIME BETWEEN '2023-03-01' AND '2023-03-31'

GA4 Event Mapping

Many user events required in Vertex AI have a direct mapping to GA4 events, especially e-commerce events.

Search-related events are more tricky, but as long as view_list is set up properly with search_term param set, it should be fine. Another way is to use view_search_results, which is an automated event if you have enabled GA4’s enhanced measurement. However, this requires the search term to be in the URL query string with predefined keys.

What about home-page-view?

There is no GA4 event that can directly map to the retail user event home-page-view. During import, page_view with a path of / is automatically used as a substitute. This is not ideal if your homepage is not at /. For example, if your homepage has multiple locales, the home page might have paths like /en and /zh.

To correctly import these events, we would have to query for these events:

CREATE TABLE `analytics_123456789.ga_homepage` (
  eventType STRING,
  visitorId STRING,
  userId STRING,
  eventTime STRING
);

INSERT INTO `analytics_123456789.ga_homepage` (eventType, visitorId, userId, eventTime)
SELECT
  "home-page-view" as eventType,
  user_pseudo_id as visitorId,
  user_id as userId,
  CAST(FORMAT_TIMESTAMP("%Y-%m-%dT%H:%M:%SZ",timestamp_seconds(CAST ((event_timestamp/1000000) as int64))) as STRING) AS eventTime
FROM `analytics_123456789.CREATE TABLE `analytics_123456789.combined_events` AS
` where event_name = 'page_view' AND `event_params`[SAFE_OFFSET(0)].`key` = 'page_path' and (`event_params`[SAFE_OFFSET(0)].`value`.`string_value` = '/zh-Hant' or `event_params`[SAFE_OFFSET(0)].`value`.`string_value` = '/en')

Import ordering of event and product catalog matters!

Note that if you import events before the product catalog, events will still be unjoined even if you filled in the correct product IDs in product catalog import. This is because product IDs are joined when ingesting user events, not vice versa.

In this case, you would have to trigger a user event rejoin job for the historical events to be joined according to the new catalog.

curl -X POST \
    -H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
    -H "Content-Type: application/json; charset=utf-8" \
    --data "{
     'userEventRejoinScope': 'UNJOINED_EVENTS'
     }" \
    "https://retail.googleapis.com/v2/projects/${your-project-nubmer}/locations/global/catalogs/default_catalog/userEvents:rejoin"

Testing out the model

Once the data is imported, we can start training the model. There are different models, each suited for different use cases, e.g. “Recommended for you”, “Others you may like”, “Frequently bought together”, etc. Each has a minimum requirement for the data. The console will not allow training of the model if the data requirement is not met.

However, even if the requirements are met, the training model can still fail with an INSUFFICIENT_TRAINING_DATA error. The message isn’t very helpful, but it likely relates to the quality of the training data. For instance, poor data quality or a high unjoined event rate could be the issue.

Trying the Similar Product model

Luckily, the “similar product” model only requires the product catalog to be imported. To start training the model, go to the Model tab of the Vertex AI Search console and create a “similar product” model. The training will take a while.

After the model is finished, we need to create a serving config to use this model. Create one in the “Serving Configs” tab. There are some configurations in serving config that we can tweak, but the default config should be good enough for testing.

After the serving config is created, we can go to the “Evaluate” tab to test the model. Select the serving config we just created and pick a product ID as input. The model should return a list of other similar products.

Conclusion

Since my historical GA4 events do not meet the model requirement, I could not try the recommendation models that I am interested in. To improve the data quality, I will be implementing methods to collect real-time user data. In the next post, we will cover how to collect real-time user data.

Background

Setting up an HTTP proxy in Go is straightforward with the built-in NewSingleHostReverseProxy. When used with Gin as middleware, the code looks like this:

router := gin.New()
proxy := httputil.NewSingleHostReverseProxy(lcdURL)
proxyHandler := func(c *gin.Context) {
  proxy.ServeHTTP(c.Writer, c.Request)
}
router.Use(proxyHandler)

But what if we want to modify the proxied response before sending it back to the client? For example, we might want to hide PII before serving internal data to third parties. For simplicity, let’s assume the body is in JSON format.

Built-in Functions vs. Middleware

Sadly, NewSingleHostReverseProxy doesn’t directly support response modification. However, the ReverseProxy instance it returns allows us to use the method ModifyResponse, which allows us to define a function to modify the response:

proxy.ModifyResponse = func(r *http.Response) error {
  b, err := io.ReadAll(r.Body)
  if err != nil {
    return err
  }
  defer r.Body.Close()

  var jsonObject map[string]interface{}
  err = json.Unmarshal(b, &jsonObject)
  if err != nil {
    return err
  }

  // Modify jsonObject here

  newBody, err := json.Marshal(jsonObject)
  if err != nil {
    return err
  }

  r.Body = io.NopCloser(bytes.NewReader(newBody))
  return nil
}

However, if you’re running the proxy alongside other API routes, you might want a unified way to modify all responses. Writing the rewrite logic as Gin middleware can achieve this.

func filterJsonBody() gin.HandlerFunc {
  return func(c *gin.Context) {
    // Create our own writer
    wb := &copyWriter{
      body:           &bytes.Buffer{},
      ResponseWriter: c.Writer,
    }

    // Inject it into gin context
    c.Writer = wb

    // Call the next handler
    c.Next()

    // Handle response modification at the end of handler chain
    originBodyBytes := wb.body.Bytes()

    var jsonObject map[string]interface{}
    err := json.Unmarshal(originBodyBytes, &jsonObject)
    if err != nil {
      c.AbortWithError(http.StatusInternalServerError, err)
      return
    }

    // Modify jsonObject here

    newBody, err := json.Marshal(jsonObject)
    if err != nil {
      c.AbortWithError(http.StatusInternalServerError, err)
      return
    }

    wb.ResponseWriter.Write(newBody)
  }
}

type copyWriter struct {
  gin.ResponseWriter
  body *bytes.Buffer
}

func (cw *copyWriter) Write(b []byte) (int, error) {
  return cw.body.Write(b)
}

Add it at the beginning of the router handler chain, so that our copyWriter instance would replace the Writer in the Gin context for all routes.

router := gin.New()
router.Use(filterJsonBody())

// Other routes
router.Use(proxyHandler)

HTTP Error?

If you add the middleware as shown and then use curl to test the API, you may encounter errors like:

• HTTP/2 stream 1 was not closed cleanly: INTERNAL_ERROR (err 2)
• curl: (18) transfer closed with x bytes remaining to read

Which of these would show up depends on your infrastructure setup. For example, cloud hosting load balancers would likely be in HTTP/2. The HTTP/2 error can be cryptic, but the curl error hints at a Content-Length header mismatch. One way to confirm the issue is to make curl show the HTTP headers by using curl -v. Indeed, the Content-Length header is set as the original body length, instead of the modified one.

Fixing the Content-Length Header in ModifyResponse Approach

Fixing this issue in ModifyResponse is straightforward; we just need to set the Content-Length header to the new body length:

r.ContentLength = int64(len(newBody))
r.Header.Set("Content-Length", strconv.Itoa(len(newBody)))

Fixing the Content-Length Header in Gin Middleware Approach

In the middleware approach, ideally, we would also rewrite the Content-Length header with the new correct length. However, in the method we would override here, WriteHeader, we can’t directly access the http.Response object. Instead, a workaround is to use Transfer-Encoding: chunked to signal that the response body is sent in chunks. This way, we don’t need to set the Content-Length header at all:

func (cw *copyWriter) WriteHeader(statusCode int) {
  cw.Header().Del("Content-Length")
  cw.Header().Set("Transfer-Encoding", "chunked")
  cw.ResponseWriter.WriteHeader(statusCode)
}

One downside of using chunked encoding is that some HTTP clients and proxies may not cache them properly. Since I use Nginx in my setup, which caches small chunked responses, this tradeoff is acceptable for my use case.

Conclusion

Setting up a reverse proxy in Go is easy; the ability to modify the response allows even more flexibility in use cases. By using Gin middleware, we easily apply a unified modification logic to all routes. However, be aware of the Content-Length header issue when modifying the response body, and choose the appropriate solution wisely based on your setup.

Background

Stripe Checkout and Google Analytics (GA4) are powerful and commonly used tools in e-commerce websites. Stripe Checkout allows you to collect payments with minimal code and a prebuilt UI, while GA helps analyze user behavior to improve sales.

Issue: 0 purchase events in GA funnel

However, I recently encountered a significant issue where GA’s purchase journey dashboard couldn’t properly plot the purchase funnel analysis when using Stripe Checkout. The number of events always dropped to 0 after the “begin checkout” step. Interestingly, the “Ecommerce Purchases” dashboard showed that the number of purchase events wasn’t actually 0. So, there must be something wrong with the purchase journey funnel.

GA Sessions, Stripe Checkout and third party domains

In Google Analytics 4, events are attributed to different sessions. Users receive a unique ID cookie when they land on a GA-enabled site, and events fired under the same ID count towards the same session. A new session is created either when a new ID is received or when a time limit has passed since the last event from a particular ID.

Learn more about sessions

To understand the issue with the purchase journey funnel, we need to consider the nature of Stripe Checkout. In Stripe checkout, there is an additional redirect step after the checkout begins. By default, Stripe checkout is a hosted page under the domain checkout.stripe.com. Only after users complete or cancel the checkout process are they redirected back to the original site. This redirect process seems to change the session ID, which breaks the purchase journey funnel.

Why does the session ID change after redirection? One possible reason is the third-party cookie constraints imposed by modern browsers. Browsers now alter cookie behaviors for third-party domains to protect user privacy, such as limiting their lifetime or adding extra per-origin isolations. Since GA cookies are sent under the domain analytics.google.com and not under the user’s application domain, they are affected by these restrictions. The redirection to and from the Stripe Checkout page likely resets the GA cookie, creating a new session ID. As a result, GA thinks the final purchase step of the purchase journey funnel occurs under a different user session, leading to the correct event count but a broken funnel.

Fixing session with cross-domain measurement

To mitigate this issue and attribute the event to the correct session (thus fixing the purchase journey funnel), we need to inform GA explicitly that users redirected back from the Stripe Checkout page already have an existing session ID. GA4 actually has an automated solution for this issue called “cross-domain measurement.” It automatically adds a query string _gl when a user clicks on a URL under any domain that you also own. This _gl query string is a unique linker ID that allows the gtag script in the receiving domain to identify existing users and sessions without relying on the presence of the same cookie ID.

You can learn more about how cross-domain measurement works and configure a list of domains you own under “Configure your domains” in your “Google tag data stream settings.”

Learn more about cross-domain measurement

However, this automated solution does not work in the case of Stripe Checkout. We don’t own or control the gtag script under checkout.stripe.com, so it won’t utilize the _gl query string we send or send back the user with the proper _gl query string when they are redirected back to our own site.

Making cross-domain measurement work manually

Nevertheless, by understanding how to mitigate cross-domain issues, we can implement a manual link for session IDs.

Before sending the user to the Stripe Checkout page, collect the GA client ID and session ID using the gtag query. The following code snippet is an example under a Nuxt.js app, using the Vue.$gtag syntax:

  Vue.$gtag.query('get', process.env.GA_TRACKING_ID, 'client_id', id => {
    store.dispatch('setGaClientId', id);
  });
  Vue.$gtag.query('get', process.env.GA_TRACKING_ID, 'session_id', id => {
    store.dispatch('setGaSessionId', id);
  });

When creating the Stripe Checkout session, include the ga_client_id and ga_session_id respectively in the success_url and cancel_url:

const checkoutPayload: Stripe.Checkout.SessionCreateParams = {
  mode: 'payment',
  success_url: `${successUrl}?ga_client_id=${gaClientId}&ga_session_id=${gaSessionId}`,
  cancel_url: `${cancelUrl}?ga_client_id=${gaClientId}&ga_session_id=${gaSessionId}`,
  ...
}

When initializing GA/gtag on our own site, check for the query string parameters ga_client_id and ga_session_id. If these values exist, we can assume that the user was redirected from the checkout flow and restore their client_id and session_id accordingly:

if (query.ga_client_id && query.ga_session_id) {
  Vue.$gtag.config({
    client_id: query.ga_client_id,
    session_id: query.ga_session_id,
  });
}

This fixes the issue of lost sessions and restores the normal functioning of the purchase journey funnel.

Another solution: Server side event recording

The server-side event recording solution is mentioned in the Stripe official documentation. It allows you to fire events directly from the server side. This approach involves sending the client ID of the checkout user to the Stripe Checkout session, which is then stored in the checkout metadata. Once the payment is successfully completed, the server can directly fire the purchase event with the corresponding IDs set.

While this solution can help in correctly logging a purchase event, it’s important to note that the session ID is not explicitly mentioned in the official guide. Therefore, it’s unclear whether this approach will work seamlessly with the purchase journey funnel.

 if (event.type === "checkout.session.completed") {
    // Record metrics using the Google Analytics Measurement Protocol
    // See https://developers.google.com/analytics/devguides/collection/protocol/v1/devguide
    const params = new URLSearchParams({
      v: "1", // Version
      tid: <GOOGLE_ANALYTICS_CLIENT_ID>, // Tracking ID / Property ID.
      cid: event.data.object.metadata.analyticsClientId, // Client ID
      t: "event", // Event hit type
      ec: "ecommerce", // Event Category
      ea: "purchase", // Event Action
    });

    request(`https://www.google-analytics.com/batch?${params.toString()}`, {
      method: "POST",
    });
  }

For more detailed information and implementation instructions, you can refer to the official guide on server-side event recording provided by Stripe.

Extra config to make analytics data cleaner

Another issue related to GA4 and Stripe Checkout is that users redirected back from the Checkout page are counted as referral traffic by default in GA. This happens because returning users are misidentified as new sessions, leading to an increase in referral traffic. To mitigate this issue, you can exclude checkout.stripe.com as a referral source.

Follow this guide from Google to set up “unwanted referrals” in your Google tag data stream settings:

Identify unwanted referrals

By implementing these additional configurations, you can ensure cleaner analytics data and improve the accuracy of your funnel analysis.