Set up Key Rewrite

The Key Rewrite feature in StreamX Mesh enables flexible transformation of resource keys. This is particularly useful when incoming data uses complex or structured keys that must be standardized for efficient identification and retrieval. By setting up key rewriting, you can simplify resource keys to improve data organization and ensure compatibility with existing data structures.

This guide walks you through configuring Key Rewrite in StreamX Mesh with regex-based transformations.

For comprehensive details on this feature, see the Key Rewrite reference.

Prerequisites

Ensure you have the following before proceeding:

  • Approximately 15 minutes to complete the setup

  • StreamX CLI installed

  • A predefined mesh.yaml file for StreamX

  • A web browser of your choice

A simple mesh.yaml and knowledge on how to serve web resources with StreamX you can gain with Create a simple website with StreamX CLI tutorial.

Step 1: Provide match-rewrite patterns pair

Consider an example, where the input system generates data with a resource key:

"blog-entries/2023-05-12/food/john-smith/page.html".

You want to modify this key to simplify resource identification in StreamX. The rewritten publication key should look like this:

"blogs/food/john-smith/page.html".

The goal is to:

  • Change "blog-entries" to "blogs"

  • Remove the date from the key

  • Apply transformation only to .html files

To achieve this, prepare match-rewrite patterns. For the given example:

  • Match pattern: "^blog-entries/([^/])/(.*)\\.html$"+

  • Rewrite pattern: "blogs/$2.html"

Step 2: Configure Key Rewrite

The transformation can be achieved by passing environment variables into StreamX Rest Ingestion. You must use the given properties to set match and rewrite pattern:

ingestion:
  rest-ingestion:
    environment:
      STREAMX_KEY-REGEX-REWRITE_PATTERNS_CONFIG1_MATCH-PATTERN: "^blog-entries/([^/]+)/(.*)\\.html$"
      STREAMX_KEY-REGEX-REWRITE_PATTERNS_CONFIG1_REWRITE-PATTERN: "blogs/$2.html"

Step 3: Validate StreamX

To validate your StreamX setup, you can publish a simple website with a blog entry and check whether the entry is published with the rewritten resource path.

Summary

Congratulations! You have successfully set up key rewrites for the StreamX Mesh.

Related content