Building a Secure AI LLM SaaS BYOK: using Infisical secret management - Andy Primawan

Building an AI LLM SaaS with a Bring-Your-Own-Key (BYOK) model? Storing customer API keys in your database is a huge security risk. Learn how to securely manage these secrets using Infisical, with a practical code example. It's easier and safer than you think.

Building a Secure AI LLM SaaS BYOK: using Infisical secret management - Andy Primawan
Building a secure AI LLM SaaS with BYOK (Bring Your Own Key) using Infisical - Andy Primawan

Introduction

As full-stack engineers, we're at the forefront of the AI boom. Building LLM-powered SaaS applications is an exciting challenge, and one feature request is becoming increasingly common: Bring Your Own Key (BYOK).

Whether it's for cost control, access to specific model versions, or managing their own rate limits, customers often want to use their own OpenAI, Anthropic, or other foundation model API keys within your application.

Your first instinct might be to add a new, encrypted column like openai_api_key to your users or tenants table in Postgres or MongoDB. It seems simple, but this approach is fraught with security risks that can undermine your entire platform's integrity. Let's talk about why that's a bad idea and explore a much more robust solution.

The Problem: The Siren Call of the Plaintext Column

Storing sensitive, third-party credentials directly in your primary application database—even if you apply some form of application-level encryption—is a security anti-pattern. Here’s why it’s so dangerous:

  • Catastrophic Blast Radius: If your application database is compromised through SQL injection, a misconfigured connection string, or a vulnerability in your ORM, an attacker gets everything. They don't just get your user data; they get the keys to your customers' kingdoms. An attacker could rack up enormous bills on your customers' accounts, access their private models, and destroy your company's reputation overnight.
  • Insider Threats: A disgruntled employee or a contractor with database access could easily export all customer secrets. Relying on application-level encryption helps, but it’s not foolproof and becomes complex to manage.
  • Secret Sprawl: Keys stored in the database can easily leak. They can show up in database backups, replication logs, application logs during debugging, or memory dumps. The more places a secret exists, the harder it is to secure.
  • Compliance and Trust: If you're aiming for compliance certifications like SOC 2 or ISO 27001, auditors will heavily scrutinize how you handle sensitive customer data. Storing keys directly in your app DB is a major red flag that signals a lack of security maturity. It breaks the principle of least privilege.

The bottom line is that your primary database is designed for application state, not for vaulting high-value secrets.

The Solution: A Dedicated Vault with Infisical

The best practice for handling secrets is to use a dedicated secret management platform. These tools are purpose-built to store, manage, and securely dispense secrets like API keys, database credentials, and certificates. They ensure secrets are encrypted at rest and provide strict access controls and audit logs.

For this task, we'll use Infisical. It's an open-source, end-to-end encrypted platform that's incredibly developer-friendly. Instead of storing the customer's API key in your database, you store it in Infisical and save only a non-sensitive reference or path to that secret in your database.

Here is the secure workflow:

  1. A customer submits their API key through your application's UI.
  2. Your backend service receives the key.
  3. Instead of writing to your database, your service uses the Infisical SDK to create a new secret. You'll generate a unique, non-guessable path for this secret, for example, byok/tenant/{tenant-id}/openai.
  4. The Infisical SDK encrypts the key and stores it securely in the Infisical vault.
  5. You save the path to the secret (e.g., byok/tenant/{tenant-id}/openai) in your tenants table in Postgres/MongoDB. This path is not the secret itself.
  6. When your application needs to make an API call on behalf of the customer, it first retrieves the secret path from your database, then uses the Infisical SDK to fetch the actual API key just-in-time.
Example of Infisical running locally, created: project, secret, identity, universal auth key - Andy Primawan

Demo Code: A Practical Example

Let's look at how this is implemented in practice. The following code is inspired by the architecture in the Dojotek AI Chatbot - backend repository.

We'll assume you have a Node.js/TypeScript backend and have the Infisical Node SDK installed (npm install @infisical/sdk).

Let's write a function to save the customer's key, and another function to retrieve the key when needed:

// src/model-provider-secrets/model-provider-secrets.service.ts
import {
  Injectable,
  InternalServerErrorException,
  NotFoundException,
} from '@nestjs/common';
import { CreateModelProviderSecretDto } from './dto/create-model-provider-secret.dto';
import { UpdateModelProviderSecretDto } from './dto/update-model-provider-secret.dto';
import { PrismaService } from '../prisma/prisma.service';
import { ConfigsService } from '../configs/configs.service';
import { InfisicalSDK } from '@infisical/sdk';
import { Prisma } from '../generated/prisma/client';

@Injectable()
export class ModelProviderSecretsService {
  constructor(
    private readonly prisma: PrismaService,
    private readonly configs: ConfigsService,
  ) {}

  private async getInfisicalClient(): Promise<InfisicalSDK> {
    const client = new InfisicalSDK({ siteUrl: this.configs.infisicalSiteUrl });
    const clientId = this.configs.infisicalClientId;
    const clientSecret = this.configs.infisicalClientSecret;
    if (!clientId || !clientSecret) {
      throw new InternalServerErrorException(
        'Infisical credentials are not configured',
      );
    }
    await client.auth().universalAuth.login({ clientId, clientSecret });
    return client;
  }

  async create(dto: CreateModelProviderSecretDto) {
    // Create DB record first to get ID as pointer key
    const created = await this.prisma.modelProviderSecret.create({
      data: {
        name: dto.name,
        type: dto.type,
      },
    });

    try {
      const client = await this.getInfisicalClient();
      const environment = this.configs.infisicalEnvironment;
      const projectId = this.configs.infisicalProjectId;
      const secretPath = this.configs.infisicalModelProviderSecretsPath;

      const createdSecret = await client.secrets().createSecret(created.id, {
        environment,
        projectId,
        secretValue: dto.secret,
        secretComment: 'From Dojotek AI Chatbot backend',
        secretPath,
      });

      const updated = await this.prisma.modelProviderSecret.update({
        where: { id: created.id },
        data: {
          secretStoragePointer:
            (createdSecret as { secret?: { reference?: string } })?.secret
              ?.reference ?? created.id,
        },
      });
      return updated;
    } catch (err) {
      // Rollback DB record on failure to store secret
      await this.prisma.modelProviderSecret.delete({
        where: { id: created.id },
      });
      throw err;
    }
  }

  async findOne(id: string) {
    const found = await this.prisma.modelProviderSecret.findUnique({
      where: { id },
    });
    if (!found) {
      throw new NotFoundException('ModelProviderSecret not found');
    }
    return found;
  }
}

This pattern elegantly decouples your application state from your customers' secrets. Your database remains clean and secure.

Why It Matters: More Than Just Encryption

Adopting this pattern gives you benefits far beyond simple encryption:

  • Centralized Management: All your secrets—for your infrastructure and your customers—live in one dashboard.
  • Audit Trails: Infisical provides detailed audit logs (available on paid plains). You can see exactly who or what service accessed a secret and when. This is invaluable for security investigations and compliance.
  • Granular Access Control: You can define strict policies on which services or environments can access specific secret paths. Your staging environment, for example, should never be able to access production customer keys.
  • Simplified Developer Experience (DX): Instead of wrestling with low-level crypto libraries or complex KMS setups, your team gets a clean, high-level SDK. This reduces the chance of implementation errors.

Conclusion

The "Bring Your Own Key" model is a powerful feature that gives your customers flexibility and control. But with great power comes great responsibility. As engineers, our responsibility is to handle that sensitive data with the highest level of care.

Storing customer API keys in your primary database is a ticking time bomb. By offloading that responsibility to a dedicated secret manager like Infisical, you not only adopt security best practices but also build a more robust, compliant, and trustworthy platform. It's a strategic choice that protects your customers, your company, and your peace of mind.

It's not just about building features; it's about building trust.

Example Code

You can find a complete, deployable example of a BYOK-enabled AI chatbot that uses Infisical for secret management in this GitHub repository:

https://github.com/dojotek/dojotek-ai-chatbot-backend

Related files:

  1. model-provider-secrets.service.ts
  2. Prisma ORM schema for ModelProviderSecret
  3. related ENV variables to connect to the Infisical