FF1 Tokenization

Overview

FF1 is now the primary driver of our ShieldConex Tokenization platform for enhanced tokenization security. ShieldConex implements updated FF1 idempotency keys as required per merchant. This provides cryptographic isolation between Partners and ensures that the same input always produces the same token for a given Partner—delivering consistent, deterministic tokenization without cross-Partner collision risk.

Since the ShieldConex 1.22.0 release, all newly created templates default to the 2.0 template version. The Templates v2 employ the FF1 tokenization as their main and only tokenization scheme.

This release has introduced FF1 tokenization and detokenization through ShieldConex Templates v2. FF1 is a NIST-approved format-preserving encryption method that offers enhanced security for ShieldConex tokenization. Templates created from scratch use FF1 as their main and only tokenization scheme.

Requirements and compatibility

FF1 requires values of at least six characters; shorter values are rejected.

For repeated tokenization of the same value to remain idempotent, keep both the idempotency group and idempotency key the same. Changing either can result in changed tokenized value.

On the other hand, it is important to note that idempotency does not carry across tokenization algorithms: moving from FPE or FPT to FF1 does not preserve the same tokens, and historical data is not migrated to preserve them.

BFID handling is unchanged with the associated BFID required per detokenization. For more information and examples, check out The ShieldConex API and FF1 Tokenization API Examples.

Why are shorter values rejected?

It is best practice because small domains are inherently insecure for format-preserving encryption (FPE), and the 6-character minimum (for decimal data) enforces a sufficiently large domain size.

The most common examples include the tokenization of card CVV and expiration date.

To enforce this requirement, FF1 rejects the encryption of short values at the point of API validation.

CVV (card_cvv)

PCI DSS Requirement 3.3.1 prohibits storing Sensitive Authentication Data (SAD)—including CVV/CVC/CID—after authorization, even if encrypted, hashed, or tokenized. The standard is explicit that cryptographic techniques do not satisfy the "do not store" rule. CVV may be collected and held only transiently before authorization (and must be strongly protected while it is in memory/transit), then rendered unrecoverable immediately afterward.

This is why people do not tokenize CVV:

• 3-digit CVV → 1,000 possible values
• 4-digit (Amex) → 10,000 possible values

Both are far below the FF1 domain-size floor of 1,000,000. Even if you padded the field to 6 characters, the real entropy remains tiny, so enumeration is trivial.

Expiration date (card_exp)

Expiration date is ordinary cardholder data (not SAD).

According to PCI DSS, it may be stored when there is a legitimate business need, provided the overall cardholder data is protected (e.g., when stored with the PAN).

Typical formats are still short:

• MMYY → ~ few hundred realistic values (months 01–12 × a limited year range)
• MM/YYYY or similar remains well under the 1,000,000 domain requirement

Standalone FF1 on a 4-digit expiry is therefore not secure for the same reasons as a short CVV.

Practical options to Store Expiration Date:

1. Preferred: Do not use format-preserving encryption on the expiry alone. Store it with conventional encryption (or a non-format-preserving token such as a UUID) if protection is required, or store it in the clear if the PAN is already properly protected/tokenized (common, because expiry by itself has limited value).
2. If format preservation is mandatory for legacy reasons:
   • Always normalize and store as MMYYYY (6 digits)
   • Or concatenate with other low-sensitivity context before applying FF1
   • Explicitly note the residual risk that the effective entropy is still low and that an attacker with an oracle can still enumerate the realistic subset of values.
3. Record the exact format you use (MMYY, MMYYYY, YYYYMM, etc.) and any padding/normalization rules so consumers of the schema know what to expect.

Template Configuration

FF1 is the format-preserving tokenization path for Template v2. ShieldConex selects it from the template version, so customers do not select the algorithm in each tokenize request.

The tokenization is intended to preserve the configured length, alphabet, and format; detokenization restores the original value.

Create a template using the steps below, then explore the Templates v2.0 features described in this section: idempotency options, field lengths, character sets, and preservation options. The video provides a visual walkthrough of template configuration and feature exploration.

1. Sign in to the certification portal and open Manage > Templates.
2. Select Create Template. All new templates use version 2.0 by default; you do not need to select a version or tokenization algorithm. Select the Partner and Client required for your role, enter a template name, and select Generate to create its TemplateRef. Use this identifier as templateRef in tokenization requests, not as the request-tracking reference.
3. Set Access Mode to Restrict Sharing to limit detokenization to the current Partner, or Allow Sharing to allow other Partners to detokenize your data.
4. Set Idempotency Group under Idempotency options. Enter an integer from -2147483648 to 2147483647, or leave it unset to use 0. Keep this setting unchanged for tokens you need to detokenize later.
5. Select Click to add new field and configure the fields you want to tokenize. Set each field's System Name to match its name in API requests. For custom FF1 fields, set Min Length and Max Length within 6–1024, with the minimum no greater than the maximum. Review the character sets and preservation options; inputs must meet the six-character minimum after data masking.
6. Save the template and test tokenization and detokenization in the certification environment. Include every required template field in your tokenization request. Use the returned BFID and tokenized values for detokenization, and confirm that you recover the original values. Repeat tokenization with the same input, idempotency group, and runtime idempotency key to confirm that the token stays the same. Match response values by name, not array position.

Recommended: Publish after confirming accuracy. After you verify that the template settings are correct and the tokenization and detokenization results match your expected values, select Publish, then save. Publishing permanently prevents further changes to the template, including unintended changes to its idempotency group. To make later changes, clone the template; the clone has a new TemplateRef and starts unpublished.

For the comprehensive guide on setting up a template configuration, please refer to ShieldConex | Templates.

FF1 Idempotency Options

Template v2.0 provides a template-level (setting) Idempotency Group under Idempotency options. In the ShieldConex Manager, this field accepts a signed integer from -2147483648 to 2147483647.

FF1 idempotency keys are supplied at runtime through the Tokenization API or iFrame SDK. For repeated tokenization of the same value to remain idempotent, keep both the idempotency group and idempotency key the same. Changing either can change the token returned.

Idempotency does not carry across tokenization algorithms. Moving from FPE or FPT to FF1 does not preserve the same tokens, and historical data is not migrated to preserve them. BFID handling remains unchanged.

If no Idempotency Group is set, it defaults to 0.

Fields

Min Length

The minimum input length accepted by the field. It must be less than or equal to Max Length. For custom FF1 fields, Manager accepts a Min Length setting from 6 to 1024. FF1 rejects values shorter than six characters.

Max Length

The maximum input length accepted by the field. It must be greater than or equal to Min Length. For custom FF1 fields, Manager accepts a Max Length setting from 6 to 1024. These are the custom-field configuration bounds shown in Manager.

Character Sets

The following are all the allowed character combinations and ranges that FF1 supports for ShieldConex.

Preservation Options

What the preservation options essentially represent for a template field is that they impose the data masking rules on the input value.

How does IdempotencyGroup affect existing tokens?

If a value has already been tokenized under a specific IdempotencyGroup (set at the template level), changing the IdempotencyGroup will produce different detokenized outputs.

This is expected behavior: the IdempotencyGroup is part of the cryptographic context, so tokens generated under one group will not detokenize correctly under another—even when they were created from the same template.

For example, the detokenization response becomes something like the following:

{
  "messageId": "<bluefin-message-id>",
  "reference": "2b209fdc5a7a4380a9c87ed5bec04bf4",
  "values": [
    {
      "name": "card_holder",
      "value": "KKOh BbV01"
    },
    {
      "name": "card_number",
      "value": "5497762600616528"
    }
  ]
}

In contrast, the idempotencyKey affects newly tokenized values generated at runtime (API level).

Environment URLs

There are two ShieldConex environments each with their own URL.

All requests must be made over HTTPS for security. Append the endpoints described below to the appropriate base URL depending on whether you are testing or running in production.

For example, this is the full URL for tokenizing data in the certification environment:

https://secure-cert.shieldconex.com/api/tokenization/tokenize

API Authentication

In order to authenticate with the ShieldConex endpoint, please follow the API Authentication guide.

FF1 Tokenization API Examples

Keep the template identifier separate from the request-tracking reference:

Include all required template fields in tokenization requests. Match objects in response values arrays by name; the API does not guarantee their order.

Use Cases

Depending on their use case, customers can choose:

A fresh token each time

Generate a new random idempotencyKey for each request. In this case, idempotencyGroup doesn’t matter for repeatability.

Repeatable tokens within separate domains

Keep both settings consistent within a domain; change either to establish a new domain.

Repeatable tokens across requests for a template

Keep both settings consistent. Most commonly, this means passing no idempotencyKey.

Sharing a template

Partners still can get idempotent tokens together. Data sharing with other entities and preventing compatibility with tokens that are not meant to be shared with that entity.

Single Tokenization Request

Request

POST /api/tokenization/tokenize

{
  "reference": "5092097aff3d43a7804f2ca1982559a8",
  "templateRef": "d00bc705ad88fefa38f71c6a1f9b548e",
  "idempotencyKey": "8705b9c8bf664302a68fdf2181508f3e",
  "values": [
    {
      "name": "card_holder",
      "value": "Jane Smith"
    },
    {
      "name": "card_number",
      "value": "4111111111111111"
    }
  ]
}

Response

{
  "messageId": "1202610301152261221834270",
  "reference": "5092097aff3d43a7804f2ca1982559a8",
  "bfid": "djM6MTIwMjYxMDMwMTE1MjI2MTIyMTgzNDI3MHxkMDBiYzcwNWFkODhmZWZhMzhmNzFjNmExZjliNTQ4ZXwwNWgyUnlBWWtuV0hQT0hRZjdYQnQ5Q0FWMjdRY0VOcU5paHVRbGJDZmpFPXxGRjFTQ1hDRVJUOTgzMjAyNjA4MjB8",
  "values": [
    {
      "name": "card_holder",
      "value": "NDJ4 ZsJ3U"
    },
    {
      "name": "card_number",
      "value": "4133157377923678"
    }
  ]
}

Single Detokenization Request

Detokenization involves combining the respective BFID and tokenized values of the tokenized entry above.

Neither templateRef nor idempotencyKey are needed for this operation.

Request

POST /api/tokenization/detokenize

{
  "reference": "2b209fdc5a7a4380a9c87ed5bec04bf4",
  "bfid": "djM6MTIwMjYxMDMwMTE1MjI2MTIyMTgzNDI3MHxkMDBiYzcwNWFkODhmZWZhMzhmNzFjNmExZjliNTQ4ZXwwNWgyUnlBWWtuV0hQT0hRZjdYQnQ5Q0FWMjdRY0VOcU5paHVRbGJDZmpFPXxGRjFTQ1hDRVJUOTgzMjAyNjA4MjB8",
  "values": [
    {
      "name": "card_holder",
      "value": "NDJ4 ZsJ3U"
    },
    {
      "name": "card_number",
      "value": "4133157377923678"
    }
  ]
}

Response

{
  "messageId": "1202610301155181221993554",
  "reference": "2b209fdc5a7a4380a9c87ed5bec04bf4",
  "values": [
    {
      "name": "card_holder",
      "value": "Jane Smith"
    },
    {
      "name": "card_number",
      "value": "4111111111111111"
    }
  ]
}

Batch Tokenization Request

Tokenizing a large volume of data is highly resource- and encryption-intensive when each entry is processed individually.

That is why customers with large databases of clear-text records, and similar use cases, can leverage ShieldConex Tokenization Batch for efficient encryption at scale.

A batch applies the top-level templateRef and idempotencyKey to tokenize all the (underlying) batch values in the API request.

The top-level reference tracks the batch request; it is not the template identifier. The examples also use a separate reference for each batch item to distinguish its request and response data.

Request

POST /api/tokenization/batch/tokenize

{
  "reference": "ff1-batch-example-248aad66965442ffac3a6a8e3dbe89e3",
  "templateRef": "d00bc705ad88fefa38f71c6a1f9b548e",
  "idempotencyKey": "8705b9c8bf664302a68fdf2181508f3e",
  "batches": [
    {
      "reference": "item-1",
      "values": [
        {"name": "card_number", "value": "4111111111111111"}
      ]
    },
    {
      "reference": "item-2",
      "values": [
        {"name": "card_number", "value": "5555555555554444"}
      ]
    }
  ]
}

Response

{
  "messageId": "1202610301140001223218168",
  "reference": "ff1-batch-example-248aad66965442ffac3a6a8e3dbe89e3",
  "batches": [
    {
      "reference": "item-1",
      "bfid": "djM6MTIwMjYxMDMwMTE0MDAwMTIyMzIxODE2OCMwfGQwMGJjNzA1YWQ4OGZlZmEzOGY3MWM2YTFmOWI1NDhlfGFXNCtRSjA1WlRJVFdXNDFIQjN1WlkvNExnR2xTV2h0U1h2cWxHN3hweWs9fEZGMVNDWENFUlQ5ODMyMDI2MDgyMHw=",
      "values": [
        {
          "name": "card_number",
          "value": "9284099259528814"
        }
      ]
    },
    {
      "reference": "item-2",
      "bfid": "djM6MTIwMjYxMDMwMTE0MDAwMTIyMzIxODE2OCMxfGQwMGJjNzA1YWQ4OGZlZmEzOGY3MWM2YTFmOWI1NDhlfGFXNCtRSjA1WlRJVFdXNDFIQjN1WlkvNExnR2xTV2h0U1h2cWxHN3hweWs9fEZGMVNDWENFUlQ5ODMyMDI2MDgyMHw=",
      "values": [
        {
          "name": "card_number",
          "value": "1282859270324374"
        }
      ]
    }
  ]
}

Batch Detokenization Request

Detokenization involves combining the respective BFID and tokenized values for each batch.

Neither templateRef nor idempotencyKey are needed for this operation.

Request

POST /api/tokenization/batch/detokenize

{
  "reference": "ff1-batch-example-1c9ec5aee2274362adb7956093e2e2fd",
  "batches": [
    {
      "reference": "item-1",
      "bfid": "djM6MTIwMjYxMDMwMTE0MDAwMTIyMzIxODE2OCMwfGQwMGJjNzA1YWQ4OGZlZmEzOGY3MWM2YTFmOWI1NDhlfGFXNCtRSjA1WlRJVFdXNDFIQjN1WlkvNExnR2xTV2h0U1h2cWxHN3hweWs9fEZGMVNDWENFUlQ5ODMyMDI2MDgyMHw=",
      "values": [
        {
          "name": "card_number",
          "value": "9284099259528814"
        }
      ]
    },
    {
      "reference": "item-2",
      "bfid": "djM6MTIwMjYxMDMwMTE0MDAwMTIyMzIxODE2OCMxfGQwMGJjNzA1YWQ4OGZlZmEzOGY3MWM2YTFmOWI1NDhlfGFXNCtRSjA1WlRJVFdXNDFIQjN1WlkvNExnR2xTV2h0U1h2cWxHN3hweWs9fEZGMVNDWENFUlQ5ODMyMDI2MDgyMHw=",
      "values": [
        {
          "name": "card_number",
          "value": "1282859270324374"
        }
      ]
    }
  ]
}

Response

{
  "messageId": "1202610301143031262272882",
  "reference": "ff1-batch-example-1c9ec5aee2274362adb7956093e2e2fd",
  "batches": [
    {
      "reference": "item-1",
      "values": [
        {
          "name": "card_number",
          "value": "4111111111111111"
        }
      ]
    },
    {
      "reference": "item-2",
      "values": [
        {
          "name": "card_number",
          "value": "5555555555554444"
        }
      ]
    }
  ]
}