Brownfield Adoption

Brownfield adoption is the process of adding Tayra to an existing application that already has cleartext PII data stored in its database. Unlike a greenfield deployment where every record is encrypted from the start, brownfield adoption requires a one-time migration to encrypt pre-existing data in place.

This guide walks through the migration process for both EF Core and Marten, including batch tuning, mixed-state handling, and rollback strategies.

When You Need This

You need brownfield adoption when:

• Your application already stores PII in cleartext (names, emails, phone numbers, etc.)
• You are adding Tayra to comply with GDPR, CCPA, or other data protection regulations
• You want to encrypt existing records without rebuilding your database from scratch

If you are starting a new application with no existing data, you do not need this guide. Simply configure Tayra and all data will be encrypted from the first write.

Prerequisites

Before starting a brownfield migration:

1. Back up your database. Take a full backup of every table or collection that contains PII. This backup is your rollback safety net.
2. Schedule a maintenance window. The migration reads and rewrites every row or document containing PII. Plan for downtime or reduced throughput during the migration.
3. Configure Tayra. Ensure AddTayra() is called in your DI setup with a production key store configured. Verify that your entity classes have [DataSubjectId] and [PersonalData] attributes applied correctly.
4. Test on a staging copy first. Run the migration against a copy of production data before touching the real database.

EF Core Brownfield Migration

Step 1: Register Migration Services

Register the EF Core migration service along with Tayra core services:

// Register migration services (requires AddTayra() and AddTayraEFCore() first)
var migrationServices = new ServiceCollection();
migrationServices.AddTayra(opts => opts.LicenseKey = licenseKey);
migrationServices.AddTayraEFCore();
migrationServices.AddTayraEFCoreMigrations();
migrationServices.AddDbContext<AppDbContext>(opts =>
    opts.UseInMemoryDatabase("TayraMigration"));

var migrationProvider = migrationServices.BuildServiceProvider();

^anchor

This registers ITayraDataMigrationService from the Tayra.EFCore.Migrations namespace.

Step 2: Create a Raw DbContext

The migration needs a DbContext configured without Tayra interceptors. If your normal DbContext registration includes UseTayra() interceptors, create a separate registration or a factory that produces a "raw" context:

// Create a raw DbContext without Tayra interceptors
var rawOptions = new DbContextOptionsBuilder<MyDbContext>()
    .UseNpgsql(connectionString)
    // Do NOT call UseTayra() here -- we need raw access to read cleartext
    .Options;

using var rawContext = new MyDbContext(rawOptions);

This ensures the migration reads existing cleartext values as-is, without any interceptor interference.

Step 3: Encrypt Existing Data

Call EncryptExistingDataAsync<T>() with your raw context to encrypt all rows of a given entity type:

// Bulk-encrypt existing cleartext data in batches
var migrationService = migrationProvider.GetRequiredService<ITayraDataMigrationService>();
var result = await migrationService.EncryptExistingDataAsync<CustomerEntity>(
    migrationDb,
    batchSize: 50);

Console.WriteLine($"\n=== Migration Result ===");
Console.WriteLine($"  Scanned:   {result.TotalScanned}");
Console.WriteLine($"  Encrypted: {result.Encrypted}");
Console.WriteLine($"  Skipped:   {result.Skipped}");
Console.WriteLine($"  Errors:    {result.Errors}");
Console.WriteLine($"  Duration:  {result.Duration.TotalMilliseconds:F0}ms");

^anchor

The MigrationResult (from Tayra.Migrations) reports how many rows were scanned, encrypted, skipped (already encrypted), and errored.

Repeat this for each entity type that has [PersonalData] fields.

Step 4: Verify and Enable Interceptors

After encrypting, verify that all rows are correctly encrypted:

// Verify all rows are now properly encrypted
var verification = await migrationService.VerifyEncryptionAsync<CustomerEntity>(
    migrationDb,
    batchSize: 50);

Console.WriteLine($"\n=== Verification Result ===");
Console.WriteLine($"  Verified: {verification.TotalVerified}");
Console.WriteLine($"  Valid:    {verification.Valid}");
Console.WriteLine($"  Invalid:  {verification.Invalid}");
Console.WriteLine($"  Duration: {verification.Duration.TotalMilliseconds:F0}ms");

foreach (var invalid in verification.InvalidRows)
{
    Console.WriteLine($"  [INVALID] Entity {invalid.EntityId}, Property: {invalid.PropertyName} — {invalid.Reason}");
}

^anchor

The VerificationResult (from Tayra.Migrations) contains Valid, Invalid, and TotalVerified counts, plus a list of InvalidRowDetail objects for any rows that failed verification.

Once verification passes, enable Tayra interceptors in your normal DbContext registration so all subsequent reads and writes go through transparent encryption:

// Alternative: pass the Tayra instance directly to UseTayra()
var tayra = provider.GetRequiredService<ITayra>();

var builderServices = new ServiceCollection();
builderServices.AddDbContext<AppDbContext>(opts =>
    opts.UseInMemoryDatabase("TayraSampleBuilder")
        .UseTayra(tayra, configure: efOpts =>
        {
            efOpts.EncryptOnSave = true;
            efOpts.DecryptOnLoad = true;
        }));

^anchor

Marten Brownfield Migration

Step 1: Register Migration Services

Register the Marten migration service along with Tayra core services:

// Register Tayra Marten migration services (requires AddTayra() first)
var migrationServices = new ServiceCollection();
migrationServices.AddTayra(opts => opts.LicenseKey = licenseKey);
migrationServices.AddTayraMartenMigrations();

var migrationProvider = migrationServices.BuildServiceProvider();

^anchor

This registers ITayraMartenMigrationService from the Tayra.Marten.Migrations namespace.

Step 2: Create a Raw Document Store

The migration needs a Marten IDocumentStore configured without UseTayra(). This ensures documents are read as cleartext:

// Create a "raw" store WITHOUT UseTayra() — reads cleartext field values as-is.
// This is required so the migration service can detect which documents need encrypting.
var rawStore = DocumentStore.For(opts =>
{
    opts.Connection(connectionString);
    // Do NOT call opts.UseTayra() here — we need raw access to read cleartext
});

^anchor

Step 3: Encrypt Existing Documents

Call EncryptExistingDocumentsAsync<T>() with the raw store to encrypt all documents of a given type:

// Bulk-encrypt existing cleartext documents in batches
var migrationService = migrationProvider.GetRequiredService<ITayraMartenMigrationService>();
var result = await migrationService.EncryptExistingDocumentsAsync<CustomerDocument>(
    rawStore,
    batchSize: 100);

Console.WriteLine($"\n=== Marten Migration Result ===");
Console.WriteLine($"  Scanned:   {result.TotalScanned}");
Console.WriteLine($"  Encrypted: {result.Encrypted}");
Console.WriteLine($"  Skipped:   {result.Skipped}");
Console.WriteLine($"  Errors:    {result.Errors}");
Console.WriteLine($"  Duration:  {result.Duration.TotalMilliseconds:F0}ms");

^anchor

Repeat for each document type that has [PersonalData] fields.

Step 4: Verify Document Encryption

Verify that all documents are correctly encrypted:

// Verify all documents are now properly encrypted
var verification = await migrationService.VerifyDocumentEncryptionAsync<CustomerDocument>(
    rawStore,
    batchSize: 100);

Console.WriteLine($"\n=== Marten Verification Result ===");
Console.WriteLine($"  Verified: {verification.TotalVerified}");
Console.WriteLine($"  Valid:    {verification.Valid}");
Console.WriteLine($"  Invalid:  {verification.Invalid}");
Console.WriteLine($"  Duration: {verification.Duration.TotalMilliseconds:F0}ms");

foreach (var invalid in verification.InvalidRows)
{
    Console.WriteLine($"  [INVALID] Document {invalid.EntityId}, Property: {invalid.PropertyName} — {invalid.Reason}");
}

^anchor

Step 5: Enable TayraSerializer

Once verification passes, enable UseTayra() in your Marten configuration so all subsequent reads and writes go through transparent encryption:

// Configure Marten with Tayra encryption.
// UseTayra() wraps the serializer so PII fields are encrypted in JSONB storage.
var connectionString = "Host=localhost;Port=5432;Database=tayra_sample;Username=postgres;Password=postgres";

services.AddMarten(opts =>
{
    opts.Connection(connectionString);

    // Enable Tayra encryption for documents and events
    opts.UseTayra(tayra);
});

^anchor

Batch Size Tuning

Both EF Core and Marten migration methods accept an optional batchSize parameter that controls how many rows or documents are processed in a single database transaction.

Scenario	Recommended Batch Size	Rationale
Default starting point	100	Safe balance between speed and memory
Small entities (few PII fields, small values)	500 - 1000	Each row produces little ciphertext overhead
Large entities (many PII fields, large text values)	25 - 50	Encryption buffers consume more memory per row
Constrained memory environment	25 - 50	Reduce peak memory usage
High-performance server with fast I/O	500 - 2000	Maximize throughput when resources allow

Start with 100 and adjust based on observed memory usage and migration speed. Monitor your database's transaction log size -- very large batches can cause long-running transactions that affect replication or lock contention.

Handling Mixed Encryption State

A mixed encryption state occurs when some rows or documents in a table are encrypted while others remain in cleartext. This can happen if:

• A migration was interrupted partway through (crash, timeout, deployment)
• New cleartext rows were inserted after the migration started but before interceptors were enabled

Detecting Mixed State

The migration service internally uses the EncryptionDetector utility (from Tayra.Migrations) to check the encryption state of each row or document. The EncryptionState enum has three values:

Value	Meaning
AllCleartext	No rows are encrypted
AllEncrypted	Every row is encrypted
Mixed	Some rows are encrypted, some are cleartext

Resolving Mixed State

If the state is Mixed, re-run the migration. The migration methods are idempotent -- they skip rows that are already encrypted (counted in the Skipped field of MigrationResult) and only encrypt cleartext rows. There is no risk of double-encrypting data.

Rollback Strategy

If the migration produces unexpected results or you need to revert:

1. Stop the application to prevent further encrypted writes.
2. Restore from backup. Replace the migrated tables or collections with the backup copy that contains the original cleartext data.
3. Re-deploy without Tayra (or with Tayra interceptors disabled) to return to cleartext operation.

Rollback is safe because:

• Your pre-migration backup contains all original cleartext data.
• Tayra never modifies the database schema -- it only changes field values. Restoring the backup reverts the data completely.
• Encryption keys created during the migration can remain in the key store (they are harmless) or be cleaned up separately.

Always Keep the Backup

Do not delete your pre-migration backup until you have fully verified the migration and run in production for a sufficient period. The backup is your only path back to cleartext if something goes wrong.

FAQ

Can I do a live migration without downtime?

It is technically possible but not recommended for most deployments. A live migration introduces a window where some rows are encrypted and others are not. While Tayra's interceptors and serializers can handle mixed state (they detect whether a field is already encrypted before attempting decryption), concurrent writes during the migration can create race conditions. For most applications, a short maintenance window is safer and simpler.

Can I restart an interrupted migration?

Yes. The migration methods are idempotent. If a migration is interrupted (crash, timeout, manual stop), simply re-run it. Already-encrypted rows are detected and skipped automatically. The MigrationResult.Skipped count tells you how many rows were already encrypted.

What happens with null fields?

Null [PersonalData] fields are left as null. Tayra does not encrypt null values -- there is no PII to protect. This applies to both the initial migration and normal interceptor operation.

How long does the migration take?

It depends on the number of rows, the number of PII fields per row, your batch size, and your database performance. As a rough guide, encrypting 100,000 rows with 3 PII fields each at a batch size of 100 typically takes a few minutes on a modern PostgreSQL server. Run a test against your staging environment to get an accurate estimate.

Do I need to migrate all entity types at once?

No. You can migrate one entity type at a time. This is useful if you want to adopt Tayra incrementally -- for example, encrypting Customer records first, then Employee records in a later maintenance window. Just ensure that Tayra interceptors are only enabled for entity types that have been fully migrated and verified.

See Also

• Entity Framework Core -- EF Core integration setup
• Marten -- Marten integration setup
• Getting Started -- Initial Tayra configuration
• Key Stores -- Production key store options