A third recovery site: hybrid Data Guard and Autonomous Recovery Service

Until recently, most organizations I work with have been comfortable running their databases and applications across two regions. Sometimes that is an Active/Standby configuration, but more often I see some form of Active/Active. The big change I have seen recently is the addition of a third site, often described as a "cyber vault" or an IRE (Isolated Recovery Environment).

To provide that third site, many customers are looking at hosting a standby database in the cloud, either in OCI or through Oracle Database@ services in a multicloud environment. A practical way to limit traffic from the on-premises environment is to only send redo to the cloud database with Data Guard, then back up that cloud database to Autonomous Recovery Service. The backup workload stays on the cloud side.

Creating a cloud standby from your on-premises database is easier than you might think. In this post, I go through the standby build and the next step: configuring backups to Autonomous Recovery Service.

One thing to keep in mind is that a standby database and backups are only part of the picture. To use this as a cyber vault or IRE, you also need to control who can access it and which systems can connect to it. You need to protect the encryption keys and test that you can restore the database to a point before an attack or data corruption occurred. Plan for all of this as you build the standby and configure the backups..

The third-site pattern: send redo to a cloud standby, then back up the standby to Autonomous Recovery Service. The worked example below uses Oracle Exadata Database Service in OCI.

I recently went through the Oracle documentation for creating a hybrid Data Guard configuration. There is a lot of information to work through, especially when you are trying to figure out what needs to be ready before you start.

I am breaking this into four parts:

1. Get the prerequisites in place.
2. Run dbcactl to prepare the primary database.
3. Run dbaascli to create the standby database in OCI.
4. Configure backups from the cloud standby to Autonomous Recovery Service.

Once the preparation is done, the standby build comes down to two commands. I will go through their parameters and what dbcactl and  dbaascli are doing, then show where Recovery Service fits into the third-site design.

For this walkthrough, I am showing an on-premises Oracle RAC primary with a standby on Oracle Exadata Database Service in OCI. I have changed the database names and hostnames to use a generic sales example. My example build used a 19c Oracle Home and DBAAS CLI 26.3.1.0.0.

NOTE: The cloud VM cluster and Oracle Home need to exist before you start. configureStandby creates the database within that cluster. Refer to the full Oracle guide for platform requirements and configurations with additional standbys.

1 What needs to be ready before I start

First, let's go through the pieces that need to be in place before running either command.

• Download dbcactl. My Oracle Support document 3099785.1, linked from the hybrid guide, has the download and installation instructions. I am using /home/oracle/dbcacl/dbcactl in the example. The folder is named dbcacl, and the executable is dbcactl.
• Have the cloud Oracle Home ready. Match the primary release, RU, and one-off patches where possible. If you need a different RU, check the combinations allowed in the guide.
• Check connectivity in both directions. The primary needs to reach the cloud, and the cloud needs to reach the primary. Check DNS, SSH, and Oracle Net for all RAC nodes and SCAN endpoints. The network also needs enough bandwidth for the initial copy and ongoing redo.
• Have a shared ACFS directory on the primary. This is where the network files and wallet configuration can be shared between instances. For this example I am using /acfs01/sales on an existing mounted ACFS filesystem.
• Prepare the primary for Data Guard. This includes standby redo logs, logging, Flashback Database, Oracle Net encryption, and enough recovery-area space. Follow the MAA recommendations in the guide.

Make sure the encryption keys are there

One thing I want to call out is the TDE wallet. Even if the on-premises primary is not encrypted, it still needs a wallet with master encryption keys for this hybrid configuration. The cloud database needs those keys.

For this example I am using a file-based TDE keystore. Before starting, I would check that the wallet is usable and that the keys exist in the CDB root and each PDB other than the seed. The TDE with Data Guard documentation goes through the key handling.

For 19c at RU 19.16 or later, these are the settings I would look at:

Parameter	Example or policy	Purpose
WALLET_ROOT	/acfs01/sales/wallet	Root directory for wallet storage; the file-based TDE wallet is under tde. Use your existing keystore location.
TDE_CONFIGURATION	KEYSTORE_CONFIGURATION=FILE	Selects the file-based keystore used in this example.
TABLESPACE_ENCRYPTION	Choose for each site	AUTO_ENABLE is the cloud policy. DECRYPT_ONLY  is an on-premises option when encrypted tablespaces must be avoided. Do not apply it to the cloud standby.

NOTE: Setting the parameters does not create the wallet or set the master keys. Also, WALLET_ROOT is static, so changing it requires a restart.

If you still need to set up the wallet, follow the TDE configuration procedure. The TABLESPACE_ENCRYPTION reference explains the encryption policies. For earlier RUs, use the release-specific settings in the hybrid guide.

Here is a quick check I can run on the primary, connected through SQL*Plus as oracle:

SELECT name, db_unique_name, database_role, log_mode,
       force_logging, flashback_on
FROM v$database;

SHOW PARAMETER wallet_root
SHOW PARAMETER tde_configuration
SHOW PARAMETER tablespace_encryption

SELECT con_id, status, wallet_type, wrl_parameter
FROM v$encryption_wallet;

I want the primary in ARCHIVELOG mode, with force logging enabled and Flashback Database prepared before moving on. If archive logging still needs to be enabled, plan for the maintenance window that change may require.

I would also check V$DATABASE_KEY_INFO and V$ENCRYPTION_KEYS in the root and relevant PDBs. An open wallet is only part of the check; I need the master keys as well. If the wallet shows OPEN_NO_MASTER_KEY, that needs to be addressed before the build.

Use your existing key-management procedure and wallet location. There is no reason to replace an existing wallet just to match the path in my example.

Finally, have the primary SYS password, the TDE wallet password, and the password you will supply for AWR administration ready. These have different purposes and do not need to be the same.

Get Recovery Service ready as well

For the backup step, I also need the Recovery Service IAM policies, network connectivity, and a registered Recovery Service subnet. For 19c, the documented minimum is RU 19.18. Broker is mandatory for this backup destination in a Data Guard configuration with dbaascli 25.3.1.0.0 and later. I would also check regional availability, service limits, and the retention policy I need. The Exadata backup guide covers these prerequisites.

The names I am using in the example

Item	Generic value
Database name on both sites	sales
Primary database unique name	sales_primary
Standby database unique name	sales_stby
Primary SCAN	onprem-scan.example.com
Standby SCAN	cloud-scan.example.com
Primary database service	sales_primary.example.com
SCAN listener port on both sites	1521
Existing cloud Oracle Home	/u02/app/oracle/product/19.0.0.0/dbhome_1

You can see that I am using sales as the database name on both sides. The unique names are different: sales_primary and sales_stby.

The other distinction to keep in mind is the SCAN name versus the database service. The SCAN gets me to the cluster, and the service identifies the database connection. Use the actual service registered with your primary listener when replacing sales_primary.example.com.

2 Run dbcactl on the primary

Figure 1. Run the preparation command on the first primary node. The standby SCAN already belongs to the provisioned cloud cluster.

Now I can prepare the primary. The diagram above shows the database and its SCAN listener, along with the cloud SCAN I will pass to the command.

I run this as oracle on the first primary node, with ORACLE_HOME and the local ORACLE_SID set for the source database. The shared directory /acfs01/sales also needs to be writable by oracle.

Here is the command with the example names:

/home/oracle/dbcacl/dbcactl \
  -silent \
  -oui_internal \
  -configureDatabase \
  -prepareForStandby \
  -dgTNSNamesoraFilePath /acfs01/sales \
  -sourceDB sales_primary \
  -gdbName sales.example.com \
  -standbyDBUniqueName sales_stby \
  -standbyScanName cloud-scan.example.com \
  -standbyScanPort 1521 \
  -primaryScanPort 1521 \
  -pdbServiceDomain example.com \
  -blobFileLocation /tmp

The command prompts for the primary SYS password. In my run, I could see it creating the Data Guard services, updating tnsnames.ora and the include-file entry, and then preparing the standby bundle.

What the dbcactl parameters mean

Parameter	Value in this example	Meaning
-silent	Flag	Runs without the graphical assistant; password prompts still appear.
-oui_internal	Flag	Workflow flag that allows for the use of prepareForStandby  (which is a hidden parameter)
-configureDatabase	Flag	Selects database configuration.
-prepareForStandby	Flag	Selects primary preparation for a standby.
-dgTNSNamesoraFilePath	/acfs01/sales	Directory for the Data Guard network configuration.
-sourceDB	sales_primary	Identifies the primary by its unique name.
-gdbName	sales.example.com	Global database name passed to the workflow.
-standbyDBUniqueName	sales_stby	Unique name planned for the standby.
-standbyScanName	cloud-scan.example.com	SCAN hostname for the existing standby cluster.
-standbyScanPort	1521	Standby SCAN listener port.
-primaryScanPort	1521	Primary SCAN listener port.
-pdbServiceDomain	example.com	PDB service domain passed to the workflow.
-blobFileLocation	/tmp	Output directory for the generated bundle.

These are the options from my successful command, with the environment names changed. If you are using a different utility version, check its supported options, especially -gdbName and -pdbServiceDomain.

NOTE: My original run included -skipFlashbackValidation and -skipForceLoggingValidation. I have left those out here so the checks run. Skipping a check does not remove the need to prepare the database.

At the end, the command tells me where it wrote the .tar file. The filename will look like this, with a generated timestamp and ID:

Successfully created blob file:
/tmp/sales_<generated_timestamp>_<generated_id>.tar

This file contains the preparation material, including network files and the TDE wallet. I need to copy it securely to the first cloud node before running the second command.

For example, I can use scp:

scp /tmp/sales_<generated_timestamp>_<generated_id>.tar opc@cloud-node1.example.com:/tmp/

Replace the timestamp and ID with the actual filename from your output. For the next command, I am using /tmp/sales_standby.tar as a simpler name for the transferred file. You can keep the generated name instead; just use the same path in --standbyBlobFromPrimary.

NOTE: The bundle contains wallet material. Keep access to it restricted during the transfer and on the destination.

3 Run dbaascli to create the standby

Figure 2. The existing cloud VM cluster hosts the new standby database. The primary and standby share DB_NAME=sales and have different unique names.

Now that the bundle is on the cloud node, I can create the standby database.

This command runs as root on the first cloud node. The Oracle Home in the command is the existing cloud home where the standby will be created.

dbaascli dataguard configureStandby \
  --activeDG true \
  --standbyDBUniqueName sales_stby \
  --standbyScanIPAddresses cloud-scan.example.com \
  --protectionMode MAX_PERFORMANCE \
  --standbyScanPort 1521 \
  --dbname sales \
  --oracleHome /u02/app/oracle/product/19.0.0.0/dbhome_1 \
  --primaryScanIPAddresses onprem-scan.example.com \
  --primaryServiceName sales_primary.example.com \
  --transportType ASYNC \
  --primaryScanPort 1521 \
  --standbyBlobFromPrimary /tmp/sales_standby.tar \
  --noDBDomain

What the dbaascli parameters mean

Parameter	Value in this example	Meaning
--activeDG	true	Requests Active Data Guard. Use an appropriate service entitlement for read-only access while apply runs.
--standbyDBUniqueName	sales_stby	Standby unique name.
--standbyScanIPAddresses	cloud-scan.example.com	Standby SCAN name or comma-separated SCAN IPs.
--protectionMode	MAX_PERFORMANCE	Data Guard protection mode.
--standbyScanPort	1521	Standby listener port.
--dbname	sales	Database name.
--oracleHome	/u02/app/oracle/product/19.0.0.0/dbhome_1	Existing target Oracle Home.
--primaryScanIPAddresses	onprem-scan.example.com	Primary SCAN name or comma-separated SCAN IPs.
--primaryServiceName	sales_primary.example.com	Primary database service.
--transportType	ASYNC	Redo transport mode.
--primaryScanPort	1521	Primary listener port.
--standbyBlobFromPrimary	/tmp/sales_standby.tar	Transferred preparation bundle.
--noDBDomain	Flag	Omits the standby database domain.

You may have noticed that I passed SCAN hostnames to parameters named IPAddresses. These parameters accept either a SCAN name or a comma-separated list of SCAN IP addresses. You can find the options in the dbaascli command reference.

I am also using --noDBDomain for the standby. That controls the database domain, so the SCAN and primary service can still have fully qualified names. If you need a standby database domain, use --standbyDBDomain and make sure it agrees with the preparation settings.

The command asks for the following passwords:

Enter PRIMARY_DB_SYS_PASSWORD:
Enter PRIMARY_DB_TDE_PASSWORD:
Enter AWR_ADMIN_PASSWORD:
Enter AWR_ADMIN_PASSWORD (reconfirmation):

What is happening while dbaascli runs

This is the part I wanted to spend a little time on. There is a lot of output from dbaascli, and it helps to understand what all those jobs are doing.

Looking through my output, I can group the work into five pieces:

1. Validate the target. Check the inputs, VM type, Oracle Home version, Clusterware, database names, disk space, memory, TDE configuration, and storage compatibility.
2. Prepare the databases. Fetch the bundle, prepare primary configuration, enable primary logging, configure primary flashback, generate standby metadata, and configure the standby database. These jobs can change the primary as well as the target.
3. Configure connectivity and encryption. Update sqlnet.ora, tnsnames.ora, listener parameters, and the standby wallet.
4. Integrate Data Guard and cloud management. Configure redo routing, verify Broker status, and update cloud automation and database registry metadata.
5. Configure AWR integration and finish. Configure AWR accounts, links, node registration, and a remote snapshot; generate database and Data Guard details; then clean up.

One thing to keep in mind is that copying the bundle is only the handoff between the two commands. The database data still needs to be copied to create the standby. The hybrid guide describes the RMAN channels used during that part of the build.

How long it takes will depend on the database size, the source workload, the cloud resources, and the network between the two environments.

My run finished with:

dbaascli execution completed

Check the standby after the build

The next thing I would do is check the database and Data Guard configuration. Keep the session log path that dbaascli prints; it is useful if you need to go back through the build.

As root on the cloud node, I can run:

dbaascli database getDetails --dbname sales
dbaascli dataguard getDetails --dbName sales

I want to see the physical standby role, a successful configuration status, and redo transport enabled on the primary.

Then, as oracle with the standby environment set, I would check Broker:

dgmgrl / 'show configuration'
dgmgrl / 'show database verbose sales_stby'
dgmgrl / 'validate database sales_stby'

Here I would look at transport lag, apply lag, the recovery state, and any Broker warnings. Since I requested Active Data Guard, I would also check that the standby is open read-only with apply running.

The completed build is a good starting point. Before relying on it for disaster recovery, I still want to validate the configuration, check the lag, and test the role transitions.

4 Back up the cloud standby to Autonomous Recovery Service

Now I have the database at the third site. The next step is to give it a recovery history. Data Guard keeps the standby current; the backups give me earlier recovery points to work with if a problem has already reached that standby.

For an ExaDB-D standby exposed through a supported OCI Data Guard association, Oracle documents this console workflow in its standby backup and restore tutorial:

1. Open the cloud standby's database details page. Confirm I am working with sales_stby.
2. Choose Enable automatic backups and select Autonomous Recovery Service as the destination.
3. Set the retention policy and backup schedule, then save the configuration.
4. Check the backup details and confirm a successful backup. The tutorial also shows creating a database from a standby backup, which is a useful recovery test.

NOTE: The hybrid command does not configure Recovery Service backups. If the console does not expose the backup action for the CLI-created hybrid database, confirm the supported backup configuration procedure for that deployment before continuing. Check the corresponding service guidance for Database@ deployments as well.

DO NOT configure Real-time redo when configuring the Autonomous Recovery service.  Real-time redo requires creating a user in the primary database which out of the control of the tooling.

Backup settings can be fine tuned and these settings are described in the Exadata backup guide.

Test the third-site recovery process

My final check is a recovery exercise: choose a recovery point, restore into the intended recovery environment, verify the keys are available, and validate the database and application data.

Oracle 19c: Cloning Encrypted PDBs and Moving TDE Keys

Oracle TDE key hierarchy. This walkthrough uses file-based wallets in Oracle Database 19c.

When cloning or refreshing an encrypted pluggable database (PDB) in Oracle Database 19c, it is easy to focus on the datafiles. Restore the backup, recover the database, and plug the PDB into the destination container database (CDB).

But don't forget the keys. With Transparent Data Encryption (TDE), the destination also needs access to the master encryption keys stored in the source wallet.

In this post I will walk through two ways to bring the keys with an encrypted PDB, and then show where those steps fit when the PDB is restored from backup. I am using Oracle Database 19c, local file-based TDE wallets, and united mode. In united mode the PDBs use the CDB's wallet, while keeping their own master encryption keys.

This builds on my posts about encrypting an Oracle PDB with TDE and Oracle wallet files for TDE and authentication.

NOTE: These are command examples to adapt and validate on your 19c Release Update. Database names, paths, passwords, and the recovery SCN are placeholders. This is a walkthrough of the key handling, not output from a completed lab.

For unplug/plug, I can bring the keys with ENCRYPT USING and DECRYPT USING, or export and import them separately. A refresh from RMAN backups also needs the historical keys during recovery into the auxiliary CDB.

In this post: Check the wallet · Unplug/plug with keys · Export and import keys · Refresh from RMAN backups · Rekey and verify · Merge wallets

TDE wallet keys: what needs to move with a PDB?

TDE has two levels of keys. The tablespace key encrypts the data, and the TDE master encryption key protects that key. The master key is stored in the keystore. Copying encrypted datafiles does not, by itself, give a different CDB access to that master key. Oracle's TDE overview

Figure 1. Move both the encrypted PDB datafiles and the required master keys into the destination environment.

For these examples I am using the following names.

Name	Purpose
PRODCDB / APPPDB	Source CDB and encrypted PDB
AUXCDB / APPPDB	Temporary CDB used for a restore from backup
TESTCDB / APP_REFRESH	Existing destination CDB and the new refresh PDB
SourceWalletPassword	Password for the source wallet
TargetWalletPassword	Password for the destination wallet
TransportSecret	Separate secret protecting keys during transport

The transport secret and the wallet passwords have different jobs. I do not need to make the source and destination wallet passwords match.

STEP #1 - Check the TDE wallet configuration

On each CDB, I first check the configuration and wallet status.

-- SQL*Plus, connected to the appropriate CDB root
SHOW CON_NAME
SHOW PARAMETER wallet_root
SHOW PARAMETER tde_configuration

SELECT con_id, status, wallet_type, keystore_mode,
       wrl_parameter
FROM   v$encryption_wallet
ORDER BY con_id;

The examples assume TDE_CONFIGURATION='KEYSTORE_CONFIGURATION=FILE', with the united wallet under WALLET_ROOT/tde. A PDB row should show UNITED; the root can show NONE. An OPEN wallet is a useful first check, but it does not prove that it contains every key needed for this refresh. Wallet status reference

For a password wallet that is closed, open it from the root. Use the password belonging to that CDB.

ADMINISTER KEY MANAGEMENT SET KEYSTORE OPEN
  IDENTIFIED BY "SourceWalletPassword"
  CONTAINER = ALL;

If auto-login already has the wallet open, check its state rather than blindly repeating the open. The key-management examples below use FORCE KEYSTORE where supported to allow access to the password wallet. Have the ewallet.p12 password available even when normal database startup uses cwallet.sso. Configuring TDE

I also record the source PDB's keys before the move.

ALTER SESSION SET CONTAINER = APPPDB;

SELECT key_id, creation_time, activation_time
FROM   v$encryption_keys
ORDER BY activation_time;

This gives me key identifiers to compare later. I am recording metadata here, not displaying the secret key material. Encryption key view

Run the key operations as an appropriately privileged key administrator (SYSKM or ADMINISTER KEY MANAGEMENT), and the PDB operations with the necessary database privileges. The examples assume both are available to the administrative session.

METHOD #1 - Unplug and plug an encrypted PDB with its TDE keys

For a planned unplug/plug, I can protect the PDB's keys with a transport secret as part of the unplug operation.

1) Unplug on the source

-- PRODCDB root; source wallet must be open
ALTER SESSION SET CONTAINER = CDB$ROOT;

ALTER PLUGGABLE DATABASE APPPDB CLOSE IMMEDIATE;

ALTER PLUGGABLE DATABASE APPPDB
  UNPLUG INTO '/secure_stage/apppdb.xml'
  ENCRYPT USING "TransportSecret";

ENCRYPT USING protects the transported keys. The XML file is a manifest; the PDB's datafiles still have to be made available on the destination host. A .pdb archive is an alternative packaging format. Unplugging PDBs

NOTE: Unplugging takes this source PDB out of service. For a refresh that must leave production running, use the backup/auxiliary workflow below and unplug the restored copy.

2) Check compatibility at the destination

For this example, the XML and datafiles are accessible at the paths recorded in the XML. If the files were staged at different paths, account for that with SOURCE_FILE_NAME_CONVERT; FILE_NAME_CONVERT controls the destination copies.

-- TESTCDB root
SET SERVEROUTPUT ON
DECLARE
  can_plug BOOLEAN;
BEGIN
  can_plug := DBMS_PDB.CHECK_PLUG_COMPATIBILITY(
    pdb_descr_file => '/secure_stage/apppdb.xml',
    pdb_name       => 'APP_REFRESH');
  IF can_plug THEN
    DBMS_OUTPUT.PUT_LINE('Compatible');
  ELSE
    RAISE_APPLICATION_ERROR(-20001,
      'Review PDB_PLUG_IN_VIOLATIONS before continuing');
  END IF;
END;
/

Resolve compatibility errors before creating the PDB, including applicable patch, component, and character-set issues. Plug compatibility check

3) Plug in the copy

The destination wallet must already exist and be open. APP_REFRESH must be a new PDB name in TESTCDB.

-- TESTCDB root; filesystem paths used for this example
CREATE PLUGGABLE DATABASE APP_REFRESH AS CLONE
  USING '/secure_stage/apppdb.xml'
  COPY
  FILE_NAME_CONVERT =
    ('/u02/oradata/PRODCDB/APPPDB/',
     '/u02/oradata/TESTCDB/APP_REFRESH/')
  KEYSTORE IDENTIFIED BY "TargetWalletPassword"
  DECRYPT USING "TransportSecret";

ALTER PLUGGABLE DATABASE APP_REFRESH OPEN;

Here, AS CLONE gives the plugged copy a new identity, and COPY creates its datafiles at the destination. DECRYPT USING supplies the transport secret, while KEYSTORE IDENTIFIED BY supplies the destination wallet password. Adapt file placement for OMF or ASM. CREATE PLUGGABLE DATABASE reference

Finish with the rekey and verification steps below. An initial restricted open is not the end of the process.

METHOD #2 - Export and import TDE keys for an Oracle 19c PDB

This is an alternative to Method #1. I use a separate export file when I want the key transfer to be an explicit step in the refresh process.

1) Export from inside the source PDB

-- PRODCDB, or AUXCDB after restoring the PDB
ALTER SESSION SET CONTAINER = APPPDB;

ADMINISTER KEY MANAGEMENT EXPORT ENCRYPTION KEYS
  WITH SECRET "TransportSecret"
  TO '/secure_stage/apppdb_keys.exp'
  FORCE KEYSTORE
  IDENTIFIED BY "SourceWalletPassword";

ALTER SESSION SET CONTAINER = CDB$ROOT;
ALTER PLUGGABLE DATABASE APPPDB CLOSE IMMEDIATE;
ALTER PLUGGABLE DATABASE APPPDB
  UNPLUG INTO '/secure_stage/apppdb.xml';

Export inside the PDB. Do not add a WITH IDENTIFIER IN filter: a PDB export carries its keys and the metadata identifying the active key. Protect the export file and deliver its secret separately. Exporting keys leaves the source keys in place. Key export syntax

2) Import into the destination root before plugging, when required

If SYSTEM, SYSAUX, UNDO, or TEMP is encrypted, first import into TESTCDB's root. Then import again inside the new PDB to associate the keys with it. For a PDB without those encrypted system tablespaces, the pre-plug root import can be skipped. Oracle's united-mode PDB procedure

-- TESTCDB root: conditional pre-plug import
ALTER SESSION SET CONTAINER = CDB$ROOT;

ADMINISTER KEY MANAGEMENT IMPORT ENCRYPTION KEYS
  WITH SECRET "TransportSecret"
  FROM '/secure_stage/apppdb_keys.exp'
  FORCE KEYSTORE
  IDENTIFIED BY "TargetWalletPassword"
  WITH BACKUP USING 'before_refresh_root_import';

3) Plug in, open the PDB wallet, and import inside the PDB

Run the compatibility check from Method #1 first. This manifest was created without ENCRYPT USING, so this branch does not use DECRYPT USING.

-- TESTCDB root
CREATE PLUGGABLE DATABASE APP_REFRESH AS CLONE
  USING '/secure_stage/apppdb.xml'
  COPY
  FILE_NAME_CONVERT =
    ('/u02/oradata/PRODCDB/APPPDB/',
     '/u02/oradata/TESTCDB/APP_REFRESH/')
  KEYSTORE IDENTIFIED BY "TargetWalletPassword";

ALTER SESSION SET CONTAINER = APP_REFRESH;

-- If the PDB's password wallet is not already open:
ADMINISTER KEY MANAGEMENT SET KEYSTORE OPEN
  FORCE KEYSTORE
  IDENTIFIED BY "TargetWalletPassword";

ALTER SESSION SET CONTAINER = CDB$ROOT;
ALTER PLUGGABLE DATABASE APP_REFRESH OPEN;

ALTER SESSION SET CONTAINER = APP_REFRESH;

ADMINISTER KEY MANAGEMENT IMPORT ENCRYPTION KEYS
  WITH SECRET "TransportSecret"
  FROM '/secure_stage/apppdb_keys.exp'
  FORCE KEYSTORE
  IDENTIFIED BY "TargetWalletPassword"
  WITH BACKUP USING 'before_refresh_pdb_import';

The PDB may initially open restricted. Complete the PDB import, rekey, and reopen before releasing it to the application. The root import and PDB import serve different purposes; do not treat the root import as completion of both steps.

Refresh an encrypted PDB from RMAN backups

I separate this into two handoffs: backup to auxiliary CDB, followed by auxiliary PDB to the existing destination CDB.

Figure 2. A backup refresh has two key handoffs: recovery into AUXCDB, then plug-in and key transfer into TESTCDB.

1) Make the historical keys available to the auxiliary

The auxiliary needs the keys required by the backup and recovery interval. Rotating the production master key does not eliminate the need for older keys. A wallet backup that predates a required rotation may be missing a key; a later wallet can contain the historical keys if they have been retained. TDE key history

For a newly prepared auxiliary, securely provision a copy of the source password wallet at its configured WALLET_ROOT/tde location. Use a wallet with the required CDB and PDB key history, not just a PDB key export. The duplicate also restores root and seed files. Keep the auxiliary's wallet separate from the live source and from TESTCDB's wallet. Preparing the auxiliary keystore

For example, the auxiliary initialization settings include:

db_name='AUXCDB'
db_unique_name='AUXCDB'
enable_pluggable_database=TRUE
wallet_root='/u01/app/oracle/admin/AUXCDB/wallet'
tde_configuration='KEYSTORE_CONFIGURATION=FILE'

This is only the relevant parameter excerpt. Prepare the auxiliary in NOMOUNT, with its own controlfile, datafile, redo, and recovery-area destinations, plus the password file and Oracle Net configuration required by your duplication method.

2) Restore the PDB into the temporary CDB

Here is a backup-based duplication example, with RMAN connected to PRODCDB as TARGET and AUXCDB as AUXILIARY. Both connections are to the root. The source backup pieces must be accessible to the auxiliary, including root, seed, PDB, controlfile, and the archived redo needed to reach the chosen SCN.

-- RMAN: connected to the two CDB roots
SET DECRYPTION WALLET OPEN
  IDENTIFIED BY 'SourceWalletPassword';

RUN {
  SET UNTIL SCN 123456789;
  SET NEWNAME FOR DATABASE TO '/u02/oradata/AUXCDB/%U';

  DUPLICATE TARGET DATABASE TO AUXCDB
    PLUGGABLE DATABASE APPPDB
    LOGFILE
      GROUP 1 ('/u02/oradata/AUXCDB/redo01.log') SIZE 200M,
      GROUP 2 ('/u02/oradata/AUXCDB/redo02.log') SIZE 200M,
      GROUP 3 ('/u02/oradata/AUXCDB/redo03.log') SIZE 200M;
}

Replace the SCN and file layout with values for your restore. This creates a new auxiliary CDB containing the selected PDB, not a PDB directly inside TESTCDB. Oracle 19c's DUPLICATE PLUGGABLE DATABASE ... TO existing_cdb form supports active duplication; it is not the backup-based command used here. RMAN DUPLICATE reference

SET DECRYPTION WALLET OPEN supplies the wallet password across auxiliary restarts during duplication. It does not transfer missing keys. If the backup pieces have a separate password-encryption layer, also supply the applicable backup password using SET DECRYPTION IDENTIFIED BY. That password does not replace the keys needed for the TDE-encrypted datafiles. RMAN SET reference

3) Transfer from AUXCDB into TESTCDB

After successful recovery and opening the restored PDB, use either Method #1 or Method #2, with AUXCDB as the source.

Create a fresh manifest from the restored PDB, and export its keys there if using Method #2. Update FILE_NAME_CONVERT to match the restored files' actual paths. In the RMAN example, %U generates filenames directly under /u02/oradata/AUXCDB/; inspect those filenames rather than assuming an APPPDB subdirectory exists.

I would create APP_REFRESH alongside the current test PDB, validate it, and then handle the application/service cutover. The destructive replacement of an existing PDB is a separate decision. This is a repeated refresh from backups, not a REFRESH MODE clone.

Rekey the cloned PDB, reopen, and verify

Once the imported keys are available, create and activate a destination master key inside APP_REFRESH.

-- TESTCDB, inside the new PDB only
ALTER SESSION SET CONTAINER = APP_REFRESH;

ADMINISTER KEY MANAGEMENT SET KEY
  FORCE KEYSTORE
  IDENTIFIED BY "TargetWalletPassword"
  WITH BACKUP USING 'app_refresh_rekey'
  CONTAINER = CURRENT;

ALTER SESSION SET CONTAINER = CDB$ROOT;
ALTER PLUGGABLE DATABASE APP_REFRESH CLOSE IMMEDIATE;
ALTER PLUGGABLE DATABASE APP_REFRESH OPEN;

Use CONTAINER=CURRENT for this PDB; there is no reason for a refresh to rotate every PDB's key. Changing the wallet password is also a different operation from generating a new master key. Key-management operations

Rekeying gives the destination a new master key. It does not make historical backups independent of their original keys. Retain those keys for as long as the corresponding recovery requirements exist. TDE key architecture FAQ

Now I check the result.

-- TESTCDB root
SELECT name, open_mode, restricted
FROM   v$pdbs
WHERE  name = 'APP_REFRESH';

SELECT time, name, cause, type, message, status, action
FROM   pdb_plug_in_violations
WHERE  name = 'APP_REFRESH'
AND    status <> 'RESOLVED'
ORDER BY time;

ALTER SESSION SET CONTAINER = APP_REFRESH;

SELECT status, keystore_mode, fully_backed_up
FROM   v$encryption_wallet;

SELECT key_id, creation_time, activation_time
FROM   v$encryption_keys
ORDER BY activation_time;

SELECT tablespace_name, encrypted
FROM   dba_tablespaces
ORDER BY tablespace_name;

I want READ WRITE, RESTRICTED=NO, and all blocking plug-in violations resolved. I also read known application data from an encrypted tablespace and check the expected refresh timestamp or business totals. Opening the PDB alone is not my application validation. Completing a PDB plug-in

Finally, back up the updated destination wallet and the refreshed PDB. WITH BACKUP protects the wallet before a change; I also want a retained copy containing the newly created key.

-- TESTCDB root, after successful rekey
ALTER SESSION SET CONTAINER = CDB$ROOT;

ADMINISTER KEY MANAGEMENT BACKUP KEYSTORE
  USING 'after_app_refresh'
  FORCE KEYSTORE
  IDENTIFIED BY "TargetWalletPassword"
  TO '/secure_wallet_backups/TESTCDB';

Keep password-wallet backups and their passwords recoverable, with access controlled separately from the database backups. An auto-login wallet stored beside encrypted backups weakens that separation. RMAN encryption and wallet backup guidance

Migrate TDE wallet keys with MERGE KEYSTORE

For one PDB, I would normally use the PDB export/import above. A wallet merge has broader scope: it adds the source wallet's keys to another wallet.

Here is a merge example using staged wallet copies, outside the live wallet directories:

-- Optional wallet administration example; not a PDB refresh step
ADMINISTER KEY MANAGEMENT
  MERGE KEYSTORE '/secure_stage/source_wallet'
    IDENTIFIED BY "SourceWalletPassword"
  INTO EXISTING KEYSTORE '/secure_stage/target_wallet_copy'
    IDENTIFIED BY "TargetWalletPassword"
  WITH BACKUP USING 'before_wallet_merge';

This updates the staged destination password wallet and leaves the source unchanged. It does not switch the live database to that staged wallet. A planned wallet migration must also configure the final location, reopen the merged wallet, and update or recreate its auto-login companion as appropriate. Coordinate that change across all consumers of the wallet. Merging and relocating wallets

NOTE: Do not copy the source ewallet.p12 over the wallet of an existing destination CDB. That destination may need keys for other PDBs. Also, MOVE KEYS removes selected keys from the source keystore and is not a substitute for a clone's export/import. MIGRATE USING is for changing the keystore provider, such as migrating to Oracle Key Vault. Key-management SQL reference

The operational check I would add to every refresh job is simple: record which backup was restored, which keys were supplied, where the destination wallet was backed up, and whether encrypted application data was successfully read. Those four items make the next refresh, and the next recovery exercise, much easier to review.

Oracle Wallet file settings

 I have been doing more research on how to set wallet locations for the Oracle database as wallets are becoming more and more important to secure your database.

Oracle wallet usage

I have found that there are different types of wallets, and  different ways to set the Wallet location depending on it's usage.

1) WALLET_ROOT

Wallet_root is a Database setting (spfile) that is used by a running database.  Below are the different sub-directories that can be created within wallet root and what they can be used for.

The most commonly used directories are

TDE - The wallet in this directory contains the encryption key(s) for the database and replaces the setting ENCRYPTION_WALLET_LOCATION in the sqlnet.ora.

TLS - The wallet in this directory contains the TLS certificate(s) used by the database to make secure TCPS connections.

SERVER_SEPS - The wallet in this location contains the login credentials for other databases or even the same database. Some examples of when this can be useful are.

• Sending real-time redo to a ZDLRA
• Creating Database Links to other databases
• RMAN channel configurations that connect to different nodes in a RAC cluster.

* Note : It was pointed out to me, that using the wallet in server_seps for RMAN channel connects will replace the usage of the "connect ..." string when allocating RMAN channels.

Subdirectory	Component / Purpose	Typical File Types	Notes & Details
tde	Transparent Data Encryption (TDE)	ewallet.p12, ewallet_*.p12	Created automatically when establishing TDE keystores. Contains master encryption keys for CDB$ROOT, non-CDBs, or isolated PDBs.
tde_seps	TDE Auto-Login / SEPS	cwallet.sso	Stores the Secure External Password Store (SEPS) auto-login file used for automatic opening of the TDE keystore.
tls	Transport Layer Security (TLS/SSL)	ewallet.p12, cwallet.sso	Stores public key infrastructure (PKI) certificates and private keys used for encrypted network communications.
eus	Enterprise User Security (EUS)	ewallet.p12, cwallet.sso	Contains credentials for centralized directory service authentication (e.g., Oracle Internet Directory / LDAP).
xdb_wallet	XML Database (XDB) Security	ewallet.p12	Used for securing Oracle XML DB HTTP, HTTPS, and FTP server connections.
server_seps	Server-Side Credential Store	cwallet.sso, ewallet.p12	Used in newer releases (e.g., Oracle 23ai) for passwordless server-side connections and integrations (such as Recovery Appliance).
mfa	Multi-Factor Authentication (MFA)	ewallet.p12, cwallet.sso	Holds certificates and PKI credentials used for native Multi-Factor Authentication integrations (e.g., OMA or Duo). Introduced in Oracle 19.28+.
bctable	Blockchain Tables	ewallet.p12, cwallet.sso	Stores the PKI private key and certificates of the blockchain table owner. Required for signing and verifying rows using the DBMS_BLOCKCHAIN_TABLE package.
<PDB_GUID>	Isolated Pluggable Database Root	Subdirectories (e.g., /tde, /tls)	A 128-bit GUID folder automatically generated per isolated PDB to isolate keystores from CDB$ROOT and other PDBs.

WALLET_ROOT Directory Structure

WALLET_ROOT/
├── bctable/                    # Blockchain Tables PKI store
│   └── ewallet.p12
├── eus/                        # Enterprise User Security credentials
│   └── ewallet.p12
├── mfa/                        # Multi-Factor Authentication wallet (19.28+)
│   └── ewallet.p12
├── server_seps/                # Server-side credential store
│   └── cwallet.sso
├── tde/                        # Master CDB/non-CDB TDE wallet
│   └── ewallet.p12
├── tde_seps/                   # CDB Auto-login keystore
│   └── cwallet.sso
├── tls/                        # TLS/SSL certificate wallet
│   └── ewallet.p12
├── xdb_wallet/                 # XML DB security wallet
│   └── ewallet.p12
└── <PDB_GUID>/                 # Isolated PDB directory
    ├── bctable/
    │   └── ewallet.p12
    ├── mfa/
    │   └── ewallet.p12
    ├── tde/                    # Isolated PDB TDE wallet
    │   └── ewallet.p12
    ├── tde_seps/
    │   └── cwallet.sso
    └── tls/
        └── cwallet.sso

2) SQLNET.ORA

The sqlnet.ora file is used by the database (during startup) and by client sessions.  Client sessions can override which sqlnet.ora is being used by setting TNS_ADMIN.

There are two locations that can be set in the sqlnet.ora file.

ENCRYPTION_WALLET_LOCATION 

This parameter is being deprecated in Oracle 26ai and it is being replaced with WALLET_ROOT mentioned previously.  WALLET_ROOT allows for easily setting individual wallet locations for each database sharing the same $ORACLE_HOME, along with the ability to set individual wallet locations for each PDB. it is recommended to no longer use this setting and migrate to WALLET_ROOT.

WALLET_LOCATION

Wallet_location is used for a number of wallets entries including SEPS, TLS, and EUS.  Because of this you need to be careful when setting the general wallet_location in the default location ($ORACLE_HOME/network/admin).  With consolidation, many environments today have multiple databases sharing the same $ORACLE_HOME and those databases often require separate, unique wallet files files for each database. 

It is best to create a separate sqlnet.ora directory for each database outside of the $ORACLE_HOME and utilize TNS_ADMIN to set your environment.

This parameter is being deprecated for use by the Oracle Database server (during startup), but not by client connections.

3) TNSNAMES.ORA

The tnsnames.ora file is used to make the connection to the database and it is possible set some wallet locations as part of the connection string.

This setting is done with the the (SECURITY= ...) block of the connect string.

SEPS_WALLET_LOCATION

This setting can be used to set the location of the SEPS wallet for each client connection. This can be useful if you make multiple connects in the same client session and wish to use different SEPS wallets for each connection. 

WALLET_LOCATION

Wallet_location is used for a number of wallets entries including SEPS, TLS, and EUS.  The usage of wallet location in the database connect string is the same as it would be within the sqlnet.ora. Using this setting will allow you to isolate which wallets is used for each connection when using the same client session.