File Retention Lock was recently released on ZFSSA and I wanted to take the time to explain how to set the retention time and view the retention of locked files.
Below is an example of what happens. You can see that the files are locked until January 1st 2025
The best place to start for information on how this works is by looking at my last blog post on authorizations.
First I will go through the settings that available at the share/project level
Grace period
The grace period is used to automatically lock a file when there has not been updates to the file for this period of time.
If the automatic file retention grace period is "0" seconds, then the default retention is NOT in effect.
NOTE: even with a grace period of "0" seconds files can be locked by manually setting a retention period.
Also, once a grace period is set (> "0") it cannot be increased or disabled.
Finally, if you set the grace period to a long period (to ensure all writes are to a file are completed), you can lock the file by removing the write bit. This does the same thing as expiring the grace period.
Below is an example
chmod ugo-w *
Running the "chmod" will remove the write bit, and immediate cause all files to lock.
Default retention
The most common method to implement file retention is by using the default retention period. This causes the file to be locked for the default retention when the grace period expires for a file.
Note that the file is locked as of the time the grace period expires. For example, if I have a grace period of 1 day (because I want the ability to clean up a failed backup) and a default file retention period of 14 days, the file will be locked for 14 days AFTER the 1 day grace period. The lock on the file will expire 15 days after the file was last written to.
In the example above you can see that all files created on this share are created with a default retention of 1 day (24 hours).
NOTE: If the grace period is not > "0' these settings will be ignored and files will not be locked by default.
.
Minimum/Maximum File retention
The second settings you see on the image above are the "minimum file retention period" and the "maximum file retention period".
These control the retention settings on files which follows the rules below.
The default retention period for files MUST be at least the minimum file retention period, and not greater than the maximum file retention period.
If the retention date is set manually on a file, the retention period must fall within the minimum and maximum retention period.
Display current Lock Expirations.
In order to display the lock expiration on Linux the first thing you need to do is to change the share/project setting "Update access time on read" to off . Through the CLI this is "set atime=false" on the share.
Once this settings is made, the client will then display the lock time as the "atime". In my example at the top of the blog, you can see by executing "ls -lu" the file lock time is displayed.
NOTE: you can also use the find command to search for files using the "atime" This will allow to find all the locked files.
Below is an example of using the find command to list files that have an lock expiration time in the future.
It is possible to set a specific date/time that a file is locked until. You can even set the retention date on a file that currently locked (it must be a date beyond the current lock data).
NOTE: If you try to change the retention date on a specific file, the new retention date has to be greater than current retention date (and less than or equal to the maximum file retention period). This makes sense. You cannot lower the retention period for a locked file.
Now how do you manually set the retention date ? Below is an example of how it is set for a file.
Setting File retention lock
There are 3 steps that are needed to lock the file with a specific lock expiration date.
1. Touch the file and set the access date. This can be done with
"-a" to change the access date/time
"-d" or "-t" to specify the date format
2. Remove the write bit with chmod guo-2
3. execute a cmod to make the file read only.
Below is an example where I am taking a file that does not contain retention, and setting the date to January 1, 2025.
First I am going to create a file and touch it setting the atime to a future data.
ZFS File Retention authorizations is important to understand if you plan on implementing retention lock on ZFS. This feature was added in release OS8.8.46. and there is a MOS note explaining how it works (2867335.1 - Understanding ZFS Appliance File Retention Policy).
In order to start using the new features, you need to grant some new authorizations that manage who can administer the new settings. Be aware that these new authorizations are NOT granted to the administrator role. You must add them to the administrator role or create an additional role.
The image above shows the File Retention Policies that can be set and which authorization is needed to administer each setting.
NOTE: The share must be created with file retention in order to have these settings take effect. You cannot add file retention to an existing Project/Share.
Now let's go through the 3 Authorizations and what they allow the administrator to do.
retentionPeriods
When an administrator is granted the "retentionPeriods" authorization they are given the authority to administer 3 of the setting for file retention
"Minimum file retention period" - This is the minimum amount of time in the future that you can set a file retention to be. If you set the file retention date manually the retention time must be at least this far if not longer in the future. If you set the "Default file retention period", it must be at least the "Minimum file retention period" if not longer. The default value for this setting is "0 seconds".
"Maximum file retention period"- This is the maximum amount of time in the future that you can set a file retention to be. If you set the file retention date manually the retention time must at most this far if not shorter in the future. If you set the "Default file retention period", it must be at most the "maximum file retention period" if not shorter. The default value for this setting is "5 years".
"Default file retention period"- This is the default amount of time in the future that you can set a file retention to be. This value has to fall within the minimum and maximum file retention period. Unless this value is set to a value greater than "0 seconds" no files are locked by default.
NOTE : The most common method used to lock files is to set the "Default file retention period" to a value greater than '0 seconds". When this is set (and file retention is turned on) any files created will be locked for this period of time.
retentionAuto
When an administrator is granted the "retentionAuto" authorization they are given the authority to set the Automatic file retention grace period.
This value controls how long after the last access time the ZFS waits to lock the file. The default setting is "0 seconds". Until this value is set to a value greater than "0 seconds" no files are automatically locked (using the Default file retention period). The only method to lock files when this value is left as "0", the default, is to manually lock files.
NOTE: A very important item to understand is that the ZFS locks the file once it has not been updated for this period of time. If you have a process that holds a file open without writing to it, for example an RMAN channel, it may lock the file before it is closed.
Be sure to set the grace period to be longer than the amount of time a process may pause writing to a file. DO NOT set it too short. If you wish to lock a file immediately after you have finished writing to it (because you have a long grace period) you can remove the "w" bit from the files using chmod. This will bypass the grace period.
If the share is configured with mandatory retention, the automatic grace period cannot be increased, it can only be lowered.
retentionMandatory
When an administrator is granted the "retentionMandatory" authorization they are given the authority to create a share with a "mandatory (no override)" file retention. This authorization is not necessary to create a "privileged override" file system.
Be aware that in order to create a file system with "mandatory" file retention the ZFS must be configured with the following settings. The "file retention" service must be running, and the file system needs to be a mirrored configuration
Remote root user login via the BUI/REST needs to be turned off in the HTTPS service
Remote root login via SSH needs to be turned off in the SSH service
NTP sync needs to be configured in the NTP service
NTP service needs to be on-line.
NOTE : You must ensure that the ZFS administrator is granted these authorizations before attempting to configure file retention. If the administration user is not granted the proper authorization you will permission errors like below.
"You are not authorized to perform this action. If you wish to proceed, contact an administrator to obtain the proper credentials.
Migrating an Oracle database from on-premise to OCI is especially challenging when the database is quite large. In this blog post I will walk through the steps to migrate to OCI leveraging an on-disk local backup copied to object storage.
The basic steps to perform this task are on on the image above.
Step #1 - Upload backup pieces to object storage.
The first step to migrate my database (acmedb) is to copy the RMAN backup pieces to the OCI object storage using the OCI Client tool.
In order to make this easier, I am breaking this step into a few smaller steps.
Step #1A - Take a full backup to a separate location on disk
This can also be done by moving the backup pieces, or creating them with a different backup format. By creating the backup pieces in a separate directory, I am able to take advantage of the bulk upload feature of the OCI client tool. The alternative is to create an upload statement for each backup piece.
For my RMAN backup example (acmedb) I am going to change the location of the disk backup and perform a disk backup. I am also going to compress my backup using medium compression (this requires the ACO license). Compressing the backup sets allows me to make the backup pieces as small as possible when transferring to the OCI object store.
Below is the output from my RMAN configuration that I am using for the backup.
RMAN> show all;
RMAN configuration parameters for database with db_unique_name ACMEDBP are:
CONFIGURE CONTROLFILE AUTOBACKUP ON;
CONFIGURE DEVICE TYPE DISK PARALLELISM 4 BACKUP TYPE TO COMPRESSED BACKUPSET;
CONFIGURE CHANNEL DEVICE TYPE DISK FORMAT '/acmedb/ocimigrate/backup_%d_%U';
CONFIGURE COMPRESSION ALGORITHM 'MEDIUM' AS OF RELEASE 'DEFAULT' OPTIMIZE FOR LOAD TRUE;
I created a new level 0 backup including archive logs and below is the "list backup summary" output showing the backup pieces.
List of Backups
===============
Key TY LV S Device Type Completion Time #Pieces #Copies Compressed Tag
------- -- -- - ----------- --------------- ------- ------- ---------- ---
4125 B A A DISK 21-JUN-22 1 1 YES TAG20220621T141019
4151 B A A DISK 21-JUN-22 1 1 YES TAG20220621T141201
4167 B 0 A DISK 21-JUN-22 1 1 YES TAG20220621T141202
4168 B 0 A DISK 21-JUN-22 1 1 YES TAG20220621T141202
4169 B 0 A DISK 21-JUN-22 1 1 YES TAG20220621T141202
4170 B 0 A DISK 21-JUN-22 1 1 YES TAG20220621T141202
4171 B 0 A DISK 21-JUN-22 1 1 YES TAG20220621T141202
4172 B 0 A DISK 21-JUN-22 1 1 YES TAG20220621T141202
4173 B 0 A DISK 21-JUN-22 1 1 YES TAG20220621T141202
4174 B 0 A DISK 21-JUN-22 1 1 YES TAG20220621T141202
4175 B 0 A DISK 21-JUN-22 1 1 YES TAG20220621T141202
4176 B 0 A DISK 21-JUN-22 1 1 YES TAG20220621T141202
4208 B A A DISK 21-JUN-22 1 1 YES TAG20220621T141309
4220 B F A DISK 21-JUN-22 1 1 YES TAG20220621T141310
From the output you can see that there are a total of 14 backup pieces
3 Archive log backup sets (two created before the backup of datafiles, and one after).
TAG20220621T141019
TAG20220621T141201
TAG20220621T141309
10 Level 0 datafile backups
TAG20220621T141202
1 controlfile backup
TAG20220621T141310
Step #1B - Create the bucket in OCI and configure OCI Client
Now we need a bucket to upload the 14 RMAN backup pieces to.
Before I can upload the objects, I need to download and configure the OCI Client tool. You can find the instructions to do this here.
Once the client tool is installed I can create the bucket and verify that the OCI Client tool is configured correctly.
The command to create the bucket is.
Below is the output when I ran it for my compartment and created the bucket "acmedb_migrate"
Step #2 - Create the manifest for the backup pieces.
The next step covers creating the "metadata.xml" for each object which is the manifest the the RMAN library uses to read the backup pieces.
Again this is broken down into a few different steps.
Step #2A - Download an configure the Oracle Database Cloud Backup Module.
The link for the instructions (which includes the download can be found here.
I executed the jar file which downloads/created the following files.
libopc.so - This is the library used by the Cloud Backup module, and I downloaded it into "/home/oracle/ociconfig/lib/" on my host
acmedb.ora - This is the configuration file for my database backup. This was created in "/home/oracle/ociconfig/config/" on my host
This information is used to allocate the channel in RMAN for the manifest.
Step #2b - Generate the manifest create for each backup piece.
The next step is to dynamically create the script to build the manifest for each backup piece. This needs to be done for each backup piece, and the command is
The script I am using to complete this uses backup information from the controlfile of the database, and narrows the backup pieces to just the pieces in the directory I created for this backup.
Step #2c - Execute the script with an allocated channel.
The next step is to execute the script in RMAN within a run block after allocating a channel to the bucket in object storage. This needs to be done for each backup piece. You create a run block with one channel allocation followed by "send" commands.
NOTE: This does not have be executed on the host that generated the backups. In the example below, I set my ORACLE_SID to "dummy" and performed create manifest with the "dummy" instance started up nomount.
Below is an example of allocating a channel to the object storage and creating the manifest for one of the backup pieces.
export ORACLE_SID=dummy
rman target /
RMAN> startup nomount;
startup failed: ORA-01078: failure in processing system parameters
LRM-00109: could not open parameter file '/u01/app/oracle/product/19c/dbhome_1/dbs/initdummy.ora'
starting Oracle instance without parameter file for retrieval of spfile
Oracle instance started
Total System Global Area 1073737792 bytes
Fixed Size 8904768 bytes
Variable Size 276824064 bytes
Database Buffers 780140544 bytes
Redo Buffers 7868416 bytes
RMAN> run {
allocate channel t1 device type sbt parms='SBT_LIBRARY=/home/oracle/ociconfig/lib/libopc.so ENV=(OPC_PFILE=/home/oracle/ociconfig/config/acmedb.ora)';
send channel t1 'export backuppiece backup_RADB_3r10k6ec_123_1_1';
}
2> 3> 4>
allocated channel: t1
channel t1: SID=19 device type=SBT_TAPE
channel t1: Oracle Database Backup Service Library VER=23.0.0.1
sent command to channel: t1
released channel: t1
Step #2d - Validate the manifest is created.
I logged into the OCI console, and I can see that there is a directory called "sbt_catalog". This is the directory containing the manifest files. Within this directory you will find a subdirectory for each backup piece. And within those subdirectories you will find a "metadata.xml" object containing the manifest.
Step #3 - Catalog the backup pieces.
The next step covers cataloging the backup pieces in OCI. You need to download the controlfile backup from OCI and start up mount the database.
Again this is broken down into a few different steps.
Step #3A - Download an configure the Oracle Database Cloud Backup Module.
The link for the instructions (which includes the download can be found here.
Again, you need to configure the backup module (or you can copy the files from your on-premise host).
Step #3b - Catalog each backup piece.
The next step is to dynamically create the script to build the catalog each backup piece. This needs to be done for each backup piece, and the command is
"catalog device type 'sbt_tape' backuppiece <object name>';
The script I am using to complete this uses backup information from the controlfile of the database, and narrows the backup pieces to just the pieces in the directory I created for this backup.
Step #3c - Execute the script with a configured channel.
I created a configure channel command, and cataloged the backup pieces that in the object store.
RMAN> CONFIGURE CHANNEL DEVICE TYPE 'SBT_TAPE' PARMS 'SBT_LIBRARY=/home/oracle/ociconfig/lib/libopc.so ENV=(OPC_PFILE=/home/oracle/ociconfig/config/acmedb.ora)';
run {
catalog device type 'sbt_tape' backuppiece 'backup_RADB_3r10k6ec_123_1_1';
catalog device type 'sbt_tape' backuppiece 'backup_RADB_3s10k6hh_124_1_1';
catalog device type 'sbt_tape' backuppiece 'backup_RADB_3t10k6hj_125_1_1';
catalog device type 'sbt_tape' backuppiece 'backup_RADB_3u10k6hj_126_1_1';
catalog device type 'sbt_tape' backuppiece 'backup_RADB_3v10k6hj_127_1_1';
catalog device type 'sbt_tape' backuppiece 'backup_RADB_4010k6hj_128_1_1';
catalog device type 'sbt_tape' backuppiece ' backup_RADB_4110k6hk_129_1_1';
catalog device type 'sbt_tape' backuppiece 'backup_RADB_4210k6id_130_1_1';
catalog device type 'sbt_tape' backuppiece 'backup_RADB_4310k6ie_131_1_1';
catalog device type 'sbt_tape' backuppiece 'backup_RADB_4410k6ie_132_1_1';
catalog device type 'sbt_tape' backuppiece 'backup_RADB_4510k6jh_133_1_1';
}
old RMAN configuration parameters:
CONFIGURE CHANNEL DEVICE TYPE 'SBT_TAPE' PARMS 'SBT_LIBRARY=/home/oracle/ociconfig/lib/libopc.so ENV=(OPC_PFILE=/home/oracle/ociconfig/config/acmedb.ora)';
new RMAN configuration parameters:
CONFIGURE CHANNEL DEVICE TYPE 'SBT_TAPE' PARMS 'SBT_LIBRARY=/home/oracle/ociconfig/lib/libopc.so ENV=(OPC_PFILE=/home/oracle/ociconfig/config/acmedb.ora)';
new RMAN configuration parameters are successfully stored
starting full resync of recovery catalog
full resync complete
RMAN>
RMAN> 2> 3> 4> 5> 6> 7> 8> 9> 10> 11> 12> 13>
allocated channel: ORA_SBT_TAPE_1
channel ORA_SBT_TAPE_1: SID=406 device type=SBT_TAPE
channel ORA_SBT_TAPE_1: Oracle Database Backup Service Library VER=23.0.0.1
allocated channel: ORA_SBT_TAPE_2
channel ORA_SBT_TAPE_2: SID=22 device type=SBT_TAPE
channel ORA_SBT_TAPE_2: Oracle Database Backup Service Library VER=23.0.0.1
allocated channel: ORA_SBT_TAPE_3
channel ORA_SBT_TAPE_3: SID=407 device type=SBT_TAPE
...
...
...
channel ORA_SBT_TAPE_4: SID=23 device type=SBT_TAPE
channel ORA_SBT_TAPE_4: Oracle Database Backup Service Library VER=23.0.0.1
channel ORA_SBT_TAPE_1: cataloged backup piece
backup piece handle=backup_RADB_4510k6jh_133_1_1 RECID=212 STAMP=1107964867
RMAN>
Step #3d - List the backups pieces cataloged
I performed a list backup summary to view the newly cataloged tape backup pieces.
RMAN> list backup summary;
List of Backups
===============
Key TY LV S Device Type Completion Time #Pieces #Copies Compressed Tag
------- -- -- - ----------- --------------- ------- ------- ---------- ---
4220 B F A DISK 21-JUN-22 1 1 YES TAG20220621T141310
4258 B A A SBT_TAPE 21-JUN-22 1 1 YES TAG20220621T141019
4270 B A A SBT_TAPE 21-JUN-22 1 1 YES TAG20220621T141201
4282 B 0 A SBT_TAPE 21-JUN-22 1 1 YES TAG20220621T141202
4292 B 0 A SBT_TAPE 21-JUN-22 1 1 YES TAG20220621T141202
4303 B 0 A SBT_TAPE 21-JUN-22 1 1 YES TAG20220621T141202
4315 B 0 A SBT_TAPE 21-JUN-22 1 1 YES TAG20220621T141202
4446 B 0 A SBT_TAPE 21-JUN-22 1 1 YES TAG20220621T141202
4468 B 0 A SBT_TAPE 21-JUN-22 1 1 YES TAG20220621T141202
4490 B 0 A SBT_TAPE 21-JUN-22 1 1 YES TAG20220621T141202
4514 B 0 A SBT_TAPE 21-JUN-22 1 1 YES TAG20220621T141202
4539 B 0 A SBT_TAPE 21-JUN-22 1 1 YES TAG20220621T141202
RMAN>
Step #4 - Restore the database.
The last step is restore the cataloged backup pieces. Remember you might have to change the location of the datafiles.
The process above can be used to upload and catalog both additional archive logs (to bring the files forward) and incremental backups to bring the database forward.
