Monday 27 May 2013

Inplace/Online Reorg phases

Online or inplace table reorganization allows the user to reorganize a table while permitting full access to that table. While inplace REORG provides uninterrupted user access to the data, the performance of inplace REORG is slower than classic or offline REORG.

During an inplace table reorganization, the entire table is not reorganized at once. Instead, portions of the table are reorganized incrementally. Data is not copied out to a temporary table space: rows are moved within the existing table object to re-establish clustering, reclaim free spaces, and eliminate overflow rows.

There are four primary phases for inplace table REORG:

  1. SELECT N pages
    During this phase, DB2 selects N pages, where N is the extent size with a minimum of 32 sequential pages for REORG processing.
  2. Vacate the range
    With the N pages selected, inplace table REORG moves all rows within this range to free pages within the table. Each row that is moved leaves behind an RP (REORG pointer) record that contains the RID of the row’s new location. The row is inserted to free pages in the table as a RO (REORG overflow) record that contains the data.
    Once REORG finishes moving a set of rows, it waits for all existing data accesses that are occurring in the table (for example, by currently executing applications) to complete. These existing accesses, called old scanners, use old the RIDs when accessing the table data. Any accesses that start during this waiting period, called new scanners, use the new RIDs to access the data. Once all of the old scanners have completed, REORG cleans up the moved rows, deleting the RP records and converting the RO records to normal records.
  3. Fill the range
    After all rows have been vacated, the rows are written back, in a reorganized format, sorted according to any indexes used, and obeying any PCTFREE restrictions defined. When all the pages in the range are filled, the next N sequential pages are selected in the table, and the process begins again.
  4. Truncate the table
    When all pages in the table have been reorganized, the table will be truncated to reclaim space by default. If the NOTRUNCATE option is specified, the reorganized table is not truncated.

Classic/Offline Reorg Phases

Offline table reorganization uses a shadow copy approach, building a full copy of the table that is being reorganized.

The shadow copy image is built within the table space that the table being reorganized resides in. Optionally, the copy can be built in a system temporary table space if the USE option is specified when issuing the REORG TABLE command.

There are four possible phases in a classic or offline table reorganization:

  1. SORT
    If an index is specified with the REORG TABLE command, or if a clustering index is defined on the table, the rows of the table are first sorted according to that index. If the INDEXSCAN option is specified, an index scan is used to sort the table, otherwise, a table scan sort is used. This phase only applies to a clustering REORG. Space reclaiming reorganizations begin at the build phase.
  2. BUILD
    In this phase, a reorganized copy of the entire table is build, either in the table space that the table being reorganized resides, or in a temporary table space specified with the REORG command.
  3. REPLACE
    In this phase, the original table object is replaced by either copying back from the temporary table space, or by pointing to the newly built object within the table space of the table being reorganized.
  4. RECREATE ALL INDEXES
    All indexes defined on the table are recreated.

You can monitor the progress of the table reorganization and determine which phase the process is currently in using the snapshot monitor or snapshot administrative views.

Online Reorg and Offline Reorg in db2 UDB


Offline reorg :
1. Database will be unavailable during offline reorg. If the db is quiesced then only instance owner will be able perform any operation .. other application will not be able to connect.

2. In offline reorg when we use "use tempspace_name" phrase then actually db2 creates a copy of that table in temp_tablespace by building structure of the table ,,, in second phase db2 starts to copy the data as well in proper way and removes dirty pages.. In last phase once this activity gets completed db2 removed the old table and places the newly organized table in its previous tablespace from temp_tablespace.
If we do not use temp_tablespace then at least double space of the table data size + index size is required to continue the offline reorg.

Online reorg :
In online reorg all application can connect to database .but applications perpormance gets slow.. hence creates lock-waits as well. Here db2 do not need temp_tablespace as db2 part by part reorganizes the table data.. Tables data are moved from one place to another place and leaves a row identifier to mention where the data pages are moved.Clean up only option is useful to clear the dirty pages.

But I personally feel online reorg is not useful for big tables. Most of the time it take s much more time compare to offline reorg timings.

Online reorg is compatible with online database backup...

Please correct me if my understanding is not correct ...

Friday 3 May 2013

DB2 Locking Basics

DB2 Locking basics

In any relational database, in order to maintain consistency among the transaction locks need to be acquired on the resource on which the transaction will be executed. Resource can be a Relational Table, or Rows. Locking provide isolation of resource between transaction thereby providing concurrency between independent transaction. DB2 provides four isolation levels.

  • Uncommitted Read
  • Cursor Stability
  • Read Stability
  • Repeatable Read
  • Currently committed (New in DB 9.7)
In order to utilize the isolation level in the transaction, they can be specified at

1) Statement level (SELECT, DELETE, UPDATE)
eg:- SELECT FROM WITH {UR, CS, RR, RS}

2) At session level
eg:- SET CURRENT ISOLATION = {UR, CS, RR, RS}


Commonly referred LOCKING terms

1) Locks
In order to use any resource, DB2 lock the resource before the transaction can use it.


2) Lock-Wait
If any transaction is requesting lock on a resource, and that resource is currently being held by other transaction, then the application requesting the lock goes into LOCK-WAIT mode until the lock on the resource is released by the primary transaction.

3) Deadlock
If two transaction are waiting on each others resources the the both the transaction goes into deadlock state.

4) Lock Escalation
For each lock acquired on a resource, DB2 maintain that information in DB2 locklist memory component. If the locklistmemory started getting exhausted, DB2 escalate all row level lock to table level lock in order to release memory.

How DB2 locking can be monitored?

1) Monitor switches
2) GET SNAPSHOT command
3) db2pd tool
4) Snapshot monitor SQL administrative routines
5) DB2_CAPTURE_LOCKTIMEOUT ( New in DB2 9.5 locking)
                                    Practical Use of DB2_CAPTURE_LOCKTIMEOUT
6) db2cos scripts
7) db2pdcfg with -catch option
8) Locking event monitor as apposed to Deadlock event monitory (New in DB2 9.7 Locking)

Lock related DB CFG parameter

1) LOCKLIST
2) MAXLOCK
3) LOCKTIMEOUT
4) DLCHKTIME

Lock related registry variable

1) DB2LOCK_TO_RB
2) DB2_KEEPTABLELOCK
3) DB2_MAX_NON_TABLE_LOCKS
4) DB2_EVALUNCOMMITED
5) DB2_SKIPDELETED
6) DB2_SKIPINSERTED

Lock monitoring - to get more details click here

Types of Recovery in Db2

Crash Recovery

It is a type of recovery which bring the database back to the consistent state after unplanned failure
It perform rollback operation of the transaction that were not committed and perform data page externalization for all the committed transactions

If the tablespace is damaged in circular logging database and cannot be made online then in order to start the database it needs to be dropped to start the database else database will not start
If the database is not started, try restarting it using RESTART command
                    db2 "restart database sample"

If there are corrupt tablespace the database restart will fail with SQL0290N error code
Then check the DB2 administration notification file for list of all tablespace that are in DROP PENDING state                       db2 get dbm cfg | grep -i diag
                              cd /home/db2inst1/sqllib/db2dump
                               vi db2inst1.nfy

Try restarting the database with DROP PEDNING clausedb2 "restart database sample drop pending tablespaces(userspace1,tbsp_taba)"

Click here to get more detail on DB2 Crash Recovery

Crash recovery can be automated or can be done manually
Enabling Automated crash recovery
Set DB CFG parameter “autorestart” to “ON”db2 "update db cfg for sample using autorestart on immediate"

Following the failure, DB2 will automatically restore the database to a consistent point by rolling forward and backup the changes from Active log files
Manually performing the Crash recovery
DB CFG parameter “autorestart” is set to “OFF”
Following the failure, “RESTART DATABASE” is issued to perform the crash recoverydb2 "restart database sample"


2)Version Recovery

  • It restores the database from the consistent full offline backup available.
  • Restore from only full offline backup operation is available.
  • Version recovery can be used with circular logging and archival logging database.
  • If the database is enabled for archival logging, then using only the FULL OFFLINE BACKUP of the database with the “WITHOUT ROLLING FORWARD” at the end of the “RESTORE” command can be used to perform version recovery.
  • All the transaction after the last full offline backup are lost in this type of recovery.
  • click here to get more details on DB2 Version Recovery

Version recovery example
Sr. No
Description
Sample DB2 Command
1Check if database is configured for Circular logging. If database is configured for Archival logging this step is not needed.db2 get db cfg for sample | grep -i log
               LOGARCHMETH1 = OFF
2Perform Offline backup of the database-- Deactivate the database
db2 deactivate db sample
-- Check if the database is deactivated. This command check for db2loggr process. If not listed then DB is deactivated
db2pd -edus | grep -i db2lo
-- Perform offline backup of the database
db2 “backup database sample to /backup compress”
3Identify the set of offline backup and determine the recovery pointdb2 “list history backup all for database sample”

Op Obj Timestamp+Sequence Type Dev Earliest Log Current Log  Backup ID
-- --- ------------------ ---- --- ------------ ------------ --------------
 B  D  20110520114322001   F    D  S0000023.LOG S0000023.LOG
----------------------------------------------------------------------------
 Contains 3 tablespace(s):

 00001 SYSCATSPACE
 00002 USERSPACE1
 00003 SYSTOOLSPACE
----------------------------------------------------------------------------
    Comment: DB2 BACKUP SAMPLE OFFLINE
Start Time: 20110520114322
  End Time: 20110520114345
    Status: A
----------------------------------------------------------------------------
 EID: 29 Location: /home/db2inst1/gilroy/bar
4Determine which version of the database is needed using the timestamp displayed from the “LIST HISTORY” command Details captured from previous command
1) Timestamp => 20110520114322001 
2) Backup image location =>/home/db2inst1/gilroy/bar
5.aVersion recovery onCircular Logging Databasedb2 “restore db sample from/home/db2inst1/gilroy/bar taken at20110520114322001 into testdb
5.b.Version recovery onArchival logging databasedb2 “restore db sample from/home/db2inst1/gilroy/bar taken at20110520114322001 into testdb without rolling forward

Redirect Restore

1) Redirect restore of the database using version recovery
a) DB is enable for archival logging.

b) Perform offline backup of the database

Offline backup is needed when the current database has to be redirected restore.(Requires, only the Offline backup image)
Online backup can also be used, but transaction done during the online backup needs to be rollforward(Requires, Online Backup image + Archive/Active Log file)c) Performing recovery of database

Version recovery if offline backup is used
Roll-forward recovery if online backup is usedd) Activate the database



2) Restore a failed Tablespace (Non-SYSCAT)

A) Restore failed tablespace from the available backup imagea) Identify the tablespace that is failed

ADMIN NOTIFICATION FILE
db2pd -tablespace -db sample
b) Restore the tablespace from the available backup image

db2 restore db sample tablespace (userspace1) online
c) Rollforward the tablespace to end of logs

db2 "rollforward db sample to end of logs tablespace(userspace1) online"
d) Optional. Take backup of database or backup of the restored tablespace

B) Backup image is not available for the failed tablespace
a) Identify the tablespace that is failed

ADMIN NOTIFICATION FILE
db2pd -tablespace -db sample
b) Check the create time of the failed tablespace

db2 "select varchar(TBSPACE,40),CREATE_TIME from syscat.tablespaces order by CREATE_TIME desc"
c) Check the available backup image
$==> db2 list history backup all for db <dbname>