Tuesday, July 22, 2008

IN (vs) EXISTS

At which situation, IN is better then EXISTS?

IN & EXISTS are processed in different way. But we can use IN in the place of EXISTS, also EXISTS can be used in the place of IN.



select * from emp where deptno in(select deptno from dept);

is typically processed as :

select * from emp, ( select distinct deptno from dept ) dept where emp.deptno = dept.deptno;

The subquery is evaluated, distinct'ed, indexed (or hashed or sorted) and then joined to the original table.

As opposed to

select * from emp where exists(select null from dept where deptno = emp.deptno)

That is processed more like:

for x in ( select * from emp ) loop

if ( exists ( select null from dept where deptno = emp.deptno )

then OUTPUT THE RECORD

end if

end loop

It always results in a full scan of EMP in EXISTS clause, whereas the first query can make use of an index on EMP.DEPTNO.


So, when is EXISTS appropriate and IN appropriate?

Lets say the result of the subquery is "huge" and takes a long time, outer query is small. Then IN is not good option. Because, it has to distinct the data. EXISTS can be quite efficient for this circumstances.

If the result of sub query is small, and then IN is good option. EXISTS is not good choice.

If both the subquery and the outer table are huge -- either might work as well as the other depends on the indexes and other factors.

When we tune the query, we can try with both IN & EXISTS and we can decide which one would be better...

If you need more info about this, please read the asktom article..

http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUESTION_ID:953229842074






How do we see what transaction is currently running on the database?

This queries are tested on oracle 9.2.0.8

Display all the statement which are available in memory?

Select * from v$sqlarea;

Note : If full SQL text is not shown in V$SQLAREA view, because it was larger than 1000 characters, this V$SQLTEXT view should be queried to extract the full SQL. So you need to use sql_text column in v$sql_text view instead of using sql_text column in v$sqlarea.

Display only currently running transactions in database?

The below query is used to find what is actively running(uncommitted transactions) in database. The column query_active_time is time between transaction starting time and commit time. For instance, the transaction started at 10AM and completed at 10.05AM. But user commits the transaction at 10.30PM. So the query_active_time will be 30 Seconds. V$transaction does not show any SELECT statement which are running in the database.

select
osuser,
username,
sid,
serial#,
program,
machine,
sql_text,
start_time,
cpu_time/1000000 cpu_time,
elapsed_time/1000000 run_time,
to_number(sysdate-to_date(c.start_time,'MM/DD/YY HH24:MI:SS'))*24*60 query_active_time
from
v$session a,
v$sqlarea b,
v$transaction c
where
((a.sql_address = b.address
and a.sql_hash_value = b.hash_value )
or ( a.prev_sql_addr = b.address
and a.prev_hash_value = b.hash_value))
and c.ses_addr = a.saddr
--and sysdate-to_date(v$transaction.start_time,'MM/DD/YY HH24:MI:SS') > 2/(60*24)


Display all the statement for only connected users?

This query will display all statement in memory only for connected sessions.

select
osuser,
username,
sid,
serial#,
program,
machine,
sql_text,
cpu_time/1000000 cpu_time,
elapsed_time/1000000 run_time
from
v$session a,
v$sqlarea b
where
((a.sql_address = b.address
and a.sql_hash_value = b.hash_value )
or ( a.prev_sql_addr = b.address
and a.prev_hash_value = b.hash_value))
and username is not null


Display all the statement with I/O statistics for connected session?

The below query provides all the transaction which are in memory with I/0 statistics. Sometime, user might be complaining that their query is Stuck. As a DBA, we can check v$sess_io view and see GETS, READS or CHANGES column continue to increase for a session, then you can tell them that the statement is not stuck.

select
a.sid,
a.username,
optimizer_mode,
hash_value,
address,
cpu_time/1000000,
elapsed_time/1000000,
sql_text,
a.sid,
block_gets,
consistent_gets,
physical_reads,
block_changes,
consistent_changes
from
v$session a,
v$sqlarea b,
v$sess_io c
where
a.sql_hash_value = b.hash_value
and a.sql_address = b.address
and a.sid = c.sid
and a.username is not null


Note : The I/0 statistics(consistent_gets, physical_reads, block_changes) continue to increase for each statement for the session. It would not show the statistics for individual statement.

Display all the long running statement in memory?

This query provides all the long transactions for connected/disconnected sessions.

Select * from v$session_longops;

Display all the long running completed transactions(connected/disconnected sessions)?

Select * from v$session_longops where time_remaining = 0

Display all the long running active transactions?

Select * from v$session_longops where time_remaining > 0

Display all the long running transactions for connected users with I/0 Statistics?

select
a.sid,
a.username,
optimizer_mode,
hash_value,
address,
cpu_time/1000000,
elapsed_time/1000000,
sql_text,
block_gets,
consistent_gets,
physical_reads,
block_changes,
consistent_changes,
d.time_remaining,
message
from
v$session a,
v$sqlarea b,
v$sess_io c,
v$session_longops d
where
a.sql_hash_value = b.hash_value
and a.sql_address = b.address
and a.sid = c.sid
and a.sql_hash_value = d.sql_hash_value
and a.sql_address = d.sql_address
and a.username is not null

How do we check which SQL is currently taking more CPU's in UNIX?
First we need to run the top command and top command will show the top CPU session.

Here is the top command partial output.

PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU COMMAND
27486 oracle 19 0 603M 603M 601M S 16.9 5.2 35:12 1 oracle

27486 is process id which takes 16.9% CPU time. The process id 27486 is equivalent to v$process.spid in database. So we can join v$process, v$session and get the required information.

Here is the query to find out which query is currently taking high CPU.

select
a.sid,
a.username,
optimizer_mode,
hash_value,
address,
cpu_time/1000000,
elapsed_time/1000000,
sql_text,
a.sid,
block_gets,
consistent_gets,
physical_reads,
block_changes,
consistent_changes
from
v$session a,
v$sqlarea b,
v$sess_io c,
v$process d
wherea.sql_hash_value = b.hash_value
and a.sql_address = b.address
and a.sid = c.sid
and d.addr = a.paddr
and a.username is not null
and d.spid= 32479

Note : 32479 is taken from Unix TOP output.

Tuesday, July 15, 2008

CLOB update is very slow

This article is written in oracle9i release 9.2.0.6.0 I just want to share this issue with all...

we are updating CLOB column in one of the table. The update statement is running through Java code. It used to take 1 second to update one row. All of a sudden, it started taking 150 second. Immediatley, developers reported to me.

Here are the direction i took initially.

1. I verified the table statistics. The stat is generating periodically.
2. Enabled the trace and ran the TKPROF. In the execution part, it is using high resources.

update MWS_COLLECTION set WEB_BODY=:1, headline=:2, STATE=:3,
SOURCE_PUBLISHED_DT=:4, author=:5
where
STORY_ID=:6
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 0 0 0 0
Execute 1 55.99 145.55 1057 9495719 8238 1
Fetch 0 0.00 0.00 0 0 0 0
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 2 55.99 145.56 1057 9495719 8238 1
Misses in library cache during parse: 1
Optimizer goal: FIRST_ROWS
Parsing user id: 67
Rows Row Source Operation
------- ---------------------------------------------------
1 update
1 TABLE ACCESS BY INDEX ROWID MWS_COLLECTION
1 INDEX UNIQUE SCAN PK_MWS_COLLECTION (object id 139890)
********************************************************************************

3. I checked the tablespace and it is LMT and SSM is AUTO.

I am not able to find anything except the trace file output shows more resources in exection part.

The LOB segment and other columns use the same tablespace. I thought, i move the LOB segment to different tablespace. But i checked with the business and they agreed to purge 90% of the data in the table. so i backup only 10% of the data into another table and truncate the table and insert the cloned data into original table. The problem is gone and again it takes only one second to update the single row.

Wednesday, July 9, 2008

Truncate table in another schema without DROP ANY TABLE Privilege

Developers used to write a Procedure to truncate the table which is in another schema. So developers require truncate table privileges to run their code. Now DBA needs to grant DROP ANY TABLE privileges to statisfy the developers code. DROP ANY TABLE is one of the most powerful and dangerous privilege and DBA can not grant to other schema's in Oracle.

Here is one way to accomplish the task. This article was written in oracle9i release 9.2.0.6.0

Let us say, We have two schemas, INVENT, INVENTADMIN. All the tables were created in INVENT and stored procedures were created in INVENTADMIN. We need to grant truncate privileges to INVENTADMIN to truncate the tables in INVENT.

SQL> connect invent/invent@db1
Connected.
SQL> create table test as select * from user_objects;
Table created.
SQL> select count(*) from test;

COUNT(*)
----------
3

-- For some reson, the piple symbol is not appearing in the blog and
-- please use piple symbol in the third line of the below procedure.

SQL> create or replace procedure stp_truncate_table(p_table_name varchar2) is
2 begin
-- the below line, there is piple synobol before p_table_name.
3 execute immediate 'truncate table 'p_table_name;
4 end;
5 /

Procedure created.

SQL> grant execute on stp_truncate_table to inventadmin;

Grant succeeded.

SQL> connect inventadmin/inventadmin@db1
Connected.

SQL> create synonym stp_truncate_table for invent.stp_truncate_table;
Synonym created.

SQL> execute stp_truncate_table('TEST');
PL/SQL procedure successfully completed.
SQL>

Now login back to invent and check the table data.

SQL>
SQL> connect invent/invent@db1
Connected.
SQL> select * from test;

no rows selected

SQL>

Resize temporary tablespace

In some Database configuration, temporary tablespace is set to autoextend. When user runs query with lot of sort or any cartesian join, then temp tablespace will increase. If the temp file gets increasingly large, then DBA wanted to resize the tablespace to more reasonable size in order to reclaim the space.

There may be multiple methods exist to reclaim the used space depending on which release of Oracle you are running. The method that exists for all releases of Oracle is to simply drop and recreate the temporary tablespace back to its original resonable size.

This excerise is tested in oracle9i on Windows. It should apply to unix too.
Step 1.

Create the new temporary tablespace.

SQL> create temporary tablespace temp1 tempfile 2 'E:\ORADATA\RMS30MC\TEMP02.DBF' SIZE 500M 3 autoextend off
4 /
Tablespace created.

Step 2
Assign the newly created tablespace to default temporary tablespace

SQL> alter database default temporary tablespace temp1;
Database altered.

SQL> select property_value from database_properties where property_name='DEFAULT_TEMP_TABLESPACE';

PROPERTY_VALUE
--------------------------------------------------------------------------------TEMP1

Now all the newly connected users(users conncted to the databaseafter completing step2) will use the temp tablespace TEMP1.

Step 3
Wait until no one is using the temp tablespace.

SELECT COUNT(*) FROM v$sort_usage WHERE tablespace = 'TEMP'
and
SELECT COUNT(*) FROM v$tempseg_usage WHERE tablespace = 'TEMP'

return 0 rows before dropping the old TEMP tablespace.

Step 4
Once we are certain that, no one is using the old temp tablespace,we drop the old temp tablespace.

SQL> DROP TABLESPACE temp INCLUDING CONTENTS AND DATAFILES;

Tablespace dropped.

Index Skip Scanning

Index skip scan is one of the new feature in oracle9i. Prior to oracle9i, if we have composit index and if we are not using leading column in the WHERE clause, then optimizer will not use the index. But in oracle9i and further versions, optimizer may use the skip scan index depends on the cardinality of the leading column and optimizer cost. The regular B-tree index will have only one range scan from top to bottom. But skip scan will do many range scan depends on the cardinatlity of the leading index column.

The advantage of index skip scan is, we can reduce the number of indexes on the table. It save index space and reduce the index maintenance. Skip scan index is not same as efficient as B-tree index. But it is better then the full table scan.

Here is an example.

SQL> desc emp;
Name Null? Type
----------------------------------------- -------- ----------------------------
EMPNO NUMBER
SEX VARCHAR2(10)
NAME VARCHAR2(20)

SQL> select count(*),sex from emp group by sex;

COUNT(*) SEX
---------- ----------
10000 F
10000 M

SQL> create index idx_emp on emp(sex,empno);

Index created.
SQL> EXECUTE DBMS_STATS.GATHER_TABLE_STATS(OWNNAME => 'MWSADMIN',TABNAME => 'EMP',ESTIMATE
_PERCENT => 10, METHOD_OPT => 'FOR ALL COLUMNS SIZE 1', CASCADE => TRUE);

PL/SQL procedure successfully completed.
SQL> set autotrace on explain;
SQL> select name from emp where empno=1934;

NAME
--------------------
Scott
Execution Plan
----------------------------------------------------------
0 SELECT STATEMENT Optimizer=FIRST_ROWS (Cost=4 Card=2 Bytes=20)
1 0 TABLE ACCESS (BY INDEX ROWID) OF 'EMP' (Cost=4 Card=2 Bytes=20)
2 1 INDEX (SKIP SCAN) OF 'IDX_EMP' (NON-UNIQUE) (Cost=3 Card=2)

Optimizer used skip scan index for the above query. During the skip scan, the composite index is accessed once for each distinct value of the leading column(s). For each distinct value, the index is searched to find the query's target values. When optimizer use the skip scan index, the query(select name from emp where empno=1934) is broken down into two small range scan as below.


select name from emp where sex = 'M' and empno=1934
union
select name from emp where sex = 'F' and empno=1934

Let us consider the below SQL query and it is using regular B-tree index. Since we use the leading index column in the where clause. This query does only one range scan, not like skip scan index.

SQL> select name from emp where sex='M' and empno=1934;

NAME
--------------------
Scott

Execution Plan
----------------------------------------------------------
0 SELECT STATEMENT Optimizer=FIRST_ROWS (Cost=2 Card=1 Bytes=11)
1 0 TABLE ACCESS (BY INDEX ROWID) OF 'EMP' (Cost=2 Card=1 Bytes=11)
2 1 INDEX (RANGE SCAN) OF 'IDX_EMP' (NON-UNIQUE) (Cost=1 Card=1)

HWM - High Water Mark

What is High Water Mark(HWM)?

HWM is boundry between used block and unused block. Blocks below HWM is used blocks and above HWM is unused blocks. Used blocks might or might not have the data. In normal operations (Insert/update), the HWM is mostly bump up and never go down. Oracle tracks the HWM for the segment in segment header.

How do we identify the HWM?

Option 1


We can use DBMS_SPACE.UNUSED_SPACE procedure.

Please remember, i used pipe symbol in the below code and it is not visible in blog. So placed ### symbol in the place of pipe sign. Please replace thie ### sign with pipe symbol when we run this code in any of your test database.

DECLARE
alc_bks NUMBER;
alc_bts NUMBER;
unsd_bks NUMBER;
unsd_bts NUMBER;
luefi NUMBER;
luebi NUMBER;
lub NUMBER;
BEGIN
FOR c1_row IN (SELECT table_name
FROM user_tables where table_name='COLLECTION') LOOP
DBMS_SPACE.UNUSED_SPACE (
segment_owner => USER,
segment_name => c1_row.table_name ,
segment_type => 'TABLE' ,
total_blocks => alc_bks ,
total_bytes => alc_bts ,
unused_blocks => unsd_bks ,
unused_bytes => unsd_bts ,
last_used_extent_file_id => luefi ,
last_used_extent_block_id => luebi ,
last_used_block => lub);
DBMS_OUTPUT.PUT_LINE( 'Data for '### c1_row.table_name);
DBMS_OUTPUT.PUT_LINE( RPAD('*',LENGTH(c1_row.table_name) + 10,'*'));
DBMS_OUTPUT.PUT_LINE( 'Total Blocks.................'### alc_bks );
DBMS_OUTPUT.PUT_LINE( 'Total Bytes..................'###alc_bts );
DBMS_OUTPUT.PUT_LINE( 'Unused Blocks................'### unsd_bks );
DBMS_OUTPUT.PUT_LINE( 'Unused Bytes.................'###unsd_bts );
DBMS_OUTPUT.PUT_LINE( 'Last Used Ext File Id........'### luefi );
DBMS_OUTPUT.PUT_LINE( 'Last Used Ext Block Id.......'### luebi );
DBMS_OUTPUT.PUT_LINE( 'Last Used Block..............'###lub );
DBMS_OUTPUT.NEW_LINE;
END LOOP;
EXCEPTION
WHEN OTHERS THEN
DBMS_OUTPUT.PUT_LINE(SUBSTR(SQLERRM,1,250));
END;
/

Here is the output for the above code.

Data for COLLECTION
************************
Total Blocks.................38656
Total Bytes..................158334976
Unused Blocks................0
Unused Bytes.................0
Last Used Ext File Id........60
Last Used Ext Block Id.......1284368
Last Used Block..............256

PL/SQL procedure successfully completed.

SQL>
HWM = Total Blocks - Unused Blocks
HWM = 38656-0
HWM = 38656

Option 2

we can use user_tables.
select sum(blocks)-sum(empty_blocks) from user_tables

Please remember, the above query gives the result based on the current statistics. Best way to find the HWM is to use DBMS_SPACE.UNUSED_SPACE(Option 1) procedure.

How do we find which table is good candidate to reset the HWM?

I have answered for this as a different topic in the same blog. Please click to answer this question.

Why do we need to reset the HWM?

1. When ever optimizer takes full table scan, it scan all the blocks below HWM. This would degrade the performance and slowdown the Full table scan process. To avoid this, we need to shrink the table to reset the HWM.

2. Oracle does not release the free space under HWM for use by other objects for a simple reason: because that space is reserved for new inserts and to accommodate the growth of existing rows. The additional benefit of resetting HWM is a more efficient utilization of space because the space freed is available for other objects.

3. When rows are inserted with direct path—for example, through Direct Load Insert (insert with the APPEND hint) or through the SQL*Loader direct path—the data blocks are placed directly above the HWM. The space below it remains wasted.

How do we reset the HWM?

Option 1
Alter table emp move tablespace

This option requires rebuilding the indexes. since the index will be invalid/unusable after running the above command. The downside is, rebuilding the index is additional overhead when we use this option. Also users can not use the application or reports while rebuilding the index.

Option 2

1. Export the data
2. truncate the table
3. import the table
4. Analyze the table

Option 3
1. copy the table data
2. truncate the original table
3. insert back.

Option 4
Use DBMS_REDEFINITION package to copy the table.

Option5
In oracle10g, we have better option to reset the high water mark. We need to enable row movement before we shrink the space. We do not need to rebuild the index after reseting the HWM. This is the best option to reset the HWM. But this feature is not available in oracle9i.

SQL@DBA1> alter table bookings enable row movement;

Table altered.

SQL@DBA1>
SQL@DBA1> alter table bookings shrink space compact;

Table altered.

If you need to read more about how to reset the HWM in oracle10g, please click this link. I wrote an article as a different topics.