I just moved from Charlotte, NC to Houston, TX for a new job. I am giving consulting/contracting a (indefinite) rest. I don’t know if I will ever be a consultant again but for now I will be a Full-Time employee working on ’our’ in-house projects. Consulting was fun. I can’t complain at all.
Let’s see how houston treats us!!!
http://3quarksdaily.blogs.com/
An excellent website. A collaborative effort in its true sense. A complete delight to read in my opinion.
Analytic functions:
All my findings regarding analytic functions will go in this post. In other words I will keep on adding my finding to this post instead of creating new one all the time. I will also make sure that everytime when I update this post I bring up to the top of the list so that people notice the change.
Below is a small but sweet example that shows the difference between using three different functions.
DENSE_RANK()
RANK()
ROW_NUMBER()
create table t1 (
a number,
b varchar2(10)
);
insert into t1 values (5, ‘ABC’);
insert into t1 values (6, ‘ABC’);
insert into t1 values (5, ‘ABC’);
insert into t1 values (3, ‘ABC’);
insert into t1 values (5, ‘ABC’);
insert into t1 values (3, ‘ABC’);
insert into t1 values (6, ‘ABC’);
insert into t1 values (4, ‘XYZ’);
insert into t1 values (8, ‘XYZ’);
insert into t1 values (4, ‘XYZ’);
insert into t1 values (8, ‘XYZ’);
insert into t1 values (4, ‘XYZ’);
commit;
SQL> select * from t1;
A B
———- ———-
5 ABC
6 ABC
5 ABC
3 ABC
5 ABC
3 ABC
6 ABC
4 XYZ
8 XYZ
4 XYZ
8 XYZ
A B
———- ———-
4 XYZ
12 rows selected.
SQL> select dense_rank() over (partition by b order by a) “denserank”,
rank() over (partition by b order by a) “rank”,
row_number() over (partition by b order by a) “rownumber”,
a,b
from t1;
denserank rank rownumber A B
———- ———- ———- ———- ———-
1 1 1 3 ABC
1 1 2 3 ABC
2 3 3 5 ABC
2 3 4 5 ABC
2 3 5 5 ABC
3 6 6 6 ABC
3 6 7 6 ABC
1 1 1 4 XYZ
1 1 2 4 XYZ
1 1 3 4 XYZ
2 4 4 8 XYZ
denserank rank rownumber A B
———- ———- ———- ———- ———-
2 4 5 8 XYZ
12 rows selected.
SQL>
Hope this helps.
Kirtan
##### UPDATES BELOW #####
The example below uses MAX() function to show highest salary in each department right next to each employee record. It also shows what an employee’s salary is compared to the highest salary in the department.
Here the key is (PARTITION BY DEPTNO) part of the analytic function. That’s what tells oracle to break data up based on the partition details.
SQL> SELECT
ENAME,
SAL,
DEPTNO,
MAX(SAL) OVER (PARTITION BY DEPTNO) MGR_SAL_BY_DEPT,
sal/MAX(SAL) OVER (PARTITION BY DEPTNO) *100 EMP_percent_SAL
FROM EMP
ORDER BY DEPTNO;
ENAME SAL DEPTNO MGR_SAL_BY_DEPT EMP_PERCENT_SAL
———- ———- ———- ————— —————
CLARK 2450 10 5000 49
KING 5000 10 5000 100
MILLER 1300 10 5000 26
JONES 2975 20 3000 99.1666667
FORD 3000 20 3000 100
ADAMS 1100 20 3000 36.6666667
SMITH 800 20 3000 26.6666667
SCOTT 3000 20 3000 100
WARD 1250 30 2850 43.8596491
TURNER 1500 30 2850 52.6315789
ALLEN 1600 30 2850 56.1403509
JAMES 950 30 2850 33.3333333
BLAKE 2850 30 2850 100
MARTIN 1250 30 2850 43.8596491
14 rows selected.
SQL>
Now by adding one more MAX() function to the query we can have max salary across departments and a comparision of what an empolyee is making compared to the maximum salary across all departments.
SQL> SELECT
ENAME, SAL,DEPTNO,
MAX(SAL) OVER (PARTITION BY DEPTNO) max_dept_sal
,sal/MAX(SAL) OVER (PARTITION BY DEPTNO) *100 dept_percent
,MAX(SAL) OVER () max_comp_sal
,sal/MAX(SAL) OVER () *100 comp_percent
FROM EMP
ORDER BY DEPTNO;
ENAME SAL DEPTNO MAX_DEPT_SAL DEPT_PERCENT MAX_COMP_SAL COMP_PERCENT
———- ———- ———- ———— ———— ———— ————
CLARK 2450 10 5000 49 5000 49
KING 5000 10 5000 100 5000 100
MILLER 1300 10 5000 26 5000 26
JONES 2975 20 3000 99.1666667 5000 59.5
FORD 3000 20 3000 100 5000 60
ADAMS 1100 20 3000 36.6666667 5000 22
SMITH 800 20 3000 26.6666667 5000 16
SCOTT 3000 20 3000 100 5000 60
WARD 1250 30 2850 43.8596491 5000 25
TURNER 1500 30 2850 52.6315789 5000 30
ALLEN 1600 30 2850 56.1403509 5000 32
ENAME SAL DEPTNO MAX_DEPT_SAL DEPT_PERCENT MAX_COMP_SAL COMP_PERCENT
———- ———- ———- ———— ———— ———— ————
JAMES 950 30 2850 33.3333333 5000 19
BLAKE 2850 30 2850 100 5000 57
MARTIN 1250 30 2850 43.8596491 5000 25
14 rows selected.
SQL>
##############UPDATES ###################
example showing the use of LAG and LEAD functions. Here, LAG is used for reading rows before the current row and LEAD is used for reading rows after the current row. Example below shows the difference between salaries of row and its previous row and row and its next row. The key here would be the ORDER BY clause in the OVER() section of the analytic function.
SQL> SELECT
2 EMPNO,
3 ENAME,
4 JOB,
5 SAL,
6 LAG(sal, 1, 0) OVER (ORDER BY sal) AS sal_prev,
7 sal - LAG(sal, 1, 0) OVER (ORDER BY sal) AS LAG_sal_diff,
8 LEAD(sal, 1, 0) OVER (ORDER BY sal) AS sal_NEXT,
9 sal - LEAD(sal, 1, 0) OVER (ORDER BY sal) AS LEAD_sal_diff
10 FROM
11 EMP;
EMPNO ENAME JOB SAL SAL_PREV LAG_SAL_DIFF SAL_NEXT LEAD_SAL_DIFF
---------- ---------- --------- ---------- ---------- ------------ ---------- -------------
7369 SMITH CLERK 800 0 800 950 -150
7900 JAMES CLERK 950 800 150 1100 -150
7876 ADAMS CLERK 1100 950 150 1250 -150
7521 WARD SALESMAN 1250 1100 150 1250 0
7654 MARTIN SALESMAN 1250 1250 0 1300 -50
7934 MILLER CLERK 1300 1250 50 1500 -200
7844 TURNER SALESMAN 1500 1300 200 1600 -100
7499 ALLEN SALESMAN 1600 1500 100 2450 -850
7782 CLARK MANAGER 2450 1600 850 2850 -400
7698 BLAKE MANAGER 2850 2450 400 2975 -125
7566 JONES MANAGER 2975 2850 125 3000 -25
EMPNO ENAME JOB SAL SAL_PREV LAG_SAL_DIFF SAL_NEXT LEAD_SAL_DIFF
---------- ---------- --------- ---------- ---------- ------------ ---------- -------------
7788 SCOTT ANALYST 3000 2975 25 3000 0
7902 FORD ANALYST 3000 3000 0 5000 -2000
7839 KING PRESIDENT 5000 3000 2000 0 5000
14 rows selected.
SQL>
The following is a list of books. There are three sections. First is the name of the book that I am currently reading. Second is the list of books that I am going to read in near future. And third is a list of books that I have read.
| Currently reading… | |
| The Inheritance of Loss | by Kiran Desai |
| Next in line… | |
| The Fountainhead | by Ayn Rand |
| The Last King of Scotland | by Giles Foden |
| Books I’ve read… | |
| Tuesdays With Morrie | by Mitch Albom |
| Legionnaire: Five Years in the French Foreign Legion | by Simon Murray |
| The Five People You Meet in Heaven | by Mitch Albom |
| The Da Vinci Code | by Dan Brown |
| One Hundred Years of Solitude | by Gabriel Garcia Marquez |
| Middlesex | by Jeffrey Eugenides |
| The Catcher in the Rye | by J.D. Salinger |
| Angels & Demons | by Dan Brown |
| One Flew Over the Cuckoo’s Nest | by Ken Kesey |
| The Count of Monte Cristo | by Alexandre Dumas père |
| The Chronicles of Narnia | by C. S. Lewis |
| Train to Pakistan | by Khushwant Singh |
| Where Wizards Stay Up Late: The Origins Of The Internet | by Katie Hafner |
| The World Is Flat | by Thomas L. Friedman |
| Freakonomics | by Steven D. Levitt, Stephen J. Dubner |
| The Harry Potter Series | by J. K. Rowling |
| Rich Dad, Poor Dad | by Robert T. Kiyosaki |
| Les Miserables | by Victor Hugo |
| The Runaway Jury | by John Grisham |
Kirtan Desai
I was reading tom kyte’s blog and found the following link there. I am not trying to copy/steal what he had there but I thought this was extremely important to publish. Not just for others but to keep reminding myself.
I think, for anyone who wants to learn anything, asking right questions is the only way to learn. Questions themselves can be directed to someone else or yourself. What’s important is that you ask questions that lead you in the right way. Check out the article below. It’s GOOD.
http://www.catb.org/~esr/faqs/smart-questions.html
Tom also wrote about this and he had his own link besides the one above on his blog. Since I am sort of “re-blogging” here, it’s only fair to provide the original links. Or just in case you want to read them both.
http://tkyte.blogspot.com/2005/06/how-to-ask-questions.html
Kirtan.
There is an excellent article by Doug Burns here regarding the role of an individual who acts as either Developmentt DBA, Developer/DBA, DBA/Developer etc. The bottom line, as pointed out in the article, is that in a development environment where a DBMS is at the center of the project/application/system, individual(s) with strong knowledge of the DBMS must be utilized.
Leaving the future of an application or a system in the hands of developers who have little knowledge of the DBMS is not a good idea. Many companies nowadays let consulting firms come in and develop what’s needed. Many time these incoming consultants come with *consulting* experience and not with product or technology specific experience.
Even though the article is listed on SQL-Server specific website, I thought the artice was generic enough, to-the-point and meaningfull. It’s a must-read.
Kirtan
COALESCE is one of many convenient ways provided by oracle to handle NULL values in the database. The Oracle COALESCE function accepts a varying length list of arguments and returns the first non-NULL value/expr in the list. If all arguments in the list evaluate to a NULL value, then the COALESCE function will return a NULL value. Instead of scanning through all the arguments, oracle takes a short-circuit evaluation approach to processing what’s provided. In other words, it evaluates each expr value and determines whether it is NULL, rather than evaluating all of the expr values before determining whether any of them is NULL. As soon as it find the first non-NULL value it will stop processing rest of the provided arguments.
There is always more than one way to achieve the same outcome. You can use COALESCE as a variety of the CASE or DECODE expression.
For example,
Using DECODE, you would do something like
SELECT DECODE(mgr, NULL, 9999, mgr) FROM SCOTT.EMP
OR
Using CASE, you would do something like
SELECT
CASE
WHEN MGR IS NULL THEN
9999
ELSE
MGR
END
FROM SCOTT.EMP;
The following demonstrates how to substitute in a default value in a SELECT statement when there are NULL values present using COALESCE function.
SELECT COALESCE(mgr, 9999) FROM SCOTT.EMP;
The statement above looks much cleaner, well at least to everyone who knows what’s going on. In examples above, 9999 is a default value that you would like to assign to any null values in the list.
But a better use of COALESCE would be when more than 1 value is passed to it as input.\
For example,
SQL> select coalesce(null,null, 5, null) from dual;
COALESCE(NULL,NULL,5,NULL)
--------------------------
5
SQL> select coalesce(null,5,null,10, null) from dual;
COALESCE(NULL,5,NULL,10,NULL)
-----------------------------
5
SQL> select coalesce(null, null, null, null,555) from dual;
COALESCE(NULL,NULL,NULL,NULL,555)
---------------------------------
555
SQL>
As you can notice COALESCE picks up the first non-null value from a given list.
hope this helps.
Kirtan
There was a question asked on InformaticaDevelopement yahoo group about parsing names. I had a piece of code saved from asktom. I don’t know when I got it from asktom, but I know I got it from asktom. I just wanted to put it out here for reference. To parse names you would have to do the following.
SQL>create table name_test ( a varchar2(15) );
Table created.SQL>insert into name_test values ( 'kirtan' );
1 row created.
SQL>insert into name_test values ( 'kirtan desai' );
1 row created.
SQL>insert into name_test values ( 'kirtan desai s' );
1 row created.
SQL>select substr( a||' ', 1, instr(a||' ' ' ')-1 ) first_name,
2 substr( a||' ', instr( a||' ' ' ') +1, instr( a||' ' ' ', 1, 2 )-instr(a||' ' ' ')-1 ) last_name,
3 rtrim(substr( a||' ', instr( a||' ' ' ',1,2)+1),' ') middle_init
4 from name_test
5 /
FIRST_NAME LAST_NAME MIDDLE_INIT
---------------- ----------------- -----------------
kirtan
kirtan desai
kirtan desai s
hope this helps
Kirtan
hope this helpsKirtan
Flashback features were there in 9i. 10g is providing something new/extra.
### From new features guide ###
What if you ever need to do a resetlogs operation for recovery purposes, you can take the
database back to a point in time before the resetlogs if you find that you made a
mistake. This provides administrators more flexibility to detect and correct human
error situations.
It is now possible to flashback the primary and standby databases to an SCN or a point in
time prior to the switchover operation. Using flashback in this way on a physical standby
database preserves the standby role. Using flashback in this way on a logical standby
database changes the role of the standby database to what it was at the target SCN/time.
### ###
Here is how you can use Flashback options.
SQL> create table ksdesai.fstes (a number(2));Table created.
SQL> insert into ksdesai.fstes values (1);
1 row created.
SQL> insert into ksdesai.fstes values (2);
1 row created.
SQL> insert into ksdesai.fstes values (3);
1 row created.
SQL> insert into ksdesai.fstes values (4);
1 row created.
SQL> insert into ksdesai.fstes values (5);
1 row created.
SQL> commit;
Commit complete.
SQL> select * from ksdesai.fstes;
A
----------
1
2
3
4
5
SQL> delete from ksdesai.fstes where a<3;
2 rows deleted.
SQL> commit;
Commit complete.
SQL> SELECT versions_xid XID, versions_startscn START_SCN,
2 versions_endscn END_SCN, versions_operation OPERATION,
3 A AS TABLE_COLUMN FROM KSDESAI.FSTES
4 VERSIONS BETWEEN SCN MINVALUE AND MAXVALUE
5 ;\
6
SQL> SELECT versions_xid XID, versions_startscn START_SCN,
2 versions_endscn END_SCN, versions_operation OPERATION,
3 A AS TABLE_COLUMN FROM KSDESAI.FSTES
4 VERSIONS BETWEEN SCN MINVALUE AND MAXVALUE;
XID START_SCN END_SCN O TABLE_COLUMN
---------------- ---------- ---------- - ------------
000A001B00078D60 660961162 D 2
000A001B00078D60 660961162 D 1
0005002E0007F815 660961134 I 5
0005002E0007F815 660961134 I 4
0005002E0007F815 660961134 I 3
0005002E0007F815 660961134 660961162 I 2
0005002E0007F815 660961134 660961162 I 1
7 rows selected.
SQL> insert into ksdesai.fstes
2 select a from ksdesai.fstes
3 VERSIONS BETWEEN SCN MINVALUE AND MAXVALUE
4 where versions_operation='D';
2 rows created.
SQL> commit;
Commit complete.
SQL> select * from ksdesai.fstes;
A
----------
3
4
5
2
1
Magical!!! Or is it?
Kirtan
####UPDATES####
Retriving delete records from a table within certain period.
SQL> create table t1 (a number);
Table created.
SQL> desc t1;
Name Null? Type
------ ------ --------
A NUMBER
SQL> insert into t1 values (1);
1 row created.
SQL> insert into t1 values (2);
1 row created.
SQL> insert into t1 values (3);
1 row created.
SQL> commit;
Commit complete.
SQL> select * from t1;
A
----------
1
2
3
SQL> create table t3 (a number);
Table created.
SQL> insert into t3 values (1);
1 row created.
SQL> insert into t3 values (2);
1 row created.
SQL> insert into t3 values (3);
1 row created.
SQL> commit;
Commit complete.
SQL> select * from t3;
A
----------
1
2
3
######################
SQL> delete from t1 where a=2;
1 row deleted.
SQL> commit;
Commit complete.
SQL> select * from t1;
A
----------
1
3
SQL> select * from t3;
A
----------
1
2
3
########################
SQL> select versions_xid XID, versions_startscn START_SCN,
2 versions_endscn END_SCN, versions_operation OPERATION,
3 A AS TABLE_COLUMN from t1
4 VERSIONS BETWEEN SCN MINVALUE AND MAXVALUE;
XID START_SCN END_SCN O TABLE_COLUMN
---------------- ---------- ---------- - ------------
000B000D0006B0E0 661139198 D 2
000B002C0006B412 661139095 I 3
000B002C0006B412 661139095 661139198 I 2
000B002C0006B412 661139095 I 1
######################
SQL> select * from t3;
A
----------
1
2
3
SQL> delete from t3 where a in (
2 select a from t1
3 VERSIONS BETWEEN SCN MINVALUE AND MAXVALUE
4 where
5 versions_operation='D'
6 );
1 row deleted.
SQL> select * from t3;
A
----------
1
3
#######################
SQL> select * from t1 a, t1 b where a.a=b.a;
A A
---------- ----------
1 1
3 3
######################
SQL> insert into t1
2 select a from t1
3 VERSIONS BETWEEN SCN MINVALUE AND MAXVALUE
4 where
5 versions_operation='D';
1 row created.
SQL> commit;
Commit complete.
SQL> select * from t1;
A
----------
1
3
2
######################
I just wrote this procedure below that does duplicate file (dataset, i should say) check.
Let’s say, you have a system where your clients/users have rights to write straight to your “staging” or “landing” tables. You have a system where multiple users are feeding multiple files everyday. You have a requirement to check whether users are sending duplicate files into your system intentionally or unintentionally. You could probably find a million ways of doing a duplicate file check. I just wrote a simple stored procedure that does that for me. Let’s have a look at it. \
create or replace PROCEDURE clob_to_hash IS
i number(5);
mychar VARCHAR2(4000);
myclob CLOB;
myraw raw(2000);
rec_cnt NUMBER(2);
CURSOR mycur1 IS
SELECT column_name AS
field_name
FROM sys.all_tab_cols
WHERE owner = 'SCOTT'
AND TABLE_NAME = 'EMP'
ORDER BY column_id;
mycur mycur1%ROWTYPE;
col_names VARCHAR2(4000);
mysqlstr varchar2(4000);
BEGIN
i:=0;
OPEN mycur1;
LOOP
FETCH mycur1 INTO mycur;
EXIT WHEN mycur1%NOTFOUND;
IF i = 0 THEN
col_names := mycur.field_name;
ELSE
col_names := col_names || ',' || mycur.field_name;
end if;
i:=i+1;
END LOOP;
close mycur1;
mysqlstr := 'SELECT xmlagg(xmlelement("emp",' || col_names || ')).getclobval() FROM scott.emp';
execute immediate mysqlstr INTO myclob;
myraw := dbms_crypto.hash(src => myclob, typ => dbms_crypto.hash_md5);
mychar := rawtohex(myraw);
SELECT COUNT(*) INTO rec_cnt FROM hashtest WHERE mchar = mychar;
IF rec_cnt = 0 THEN
INSERT INTO hashtest VALUES(mychar);
ELSE
INSERT INTO hashtest VALUES('duplicate data set');
END IF;
COMMIT;
END;
This procedure creates a list of column names of ‘emp’ table from sys.all_tab_cols in order they were created. It builds a clob of xml data out of the emp table and create an MD5 key for that clob. It converts the ‘raw’ value returned by DBMS_CRYPTO.HASH function to a ‘hex’ value. After that it tries to find a row in the “hashtest” table to see if the same ‘hash’ key exists. If it finds duplicate records, it write ‘duplicate data set’ in the hashtest table. otherwise it writes the ‘hex’ value generated from the ‘hash’ value which was returned in ‘raw’ format by DBMS_CRYPTO.HASH function.
SQL> truncate table hashtest;
Table truncated.
SQL> exec clob_to_hash;
PL/SQL procedure successfully completed.
SQL> desc hashtest;
Name Null? Type
----------------------------------------- -------- ----------------------------
MCHAR VARCHAR2(255)
SQL> select * from hashtest;
MCHAR
--------------------------------------------------------------------------------
4FCD4351B718B46FC8DCE7BA01B3F910
SQL> exec clob_to_hash;
PL/SQL procedure successfully completed.
SQL> select * from hashtest;
MCHAR
--------------------------------------------------------------------------------
17A6E53EA979BFB0E67CCE2E5BDDB769
duplicate data set
SQL> update scott.emp set sal=sal+100 where empno=7844;
1 rows updated.
SQL> commit;
Commit complete
SQL> exec clob_to_hash;
PL/SQL procedure successfully completed.
SQL> select * from hashtest;
MCHAR
--------------------------------------------------------------------------------
17A6E53EA979BFB0E67CCE2E5BDDB769
duplicate data set
4FCD4351B718B46FC8DCE7BA01B3F910
That’s it!!! With a little modification, you could pretty much use this to compare any two datasets.