Monthly Archives: March 2009

Grid Control, OMS : large number of defunct-processes

On my Grid Control management-server (Suse Linux, OMS version 10.2.0.4) a very large number of <defunct> processes arose what eventually caused the OMS not to respond anymore.

Looked like this:
oracle   16932 15961  0 Mar03 ?        00:00:00 <defunct>
oracle   16987 15961  0 Mar03 ?        00:00:00 <defunct>
oracle   17027 15961  0 Mar03 ?        00:00:00 <defunct>

… etc.

The process what caused this appeared to be the iasconsole:

oracle   15961     1  0 Mar03 ?        00:05:10 /software/oracle/product/GC10g/oms10g/perl/bin/perl /software/oracle/product/GC10g/oms10g/bin/emwd.pl iasconsole /software/oracle/product/GC10g/oms10g/sysman/log/em.nohup

Stopping and starting did clean up the defunct-processes, but only temporarily.

Two solutions

1. Not supported by Oracle :-) , solution contributed by a guy called Seb on the forums (but it works!):
Stop the iasconsole.
– Edit $ORACLE_HOME/bin/IASConsole.pm
– Modify the following line:
from #my $ua = LWP::UserAgent->new(keep_alive=>1);
to my $ua = LWP::UserAgent->new;
– Start the iasconsole.

2. Follow the note 391894.1
– It’s the bug 5504078 Abstract: EMWD.PL SPWANS DEFUNCT PERL PROCESSES AFTER OMS PATCH 10.2.0.2:
The script attempts to locate a file called WINDOWS_NT which on Unix of course does not exist. Consequently a defunct process is created.
– Stop the iasconsole:
emctl stop iasconsole
– cd $ORACLE_HOME/bin
touch Windows_NT
chmod 544 Windows_NT
– Start the iasconsole:
emctl start iasconsole

By |March 31st, 2009|Categories: grid control|Tags: , |0 Comments

Time-outs in web-forms environment

m4s0n501

Always confused with time-outs in a web-forms environment, so I wrote it in very short terms down to be memorized. Note 549735.1
on Metalink is the best source by the way where it is explained what time-outs you can use best, in this kind of environments

Three parameters of vital importance:

* FORMS_TIMEOUT to be configured in .env-file = per application, defaults to 15 minutes.
* Heartbeat to be configured in formsweb.cfg (default), or in .env-file, defaults to 2 minutes.
* Session-timeout to be configured in web.xml, for example:
<session-config>
<session-timeout>11</session-timeout>
</session-config>

– No timeout occurs when:
* heartbeat < forms-timeout.
– Time-out occurs when:
* heartbeat > forms-timeout      or
* heartbeat > session-time out

For example : two applications running on the same BI_FORMS- java container, one needs to be timed-out after 10 minutes, the other the customer don’t want to be timed-out:

———————-Application timing out           Application not te be timed out
FORMS_TIMEOUT            10                                                                         15
heartbeat                              12                                                                         12
session-timeout                 13                                                                         13

So in this case I used the formsweb.cfg (heartbeat) and the web.xml (session-timeout) to be the ‘fixed’ timeouts. And per application (.env-file) I can decide if they are timed-out or not.

By |March 31st, 2009|Categories: App. Server||0 Comments

Every 5 seconds vpxoci-error in emagent.trc, ora-25228

Noticed that the following error was popping up in the emagent.trc of a 10.2.04 – Grid Control-agent  on a specific node, every 5 seconds. Annoying, unnecessary:

2007-09-18 12:15:14 Thread-134875 ERROR vpxoci: Error on dequeue from SYS.ALERT_QUE: ORA-00604:
error occurred at recursive SQL level 1
ORA-06502: PL/SQL: numeric or value error
ORA-06512: at line 30
ORA-25228: timeout or end-of-fetch during message dequeue from SYS.ALERT_QUEM

Metalink  gave me doc-id 738638.1 as the cause, and gave me a solution: disable an ‘after event’-trigger.  First of all I couldn’t find a trigger (had to search 8 instances..) with the statement they provided. Second of all I couldn’t disable this trigger, as it was an application-trigger, and in use.

The statement they gave was:

select trigger_name,trigger_type,triggering_event from dba_triggers where trigger_type = ‘AFTER EVENT’ and triggering_event =  ‘ERROR';

The following gave me more results:

select trigger_name,trigger_type,triggering_event from dba_triggers where trigger_type = ‘AFTER EVENT’ and triggering_event like ‘%ERROR%';

So I found the trigger, but could not disable this one.  So I updated the trigger with:

begin

if ora_login_user != ‘DBSNMP’ then

…..

end if;

end;

/

Did not need to stop the database, error disappeared right away.

By |March 27th, 2009|Categories: grid control||0 Comments

Connecting AS SYSDBA results in "Connected to idle instance"..

Most of the time I don’t even think about how I connect to a database: set the environment – variables,  connect /as sysdba, and voila..

Then I ran into a node in which this was not as simple as I thought. Only connections through the listener were accepted. After serious investigation in discovering some typo’s, a vague bell was ringing…. Not the first thing I would think of… :

The Linux-command:  ‘uname -a’  gave me  ‘x86_64′.  A 64-bits machine!  The database and application server turned out to be running in 32-bits emulation mode.

Switched my session into 32-bits-emulation: with the Linux-command ‘ linux32 bash’ , and the problem was solved.  Some things can be time consuming but simple (when you think of it…).

By |March 23rd, 2009|Categories: Database||0 Comments

Bad Hummingbird performance (doc.mgmt.system)? Upgrade to 10.2 !

Hummingbird, the document management system, uses an Oracle database 91 on Sun at this company. Performance of several functions were very bad. 40 seconds to give me a document in some cases.  Bad sql-statements all over the place.

Installed a Linux system with 10.2.0.4 on it, made an export on SUN of the user DOCSADMIN, moved it to the Linux machine, imported it and transformed the tnsnames on the DM-server of Hummingbird. Very straightforward.  It gave Hummingbird a performance-boost. The select statement of 40 sec went down to 2 seconds. Be aware: turn on the statistics on SYS ! This will be done automatically when you create a database with DBCA by the way.

By |March 16th, 2009|Categories: Database||6 Comments