Apps Technology Stack

Interesting enhancements in 12.0.6 – R12.ATG_PF.A.DELTA.6

Posted on December 2, 2008. Filed under: Apps Technology Stack, DBA/Admin Tips, R12, Technology | Tags: , , |

Oracle Applications Technology Stack Release Update Pack 6 (R12.ATG_PF.A.DELTA.6)

Autoconfig has two new options

1. Parallel Mode
2. Profile Mode

Parallel Mode

This mode enables autoconfig to be executed parallel across all mid-tiers. Oh man, i love this feature … it really helps when you have multiple mid-tiers.

Just add -parallel to your normal adconfig command

Profile Mode

This option provides a drill down HTML report of the autoconfig run.

Just add -profile to your normal adconfig command

For more information on above options refer to Metalink Note 742110.1


Read Full Post | Make a Comment ( 1 so far )

Unable to generate forwarding URL. Exception: oracle.apps.fnd.cache.CacheException

Posted on November 18, 2008. Filed under: Apps Technology Stack, DBA/Admin Tips, Diagnostics, R12, TroubleShooting | Tags: , , |

While trying to access the login page below error is reported.

“Unable to generate forwarding URL. Exception: oracle.apps.fnd.cache.CacheException”

This simple message doesnot give much information. Digging into logs yielded below detailed error message.


html: Servlet error
        at oracle.apps.fnd.cache.AppsCache.get(
        at oracle.apps.fnd.profiles.Profiles.getProfileOption(
        at oracle.apps.fnd.profiles.Profiles.getProfile(
        at oracle.apps.fnd.profiles.ExtendedProfileStore.getSpecificProfileFromDB(
        at oracle.apps.fnd.profiles.ExtendedProfileStore.getSpecificProfile(
        at oracle.apps.fnd.profiles.ExtendedProfileStore.getProfile(
        at oracle.apps.fnd.common.logging.DebugEventManager.configureUsingDatabaseValues(
        at oracle.apps.fnd.common.logging.DebugEventManager.configureLogging(
        at oracle.apps.fnd.common.logging.DebugEventManager.internalReinit(
        at oracle.apps.fnd.common.logging.DebugEventManager.reInitialize(
        at oracle.apps.fnd.common.logging.DebugEventManager.reInitialize(
        at oracle.apps.fnd.common.AppsLog.reInitialize(
        at oracle.apps.fnd.common.AppsContext.initLog(
        at oracle.apps.fnd.common.AppsContext.initializeContext(
        at oracle.apps.fnd.common.AppsContext.initializeContext(
        at oracle.apps.fnd.common.AppsContext.<init>(
        at oracle.apps.fnd.common.WebAppsContext.<init>(
        at oracle.apps.fnd.sso.Utils.getAppsContext(
        at oracle.apps.fnd.sso.AppsLoginRedirect.AppsSetting(
        at oracle.apps.fnd.sso.AppsLoginRedirect.init(
        at com.evermind[Oracle Containers for J2EE 10g ( ].server.http.HttpApplication.loadServlet(
        at com.evermind[Oracle Containers for J2EE 10g ( ].server.http.HttpApplication.findServlet(
        at com.evermind[Oracle Containers for J2EE 10g ( ].server.http.HttpApplication.findServlet(
        at com.evermind[Oracle Containers for J2EE 10g ( ].server.http.HttpApplication.getRequestDispatcher(HttpApplicati
        at com.evermind[Oracle Containers for J2EE 10g ( ].server.http.HttpRequestHandler.doProcessRequest(HttpRequestHan
        at com.evermind[Oracle Containers for J2EE 10g ( ].server.http.HttpRequestHandler.processRequest(HttpRequestHandl
        at com.evermind[Oracle Containers for J2EE 10g ( ]
        at com.evermind[Oracle Containers for J2EE 10g ( ]
        at com.evermind[Oracle Containers for J2EE 10g ( ].util.ReleasableResourcePooledExecutor$
Caused by: oracle.apps.jtf.base.resources.FrameworkException: Error in IAS Cache: java.lang.NullPointerException: null Connection
        at oracle.apps.jtf.cache.IASCacheProvider.get(
        at oracle.apps.jtf.cache.CacheManager.getInternal(
        at oracle.apps.jtf.cache.CacheManager.get(
        at oracle.apps.fnd.cache.AppsCache.get(
        … 30 more
Caused by: oracle.apps.jtf.base.resources.FrameworkException:
An exception occurred in the method CacheAccess.get
The base exception is:
null Connection
        at oracle.apps.jtf.base.resources.FrameworkException.convertException(
        at oracle.apps.jtf.base.resources.FrameworkException.addException(
        at oracle.apps.jtf.base.resources.FrameworkException.<init>(
        at oracle.apps.jtf.base.resources.FrameworkException.<init>(
        at oracle.apps.jtf.base.resources.FrameworkException.<init>(
        at oracle.apps.jtf.base.resources.FrameworkException.<init>(
        at oracle.apps.jtf.base.resources.FrameworkException.<init>(
        … 34 more

There could be many root causes for this error. If you look closely into the errorstack, it complains about null connection.

Caused by:
Error in IAS Cache:
null Connection

Every access to the login page ( (in R12) establishes a connection to the database as GUEST user using the DBC file. (In 11i, the url page is a static html which doesnt establish a connection)

If the mid-tier gets a connection, it displays the login page else it throws “oracle.apps.fnd.cache.CacheException”.

Now lets look at some possible reasons for not getting a connection.

1. Database is down
2. Database is up but the listener is down.
3. Archive Location is full, which prevents new connections till freed-up.
4. The instance was recently cloned and the GUEST user password is not in sync.

When the instance is cloned from source, there is a remote chance that the guest user password is not in sync. By sync, i mean the profile option value and the dbc file value for guest user password.

Steps to verify the values (perform below steps in the mid-tier)

4.1 DBC File value



4.2 Profile Option Value

sqlplus apps/[apps_pwd]
select fnd_profile.value(‘GUEST_USER_PWD’) from dual;


Values from 4.1 and 4.2 should be in sync and in our case it is. Now lets connect to the database using the guest password.

4.3 GUEST User Connectivity Check

sqlplus apps/[apps_pwd]


5. Ok, i didnt do a clone and the GUEST user password is in sync. What next.

The next possible reason, the database is not allowing any new sessions. May be the maximum number of sessions defined at the database level had already reached.

If you had followed all the steps sequentially as defined above, you would have realised that database does accept new connections, since you were able to verify the GUEST user password profile option value. If SESSIONS parameter value had been reached you would have got the error ORA-00018: maximum number of sessions exceeded.

6. Now what, my database is accepting new sessions.

Next option is to look for stale database connections.

ps -fu [oracle_user] | grep -i ‘LOCAL=NO’

This command will give you a list of stale connections. Use below command to kill them and to free up resources.


ps -fu [oracle_user] | grep -i ‘LOCAL=NO’ | grep -v grep | awk ‘{print “kill -9 “$2}’ | ksh

Hope one of the above mentioned steps resolves your issue when you encounter “Unable to generate forwarding URL. Exception: oracle.apps.fnd.cache.CacheException”


Read Full Post | Make a Comment ( 6 so far )

How to clear cache for a specific component in R12?

Posted on November 11, 2008. Filed under: Apps Technology Stack, Configuration, DBA/Admin Tips, Diagnostics, R12, Technology, TroubleShooting | Tags: , , , , |

As we all know the normal tendency for clearing cache in Oracle Apps is to delete the $COMMON_TOP/_pages directory. In 11i, after deleting we dont have to recompile the jsps but in R12 we have to recompile the pages since s_jsp_main_mode is set to justrun by default which is in turn updated in orion-web.xml. Click here for more info.

Ok coming to this post, how can i clear cache for a specific component of Oracle Apps R12 viz., iprocurement, istore etc., Is this even possible ? Yes it is and this option comes handy when somebody modifies a jsp page which is part of say istore, then it makes sense to clear cache for istore and not for the whole server.

Clearing cache for specific Component/Application in R12

Login as sysadmin
Select “Functional Administrator” responsibility
Go to Core Services ==> Caching Framework ==> Tuning
Query the application (for example iStore) or by Name or code.
Select the component and clear the cache.

Note: For some specific configuration changes, we need to bounce Apache also.

Clearing Global Cache in R12

a) Go to Functional Administrator Responsibility
b) Go to Core Services ==> Caching Framework
c) Go to Global configuration and clear all the cache from there.(Only If you want to clear all the  cache)

Hope this helps…..


About Sam: Sam is a guest author for this blog and had performed various installations & upgrades. He is well versed with tech stack components and applications maintenance. He can be contacted at

Read Full Post | Make a Comment ( 1 so far )

OPATCH_JAVA_ERROR : An exception of type “OPatchException” has occurred

Posted on November 4, 2008. Filed under: Apps Technology Stack, Database, DBA/Admin Tips, opatch, Patch, R12, TroubleShooting | Tags: , , |

While trying to apply a patch or listing inventory, using opatch utility (be it on the Database node or for the Applications Tech Stack) below error is encountered:

LsInventory: OPatch Exception while accessing O2O

OPATCH_JAVA_ERROR  : An exception of type “OPatchException” has occurred:
OPatch Exception:
OUI found no such ORACLE_HOME set in the environment
Can not get details for given Oracle Home
An exception occurs

There is no Interim Patch

OPATCH_JAVA_ERROR  : An exception of type “OPatchException” has occurred:
Can not get a list of inventory on this home.

ERROR: OPatch failed because of Inventory problem.


Imagine you are trying to apply a patch to 10.1.3 Tech Stack Oracle Home on R12.

What did i do wrong ?

1. Did i source the corrent environment file ?

Yes, i did, my echo $ORACLE_HOME command points to 10.1.3 home instead of the default 10.1.2 home. hmmm ok

2. Am i using the correct opatch utility?

yes, which opatch is pointing to $ORACLE_HOME(10.1.3)/Opatch/opatch

3. Is the inventory location accessible ?

Yes, oraInst.loc under Oracle Home (10.1.3) is pointing to a valid directory.

$ cat oraInst.loc

4. Does the inventory has information about the Oracle Home you are trying to patch?

How do i check that ….

$ cd /var/opt/oracle/oraInventory

$ ls -latr
total 112
drwxrwxr-x   3 username    dba  ..
drwxrwxr-x   2 username    dba  oui
-rwxrwxr-x   1 username    dba  sessionContext.ser
drwxrwxr-x   2 username    dba  ContentsXML
-rwxrwxr-x   1 username    dba
drwxrwxr-x   2 username    dba  Contents
drwxrwx—   3 username    dba  logs
-rwxrwxr-x   1 username    dba  install.platform
drwxrwxr-x   6 username    dba  .

$ cd ContentsXML
$ vi inventory.xml

<HOME NAME=”SID_TOOLS__masked_oracle_home_tech_st_10_1_2″ LOC=”[masked_oracle_home]/tech_st/10.1.2″ TYPE=”O” IDX=”1″/>


ooops … i dont …. THIS IS THE ISSUE

So what should i do now?

Pretty simple, add an entry for this Oracle Home just above or below HOME_LIST tag.

<HOME NAME=”SID_TOOLS__masked_oracle_home_tech_st_10_1_3″ LOC=”[masked_oracle_home]/tech_st/10.1.3″ TYPE=”O” IDX=”2″/>

After making changes,

<HOME NAME=”SID_TOOLS__masked_oracle_home_tech_st_10_1_2″ LOC=”[masked_oracle_home]/tech_st/10.1.2″ TYPE=”O” IDX=”1″/>
<HOME NAME=”SID_WEBOH__masked_oracle_home_tech_st_10_1_3″ LOC=”[masked_oracle_home]/tech_st/10.1.3″ TYPE=”O” IDX=”2″/>

Save the file.

Now you retry the same command it should go through.

If you dont want to update your inventory.xml which is under the global inventory location, you can pass a parameter to the opatch utility to use a different inventory location provided that inventory has information about this oracle home.

opatch -invPtrLoc [New .loc file location]

Happy troubleshooting !!!!


Read Full Post | Make a Comment ( 3 so far )

How to identify if a port is listening on a unix server

Posted on October 6, 2008. Filed under: Apps Technology Stack, DBA/Admin Tips, Diagnostics, Operating System, R12, TroubleShooting | Tags: , , , |

We often get into a situation wherein we have to check if a particular port is being used by some process.

netstat -an | grep [port number] | grep LISTEN

Command to identify the OS process associated with a port

lsof -i tcp:[port number]



Read Full Post | Make a Comment ( 1 so far )

Blank AppsLogin / AppsLocalLogin Page in R12?

Posted on September 12, 2008. Filed under: Apps Technology Stack, Configuration, Diagnostics, R12, TroubleShooting | Tags: , , , |

There were quite a few times i had encountered this blank page issue and the root cause was not the same all the times. There are many reasons for this issue to occur. Listed below are some of them.

Scenario 1: After a clone

Solution :

1. stopall — to shutdown opmn managed services viz., opmn, forms, oacore & oc4j
2. cd $COMMON_TOP/_pages
3. mv _pages _pages_old
4. cd $FND_TOP/patch/115/bin or export PATH=$FND_TOP/patch/115/bin:$PATH
5. Compile all jsps.

perl –compile –flush -p 2

(-p represents parallel threads, if you have enough horse power on your server, you can increase it to 8 or 10)

6. startall

Scenario 2: Database Listener is down

Solution: Obvious answer, start the database listener.

Scenario 3: Database Archive Locaion is 100% full

Solution: Either temporarily move the archive files to another location or delete older archive files (provided you had taken a cold backup sometime).

Scenario 4: IAS Cache initialization failed

This issue occurs only if you have more than one mid-tier and you had enabled Distributed Java Caching.

Error Message in log file:

Exception in static block of jtf.cache.CacheManager. Stack trace is: oracle.apps.jtf.base.resources.FrameworkException:
IAS Cache initialization failed.
The Distributed Caching System failed to initialize on port: 12357.
The list of hosts in the distributed caching system is: [node 1] [node 2].
The port [port no] should be free on each host running the JVMs.
at oracle.apps.jtf.cache.IASCacheProvider.init(
at oracle.apps.jtf.cache.CacheManager.activateCache(

In this scenario, s_java_object_cache_port value was listening on node 2 but not on node 1. So whichever connection went to node 1 it returned a blank page and occasionally it displayed 500 Internal Server error.



1) Bring down opmn services on both the boxes.
2) Verify no process is listening for s_java_object_cache_port value. click here to view the commands.
2) Change s_java_object_cache_port on all the nodes in the context file
3) Run autoconfig


s_java_object_cache_port value is referrenced in two places

1) $INST_TOP/ora/10.1.3/javacache/admin/javacache.xml
2) Profile option name JTF_DIST_CACHE_PORT

Change the port value to a new number and bounce opmnctl services. Dont forget to change it in $CONTEXT_FILE. (To take effect during next autoconfig)


Use commands listed in step 2 of autoconfig option.

If the new port mentioned still doesnt listen on the boxes you can use below workaround. This workaround will resolve the “IAS Cache initialization failed” even if the port is not listening.


Set LONG_RUNNING_JVM=false in $INST_TOP/ora/10.1.3/j2ee/oacore/config/
Bounce opmn services.

In 11i, if the database listener is down or if the archive location is full, atleast we get the internal server page while trying to access the login page, but in R12 nothing happens except for the blank page and apache error log will have an entry “oc4j_socket_recvfull timed out” with additional error message mod_oc4j: request to OC4J [mid-tier servername:ajp port] failed: Connect failed

So the moral of the story is recompiling the jsps is not always the solution if you get a blank login page.


Read Full Post | Make a Comment ( 2 so far )

How to change Java heap size in R12 to avoid java.lang.OutOfMemoryError: Java heap space error?

Posted on September 11, 2008. Filed under: Apps Technology Stack, Configuration, Diagnostics, Operating System, R12, Technology, TroubleShooting | Tags: , , , , |

Recently we had encountered a situation, wherein end users where not able to get the R12 login page and
it just hangs, but all the opmn processess were up and running – status gives status as
alive for all the components viz., HTTP, oafm, oacore & forms. The same environment was available and
end users were able to access without any isuses 15 minutes before. This error happens only when
multiple people login to the same page and perform simillar activity like employee self service form
etc. So where and what exactly could be the problem.

This is how we approached and resolved the issue.

First we checked apache error log. The following error was reported in the error log.

[warn] [client IP Address ] oc4j_socket_recvfull timed out
[error] [client IP Address ] mod_oc4j: There is no oc4j process (for destination: application://oacore) available to service request.
[error] [client  IP Address] mod_oc4j: request to OC4J [servername:OC4J AJP Port Range for Oacore]
failed: Connect failed

Above error message does give us a hint that the problem is with oacore, but as i said earlier the oacore is alive according to opmn.

Next, we verified the oacore log file.


Error Message in the file

Exception in thread “OC4JMonitorThread” java.lang.OutOfMemoryError: Java heap space

While checking the error in file, parallely we opened another window to see the CPU and memory usage and we could see one java process was taking more than 100% CPU. This process was spawned by the opmn -d process and the process id didnt match with the oacore process id. Hint: status will give the status as well the process id. It looked like a end client java process.

At this stage we had 3 options.

1. Kill the java process which is consuming high cpu.
2. Bounce oacore component
3. Adjust java (jvm) memory parameters

Each of the above options has its own implications and advantages. To minimize the downtime we decided  to kill the java processes, and the moment we killed the java process, all the browsers which were hanging reported instantaneously Internal Server Error. This proved to be a bad decision.

So we moved to the next option, bouncing the oacore service which we are sure will resolve the issue  (temporarily) and it did as expected. Basically when you bounce the services all the existing
connections and processess will be released which results in more available memory when you re-start the services.

Ok, now we tackled the problem and provided a temporary solution but we need to find a long term solution. This is option 3, adjusting java memory size.

Steps to change the heap size.

First, you need to identify how much is the maximum heap size that you can set. Click here.

Once you had identified the maximum heap size, we need to change the configuration files to make it

Step 1: Edit opmn.xml file.

Location : $INST_TOP/ora/10.1.3/opmn/conf/

Open opmn.xml

Search for string Xms or Xmx or module-id=”OC4J”

This search should lead you to below location

‘<‘process-type id=”oacore” module-id=”OC4J” status=”enabled” working-dir=”$ORACLE_HOME/j2ee/home”‘>’
‘<‘category id=”start-parameters”‘>’
‘<‘data id=”java-options” value=”-server -verbose:gc -Xmx512M -Xms128M ……]

The default value for Maximum (-Xmx) and Minimum (-Xms) heap sizes are 512M and 128M respectively.

Again here you have options, you can set both Xms and Xmx has the same value as Xmx if you feel all your sessoins require higher memory or set a lower value for Xms and the maximum value for Xmx. Dont forget to change the values under ‘<‘category id=”stop-parameters”‘>’

opmn.xml also contains jvm configurations for other components – oafm & forms.

Step 2: Edit file.

Location : ($INST_TOP/ora/10.1.3/j2ee/oacore/config)

This step is optional since we had already made changes in opmn.xml but there is no harm in making the change here. This step will come handy for troubleshooting specific components of Oracle viz., configurator, iSupplier or any other option which heavily utilizes/consumes CPU/memory.

Search for string Xms or Xmx or wrapper.

Option 1: If you find any of the above parameters change the values corresponding to the value you had  mentioned in opmn.xml or even more than that, as long as you dont exceed the maximum heap size limit.

Option 2: If you DO NOT find any of the above parameters, then make an entry like this, under the heading “# Java Object Cache Configuration Parameters”

wrapper.bin.parameters=-Xms[Value]M -Xmx[Value]M -XX:NewSize=256M -XX:MaxNewSize=256M

Step 3: Edit Applications Context file


Location: $INST_TOP/appl/admin/SID_hostname.xml

search for string s_oacore_jvm_start_options

Change Xms and Xmx value. Repeat the same step for parameter s_oacore_jvm_stop_options.

Changes made in Step 3 will take effect the next time you run autoconfig, whereas Step 1 & 2 changes will take effect the next time you bounce opmn services, but the values are not permanent in the sense these will be wiped off next time you run autoconfig. Yes you can preserve the changes by placing it inbetween Begin and End customizations.

You can also increase the Garbage Collection threads parameter (-XX:ParallelGCThreads) to a higher value if you have JDK 1.5 or more than 2 cpus or more memory. For more information on this you can refer to Metalink Note: 362851 – Guidelines to setup the JVM in Apps Ebusiness Suite 11i and R12.


Read Full Post | Make a Comment ( 8 so far )

How to identify/determine the maximum JVM Heap Size for OC4j components in R12 / 10gAS

Posted on September 11, 2008. Filed under: Apps Technology Stack, Configuration, Operating System, R12, TroubleShooting | Tags: , , , , , , |

Below command will help you to identify the maximum memory heap size than can be allocated to a JVM

java -mx[value]m -version

Start with a higher value like 4G or higher and slowly cut down to the maximum allowed value.


$ java -mx4096m -version

Invalid maximum heap size: -Xmx4096m
The specified size exceeds the maximum representable size.
Could not create the Java virtual machine.

As you can see my system cant support 4Gig for a JVM.

Now lets reduce to a lesser value, viz., 3Gig

$ java -mx3072m -version
java version “”
Java(TM) 2 Runtime Environment, Standard Edition (build
Java HotSpot(TM) Server VM (build jinteg:07.27.06-16:18 IA64, mixed mode)

Now its clear that the maximum allowed value is 3gig.


Read Full Post | Make a Comment ( 3 so far )

Active Users concurrent program never completes after clone

Posted on July 28, 2008. Filed under: Apps Technology Stack, Configuration, DBA/Admin Tips, Diagnostics, R12, TroubleShooting | Tags: , , |

Recently we did a clone (R12). Everything worked as expected, except for Active Users Concurrent program which never completed when submitted. The status remains Running – Normal for hours together. But “Prints environment variable values” concurrent program completed within no time.

This is how the issue was approached and resolved.

1. This is not a concurrent manager (FNDLIBR) issue, since the concurrent managers came up without any problem. – This eliminates concurrent manager to be part of the problem.
2. How about running some other concurrent program viz., Users of a responsibility or Active Users of a Responsibility. Same behaviour as Active Users – request never completes.

3. Now it is clear that the problem is with concurrent programs which has (Oracle Reports) .rdf defined as an executable, since “Prints environment variable values” completed sucessfully.

Thinking it could be a clone issue, i recompiled all the reports using adadmin, still the same issue.

To get more information, i enabled trace for Active users concurrent program (How to enable trace)and resubmitted it. Strange, it didnt even generate a trace file, but a session had been established on the database. Checking for the sql being run by the session, returns nothing.

hmmm now what …

4. Decided to run the report manually from the command prompt. (How to)

Atlast ….

Error: [cache directory location] does not exist.

Where is this defined ?

Checking $INST_TOP/ora/10.1.2/reports/conf/rwbuilder.conf revealed that the cacheDir value defined (same as the missing directory) in this file did not physically exists on the server. Created the cache directory and

Re-ran the report manually from command prompt, Guess what, the active users concurrent request completed sucessfully and even generated a trace file.


Read Full Post | Make a Comment ( 2 so far )

How to compile a single jsp file in Oracle Applications 11i/R12

Posted on May 28, 2008. Filed under: Apps Technology Stack, DBA/Admin Tips, R12, Scripts | Tags: , , , , |

You can follow below steps to compile a single jsp file (seeded or custom) in Oracle Applications 11i/R12.

1. Copy the jsp file to $COMMON_TOP/webapps/oacore/html (this location translates to $OA_HTML).

Placing the file under $OA_HTML enables end users to access the file using the url http://hostname.domain:port/OA_HTML/%5Bjspfile.jsp%5D

2. export PATH=$PATH:$FND_TOP/patch/115/bin
3. – -compile -s ‘[jspfile.jsp]’ (no space between the two dashes before compile)

No need to bounce apache server after compiling the file to access the new content.

Below is the command line help for command

ojspCompile.jsp command help

syntax: ./ COMMAND {ARGS}
COMMAND –compile update dependency, compile delta
–create rebuild entire dependency file
-delta.out update dependency, list delta to file
-dep.out update dependency, output heirarchy to file

ARGS -s matching condition for JSPs filenames
-p number of parallel compilations
-log to override logfile from ojspCompile.conf
You are
recommended to set the log file location
outside of any network file system shared (NFS) area/drive.
-conf to override ojspCompile.conf
–retry retry previously failed compilation attempts
–flush forces recompilation of all parent JSPs
–quiet do not provide an actively running progress meter
–fast instantly fail jsps that are *possibly* invalid

example1: –compile -s ‘jtf%’ -p 20 –retry
example2: –compile -s ‘jtflogin.jsp,jtfavald.jsp’ –flush
example3: –compile –fast –quiet


Read Full Post | Make a Comment ( None so far )

« Previous Entries

Liked it here?
Why not try sites on the blogroll...