Diagnostics
Unable to generate forwarding URL. Exception: oracle.apps.fnd.cache.CacheException
While trying to access the login page below error is reported.
“Unable to generate forwarding URL. Exception: oracle.apps.fnd.cache.CacheException”
This simple message doesnot give much information. Digging into logs yielded below detailed error message.
$INST_TOP/logs/ora/10.1.3/j2ee/oacore/oacore_default_group_1/application.log
html: Servlet error
oracle.apps.fnd.cache.CacheException
at oracle.apps.fnd.cache.AppsCache.get(AppsCache.java:228)
at oracle.apps.fnd.profiles.Profiles.getProfileOption(Profiles.java:1485)
at oracle.apps.fnd.profiles.Profiles.getProfile(Profiles.java:354)
at oracle.apps.fnd.profiles.ExtendedProfileStore.getSpecificProfileFromDB(ExtendedProfileStore.java:211)
at oracle.apps.fnd.profiles.ExtendedProfileStore.getSpecificProfile(ExtendedProfileStore.java:171)
at oracle.apps.fnd.profiles.ExtendedProfileStore.getProfile(ExtendedProfileStore.java:148)
at oracle.apps.fnd.common.logging.DebugEventManager.configureUsingDatabaseValues(DebugEventManager.java:1259)
at oracle.apps.fnd.common.logging.DebugEventManager.configureLogging(DebugEventManager.java:1114)
at oracle.apps.fnd.common.logging.DebugEventManager.internalReinit(DebugEventManager.java:1083)
at oracle.apps.fnd.common.logging.DebugEventManager.reInitialize(DebugEventManager.java:1050)
at oracle.apps.fnd.common.logging.DebugEventManager.reInitialize(DebugEventManager.java:1037)
at oracle.apps.fnd.common.AppsLog.reInitialize(AppsLog.java:595)
at oracle.apps.fnd.common.AppsContext.initLog(AppsContext.java:941)
at oracle.apps.fnd.common.AppsContext.initializeContext(AppsContext.java:926)
at oracle.apps.fnd.common.AppsContext.initializeContext(AppsContext.java:891)
at oracle.apps.fnd.common.AppsContext.<init>(AppsContext.java:751)
at oracle.apps.fnd.common.WebAppsContext.<init>(WebAppsContext.java:1014)
at oracle.apps.fnd.sso.Utils.getAppsContext(Utils.java:525)
at oracle.apps.fnd.sso.AppsLoginRedirect.AppsSetting(AppsLoginRedirect.java:120)
at oracle.apps.fnd.sso.AppsLoginRedirect.init(AppsLoginRedirect.java:170)
at com.evermind[Oracle Containers for J2EE 10g (10.1.3.0.0) ].server.http.HttpApplication.loadServlet(HttpApplication.java:2
232)
at com.evermind[Oracle Containers for J2EE 10g (10.1.3.0.0) ].server.http.HttpApplication.findServlet(HttpApplication.java:4
622)
at com.evermind[Oracle Containers for J2EE 10g (10.1.3.0.0) ].server.http.HttpApplication.findServlet(HttpApplication.java:4
546)
at com.evermind[Oracle Containers for J2EE 10g (10.1.3.0.0) ].server.http.HttpApplication.getRequestDispatcher(HttpApplicati
on.java:2822)
at com.evermind[Oracle Containers for J2EE 10g (10.1.3.0.0) ].server.http.HttpRequestHandler.doProcessRequest(HttpRequestHan
dler.java:740)
at com.evermind[Oracle Containers for J2EE 10g (10.1.3.0.0) ].server.http.HttpRequestHandler.processRequest(HttpRequestHandl
er.java:451)
at com.evermind[Oracle Containers for J2EE 10g (10.1.3.0.0) ].server.http.AJPRequestHandler.run(AJPRequestHandler.java:299)
at com.evermind[Oracle Containers for J2EE 10g (10.1.3.0.0) ].server.http.AJPRequestHandler.run(AJPRequestHandler.java:187)
at oracle.oc4j.network.ServerSocketReadHandler$SafeRunnable.run(ServerSocketReadHandler.java:260)
at com.evermind[Oracle Containers for J2EE 10g (10.1.3.0.0) ].util.ReleasableResourcePooledExecutor$MyWorker.run(ReleasableR
esourcePooledExecutor.java:303)
at java.lang.Thread.run(Thread.java:595)
Caused by: oracle.apps.jtf.base.resources.FrameworkException: Error in IAS Cache: java.lang.NullPointerException: null Connection
at oracle.apps.jtf.cache.IASCacheProvider.get(IASCacheProvider.java:712)
at oracle.apps.jtf.cache.CacheManager.getInternal(CacheManager.java:4802)
at oracle.apps.jtf.cache.CacheManager.get(CacheManager.java:4624)
at oracle.apps.fnd.cache.AppsCache.get(AppsCache.java:216)
… 30 more
Caused by: oracle.apps.jtf.base.resources.FrameworkException:
An exception occurred in the method CacheAccess.get
null
The base exception is:
null Connection
at oracle.apps.jtf.base.resources.FrameworkException.convertException(FrameworkException.java:607)
at oracle.apps.jtf.base.resources.FrameworkException.addException(FrameworkException.java:585)
at oracle.apps.jtf.base.resources.FrameworkException.<init>(FrameworkException.java:66)
at oracle.apps.jtf.base.resources.FrameworkException.<init>(FrameworkException.java:88)
at oracle.apps.jtf.base.resources.FrameworkException.<init>(FrameworkException.java:202)
at oracle.apps.jtf.base.resources.FrameworkException.<init>(FrameworkException.java:218)
at oracle.apps.jtf.base.resources.FrameworkException.<init>(FrameworkException.java:249)
… 34 more
There could be many root causes for this error. If you look closely into the errorstack, it complains about null connection.
Caused by:
oracle.apps.jtf.base.resources.FrameworkException:
Error in IAS Cache:
java.lang.NullPointerException:
null Connection
Every access to the login page (http://machine.domain.com:port/OA_HTML/AppsLogin) (in R12) establishes a connection to the database as GUEST user using the DBC file. (In 11i, the url page is a static html which doesnt establish a connection)
If the mid-tier gets a connection, it displays the login page else it throws “oracle.apps.fnd.cache.CacheException”.
Now lets look at some possible reasons for not getting a connection.
1. Database is down
2. Database is up but the listener is down.
3. Archive Location is full, which prevents new connections till freed-up.
4. The instance was recently cloned and the GUEST user password is not in sync.
When the instance is cloned from source, there is a remote chance that the guest user password is not in sync. By sync, i mean the profile option value and the dbc file value for guest user password.
Steps to verify the values (perform below steps in the mid-tier)
4.1 DBC File value
$ grep -i GUEST_USER_PWD $FND_SECURE/[SID].dbc
GUEST_USER_PWD=GUEST/GUEST
4.2 Profile Option Value
sqlplus apps/[apps_pwd]
select fnd_profile.value(‘GUEST_USER_PWD’) from dual;
FND_PROFILE.VALUE(‘GUEST_USER_PWD’)
——————————————————————————–
GUEST/GUEST
Values from 4.1 and 4.2 should be in sync and in our case it is. Now lets connect to the database using the guest password.
4.3 GUEST User Connectivity Check
sqlplus apps/[apps_pwd]
select FND_WEB_SEC.VALIDATE_LOGIN(‘GUEST’,’GUEST’) from dual;
FND_WEB_SEC.VALIDATE_LOGIN(‘GUEST’,’GUEST’)
——————————————————————————–
Y
5. Ok, i didnt do a clone and the GUEST user password is in sync. What next.
The next possible reason, the database is not allowing any new sessions. May be the maximum number of sessions defined at the database level had already reached.
If you had followed all the steps sequentially as defined above, you would have realised that database does accept new connections, since you were able to verify the GUEST user password profile option value. If SESSIONS parameter value had been reached you would have got the error ORA-00018: maximum number of sessions exceeded.
6. Now what, my database is accepting new sessions.
Next option is to look for stale database connections.
ps -fu [oracle_user] | grep -i ‘LOCAL=NO’
This command will give you a list of stale connections. Use below command to kill them and to free up resources.
CAUTION: BELOW COMMAND WILL KILL THE CONNECTIONS
ps -fu [oracle_user] | grep -i ‘LOCAL=NO’ | grep -v grep | awk ‘{print “kill -9 “$2}’ | ksh
Hope one of the above mentioned steps resolves your issue when you encounter “Unable to generate forwarding URL. Exception: oracle.apps.fnd.cache.CacheException”
cheers,
OraclePitStop.
How to enable trace with BIND variables and WAITs for a concurrent program?
In this link, we had mentioned about tracing a concurrent program, but the drawback of this method is it doesnt yield information about the BIND variable values. So in this post we will talk about it.
Scenario
X submitted a concurrent request and its running for a long time and never completes. X approached you to see what is the problem with the concurrent request.
Steps
1. Inform X to logout from his ERP session.
2. Login as sysadmin user
3. System Administrator -> Profile (System)
4. Values: User = X, Profile = Initialization SQL Statement – Custom (you can also mention Initialization%Custom)
5. Under user field enter the following
begin fnd_ctl.fnd_sess_ctl(”,”,’TRUE’,’TRUE’,’LOG’,’ALTER SESSION SET EVENTS=”10046 TRACE NAME CONTEXT FOREVER, LEVEL 12” TRACEFILE_IDENTIFIER=”[ANY_IDENTIFIER]”‘);end;
[ANY_IDENTIFIER] = Any value to identify the trace file.
LEVEL 12 – Both Binds and Waits
LEVEL 8 – Only WAITS
LEVEL 4 – Only BIND Variables
All above levels will include the sql statements being executed and not only the binds and waits.
6. If tracing had been enabled at the concurrent program level as mentioned in this link, please disable it by removing the check box. [IMPORTANT]. If you have trace enabled for the concurrent program, then it will take precedence for the profile option.
7. Kill / cancel the long running concurrent request (if its still running)
8. Request X to login to ERP and instruct X to submit the concurrent request again and logout. Nothing else. This will ensure we capture the trace information specific to the concurrent request and not for other actions.
At this stage, you will find a trace file generated at $ORACLE_HOME/admin/SID_Hostname/udump with [ANY_IDENTIFIER] as part of it.
Inside the trace file you will be able to see information about the bind variable values being passed to and used by the concurrent program.
Eg:
Bind#0
oacdty=12 mxl=07(07) mxlc=00 mal=00 scl=00 pre=00
oacflg=13 fl2=206001 frm=00 csi=00 siz=8 off=0
kxsbbbfp=9fffffffbf2e88d8 bln=07 avl=07 flg=09
value=”[BIND VARIABLE VALUE]”
Bind#1
oacdty=12 mxl=07(07) mxlc=00 mal=00 scl=00 pre=00
oacflg=13 fl2=206001 frm=00 csi=00 siz=8 off=0
kxsbbbfp=9fffffffbf2e88f8 bln=07 avl=07 flg=09
value=”[BIND VARIABLE VALUE]”
Note: Make sure you remove the profile option value for X after the job is done, else every action of X will generate a trace file in udump directory.
Happy Trouble shooting.
cheers,
OraclePitStop
How to clear cache for a specific component in R12?
As we all know the normal tendency for clearing cache in Oracle Apps is to delete the $COMMON_TOP/_pages directory. In 11i, after deleting we dont have to recompile the jsps but in R12 we have to recompile the pages since s_jsp_main_mode is set to justrun by default which is in turn updated in orion-web.xml. Click here for more info.
Ok coming to this post, how can i clear cache for a specific component of Oracle Apps R12 viz., iprocurement, istore etc., Is this even possible ? Yes it is and this option comes handy when somebody modifies a jsp page which is part of say istore, then it makes sense to clear cache for istore and not for the whole server.
Clearing cache for specific Component/Application in R12
Login as sysadmin
Select “Functional Administrator” responsibility
Go to Core Services ==> Caching Framework ==> Tuning
Query the application (for example iStore) or by Name or code.
Select the component and clear the cache.
Note: For some specific configuration changes, we need to bounce Apache also.
Clearing Global Cache in R12
a) Go to Functional Administrator Responsibility
b) Go to Core Services ==> Caching Framework
c) Go to Global configuration and clear all the cache from there.(Only If you want to clear all the cache)
Hope this helps…..
Thanks,
Sam.
About Sam: Sam is a guest author for this blog and had performed various installations & upgrades. He is well versed with tech stack components and applications maintenance. He can be contacted at gksam4u@gmail.com
Read Full Post | Make a Comment ( 1 so far )How to enable / generate trace for a form 11i / R12
This step can be performed by any user who has access to the professional approach (ie., the forms not the jsp). By access i meant any responsibility which launches the form.
Steps: (In the forms window)
1. Click on Help -> Diagnostics -> Trace
2. Under trace you have 5 options – select the appropriate option
3. Key in the apps password in the prompted window.
4. The next window will specify the logfile name and location.
cheers,
OraclePitStop
Blank AppsLogin / AppsLocalLogin Page in R12?
There were quite a few times i had encountered this blank page issue and the root cause was not the same all the times. There are many reasons for this issue to occur. Listed below are some of them.
Scenario 1: After a clone
Solution :
1. adopmnctl.sh stopall — to shutdown opmn managed services viz., opmn, forms, oacore & oc4j
2. cd $COMMON_TOP/_pages
3. mv _pages _pages_old
4. cd $FND_TOP/patch/115/bin or export PATH=$FND_TOP/patch/115/bin:$PATH
5. Compile all jsps.
perl ojspCompile.pl –compile –flush -p 2
(-p represents parallel threads, if you have enough horse power on your server, you can increase it to 8 or 10)
6. adopmnctl.sh startall
Scenario 2: Database Listener is down
Solution: Obvious answer, start the database listener.
Scenario 3: Database Archive Locaion is 100% full
Solution: Either temporarily move the archive files to another location or delete older archive files (provided you had taken a cold backup sometime).
Scenario 4: IAS Cache initialization failed
This issue occurs only if you have more than one mid-tier and you had enabled Distributed Java Caching.
Error Message in log file:
Exception in static block of jtf.cache.CacheManager. Stack trace is: oracle.apps.jtf.base.resources.FrameworkException:
IAS Cache initialization failed.
The Distributed Caching System failed to initialize on port: 12357.
The list of hosts in the distributed caching system is: [node 1] [node 2].
The port [port no] should be free on each host running the JVMs.
at oracle.apps.jtf.cache.IASCacheProvider.init(IASCacheProvider.java:214)
at oracle.apps.jtf.cache.CacheManager.activateCache(CacheManager.java:1451)
In this scenario, s_java_object_cache_port value was listening on node 2 but not on node 1. So whichever connection went to node 1 it returned a blank page and occasionally it displayed 500 Internal Server error.
Solution:
Autoconfig
1) Bring down opmn services on both the boxes.
2) Verify no process is listening for s_java_object_cache_port value. click here to view the commands.
2) Change s_java_object_cache_port on all the nodes in the context file
3) Run autoconfig
Non-Autoconfig
s_java_object_cache_port value is referrenced in two places
1) $INST_TOP/ora/10.1.3/javacache/admin/javacache.xml
2) Profile option name JTF_DIST_CACHE_PORT
Change the port value to a new number and bounce opmnctl services. Dont forget to change it in $CONTEXT_FILE. (To take effect during next autoconfig)
Verification:
Use commands listed in step 2 of autoconfig option.
If the new port mentioned still doesnt listen on the boxes you can use below workaround. This workaround will resolve the “IAS Cache initialization failed” even if the port is not listening.
Workaround
Set LONG_RUNNING_JVM=false in $INST_TOP/ora/10.1.3/j2ee/oacore/config/oc4j.properties
Bounce opmn services.
In 11i, if the database listener is down or if the archive location is full, atleast we get the internal server page while trying to access the login page, but in R12 nothing happens except for the blank page and apache error log will have an entry “oc4j_socket_recvfull timed out” with additional error message mod_oc4j: request to OC4J [mid-tier servername:ajp port] failed: Connect failed
So the moral of the story is recompiling the jsps is not always the solution if you get a blank login page.
cheers,
Ram.
SQL Query to view sql of a session
SELECT
T.sql_text
FROM
v$session S,
v$sqltext_with_newlines T
WHERE
S.sid = &sid
AND S.serial# = &serial
AND S.sql_address = T.address
AND S.sql_hash_value = T.hash_value
ORDER BY T.piece
/
Replace v$session with gv$session if the database is running on RAC configuration.
cheers,
Ram.
Post Clone – Forms Error – oracle.forms.engine.RunformException: Forms session failed during startup: no response from runtime process
We recently did a clone and everything went well – autoconfig ran properly – services started sucessfully – but when we selected a responsibility which will launch a form it errored out with below message :
formsweb: Forms session exception stack trace:
oracle.forms.engine.RunformException: Forms session failed during startup: no response from runtime process
at oracle.forms.servlet.RunformProcess.connect(Unknown Source)
at oracle.forms.servlet.RunformProcess.dataToRunform(Unknown Source)
at oracle.forms.servlet.RunformSession.dataToRunform(Unknown Source)
at oracle.forms.servlet.ListenerServlet.doPost(Unknown Source)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:763)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:856)
Checking the log file revealed more information
formsweb: Forms session aborted: runtime process failed during startup with errors /usr/lib/hpux32/dld.so: Unsatisfied code symbol ‘nnftboot’ in load module ‘/opt/var/applbin/apps/tech_st/10.1.2/lib32/libclntsh.so.10.1’.
/usr/lib/hpux32/dld.so: Unsatisfied code symbol ‘nnfoboot’ in load module ‘/opt/var/applbin/apps/tech_st/10.1.2/lib32/libclntsh.so.10.1’.
/usr/lib/hpux32/dld.so: Unsatisfied code symbol ‘nnfoboot’ in load module ‘/opt/var/applbin/apps/tech_st/10.1.2/lib32/libclntsh.so.10.1’.
/usr/lib/hpux32/dld.so: Unsatisfied code symbol ‘nnfhboot’ in load module ‘/opt/var/applbin/apps/tech_st/10.1.2/lib32/libclntsh.so.10.1’.
/usr/lib/hpux32/dld.so: Unsatisfied code symbol ‘nnfhboot’ in load module ‘/opt/var/applbin/apps/tech_st/10.1.2/lib32/libclntsh.so.10.1’.
/usr/lib/hpux32/dld.so: Unsatisfied code symbol ‘nnflboot’ in load module ‘/opt/var/applbin/apps/tech_st/10.1.2/lib32/libclntsh.so.10.1’.
/usr/lib/hpux32/dld.so: Unsatisfied code symbol ‘nttini’ in load module ‘/opt/var/applbin/apps/tech_st/10.1.2/lib32/libclntsh.so.10.1’.
/usr/lib/hpux32/dld.so: Unsatisfied code symbol ‘ntusini’ in load module ‘/opt/var/applbin/apps/tech_st/10.1.2/lib32/libclntsh.so.10.1’.
/usr/lib/hpux32/dld.so: Unsatisfied code symbol ‘ntpini’ in load module ‘/opt/var/applbin/apps/tech_st/10.1.2/lib32/libclntsh.so.10.1’.
/usr/lib/hpux32/dld.so: Unsatisfied code symbol ‘ntzini’ in load module ‘/opt/var/applbin/apps/tech_st/10.1.2/lib32/libclntsh.so.10.1’.
As a normal standard practice we normally recreate all the symbolic links under APPL_TOP, but still we got this error. Now heres the
Solution :
Stop the forms services (if possible run adopmnctl.sh stopall)
cd $ORACLE_HOME/forms/lib32
make -f ins_forms.mk install
restart the services.
cheers,
OraclePitStop
Active Users concurrent program never completes after clone
Recently we did a clone (R12). Everything worked as expected, except for Active Users Concurrent program which never completed when submitted. The status remains Running – Normal for hours together. But “Prints environment variable values” concurrent program completed within no time.
This is how the issue was approached and resolved.
1. This is not a concurrent manager (FNDLIBR) issue, since the concurrent managers came up without any problem. – This eliminates concurrent manager to be part of the problem.
2. How about running some other concurrent program viz., Users of a responsibility or Active Users of a Responsibility. Same behaviour as Active Users – request never completes.
3. Now it is clear that the problem is with concurrent programs which has (Oracle Reports) .rdf defined as an executable, since “Prints environment variable values” completed sucessfully.
Thinking it could be a clone issue, i recompiled all the reports using adadmin, still the same issue.
To get more information, i enabled trace for Active users concurrent program (How to enable trace)and resubmitted it. Strange, it didnt even generate a trace file, but a session had been established on the database. Checking for the sql being run by the session, returns nothing.
hmmm now what …
4. Decided to run the report manually from the command prompt. (How to)
Atlast ….
Error: [cache directory location] does not exist.
Where is this defined ?
Checking $INST_TOP/ora/10.1.2/reports/conf/rwbuilder.conf revealed that the cacheDir value defined (same as the missing directory) in this file did not physically exists on the server. Created the cache directory and
Re-ran the report manually from command prompt, Guess what, the active users concurrent request completed sucessfully and even generated a trace file.
cheers,
OraclePitStop.
« Previous Entries
OraclePitStop - Lets share some knowledge 