Web Plugin Framework stopped working
Web Plugin Framework (web admin UI) stopped working on a Remote Expert Mobile HA Cluster
|Problem Summary|| Remote Expert Mobile Server is installed in HA mode, i.e. Master node and at least one Slave node is installed.
While accessing web plugin framework using following URL:
User fails to use web admin UI with error message being displayed based on web page being loaded.
Server log on master node would contain ERROR messages related to "PerformanceLogCache". Below is an example:
2015-04-20 23:03:06,325 ERROR [org.infinispan.loaders.file.FileCacheStore] (MSC service thread 1-10) | ISPN000062: Error while reading from file: /opt/cisco/10.6.1.10000-3/REAS/domain/servers/appserver-REAS-A/data/infinispan/fcsdk/PerformanceStatistics/-978329600: java.io.EOFException: The stream ended unexpectedly. Please check whether the source of the stream encountered any issues generating the stream.
|Error Message||Web plugin page will receive 500 error or some other error based on page being requested,|
Issue is caused by time not being in sync on between Master and Slave(s) node of the HA Cluster.
This result in infinispan cache being corrupted hence cluster fails to respond correctly.
|Recommended Action|| Resolution of this issue require 2 steps:
Step 1 Ensure time and date are in sync between all REAS cluster elements:
It is recommended to use NTP service for this purpose. Please check with your network administrator if you are unsure about your NTP server.
Below are some example commands to help you with NTP server config:
server <your ntp server address>
# service ntpd status
synchronised to <ntp server name> at stratum 3
Step 2 Delete corrupted cache files:
Following steps can be taken to delete corrupted cache files:
service reas stop
cd <reas server install directory>/domain/servers/appserver-<node address>/data/infinispan/fcsdk/PerformanceStatistics/
Note: If the files names start with a - e.g. -12345 you need to run rm -- -12345 to remove it.
service reas start
|Release||Remote Expert Mobile|
|Associated CDETS #|| None|