There have been cases in analytics repository migration where the entire repository (popularly known as rpd in the developers circle) files needs to be moved to other environments due to code merge issues or developers not adhering to coding standards or due to any unforeseen issues. In such cases where the repository has multiple physical databases configured, it becomes a tedious and time consuming process to update the connection pool credentials (UID, PWD Data source name each of the connection pools created) manually and is also prone to mistakes. These errors might lead to incorrect database being queried upon or account being locked. To avoid such issues, one can use the nqUDML executables to automate this entire process.
 
Process to update development rpd with QA credentials during the migration.
 
Open the existing QA repository using the Administration Tool. Copy + Paste the connection pool to a notepad file. One can see the UDML definition of the connection pool. This UDML definition gives the Datasource name, UID and the encrypted password. Save this notepad file (dev2qa.txt or any name which identifies it with what it contains)
 
By using nqUDMLExec command line utility provided with Oracle BI, one can update the connection pool. It requires and accepts the following arguments.
 
-U : Administrator
-P : Password of the base repository
-I  : input script (in this case is the dev2qa.txt file)
-B : Base repository (in this case is the development rpd)
-O : Output repository (the updated repository with development metadata and QA credentials)
 
Usage: 
:\>nqUDMLExec –U Administrator –P SADMIN –I dev2qa.txt –B dev.rpd –O qa.rpd
 
 
Advantages
 
·         Multiple connection pools can be updated at the same time without manual intervention.
·         After the first time setup of the passwords in the connection pool by the DBA/OBI Administrators, they are encrypted and cannot be misused by developers.
·         Static variables, used in DSN names and UIDs can also be updated using this method.
·         Local repository users and groups and also be administered by having a one-time setup patch file for each environment.

offline page can be achieved using following techniques -
 1.URL Redirect
 2.using web.config / .htaccess file
 3.using default.htm
 4.Siebel offline page using IIS URL Rewrite module
 
To host the offline page in Siebel during the application maintenance window, follow the below steps.
1. Create a new html file with the message you want to show when the Siebel is servers are down.
2. Rename the default.htm to orginal_default.htm in C:\ sia81\SWEApp\PUBLIC\enu
3. Copy the new htm offline page to C:\ sia81\SWEApp\PUBLIC\enu\default.htm
When you go to the application URL you will see the offline page
http://localhost/callcenter_enu
To bring back the normal Siebel screen, replace the default.htm with original default.htm file.
Siebel web server uses the default.htm as the entry point into the application. This file contains a function called GotoUrl(url), when you change the default.htm with offline page it displays the offline page.
function GotoUrl(url)
{
// Append the current hostname to the server request so that the server has
// the top level host name. This is needed to support reverse proxy servers.
url += "&SWEHo=" + this.location.hostname;
this.location = unescape(this.location.pathname) + url;
}
GotoUrl('start.swe?SWECmd=Start');
To host the offline page in Siebel during the application maintenance window, follow the below steps. 1. Create a new html file with the message you want to show when the Siebel is servers are down. 2. Rename the default.htm to orginal_default.htm in C:\ sia81\SWEApp\PUBLIC\enu 3. Copy the new htm offline page to C:\ sia81\SWEApp\PUBLIC\enu\default.htm When you go to the application URL you will see the offline page http://localhost/callcenter_enu To bring back the normal Siebel screen, replace the default.htm with original default.htm file. Siebel web server uses the default.htm as the entry point into the application. This file contains a function called GotoUrl(url), when you change the default.htm with offline page it displays the offline page. function GotoUrl(url) { // Append the current hostname to the server request so that the server has // the top level host name. This is needed to support reverse proxy servers. url += "&SWEHo=" + this.location.hostname; this.location = unescape(this.location.pathname) + url; } GotoUrl('start.swe?SWECmd=Start');

 Siebel Application Deployment Manager is not designed to deploy changes made to the configuration of enterprise parameters, servers, and components.These changes can be made either through the server management screens and views in the Siebel Web Client or through the Siebel Server Manager (srvrmgr) command line utility.
We already know that the storage location of this information is the siebns.dat file, which is managed by the Siebel Gateway Name Server.
Because some of the parameters refer to enterprise-specific paths or host names such as the file system path or the chart server host name, we cannot simply copy the siebns.dat file from one enterprise to the other.
The cfgmerge command line utility is one possible solution to achieving a merge of configuration stores.
The utility compares two siebns.dat files—target and source—and produces an input file for the Siebel Server Manager (srvrmgr) command line utility. This file contains the necessary commands to change those parameters in the target environment that have been modified in the source environment. The input file must be reviewed, modified if necessary, and then executed using the srvrmgr command line.
Deploying New Component Definitions from Source to Target Enterprises
The cfgmerge utility can produce input files on the enterprise level and the individual server level. It can only compare parameters, component definitions, and server components that are already present in both siebns.dat files and have the same names in both files.
If we wish to use the cfgmerge utility to deploy new component definitions from the source system to the target system, we have to create them manually in the target system by simply copying and renaming an existing component definition in the graphical user interface or by using a script and then running the utility.
 
Using the Cfgmerge Utility
The following list gives an example procedure of how to use the cfgmerge utility:
 
Log in to the Siebel Server Manger (srvrmgr) command line against the source enterprise.
Use the backup namesrvr command to create a backup of the current siebns.dat source file.
Rename the backup copy to siebns_source.dat.
Log in to the Siebel Server Manger (srvrmgr) command line against the target enterprise.
Use the backup namesrvr command to create a backup of the current siebns.dat target file.
Rename the backup copy to siebns_target.dat.
Copy both files created in the previous steps to a temporary directory on a machine where a Siebel Gateway Name Server is installed.
On this machine, open a command shell and navigate to the bin directory of the Siebel Gateway Name Server installation folder.
Execute a command similar to the following to create the input file at the enterprise level.
 
cfgmerge -l ENU -i D:\cfgmerge\siebns_source.dat,D:\cfgmerge\
siebns_target.dat -e SIEB_DEV,SIEB_TEST -o D:\cfgmerge\DEV_TEST_
ENT.txt
The above command invokes the cfgmerge utility. The -l parameter takes a three letter language code. The -i switch is followed by a comma-separated list of the full paths to the file representing the source siebns.dat and the file representing the target siebns.dat. The -e switch is followed by a comma-separated list of the names of the source enterprise and the target enterprise. The -o parameter value specifies the path where the output file should be written to.
Execute a command similar to the following to create the input file at the server level:
 
cfgmerge -l ENU -i D:\cfgmerge\siebns_source.dat,D:\cfgmerge\
siebns_target.dat -e SIEB_DEV,SIEB_TEST -s Dev_1,Test_1 -o D:\
cfgmerge\DEV_TEST_SRV.txt
The above command is similar to the one that generates the enterprise level output. The difference is the -s switch, which is followed by a comma-separated list of the names of the two Siebel servers to compare.
 
If necessary, we repeat the command for any additional Siebel servers in the source or target enterprise.
 
Review and modify the output files if necessary. It is mandatory to thoroughly review the output files of the cfgmerge utility in order to avoid unwanted changes being applied to the target enterprise configuration.
 
Log in to the Siebel Server Manager (srvrmgr) against the target enterprise and execute a command similar to the following to apply the changes:
 
 
read D:\cfgmerge\DEV_TEST_ENT.txt
read D:\cfgmerge\DEV_TEST_SRV.txt
The read command opens the specified file and executes all commands in that file.
 
The process is completed by shutting down and restarting all services of the target enterprise. If end users are affected by the shutdown, we must of course ensure that the end user community is notified of the downtime well in advance.

When an end user clicks a view tab to navigate to a new view, a chain of processes is invoked on different machines in order to handle the request. For example, the Siebel Web Server Extension must connect to the application object manager (AOM) on the Siebel server. The AOM will issue an SQL statement to the Siebel database and the Siebel Web Engine (SWE) will put the view layout and data together and render the view.

In order to be able to make exact measurements to determine performance bottlenecks quickly, Siebel engineers have included instrumentation points in the core program code of all Siebel software components.

This feature is commonly known as Siebel Application Response Measurement (SARM). The following diagram explains how SARM works:

From the above diagram, we can learn the following about Siebel Application Response Measurement:

1. Requests from the web client are processed on the Siebel Web Server Extension (SWSE) on the web server machine.

2. The request is passed on to the application object manager on the Siebel server machine.

3. The application object manager requests data from the Siebel database.

4. The database returns the data.

5. When the Siebel code executes, a buffer in the machine's physical memory is filled with information about each single event.

6. Once the memory buffer is full or a timeout is reached, the buffer's content is flushed to files on the hard disk. These files have a .sarm suffix.

7. Administrators can use the sarmquery command line utility to convert the binary content of the .sarm files to readable formats such as CSV (comma separated values) or XML. Alternatively, SARM information can be viewed directly at the command prompt.

Next, we will discuss these major administrative tasks related to Siebel Application Response Measurement:

• Enabling SARM

• Managing SARM files

• Using sarmquery to read SARM files

• Automating SARM data retrieval

Enabling SARM

The default behaviour of Siebel software is not to produce .sarm files. In order to turn on the production of .sarm files, we have to set parameters (for Siebel Servers and their components) or environment variables (for all other variants of Siebel software such as the Siebel Web Server Extension or Siebel clients).

Enabling SARM for Siebel Servers and Components

We can use the graphical user interface (GUI) provided by the views in the Administration - Server Configuration screen or the Siebel Server Manager (srvrmgr) command line utility to set the SARM-related parameters. It is recommended to enable SARM only for a specific set of server components rather than for an entire Siebel server in order to keep the performance impact to a minimum.

The following table describes the SARM-related parameters:

The following procedure describes how we can enable SARM for an application object manager component:

1. Log in to the Siebel Server Manager (srvrmgr) command line using a command similar to the following at the Siebel server's bin directory:

   srvrmgr /g osappeval4 /e SIEBEL_EVAL /u SADMIN /p TJay357D /s Eval_1

The above example command will connect to the Siebel Gateway Name Server on the host osappeval4. The enterprise name is SIEBEL_EVAL, username is SADMIN, and the server to be set at the prompt is Eval_1.

1. At the srvrmgr> prompt, enter a command similar to the following:

   change param sarmlevel=2,sarmperiod=1,sarmthreshold=100 for comp sccobjmgr_enu

The above example command enables SARM at level 2 for the English Siebel Call Center object manager. The memory buffer's content will be flushed once a minute; only events that took longer than 100 milliseconds will be written to the .sarm files. These parameter changes are effective immediately, so no component or server restart is required.

Enabling SARM for Other Siebel Software Units

Because of the fact that only Siebel servers and their components can read the parameters from the Siebel Gateway Name Server's configuration store, we must use environment variables to enable SARM for other Siebel software units such as the Siebel Web Server Extension (SWSE) or Siebel clients such as Siebel Tools or the Siebel Mobile Web Client.

The following table relates the SARM environment variables to the parameters discussed above. Basically, the environment variable's name can be derived by adding the prefix "SIEBEL_" to the parameter name.

The following procedure describes how to set the SARM environment variables on Microsoft Windows-based operating systems:

1. Right-click the My Computer icon and select Properties.

2. In the System Properties dialog, click the Advanced tab.

3. Click the Environment Variables button.

4. In the System Variables area, click the New button.

5. Enter the correct values in the Variable Name and Variable Value fields and click OK.

6. Repeat the previous step for each environment variable you need to set.

7. Restart the application (for example the Siebel Developer Web Client or the web server).

We can verify that SARM is properly enabled by inspecting the log directory of the installation folder of the Siebel software unit. SARM files should appear in this directory after the time specified in the SARMPeriod parameter. The screenshot below shows the Siebel Server's log directory with .sarm files:

The naming convention for sarm files is as follows:

A_<Server component name>_T<Timestamp>_P<Process Id>_N<counter>.sarm

If we used the SARMLogDirectory parameter to specify a different storage location for .sarm files, we must inspect the respective directories.

Managing SARM Files

It is imperative for Siebel administrators to understand how to create usable sets of .sarm files without cluttering the server's hard drives. In environments with hundreds or even thousands of end users, the amount and size of .sarm files can grow out of bounds quickly.

The following techniques ensure that we harvest a reasonable amount of information without affecting the overall system performance more than necessary.

Always set the SARMThreshold parameter to a value of 10 or higher. This avoids too much data being written to the .sarm files. Because SARM is often used to trace long-running and therefore performance-hampering events, we can ignore the events that execute very fast.

Setting the SARMThreshold for example to a value of 100 reduces the amount of data written to the .sarm files by an average of 90%. This can be easily verified by comparing .sarm file sizes at different settings of this parameter.

Adjust the amount of .sarm files for a single process using the SARMMaxFiles parameter. After a short evaluation period, we should be able to determine the ideal number of files. We should always have the amount of files that is needed to cover a typical monitoring period. For example, if we monitor a work day (9:00 am to 5:00 pm) we should set the SARMMaxFiles parameter to a number which ensures that .sarm files for an individual process do not get overwritten before eight hours.

With regard to reading the .sarm files with the sarmquery utility or using the Siebel Diagnostic Tool, the default folder—the log directory of the Siebel software unit—should be kept as the storage location. However, we can use scheduled scripts to copy the .sarm files to an archive folder to avoid loss of data.

Using Sarmquery to Read SARM Files

The sarmquery command line utility is the primary means of converting the binary content of the .sarm files to readable output. We can choose between various output formats such as text, XML, or csv files or we can consume the output directly at the command prompt.

We can locate the sarmquery executable in the bin directory of the Siebel server's or the Siebel Gateway Name server's installation folder. In addition, the Siebel Management Agent uses sarmquery to support the Siebel Diagnostic Tool, which is discussed later in this chapter.

When we enter sarmquery at a command prompt in the Siebel server's bin directory, the utility reveals the basic usage information. The following screenshot shows the output of the sarmquery command in a Microsoft Windows command shell:

It is recommendable to use the information provided to produce text files that contain the information obtained by the -tips, -help, and -macrosyntax options.

For example, a command similar to the following will create a tips.txt file with useful first step information in the Siebel server's bin directory:

sarmquery -output verbose=tips.txt -tips

The above command uses the -output option followed by the verbose key with a value of tips.txt and the -tips option. This command produces a text file of the standard output of the -tips option. We can repeat this command to create a help.txt and macrosyntax.txt file by modifying the above command accordingly.

The main syntax of the sarmquery utility is as follows:

sarmquery -option key=value -input <directory or file>

The sarmquery command line utility accepts several options, which must be preceded by a dash ("-") and followed by a key=value pair. The -input option must be followed by the name of either a single .sarm file or a directory that contains .sarm files. If we use the directory option, sarmquery will scan the given directory and all its subdirectories recursively for .sarm files.

The following table describes the major options and example key=value pairs for the sarmquery utility:

Each option or key can be abbreviated as long as the abbreviation is not ambiguous. For example, the following is a valid sarmquery command:

sarmquery -i ..\log -agg sub -sel area=dbc

The above example uses the -input option (abbreviated to -i), the -aggregate (-agg) option followed by the subarea (sub) key and the -select (-sel) option. The abbreviation technique might come handy while using the command line but we should consider using the full text when creating scripts or macro files for the sake of better readability.

For a complete list of possible keys, we can consult the help file (generated using the -help option) or the Performance Tuning Guide in the Siebel bookshelf.

In the following, we will explore common scenarios for sarmquery:

• Specify the start and end time

• Application performance by area and subarea

• Time histograms

• Identify slow performing objects

• Create SARM output files

Specifying the Start and End Time

In a typical sarmquery scenario, we most probably want to analyze data from a specific timeframe. In order to do so, we must use the -select option for the starttime and endtime keys in every sarmquery call we make.

We can use a command similar to the following to obtain a list of users who logged on in the specified timeframe:

sarmquery -input ..\log -aggregate user -select starttime="09:00:00" -select endtime="17:00:00"

The above command—issued at the bin directory of the Siebel server—will read all .sarm files in the Siebel server's log directory and create an aggregate plot for all users who had a session between 9:00 AM and 5:00 PM. If no date is given, sarmquery takes the current day into consideration. If we wish to specify a date, we must use the same syntax as in the example below:

-select starttime="2010-05-20 09:00:00"

The example specifies 9:00 AM on the 20th of May 2010 as the start time.

The output of the above command is similar to the following:

The previous screenshot shows the typical output of sarmquery at the command line. Aggregates are plotted as horizontal bar graphs with the average response time as the scale.

Application Performance by Area and Subarea

When SARM collects event data, it not only collects the timestamp, the user, and response timings but also the area and subarea of the Siebel application infrastructure where the event occurred.

A full list of SARM data collection areas can be found in the Siebel Performance Tuning Guide in the Siebel bookshelf documentation. The following table lists the most prominent areas:

Each area has one or more subareas. We can obtain a comma-separated value (CSV) file listing all areas and subareas using the following command:

sarmquery -output map=map.csv

The above command produces a text file in CSV format, which describes all available SARM areas and subareas.

Analyzing Areas, Subareas and Instances

We use a sarmquery command similar to the following to get a horizontal bar chart, which helps us identify the area in which most of the process time is spent:

sarmquery -aggregate area -input ..\log

The above command reads all .sarm files from the log directory of the Siebel server and produces a chart similar to the screenshot below:

The above screenshot shows the output of the -aggregate area option. The chart lists all areas (sorted descending by the percentage of process time spent in the area) for which events have been recorded in the .sarm files. In the above example, 25.35% of the total process time recorded by SARM is spent in events that belong to the INFRA area.

We can "drill down" into each area by using commands similar to the following:

sarmquery -select area=objmgr -aggregate subarea -input ..\log

The above command uses the -select option to filter the output for the OBJMGR area and creates a bar chart that shows the share of process time spent for each subarea.

We can further "drill down" on the data. For example to see a list of business services that use the processing time, we use a command similar to the following:

sarmquery -select subarea=objmgr_bsvc_invoke -aggregate instance -input ..\log

The above command uses the -aggregate instance option. The instance key typically provides the name of the object involved. In the case of the subarea objmgr_bsvc_invoke, business service names are shown in the instance field.

Time Histograms

A common server monitoring scenario is the retrieval of application response data over time. The data contained in the .sarm files can be interpreted in this way using the -aggregate time option. A command similar to the following produces a histogram of the application response time for a specific day:

sarmquery -input ..\log -aggregate time=10 -select starttime="2010-02-18 00:00:00" -select endtime="2010-02-19 00:00:00"

The above command produces a response time plot at 10-minute intervals (specified by the time=10 key=value pair) over the period specified as the timeframe between the starttime and endtime key values.

Identifying Slow Performing Objects

One of the main purposes of SARM is the quick and easy identification of performance bottlenecks. We have learned above that we can aggrgate the response time information by area, subarea, and instance.

By using the correct subarea as a filter criteria in the -select option, we can create response time histograms for objects such as views, scripts, or business services and identify slow performers.

The following example command allows us to identify slow performing views with a build time over five seconds:

sarmquery -input ..\log -agg instance -select subarea=SWEPAGE_VIEW_BUILD -select selftime=5000

The above command produces a response time histogram for all views that required more than five seconds (specified as 5000 milliseconds in the selftime key value) to build (subarea=SWEPAGE_VIEW_BUILD). The following screenshot shows the output of the example command:

Views of the server management screens typically take a longer time to build because data is being fetched from the Siebel Gateway Name Server. The high response times for the Home Page View are resulting from the fact that it is the application's start view.

Similar commands can be issued using other areas such as SCRIPT (to identify slow running scripts) or OBJMGR (to identify slow performing business layer objects) or their respective subareas.

Creating SARM Output Files

The sarmquery command line utility can be used to export the content of the .sarm files to various output formats such as plain text, comma-separated values (CSV), or XML. Typically, these files are loaded into third-party software such as Microsoft Excel to further analyze the SARM data sets. We use the -output option to create SARM output files.

One of the most prominent output commands can be seen in the following example:

sarmquery -input ..\log -output sarm=sarmdata.csv#10000

The above command writes the raw SARM data to a series of CSV files. By specifying the suffix #10000, after the file name, we determine that each CSV file will contain only up to 10000 rows of data.

The CSV files can then be consumed for further analysis in spreadsheet applications such as Microsoft Excel or business intelligence tools such as Oracle Business Intelligence Enterprise Edition. The following screenshot shows a SARM .csv output file opened in Microsoft Excel:

The filter feature of Microsoft Excel allows quick creation of filtered sets of data for easier analysis.

In contrary to the raw data output of the sarm key, we can use the agg key of the -output option to write aggregated information to text or XML files. A command similar to the following yields an XML file containing information aggregated at the area level:

sarmquery -input ..\log -aggregate area -output agg=area_agg.xml

The above command uses the agg key followed by an XML file name and produces an output similar to the following screenshot.

<?xml version="1.0" encoding="UTF-8" ?> - <xml> <sarm version="20041117"/> + <Group type="Area" name="INFRA" pctCount="0.054" pctSelfTime="25.350" pctInclResp="35.387"> - <Group type="Area" name="OBJMGR" pctCount="0.316" pctSelfTime="23.219" pctInclResp="25.099"> - <Statistics> + <Self count="4871"> + <Inclusive count="665"> </Statistics> </Group> - <Group type="Area" name="SWEPAGE" pctCount="0.028" pctSelfTime="11.079" pctInclResp="7.157"> - <Statistics> - <Self count="430"> <TimeFrom>2010-01-27 07:18:08</TimeFrom> <TimeTo>2010-02-28 05:33:00</TimeTo> <SearchableTimeFrom>2010-01-27 07:18:ll</SearchableTimeFrom> - <ResponseTime units="ms"> <Average>937.010</Average> <Total>402914.472</Total> <Maximum>24627.637</Maximum> <Minimum>0.005</Minimum> </ResponseTime>

The benefit of the XML format is that SARM data can be made available to third-party analysis software more easily.

Automating SARM Data Retrieval

In order to provide the Siebel CRM project team with fresh and easily readable performance data, we must take the following into consideration:

• Ensure that .sarm files are collected and archived according to the monitoring policies of our company. We can use shell scripts to copy .sarm files to safe locations so they do not get overwritten by Siebel server processes.

• Create shell scripts, which execute the sarmquery commands that produce the most usable output for our project. We can use the SARM macro language to produce more flexible script files.

• Use a scheduling facility such as the Windows task scheduler to invoke the scripts at a regular basis.

• Provide SARM output files to third-party analysis systems for better visualization of the data.