Lets explore few basic troubleshooting steps required during performance issues related to CheckIn / CheckOut process using Siebel Tools.

1. Check if this problem is happening on all machines or single machine. Also check whether the all developers are affected or single developer. This step helps in isolating the root cause of the performance problem.

2. Check if performance issue occurs with all projects or just a specific project. If the performance issue occurs when user checkouts a table project, such as, Table Organization, Table Person, etc then the reason for slow response times is related to the check-out process with tables that contains a long column defined as a data type.

3. Sometimes slow performance during Checkin – CheckOut is due to Network. Therefore, it is necessary to involve network administrator in order to analyze how the network is utilized during check-out and/or check-in.

4. Depending on the size of the project, the check-in / check-out process might require some time. Do not interrupt the process, as doing so can leave your repository in an unstable state. If for any reason the check-in process is interrupted, perform it again to complete any unfinished tasks and unlock the projects on the server.

5. Check if server is running out of resources. It is a good idea to restart the server and see if that makes any difference in performance.

6. Check ODBC settings: Faulty ODBC settings may cause performance issues. Go to "Configure" on the ODBC settings (Navigate to Start >Programs >Administrative Tools >Data Sources (ODBC)) then test the connection.

For Oracle Database, verify that following ODBC setting in Siebel Tools and Siebel Server is done as per the information documented in the bookshelf

PacketSize = 0
EnableScrollableCursorsEnableScrollableCursors = 0
ColumnsAsChar = 1
ColumnSizeAsCharacterColumnSizeAsCharacter = 1

For details refer Siebel Installation Guide for Microsoft Windows: Servers, Mobile Web Clients, Tools > Installing Siebel Tools > Post installation Tasks for Siebel Tools > Verify Siebel Tools ODBC Data Sources.

7. Check if ODBC tracing has been turned on. If yes, turn off the ODBC tracing. This can be done via ODBC Data Source Administrator > Tracing tab. Re-start Siebel Tools after turning off ODBC tracing and see if there is any performance improvement.

8. Check to make sure that Siebel Server and Siebel Tools are on the same patch level.

9. 3rd Party Software:
Sometimes running Anti-virus software can cause issues with Checkin / Checkout.

10. If check-out/check-in operation is over the WAN then this could cause slow performance. There is no direct solution that can be implemented to increase the speed. Workaround in these cases would be:
-Remove any unused repository from the server database.
-Any project the developers in offshore center need to perform CheckOut/CheckIn constantly, break them up into smaller projects (i.e. by moving the object definitions into separate projects).

11. The number of redundant Siebel repositories in the database, can significantly slow down the performance. Any redundant repositories should be archived off and then deleted using Siebel Tools.  use Siebel Tools to review the 'Repositories' node in the Object Explorer, or run the following SQL:

select count(*) from S_REPOSITORY;

Note: Deleting repositories can take a long time to perform and is very CPU and database intensive. Deleting Siebel repositories directly via SQL is not supported. The only supported method is via Siebel Tools.

12. Missing or corrupt indexes in the database: Verify with the DBA that there is no missing or corrupt index in the database. When the database is imported, it can happen that the repository indexes may not have been created. This can be shown in the install logs, and will be reflected in poor performance for all parts of the application using the same tables. Similarly if any database index is missing or corrupted for any reason, the same will apply. If this is the case then  recreate the indexes.

13. It is extremely important to verify that the environment meets the recommended system requirements. Validate all the components of the Enterprise and confirm they meet all the requirements as documented in: Systems Requirements and Supported Platforms Guide (SRSP). Environment Verification Tool (EVT) is available to assist with this step.  
 

The only supported way in which you can delete a repository is by way of deleting the appropriate ‘Repository’ record within the Siebel Tools application. By doing this it will perform a cascade delete on all the child objects associated to the Repository object.

To delete a Siebel Repository:

1. Click the Repository object type in the Object Explorer.
2. In the Object List Editor, click anywhere in the row for the repository you want to delete. Set the Inactive to True. Save it.
3. Choose Edit > Delete Record.
4. Click outside the record to commit the Delete action.

However, sometimes Database or Oracle errors when trying to delete repositories from Siebel tools. Deleting a repository takes a long time and requires system resources such as rollback segment, cursors, tablespace, and so on.

Following are few common database error:

• ORA-1650: unable to extend rollback segment
• ORA-01000: maximum open cursors exceeded
• An error has occurred while deleting the record. Please try to delete again.
• An error has occurred writing to a record. Please continue or ask your systems administrator to check your application configuration if the problem persists

we are exploring the errors specific to Oracle database (not SQLServer, DB2, AIX etc)

To troubleshoot these type of errors:

1. Make sure you run the deletion when there are less or no users in the system.

2. "OPEN_CURSORS" is set in the initialization file. Increase the value of 'OPEN_CURSORS' in the oracle initialization file as per Siebel Installation Guide for Microsoft Windows: Servers, Mobile Web Clients, Tools > Guidelines for Configuring the RDBMS > Configuring Oracle for Siebel Applications. If you set this value lower, Oracle may return an error to the application that prevents you from continuing (because of error > ORA-01000: maximum open cursors exceeded)

3. Make sure you create a large rollback segment or at least to set 'AUTOEXTEND' to 'ON'. The DBA might well be the best person to predict the size of the rollback segment required.

Siebel Tools does not make use of any specific rollback segment. It cannot be explictly specified. It is recommended to switch off smaller rollback segments for duration of the task if they are not properly sized. The most common error occurs when the extent sizes are too small in the rollback segment and it cannot be extended any further.

4. Check the four critical ODBC settings which need to be set to the following values:

PacketSize = 0
EnableScrollableCursors = 0
ColumnsAsChar = 1
ColumnSizeAsCharacter = 1

5. ORA-04030: out of process memory when trying to allocate string bytes (string,string). This error is caused when Operating system process private memory has been exhausted. Please check with your DBA or operating system administrator to increase process memory quota. There may be a bug in the application that causes excessive allocations of process memory space.

Limitations :

• Haley cannot insert a record.
• Haley cannot delete a record.
• Haley cannot replace browser script that interacts with the user or desktop.
• Haley cannot interact with the user to get information or answer questions.
• There is no way to make dynamic search specifications for business components. (Haley would have to be part of workflow.)
• Haley cannot interrogate the application for things like LoginName, PositionId, PositionName etc. (Haley would have to be part of workflow.)
• One cannot set values or get values from an entity that is not related to the current record. So, for example, if I want to update a record in a business component that is not related to the current business component, I cannot do it. (Haley would have to be part of workflow.)
• Haley cannot disallow duplicate child records. In order to “see” all the records for a particular parent, a runtime event would have to be configured on the parent record. It is only then that Haley can iterate over the records. It does iterate by the way. This fires the BusComp_ChangeRecord event for each record in the child business component.

What it can do:

• Get/set profile attributes
• Get/set field values
• Get the active view name
• Can be invoked from script/workflow/runtime event

The .srf file is a compressed binary database. Siebel eBusiness Applications use compression techniques to optimize  the size and performance of the .srf file at compilation time. Each time a repository object gets written or changed, there is a change to internal block allocation, causing the file size to change.

A partial compile will optimize less than a full compile, because the optimization is confined to the objects being written at that time, without looking at their relation to the repository as a whole. Therefore, it is fairly common to see an increase in .srf file size after a partial compile, even if no objects or properties have been added.

A full compile may also result in slightly different file sizes, as with this example of a comparison between a .srf file compiled from identical repositories in development, test and UAT:

• SIEBEL_DEV.SRF  ---- 10,453 KB
• SIEBEL_TEST.SRF ---- 10,496 KB
• SIEBEL_UAT.SRF  ---- 10,489 KB

The difference in size is due entirely to the optimization functionality, and not to the environment or the machine. Successive full compiles on the same machine may also show file size changes.

If there is a significant change in file size, for example, if a 10 MB file becomes 5 MB or 15 MB, investigate the following:

1.Ensure that prior to compiling, the administrator has opened the correct repository file. To verify the repository name in Siebel Tools, choose File > Open Repository. The highlighted value is the repository whose objects are currently being displayed in the Tools application and the one which will be compiled.

2.Ensure that the database sort order is set up to use binary sort order. Databases used to hold a repository must use binary sort order. Please refer to the Siebel Server Installation Guide for the appropriate operating system for further information. In the Creating the Database section for the appropriate RDBMS, refer to the section Creating the Database > Creating and Setting the Language Characteristics of Your Database.

SRFDiff has been introduced in Siebel 8.1 and "Using Siebel Tools Version 8.1" > Chapter 12: Managing Repositories > Comparing Two Repository Files Using SRFDiff explains how to use it. There it mentions about "major schema version" and "minor schema version".

"major schema version" comes from S_APP_VER.DB_SCHEMA_VER column and "minor schema version" comes from S_APP_VER.DB_MINOR_VER column. They are inserted while Siebel Database is set up

The purpose of Siebel Audit Trail is to tracks changes to business component data made by users in the application, changes which are made through the Object Manager. Therefore changes made directly to the data in the database would not be tracked by Siebel Audit Trail.

Siebel Audit Trail works through the business component layer. In the set up of Audit Trail, business components are specified to be audited operations to be tracked, and fields to be tracked.

The following operations can be tracked by Siebel Audit Trail:

1.New Record
2.Modify/Update Field
3.Copy Record
4.Delete Record
5.Associate Record
6.Read Record
7.Export Record (supported only in version 8 and above)

A full SRF compile usually completes in under an hour for the standard Horizontal and Industry repositories. Many times developers report that a full SRF compile is very slow and is taking a significantly longer time. The following suggestions can help narrow down what is causing the slow performance when compiling a new SRF file.

1. Set up Dedicated Web Client logging for Siebel Tools.

The Siebel_Log_Events environment variable will create a SIEBDEV.LOG file in the Siebel tools log directory. This log file documents the sql used when Tools is compiling a new SRF. The file includes the sql statements, their timings and timestamps. Review the siebdev.log file for slow sql timings and large timestamp gaps. Please review this metalink document for more information on Dedicated Web Client logging.

How Should Client Side Logging Be Set?

How to enable server side logging?

. Narrow down the scenario where the SRF compilation has slow performance.

Possible causes for the slow performance could be the network, the server database, or the specific repository. It is helpful to know under what conditions the SRF compile is slow and where it is fast.

Is the slow compile for all developers or just some?

Is the SRF compile slow when connected to the local or the server database?

Does the slow compile performance happen for all of the server databases or just dev, test or production?

This is to see if the slow compile is specific to one database. In this case, comparing the SIEBDEV.LOG files from a fast and slow compile can help confirm what is slow.  If the slow srf file is specific to a database, have the statistics been updated recently?  If the SRF compile is slow for the server or the local database, verify that the statistics have been collected recently.

Where is the server database and where are the local developers?
If the slow performance is for local developers on a specific machine, verify the network performance is good.

Is there antivirus software running when the SRF is being compiled?   Is the  SRF files or the Siebel Tools install directory excluded from the anti-virus checking?  Antivirus software running in the Siebel Tools installation directory is the most common cause of slow SRF compiles.  Try compiling the srf to a network drive that is known to be excluded from the antivirus processing. 

If compiling a specific repository is slow, verify if the repository was recently moved to that database or if there have been large changes to that repository recently.

3. Does the database have many repositories in the local or server database?

A large number of repositories in the Siebel tables can slow down local and server SRF compiles. Test compiling in a local or server database with a few repositories in the Siebel tables. For example, try compiling a new SRF from the production database, which should have one production repository in the Siebel tables.

4. The following documents discuss product defects that can cause slow SRF compile times.

There are some unexpected performance behaviors when compiling with Siebel Tools 8.0 with the ST eScript Engine enabled.

In general, when encountering a slow SRF compile time, first review the SIEBDEV.log file.  Then verify under what conditions the SRF compile is slow.  If the full compile takes longer than an hour, then narrow down the scenario where the full compile is slow. 

The following is an example of a 7.7 siebdev.log file (c:\sea\tools\log\siebdev.log file) where the slow compile was caused by anti virus software scanning the Siebel Tools directories.

Review the log file to find the time stamp gaps.  This is a log from a single applet object compile that took nine minutes.

Siebdev.log

...

ObjMgrBusCompLog Delete 4 0 2011-03-22 15:27:19 BusComp "Repository Application Toolbar" at 179a37b8 was deleted

ObjMgrBusCompLog Delete 4 0 2011-03-22 15:27:19 BusComp "Repository Page Tab" at 17caec30 was deleted

ObjMgrBusCompLog Delete 4 0 2011-03-22 15:27:19 BusComp "Repository Toolbar" at 17c0d868 was deleted

ObjMgrBusCompLog Delete 4 0 2011-03-22 15:27:19 BusComp "Repository Toolbar Item" at 17960868 was deleted

ObjMgrBusCompLog Delete 4 0 2011-03-22 15:27:19 BusComp "Repository System Activity Object" at 17c93930 was deleted

***ObjMgrBusCompLog Delete 4 0 2011-03-22 15:33:44 BusComp "Repository Index" at f1f2d18 was deleted

ObjMgrBusCompLog Delete 4 0 2011-03-22 15:33:44 BusComp "Repository Index Column" at f0c12c0 was deleted

ObjMgrBusCompLog Delete 4 0 2011-03-22 15:33:44 BusComp "Repository Column" at f13a190 was deleted

FDRLog FDRDetail 4 0 2011-03-22 15:33:44 Total # Buffer Wraps = 8

FDRLog FDRDetail 4 0 2011-03-22 15:33:44 Last Wrap Duration = 391 secs

FDRLog FDRDetail 4 0 2011-03-22 15:33:44 Next Absolute Record ID = 16128
...
0 2011-03-22 15:33:44 Business Service 'Siebel Tools Object Compiler' invoke method 'Start' Execute Time: 395.755 seconds.

ObjMgrBusServiceLog InvokeMethod 4 0 2011-03-22 15:33:44 End: Business Service 'Siebel Tools Object Compiler' invoke method: 'Start' at c1fd9b8

ObjMgrBusServiceLog Delete 4 0 2011-03-22 15:33:44 Business Service 'Siebel Tools Object Compiler' was deleted at c1fd9b8

***ObjMgrBusServiceLog Delete 4 0 2011-03-22 15:36:29 Business Service 'Siebel Tools Server Debugger UI' was deleted at 1fa37e8

ObjMgrLog Info 3 0 2011-03-22 15:36:29 (modpref.cpp (922)) SBL-DAT-50808: SharedFileWriter: C:\sea77\Tools\bin\JO_CMS&Siebel Tools.spf_IT10120013X_2176_3840: Unlink succesfull.