ISSEL Log Companion Enterprise 4.4.0 Release Notes

Updated 2008-07-17

This document describes installation notes, known issues, and miscellaneous engineering notes for the ISSEL Log Companion Enterprise 4.4.0.

For a description of the functionality of ISSEL Log Companion Enterprise 4.4.0, see the ISSEL Log Companion Enterprise User Guide.

This document describes the following:

New in 4.4.0 Enterprise

System Requirements

Installation Procedure

New in 4.4.0 Enterprise

These are the news in this release:

Added Special 7 log file type for custom log file formats.

Fixed crash bug if a log path is no longer avaialble.

For previous changes and fixes please see the Revision History.

System Requirements

Windows 2000 (Workstation or Server), Win XP, Windows 2003 Server
x86 compatible, 256 MB RAM, 5 GB HD.

At present the above are the only supported OSes. No problems are forseen with operations on Windows Vista, though this is not tested.

The ISSEL Log Companion Enterprise has very limited system requirements. When running it will use less then 10 MB of RAM, irrespective of the size of the log files being processed. The Log Companion itself uses less then 4 MB of disk space however the log files it processes can use any amount of disk space so the disk drives should be dimensioned accordingly.

Installation Procedure

The installer backs up any current ISSEL directory (C:\Issel) to the Temp directory before it installs any new files. The backed up directory is called "issel_org". Note: If the Log Companion is ever installed again this directory will be overwritten.

If you are already using an older version of ISSEL Log Companion Enterprise you should un-install it before installing the new Log Companion Enterprise version according to the descriptions below. However, make sure you do NOT remove the C:\Issel directory.

If you are already using ISSEL Log Companion Pro you should un-install it before installing the Log Companion Enterprise according to the descriptions below.

Installation procedure is as follows:

Download the installer application from the ISSEL web site.
Double click the installer file to run the installation program.
The installer consists of three parts:
- A self-extracting executable. When it opens select Setup. This extracts all the files from the archive to the temp directory and runs a batch file.
- The batch file creates the directories, backups the ISSEL directory and copies the files to the correct places and then starts the installer for the VB program.
- The VB installer shows a splash screen. Select Setup. This installs the VB program and its associated files.

When installing Log Companion Enterprise over a previous version of the same program all your settings and configurations should be kept intact. The entries for the different configurations should still be pointing to any customised exclusion- or inclusion-files you were using previously.
Note:All exclusion files in the C:\Issel\Exclusions and C:\Issel\Inclusions will be overwritten with the latest versions during installation. Therefore, always keep any custom exclusion files in other locations.

Every new version of Log Companion comes with the latest Robot Exclusions file from ABC. After installing an upgrade you should change your Log Companion configuration settings to point to the latest version of this exclusion file which can be found under C:\Issel\Exclusions\. However, any previous robot exclusion file are also kept intact by the installer and can continue to be used.

When a release adds support for a new log file type you should make sure the right log file type is selected in your old configurations. You may need to open every configuration and re-select the Log File Type setting.

Release 3.2.0 introduced wild cards for IP-exclusions. If you are upgrading from an earlier version of either Log Companion Enterprise or Log Companion Pro you have to edit all your IP-exclusion/inclusion files. Entries which previously did NOT have any trailing wild cards must have the trailing wild cards ADDED to represent the same exclusion/inclusion as before. Without the wild cards they will be taken to mean exact matches. Also note that 3.2.0 offers new combinations for wild cards and IP-ranges, examples of which can be seen in the included default exclusion/inclusion files.

The installation is now complete and the programme can now be started from the Start menu under Programs/Issel. When you start up the new release of the Log Companion Enterprise for the first time make sure you examine any new features and select the settings you prefer. Typically any new features will be disabled by default so they should not affect processing unless you want them to.

If you have previously been using the Log Companion Pro it is possible to re-use any configs you may want to keep from that system as follows:

You should add your old configs with this method before you add or delete any other configs to the Log Companion Enterprise.
Create a new config in the Log Companion Enterprise.
Copy your old config into the folder C:\Issel\Config\X\ where X is the order of the configs you create starting with 1. Do not make any changes to the config file in the folder C:\Issel\Config\0\ which is the system default config.
Delete the previous config file which was in the folder and rename you own config to the same name i.e. filter_config.txt
In the Log Companion Enterprise you can now select the config you created earlier and click the Edit button. The Config screen should open with the data for your old configuration.

Uninstall Procedure

To remove the ISSEL Log Companion Enterprise from your machine follow these steps:

Open the "Add/Remove" control panel, select the ISSEL Log Companion Enterprise and remove it.

Do not delete the ISSEL directory if you are doing an upgrade from an older version of Log Companion Enterprise. If you are upgrading from Log Companion Pro to Log Companion Enterprise you should save any config files or exclusion files you want to keep and then delete the Issel directory.

Otherwise delete the directory C:\Issel and all its content.

Known Issues

On some systems the installer (the CMD window) halts asking for permission to overwrite certain installation files. If this happens you can select ALL to overwite all the necessary files. Only files in the Issel directory are over written. This issue is probably based on the security settings for the account used to install the software.

On some systems the installer will halt with a message about a problem installing an ABCEWizard.dll. This message can be safely ignored.

After the first installation of the Log Companion, the first Processing of a log file may fail due to internal issues with Windows short file names. In that case open and close the configuration and then execute the processing again.

If you are not sure what type of log files you have, select the Auto Detect setting which will detect the log file format for you. However, if the log file is a totally unknown format the Log Companion may still treat it as an NCSA log so please check that the output seems valid.

There is a small problem with handling extra Header fields in IIS log files such as those that shows up when the web server is restarted or the logging configuration is changed. When the Log Companion runs into such a field in the middle of a log file (instead of in the beginning) it will simply exclude such lines. As long as the actual field definitions does not change (for example when a web server is re-started) there are no further problems. If the definition is changed then the rest of the output will probably be garbage since the fields will be parsed according to the original header in the file. To avoid this the log file has to be split where the definition occurs and the different sections of the log be processed separately.
Note: From version 3.2.0 the Log Companion can handle IIS5 logs which does not start with a proper Web Header Header if the Log Companion is set to auto detect the log file format (as long as there is a Web Header available somewhere in the log) but will still not detect extra Web Headers after the first has been detected.

There is an issue with the function converting cookies to parameters. Entered Cookie names which matches the beginning of another cookie name will be parsed if they appear before the intended cookie in the cookie field in the log file. This does not affect the function to keep a single persistent cookie.

The Group function is new in v4.0 and has occasionally caused crashes in testing. We have tried to find all the related bugs but please report any other you may find. This function has been disabled in 4.1.0, no new Groups can be created.

Licensing

To purchase the ISSEL Log Companion Enterprise contact ISSEL at sales@issel.co.uk sending the Log Companion Enterprise ID which can be found at the top of the screen after you have started the program. On receipt of your payment you will be sent a License Key. This key can be entered into the Log Companion Enterprise by clicking the Enter button directly below where the ID is found. On entering a correct license key a confirmation message will appear and the text at the top of the screen will now show that it is a licensed version.

More Help

This is complex software and as such there may be issues that you will encounter. Please report these to support@issel.co.uk for us to investigate.

We also welcome suggestions for improvements and new features. These should also be sent to support@issel.co.uk.

For our contact information please go to: www.issel.co.uk

Revision History

Release 4.3.0
Date: 2007-11-26

Config Replace function now also available from the Pop-up menu.
Now can exclude Highwire log lines which contain "Error404", by treating those messages like Status codes.
Added fully capable filter pattern matching. This can be used for URL's, IP's, Query Strings and Domains.
Added pattern Exceptions, such that a filter can have an exception defined, separated by a pipe-char. This can be used for URL's, Query Strings and Domains.
ABCE robot list Exceptions are now honoured. Currently One exception is accepted.

Release 4.2.1
Date: 2007-07-02

Added support Highwire Service Log format.
Updated ABCE based IP-exclusion list.
Updated ABCE based Status code Inclusion list to No longer include 206.

Release 4.2.0
Date: 2007-03-12

Added support for new ABCE robot exclusion list format.
Added function for removing Other robots i.e. Not on the ABCE list (so the ABCE list does not have to be edited).
Added function for Global Replace. One setting can be changed for multiple Configs per Replace operation.
Added pop-up menu for viewing Event Log, available once the config has been accessed once.
Fixed bug where adding a non-existing path as Input field could cause crasch.

Release 4.1.2
Date: 2006-03-16

Added function for "Find Again (F3)" so you don't have to re-open the Find for each search.
Can now parse User fields in NCSA logs either with or without surrounding double quotes. Useful for Domino logs.
Domain filtering disabled by default, to work better with upgraded systems where this function did not exist.
Fixed bug which disabled the Find function after config execution.

Release 4.1.0
Date: 2005-12-05

Added function to Exclude/Include web sites (domains) based on either Server or Hosts field.
Added status bar which shows which config is being processed at a particular time.
Directory paths in the Event log and Activity window now uses long names for increased readability.
Disabled grouping function as it is not production quality at this time.

Release 4.0.1
Date: 2005-09-09

Modified the Special 5 log file type for custom log file format.

Release 4.0.0
Date: 2005-08-23

Cleaned up user interface. Now with menus!
Added context sensitive pop-up menu.
Added Group functionality.
Added Exclude/Include option for Robots.
Added Disable option for all exclusion lists.
Added function to create batch file for running all enabled configs.
The grid now scrolls when scroll bars moved.
If the LogEngine crashes it now more likely that there will be traces in the event log.
Added Special 5 and Special 6 log file types for custom log file formats.
Max line length increased from 2400 to 4000 characters.
Fixed bug so that the config naming duplicate check is now case insensitive.
Fixed a bug handling a system file which could be empty after another problem.
Added message for handling non-existent short names caused by the Mac.
Made the loading of the config file more resistant to misconfigured config lines.

Release 3.8.0
Date: 2004-06-14

Added function to merge hourly IIS logs into daily.
Added function to summarise all the logs processed in the Event Log.
Increased max line length from 1900 to 2400 characters.
Modified the init-directory for exclusion dialogues to the same as the last one.
Added Special 4 log file type for custom log file format.
Fixed bug where a modified header would also get written to the bad output.
Fixed bug where modified header would Not get written to split logs.

Release 3.7.0
Date: 2004-05-17

Added function to delete old processed log files.
Added function to delete old archived log files.
Increased max line length from 1900 to 2400 characters.
Modified the Find function to also search against the sitename insert field.
Added Special 3 log file type for custom log file format.
Fixed bug with Find which would not scroll the view when a match was found very early in the list.
Fixed NCSA log cookie parsing bug which could skip the first character in the cookie.

Release 3.6.1
Date: 2004-03-24

Fixed bug where the Stop button could show up in the wrong place.

Release 3.6.0
Date: 2004-03-09

Added function to exclude lines without query strings.
Added function to create a Batch File for the selected config.
Now does not pre-select log file names when browsing for logs.
Removed meaningless Read-Only check-box from all common file dialogues.
The Input and Temp directories now gets created automatically if they do not exist.
Fixed bug which could sometimes stop the next execution after a stop of a previous execution.

Release 3.5.2
Date: 2004-02-16

Modified Special 2 log file support.
Fixed bug so that split output logs can also be compressed.
Fixed bug so that ABCE wizard now is not visible in this product.
Fixed bug so that ABCE related directories should not be restored by mistake.

Release 3.5.1
Date: 2004-02-12

Added support for IIS6 log files. Select AutoDetect to handle IIS6 logs.
Added Stop function so that processing can be manually stopped at any time.
Added function to optionally split log files after a user selectable number of lines.
The output directory now gets created automatically if it does not exist.

Release 3.4.1
Date: 2004-01-16

Added option to include only traffic from a particular month and/or year.
Added Special 2 log file type for custom log file format.
Removed separate 'contype' exclusion as this is nowadays included in the robot list.
Updated the default list of search parameters.
Fixed bug so that empty Robot exclusion file lines does not automatically result in exclusion.
Fixed bug so that no special log file warning is given when a config using a special log type is opened.
Fixed bug so that Log File list cannot be edited by user.
Fixed bug so that server insertion field is properly enabled and disabled.
Fixed bug so that Exclude/Include list for URL and Query cannot be edited by user.

Release 3.3.0
Date: 2003-12-11

Added function to split NCSA logs into days if they cover longer periods.
Added the ability to select multiple configurations.
The following functions can now operate on multiple selected configurations: enable, disable, copy, delete, move, run.
Added function to Find configs. Searches are performed on both config names and the log file field.
Added more informative status messages during processing.
Modified the init-directory for most dialogues to be the same as the last one.
Modified the internal timer to go faster between internal states.
Fixed bug with processing extreme lines used by hackers to break web server security.
Fixed bug so that the up and down buttons now keeps the whole row selected.

Release 3.2.0
Date: 2003-10-27

Added option to Auto-detect log file formats.
Added option to perform Host Copy for NCSA logs.
Added double click function to the list view
Added keyboard shortcuts to Browse, Edit and View buttons.
The config list view now remember the last selected config and scrolling position.
The config list view now adjusts the scroll position when rows are added, copied or moved around.
Added ability to use wild cards with IP exclusions.
This also means that exact matches can now be performed.
Added ability to match IP addresses using ranges.
Added ability to parse NCSA log files with Host Header field.
Added ability to parse NCSA log files with extra redundant fields at the end of the lines.
Added ability to detect and parse Web Headers in IIS5 logs even if they are not at the top of the file.
Modified Config list to also display config ID.
Fixed crashing bug in processing of ZIP files which are too small (i.e. empty or corrupt).
Fixed bug with leading URL wild card exclusions which could be fooled by matches inside a string.
Fixed bug with leading Query String wild card exclusions which could be fooled by matches inside a string.
Fixed bug so that Port label is displayed properly.
Fixed bug so that tabbing between fields is now in correct order.
Fixed bug so that server text field now is enabled and disable according to Host Copy setting.
Fixed bug so that the default log directory is shown in Enterprise list if the log directory is reset.
Fixed bug so that clicking on a row in the list view always selects the whole row.
Fixed bug so that range selection is now disabled since this is not supported.

Release 3.1.5
Date: 2003-06-19

Fixed bug when adding S-IP field to IIS5 logs so that any extra space at the end of the line is properly handled.
Fixed bug so that Host Copy can now be used even if there is no S-IP field in the log file to start with.
Fixed persistent cookie parsing bug. Now parsing does not stop when plus sign found in the cookie value.
Fixed crashing bug when unzipping a Zip file of zero size.
Fixed bug in the File menu OK command in Edit window which would end the application instead of just the edit window.
Fixed crashing bug when adding a new config and then canceling it directly.
A Batch file (LogBat.bat) is now included in the Issel\Bin directory which can be used to simplify Windows scheduling.

Release 3.1.3
Date: 2003-05-29

Fixed crash bug when trying to archive to directory which was not possible.
Changed "Save Config As" function to use Save dialogue, not Open.

Release 3.1.2
Date: 2003-04-28

Now able to process zipped log files.
Added ability to compress the output log files.
Added ability to compress archived source log files.
Fixed bug so that Event log gets created if it doesn't exist, instead of resetting it to default value.
Removed out-of-date eRoom application function.
Removed Hit List application function since it does not apply to multiple configurations.

Release 3.0.0
Date: 2003-02-03

First release of Log Companion Enterprise.