DP4 Unix/Linux Installation Guide

Contents

  • Introduction
  • Unix/Linux Versions Supported
  • DP4 Packages Available
  • Installation
  • Operational Considerations
    • Key Assignment
    • Backup_and_Restore
    • The kill command
    • Server Shutdown
    • Processes and Inter-Process Communication
    • Printing from DP4
    • Program Development
    • Copying Files To and From Unix

Introduction

This guide tells you how to install and configure DP4 for Unix or Linux, and other similar operating systems such as AIX. It also covers operational considerations particular to the Unix environment, such as program development with QA Build and DP4 C, working with Unix printers and copying files to and from Unix. In general this document's use of the name Unix should be taken to include all of the supported Unix-like operating systems. Similarly the name DOS is used to refer both to MS-DOS, and all (Intel based) varieties of Microsoft Windows including Windows NT)

Please note that if you are upgrading from release 4.523 to release 4.525, any C applications you have built yourself need to be recompiled and relinked with the new DP4 C Libraries. 4.523 DP4 binaries will not run against a 4.525 DP4 for Linux installation, because of fundamental changes to various parts of the DP4 architecture. There are no source code incompatibilities.

If you are used to using DP4 for MS-DOS or Windows, you will not have any problems running on Unix or Linux. However if you are not used to Unix or Linux at all the underlying operating system may be rather daunting. You may like to refer to our Novice's Guide to Unix and Linux.

Unix/Linux Versions Supported

DP4 is available on a wide range of Unix and Linux platforms.

  • DP4 Unix programs will run on any Intel 386+ based Unix that supports the Intel Binary Compatibility Standard. However, network programs have to be built specially for each version of Unix due to portability problems. Currently DP4 is tested on Unixware 7.1. We also have a set of Linux executables that have been tested on SuSe and Red Hat Linux. These run faster and more reliably than the Unix executables, which require the Linux ibcs package to be installed.
  • A restricted set of DP4 utilities (run time only or server only) is available for a range of Non Intel machines including the RS6000, HP9000, and Sun Solaris. In general, we are only able to support these where clients can give us access to suitable machines to build the required components. Contact us if you would like a version of DP4 to run on some other machine.

DP4 Packages Available

On Intel processors, the complete DP4 toolset is available as below:

  • ADCUNX. Source code components for building ADCs
  • CCLIN. DP4 C libraries and objects for Intel Linux
  • CCSCO. DP4 C libraries and objects for SCO Unix
  • CCUNX. DP4 header files and C library source
  • CCUW7. DP4 C libraries and objects for Unixware 7
  • DIPUNX. DP4 OSDI for Unix/Linux
  • LINLFS. Replacement DP4 files built on United Linux. These programs may not run on all Linux distributions. They are needed if you want to create/access database files larger than 2 Gigabytes. They are a partial replacement for the LINXSUSE files and must be installed immediately after those files if required.
  • LINXSUSE. Base DP4 programs built on SuSe Linux. These programs should run on any version of Linux with a sufficiently recent kernel.
  • NTBUNX. DP4 NetBIOS Networking programs for NCR3000 Unix.
  • SQLLIN. DP4 SQL programs for Linux.
  • SQLUNX. DP4 SQL programs for Intel based Unix build on SCO.
  • SQLUW7. DP4 SQL programs for Unixware 7 (optimised for Pentium Pro).
  • TCPATT. DP4 TCP/IP networking programs for NCR3000 Unix.
  • TCPLIN. DP4 TCP/IP networking programs for Linux.
  • TCPSCO. DP4 TCP/IP networking programs for SCO Unix and SCO Openserver 5.
  • TCPUW. DP4 TCP/IP networking programs for Unixware 2.
  • TCPUW7. DP4 TCP/IP networking programs for Unixware 7.
  • UNXSCO. Base DP4 programs (including the DBMS, QA Build and Report Writer) for Intel Unix built on SCO Open Server 5.
  • UNXUW. Base DP4 Programs built on Unixware 2. This is not a complete DP4 set. It may be installed after the UNXSCO set if desired.
  • UNXUW7. Base DP4 programs built on Unixware 7.1 and optimised for Pentium Pro Processor. Most of these programs also run on other versions of Intel Unix, but srv3,trm3,reorgdb and dbcheck require support for a 64 bit file system, so only run on Unixware 7.1, (and not Unixware 7).

As of release 4.525 support for Unixware 2 and NCR3000 variants is dropped, and SQL support is included in base DP4 so there are longer any SQL packages. Open Server 5 software will be supported on current ibcs 2 compatible flavours of Unix, but not on obsolete versions of Unix. As far as we know none of our current Intel variants run on Intel Solaris platforms.

On non-Intel processors, DP4 'MixNet' server implementations are available. These provide the database server components, networking interfaces and selected utility programs only (usually including QAB, Report Writer and Importer). Other DP4 utilities can be accessed from workstations using the network facilities.

We discourage you from writing your own DP4 C programs to run on non-Intel servers as DP4 programs must adhere to strict standards to work correctly in such environments. Nevertheless the C programming libraries are available if required on such platforms. ADCs are not officially supported on non Intel platforms.

  • UNXAIX. Run time only base DP4 set + TCP/IP networking programs for the IBM RS6000.
  • UNXHP. Run time only base DP4 set + TCP/IP networking programs for the HP9000.

Versions of DP4 exist for the Sun Solaris, DG Aviion and various other machines.

All the DP4 Unix packages are supplied on the DP4 CD ROM.

Installation

Up to and includeing release 4.523 DP4 must be installed in /usr/datafit/ and associated sub-directories. After installing, it is possible to move most files elsewhere and use the DP4 system setup utility, DFSETUP, to match the new directory structure. From release 4.525 there are no restrictions on where you install DP4. We suggest that you use a suitable subdirectory of the opt directory, for example /opt/dp4.

DP4 Unix packages are supplied as tar archives. The tar command is used to read the DP4 files from the install media.

Installation from CD ROM

On the DP4 CD ROM individual Unix/Linux packages are supplied as tar files with the same name as the package in the Unix sub directory for the version of DP4 being installed. For example the 4.520 programs are contained in \520\unix. Any Unix machine should be able to read the DP4 CD ROM directly (if you can work out how to mount the CD ROM!*). However, you may well find it easier to transfer the files to the Unix machine via the ftp utility, (assuming the target machine is configured as an ftp server). If you do this remember to set the transfer mode to binary with the bin command before transferring files. If you are using the standard Windows version of ftp it is a good idea to specify the filename twice when using the put command like this: put linxsuse.tar linxsuse.tar
If you do not do this the file will probably be transferred as UNXSCO.TAR.

*Typical commands for mounting the CD ROM would be mount −o ro /dev/cd0 /opt/dp4/cdrom (on Unix)
mount −o ro /dev/cdrom /opt/dp4/cdrom (on Linux)
You will probably need to login as root in order to do this, and you will need to use mkdir to create the /opt/dp4/cdrom directory if it does not already exist.

Installation Summary

It is essential that you perform the installation procedure in the correct order, as outlined here:

  1. For an upgrade installation preserve information from your existing system database.
  2. Log on as root or a similar user with administrative privileges when performing the installation.
  3. Prior to a first time installation add the "datafit" user (This is only necessary when installing 4.523 or an earlier release).
  4. For a first time installation install the DP4 licence file.
  5. Install the base system by restoring the file holding the appropriate base system installation.
  6. Install Linux large file support by restoring the linlfs.tar file holding the replacement DP4 binaries.
  7. Run the DP4 installation program, dp4install
  8. For an upgrade installation restore the information saved from the previous Installation
  9. Install any additional DP4 components required, and run the dp4install program after installing each one.
  10. Make any other necessary adjustments to your Unix/Linux installation.

  1. Preserve Information from current system database

    1. You must export the User list from the existing system database. This is the list of users and their passwords as set up using MENUUSER.

      Run menuedit -db system
      • Select Users table
      • Select Print user list

      • Print the user list to a file, so that you can re-load the information when the new system is loaded

    2. You must export the Device setup from the existing system database. This is the set of printers and devices (and logical devices) set up using DFSETUP. Run dfsetup
      • Select Printer characteristics
      • Select Export device information

      • Print the device information to a file

  2. Use a suitable login for the installation

    It is advisable to log in as root prior to installing DP4. If you do not do this it is possible that certain files may not be installed, or that permissions will not be set correctly. Make sure no DP4 programs are running before upgrading an existing installation.

  3. Add the Datafit User

    This step is only required when installing 4.523 or an earlier DP4 release, and should be skipped if you are installing 4.525 or a later release.

    Using system administration procedures, add a new user called "datafit", whose home directory is /usr/datafit/ . You do not need to give this user special privileges. A typical procedure would be as follows:

    1. Ensure that you are logged in as "root"

    2. To create an account for a new user, type:

      useradd −m −d /usr/datafit/ datafit

    This creates a login account for a user logging in as "datafit". The −m and −d command tails define and create a default home directory called " /usr/datafit/ " for this user.

    On some versions of Unix the default home directory for a user called datafit may not be /usr/datafit. It may be /home/datafit or something similar. It is not absolutely necessary to install DP4 in the /usr/datafit directory. However the DP4 licence file for release 4.523,datafit.sys, file MUST be acceible using the name /usr/datafit/datafit.sys. You can use the ln command to make it appear that DP4 is installed in /usr/datafit (e.g. ln −s /opt/dp4 /usr/datafit ).

  4. Installation of DP4 licence disk

    If you are installing DP4 for the first time on a machine you need to install a DP4 licence file (dp4.sys from release 4.525, datafit.sys in earlier releases). For subsequent installations you can continue to use your existing licence file. The licence file must be present for DP4 to function correctly. If you are planning to install DP4 in an identical configuration on many machines you may prepare a modified licence file that has been suitably preconfigured, which can be installed by any means you choose provided the licence file is created in the correct location with appropriate permissions.

    You may have been supplied with a Unix format diskette containing your DP4 licence. Such diskettes were supplied at a time when the DP4 licence file had to be installed as /usr/datafit/datafit.sys. Therefore the following instructions assume you are installing DP4 in /usr/datafit. If you are installing release 4.525 and don't want to install in /usr/datafit we suggest you follow these instructions, and then use mv to move the licence to the correct location.

    If your licence diskette is readable on a DOS or Windows PC, then you do not have a Unix licence diskette, and you should use FTP to transfer the DP4 licence file to the appropriate location on the Unix machine.

    To install a DP4 licence from a Unix format licence diskette proceed as follows:

    1. Log in as "root" (or another user with similar privileges). Change directory to /usr/datafit/

    2. Insert the diskette that holds the DP4 licence file into the diskette drive and use the appropriate command to extract the tar file containing it, e.g. tar -xv on Unix or tar -xPvf /dev/fd0 on Linux. Files on this disk are restored to /usr/datafit/ . Among these is datafit.sys

      Now DP4 licence files will normally supplied via e-mail, or possibly on an MS-DOS format diskette, and you should use binary mode ftp to transfer the file to the appropriate location on the Unix machine, i.e. /usr/datafit if you are installing 4.523 or an earlier release, or the directory where you wish to install DP4 if you are installing 4.525 (e.g. /opt/dp4).

    You can convert a DP4 licence file for another operating system to one for Unix/Linux by the following method:

    1. Copy the licence file to another directory. e.g. copy c:\dp4\win\dp4.sys c:\dp4
    2. Invoke DFSETUP like this: dfsetup -remote unix -from c:\dp4\
    3. Use the directory assignment option to set all the directory names appropriately for Unix.

    4. Transfer the modified licence file to the appropriate directory on the Unix machine, e.g. using binary mode ftp.

  5. Installing Base DP4 Components

    Once the DP4 licence file is installed (and the datafit user has been created if necessary) you can install the base system, including the DP4 for Unix programs and the system and base dictionary databases.

    If you are installing from the tar files on the DP4 CD ROM and have already transferred the appropriate tar file to your Unix machine extract it like this:

    tar −xvf filename.tar

    If you are installing files downloaded from the DP4 FTP site you will have probably downloaded .gz files. Use gunzip to extract the .tar file from the .gz file.

    You can omit the v option if desired, (v causes the filenames to be echoed to the screen as they are restored). Replace filename.tar with the package name of the appropriate set of DP4 programs for your computer, e.g. unxuw7.tar or linxsuse.tar. If you are installing release 4.523 or earlier on Linux (and possibly some versions of Unix), you need to use this command instead.

    tar −xPvf filename.tar

    The P option prevents the tar utility from removing the leading / from the absolute paths stored in the TAR files. If it is not used tar will restore files to the wrong location.

    If you are installing 4.525 or a later release the .tar files contain only relative path names such as bin dbs, so tar must be run from the DP4HOME directory, and there is no need to specift the -P option.

    In either case the files are installed into the following subdirectories of your base installation directory:

    bin/ Contains the DP4 Unix or Linux programs.

    For release 4.523 or earlier you are recommended not to change the location of the terminal manager (trm3) and database manager (srv3) files if you have a requirement to run programs converted from MS-DOS using EXE2UNIX. Such programs expect these files to reside in /usr/datafit/bin/

    dbs/ Contains the database data and index files (system and base dictionary). N.B. These databases are always installed into these directories. If you decide to keep databases elsewhere you will have to move them manually to the correct locations after the installation. It is particularly important to remember to do this if you upgrade an existing installation to a new version of DP4, otherwise you may end up using an out of date version of the system database.

  6. Installing Large File Support

    If you are installing on Linux and want to install support for large database files, then transfer the linlfs.tar file to your installation directory and extract it, exactly as you did the linxsuse.tar files.

  7. Running dp4install

    Once you have installed the base set and before installing any other DP4 packages you must run the dp4install program by typing:

    ./dp4install

    All DP4 packages come with a dp4install program, which generally just tries to ensure that file permissions are correct. Up to release 4.523 all files are changed to be owned by the datafit user. From release 4.525 the dp4install file does not change the ownership of files, so you may need to run chown and chgrp after the installation has completed to set an appropriate owner for the files - (the tar files on the installation media cannot have the correct owner as they are created on another system).

    For release 4.523 the dp4install provided with the base system also checks you have correctly created the 'datafit' user and, if you have, creates a number of subdirectories, for the different classes of DP4 files:

    • /usr/datafit/bkp/ for database backups
    • /usr/datafit/log/ for database transaction logs
    • /usr/datafit/rlb/ for database rollback files
    • /usr/datafit/spl/ for print work files
    • /usr/datafit/tmp/ for temporary files.
    • /usr/datafit/prn/ for files output by DP4 programs such as Report Writer.
    • Various others (probably including some that are not actually used any more)

    dp4install also sets up your environment, like this:

    • Release 4.523

      • The ownership of all the installation files is changed to "datafit"

      • The /usr/datafit/.profile file is updated to include the path for the DP4 Unix programs directory, and to set the environment variable, DATAFIT, to specify the location of the licence file, datafit.sys . These lines are added:

        DATAFIT=/usr/datafit; export DATAFIT
        PATH=$PATH:/usr/datafit/bin; export PATH

        The DATAFIT environment variable is not currently used by DP4! The licence file datafit.sys must be in the /usr/datafit directory! It includes the name of your organisation and licence number. It also contains the pathnames of the location of the database files, and the location of the SYSTEM database.

      • The system database name is set to "system" by running:

        sysdb /usr/datafit/dbs/system

        It is not an especially good idea to leave a hard coded pathname in the system database setting like this, as one or two DP4 utilities may have problems with this. However it is a useful means of forcing DP4 to run regardless of datafit.sys settings.

      • USERDATA is now run to create the DP4 terminal configuration file, userdata.sys . You must select a terminal type that matches your current terminal from the list of terminals it presents by typing the corresponding letter.

      • DFSETUP is run to configure DP4 for use. Your licence file may have been pre-configured with sensible values so you can use it immediately, but you should check the various directory assignments match your requirements in any case.

    • Release 4.525

      • The permissions of all the installation files is changed to rw-rw-rw or rwxrwxrwx as appropriate

      • A shell script file named .dp4 is created. It modifies PATH, LD_LIBRARY_PATH, and defines DP4HOME for the new installation. If users want to be able to run DP4 programs they should add the line:
        . dp4 install directory/.dp4
        to their .profile login files. For example if DP4 is installed in /opt/dp4 the line should read:
        . /opt/dp4/.dp4
        In this case DP4HOME points to /opt/dp4, and PATH and LD_LINRARY_PATH contain /opt/dp4/bin. Both the DP4 licence file, dp4.sys, and the DP4 configuration file, dp4.ini, should be kept in the $DP4HOME directory. (Purists may like only to store links to these files in the DP4HOME directory, and to place the actual files in a directory such as /etc/opt/dp4)

        The licence file dp4.sys contains the name of your organisation and the licence number. It also contains the pathnames of the location of the database files, and the location of the SYSTEM database.

      • The system database name is set to "system" by running:

        sysdb $DP4HOME/dbs/system

        It is not an especially good idea to leave a hard coded pathname in the system database setting like this, as one or two DP4 utilities may have problems with this. However it is a useful means of forcing DP4 to run regardless of current settings.

      • USERDATA is now run to create the DP4 terminal configuration file, userdata.sys . You must select a terminal type that matches your current terminal from the list of terminals it presents by typing the corresponding letter.

      • DFSETUP is run to configure DP4 for use. Your licence file may have been pre-configured with sensible values so you can use it immediately, but you should check the various directory assignments match your requirements in any case.

    More information on running USERDATA and DFSETUP as part of Unix installation can be found here and on running them generally in the Configuration Utilities section of the Guide to DP4 Configuration.

  8. Restore Information saved from Previous Installation

    1. Re-load the user list

      Specify the filename you used when you saved the user information in place of users.prn:

      mappost -db system users.prn

    2. Re-load the device information. Specify the filename you used when you saved the device information in place of devices.prn:

      mappost -db system devices.prn

      If you are upgrading from quite an old version of DP4 it may be the case that some of the standard print devices have changed and should not be preserved. In this case you need to follow a slightly different procedure.

      • Repeat step 2 above to export devices from the new system database (before running the MAPPOST devices.prn). Use a different filename from before.
      • run mappost -db system devold.prn where devold.prn is the filename you used to save your original devices.
      • run mappost -db system devnew.prn where devnew.prn is the filename you used to save the print devices from the new system database.

      This will ensure that any "standard" devices will be defined as issued by Itim Technology Solutions.

  9. Installing Additional Products

    Having installed the base system, you may install any additional products in the same way, for instance "UNIX C Programming Tools". If you are installing 4.523 components run dp4install after installing each product to ensure the file ownership is set to "datafit" (the dp4install file works by changing the ownership of files listed in the file /usr/datafit/packlist restored with each product you install.) In fact you can skip this step as long as you remember to set ownership and permission on all the files in the installation directory and subdirectories appropriately after installing all the products required.

  10. Adjusting Unix Limits and Permissions

    • ULIMIT/ulimit

      Unix users are assigned various limits, such as a limit for the maximum number of blocks that they can write to a single file, and a limit on the number of files that can be simultaneously open. If, for example, the file size limit is set too low the DP4 database manager could report an error:

      Fail error reported by DP4, code 25
      Error in writing file - check disk space
      Any key to continue

      These limits can usually be set with the ulimit program. The default values may be too low for DP4 to work properly. Please consult your Unix documentation and set appropriate hard and soft limits as follows:

      • CPU time should be unlimited.
      • The maximum files per process should be at least twice the maximum number of simultaneous users of the DP4 system plus four times the number of databases.
      • The maximum file size should be set to the maximum anticipated size of your application database files (assuming these are the largest files).

      DP4 attempts to remove all such limits on itself when it loads, but it may well not be able to do so.

    • UMASK

      In Unix and Linux a variable called umask, helps define the permissions files created by programs are assigned. The permissions specified by a program are masked by the umask value to restrict access to the file. In some versions of Unix the environment variable, UMASK, can be set in the user's .profile to setup the read/write attributes for new files created by that user. In Linux, and most versions of Unix the umask is set by running in the program of the same name. It is advisable to set umask to 0. If umask is set so that one user cannot access a shared file created by another user, the database server may report a system error message:

      Fail error reported by DP4, code 18
      Unable to open file
      Any key to continue

      Where relevant the UMASK value can be set in the /usr/datafit/.profile file, with this syntax:

      UMASK=0

      Or by running the umask program like this:

      umask 0

      Unix will XOR the UMASK setting with, 666, the mode specified by all DP4 programs when they create files. An important implication of this is that you must not rely on the default setting of UMASK, which is 022. When Unix XORs 666 with 022, the permissions are reduced to 644. This will deny write access to g(roup) and o(ther) classes of user

Operational Considerations

Key Assignment

The <Esc> key will often not work with DP4, because it is generally used as a "lead-in" to the sequences generated when "grey keys" are pressed, (though pressing it two or three times in succession will work). The <F3> key is usually assigned instead. You may change this using the DP4 utility DFSETUP. The Esc key will work normally with terminal type U, which you can use if your terminal emulates a DEC VT220 in 8 bit mode.

On Linux it may be necessary to load a different keyboard mapping to get the "grey" keys to work properly if you run from a console rather than from an xterm. On SuSe Linux install ibcs support and try the following: When you install iBCS it should create a file called /usr/doc/packages/ibcs2/PROD.Patches/SCO.map If you copy this file to directory /usr/lib/kbd/keymaps/i386/ then loadkeys SCO will reprogram the keyboard so that it works properly with DP4. You can restore the default keyboard behaviour by typing loadkeys uk (or whatever the appropriate abbreviation is for your country). This file may not be provided with other Linux distributions. However if you can get hold of a copy of it it will probable work on your distribution.

An X Windows version of trm3 is available for Linux. If you use this then you should use a Windows userdata.sys (created with userdata -os windows). All grey keys work properly with this.

Backup and Restore

By default, database backup and restore is made to the bkp subdirectory of your DP4 installation. You may change this to be other locations on the file system, but you cannot make this a device (for example /dev/fd0). DP4 expects to find a file system, not a device.

The kill command

The kill command is used to send a signal to specified processes. It can be used to terminate a process which appears to have hung. Most DP4 programs will respond (by terminating cleanly) to the default signal , which is signal 15 (SIGTERM). The signalled process must belong to the current user unless the user is the super-user. Refer to your Unix manual for a full description of the kill command. When you kill a DP4 program it will attempt to clean up - for example by killing its terminal manager and logging off from the database manager. In 4.523 killing a trm3 process will attempt to kill the program it belongs to (there are no trm3 processes in 4.525). You should allow up to 6 or 7 seconds for the kill to succeed. If the process has still not terminated by then you can use kill -9 to force it to terminate immediately. In this case you will need to manually clean up the program's terminal manager, and its IPC channels.

Server Shutdown

If you load any of the server or networking programs in the background, you can shut them down using the kill command, but DO NOT use kill −9 . This will terminate the processes and leave IPC channels open that will stop further DP4 programs from loading correctly.

If you are using release 4.525 you are recommended to use dbdaemon -start and dbdaemon -stop to start and stop DP4 services. dbdaemon uses the [preload] and [startup] sections of the DP4 Configuration file to decided which DP4 components to run as part of the DP4 service. If DP4 services are running under dbdaemon, either running dbdaemon -stop, or running kill against the dbdaemon pid will cleanly terminate all DP4 processes and clean up all DP4 IPC channels. For more information on dbdaemon refer to the Guide to DP4 Networking and Resilience, and relevant portions of the Guide to DP4 Configuration.

Processes and Inter-Process Communications

DP4 for Unix uses the inter-process communication (IPC) facilities of the operating system to pass messages between the the application programs and the database manager/terminal manager.

If your programs abort, you may need to perform tidy-up operations for processes and inter-process communication structures, as explained in this section. The DP4 C library attempts to catch signals that commonly cause programs to abort and then to clean up IPC information that could cause problems to other programs, but in some circumstances this may fail for one reason or another.

You can use the ps command to check what processes are running on Unix or Linux. ps −e prints information about every process that is running. On Linux the ps command is rather different from standard Unix, and you use ps ax instead (you can probably get away with just ps a so long as no DP4 programs are running without a controlling terminal).

You can use the ipcs command to display information about IPC, and the ipcrm to delete resources from programs that have aborted.

IPC Example (4.523)

The details of this section are specific to release 4.523, For a version of this section appropriate to 4.525 read IPC Example(4.525). However, the 4.525 example does not cover using ipcrm in so much detail (mainly because it should hardly ever be necessary to use it.)

This example shows a DP4 application, my_app , running under Unix. To run my_app on a database called "test", you use these commands:

my_app −db test

my_app loads and executes in this sequence:

  1. my_app loads and initialises the database server (if it is not already running)

  2. my_app loads and initialises the terminal manager

  3. The my_app program is ready and starts executing its own code calling the DP4 database manager or terminal manager as appropriate.

You may (on Unix and AIX) examine the process state with the following command:

ps −fu datafit

This then displays information about every process now running for the datafit user:

UID     PID PPID C STIME    TTY     TIME COMMAND
datafit 198 160  0 14:12:20 console 0:00 -ksh
datafit 212 198  0 14:48:12 console 0:00 my_app /usr/datafit/applics/my_app
datafit 213 212  0 14:48:12 console 0:00 srv3 -nodetach
datafit 214 212  0 14:48:12 console 0:01 trm3

PID is the process identity, and PPID is the process identity of the parent process.

Run ipcs to display the IPC status:

Message Queues:

T ID  KEY        MODE        OWNER
q 200 0x33565253 -Rrw-rw-rw- datafit
q 1   0x334d5254 -Rrw-rw-rw- datafit

Shared Memory:
T ID 	 KEY        MODE        OWNER
m 400 0x33565253 --rw-rw-rw- datafit
m 401 0x00000000 --rw-rw-rw- datafit

Semaphores:
T ID  KEY        MODE        OWNER
s 40  0x33565253 --ra-ra-ra- datafit
s 41  0x00000000 --ra-ra-ra- datafit

If my_app hangs, the system can be restored as follows:

  1. Kill the my_app executable process:

    kill 212

  2. Wait 6-7 seconds. Check with ps if the terminal manager has terminated. If not kill the associated terminal manager process:

    kill 216

  3. Remove associated inter-process communication facilities. To do this, on Unix use the following command:

    ipcrm -s 41 -s 40 -m 401 -m 400 -q 200 -q 1

    where:

    -q 200 and -q 1 Remove the message queue identifiers, 200 and 1, from the system and destroy the message queues and data structures associated with them

    -m 401 and -m 400 Remove the shared memory identifier, 401, from the system. The shared memory segment and data structure associated with it are destroyed after the last detach

    -s 41 and -s 40 Remove the semaphore identifiers, 41 and 40, from the system and destroy the set of semaphores and data structures associated with them. The various ids will vary. The DP4 IPC can usually be recognised by the key or owner.

Care should be exercised when killing processes and inter-process facilities. ipcrm should never be used when other DP4 programs are still running as you may accidentally delete information used by these programs.

Refer to your Unix manual for a complete description of the ps(1) , kill(1) , ipcs(1) and ipcrm(1) commands.

On some versions of Linux the ipcrm command works differently and you may well have to invoke it several times deleting one item at a time. Instead of -s -m etc you specify IPC objects by type name (e.g. shm). See your Linux documentation for full details

When DP4 programs are closed down cleanly almost all DP4 IPC is removed. In release 4.523 two very small items, a queue which usually has key 0x334d5254 and shared memory with key 0xdaafade3 are never deleted once created, as to do so could cause problems if two or more DP4 programs start or stop simultaneously.

Example (4.525)

This example shows the DP4 application, dfsetup , running under Linux in a user account named dp4user.

dfsetup loads and executes in this sequence:

  1. dfsetup loads and initialises dbdaemon (if the DP4 service is not already running)

  2. dfsetup starts executing its own code calling the DP4 database manager or terminal manager (which is a shared library and not a separate process as in earlier releases) as appropriate.

You may (on Linux) examine the process state with the following command:

ps x

This then displays information about every process now running for the logged on user:

  PID TTY STAT TIME COMMAND
13788  p0 S    0:00 -bash
13801  ?  S    0:00 dbdaemon -start
13803  ?  S    0:00 errlog -start
13805  ?  S    0:00 srv3 -AUX
13807  ?  S    0:00 tcpmgr -DISPLAY_ADDRESS -AUX -DEBUG_CONNECT
13808  ?  S    0:00 tcpmgr -DISPLAY_ADDRESS -AUX -DEBUG_CONNECT
13810  ?  S    0:00 tcp3 -DEBUG_CONNECT -SERVER_NAME localhost
13815  p0 S    0:00 dfsetup
13817  p1 S    0:00 -bash
13832  p1 R    0:00 ps x

PID is the process identity, and TTY is the controlling terminal of the process. None of the dbdaemon processes have a controlling terminal so they will not show up without the x option.

Run ipcs to display the IPC status:

------ Shared Memory Segments --------
key       shmid     owner     perms     bytes     nattch    status
0x34504433 17792     dp4user   666       144       6
0x00000000 17793     dp4user   666       8320      2
0x00000000 17794     dp4user   666       9816      2
0x00000000 17795     dp4user   666       3080      2
0x00000000 17796     dp4user   666       8320      1         dest
0x00000000 17797     dp4user   666       8320      2
0x00000000 17798     dp4user   666       8320      2
0x00000000 17799     dp4user   666       8320      4

------ Semaphore Arrays --------
key       semid     owner     perms     nsems     status
0x00000000 30464     dp4user   666       1
0x00000000 31105     dp4user   666       1
0x00000000 30594     dp4user   666       1
0x00000000 30979     dp4user   666       1
0x00000000 30724     dp4user   666       1
0x00000000 30725     dp4user   666       3
0x00000000 30726     dp4user   666       1

------ Message Queues --------
key       msqid     owner     perms     used-bytes  messages
0x34504435 6528      dp4user   666       0           0
0x34504432 6529      dp4user   666       0           0
0x34504430 6530      dp4user   666       0           0

If you compare this information with the corresponding information for the 4.523 example, you will see that it is considerably more complex. There are two reasons for this:

  • DP4 is loaded in a more elaborate configuration than the 4.523 example - the named connection configuration is being used, and not a simple stand-alone configuration
  • Release 4.523 allocated one shared memory segment, one semaphore, and one message queue per component of the DP4 service, plus one permanent shared memory segment for control purposes. From release 4.525 there is one shared memory segment and one semaphore per running program (including one for each copy of tcpmgr), and one message queue for each server type component of the DP4 service. tcpmgr creates a number of other semaphores and shared memory segments, and there is still a control shared memory segment (the one with key 0x34504433).

Another point to note is that most of the 4.525 IPC uses private keys. Only the Message Queues and the control shared memory segment have non-private keys. This may make it quite difficult to work out which shared memory segment is which. The -p option of ipcs may help with this.

If DFSETUP hangs, the system can be restored as follows:

  1. Kill dfsetup executable process:

    kill 13815

  2. Wait 6-7 seconds. Check with ps if the process has terminated. If not repeat the kill with kill -9

  3. It is probably a very bad idea to try to clean up any DP4 IPC while the DP4 service is still running - any orphaned shared memory segments and semaphores will not be interfering with the operation of DP4. If you are sure there is a significant amount of IPC that has not been cleaned up when it should have been use dbdaemon -stop to kill dbdaemon, or kill the dbdaemon process (kill 13801) in this case). Wait several seconds for the DP4 service processes to terminate. If there were any other running programs they will get FAIL error 9 when they next access the DP4 database manager, but should terminate cleanly. After this you can use ipcrm to remove any remaining IPC for DP4, in the same way as was described for the 4.523 example.

  4. If you are absolutely determined to remove IPC for DP4 programs while leaving other DP4 programs running you may find it useful to note the following points:

    • In 4.525 never remove IPC entries with non zero keys. These entries are used by many DP4 programs and deleting them will inevitably cause serious problems.
    • The only IPC that may safely be deleted is the per program shared memory segment (currently always 8320 bytes in size), and the semaphore created by the program to control it. You can use ipcs -p to display owner information about IPC. Note that the control segment and one very important semaphore with a private key are created by the first DP4 program that runs. Never delete semaphores for the pid that created the control segment.
    • Normally killing dbdaemon will clear up all DP4 IPC. Even it does not it will usually be harmless, and DP4 can be restarted later on. However, if a DP4 service is terminated with kill -9 or otherwise dies horribly, dbdaemon will not be able to reload itself if the 144 byte control segment is still in existence, because the IPC channel used by that part of the DP4 service will be considered to be already occupied. In this case deleting the control segment will allow DP4 to be reloaded.

Keys for IPC in DP4

Traditionally DP4 used hard coded keys from its IPC. This is considered to be bad practice as there is a remote possibility of a clash with other software. In 4.523 you can make dp4 use system generated IPC keys by creating a file called dp4ftok in the /etc/ directory. A simple one line text file should be enough - the contents of the file do not matter, except that it should not be of zero size. Make sure that the file has read write access for all users (chmod 666 /etc/dp4ftok). You must make sure all your Unix programs have been rebuilt with 4.523 libraries before using this facility, as old programs will stop working.

Because IPC for release 4.525 is completely different from that for release 4.523 a different set of IPC keys are used for DP4. The default, hard-coded, keys all begin with 0x345044 on Intel computers (this is DP4 backwards). To use system generated IPC keys create a file called edp4ftok, either in the $DP4HOME directory, or in the /etc directory. As with the corresponding file for 4.523, the file can be a one line text file, but must not be of zero size, and must acccessible to all users.

Printing from DP4 on Unix

When a DP4 program prints under Unix, it issues an lp command. This command is hard-coded into the terminal manager ( trm3 ); you cannot change this. It is possible to create a customised lp command that must be located in your PATH before the directory for the Unix system version of lp . Note that any such replacement must be a binary file - a shell script does not work.

The terminal manager issues the following command when sending a file to the printer:

lp -c -s -d device_name <spool_file>

device_name is either prnX (where X is an integer, 1 to 7), or user-supplied, used if you define an environment variable of the form PRNX .

Example 1: Printing with Device Groups

In this example, DFSETUP has been configured such that a DP4 "device group", called HP_LASER_II, is attached to prn3.

DP4 SYSTEM SET UP Device Group description

DG name       HP_LASER_II               Soft form feed ? Yes
Device number 115                       Null character   !
Description   HP LaserJet (Single Bin)  Special device ? No
Page length   64                        Hidden device ?  No
Page width    78                        Printer port     PRN3

All printing through HP_LASER_II uses a Unix printer called 'prn3'.

Example 2: Using Existing Printer Names

In this example, you are installing DP4 on a Unix system that has two printers 'hplaser1' and 'hpinkjet' already configured. The DP4 programs should use the existing printers.

The terminal manager allows you to alias printer names by creating suitable environment variables in your .profile , as in:

PRN3=hplaser1;export PRN3
PRN4=hpinkjet;export PRN4

You then define suitable DP4 print devices which print to PRN3 and PRN4 respectively.

Program Development

The DP4 C/C++ Programmer's Reference and QA Build Developer's Guide describe how to develop application programs. For information specific to application development under Unix and Linux see the following notes:

  • Development with QA Build
  • How to convert DP4 C programs from DOS to Unix
  • The DP4 C library, ccown.a
  • Compiling C Programs
  • Multi-Threading Issues
  • Data Alignment
  • Filename Issues
  • Debugging DP4 C application programs

Copying Files to and from Unix

This section describes a number of ways to copy files between Windows/DOS machines and Unix. Some work on all variants of Unix, whilst others do not.

All DOS text files use a carriage-return/line-feed combination, CR-LF (hex 0D and 0A), to indicate a new line, and sometimes a CAN (hex 1A) at the end of the file. Unix system text files use a single LF character to indicate a new line, and sometimes an EOT (hex 04) at the end of the file. (On any platform the DP4 editor DP4ED can read files that use either convention. DP4ED always saves files in the native format for the operating system it is working on. Some Unix C compilers reject MS-DOS format files.

Some, but by no means all, Windows/DOS text editors can also read Unix text files (e.g. edit can, but notepad cannot).

In the absence of any other suitable utilities, you can use DP4 network facilities to transfer files with the NETCOPY command. See the DP4 Network Resilience Developers Manual for full details. However it is usually easy to configure Unix and Linux machines to act as ftp servers, and you can use the ftp program supplied with Windows to transfer files. There are a large number of Windows based ftp programs with a simple user interface that you can use instead of the traditional command line ftp.

The FTP program can convert files between DOS and Unix formats, and so can the DP4 version of the tar utility.

DP4 databases are identical on all platforms - you can transfer a DP4 database between any two platforms and it will work identically. If using a program such as ftp you must ensure that database files are transferred in binary mode

The tar Utility

Itim Technology Solutions can supply a DOS or Win32 Console version of the tar utility for reading and writing tar archives. This gives an easy method of transferring files to and from a Unix file system as it can perform the conversions required for text files, and transferring one tar archive, is usually easier than transferring many small files. It also allows date and time information for the file to be preserved. We use this program to prepare all the Unix packages we issue, and to transfer the source code of DP4 onto Unix and Linux machines. The utility is fully described here.


What's New in the DP4 for Unix Installation Guide

What's New in DP4 for Unix Installation Guide

This page will tell you about recent changes to this manual. The most recent changes are mentioned first.

New in 4.622/4.526

  • 4 Nov 2004 The Linux release of DP4 now supports databases of effectively unlimited size. Note on Large File Support added.

New in 4.621/4.525

  • 31 Mar 2004 Note on multi-threading added.

  • 24 Mar 2004 Covers the changes to DP4 for Unix and Linux for release 4.525

New in 4.620

  • 17 Oct 2002 What's New Page Added

New in 4.523/4.619

  • This Installation Guide was new for this release.


DP4 Program Development on Unix and Linux

QA Build Programming

If several users wish to develop QA Build programs on the same database, they must ALL use the −share command tail on PROGMAKE.

Programs are locked by PROGMAKE to prevent two users concurrently updating the same program. A program is unlocked when the user selects a new program name or exits from PROGMAKE.

When QAB programs are compilied on Unix the maximum program size will be 64K. This means that your programs may not execute on 16 bit operating systems unless you build a custom version of PROGRUN. Specify the command tail -size 42000 if you want your QAB programs to be able to execute using the standard progrun on MS-DOS or other 16 bit operating systems.

Converting DP4 DOS Applications

Up to release 4.523 you can convert DP4 DOS application programs that are operating system-independent (explained below) by using the EXE2UNIX utility. EXE2UNIX is not supported from release 4.525 (because 4.525 uses ELF only facilities for communicating with the DP4 database manager). EXE2UNIX needs a special, compiler-specific, dummy file. For example, when using Metaware High C, you use HCDUMMY.UNX. Not all versions of Unix support programs built this way, and it is very difficult to debug such programs on Unix, so you are recommended to compile a "native" version of your application whenever possible.

With the .EXE version of the application program and the dummy file in the current directory, the application can be converted with the following command line (this can be done wither on DOS or Unix):

exe2unix program.exe

Ensure that the resulting file is set with the 'x' access mode when installed.

OS-independent programs must be compiled as small model programs and linked using the DP4 "root" module, xxroot2.obj, where xx represents the compiler used.

Unix C Library

The DP4 Unix C library ccown.a, which is installed in the lib subdirectory of your DP4 install directory, contains the modules required to support the development of application programs written in DP4 C. (From release 4.525 and on Linux ccown.a is in fact a linker script that causes a number of other libraries to be seached.) You should install the appropriate DP4 C library package for your version of Unix. The QAB libary is ccqab.a or ccqabd.a for the debug version.

The easiest way to compile and link a program with one or more of these libraries is to add the full name of the library to the command line, not to try using the -l option. For example cc -o test test.c $DATAFIT/lib/ccown.a will compile and link a program test from test.c and the DP4 C library. If you leave the DP4 C libraries and headers in their default locations you will need to use the -I option, and from 4.525 the -L option:

  • 4.523 and earlier

    cc -o test -I$DATAFIT/include test.c $DATAFIT/lib/ccown.a

  • 4.525 or later

    cc -o test -I$DP4HOME/include test.c -L$DP4HOME/lib -L$DP4HOME/bin $DP4HOME/lib/ccown.a

The command given above for 4.525 causes the location of the DP4 shared libraries to be hard coded into the executable, which you may or may not consider to be desirable. If you want to avoid this happening copy the DP4 .so files from the bin subdirectory of your DP4 installation into the directory where you are building your program. The libraries will then be loaded from the current directory so the path won't be in the executable. In this case you will need the LD_LIBRARY_PATH environment variable to point to the appropriate location at run time.

C Tips

  • Don't use C++ style comments in C programs. It's not valid C (at least not unless you have ISO C99) and some Unix C compilers reject them.

  • Include standard headers such as stdlib.h before DP4 header files. This gives DP4 a chance to undo any damage done by the standard header files.

  • Try not to use OS specific functions (such as WIN32 APIs) in C programs. If you have to then put OS specific stuff in separate modules, or inside #if defined(WIN32) sections so that you can port programs more easily.

  • There are special considerations when running on non Intel hardware. If you are ever asked to port programs to non Intel Unix, be very careful before agreeing. DP4 Programs designed to be object code portable, and which only access DP4 databases (not other files) are usually portable, other programs will need modification. Until you have a good understanding of the differences between Intel and SPARC chips your program will probably have non portabilities in it - especially if it casts pointers from one type to another.

Compiling C Programs

The Unix program cc allows you to compile, link and assemble using one command. cc examines the input file extension and executes the appropriate code to process the file, as in:

cc test.c Compile and link the C module
cc test.o Link the object module
cc test.s Assemble and link the assembler module

On some machines the C compiler may be called gcc or c89 rather than cc, or there may be several C compilers and the best one to use for DP4 may not be cc itself. There is almost always an environment variable CC which contains the name of the C compiler.

Although you can compile and link in one step we recommend using the Unix make program to build C programs. This allows you to construct a file showing exactly how each program in a system is built and what other files it depends on.

The C compiler normally accepts a large number of options which affect its behaviour. It is a good idea to use the environment variable CFLAGS to store the flags needed for a particular version of Unix. This avoids you having to store this value in make files, shell scripts etc. Our practice is to create a single shell script for each platform, always called startup.sh, which contains appropriate definitions of these and other similar variables used in building C programs. This allows us to build programs the same way on all platforms, using the same make file on all platforms. Porting DP4 to a new version of Unix is then almost as simple as creating the new version of the startup.sh file.

Multi-threading Issues

Prior to release 4.525, DP4 does not support use of multi-threading. If an application uses the pthreads library it must provide its own mechanism to ensure that at most one thread is inside the DP4 libraries, and save and restore DP4 global variables as required.

In 4.525 DP4DBAPI applications can multi-thread providing they set use_pthreads=1 in the [progname] section of dp4.ini. See documentation of the UNIX section in the Guide to DP4 Configuration for details. Please note that at the time of writing the use of multi-threading has not actually been tested on Linux/Unix, though the 4.525 DP4 libraries are almost source code identical to the Windows 4.621 release, which has been tested with multi-threading. Applications using the traditional DP4 C library should probably never use pthreads, unless only one thread calls DP4 functions, because this library makes such heavy use of global variables.

The use of fork() is also supported. A forked process should obtain a new connection to the database manager and re-open any databases it needs. Using fork() is potentially more efficient than using threads, because each process has its own queue into the DP4 database manager, whereas with threads all threads share a single queue.

Data Alignment

On Intel processors DP4 C programs may require data to be packed on single byte boundaries, as this is the way the data is stored on the database, and historically, Intel processors were always able to access data efficiently however it is aligned. This can cause problems when compiling DP4 C.

  • You need to know how to make your C compiler pack data like this. Most compilers either accept the −Zp1 option, or implement a #pragma pack(1) option. Neither of these are always documented. It is worth writing a small test program to check whether either or both of these options work:
    #include <stdio.h>
int main()
{
  struct FRED
  {
     char dummy;
     double data;
  } fred;
   printf("%d\n",sizeof(fred));
}

    The program will print out 9 if data is packed on 1 byte boundaries. Otherwise it will print out a value between 10 and 16 depending on how data is aligned. For a compiler to work with DP4 you must be able to find some method of compilation that causes the answer to be 9. (This is not the case on non Intel processors where DP4 can cope with data that is aligned on larger boundaries.)

  • On some versions of Unix System header files and libraries may misbehave with programs compiled with the Zp1 option or the #pragma pack(1) option.This problem has been observed on several different platforms including SCO Unix and Linux. If possible use the #pragma pack(1) option rather than Zp1, and include any non DP4 header files before the DP4 header files. If you do this you can set the pack() pragma only after the System header files have been included.

If you use number system 9 the data in your program will be aligned so that it satisfies all alignment restrictions. Nevertheless you must still compile in a way that forces single byte alignment. The reason for this is that where tables contains fields with roles the compiler may otherwise insert additional unecessary padding bytes to round up the aggregate size of the group of fields with a particular role, but the DP4 program MAKELINK does not.

From release 4.525 the standard DP4 include files and the header files generated by LIBMAKE contain appropriate calls to #pragma pack, so there should be no longer be any need to add your own #pragma pack calls.

Filename Issues

MS-DOS filenames can have an eight character filename and a three character extension. Windows 95 and NT allows long filenames but no distinction is made between upper and lower case characters. The separator (.) between the filename and extension is implied, not explicit.

Unix filenames are also long. A distinction is made between upper and lower case characters in Unix. A period (.) is often used as a separator to show name and type, for example "hello.c", but in this case (.) is simply one of the characters in the filename. It is not uncommon to see files on Unix with several "suffixes" (as the part of a filename beginning with a dot is known).

Directories are separated with a forward slash (/) in Unix, but with a back slash (\) in DOS and Windows (though in fact a forward slash also works). This can cause minor problems with the shell where entering a back slash causes the next character to be taken as a literal, so that deleting an unintentionally typed back slash may require you to hit the backspace key several times.

When copying files between DOS and Unix, or when porting applications, you should be aware of these considerations. One point worth remembering is the padding of names with spaces when moving applications from DOS to Unix.

DP4 file-access functions, such as tf_open() and tf_check() , convert the filename argument to lower case before attempting to access the file you cannot access a file with a name containing upper case letters using dp4 functions. The following example illustrates the use of these functions, and why filenames must, when converted from DOS to Unix, never end with spaces. Suppose example1.txt exists as a file in the current directory, and NAMES is a table, containing a field called file_name defined as a character field twenty characters long.

This record exists on the database:

NAME_INDEX = 1
FILE_NAME = "example1.txt "

The C program extract is:

names.name_index = 1;
if (ftchprm(EQUAL,&names.l))
...
if (tf_check(names.file_name))
...

Under MS-DOS, the file is found because the filename is restricted to 12 characters.

Under Unix, "example1.txt" is not found using release 4.523 or earlier because Unix includes the eight spaces at the end of the filename and looks for "example1.txt        ". From release 4.525 Unix/Linux DP4 automatically strips trailing spaces from filenames. The Windows version of DP4 does not need to strip trailing spaces from filenames, because the operating system does this itself.

Debugging on Unix and Linux

On most versions of Unix and Linux you can debug programs by compiling them with the -g option. You can debug at a source code level using a program such as sdb or gdb (the name will depend on the version of Unix/Linux and the C compiler you are using). It is worth remembering that if you use only ANSI and DP4 C APIs your program should be completely portable, so that you may be able to debug it on another platform if you prefer. For example we often debug problems reported in the Unix version of DP4 by reproducing them in the Windows version and debugging with Visual C++.

If you want to put diagnostics into a DP4 program you may find it more convenient to use sprintf() followed by showc(), or fprintf() than printf(). If you use printf() your diagnostics may interfere with terminal manager output, or be interfered with by it.

A quick and dirty method of narrowing down the area of a program where a problem occurs is to insert calls sys_request(2,a,b) where a and b are numbers that identify where you are in the program.


Novice's Guide to Unix and Linux

Unix (with Linux) is probably still the most important operating system in the world. Most other operating systems in common use incorporate many features of Unix. For example the MS-DOS command line does input output redirection and piping in the same way as Unix, and has a similar directory tree. Windows NT executables use a file format directly derived from Unix, and the OBJ files created by Win32 C compilers are in a format known as COFF (common object file format) which was invented for Unix. NT security, kernel and networking functions incorporate many ideas from Unix. (On the other hand new versions of Unix are incorporating more and more Windowsy features)

Unix can be very daunting to the novice. Unix seems to have been invented for computer users who cannot type and have a religious objection to words with vowels in. Most Unix commands have very short and obscure names. The other things which make Unix difficult to use are:

  • "Orthogonality". Unix programs hate providing a feature already found in another utility. It is often necessary to run several programs in a row, piping the output of one to the input of the next to accomplish common tasks.
  • The manuals. These are extremely comprehensive, but very hard to understand as the writers love explaining things in abstract and hate giving useful examples.
  • "Batch mode ". Unix utilities are generally not very interactive and are not user friendly. Most utilities are designed to be easily controllable from shell scripts (similar to batch files) . These even extends to things like text editors. This makes it possible to automate administration tasks to a remarkable extent, but it doesn't help novice users at all.

However, if you are used to using a DOS command prompt, you will be able to use Unix without too much difficulty. Although Unix is difficult, the various releases of Unix are very similar to each other and use most of the same commands. Installation and administration of a Unix machine is a bit more complicated, and tends to be release specific.

Unix servers are usually accessible without leaving your normal PC. You can use a program called telnet to log in and run programs on a Unix server from your Windows PC.

Advantages of using Unix or Linux

  • Unix is EXTREMELY robust. Unix machines almost never crash unless there is a hardware problem, and can run for months without being rebooted. Linux is not so robust - an ill behaved program can bring Linux down, but it still has a mean time between failures that NT can only dream of.
  • Unix and Linux are not greedy - they need significantly less memory and hard disk space than NT, and make better use of the hardware (i.e. run programs faster).
  • Unix is a genuine multi-user OS unlike NT (unless you are using NT Terminal Server). Many users can use a single Unix machine with cheap dumb terminals or from TELNET sessions on PCs. In either of these configurations there is no need for DP4 networking, so it is an excellent platform for DP4.

When you boot a Unix machine or connect to it with TELNET you will have to log in by supplying a username and password. After logging in you will get a command prompt, which is traditionally just a $

Cunning differences from MS-DOS/Windows

  • There are no drive letters. Everything is a subdirectory of /. (Unix paths use / instead of \).
  • Filenames are case sensitive. X.X and x.x are two different filenames. In DP4 all filenames are converted to all lower case to ensure programs port from other operating systems. In general it is best to use lower case filenames. Login names and passwords are also case sensitive. Make sure Caps Lock is not on when you log in, otherwise Unix may think you are running on a terminal without lowercase letters, convert everything to lowercase automatically, and leave you without a way of entering capital letters!
  • Unix does not have file types in the same way as MS-DOS/Windows. "." is just a character that can appear in a filename. Suffixes are used to denote file types to some extent, but Unix will quite happily execute executables called fred.xyz , fred.exe, or plain fred (by far the most common kind of executable). Unix will also happily execute a shell script called fred. Unlike DOS or Windows it does not look at the file type to decide what kind of file it is dealing with. (This may not be true with all X Windows programs). Instead it looks for "magic numbers" at the start of the file's data. This can sometimes cause problems. For example if you copy an executable to a different machine, try to execute it, and the magic number is not recognised, Unix may try to execute the program as a shell script!
  • On Unix EVERYTHING is a file. For example there is a directory called /dev , with entries called things like cdrom,mem,hd0. You can dump the entire contents of the computer memory to screen by typing cat /dev/mem ! (You can dump your hard disk in a similar way) Accessing things like tape streamers is often partly a matter of discovering the appropriate obscure device name in this directory.
  • The same physical file or directory on a Unix machine can manifest itself under several filenames and in different directories. This is done using "links", usually "symbolic" links. These are a little bit like Windows short cuts, but there is even less of a difference between opening a link and opening a short cut - programs that open a link will not usually even be aware that they have done it. Links are very commonly used to map files into "standard" places.
  • Unix files have an owner (one of the user accounts), and a group (each user account is a member of a group) , and a set of permissions, things the owner, members of the group, and other users, (world) are allowed to do to the file. You will frequently find that you do not have permission to access a file. When this happens you will need to change the permissions or owner of the file, which may involve acquiring "root" privileges. (This is very similar to logging on as administrator on an NT machine). See chmod and chown for details.
  • Unix does not search the current directory when you type in a program name on the command line. At least not unless ./ appears in the path.
  • There are some differences from MS-DOS in how you change or access environment variables. Here is how you would change your path to include a directory called /fred.
    PATH=/fred : $PATH
    export PATH
  • If you are running in a TELNET session you may find the backspace key does not work on the command line. In this case the Del key will probably be a backspace. You can force the backspace key to work nicely using this command: stty erase <backspace> (I.E. Type the backspace, after stty erase and then hit enter)
  • Unix command tails always begin with a - (like DP4), but you can combine them. So instead of doing ls -l -t you can enter ls -lt instead. Quite frequently programs will need half a dozen obscure single letter command tails to do anything useful. Linux has a new kind of command tail (from Unix 98) that starts with --. These command tails are more like DP4 ones, i.e. proper words.

Useful commands

more

Unix commands don't page their output nicely. You can use more in the same way as on MS-DOS. e.g. ls -lt | more. Sometimes there is a more powerful equivalent called pg or less . Backspace may or may not let you go back a page.

ls

Directory listing. Use ls -l for listing with date/time. ls -lt lists by date/time. Sometimes lc gives a multi-column listing.

You can specify a filename to restrict the search:

ls a* looks for files beginning with a

ls *a* looks for files with an a in the name.

N.B. *.* matches files with a . in the name, NOT all files.

mv

Used to rename files. e.g. mv oldfilename newfilename

cd

Change directory. cd by itself takes you back to your login directory.

pwd Displays the current directory.
cat

Can be used to "type" files.

e.g. cat fred.c

cp

Used to copy files. e.g. cp file1 file2

You have to give two arguments. So to copy a file to the current directory from somewhere else you do something like this:

cp /usr/datafit/bkp/system.dat .

su

Used to become super-user (root). You will be asked for the root password. Running su is not the same as logging on as root, as you don't get the root path or environment. This may mean that administration tools are not in your path. Log on as root to administer a Unix machine, or sysadm on some flavours.

mv

Used to rename files. e.g. mv oldfilename newfilename

rm

Used to delete files. You can use rm -r to delete whole directory trees. (Be careful - there is no undelete!)

chmod

Used to change the permissions for a file.

The most common uses for this program are to make something executable, or to make something writable.

chmod +x filename , makes filename executable

chmod +w filename , makes filename writable.

You can also specify desired accesses with octal numbers got by adding up 1=execute 2=write 4=read

e.g chmod 666 filename makes filename readable and writeable by everyone.

chown

e.g. chown fred filename . Makes fred the owner of filename . You may well want to run chown after extracting a tar file from another machine

chgrp

e.g. chgrp system filename. Makes filename part of the "system" group.

ps

Displays running programs. This has different command tails depending on what you are running on.

ps -a or ps -e or ps -u datafit are likely to be useful.

man

Displays an online manual. You have to say what you want to read. e.g. man ps will tell you about the command tails that ps accepts.

On Linux a program called info is used as well.

Unfortunately man pages delight in explaining commands in a highly abstract way, which is usually extremely hard to understand, and hardly ever give examples.

find

Used to find a file on the machine. For example: find / -name *fred* -print lists files with fred in the name.

grep

Finds text in files. You can use it in a simple way. e.g. grep Fred * looks for files containing Fred .

This program is VERY complicated. Use man to find out more. Note that the search string is case sensitive.

tar

This program is used to "zip" files, though it does not necessarily compress them. DP4 is installed using this program. tar -x can be used to extract files, or tar -xv if you want to see what is going on. On Linux you may need to specify tar -xPv or tar -xP .

There is a DP4 version of tar which can be used to create tar files and tar format diskettes. winzip can read tar files.

tar diskettes are not readable using DOS or Windows (except with tar) , because the area used for the FAT and directory is just used for data. They need to be reformatted to be usable as ordinary diskettes.

telnet

You can use this program from a Windows (or another Unix PC) to remotely access any of the Unix machines in this office.

ftp

You use this program to transfer files from your local machine to or from a Unix machine. Separate instructions are given later on.

ipcs

Used to display stuff called shared memory, queues and semaphores. You may need to use this program if there is a problem with DP4

ipcrm

Used to clean up the information displayed by ipcs. The ipcrm on some distributions of Liunx is completely different from the Unix program.

vi

The Unix text editor. It's hard to use until you get used to it.

If DP4 is installed and working dp4ed may be an easier alternative, but cannot be used for files needing Tab characters (e.g. makefiles)

ln

In Unix or Linux it is possible to create "alias" names for files. For example DP4 is always installed in a directory called /usr/datafit. However you are likely to want to install it somewhere else, such as /home/dp4 instead. Using ln you can create a "link" which will fool DP4 into thinking it is installed in /usr/datafit when it is really in /home/dp4.

Typically this is done with ln -s. Consult man ln for details (which may be different on different machines).

Ctrl+Alt+Func key

Usually Unix comes with lots of "virtual consoles", and you can use this key combination to switch to another. This can be useful if something goes wrong! On some machines you use Alt+Print Screen, followed by a number key instead.

kill

Used to kill programs that have hung. You have to use ps to disover the "pid" (process id). Then kill nnn where nnn is the pid should kill the program. If it doesn't kill -9 pid will definitely work.

init

Used to shutdown or reboot the machine. init 0 does a shutdown. init 6 does a reboot. Ctrl+Alt+Del works on Linux.

You have to be "super user" to do this. Either log in as root, or run su and give the root password.

doscp,dosdir

Unix machines are often capable of reading and writing information from DOS diskettes or hard disks. The commands used to do this tend to be different on each flavour of Unix. This can be useful - for example on a "dual boot" machine when you want to copy files from a Unix to a DOS partition.

Basic FTP

ftp is used to transfer files to and from Unix servers (or other servers - you can't tell what OS an ftp server is running).

An extremely important point to bear in mind is that Unix text files have a different format from DOS. When copying to and from Unix ftp can do one of three things:

  • It can translate all files between the formats. This is called "ascii" transfer. Copying anything other than a text file like this will probably corrupt the file completely.
  • It can copy all files in "binary" mode - without translation. DOS format text files are not liked by most Unix programs (for example C compilers may well refuse to compile the files). DOS/Windows text editors may or may not work with Unix files. You can use dp4ed to convert files to the "native" format for your OS, as it can read both and saves in the right format for the OS you are running on.
  • It can use guesswork to discover whether to use a binary or an ascii transfer. This is obviously a bit risky, and is not available with the regular Windows ftp client. It works well with most fancy ftp tools that are used for updating web sites.

You have to make sure you are an appropriate mode when copying files between DOS and Unix.

Most Unix machines set up an ftp server by default when Unix /Linux is installed. If this doesn't happen you will have to set up the server yourself. On Red Hat Linux this involves finding and installing something called wuftpd. Don't try it if you're feeling stressed!

open <server_name>

Creates a connection with a Unix or Linux server. e.g.

open rs6000

When you do this you will be asked to supply a username and a password. Different machines have different restrictions on what accounts you can use:

  • Some machines won't let you use an account with a blank password.
  • Some machines won't let you use the root account.

Depending on what you log on as there will also be restrictions as to what you can copy to and from the Unix machine.

Inactive ftp connections time out after a fairly short time - usually 5 or 15 minutes, so you may need to use this command several times.

put <srcfilename> <tarfilename>

Copies a file to the Unix server. Both filenames can include path information. You don't have to specify the tarfilename, in which case the file will be copied to your current directory on the target machine and the filename is converted to upper case. This is very annoying so you should specify the filename both times.

e.g. put x:\521\unix\unxsco.tar unxsco.tar

Tip: If you want to transfer a lot of files, it may well be easier to build a tar file containing all the files and ftp that

get <srcfilename> <tarfilename>

The reverse of put - copies a file from the Unix server. You can usually miss out <tarfilename>

ascii

Switches ftp to using ascii transfer

bin

Switches ftp to using binary transfer

ls

dir

Used to list contents of directories on the server. Pretty much the same as the regular Unix ls .

dir is equivalent to ls -l (but won't exist on most Unix ftp clients)

cd

pwd

Used to change and display directory on the server. Again the same as the regular Unix commands.

lcd

Used to display and change your directory on the local machine.

close

Terminates your connection.

quit

Exits ftp

help

Lists the available ftp commands. I've only included a selection - but I've hardly ever used any of the others.

Basic vi commands

vi is supposedly a wonderful editor. Some people like it so much they have even gone to the trouble of writing versions for MS-DOS/Windows. I am not a very skillful user of vi, so this summary is very rudimentary. Try the man pages for more information. When running vi, it is important that the term environment variable is set correctly. If it is not vi is likely to be unusable. This is especially the case if you are running vi in a telnet session. Beware of editing files with lines longer than 80 characters. This can be very confusing as it is all to easy to find yourselef editing something different from what you thought you were editing.

You start vi with vi filename where filename is the name of the file you want to edit. This will open the file. You can probably use the four cursor arrow keys to move round the file. If not term is not set correctly, or TELNET isn't set up .properly. To start editing the file you press i . After that characters you type will appear in the file. Backspace will delete characters you type in but not existing parts of the file. Esc will take you out of the edit mode again (and move the cursor back one space!), which you need to do when you want to use other commands. x will delete the character at the cursor. You can use a instead of i to enter an edit mode. You do this to append to the end of a line.You can go to a specific line in the file by entering a : followed by the desired line number. You can find the string fred in the file by entering /fred/ . Entering / repeatedly will cycle through all the occurrences of this string in the file. There is also a search and replace facility but I don't know how to use it. If I want to do serious editing I generally use ftp to transfer the file to DOS/Windows where I can use an editor I actually like. dd deletes a line, dw deletes a word. Ctrl+F pages down through the file. :x saves and exits. :q! exits without saving. :w saves without exiting.

Installing Unix/Linux

I'm not going to explain this in detail, but just give you a few tips:

  1. Read the Install Manual First
  2. Read the Install Manual Again
  3. Make sure you know EXACTLY what hardware you have in the machine. You'll need to know the type of CDROM and hard disk, graphics card and network card. Unix may, but usually won't, do hardware detection. For example, once when I was setting up X on a Unixware 7 machine, I was told by the person who gave me the machine I had a Matrox Millennium graphics card. Unix offered me a choice of three models of Matrox Millennium (and a host of other Matrox cards). None worked. In the end I took the lid off and found I had an ATI card. I swapped it for an S3 Trio (because I already had that card working in an old version of SCO Unix). Unixware 7 then detected that card quite happily. Linux is likely to be much better at detecting stuff than Unix. Red Hat seems better than SuSe at this (though this may just because we have a slightly newer release - The range of hardware Linux supports is extending rapidly all the time). Red Hat seems to be geared to simplifying installation by removal of choice (a bit like Windows really). If you are thinking of buying a PC with a view to running Unix or Linux on it, you should pick one with popular hardware.
  4. When you install Unix or Linux on a PC in a Windows NT network you will probably need to assign a static IP address. If you use DHCP you won't be able to get to it by name from Windows machines. Assign a static IP address and then add an "A" record to the DNS manager on the NT servers. This will make the machine addressable by name, and make it easy to run DP4 networking later on. Make sure that you name the machine appropriately. If you get this wrong you may get problems such as having the machine name only on a loopback connection in the hosts file which will stop TCPMGR working.


The DP4 tar Utility

A version of the Unix tar (tape archive) program is available from Itim Technology Solutions. tar saves and restores a collection of files into a single .tar file, which can then be transferred easily to another machine via the FTP utility. This is much more convenient that transferring many files individually.

Tar Utility Syntax

The syntax of the tar utility is:

tar [key] [filename filename filename...]

where key is one of:

-c Create a new archive file and add the named files to it

-x Extract the named files from the archive file. If no filenames are specified, every file is extracted

-t List the names of files in the archive

Additional arguments may be supplied:

-f filename Take filename as the name of the archive file.

-v Verbose. For -c or -x print each filename as it is processed. For -t provide a full directory listing

-w Print each filename to be processed and wait for user input; the action will be skipped unless it is confirmed by pressing the letter 'y' (yes)

-m Do not restore the file modification times; the modification time of each file is the time at which it is extracted from the tape or disk

-D Perform automatic conversion to, and from, MS-DOS format. This option can be specified multiple times. Specifying it a second time turns automatic conversion off for subsequent files, specifying it a third time tuens automatic conversion back on again, and so on.

-Z Perform automatic compression or uncompression: see below

-F <filename> Process the list of files named in <filename> . This argument may be repeated as often as required

Creating an Archive File

Arguments to the tar utility can be combined in any reasonable way, for example:

tar -cvf file.tar -F listfile

and

tar -c-v-f file.tar -F listfile

are equivalent. Both of these command lines create a new archive file, containing the files listed in listfile. While the archive is being created, the names of files are displayed as they are processed. The archive file is a raw disk file and can be used to transfer files to Unix machines.

Some points worth noting are:

  • If you include more arguments after listfile, they are taken as the names of files to process
  • Wildcards can be used and are expanded according to the usual MS-DOS/Windows conventions
  • Any filename prefixed with '@' is treated as the name of a file containing a list of further arguments, in similar way to the -F option. The previous example could also be entered as: tar -cvf file.tar @listfile

Source and Target File Renaming

An argument of the form:

<targetname>=<sourcename>

causes automatic renaming. With the -c option, the MS-DOS file 'sourcename' is stored in the archive under the name ' targetname . For example, the command: tar -cvf file.tar /opt/dp4/dbs/system.dat=c:\dp4\db\system.dat

could be used to prepare a disk on an MS-DOS machine for transfer to a Unix system. The command:

tar -xvf file.tar c:\dp4\db\system.dat=/opt/dp4/dbs/system.dat

could be used to recover the file back onto an MS-DOS machine.

File Conversion Options

If the -D option is supplied:

  • Files copied to archives have CR/LF converted to LF
  • Files copied from archive have LF converted to CR/LF

If the -Z option is supplied:

  • Files copied to tape are automatically compressed and the suffix .Z added to their name (if it is not already present)
  • Files copied from tape will be uncompressed if they appear to be compressed and any suffix .Z removed from their name. The algorithm used is compatible with the Unix compress, uncompress and zcat utilities.

Work Files

In operation, tar may create temporary files with the names tar.tmp and tar.$$$ .

Tar Utility Restrictions

Be aware of the restrictions that the DP4 tar utility imposes when investigating its compatibilities with your Unix tarcommand:

  • The -r (append to end of tape) and -u (update tape) options are not supported
  • Genuine magnetic tape is not supported
  • Wildcard file names are expanded according to the MS-DOS convention; directory names are not followed recursively as occurs in the Unix version
  • Linked files are ignored
  • Permissions are ignored when restoring on MS-DOS/Windows
  • Files are stored with permission rw-rw-rw- or rwxrwxrwx and user and group id both set to 100. You can use -X to toggle x permission on and off.
  • In the UK the environment variable TZ must be set to GMT0BST in the winter or GMT1BST in the summer; otherwise the modification times on the tape correspond to the wrong time zone. In other countries set the TZ variable according to the number of hours you are in front of or behind Univeral Time (Greenwich Mean Time)
  • When extracting a tar file prepared on a Unix/Linux machine no attempt is made to deal with Unix filenames which are not valid filenames on MS-DOS Windows (e.g. con).
  • The -D and -Z options cannot be combined.

  • The -D option will destroy any non-text file.


DP4 Configuration Utilities

in this topic the "home" directory of DP4 is referred to from time to time. Up to release 4.523 this means the /usr/datafit/ directory. From release 4.525 it is the directory specified in the DP4HOME environment variable, or /opt/dp4 if DP4HOME is left undefined.

USERDATA

The DP4 utility USERDATA allows you to specify the terminal type, from the list it displays, by typing the corresponding letter.

This results in the automatic creation of userdata.sys in the current directory. At install time dp4install runs this program from the base directory of the DP4 installation. userdata.sys stores more than merely the terminal-type information, and its role in defining the DP4 configuration is discussed fully in the next section.

DFSETUP

After selecting the terminal type, the DP4 utility DFSETUP runs. Your DP4 licence file should have been pre-configured with sensible values so you can use it immediately, but you should check the various directory assignments match your requirements in any case.

DFSETUP allows you to configure various aspects of your DP4 installation such as the location of the different classes of DP4 file (there are some restrictions, see File_Location below), logging options, user preferences and printer set up.

The changes you make in DFSETUP are written to one of three locations:

  • userdata.sys in the current directory

  • the DP4 licence file, dp4.sys in the $DP4HOME directory (release 4.525 and on), or datafit.sys in /usr/datafit/ (release 4.523 and earlier)

  • system database in /usr/datafit/dbs/ (4.523) or the dbs subdirectory of $DP4HOME (4.525)

When DFSETUP is run from the command line and updates userdata.sys it will read userdata.sys from the "home" directory if one is not present in the current directory. DFSETUP always writes changes to userdata.sys in the current directory. You can therefore arrange for each user to have their own copy of userdata.sys by ensuring each user works in a different directory.

DFSETUP can be invoked from the command line or from the DP4 system menus (Option 1 - System configuration ). This affects where the changes are stored as shown by the following table. (The letter C indicates that DFSETUP was run from the command line, M indicates that it was run from the system menus).

DFSETUP Option | USERDATA.SYS | DP4 Licence file | SYSTEM
Directory Assignment |  | CM |
Console/Transaction logging |  | CM |
User Preferences | C |  | M
Keyboard Customisation | CM |  |
Printer Characteristics |  |  | CM
Serial Number |  | CM |
Terminal Characteristics | CM * |  |

* If you use the Terminal Characteristics option, the information is saved to terminal.sys in the current directory. Should you do this, rename terminal.sys to userdata.sys to use the new settings. The settings are not written to userdata.sys directly so that you can prepare a file for use on another machine.

Changing the Location of Files

Up to release 4.523 you can only change the location of certain DP4 files. Up to release 4.523 DP4 expects to find datafit.sys in /usr/datafit/. 16 bit programs converted to Unix with exe2unix expect to find the terminal manager (trm3) and database manager (srv3) in /usr/datafit/bin/.

From release 4.525 DP4 files can be installed in any directory. The only restriction is that the DP4HOME environment variable must point to the directory containing the DP4 licence file (dp4.sys) and the DP4 configuration file (dp4.ini)

The location of databases can be changed by following this procedure:

  1. Create a backup copy of the DP4 licence file (e.g. by using cp to copy it to a backup directory)
  2. If necessary create the new directory where databases are to be stored, and copy the system database to the new location.
  3. Run dfsetup and choose the Directory Assigment option. Specify the new location for the databases. DFSETUP will warn you if the system database cannot be found in the new location (which may happen if you have mistyped the new location, or failed to copy the system database, or there is a permission problem).
  4. Exit from all DP4 programs and make sure srv3 is no longer running, for example by running shutdown −stop. Check that you can still run dfsetup correctly to verify that the system database is correctly accessed from the new location.
  5. Move all the other databases to the new location.

It is possible, but not recommended to store the system database separately from other databases doing the following:

  1. Move the system database to its new location

  2. Run the DP4 utility, sysdb , giving the new location and name of the system database, as in:
    sysdb /newdirectory/system