Preface

Intended Readership

The Guide to DP4 Networking and Resilience is intended for technical readers who wish to set up DP4 systems that will access DP4 databases over a network. It contains information both about setting up networking configurations, and aspects of the DP4 programming that are different in a resilient environment.

This manual focuses on our preferred Network platforms: TCP/IP protocol and Windows or Unix/Linux computers. Other protocols and platforms are covered, but fairly briefly. Contact us if you would like more detail to be included for other specific platforms.

Terminology

Roughly speaking:

  • network manager, and server are synonymous. The former term applies to the specific software component that enables a machine to act as a 'server' to other machines for the DP4 databases on that machine.

  • Similarly network requester and client are synonymous. The former term applies to the specific software component that enables a machine to act as a 'client' accessing data from DP4 databases on other machines.

  • Machines used by real users to interact with the system, (usually those on which the network client is used), are sometimes referred to as terminals.

  • Resilience

    In the context of DP4 networking resilience is a term used to to describe the behaviour of the system in the event of a broken network connection. DP4 systems achieve resilience when DP4 databases reside on more than one machine. Using a resilience configuration allows a system to cope with network failures, and to spread work between a number of different servers.

  • DP4 Configuration File

    Frequent mention is made of the DP4 configuration file. Up to and including releases 4.523 and 4.619 this file was only used on Windows, is called datafit.ini and must be stored in the "Windows" directory. From release 4.524/4.620 onwards the file is called dp4.ini, is used on all platforms, and is usually stored in the directory where DP4 executables are installed. The DP4 configuration file is described in detail in the Guide to DP4 Configuration (DFIni.chm).


What's New in the DP4 Network Resilence Developer's Guide

What's New in DP4 Network Resilence Developer's Guide

This page will tell you about recent changes to this manual. The most recent changes are mentioned first.

New in 4.622

  • 5 Oct 2004 -prefer_vital command tail described.

  • 5 Oct 2004 New database open modes have been added, allowing the default network layout specified via AUXDISTR command tails or persistent network layout records to be over-ridden. See Database Open Modes Affecting Network Layout for details.

New in 4.621

  • 24 Mar 2004 Updated with information about 4.525 Linux/Unix DP4.
    -local_port command tail described.

  • 22 Sep 2003 new −debug_connect option described.

  • 24 Jul 2003 The material in this guide has been substantially revised and rearranged for 4.621.

  • 20 May 2003 TCPW is now multi-threading. This gives much improved performance in the "named server" configuration where more than one program is being run, and more than one server is in use. TCPW also bypasses TCP/IP completely when it detects that the TCPMGR to be called is on the same machine, and calls the database manager for TPCMGR directly (it automatically discovers which IPC channel TCPMGR is using and forwards messages to it directly). Multi-threading can be controlled using the new -threads n command tail.

  • 29 Apr 2003 TCPMGR automatically accepts connections on any available network adapter. In particular it can connect on dial-up connections created, or DHCP IP addresses assigned, after TCPMGR was started.

  • 29 Apr 2003 TCPW returns instantly if the port for DP4 is not active on the server (indicating that the server is offline). This won't work for machines with a permanent IP address that are turned off. You can use the new connect_timeout setting to control how long DP4 waits when attempting to connect to a server

New in 4.620

  • 23 Apr 2003 Named connection Configuration recommended order for loading programs corrected (previous instructions failed when DP4 loaded as application rather than service).

  • 11 Mar 2003 Operation of Table of Contents improved

  • 30 Oct 2002 Named connection Configuration described, including new "Reverse Connection" facility

  • 17 Oct 2002 What's New Page Added, and a few new links added between pages.

  • 23 Apr 2002 Information about Security Command tails for Network Manager added.

  • 22 Feb 2002 Information about DP4 Terminal Server added.

  • 12 Oct 2001 A few notes added about operation over the Internet or other WAN.

New in 4.619

  • 11 April 2001 Minor additions to trouble shooting section


Overview

DP4 networking allows DP4 programs running on one machine to transparently access a DP4 database located on another machine by transferring calls to the database manager using network messages.

A number of different programs are used with DP4 networking. An alphabetical list of the various components and a summary of their use is contained in DP4 Networking Components. This topic introduces you to the most important ones.

Basic Components

There are two basic components, which usually have a different name for each operating system/network protocol combination.

  • The DP4 network requester or client is the principal client side component. To DP4 applications the DP4 network requester appears to be exactly like the regular DP4 database manager. The network requester routes calls to the DP4 database manager over the network to the second component:

  • The DP4 network manager receives calls from DP4 network requesters and passes them to the DP4 database manager. This is the principal server side component. To the database manager the network manager appears to be an ordinary DP4 application program.

A third component, the network router (AUXDISTR) is used in some Resilience configurations. The network router is used to select one or more servers to which to pass requests from applications. It can also manage distributed transactions - single logical transactions that update databases on more than one server.

Given an appropriate configuration of these components, a client program can log onto a database manager either on its own machine or on any other server machine on the network. Any machine may act as either a client or a server, or both a client and a server, with capabilities dependent on what DP4 networking components are loaded.

The selection of which server(s) an application uses can be made implicitly - as determined by the DP4 components loaded and the configuration in which they are loaded, and, (when there is one), configuration information on the local database. The various configurations are described in DP4 Networking Configurations.

Alternatively, the selection for a particular access can be made explicitly by the application program. A simple example of a program with explicit selection would be a program designed to move data, for instance price changes, from a remote database to the local database on a terminal. The program would use the Network API described in Information for Programmers.

Thin Client Components

Rather than running a traditional client-server configuration it is possible to run in a "thin client" configuration where DP4 programs are run on a server machine but use additional DP4 Terminal Server components to interact with the user on a client machine.

There are two additional client-side components for running DP4 terminal server (which is only supported on TCP/IP protocol):

  • TSTCPx is the client-side terminal server which manages the user interface in accordance with instructions sent to it from the remotely running application.
  • TSRUN. This is a small program used to launch programs on the server from the client.

There are no special server-side components required for DP4 Terminal Server, which is built into DP4 versions 4.523/4.619 or later. Machines used as servers for DP4 Thin Clients, are sometimes referred to as Application Servers.


Alternatives to DP4 networking

Before you decide to deploy a solution using DP4 networking you should consider the alternatives. Typically these involve using a "remote display protocol" which allows applications to run on a server, but display on a client machine elsewhere. For example on Unix/Linux machines you are likely to achieve improved performance by using a Telnet connection to run Unix programs rather than DP4 networking. Another major benefit of this type of solution is reduced cost of deployment and administration.

Telnet is very simple to set up, though you need to pick an implementation of Telnet that correctly supports one of the supported terminal types in userdata.sys. You can also use Telnet to run programs on Unix or Windows NT servers from Windows clients. In the latter case you have to use version 4.52x of DP4 and use a different version of the terminal manager.

Telnet can also be used to run DP4 programs on Windows NT and 2000 servers, but only using text mode.

Telnet is not a secure protocol, so should not be used over the Internet unless a VPN is available. Telnet is also unsuitable if you require resilience - if a Telnet session breaks the programs running in it are normally aborted.

DP4 also works well with Citrix WinFrame, NT Terminal Server Edition, or Windows 2000 Terminal Services. The Windows DP4 Installation guide contains information on how to install DP4 for these environments.

From release 4.619 DP4 has its own version of such a protocol using the DP4 Terminal Server.


Prerequisites for DP4 networking

  1. Before setting up DP4 networking, at least in a development environment, you should first ensure that stand-alone DP4 is installed and working on both server and client machines. (However if you are installing the DP4 Terminal Client it is preferable NOT to have DP4 installed). The appropriate DP4 installation guide should help you with installing DP4. Once you have everything working you can remove databases on the client machines unless you are using some form of resilience configuration. On Windows NT family operating systems you will need to be an administrator in order to set up DP4 and DP4 networking, as you will want to be able to configure the computer and start and stop services.

  2. Make sure the required networking components of the operating system are correctly set up. This is usually very easy for Win32 or Linux machines. It can be very difficult on Unix and other platforms.

  3. DP4 TCP/IP network clients find the DP4 server by name. In order to be able to successfully connect to the DP4 server you must be able to 'ping' the server by the same name that you intend to specify for DP4 networking. On a LAN it is very unusual for there to be problems connecting via DP4 once you have got the network to a state where ping works. If you are operating over the Internet or a private WAN there are some additional considerations:

    • It is possible that you may be able to contact the server by name even when ping does not work - for example a Unix server might run inetd without a ping daemon, or ping may be stopped by a firewall. The Trace Route (tracert) utility may be helpful in diagnosing connection problems.

    • If you are going through a proxy server, firewall, or router, then you will probably need to "open the port" that DP4 will use (usually 5000). Consult the documentation for the relevant software/hardware to find out how to do this.

  4. Even if you are planning to use a resilience configuration, then it is a good idea to initially set up machines to use either server only or client only networking first. If you set up multiple resilience immediately you may end up with an apparently working network when in fact all you have is a local connection.


Cross Platform Networking

Networking across Operating Systems

As you might expect, it is perfectly possible for a request from a DP4 program running on a Windows based Client to be handled by a Unix or Linux server (even one running on a different type of processor). In fact any DP4 network manager can handle requests from any DP4 network client as long as they are from compatible releases of DP4, and the network protocols are compatible.

In release 4.520/4.616 DP4 networking was substantially reworked and for TCP/IP only is not compatible with earlier releases. It is not possible to access a 4.520 tcpmgr from a pre 4.520 TCPx, or vice versa. Attempting to do so will cause a DP4 system error, but should not cause problems other than to the affected client.

There are special considerations when setting up an environment where Windows clients running DP4 4.6xx access a server running 4.5xx DP4 on Linux or Unix. If you are using the client-only configuration, then your Windows programs will access the system database on the Unix server. The 4.5xx system database is not compatible with 4.6xx. Therefore when setting up the server you must install the 4.6xx System database from the Windows installation onto the Unix server. 4.5xx programs Unix can use the 4.6xx system database with no problems, though the display may look a little odd in a few programs.

Networking across Protocols

You cannot pass requests from a requester running one protocol into a network manager running on another protocol, (and for NetBIOS the underlying transport must also be the same as is explained in the NetBIOS Networking notes). However, it is possible, though rather unusual, for a network manager to pass its requests into a second network requester running another protocol.

If you want to do this, then load the machine as you would a normal server, replacing the command that loads the database manager with one that loads a network requester instead. See Using the Network Manager as a Bridge for further details.

In principle you can also run two network managers for different protocols on the same server, one handling requests for one protocol, and one for another. However, this is unsafe if multiple resilience clients are used - in this case you should use a Bridge Configuration. For ordinary servers, if you want to load servers for more than one protocol simply duplicate the lines that load the network manager for one protocol, changing the name of the network manager, and any network specific command tails as required.


Overview of DP4 Networking Configurations

There are numerous ways in which a machine may use DP4 networking to act as either a server to or a client of other machines (possibly both at once). This topic introduces the various possible configurations and the names used for them.

You will see in the list below, that several of the possible configurations use "Resilience" in their names. In some ways this is misleading as you may want to use a particular configuration for reasons other than resilience. In addition in some cases more than one DP4 networking configuration can be used to implement the same Resilience Architecture.

The wide range of possible configurations, may seem rather intimidating at first, particularly if you start to worry about which permutations of machines at either end of a connection are compatible. In general you can assume that any configuration that has network client capabilities will be able to access databases on machines with any configuration that has server capabilities. However there are a few restrictions and limitations. These are summarised at the start of the sections dedicated to each configuration.

You may find How to Choose an Appropriate DP4 Networking Configuration useful in deciding which of the various possibilities to use on a given machine.

The DP4 Networking Configurations

These are the most commonly used configurations of DP4 networking.

  • Server Only Machines (Simple Servers)

    A server only machine runs one or more DP4 network managers to allow client machines to access the DP4 databases stored on it. The clients may use any of the client architectures described below. Simple servers may also run DP4 applications. The network manager program and the applications on these machines all access the local databases - there is no access to databases on other servers. Instructions for loading and using DP4 as a Server machine can be found here.

  • Client Only Machines (Simple Clients)

    A Client Only machine runs a DP4 network client and always accesses DP4 databases across the network. There is no "local" database on the clients (and so no fallback if the network is down). This configuration is easy to set up, and is suitable for environments where the network is reliable and fast, and downtime is permissible.

  • Local Database Reslience Client

    In a local resilience configuration the DP4 database is sometimes accessed across the network and sometimes locally, either for performance reasons, or because the network is down. The DP4 networking software and the application cooperate in handling network problems, and ensuring local and remote databases are kept in synchronisation. There are several different types of local resilience configuration which differ in how network errors are handled and where the database is updated.

  • Multiple Resilience Clients

    In a Multiple Resilience Client configuration the DP4 database is accessed across a range of machines, usually but not necessarily including the local machine, and usually more than one server machine. Multiple resilience uses the AUXDISTR router ADC as well as the regular network requester. Using AUXDISTR imposes some restrictions on a DP4 system, so it is necessary to design a system with AUXDISTR in mind if it is planned to use it.

  • Multiple Resilience Combined Client/Server

    This type of configuration is similar to the last, but in addition machines are capable of acting as a DP4 database server for other machines on the network.

    In a typical Multiple Resilience Combined Client/Server configuration terminals mainly access and update a local database. Information that must be synchronised across the whole system may be accessed from a "central" server. (If this central server is not available, some aspects of the system may have to be disabled). The central server uses the server capability of the terminals to collect transaction information.

  • Named Connection Configuration

    In a named connection configuration machines are also capable of acting both as clients and servers. Applications have complete control over where databases (and remote files) are accessed, but whereas with the other resilience configurations the database may appear to be distributed because the DP4 interface hides the details of exactly how the various machines are contacted and kept synchronised, with this configuration the application takes control of this, by opening a number of connections to specific machines and databases.

  • DP4 Terminal Server

    The final scheme is a DP4 Terminal Server configuration. This scheme offers potential performance benefits and radically simplifies deployment of systems, the value of which may more than offset the loss of resilience inherent in such a system. Compared with using Telnet or a product such as Terminal Services for Windows 2000 it is more flexible - for example a program running on the server can make use of POS devices on the client via the OSDI, and a program running on a Unix server will work exactly like a native Windows program on a Windows client (and vice versa). Background information on DP4 Terminal Server can be found here. One way of achieving a measure of resilience in this type of solution is to use multiple application servers.


How to Choose an Appropriate Networking Configuration

  1. Before deciding which DP4 networking configuration to use, you should consider whether or not you should really be using it at all. Refer to Alternatives to DP4 Networking for some possible alternatives.

  2. Once you have decided you do want to use DP4 networking the next step is to decide on a Resilience Architecture (assuming you need one.)

  3. The final stage in deciding the appropriate configuration for a particular machine is to analyse where databases will be accessed. The table below summarises the possible scenarios and your available options for each of them.

Does machine have DP4 databases? | Does machine use DP4 databases elsewhere (including indirectly with DP4 Thin Client) ? | Do other machines use DP4 databases on this machine or use this machine as a DP4 Terminal Server? | Suitable Configurations
No | Yes | not applicable | (Simple) Client Only Networking Multiple Resilience Client with −nolocal
Yes | No | No | Standalone DP4
 | No | Yes | (Simple) Server Only Networking
 | Yes | No | Local Database Resilience Client Multiple Resilience Client Named Connection Configuration
 | Yes | Yes | Multiple Resilience combined Client/Server Named Connection Configuration

The table above is sufficient to choose the correct configuration unless this machine acts as a client of other machines. In the latter case your choice of which configuration to use may be limited by two other factors:

  1. Are you use a network protocol other than TCP/IP, or running on a Legacy operating system?

    If the answer to either of these questions is Yes then you cannot use the Named Connection configuration.

  2. Does this machine require access to databases on more than one other machine?

    If the answer to this question is Yes, and your answer to question 1 was also yes, you must use a Multiple Resilience configuration.

    If the answer to this question is No, then there is probably no point in using either Named Connection or Multiple Resilience configurations, unless the answer to this question is Yes for some other machine that is part of the system, and you wish to use the same configuration everywhere so far as is possible.

In the case where you are using TCP/IP protocol and do require access to multiple machines you may still have more than one possible configuration. The following should help you decide which to use:

  • If you need support for distributed transactions, where a single transaction updates the database on more than one machine at once, or if a single instance of a program accesses data from more than than one remote server and you want to minimise the amount of awareness the application has of this, then you should use a multiple resilience configuration.

  • On machines where you would like to be able to use the DP4 Thin Client to run applications against remote servers you must not use Multiple Resilience configurations. (However, a Multiple Resilience Combined Client/Server can act as an application server for DP4 Thin Clients.)

If you are still unsure of the most appropriate configuration then please refer to the Comparison of DP4 networking Configurations.


Multiple Server Resilience Architectures

Resilience Architectures

Designing Network Resilience into a system can be done in several ways, and in some cases the same Resilience Architecture can be implemented using more than one DP4 networking configuration. The following outlines the various possibilities:

  • Mirroring Architecture

    If you can keep an up to date copy of your database on a backup server you can provide resilience by switching to the backup server in the event the live server fails.

    One way of maintaining a database mirror, is to periodically run the DYNABACK program to back up a live database, or its log file. The whole database, or the log file can be transferred to the backup server, which is only brought into use when the main server fails for some reason. This type of configuration does not require any form of DP4 resilience so does not introduce complications (such as the Additional Commit Failures).

    However some care will be needed to design a mechanism that means transferring operation to the backup server is relatively painless. Also in the event of an unplanned failure of the main machine the last transaction log will be inaccessible unless the disk it was written to can be swapped into the other machine very quickly.

  • Local Resilience Architecture

    In this architecture applications use both a local database and a connection to a central database. Typically static and infrequently changing information is stored locally. Sometimes the local database only contains a subset of information to be found on the central database - such a scheme would be reasonable for applications such as book or CD retailing where the possible range of items is very large. Transaction information is normally posted to the central database. If the network is unavailable transaction information is stored locally, and depending on the application, some functionality may be lost.

    Programs may need to be specially written to update the local database from information stored on the central database, and to update the central database with transactions created when the system was off-line. However the −distributed or −synchronise options of the RECASTDB utility may be able to do this for you.

    Typically this archictecture is implemented using Local Database Resilience Client. However it can equally well be implemented using Multiple Resilience, though there are probably few reasons to do so.

  • Duplex Resilience Archictecture

    In a configuration of this type, two or more servers attempt to mirror each other. Database updates from clients are sent to all servers, and the terminal waits for an acknowledgment from all. If a server becomes inaccessible, the terminal simply continues with the remaining servers. It attempts to re-establish duplex operation when an active server detects that another previously off-line server is once again accessible, and notifies the terminal.

    The system will normally incorporate a user written program to reconcile the databases after a broken network connection is restored. A generic utility ONLINE, is available to do this automatically (see Automatic Reconcialiation for Duplex Resilience), but in the constraints this imposes on system design mean that using ONLINE may be impractical.

    This archictecture requires Multiple Resilience.

  • Collector Architectures

    In a Collector architecture terminals, usually configured using Multiple Resilience as Combined Client/Servers, mainly access and update a local database. Information that must be synchronised across the whole system (such as system wide unique numbers) may be accessed from a "central" server - if this central server is not available, some aspects of the system may have to be disabled. The terminals do not normally update the database on the central server (because this could cause contention). Instead the central server runs a number of programs known as collectors or pollers, which continuously poll each terminal in turn to discover if there are new transactions which require corresponding updates on the central server. Ihe "central" server is therefore also configured as a combined client server.

    Collector applications must be carefully designed so that they can collect relevant information once and once only. However this is usually fairly simple. The beauty of this type of architecture is that it does not matter very much if the collector falls behind in busy periods, or if a workstation loses its connection to the server for a while. The collector will 'catch up' in both cases sooner or later. Of course you need to bear in mind considerations such as the maximum permissible time lag for collecting transactions - a collector architecture would probably not be appropriate for doing things like maintaining bank balances, where it is important to update the central database in real time.

  • Pusher and Push Collector Architecture

    While the collector architecture works well on small LANs, the overhead of running such collector programs over a WAN may mean it is better to update the central databases directly from the terminals using a pusher or push collector, as this both reduces the load on the central server and allows more asynchronous processing. A pusher is a program similar to a collector running on a central server, but each terminal takes responsibility for pushing its own transactions rather than waiting for them to be collected.

    Pusher programs may need to deal with the possibility of contention errors, since several terminals may be pushing transactions at the same time. One way of eliminating contentions is to keep the pusher program itself very simple, and to keep most of the collector logic on the central server. The collector no longer contacts terminals itself to process new transactions but waits for them to arrive on the central server.

With a collector or push architecture, you may not need to use Multiple Resilience at all. The Named Connection configuration is particularly well suited to the push architecture. Indeed it may well offer enhanced performance over multiple resilience, because the push programs running on the terminals will slow down applications running locally if you use a multiple resilience configuration, but not at all if you use the Named Connection Configuration.

Whichever architecture is adopted, most transfer of transaction data from terminals to central servers is best handled outside the regular client applications, as this reduces the dependence of the client application on the central server.


Design Constraints with Duplex Resilience Architecture

There are unavoidable limitations in using multiple server resilience to duplex (mirror) databases between different servers due to the possibility that servers can be updated in isolation by different workstations. The servers subsequently need to be reconciled. For example it might happen that the connection between workstation 1 and server 1 is intact, and between workstation 2 and server 2, but other connections are broken.

In the original design for DP4 resilience the duplex configuration was always used and reconciliation was carried out automatically by the ONLINE program.

The use of this program imposes a lot of additional constraints on database design. ONLINE is no longer often used, as it can prove to be unsatisfactory because of the subtle errors that failing to adhere to these constraints can introduce into a system.

If you want to use a duplex resilience architecture (with or without ONLINE) you need to bear the following in mind:

  • Records to be insertions must have a guaranteed unique primary key, so that there is no possibility of, say, two different transactions on the two servers having the same key. The conventional way to achieve this is to include the workstation identification in the key of the records to be inserted.

  • Amendments should not be made that depend on the prior state of the record. For example, an application that attempts to maintain a real-time stock balance is unsafe. If the servers are disconnected, and two workstations attempt to sell one item from the different servers, the stock balances on each will be decremented by one, a simple reconciliation program such as ONLINE will take no action since the records are the same. The remedy for this amendment problem is to have a background program on the servers which carries out the amendments on its own local database using the inserted transaction records.

  • It is not possible to reliably generate system wide unique numbers, for instance invoice numbers or order numbers, In particular, autoincrement numbers must not be used.


Comparison of DP4 Networking Configurations

When looking at the table below please remember that Named Connections are only available on TCP/IP protocol, and are not available on legacy operating systems.

Configuration(s) Capabilities Benefits Limitations
Simple Client Only Networking

Named Connection configuration

Each connection is limited to databases on one server, but where available C/C++ programs can create multiple connections to transfer data between servers.

The Resilience API is not used in these configurations.

Applications are responsible for handling of, and recovery from, network errors.

There are no significant differences in the operation of DP4 compared with standalone operation (other than unavoidable performance degradation), so no changes are required to applications originally designed for stand-alone operation unless access to multiple servers is now required.

On Windows platforms connections to different machines are completely independent of one another, so a slow connection to one server will not slow down programs running against other servers.

Named connections (also available to client only machines) allow a client to connect to any machine running as a DP4 server anywhere accessible to the machine via TCP/IP.

Named Connection configuration supports "reverse" connection allowing most efficient scheme for two way transfer of data between machines.

 Connecting to multiple servers is not transparent to application - so any kind of cross server activity requires explicit code in an application.

Currently No support for distributed transactions
Local Database Resilience Client

Each connection is limited to databases on the local machine and one other server, but where available C/C++ programs can create multiple connections to transfer data between servers, as for Simple Clent Only and Named Connection configurations.

Network error handling is determined by the Resilience command tail supplied to the database manager, unless applications take responsibility for handling and recovering from network errors.

Tables can be statically configure to be local or remote.

Applications originally designed for stand-alone operation do not usually require modification. Using local/remote facility allows even some cross server activity to be transparent to application.

Supports transactions distributed betweeen local and remote database.

Databases on client machines are not accessible to other machines

Connections to different machines are NOT completely independent of one another, so a slow connection to one server will slow down programs running against another server.

Limited API support for controlling whether any particular table is accessed locally or remotely - so writing programs to synchronise data may be more complicated than with other configurations.
Multiple Resilience Configurations

A single connection can access databases on up to 99 machines, and update any or all of them in a single transaction.

Programs can use the DP4 Resilience APIs to select which machines a particular database access uses.

 Offers most sophisticated scheme for controlling cross-server activity transparent to applications.

Supports Distributed transactions.

Programs can only access machines in a pre-determined list of 99 servers.

Connections to different machines are NOT completely independent of one another, so a slow connection to one server will slow down programs running against another server.

Multiple Resilience causes changes to the operation of DP4 which may require you to make significant changes to applications and databases designed for stand-alone operation.

Incompatible with DP4 Thin Client


(Simple) Client Only Networking

(Simple) Client only machines use a DP4 network requester program to communicate with a single, usually fixed, server. A DP4 network requester replaces the regular DP4 database manager on these machines. All database accesses by application programs on the client machines are handled by the specific server machine with which they are in communication; there are no databases on the client machine.

Obviously, at least one machine somewhere must be configured to act as a server. You can use any of the possible server configurations as a server for somple network clients. Refer to How To Choose an Appropriate Networking Configuration for a comparison of the various possibilities. The simplest server configuration, the Server Only configuration is described here.

Instructions for loading a machine as a simple client only machine can be found here.

Limitations

Simple client only configurations have the following limitations compared with resilient configurations:

  • If a server or the link to it fails, the clients using that server will terminate (unless they contain their own specially written code to handle network errors).

  • Different client machines can access different servers, but each client can access only one server (unless you use the −server_name command tail or your application uses named connection facilities as described in The -server_name Command Tail for Applications).

Programs such as DBCHECK, DBBACKUP and DBRESTOR, when run on a client, work on the remote database. It is strongly recommended that such programs be run only on server machines. Running these programs over a network will cause a large increase in the amount of data transfer across the network and adversely affect performance of the system.


Local Database Resilience Clients

In a configuration of this type the applications have access to a local copy of the database, in addition to the database on the server. The database manager initially handles messages sent by the application program, and decides whether to process them itself or to pass them on to the network client.

In a typical configuration of this type static or infrequently changing information is held on the local database, but transation data is normally written to the remote database. In the event of a network problem the application seamlessly switches to storing transaction data locally. The system would incorporate programs to transfer such locally held transaction data up to the remote server, and to download updates to the infrquently changing data (for example price updates).

Each client accesses certain tables on its local database and certain other tables on the database on one of the server machines. By default, all the base dictionary tables are local, and all the application tables are remote; this default can be changed using makedb -expert. Updates can be made to both sets of tables. The system database is always accessed entirely locally.

Obviously, at least one machine somewhere must be configured to act as a server. You can use any of the possible server configurations described elsewhere as a server for local database resilience clients. Refer to How To Choose an Appropriate Networking Configuration for a comparison of the various possibilities. The simplest server configuration, the Server Only configuration is described here.

Instructions for loading a machine as a local database resilience client can be found here.

Limitations

  • In a local resilience configuration, different client machines can access different servers, but each client can access only one remote server. (However, you can use the Named Connection Facilities to make an application use a different server in this configuration, provided you set the default server by using the -server_name servername command tail on the network requester, and not by specifying -server n on the database manager.)

  • With the exception noted in Resilience Command Tails for the command tail -NOFAIL, parent and orphan checking and child deletion are carried out on the local database for records posted to the local database, and on the remote database for records posted to the remote database. No cross-database checking or child deletion is ever done.

  • It is not possible for other machines to access the local database on client machines. You must not attempt to run the network manager on machines running with the Local Resilience Client software. If access to the local database is required from other machines you must use either a Multiple Resilience Combined Client/Server or a Named Connection configuration.

  • There is no automatic provision for moving data from the local database to the remote when connection is reestablished. An application program must be written to do this.

  • From the point of view of the programmer, the local database resilience configuration is less controllable than a Multiple Resilience or Named Connection configuration. For example, a program cannot easily find out whether a table should be stored locally or remotely, nor change the error handling mode from the default.


Multiple Server Resilience

In these configurations each application program can access a local database and also multiple servers. Each database table is either processed locally or stored on one or more servers (possibly including the local machine); fetches on a table stored on multiple servers can be processed by any one of the servers and updates to such a table are broadcast to all the servers holding the table.

The network router AUXDISTR, is responsible for routing requests from applications to the appropriate locations, passing requests to either the the local database manager, or one or more remote servers (via the network requester) as appropriate.

Each table on a database is marked on the database as either local or remote. (This is done in the same way as for Local Database Resilience Clients, i.e. by using makedb -expert to set or clear the "local table only" flag for the table.) Operations on local tables are directed to the local database, and operations on remote databases are broadcast to a set of servers. The default set of servers to use for a 'remote' operation is determined by command tails, but can be over-ridden on a per-table basis by posting records to a network layout table.

Multiple Reslience Configurations divide into two main types, according to whether or not the machine also acts as a server for other machines on the network. For configurations without server capabilities there may or may not be a local database. Instructions for loading the different configurations can be found in How to Load Multiple Resilience Clients and How to Load Multiple Resilience Combined Client/Servers.

Limitations

  • When a multiple resilience configuration is in use machines are limited to talking to 99 pre-defined servers. The −server_name command tail is not available in these configurations.

  • You cannot use the client side of the DP4 Thin Client on a machine running a Multiple Resilience configuration. It is however possible for a machine running AUXDISTR to act as an application server for machines running the DP4 thin client.

  • There are some important changes and limitations to DP4 functionality when a multiple resilience configuration is loaded, which programmers and database designers need to be aware of. These are discussed in Implementation Details.

    Programs such as DBBACKUP and DBRESTOR, when run on a client, work on the local database and not on the remote database.


Named Connection configuration

A machine that is running in the named connection configuration is loaded as a network client that is capable of:

  • Acting as server, both for its own network client and any other client machine on the network
  • Acting as a client, both of its own server, and any other server on the network.
  • (Optional) Acting both as an application server for other machines using DP4 Thin Client, and as a DP4 Thin Client of other application servers.

Instructions for loading a machine in the Named Connection configuration can be found here.

This configuration has a combination of facilities not otherwise simultaneously available in DP4 networking

  • If you mainly want to run programs against one server, or locally, but would sometimes like to run against another server (this might be something as simple as wanting to run against a database held on a handheld device from your PC) this configuration allows you to do so easily without having to stop the DP4 service and load it again in another configuration. You can run a program against another machine simply by specifying the appropriate name with the −server_name servername command tail. Alternatively, applications can use the named connection facilities in DP4 to make their own connections to other machines.

  • It is possible, with some restrictions, for a server to "call back" a client that is loaded in this configuration, as explained in the next section. It is impossible to do this for a client loaded with the "local resilience" configuration. For details of the callback facility refer to Reverse Connection Facility in the configuration section.

Limitations

  • Currently, as, already mentioned, there is no support for distributed transactions. This is most significant limitation of the system compared with Multiple Resilience, and it will be removed in a future release of DP4. If your system does not require distributed transactions and is currently using AUXDISTR you should consider updating your system to use Named Server Configuration instead, as it will probably give improved performance.

  • The Named Connection configuration is only supported for TCP/IP protocol and only on Win32 and Unix like platforms.

  • Prior to 4.621 if you use the -rocopy or -nocopy command tails on TCPMGR you will also be prevented from running programs that use DP4 sf_() functions (this includes most of the database backup/restore/maintenance utilities) locally. This restriction is removed in 4.621, because local connections are rerouted to bypass TCPMGR.

Network Error Handling

There are no special facilities for error handling in this configuration. Resilient Programs that are specifically written for this configuration must take control of error handling rather than relying on the default sys_error() and sys_fail() behaviour of DP4. When a network error occurs an application must close the affected connection, but can try to reopen it afterwards if desired.


DP4 Terminal Server Clients

DP4 Terminal Server

The DP4 Terminal Server allows you to avoid installing more than than a very few DP4 components, (collectively named the DP4 Thin Client), on client machines. Rather than running various programs on the client which access a database via a network server, you run one user interface program which provides the user interface for applications running on the server. You can find background information on the DP4 Terminal Server in the description of the DP4 Thin Client in the Introduction to DP4.

The DP4 Thin Client is supplied as the minimal set of DP4 components needed on the client together with an install program. If you only want to run DP4 like this is all you need to install. You will probably want to customise the installation by pre-editing the DP4 configuration file with any settings you need. For Win32 clients the ideal way to do this is by adding entries to the tscw32.inf file, or running a second .inf file. If your application server is not called dp4server or accessible under that name, you at least need to change the entries in the .inf file which specify server names.

You may also want to add an OSDI to the installation, if you want to be able to access POS or similar devices attached to the client.

A DP4 Thin Client Client installation is not incompatible with a client only network installation, or a local resilience configuration, or a named connection configuration so you can run a mix of applications. For example you might use a local resilience configuration with a few key applications installed on the client, and arrange to run other applications via the Terminal Server. DP4 Thin Client is incompatible with a Multiple Resilience Configuration - if a machine is running AUXDISTR it will not be able to run programs on other machines using the DP4 Thin Client.

Instructions for loading machines so they can to act either as Application Servers, or DP4 thin clients can be found here.

Displaying Bitmaps and Icons

If both the application server and the client are running Win32 operating systems bitmaps and icons built into the application will be correctly displayed on the client. On Win32 clients bitmaps and icons stored on the server can also be displayed regardless of the server operating system. If filenames specified in maps include a path this must be relative to the server, not the client. In each case the necessary files are transferred across the network once and are held in a cache similar to that used by Web Browsers for web pages. If files are updated on the server the new version will automatically be used next time the file is accessed.


General Remarks about Loading DP4 Networking

Command Tails

The "How to Load..." sections cover only the most important command line options. For a complete listing of all the other options available with the DP4 networking programs refer to Command Tails in the Quick Reference section.

Windows platforms

It is often useful to have several alternate startup sections in the DP4 configuration file with different configurations of DP4 networking. The DP4 loader program accepts the command tail
−startup=section_name and processes the appropriate section, when you start the DP4 service.

On Windows and Windows CE platforms (apart from Windows 3.x) DP4 networking is run as part of the DP4 service loaded by srvw32.exe/srvwce.exe You configure which components as run are part of the DP4 service by editing the [startup] section of the DP4 configuration file as described here. Therefore, on Win32 platforms DP4 networking can be autoloaded by DP4 applications and you may not need to start it at boot time. However, on Windows NT family machines, DP4 will be typically run as an automatically started service, because ordinary users are not allowed to stop and start services, and because machines will not be able to act as servers when there is no user logged on otherwise.

Unix/Linux Platforms

Up to and including release 4.523 DP4 networking components must be manually loaded, as described for Other Platforms below. However, as of release 4.525, DP4 networking components may be run as part of the "DP4 service" controlled by dbdaemon, though you can continue to load components manually if required.

There are several advantages to using dbdaemon:

  • DP4 applications can auto-load any DP4 configuration, for example automatically running dbcheck or dbrecov as part of the startup process.
  • You can use the errlog program to obtain a single log file containing diagnostics from all components run under dbdaemon.
  • You can use dbdaemon -stop to cleanly unload all components of the DP4 service.

  • You can store alternate DP4 configurations in the DP4 configuration file, exactly as on Windows, and use the -startup=section_name command tail on dbdaemon in the same way as for srvw32 on Windows.

You can combine use dbdaemon and shell scripts to enhance its capabilities. For example dbdaemon will not expand environment variable names, but you can invoke a shell script that does. An example of this is discussed in Accessing Environtment Variables.

Other Platforms

Currently, on non-Windows platforms and Windows 3.x, DP4 networking components must be manually loaded before attempting to run a DP4 program, otherwise the DP4 program will try to auto-load the regular DP4 database manager and access databases locally. Typically you will write a batch file or shell script that loads your desired configuration, and arrange for this to be run when the machine is booted.

The command lines shown in this guide are suitable for inclusion in such scripts.


Unloading DP4 Networking

If not detached, the network manager can be terminated by closing its Window with Alt+F4 (on Windows), or by pressing the Esc key (on Unix). You cannot do this while there are active connections. On Win32 platforms stopping the network manager does not stop the DP4 service as a whole, so it will not interfere with applications running locally unless the machine is running a Multiple Resilience combined client server configuration or the Named Connection Configuration.

On Win32 platforms you can stop the DP4 service, which will terminate all DP4 network connections and shutdown all loaded DP4 components cleanly. This works for any configuration of DP4 networking.

You can also shutdown a network client by using the DP4 shutdown program. However the shutdown program requires to be able to make a connection in order to work, so shutdown won't work if you cannot make a connection at all. (So on MS-DOS network clients you may have to reboot in this situation.)

On Unix DP4 network programs can be terminated with the kill command. You should not use kill -9, or at any rate you should first try a normal kill and wait for the network timeout period (10 seconds by default) to elapse before issuing kill -9. When DP4 is killed with kill -9 you must also use ipcrm to remove the IPC queues, semaphores and shared memory created by DP4.


-server_name name Command Tail for Applications

From 4.523/4.619, except on legacy platforms, all standard DP4 utilities (or DP4 applications built with the DP4 C library from that release) also accept the -server_name servername command tail. If DP4 is loaded in either a client-only, a local resilience or a named connection configuration using TCP/IP protocol, this allows the program to run against the named server rather than the default server.

From version 4.621 servername may also be given in the form #ip_address - for example #192.168.160.12 . This is primarily intended for creating "reverse" connections in the named server DP4 configuration, and is not recommended for normal use.

Applications can also connect to a specific server using the srv_connect() function. See "Named Connection Configuration" for further details.

Notes

  • Don't assume because DP4 Network clients (and the DP4 database manager in Local Database Resilience configuration) accept both -server N command tails and -server_name name, as a means of selecting server, that you can do the same thing with applications - you can't.

  • In multiple resilience configurations (any configuration using AUXDISTR), the -server_name servername command tail has no effect.


Server Numbers

If more than one network manager program is being run in a network, the various network managers on the servers can (and must for NetBIOS and for multiple resilience configurations) be given numbers to distinguish them. These numbers are allocated in the range 0 to 9999 (or 65535 for TCP/IP only).

In Multiple Resilience configurations the number should be between 1 and 99. Server number 0 has special significance to the multiple resilience software and must be avoided if multiple resilience is in use.

With NetBIOS networking the server number is used to construct a name such as DP4_NETBIOS_000, and clients are able to find the appropriate server automatically.

For other network protocols some form of address mapping is usually required, as described in Mapping of Server Numbers to Network Addresses.

The information in the following section applies strictly only to TCP/IP. However a very similar facility to the [tcpip_servers] section described there is also implemented for IPX/SPX. Therefore it is worth reading this section for IPX/SPX networking as well. The IPX/SPX notes will explain the differences.

Mapping of server numbers to TCP/IP names

With DP4 TCP/IP networking the DP4 TCP/IP requester will generate a name for the server it tries to contact based on the server number as follows:

Server Number | Generated name | Comments
0 | dp4server | This number must not be used in a multiple resilience configuration
1-9 | dp4server01-dpserver09 | A leading zero is added to the number for servers 1 to 9 only, for compatibility with older releases.
10-99 | dp4server10-dp4server99 |
100-65535 | dp4server100-dp4server65535 | These numbers must not be used in a multiple resilience configurations. They are intended to be used in large networks where machines have a preallocated number that is available in the environment. Numbers in this range were not supported prior to release 4.520.

Mapping of Server Names to network addresses

Except on Windows platforms, the names used for servers must be resolvable by TCP/IP itself. You can either add entries to the hosts file, where this is in use, or you can use a name server (DNS) to add CNAME records for the names you need, which point to the real server names, or you can use DHCP and Dynamic DNS or, on Windows only, DHCP + WINS. DNS and DHCP make it much easier to manage a network, and you should use them if possible.

If you are using a hosts file it should contain entries of the form
192.168.100.1 ntworkstation dp4server09

This entry indicates that DP4 server 9 is running on a machine identified as ntworkstation having the IP-address 192.168.100.1.

  • Under Windows 9x family operating systems and Windows for Workgroups the hosts file is usually located in the "Windows" directory.
  • Under Windows NT family operating systems the hosts file is located in the %systemroot%/system32/drivers/etc directory.
  • On UNIX systems the hosts file is traditionally located in the /etc directory.

Specifying IP addresses directly

If absolutely necessary you can specify IP addresses for servers by specifying -address options on the command line for the tcp requester. For server number 0 (normally only used in simple client-server and local resiliency configurations), specify -address a.b.c.d . For other server numbers specify -address_n a.b.c.d , where n is the same as for the corresponding dp4serverN name. You are strongly urged not to use this method. It did not work properly in some earlier versions of DP4 networking.

Use of [TCPIP_SERVERS] section of the DP4 configuration file under Windows

On Windows (all versions) you can add a special section to the DP4 configuration file which maps these names to the real names that TCP/IP uses to reach the servers. You cannot use IP addresses in these entries. The syntax of this as follows:
[tcpip_servers]
dp4server=servername
dp4server01=servername01
..
..
dp4server09=servername09
dp4server10=servername10
..
..
dp4server99=servername99
dp4server100=servername100
..
dp4server65535=servername65535

When looking for server number NN, tcpw first constructs the dp4serverNN name in the normal way, and then looks for a translation of this name by looking for a corresponding entry in the [tcpip_servers] section of the DP4 configuration file. It is important that you number your servers consistently. For example, it is perfectly possible to accidentally configure different clients so that they use the same server number for different DP4 servers (for example one has dp4server01=fred, and the other has dp4server01=jim), or to use different server numbers for the same DP4 server. This may potentially lead to unforeseen problems, because the multiple resilience software passes the "online" status of the various servers around the network based on DP4 server numbers.


Server Numbers

Mapping of Server Numbers to Network Addresses

The information in the following sections applies strictly only to TCP/IP. However a very similar facility to the [tcpip_servers] section below is also implemented for IPX/SPX. Therefore it is worth reading this section for IPX/SPX networking as well. The IPX/SPX notes will explain the differences.

Remember, that except for multiple resilience configurations, you don't usually need to use server numbers at all with TCP/IP. You can use the -server_name command tail and real TCP/IP names.

Mapping of server numbers to TCP/IP names

With DP4 TCP/IP networking the DP4 TCP/IP requester will generate a name for the server it tries to contact based on the server number as follows:

Server Number | Generated name | Comments
0 | dp4server | This number must not be used in a multiple resilience configuration
1-9 | dp4server01-dpserver09 | A leading zero is added to the number for servers 1 to 9 only, for compatibility with older releases.
10-99 | dp4server10-dp4server99 |
100-65535 | dp4server100-dp4server65535 | These numbers must not be used in a multiple resilience configurations. They are intended to be used in large networks where machines have a preallocated number that is available in the environment. Numbers in this range were not supported prior to release 4.520.

Mapping of Server Names to Network Addresses

Except on Windows platforms, the names used for servers must be resolvable by TCP/IP itself. You can either add entries to the hosts file, where this is in use, or you can use a name server (DNS) to add CNAME records for the names you need, which point to the real server names, or you can use DHCP and Dynamic DNS or, on Windows only, DHCP + WINS. DNS and DHCP make it much easier to manage a network, and you should use them if possible.

If you are using a hosts file it should contain entries of the form
192.168.100.1 ntworkstation dp4server09

This entry indicates that DP4 server 9 is running on a machine identified as ntworkstation having the IP-address 192.168.100.1.

  • Under Windows 9x family operating systems and Windows for Workgroups the hosts file is usually located in the "Windows" directory.
  • Under Windows NT family operating systems the hosts file is located in the %systemroot%/system32/drivers/etc directory.
  • On UNIX systems the hosts file is traditionally located in the /etc directory.

Specifying IP addresses directly

If absolutely necessary you can specify IP addresses for servers by specifying -address options on the command line for the tcp requester. For server number 0 (normally only used in simple client-server and local resiliency configurations), specify -address a.b.c.d . For other server numbers specify -address_n a.b.c.d , where n is the same as for the corresponding dp4serverN name. You are strongly urged not to use this method. It did not work properly in some earlier versions of DP4 networking.

Use of [tcpip_servers] section of the DP4 configuration file under Windows

On Windows (all versions) you can add a special section to the DP4 configuration file which maps these names to the real names that TCP/IP uses to reach the servers. You cannot use IP addresses in these entries. The syntax of this as follows:

[tcpip_servers]
dp4server=servername
dp4server01=servername01
..
..
dp4server09=servername09
dp4server10=servername10
..
..
dp4server99=servername99
dp4server100=servername100
..
dp4server65535=servername65535

When looking for server number NN, tcpw first constructs the dp4serverNN name in the normal way, and then looks for a translation of this name by looking for a corresponding entry in the [tcpip_servers] section of the DP4 configuration file.

When multiple resilience is in use it is important that you number your servers consistently. For example, it is perfectly possible to accidentally configure different clients so that they use the same server number for different DP4 servers (for example one has dp4server01=fred, and the other has dp4server01=jim), or to use different server numbers for the same DP4 server. This may potentially lead to unforeseen problems, because the multiple resilience software passes the "online" status of the various servers around the network based on DP4 server numbers. (See Network Status Table with Multiple Resilience for details).


How to Load the DP4 Network Manager

How to Load the DP4 Network Server

  • On Win32 platforms DP4 networking components are loaded by modifying the [startup] section of the DP4 configuration file. To load the server you can use the following startup section:

    [startup]
    1=dp4srvr.w32
    2=tcpmgr.w32

  • On Window CE platforms the DP4 configuration file is also used and you can use the following startup section:

    [startup]
    1=dp4srvr.wce
    2=tcpmgr.wce

  • On Unix and Linux machines DP4 Networking components can be loaded from the shell, or in a shell script. The server can be loaded simply by entering tcpmgr at a shell prompt. Alternately, from release 4.525 you can use dbdaemon and the following startup section in the DP4 configuration file:

    [startup]
    1=srv3
    2=tcpmgr -display_address

When loaded like this TCPMGR will usually display information about connected clients. For more information about this display, and how to suppress it if it is not required, please refer to Network Manager Display.

If the server is accessed from machines running one or other of the Multiple Resilience Configurations, you should add the command tail -server n where n matches the server number used to access the server from client machines. It is also necessary to do this if you are using a network requester with the -server n command tail using any network protocol other than TCP/IP. Please refer to the section on server numbers for further details.

Using the Network Manager as a Bridge

It is possible to replace the local DP4 database manager usually used by the Network Manager with another program, so that the network manager does not actually use a local database. There are three main reasons you might want to do so:

  • You may wish to replace the DP4 database with another database manager, such as Oracle, but continue to use existing DP4 programs. In this case you would load the server as follows on a Windows server:

    [startup]
    1=dp4svr.w32 -aux
    2=auxoracl.w32
    3=tcpmgr.w32

    The same general format would be used for loading any other ADC on a server machine.

  • You may want to feed network requesters using one protocol, into a system using another protocol. For example you might want to allow old MS-DOS machines that only had NetBIOS protocol to access data from a system networked using TCP/IP. You would then load a machine as a network bridge to achieve this:

    [startup]
    1=tcpw.w32 -server_name servername
    2=ntbmgr.w32

    Here NTBMGR will receive calls from the MS-DOS machines, and make calls to the regular DP4 interface. These will then be passed to the "real" database server by tcpw.w32.

  • In another situation, very similar to the last, you may need to need a bridge where both sides use the same network protocol. The two most common reasons for doing this are:

    • You want to concentrate a number of clients, so that they all appear to be coming from the same location on the network (this may be helpful if there is limit on the available bandwidth or number of connections). This may allow you to run more clients simultaneously than would otherwise be possible.

    • You want to access data over a WAN link from terminals which only have LAN connectivity.


Network Manager Display

The network manager usually displays information about the clients that are currently connected to it. Either the name or network address of connected clients will be displayed, depending on whether the name is available or not. The number of users is also shown. This is the number of connected machines, not the number of DP4 programs using the server, which may be greater. If you wish you can disable this display by specifying the −detach option on the command line. If you are loading DP4 from a shell script on Unix or Linux you may also want to do this if tcpmgr is not the last line in the shell script. Alternatively you can put an & character at the end of the line that loads tcpmgr to force the shell script to continue.

Many DP4 users are in the habit of automatically specifying -detach on tcpmgr. In at least one case a customer requested an enhancement to be display exactly the same information as tcpmgr does display, unaware that the only reason he did not see the information was that he had specified -detach! We think this information, while it could be improved on, should be of use to network or system administrators. In older versions of DP4 networking this information was somewhat unreliable. In 4.520 and beyond it is trustworthy. When upgrading to 4.520/4.616 networking or later you must also upgrade your system database as the maps used are different from earlier releases.

On Windows NT and Windows 2000 you will only be able to see the output from tcpmgr if you enable the option to allow the DP4 service to interact with the desktop. "Allow service to interact with desktop" is one of the properties that can be set in the Startup Properties Properties for the service in the "Services" Control Panel Applet. From release 4.620 this property is set automatically if you use srvw32 −install to create the DP4 service. With earlier releases you need to use Control Panel to change the DP4 service configuration.

If you wish you can get the display to timeout after a specified timeout by specifying the −scrnsave option with the desired timeout in seconds. On Windows you will normally rely on the Windows screen saver. You should use a simple screen saver on a machine that acts as a DP4 server, such as the NT Logon screen saver, as some screen savers may impact performance.

Network Manager Display on Combined Client/Server Machines

If a machine is loaded as a combined client/server using either the Named Connection configuration or the Multiple Resilience Combined Client/Server, then for technical reasons, the normal display is suppressed.

On Windows platforms, in these configurations, the network manager will automatically behave as though −detach was specified, and will send diagnostic and other output to the DP4 Error Log (if it is running). On other platforms the network manager will usually default to its -quiet display mode. This simply displays messages teletype fashion as clients log on and off. If enabled, diagnostic messages will also appear in this output.

Network Manager Display under dbdaemon on Unix/Linux

If tcpmgr is loaded under dbdaemon on Unix or Linux it automatically behaves as though -detach was specified. When tcpmgr is run like this it does not have a terminal and cannot be run interactively. However you can still display connection information from tcpmgr: put the command tail -display_address as the first command tail to tcpmgr. On Linux and Unix, tcpmgr creates a separate tcpmgr process for each connected machine. If you supply this command tail each tcpmgr process will replace it with the ip address it is acting a server for. This will normally show up when you run the ps program to display running processes (you may need to supply a command tail to get ps to show tcpmgr and other proceses running as daemons, probably -d on Unix and x on Linux). This information will also allow you to disconnect a specific machine without shutting down DP4 completely: just use kill on the connection you want to terminate.


How to Load a Client Only Configuration

How to Load Client Only Configuration

The easiest way to load the TCP/IP client software is as follows:

  • Windows

    On Windows clients use the following [startup] section in the DP4 configuration file.

    [startup]
    1=tcpw.w32 -server_name servername

  • Windows CE

    On Windows CE clients use the following [startup] section in dp4.ini.

    [startup]
    1=tcpw.wce -server_name servername

  • Unix/Linux

    On Unix and Linux use the following command in a shell or a shell script:
    tcp3 -remain -server_name servername

    If you are using 4.525 and dbdaemon, use the following command in the [startup] section of dp4.ini
    1=tcp3 -server_name servername
    The -remain command tail is not used with release 4.525, whether or not dbdaemon is used to load DP4.

  • MS-DOS

    On MS-DOS use the following command at the prompt or in a batch file:
    tcp2 -remain

In each case servername should be replaced by the name you use to ping the machine where tcpmgr and the DP4 server will run. From version 4.621 servername may also be given as #<ip_address> e.g. #192.168.160.12 . This is primarily intended for creating "reverse" connections in the named server DP4 configuration, and is not recommended for normal use. The -server_name command tail is not available on MS-DOS.

The commands given above cause the DP4 requester to remain loaded permanently. It is possible to load the requester so that it unloads after the last DP4 program has terminated (the requester will remain loaded in any case until at least one DP4 program has been run). To do this omit the -remain command tail on MS-DOS and Unix/Linux (4.523). On Windows (or Windows CE), or 4.525 Linux/Unix add -noremain to the [startup] line for the tcpw/tcp3

Accessing Environment Variables

Batch files, or the Linux/Unix shell (though not dbdaemon), will automatically replace variables such as %server_name or $servername (as appropriate to the platform) with the value of the variable as defined in the environment. This can be a very convenient method of avoiding having to generate separate configuration scripts for each site. Although the Win32 tcpw is not loaded from a script, the tcpw program itself will use the environment to replace a servername beginning with a % sign with the value of the corresponding environment variable. (Don't put a trailing % sign at the end of the name in this case.)

This is not the case for Unix/Linux programs loaded by dbdaemon. If you want to use dbdaemon, and take advantage of environment variable options, make dbdaemon invoke a shell script rather than a DP4 command directly. For example:
[startup]
1=sh dp4start

dp4start might contain the following command:
tcp3 -server_name $DP4SERVER

If tcp3, or any other DP4 component, is loaded like this, it will run exactly as though it had been started directly from dbdaemon, and dbdaemon -stop will still unload it. Rather than invoking sh explicitly you may prefer to make your script executable using chmod +x. If you do this the line 1=dp4start will work perfectly well.

You may also be able to configure your DNS server to translate one name to another by adding a CNAME, or alias, record to its database.

If you do not specify the -server_name option the tcp requester will look for a server called dp4server.

Using Server Numbers

The -server_name option was introduced in version 4.520 and is only available with TCP/IP networking. It is not available for other protocols or with earlier releases of DP4 TCP/IP networking.

All DP4 networking programs (both clients and managers) accept a command tail of the form -server N where N is a number. In a way that varies depending on the protocol and platform, network requesters will locate the correct network manager to use by discovering the network manager with the desired server number. See the section on server numbers for more information.


How to Load a Local Database Resilience Client

Client Machines

Client machines are loaded as follows:

  • On Win32 platforms:
    [startup]
    1=tcpw.w32 -distributed -server_name servername
    2=dp4srvr.w32 -distributed

  • On Windows CE platforms:
    [startup]
    1=tcpw.wce -distributed -server_name servername
    2=dp4srvr.wce -distributed

  • On Unix and Linux:
    tcp3 -distributed -server_name servername -remain
    srv3 -distributed -remain

    If you are using 4.525 and dbdaemon you would use this startup section instead:
    [startup]
    1=tcp3 -distributed -server_name servername
    2=srv3 -distributed

  • On MS-DOS:
    tcp2 -distributed -server_name servername -remain
    srv2 -distributed -remain

In each case servername is replaced by the pingable name of the DP4 server machine. As before you can load DP4 temporarily. The -remain command tail must be omitted everywhere. On Windows add the -noremain option to the startup line for either or both dp4srvr.w32 and tcpw.w32 (it does not matter which.)

The order of the commands given above is in fact usually interchangeable. However, you are recommended to adhere to the order given above, as it is the only one officially supported on all platforms.

In a local resilience configuration there are a number of special options which determine the behaviour of the system in the event of a network error. These Resilience Command Tails are supplied to the database manager (e.g. srv3 or dp4srvr.w32), and not to the network programs. Without one of the Resilience Command Tails, if the server or the link to it fails, the application terminates with an error message.

−READLOCAL

Programs such as DBBACKUP and DBRESTOR, when run on a client, work on the local database and not on the remote database. However, if the command tail −readlocal is supplied to the database manager on the client instead of −distributed , then these programs work on the remote database. −readlocal is used only on the database manager - you always use −distributed when loading the network requester in local database resilience mode.

Please note the highly misleading name of this command tail: −readlocal actually means read remote!

Use of Server Numbers

Where more than one server is in use it is possible (and except for TCP/IP mandatory), to select the server not by specifying a value for -server_name, but instead specifying -server n , where n is a number between 1 and 65535 for TCP/IP, or 1 and 9999 for NetBIOS. In the local resilience configuration the -server N command tail is specified on the database manager (srv3,dp4srvr.w32) and not on the network requester.

For example, on Unix and Linux:
tcp3 -distributed -remain
srv3 -distributed -remain -server N

For further information refer to the section on server numbers.

If you use this method to select the server, applications will not be able to select a different server using the -server_name command tail.


Network Error Handling Options with Local Database Resilience

Various command tails can be supplied to the database manager (SRVx/DP4SRVR), that alter the behaviour of the system in the event of a network failure:

Error Mode | Command Tail | Behaviour in event of network failure
0 | (None) | An application will receive a network error (sys_fail(7)) if a network error occurs when it is running. This will usually cause it to terminate.
0 −nodisable

An Application will still run if the network manager has terminated when it starts up; it will then access the local database exclusively.

The application will still terminate if the server terminates once it has got started.
1 −nocrash

A network error will not cause the application to terminate; instead it will switch to accessing the local database exclusively. Any transaction in progress at the time will receive a commit failure.

If the read-only database manager (MS-DOS only) is in use on the local machine, the application can continue to post to the remote tables but will continue to receive a commit failure for every transaction.

2 −nofail

A network error will not cause the application to terminate; instead it will switch to accessing the local database exclusively. A transaction in progress at the time will be posted entirely to the local database. In this mode, parent and orphan checking are done on the local database as well as on the remote database, because it is not known during the checking phase of a commit whether the data will finally be posted to the remote or the local machine.

If the read-only database manager (MS-DOS only) is in use on the local machine, this mode of operation reverts to being the same as the NOCRASH mode.

There is no means of distinguishing the -nodisable mode from the default option. This does not matter, because in the only situation where they differ the program will not be running yet, and will either run or not run according to whether -nodisable is set or not set.


How to Load the Named Connection Configuration

In this configuration there are no separate instructions for client and server machines - all machines using this configuration can act as both clients and servers.

Machines are loaded as follows:

  • On Win32 platforms:
    [startup]
    1=dp4srvr.w32 -aux
    2=tcpw.w32 -server_name servername
    3=tcpmgr.w32 -aux

  • On Windows CE platforms:
    [startup]
    1=dp4srvr.wce -aux
    2=tcpw.wce -server_name servername
    3=tcpmgr.wce -aux

  • On Unix and Linux:
    srv3 -aux -remain
    tcp3 -server_name servername
    tcpmgr -aux

    With 4.525 and dbdaemon you would use the following startup section:
    [startup]
    1=srv3 -aux
    2=tcp3 -server_name servername
    3=tcpmgr -aux

In each case servername is replaced by the pingable name of the DP4 server machine to be used as the "default" server. (If you are running over the Internet the name may not actually be pingable, but it must be resolvable to the correct IP address). Usually servername will be the name of the machine itself so that it usually acts as its own server. From 4.525/4.620 you can use the name localhost, assuming your TCP/IP configuration supports this. This normally equates to the "loopback" address 127.0.0.1 . This may be convenient if you want to load this configuration on a large number of machines and don't want to have to tailor each machine's DP4 configuration individually. You could use the name of another server, if you usually want to run against its databases.

You cannot use localhost on older versions of TCPMGR because it did not listen on this address.

You can load DP4 temporarily. Add the -noremain option to the startup line of the network client (tcpw/tcp3).

Named Connection + DP4 Terminal Server

If you include the commands required for running DP4 Thin Client and enable TCPMGR as an application server, you will have a machine that has the full range of DP4 networking capabilities.

The commands for running this configuration on Win32 platforms are:

[startup]
1=dp4srvr.w32 -aux
2=tcpw.w32 -server_name servername
3=tcpmgr.w32 -aux -allow_execute -rocopy
4=tstcpw.w32

If your default server_name is localhost, or some other connection to self, you must remember to specify a different server when running programs with tsrun. If you accidentally try to run a program as a thin client of the local application server, the application will usually fail (possibly horribly). With 4.621 the application will simply fail to start.

Reverse Connection Facility

A particular advantage of this type of configuration is that two machines can each act as a server for the other's network requests, and that you can arrange for a server to automatically run a program of your choice on the "reverse" connection to a client. The program is run each time a connection is made to the server from a machine that is not already connected. Using this facility you may be able to reduce the amount of data transferred across the network for tasks such as synchronisation of data between two machines. (From 4.620 RECASTDB can be used for this purpose).

To use this facility you must load tcpmgr with the command tail -reverse, and in the DP4 configuration file specify a command line to be run in the onconnect entry. When a new connection is created this command line will be executed. The IP address of the incoming connection is specified as a final argument, (prefixed with a #). Therefore if you make the final argument of the onconnect string -server_name, or -synchronise (for RECASTDB) the program will automatically run against the correct client machine.


How to Load Multiple Resilience Clients

In the information on this page [] is used to delimit optional items on command lines, the [ and ] should not appear in actual commands. Similarly ... is used to indicate items that may be repeated.

If a machine is loaded a pure client its local databases are inaccessible to other machines. If a machine is required to access datbases on other machines, and to allow access to its databases to other machines then it should be loaded as a Combined Client Server Machine.

Pure Client machines machines that do not act as servers are loaded as follows:

  • On Win32 Platforms:

    [startup]
1=tcpw.w32 -multiple
2=dp4srvr.w32 -aux
3=auxdistr.w32 -use n1 [-use n2 ...] [-nolocal] [resilience
option]

  • On Windows CE Platforms:
[startup]
1=tcpw.wce -multiple
2=dp4srvr.wce -aux
3=auxdistr.wce -use n1 [-use n2 ...] [-nolocal] [resilience
option]

  • On Unix/Linux Platforms:
    tcp3 -multiple -remain
    srv3 -remain -aux
    auxdistr -remain -use n1 [-use n2 ...] [-nolocal] [resilience option]

    If you are using 4.525 and dbdaemon you would use the following [startup] section:
[startup]
1=tcp3 -multiple
2=srv3 -aux
3=auxdistr -use n1 [-use n2 ...] [-nolocal] [resilience
option]

On all platforms the −use command tail is specified one or more times and specifies the default set of servers used to access DP4 databases. This server list can be over-ridden on a per table or per database operation basis using the network resilience API. An alternative form of this option −mustuse n is available and designates a server as a "vital" machine as explained here.

See below for the use of the -nolocal option.

Resilience Command Tails

There are a number of resilience command tails, which control behaviour in the event of a network error. These are supplied to AUXDISTR (not the database manager as in local resilience). They are explained in the section on Network Error Handling. Without one of these command tails, programs will terminate if there is a network problem.

The -nolocal Option - Multiple Resilence without Local Database

Normally Multiple Resilience clients have a local database. If you want a client to use multiple remote servers but not to have a local database you include the −nolocal option on the command line to AUXDISTR. The client will ignore the flags indicating which tables are local and which are remote, and will treat all tables as remote.

Obviously such a system is not as resilient as one with a local database.


How to Load a Multiple Resilience Combined Client/Server

In the information on this page [] is used to delimit optional items on command lines, the [ and ] should not appear in actual commands. Similarly ... is used to indicate items that may be repeated.

Combined client/server machines will be both requesters and servers. In this case the network manager must be loaded and AUXDISTR must be informed of the number of the server which is running on its own machine. The instructions for loading DP4 networking are as follows:

  • On Win32 Platforms:

    [startup]
    1=tcpw.w32 -multiple [-server servernr]
    2=dp4srvr.w32 -aux
    3=auxdistr.w32 -server servernr -use servernr -use n1 [-use n2 ...] [resilience option]
    4=tcpmgr.w32 -aux -server servernr -aux [-online]

  • On Windows CE Platforms:

    [startup]
    1=tcpw.wce -multiple [-server servernr]
    2=dp4srvr.wce -aux
    3=auxdistr.wce -server servernr -use servernr -use n1 [-use n2 ...] [resilience option]
    4=tcpmgr.wce -aux -server servernr -aux [-online]

  • On Unix/Linux Platforms:

    tcp3 -multiple -remain [-server servernr]
    srv3 -remain -aux
    tcpmgr -aux -server servernr -detach [-online]
    auxdistr -remain -server servernr -use servernr -use n1 [-use n2 ...] [resilience option]

    If you are using 4.525 and dbdaemon use the following [startup] section:
    [startup]
    1=tcp3 -multiple [-server servernr]
    2=srv3 -aux
    3=tcpmgr -display_address -aux -server servernr [-online]
    4=auxdistr -server servernr -use servernr -use n1 [-use n2 ...] [resilience option]

Note the use of the -detach command tail on tcpmgr on Unix/Linux. In the multiple resilience configuration tcpmgr is currently not capable of displaying information about the connected machines, so the -detach option should be used.

The additional command tail -aux on the network manager is to notify it of the existence of the ADC program AUXDISTR and to ensure that it communicates directly with the database manager and not via AUXDISTR (which would lead to disaster as AUXDISTR would then possibly send the request back to the network manager or another requester!.

The −online command tail on TCPMGR is only used in Duplex configurations. It causes TCPMGR to notify AUXDISTR at commit time when a server that has been offline comes back online. AUXDISTR will reconnect to the server and start sending updates to it once again. However, any transactions posted while the server was offline need to be transferred to the newly restored server. This must either be done with a specially written application, or the DP4 ONLINE utility.

On all platforms the −use command tail is specified one or more times and specifies the default set of servers used to access DP4 databases. Note the use of −use servernr to specify the local machine as one of the servers to be used in the default set, assuming that it is to be so used.

This server list can be over-ridden on a per table or per database operation basis using the network resilience API. An alternative form of this option −mustuse n designates a server as a "vital" machine here.

The −server servernr command tail is optional on the network requester (e.g. tcp3). For certain operations on the local server AUXDISTR needs to go via the network manager rather than directly to the database manager (for example checking the local network manager is online). In the past it has sometimes appeared that using network communication between a network requester and a network manager running on the same machine is unreliable. If you specify the −server servernr command tail on the client, it may use, depending on the operating system and network protocol, a "backdoor" method of calling the network manager. This is usally faster than using real network calls. If you don't specify the −server servernr command tail on the client, it will communicate with its local network manager in exactly the same way as every other server. As far as we know there is no need to use the "backdoor" method on any current networks. In 4.621 tcpw will automatically detect which connections are local and avoid unecessary network calls, and the −server servernr is redundant.

Resilience Command Tails

There are a number of resilience command tails, which control behaviour in the event of a network error. These are supplied to AUXDISTR (not the database manager as in local resilience). They are explained in the section on Network Error Handling. Without one of these command tails, programs will terminate if there is a network problem.


Network Error Handling Options with Multiple Resilience

The behaviour of applications when all the servers required for a given operation fail is normally controlled by optional command tails specified when AUXDISTR is loaded, and is also dependent on whether or not there is a local database. However programs can change how errors are handling by posting to the network status table. The table below summarises the available error modes, the command tails needed to select them as the default error mode for programs that do not set an explicit error mode for themselves, and the behaviour of the system when they are used.

Error Mode | Command Tails | Behaviour with local database | Behaviour without local database
0 | (none) | Application receives sys_fail 7 (Network error), and will terminate unless it contains special error handling. | As for system with local database
1 | −nocrash | Revert to local database only Updates pending on remote servers at time network problem arose lost Future updates that should be posted remotely posted to local database | Wait for a server to restart Updates pending on remote servers at time network problem arose lost
2 | −nofail | Revert to local database only Updates pending on remote servers at time network problem arose posted to local database Future updates that should be posted remotely posted to local database | Wait for a server to restart, Updates pending on remote servers at time network problem arose will be resent when network is restored
3 | −nocrash −readlocal | Revert to local database for reads Subsequent updates that would be sent to remote servers are ignored | If read, wait for a server to restart If update, ignore error
4 | (not available) | Ignore error, subsequent reads return 'not there' Subsequent updates are ignored | As for system with local database

If an update is lost or ignored, the next commit will fail.

Error mode 4 cannot be selected as the default using command tails, but is accessible under program control (by posting to the network status table.)

If you use −nocrash or −nofail with −mustuse n you need to be aware that the −mustuse n behaviour will only happen while at least one database server is online. If all servers go offline, so that the system reverts to completely local operation, commits that would normally fail because the "vital" server is offline will succeed.


Automatic Server Reconciliation for Duplex Resilience Configurations

When a server is reconnected to a network with two servers following a failure of any kind, the program ONLINE can be used to reconcile the data on the two servers.

In the worst situation, neither server has failed. They have simply become disconnected, with some workstations continuing to operate on one, and others on the second. ONLINE reconciles the two databases. It is always successful in the sense that the databases will be the same for all records that have been updated during the session, but there are circumstances when this still does not give the required result.

ONLINE works by comparing the transaction logs on each server against the database on the other. It can therefore only be used where transaction logging is enabled. ONLINE performs the following operations:

  • Records inserted on one server are copied to the other if they are either not present on the other or the matching record has an earlier last update time

  • Records amended on one server are copied to the other if they are either not present on the other or the matching record has an earlier last update time

  • Records deleted on one server cause a deletion on the other if they are present on the other and the matching record has an earlier last update time

    Records match if they have the same primary key.

For this reconciliation to be useful, the application must conform to guidelines discussed in Design Constraints with Multiple Server Resilience.

To use ONLINE run it against your local database specifying the number of the server to be used for reconciliation with the -server n command tail. For example:

online -db salesord -server 1

See Command Tails for other command tails that can be used with ONLINE.


Using other ADCs with Multiple Resilience

The multiple server resilience software requires the special ADC called AUXDISTR to be loaded. Users may wish to load their own ADC programs to intercept messages passed from the application to AUXDISTR; the following explains how this can be done (and also how users may load multiple ADCs). Note that the explanation is "top-down", as it is easier to work out the correct command tail in this direction. However, the programs are actually loaded in the opposite order.

  1. The first ADC in the chain receiving messages from the application should be loaded without any special command tail. This will cause it to use the same IPC mechanism that the database manager normally uses, so that it receives the messages from the application correctly.

  2. The next ADC should use the command tails: -aux -chain 1 and subsequent ADCs should specify -aux -chain 2 , -aux -chain 3 and so on to indicate their position in the chain. The -chain n command tail causes the ADC to modify the IPC mechanism it uses (by reading a different queue, semaphore or whatever) so that messages are passed down the chain of ADCs in the correct order.

  3. Finally the database manager SRVn should be given the same command tails as the last ADC in the chain.

  4. If the network manager (usually TCPMGR) is also loaded, it should usually be given the same command tails as the database manager, to ensure that it communicates directly with the database manager and not using the ADC. Alternatively you can specify the command tails that match the ADC you want to intercept the network manager. You must not let the network manager pass its calls to AUXDISTR - you must specify at least -aux.

The database manager and ADCs should be loaded in reverse order to that in which messages pass down the chain, as each tests for the presence of the other programs in the chain when it is loaded.

Example Configuration for Unix

For example, the following sequence of UNIX commands loads the ADC called USERADC on top of the appropriate programs for multiple server resilience:

tcp3 -remain -multiple
srv3 -remain -aux -chain 1
tcpmgr -remain -detach -aux -chain 1 -server 1
auxdistr -remain -chain 1 -server 1 -use 1 -use 2
useradc -remain

You should normally load user ADCs above auxdistr rather than underneath. (A possible exception might be if you were using AUXDEBUG to track which database requests were being handled locally.) AUXDISTR uses some special functionality in the database manager, which makes putting ADCs underneath AUXDISTR difficult.


How to Load the DP4 Terminal Server

Server Side Configuration

Machines that act as Application Servers for DP4 thin clients should be loaded as DP4 servers using any of the networking configurations with server capabilities. There are no additional components to load on the application server, except that you need to ensure that all the applications you want to run are installed on the server, and you need to enable support for remote execution by specifying the -allow_execute option on the command line for TCPMGR. For security reasons it is a good idea to also specify the -rocopy option at the same time, though this will prevent some DP4 utilities from being run from requester machines. See Secure Operation of DP4 in the Guide to DP4 Configuration for more information about Security.

Thus the [startup] section on the application server might look like this:

[startup]
1=dp4srvr.w32
2=tcpmgr.w32 -allow_execute -rocopy

Machines running as any kind of server can act as an application server - the same command tails should be used on the line that loads TCPMGR in the Named Connection Configuration or Multiple Resilience Combined Client/Servers.

DP4 Thin Client

The DP4 Thin Client is loaded with the TSTCPx as an additional program to your chosen regular DP4 configuration. You will most likely be using the client only networking configuration, but the thin client is also compatible with the Named Connection configuration and the Local Database Resilience Client.

  • On Windows clients add the following additional command to your [startup] section in the DP4 configuration file.

    n=tstcpw.w32

    It does not actually matter where in the sequence this component is loaded, but it is probably most logical to load it immediately after the program that provides the application level database manager. For example, if you are using client only networking you would use the following:

    [startup]
    1=tcpw.w32 -server_name servername
    2=tstcpw.w32

    Refer to How to Load the Named Connection Configuration, for an appropriate [startup] for configuring a machine so that is can act both as an application server and a thin client simultaneously.

    On Windows NT,2000,XP if you will be running the DP4 system as a service you must allow the service to interact with the desktop, otherwise the DP4 thin client will not work properly.

    The server name specified for tcpw does not prevent you from running applications on more than one application server. It merely establishes a default server. When applications are started you can over-ride the server name.

  • The details for Windows CE are essentially the same as for regular Windows. On Windows CE clients use the following additional command in the [startup] section in dp4.ini.

    n=tstcpce.wce

  • On Unix and Linux use the following command in a shell or a shell script:
    tstcp3

    If you are using 4.525 and dbdaemon use the following additional command in the [startup] section in dp4.ini.
    n=tstcp3

Running Programs with the DP4 Terminal Server

To start an application running on the server the TSRUN program is used. The way to use this program is as follows:

  1. Construct the command line that you would use to run the application normally on the server.
  2. If the server for the application is not the default application server prefix this command with -server_name servername.
  3. Supply this command line as a set of command line arguments to TSRUN.

You can also use TSRUN to start an application running completely remotely on the server. In this case add the command tail -remoteonly.

On Windows you can set up short cuts or batch files with the appropriate command lines for programs you want to run using TSRUN. Similarly on Unix you could create a shell script. On Windows CE this is not possible. Therefore from 4.620 TSRUN supports a second method of running programs : if TSRUN is invoked with no arguments at all then it looks for:

[tsrun]
command=Command Line

in the DP4 configuration file. Unfortunately the command line cannot include command line options for TSRUN, only for the program to be run. In particular you cannot use the -server_name command tail to run programs on another application server. If you need to run more than one program from a TSRUN short cut you can copy TSRUN to another name. In fact TSRUN looks for the command in the section with the name of the currently running executable. This method of running programs also works on regular Windows.

If TSRUN is started with no command line and there is no command entry in the [tsrun] section of the DP4 configuration file, TSRUN will invoke the DP4 program itself.


NETCOPY

The program NETCOPY copies files from the database server machine to the local machine (or vice versa); its syntax is:

NETCOPY -FROM FILENAME [ -TO FILENAME ] [ -TOSERVER ]

If the TO is omitted, the file is copied to the same name. If -TOSERVER is specified, the file is copied from the local machine to the database server. NETCOPY does not accept wild card filenames.

  • NETCOPY does not rely on any kind of file sharing or network drives etc. It can be used between any two machines which can be linked by DP4.

  • In Windows environments NETCOPY could allow a malicious user to gain unauthorised access to a server's files, and change or view information that would normally be denied to him. For this reason you may wish to disable this program. This can be done using the -nocopy or -rocopy options on tcpmgr. See Command Tails for more information.

If local database resilience is in operation, then the database manager must be loaded with the command tail -READLOCAL for this program to read from the remote machine. No doubt there is a good explanation for why this command tail has such a misleading name, but please remember that it is misleading.

If multiple server resilience is in operation, the additional command tail -server n is required to specify which remote server is to be used.

Synchronising date/time

Nowadays NETCOPY is mainly used to ensure that the system clock on two machines is synchronised. To "copy" the current date/time setting (instead of a file), use: NETCOPY -DATE [-TOSERVER]

Note that on many platforms a program requires "root" or "administrator" privileges to be able to change the date or time on the system.


Troubleshooting

When DP4 networking is in use applications may not be able to report errors until they have managed to connect to the network server. When there is a problem the application may simply fail silently (it will usually return an exit code of 5 when this happens). Obviously this can make problem investigation a little tricky. (This should not happen with DP4 Enterprise releases.)

If a call to the DP4 system fails because of a network problem you will normally encounter a FAIL 7 error message. Some additional information on the various additional error codes associated with this message is given here.

If you have trouble making or keeping connections with DP4 networking there are diagnostic facilities that may be of use. However these diagnostics are primarily designed for use by Itim Technology Solutions, and you may have trouble understanding them. You can find general information on enabling DP4 diagnostics in the Guide to DP4 configuration here.

On Windows (including Windows CE), and, from 4.525, on Unix and Linux, the diagnostics are (at least optionally) written to the DP4 error log program. The easiest way to turn on DP4 error logging, is to stop the DP4 service (if it is started), and reload it like this:
srvw32 -start -load_errlog -debug_echo
On Windows CE you use srvwce, and on Unix and Linux dbdaemon, instead of srvw32.

You can turn on error logging by default by setting the entry for load_errlog in the [system] section of the DP4 configuration file to 1. Error logs are usually written to the file debug.out (the name is also a DP4 configuration file entry) which will be located in the current directory for the DP4 service (normally the directory where it is installed). On Windows NT/2000 you must enable the DP4 service to interact with the desktop for the error log to output to the screen. Alternatively you can use the -load option in place of −start when you run srvw32.

If you are using 4.523 on Unix or Linux the diagnostics are written to various files and cannot be seen on screen unless you also specify −echo on the command lines for the DP4 network programs. See DP4 Networking Notes for Unix and Linux (4.523) for more details.

There are three levels of diagnostics, enabled by options on the command line to the network requester and manager:

  1. −debug_error , available from 4.520 onwards, causes diagnostics to be written only when there is an error condition - either a bad return code from the operating system, or a system error or fail error from DP4. This option can safely be left turned on all the time, and may help to diagnose intermittent problems.
  2. −debug_connect , available from 4.621 onwards, includes the effect of −debug_error (causing diagnostics to be written when there is an error condition), but includes some additional diagnostics related to the making and breaking of connections between the requesters and servers, which are produced whether or not there is an error condition. This option can also safely be left turned on all the time, and may help to diagnose intermittent problems.
  3. −debug turns on diagnostics for every stage of every operation in the network manager or requester. It should normally only be used when you have trouble getting or maintaining a connection at all. This option has a severe impact on performance, and should never be used in a production environment.

Trouble shooting TCP/IP - things to try

−nonames Option

If you have trouble making a connection, it is worth experimenting with the −nonames option on tcpmgr. By default tcpmgr attempts to discover the hostname for all incoming connections. Occasionally, particularly in a network contain machines running a mixture of operating systems (especially non Windows operating systems), this call can take a VERY long time to return. This can cause new connections to fail, and existing connections to time out.

−nokeepalive Option

Occasionally, on some networks, (again particularly in mixed environments), the DP4 server may report a client has died because it failed to send a keepalive message. In older versions of DP4 this could happen when the server was busy, as it incorrectly checked the keep alive timeout without making sure there were no more messages pending. However because TCP/IP packets are not necessarily sent when DP4 requests that they should be, (because of buffering inside the TCP/IP itself), it can happen that defective TCP/IP implementations fail to send keepalives from DP4 when they should. In this case using the −nokeepalive option on the server will cure the spurious timeout. You should not use the −nokeepalive option unless necessary as it prevents the detection of failed client machines. If you specify it on the server, you should also specify it on all the clients, as otherwise now unnecessary, harmless, but bandwidth consuming messages are still sent by idle requesters every few seconds.

If you use this option, then if a client is rebooted or turned off, or fails unexpectedly in some other way, the connection at the server end will not be terminated and cleaned up automatically. However, if the client is restarted the old connection will be cleaned up, assuming it has the same IP address as before (which will normally be the case unless DHCP is in use and the machine's lease on the IP address has expired).

−socket n Option

It is possible, though unlikely in practice, for DP4 TCP/IP networking to fail because of a conflict with other software, which may happen to use the same port number as DP4. You can force DP4 to use a different port number (the default is 5000) by specifying a value with the −socket option on the command line, for example −socket 6000 . You should avoid port numbers that are commonly used by programs such as FTP,Telnet etc. Numbers above 5000 are probably best. If used, this command tail must be specified and given the same value everywhere.

The use of port 5000 can cause problems on Windows XP machines. Please refer to article A4000024.htm on the DP4 Knowledge Base for details.

Mixed networks

If you are trying to connect from a Windows TCP/IP client to another operating system, particularly a legacy operating system such as FlexOS or IBM 4690 OS you may have problems because of incompatibilities in the underlying network protocol. This is a distinct possibility if you are using Windows 98/ME/2000/XP or later, but is much less likely if you are using Windows 95/NT4.

For Windows 2000/XP try adding the following DWORD values to the registry at
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters :
SackOpts=0
tcp1323Opts=0

The machine must be rebooted for these options to take effect. Disabling/Re-enabling the network connection has no effect.


TCP/IP Specific Aspects of DP4 Networking

Keep Alives, Operation over a Wide Area Network

DP4 TCP/IP software uses a system of keep alive messages from the client to the network manager to manage connections. This enables the network manager to drop connections and perform the necessary tidying up if a client application terminates abnormally or if the physical network connection is broken. The scheme works as follows: The network manager maintains a timer for each active connection. This timer is restarted whenever a message is received on that connection. If an application does not access the server for a specified length of time, TCPn sends a keep alive message to restart the network manager's timer on that connection. If the timeout period on the network manager expires without being reset by a message, which may be either a genuine database request or a keep alive message, the connection is dropped and the user(s) logged off the database manager. The various timeouts are determined by the timeout parameter as follows:

  • The network manager checks each connection's timer every timeout/10 seconds.
  • The network manager drops a client connection if more than 75% of the timeout period elapses without a message from the client. This timeout can be changed using the -deadtime command tail.
  • The client attempts to send a keep alive message if more than 50% of the timeout period has elapsed since the last communication with the network manager. (Prior to 4.620 clients sent messages after 30% of the timeout period)

It is essential that any timeout value supplied on the command line be the same for all clients and network managers on the network. The command tail −deadtime n can be used on tcpmgr to extend the period that the network manager will wait for a message or keepalive before disconnecting the client. n is given in seconds. Obviously only values of n greater than the timeout value make sense. This may be of use on slow networks where the delivery of keepalive messages may be unreliable for some reason (such as unsafe buffering algorithm within the TCP/IP implementation). Keep alive messages are handled much more reliably in 4.520 and later versions of DP4 networking. Older versions of the network manager might fail to see a keep alive because the network manager was busy processing other requests. This cannot happen after 4.520/4.616

It is possible to turn off keepalives if necessary. See information on the nokeepalive option. You may wish to disable keepalives where a WAN is in use, especially if there are a large number of users, or keeping connections live unnecessarily is expensive. If keepalives are not disabled a message will be sent every few seconds, keeping the line active.


Creating Named Connections from C/C++ Code

The information in this section allows you to create a named connection to a specified TCP/IP machine name using DP4 networking. You can create named connections with the following configurations of DP4 networking:

  • Simple Client Only Networking
  • Local Database Resilience Client
  • Named Connection Configuration

The recommended method of using named connections is to use the dp4dbase C++ interface to DP4. This is a source code compatible replacement for the old DP4DYN based dp4class interface into DP4. The only change from a traditional dp4class program, is that the Open() method used to establish communication with DP4 has an optional server_name parameter. All you have to do to create a named connection is specify the desired server name. You should probably avoid using the global "theConnection" available in the dp4clex.cpp replacement dp4dbsasx.cpp in programs using named connections. The dp4dbase files can be found in the dp4dyn subdirectory for each release of DP4 after 4.620.

Checking for Success

If DP4 is not running in a networked configuration, attempting to open a named connection will apparently succeed, but open a local connection!. If necessary you can use the Network Status table to check that the connection is using DP4 networking after you have created it.

C Library

In theory you can use the regular DP4 C library with named connections. However this is strongly discouraged, because of the difficulty of managing global variables such as server_handle, df_process etc. These all have to be saved and restored each time you want to switch the connection you are working with. If you want to program in C rather than C++ the best option is to use the dp4dbapi.h version of the C library. This is very similar to the usual C library, but without any global variables. dp4dbase is built around dp4dbapi.

The header file dp4dbapi.h is in the include subdirectory for DP4 releases from 4.620. You should link with ntown.lib as for a C program.


Resilience Application Program Interface

Most of the information in this section is applicable only to Multiple Resilience Configurations, and to a lesser extent to the Local Database Resilience Client. For all other configurations the only part of this section that is at all relevant is that relating to the Network Status table, as you can use to check how DP4 is configured. Most of the pages in this section start with This page applies to ... to indicate which configurations they apply to.

Overview

The Resilience API is primarily based on using two special tables on the database. Reading and updating these tables allows you to find out about and control the servers your application is using.

  • The Network Layout table (@NETWORK) specifies which server or servers are to be used for the database or for individual tables on it. It is only used with multiple resilience.

  • The Network Status Table (@NET_STATUS) table can be interrogated to discover what resilience configuration (if any) is in effect and whether servers are on- or off-line. It can be updated to force servers on- or off-line. This table is available in any DP4 configuration, though its use will vary according to the configuration.

When the resilience software is loaded, operations on these tables do not access data on the database, but instead manipulate control information held in the memory of the resilience programs. Updates to these table take effect immediately, without a wait until the next db_commit() or db_checkpoint() . This makes it possible to change servers within a commit unit, and create transactions distributed across multiple servers. See Operation of @Network with Multiple Resilience Loaded for details.

Because the @NETWORK and @NET_STATUS tables appear to be ordinary database tables, they can be manipulated by programs written in C or DP4 QA Build, or by utilities such as BROWSER.

A global variable, broken, can also be used to test network status.

A second technique, fully available only to the C/C++ programmer, allows access to be switched from the local to the remote server(s) and back again, by adding a special value, LOCN_OVERRIDE to the flags in fetch and update operations.

When a muliple resilience configuration is in use there are some important changes to DP4 functionality. In particular the operation of secondary indexes and commits may be affected. This is discussed in more detail in Multiple Resilience Implementation.


Network Status Table

This table is called @NET_STATUS. This table available in any DP4 configuration, though its use varies considerably according to the configuration. No records exist in this table on the database: the records which are returned when the table is read are constructed within the programs providing the DP4 service.

The fields in this table are:

Field Name | Notes
net_status.info_from.server | Server from which data required. This is the first part of the key to this table
net_status.info_about.server | Server about which data required. This is the second part of the key to this table
net_status.net_mode

Server status as summarised in the table below:

Value | Meaning
0 | Status not known
STAY_OFFLINE (3) | Server believed to be off-line. Can be used when forcing a server off-line by posting to network table
OFFLINE (2) | Server believed to be off-line
ONLINE (1) | Server believed to be on-line

The values ONLINE, OFFLINE,STAY_OFFLINE are defined in network.h.
net_status.err_mode

Value indicating network error handling mode for this program.

  • For Multiple Resilience Networking the value is one of the error modes listed in Network Error Handling Options with Multiple Resilience

  • For Local Database Resilience the value is one of the error modes listed in Network Error Handling Options with Local Database Resilience

  • For any other Configuration of DP4 err_mode is 0, indicating there is no special handling for network errors.
net_status.resil_type

Current resilience type on local machine as follows:

Value | Meading
0 | No networking, no resilience (Standalone configuration) or Named Connection Configuration connection to local machine‡
1 | (Simple) Client Only Networking or Named Connection Configuration connection to remote machine‡
2 | Networking with Local Database Resilience Client
3 | Networking with AUXDISTR. Does not switch to local database on network errors.
4 | Networking with AUXDISTR. Switch to accssing and possibly updating local database on network errors. The err_mode field will indicate how the local database is used, as described in Network Error Handling Options with Multiple Resilience
5 | Networking with AUXDISTR. No local database (-nolocal option specified)

‡This use of the resil_type field for Named Connection Configurations is explained further here.
net_status.this_machine.server | Server number of local machine (Only valid in Multiple Resilience Configurations). Zero for client only machines and in other Configurations.
net_status.file_server | Server number to which remote file functions such as sf_open(),sf_read() etc, are directed. You can select the server for these operations by updating the network status table.
net_status.remote_files | This field is only valid in the local database resilience configuration, and indicates whether remote file operations are directed to the local machine or the remote machine. (See information on -readlocal for details).

Other fields in this table are reserved for internal use

Use of Network Status Table

For further information on how this table is used please refer to either:

  • Network Status Table for Configurations not using Multiple Resilience (i.e. all configurations where AUXDISTR is not used)

or:

  • Network Status Table for Configurations using Multiple Resilience (i.e. all configurations where AUXDISTR is used)


Network Status Table in Configurations not using Multiple Resilience

Except in multiple resilience configurations there is exactly one record in the Network Status table. This record has both key fields set to 0. You can read the record with a call to rec_fetch_main(EQUAL,...). This allows you to discover the networking configuration by examining the values in the resil_type and err_mode fields.

Although the software does not prevent it, there is no point in posting to the network status table unless you are using DP4 in a Multiple Resilience Configuration. Updates have no effect on the operation of DP4 in these configurations.

Net Status Table in Named Connection Configuration

‡From 4.621, and currently on Win32 platforms only, when Named Connection configuation is in use the network status will be returned differently according to how requests to the database manager are routed. Connections to the local machine will return the network status information as though the system was running in standalone mode. Connections to a remote machine will return the network status information as though the system was running simple client server networking. On other platforms the network status will currently always indicate a network connection. This is reasonable, because on those platforms the requests are sent to the network. DP4 detects whether a connection is local or remote by attempting to bind a socket to the IP address of the server. This will only succeed if the server is local.


Network Status Table in Configurations Using Multiple Resilience

Checking the Network Status

Both the client (within AUXDISTR) and each server (in the network manager) maintain information about which servers are currently running. This information may be accessed by reading the network status table. The information is accessed from the local client program by setting info_from.server to zero, and from a server by setting info_from to the appropriate server number. The info_about field should be set to the server number about which information is required. The rec_fetch_main(EQUAL,...) function or an equivelent should be used to read the record.

If no record is returned then the server from which information was requested did not respond. Note however that no attempt is made to contact the specified server if the client believes that it is currently off-line. For this reason you may want to force the client to attempt to contact the server by updating the network status table.

If a record is returned, then the net_mode field indicates the status of the server about which information is required and is s set to either 0 (unknown), 1 (ONLINE) or 2 (OFFLINE).

It is also possible to search through this table using rec_fetch_main(NEXT,...)and so on, but records will not be returned except those for which info_from.server is known by the local client to be online and info_about.server is known to the local client.

Refer back to Network Status table for an explanation of the various fields in this table.

Status Information from Clients and Servers

The meaning of each type of server status information is as follows:

  • Information from client

    The status of each server is initially set to zero (unknown) , but is changed when a message is first sent to that server and is subsequently always either OFFLINE or ONLINE. It will be set to ONLINE unless the server fails to respond to a message, when it will be changed to OFFLINE. No further attempt will be made to contact a server with OFFLINE status (so the status will remain as OFFLINE) unless one of the following occurs:

    • The application forces an attempt to contact this server (see updating the network status table).

    • Another server indicates that the off-line server is now running again. (Only servers for which the -online option is specified on the command line will ever send such an indication).

  • Status Information from Servers

    Each server holds information about every other server, which it collates from information sent by clients during commits. A client sends its status information about the other servers at this time and this is used to determine the status of other servers. If conflicting information arrives from various clients, a report that a server is online takes precedence over a report that it is off-line (as the former implies that communication with the server took place recently, but the latter may be based on out-of-date information).

    Information received from a client is retained only for a certain length of time, so that the server status is never based on old information. If no fresh information about a server arrives, its status will eventually revert to 'unknown'.

    The time for which information is held defaults to five minutes, but may be altered by loading the network manager with the command tail: -stimeout nn where nn is the required timeout in seconds.

Checking Network Integrity

In a Multiple Resilience Configuration the network status record in which the info_from and info_about fields both zero has a special meaning. The 'net_mode' field in this record has the value ONLINE if every server about which the local client has information is currently ONLINE, and is set to OFFLINE if any one such server is OFFLINE. This allows the current integrity of the network to be tested with a single fetch on this table. The difference between this and testing the 'broken' variable is that the latter is set TRUE when a server goes off-line, but is not reset if it subsequently comes back online.

Effect of Updating the Network Status Table in Multiple Resilience Mode

The application program can force AUXDISTR to attempt to access a server which AUXDISTR currently believes is off-line by posting to the @net_status table with a value of net_mode = ONLINE and then attempting a normal database access. Before posting The info_from field should be set to zero, the info_about field should be set to the relevant server number, and the net_mode field should be set to ONLINE.

Afterwards the @net_status table can be read as described above to discover whether the server is indeed now ONLINE.

Similarly, posting @net_status with net_mode = OFFLINE forces the connection to the server to be dropped (as though the server had terminated). This may be useful if you know you are not going to access a particular server again for a while, and do not want to stay connected to it unnecessarily. Release 4.622 introduces a new net_mode STAY_OFFLINE, which you can also use to force a server offline. The difference between posting a server OFFLINE and STAY_OFFLINE as as follows: when a server is posted OFFLINE, although any current connection is dropped, AUXDISTR will reconnect to the server automatically next time a database operation should be directed to that server - rather than being truly OFFLINE the server has reverted to its initial UNKNOWN state; when a server is posted STAY_OFFLINE the server will not be reconnected automatically, the server is truly OFFLINE, exactly as it would be after a network error. AUXDISTR will normally only reconnect to the server when the application posts the the server ONLINE again. (However, if the -ONLINE mode option is used on the network manager, and AUXDISTR receives a message that the server is online, the server will be reconnected exactly as it would be after a disconnection caused by a network error)

The application program can change to error handling option by posting to the @net_status table with both key fields set to zero, and the err_mode field set to the required value. (See Network Error Handling Options with Multiple Resilience)

Other posts or deletions on this table are ignored.

Updates to the @net_status table take effect immediately, and are not postponed to the next commit() or checkpoint().


The broken Variable

This page applies to Local Database Resilience and Multiple Resilience Configurations Only

The global variable broken is set to TRUE whenever the software tries and fails to access a remote machine. It is never reset to FALSE by the system software (though it can be reset by the application program if required), so it may be conveniently tested at any time to determine whether a network problem has occurred.

Note that the servers accessed by an operation may not be the ones expected. For example:

  • A fetch is usually directed to only one of the servers on which the table being accessed is held. Also some fetches may be "buffered" so that not every fetch() actually has to call a server. (This can improve performance considerably). Buffering of fetching can be disabled with the -noblock command tail on auxdistr.

  • Database updates, (the rec_post() and rec_kill() functions), may not be done immediately, but may be batched up by the system and done later; in this case no server will be accessed during the processing of the function.

  • A fetch or commit may cause batched-up updates to be sent, so may access servers other than those it directly affects.


Location Override for Fetches and Updates

This page applies to Local Database Resilience and Multiple Resilience Configurations Only

The special value LOCN_OVERRIDE can be added to the flags for a fetch or update operation. This will cause the default setting of the local/remote flag for the table involved to be reversed; if the table is marked as local, the operation will be done on the remote server(s), and if the table is marked as remote it will be done on the local database.

In multiple resilience configurations you can check the state of the network.local flag for the table to see whether a table is marked as remote or not.

Unfortunately there is no easy way to check whether a table is local or remote for local resilience - you have to fetch the @reldata record for the table and test the F_LOCAL flag in reldata.rel_flags, remebering to invert the value for base dictionary tables!.

In multiple server resilience, the set of servers used for a database operation can be more finely controlled using the @NETWORK table as described earlier. However it is often convenient to use this flag where a program is transferring data from a local database to a server, or vice versa.

In QAB programs LOCN_OVERRIDE can be specified for database update operations (by changing the QKILL_FLAGS or QPOST_FLAGS ) variables. (See the QAB Standard Variable Reference for details). However LOCN_OVERRIDE cannot be specified in database reads, unless they are done in a C exit, so you would have update the @NETWORK record for the table instead.


Network Layout Table

This page applies to Multiple Resilience Configurations Only

The Network Layout table is called @NETWORK and contains the following fields:

Field Name | Description
network.number | Table number to which layout applies, or zero for the default layout.
network.parent.number
network.role.number
network.old 		Reserved: must be set to zero
network.dtype | Not used
network.machine | Array of server numbers to use
network.local | Local/remote flag
network.nr_machines | Number of entries used in server array (network.machine)
network.vital_machines | Number of entries in server array (network.machine) that refer to servers that must be online when programs try to update records on them.
network.nr_prefixes
network.prefix_size 		Reserved : Must be set to zero

network.machine should have valid, different server numbers entered from the lowest element upwards. Zero is a valid server number, meaning 'local'. AUXDISTR will consider as many elements as the value of network.nr_machines. So, if the values in network.machine are:

4 3 0 1 0 0 0 ...

if nr_machines is 2, the servers to be accessed are 2 and 3;
if nr_machines is 3, the servers to be accessed are 2, 3, and the local machine;
if nr_machines is 4 the servers to be accessed are 2, 3, 1 and the local machine;
if nr_machines is more than 4, there is an undefined effect from supposedly accessing the local machine twice.

The vital_machines field sets an initial number of machines in network.machine which must be online when an application tries to update them. For example, if a program is set to update a table on server 1 and 2, so that nr_machines is 2, the update will normally succeed if either server is online. If vital_machines is set to 1, then assuming 1 is the entry in machine[0] an update will only succeed when server 1 is online (unless AUXDISTR has reverted to local mode because no servers are available).

The network.machine array is usually ignored if the local flag is set, in this case only the local database is accessed or updated, although the LOCN_OVERRIDE flag can override this. See How the 'local Flag is Used for full details of the local flag.

Operation of @Network with Multiple Resilience Loaded

While multiple server resilience is in operation, fetches, posts and deletions on the @Network table do not go to the database, but operate on the internal layout data held within the AUXDISTR program. If there is more than one application program on a machine, each has its own entry in the AUXDISTR tables. Thus, each application program can be accessing different servers.

One entry will appear in this table for each table on the database, and also one extra entry (with key 0) representing the default layout. When a program starts up, the default layout is as specified by the command tails, and every table follows the default layout unless there is an overriding persistent entry in the local database (see below). (If the -nolocal command tail is in use then persistent entries will be read from whichever server(s) are used for accessing the @network table).

  • Posting an entry with a zero key to the @NETWORK table overrides the default layout. The layout for every table currently following the default will be changed.

  • Deleting an entry with a zero key from the @NETWORK table causes the default layout to revert to that specified by the command tails. The layout for every table currently following the default will be changed.

  • Posting an entry with a non-zero key to the @NETWORK table overrides the layout information for the specified table. The specified table will no longer follow the default or the information held on the database, but will use the posted layout instead.

  • Deleting an entry with a non-zero key from the @NETWORK table causes the specified table to follow the current default layout. Tables with a persistent overriding layout do not revert to that layout, but also revert to the current default layout.

Updates to the @NETWORK table take effect immediately, and are not postponed to the next commit()or checkpoint().

An application can fetch records in the @network table at any time to check how a particular table is accessed and updated. For example if you want to use the LOCN_OVERRIDE flag to temporarily switch how a table is accessed (for example by fetching remotely and posting locally, or vice versa) you could check the current setting of network.local to see whether to apply the LOCN_OVERRIDE flag to calls to rec_fetch() or calls to rec_post().

Persistent @Network records

Records may exist in the @Network table on the database to set the default layout for tables; these must be posted while the system is in non-resilient mode. In order to do this you need to know the table numbers for which records are required. You can use BROWSER -BASEDICT to do this (but you will need to run outside the DP4 menu and set your access levels to 255 255 with DFSETUP), or you can write a simple application program to create the required records.

The 'local' field in persistent records is ignored as this information is set up using MAKEDB −expert and held in the @RELDATA table. See How the 'local' Flag is Used for details.

You should not create a permanent record with a zero key in an attempt to override the default layout - any such record will always be ignored. Permanent records for the @reldata and @network tables will also be ignored, because reading them would involve an endless recursion.


How 'local' flag is used

This page applies to Multiple Resilience Configurations Only

This section describes the use of the 'local' flag and its interaction with the location override flag.

The location information for a table stored on the database is as follows:

  • Local/Remote Flag

    A flag in the @RELDATA entry for the table, which can be set to either 'local' or 'remote' using MAKEDB -expert. This flag is used in both local resilience and multiple resilience configurations.

  • List of Server Numbers

    Optionally, and for multiple resilience only, a persistent entry in the @NETWORK table giving a list of server numbers for the table.

When an application first accesses a particular table, AUXDISTR creates an in-memory copy of an @NETWORK table entry for that table as follows: the list of server numbers is copied from the entry on the database, if there is one, and otherwise is taken from the list of default servers supplied by command tails. The network.local flag is copied from the @RELDATA table entry (unless the −nolocal command tail is used on AUXDISTR, in which case network.local will be set to No). The value of local contained in any persistent @network record for the table is ignored.

Subsequently this in-memory copy of the @NETWORK table entry is used to determine which servers to use whenever the table is accessed.

rec_post() and rec_kill()operations on the @NETWORK table operate exclusively on the in-memory copy of the table and are not passed on to the database. Posting to @NETWORK can change the list of server numbers and the 'local' flag. Deleting from @NETWORK causes the original default entry to be restored.

AUXDISTR determines which servers to use for an operation on a database table as follows:

  1. The in-memory copy of the @NETWORK table entry is looked up.
  2. If the value LOCN_OVERRIDE has been added to the flags passed by the application, the value of the network.local field is temporarily reversed.
  3. If the (possibly reversed) network.local field is TRUE the local server is used; otherwise the servers in the list are used.


Database Open Modes Affecting Network Layout.

This page applies to Multiple Resilience Configurations Only

Release 4.622 introduces two special flags that can be passed to the db_open() function, which cause the normal rules for initial network layout to be completely over-ridden. In theory you can still update the network layout table to change where a database is accessed, but it is not intended that you should do so. These flags are intended to provide a facility for programs which want to work with a database on one particular server to be able to do so without having to worry about how or whether AUXDISTR is loaded.

  • LOCAL_ONLY

    If this flag is specified AUXDISTR will set the initial layout for all tables so that all tables are accessed locally, and the network.local flag is TRUE for all tables. Any persistent network records which would normally be used to set the default layout for tables are ignored. However this flag will be ignored if the -nolocal command tail was passed to AUXDISTR.

  • FILE_SERVER_ONLY

    This flag is similar to the LOCAL_ONLY flag, but if the program has previously set a server for remote file operations by updating network_status.file_server field, then the initial layout for all tables is that all tables are marked remote (even if the server is in fact the local server in a combined server/client configuration) and accessed on that server only. If the program has not set a file server then FILE_SERVER_ONLY is exactly equivalent to LOCAL_ONLY.

Please note the following limitations of these two flags:

  • These flags are only effective when the process opening the database does not not already have an open database handle for the database. They cannot be used to change the initial layout for a database that the program has already opened and not yet closed.

    If you want to use these flags and are unsure as to whether the database is already open, you should either open the database, close it, and then open it again, or you should obtain a new connection handle to the database manager and open the database using that handle.

  • If you delete the default network layout record (the table 0) record, the default layout will revert to that specified by the command tails passed to AUXDISTR, but AUXDISTR will continue to ignore persistent network layout records. If you want to revert to normal operation then you should close the database and re-open it without using these special flags.

  • Updating the network status table after the database has been opened has no effect on the server to which database operations are directed when you specify FILE_SERVER_ONLY. All database operations are directed to the file server that was in effect at the time the database was opened.

Both these flags are ignored except by AUXDISTR.


Resilience - Implementation Details

The information in this section applies only to Multiple Resilience Configurations, and to a lesser extent, to Local Database Resilience Configurations. It explains some of the changes to the operation of DP4 when Resilience is in effect.

Database Reads

The information on database Reads applies only to Multiple Resilience Configurations.

Timestamps

All records read via AUXDISTR are returned to the user with a zero timestamp. This has the effect of making it impossible to update a record updated by another user since the start of the reading programs transaction. This can result in commit failures that would not arise in a standalone or simple client/server configuration. For details of this effect see Effect of Networking on Commit Failures.

Reads from Multiple Servers

If a table is set to be accessed on more than one server AUXDISTR will normally only read any particular record from one server. However, there are two situations where internally the record is in fact read from multiple servers:

  • A fetch is always sent to all servers for which there is not yet a transaction in process. Transactions are started by any database read or update operation on that server, and ended by a commit or rollback.
  • If the record is to be read with the LOCK or UNLOCK flags.

The details of how AUXDISTR selects the server to read from when the read only needs to go to one server depend on the release of AUXDISTR in use:

  • Up to release 4.620 AUXDISTR selects a server at random on every read where more than one server could be used. This can cause problems with secondary indexes even if the secondary index is unique. This problem is described in AUXDISTR and Secondary Indexes. AUXDISTR tries to remedy this defect using the "Block Fetch Optimisation", but this is not totally effective. Up to release 4.620 secondary indexes are unreliable where a table can be accessed on more than one server. In fact patch releases of AUXDISTR for 4.619 and 4.620 are available which work like the 4.621 release described next.

  • From release 4.621 and in patch releases of 4.523/4.619 dated 11 July 2003 or later, AUXDISTR works in quite a different way:

    • If one of the servers in the list is the local server (i.e. it matches a value specified in the -server command tail to AUXDISTR), that server is always used to access the table.

    • Otherwise, each table area in a calling program is assigned to a fixed server at random. For example if you have two copies of a customer table, and the customer table is configured to be accessed on two servers one copy may always be fetched from one server, and one from the other, or it may happen that both are accessed on the same server. The server to be accessed is determined using an index into the list of possible servers based on a random number assigned when the database is opened, and the memory location and table number being read. Closing and reopening the database changes the random number and hence may change the server. Updating the network layout table will cause a different server to be accessed if the server in the chosen position is changed.

      If the server first selected is offline, AUXDISTR will try the remaining server(s) sequentially from the chosen starting position.

    The motivation for this change is to make secondary indexes work reliably. However, the problem described in AUXDISTR and Secondary Indexes may still arise if a server goes offline between successive calls to fetch(NEXT) or fetch(PREV)

Update operations

Update operations involving a table (rec_post(),rec_kill() and rec_autoinc()) held on more than one server are broadcast to all the relevant machines. Usually this can only apply to Multiple Resilience Configurations. However, if the Local Database Resilience using the -nofail error handling option is in effect, updates destined for the remote server are initially posted locally as well. At commit time, such updates are discarded from the local database unless the remote server is offline.

Commit operations

This section applies to both Local Datase Resilience and Multiple Resilience Configurations.

In a Resilience Configuration network failures can give rise to commit failures. This is described in Effect of Networking on Commit Failures.

If the updates performed during a single transaction are distributed between more than one server, DP4 needs to invoke special handling for committing the transaction. This special handling is described in Implementation of Commits.


AUXDISTR and Secondary Indexes

There is a potential problem when using the multiple server resilience software reading data in tables that may be fetched from more than one server via secondary indexes.

Internally every record on a database is allocated a unique identifier (sometimes referred to as a row id). This identifier is added as an extra invisible key component to every index entry. Whenever a record is fetched from a DP4 database the row id of the record is remembered. If the next operation is a fetch(NEXT) or fetch(PREV) the saved row id is used as part of the key. Normally this ensures that if a program is reading through a table on a secondary index each record will be returned exactly once, even if there are several records all with apparently the same key.

The saved row id is not used if the index actually used to read the data is the primary index for the table (which will be the case if the record is fetched on the main index, or a secondary index for which the key is a leading subset of the primary key). In this case the database manager sets the row id to a suitable value to ensure that the correct record is found.

If a table is accessed on more than one server this algorithm may break down when reading a table on a secondary index. For example suppose a customer table contains the following data:

Suppose a customer table contains the following data:

Number | Name | Row ID on server 1 | Row ID on server 2
1 | ALBERT | 1 | 6
2 | JONES | 2 | 2
3 | JONES | 3 | 3
4 | JONES | 4 | 5
5 | JONES | 5 | 4
6 | SMITH | 6 | 1

and that the application program attempts to search through the data using a secondary index with the customer name as key.

If all the fetches are directed at server 1, the orders are returned in the order 1,2,3,4,5,6. If all the fetches are directed at server 2 the orders are returned in the order 1,2,3,5,4,6. However, if the fetches are divided up so that the first fetch is directed to server 2, the second to server 1, and the third and others to server 2 again the records will be returned in this order 1,2,6. Records 3,4,5 will be skipped, because when server 2 sees its second fetch it has a saved row_id of 6 which is greater than the row ID for any of the JONES records. In other scenarios it is possible to find the same record being fetched twice - once from one server and once from the other.

Previously this page advised that non-unique secondary keys should be modified so that they are in fact unique. Usually this involves adding the primary key as additional keys on the secondary index. In fact this does not solve the problem, although it certainly reduces it to some extent. The DP4 database manager should recognise such indexes as a special case and ignore the stored row id in the same way as for primary indexes. However, at the time of writing no such enhancement has been made.

In 4.621 and in patch releases to 4.619/4.620 AUXDISTR no longer switches between servers in the same way, and so the problem cannot arise, except in the event of a server going off line. If the table can be read locally the problem will never arise with 4.621 (because the local server cannot go offline).

If the table is not accessed locally the only complete solution to this problem is to switch tables to be read on a secondary index to use one server only. If the index is unique passing the value NO_FINDLIST as an additional flag to fetch will cure the problem at the cost of some performance degradation. However the DP4 database manager should be enhanced to automatically determine if the saved row id is valid, especially in view of the fact that the NO_FINDLIST flag is not available in QAB.


Implementation of Commit Operations

The db_checkpoint(), db_commit() and db_update() functions are implemented in so that, as far as possible, if updates have been made to the database on more than one machine, these updates are either all committed together or all fail together.

Commit operations are normally atomic. When more than one server needs to be updated commit becomes a two phase process. In the first phase all servers (including the local machine) are asked if the commit should succeed. If any server reports that the commit should fail (for example because a record posted only to that server causes a contention error, or violates a database constraint), then the transaction will be rolled back on all servers. If all servers report that the commit should succeed then the second phase of the commit is invoked on all the servers to update all the databases. It may happen that a server goes offline between the two phases of a commit operation. A commit is considered to have succeeded if all "vital" servers are still online at the end of the commit operation, or, in the case where there are no vital servers, if at least one server is still online.

Transactions that update multiple servers are more expensive than single server transactions because of the overhead of synchronisation between the servers. Therefore they should be avoided where possible. Prior to release 4.520 the implementation of split server transactions was unreliable and could cause various internal errors to be generated, or even violate the normal serialisation rules, (which should prevent a process seeing partial transactions of other processes). This is no longer the case, and split transactions can be used where necessary and appropriate.

With the exception described in Error Handling Options for Local Database Resilience, parent and orphan checking and child deletion are carried individually by each database server. No cross-database checking or child deletion is done.

In a resilient network configuration, additional commit errors may be encountered as described here.


Effect of Networking on Commit Failures

Network failures

This section applies to Local Database Resilience and Multiple Resilience Configurations Only

If a commit failure occurs due to network failures encountered by the DP4 networking software and the resilience mode means that a network error is not generated, then the global variable fail_code will be given the value COMMS_FAILURE.

Additional Commit Failures with AUXDISTR

This section applies to Multiple Resilience Configurations Only

An unexpected feature may be noticed when running programs in a multiple resilience configuration. The effect is that commit failures may occur at times when they would not have occurred with a single server. Consider the following scenario:

  1. User 1 reads record R, modifies it, posts it back and commits
  2. User 1 reads another record
  3. User 2 reads, modifies and posts back the same record, and commits
  4. User 1 reads, modifies and posts back the record

In configurations without AUXDISTR running on the client there is no commit failure, but with AUXDISTR user 1 receives a commit failure in transaction D. The reason is as follows. The rule for commit failures is:

  • a failure will occur if another user has updated a record being posted during the course of the transaction. The start of a transaction is defined as the first database fetch since the end of the previous transaction (or the start of the program), as DP4 does not provide an explicit 'start transaction' command.

In the example above transaction D is regarded as starting after the fetch in stage B, and the record R has of course been modified by user 2 during this time.

If there is only one server, the server recognises that the record being posted back in D was not read until after the end of transaction C, so it does not enforce the rule so strictly. But with AUXDISTR loaded the time at which the record was read cannot be determined, because AUXDISTR zeros the timestamps of all records read in case they are being read from one server and posted to another.

The remedy is to include a spurious database update in the program being executed by user 1, which does not alter any data on the database but simply marks the start of the next transaction.

Future versions of DP4 may cure this problem, as the current behaviour can be very annoying. Unfortunately fixing it is not likely to be easy, as unless the timestamp is set to zero there is a slight possibility of not generating a contention when one should be generated. In fact the behaviour may be marginally worse currently than with very old releases of AUXDISTR, as they passed some calls to fetch() directly to the database manager, and did not zero the timestamp.


DP4 Networking Components

The following table lists the various programs that are required to set up a networked DP4 configuration, and other network related DP4 components described in this manual.

Name | Network Type | Purpose
AUXDISTR | All | Routing program for Multi-Server configurations.
DOS2.EXE

All (MS-DOS only)

 Provides skeleton local DP4 functionality needed by AUXDISTR where there is no local database. Used with AUXDISTR -NOLOCAL.
dbdaemon | All (Unix/Linux) | Program Loader for Unix platforms - responsible for starting and stopping components used as part of the DP4 service, including the database manager and DP4 networking components. This component, introduced in release 4.525, performs essentially the same function as srvw32/srvwce do on Windows platforms.
DP4SRVR.W32 or DP4SRVR.WCE | All | Database manager for Win32 platforms - handles operations on DP4 databases on the machine on which it is running

FLX5.286

Flexnet

Network Client
FLXDISTR.286 | Flexnet | Combined AUXDISTR and FLX5 Router/Network Client. Should be supplied all command tails that would normally be specified on AUXDISTR and TCP5.

FLXMGR.286

 Flexnet Network Manager

IPXMGR.NLM

IPX/SPX

Network Manager

IPXx

IPX/SPX

Network Client

NETCOPY

 All

Utility program that can:

  • Copy files between two machines connected with DP4 networking
  • Synchronise the time between two machines connected with DP4 networking

NTBMGR

Netbios

Network Manager

NTBx

Netbios

Network Client

ONLINE

All

Database reconciliation program for Duplex Resilience
SRV2TINY.EXE

All (MS-DOS only)

 Cut down database manager. Provides read only database access on local machine. Uses less memory than SRV2
SRVx

All

 Database manager for non Windows platforms - handles operations on the DP4 databases on the machine on which it is running
SRVW32.EXE and SRVWCE.EXE | All (Windows) | Program Loader for Windows platforms - responsible for starting and stopping components used as part of the DP4 service, including the database manager and DP4 networking components.
TCPMGR | TCP/IP | Network Manager
TCPx | TCP/IP | Network Client
TSRUN | Any | Used to run programs on a remote server. Primarily used with DP4 Terminal Server
TSTCPx | TCP/IP | Provides Client side for application programs running with DP4 Terminal Server

In the list above, x is an operating system specific suffix as follows:

w (.w32) = Win32 platforms
w (.wce) = Windows CE platforms
w (.exe) = 16 bit Windows
7 = OS/2
2 = MS DOS or PC DOS
3 = UNIX
5 = FlexOS
7 = OS/2

Flexnet programs can be used to connect to TCP/IP servers or requesters running on other operating systems.


Command Tails

Command Line options for DP4 network programs.

This section summarises in table form the command tails available for and network requesters and manager respectively in all the operating environments supported. For further details on individual operating environments see the respective sections of this document. A few options that are rarely if ever used now have been omitted, but are documented on the platform specific pages. The command tails for AUXDISTR and SRVn are NOT listed here (they may be added in a later version). Please consult the pages that deal with loading specific network configurations.

DP4 programs do not usually complain if they are passed a spurious command tail. Therefore it is rather easy, especially in a resilience environment, to put a command tail on the wrong program.

Tails Used on Both Network Manager and Network Requester.

Tail | Parameter | Action
-debug -debug_connect -debug_error |  | Enable diagnostics. -debug_error = error diagnostics only. -debug = complete diagnostics. -debug_connect=error diagnostics + diagnostics relating to making and breaking connections. Refer to Unix Specific information for command line options for Unix diagnostics.
-dosidle |  | Should be used on MS-DOS versions of DP4 network programs loaded in Windows DOS Boxes. See 16 bit Windows notes for more information.
-lana | lananum | Works for Win32 NetBIOS software on Windows NT. Selects the underlying network transport to be used. See NetBIOS notes for more information.
-nokeepalive |  | turn off keepalive mechanism Should be specified on both server and client when used. TCP/IP specific
-server | 1 - 65535 | server number for remote operations. Restricted to 1..99 for multiple resilience.
-socket | port number | specifies the port number to use when making a tcp connection and must be specified at both client and server. Port 5000 is used by default. Applies only to TCP/IP programs
-timeout | seconds | specifies the length of time to wait for a reply before returning an error The default timeout for status and keepalives is also based on this value, so when used the same value should be given on both manager and requester.

Network Requester command tails

Tail | Parameter | Action
-address_nn | a.b.c.d | Specify ip-address of server nn. You should not normally use this option. Applies to TCP/IP only.
-connect_timeout n

Specify number of seconds to wait for connections to be established. The default value is the same as the regular timeout. A value of 0 will cause TCPW to wait indefinitely if a connection is possible, but will usually return immediately if the server is not running. However for machines with a permanent IP address listed in DNS that are unreachable it may take up to 20 seconds for a connection attempt to fail. You may want to specify a much smaller value to guard against this, particularly if you use a poller type program that is connecting to multiple remote servers looking for updates to collect. On a LAN a connect timeout of 1 is almost always sufficient. Over the Internet it may take 3 or 4 seconds to establish a connection.

Applies to Windows TCP/IP only.
-distributed | Selects local resilience operation. Click here for further information.
-dp4serverNN | net.node.socket | Alternative to use of spx_servers section for ipxw. Applies to IPX/SPX networking only. You cannot use this command tail with TCP/IP networking.
-local_port n

Applies to TCP/IP only. Specifies a port number to use for the local end of connections. Normally the operating system will pick a port automatically, and there is no need to use this command tail. However this causes problems when connecting to FlexOS servers, and in this case it is better to use a fixed port number.

Currently only implemented on 4.621+ Windows TCP/IP clients, but might be added to Unix/Linux in future.
-multiple | Selects multiple resilience operation. Click here for further information.
-noblock turns off block fetch optimisation - see below. Does not apply from 4.621
N.B. This tail was previously documented incorrectly.
-nodetach | do not detach from console, for debugging use. Applies to Unix/Linux and 4690 OS.
-nofaster turns off block post optimisation - see below. Does not apply from 4.621.
N.B. This tail was previously documented incorrectly.
-remain
-noremain 		-noremain: stop DP4 service after all DP4 applications have terminated (Windows)
-remain: stay loaded after all DP4 applications have terminated (others)
-retry | number | number of times to try to make TCP connection before returning an error, default 3. Applies to tcp2.exe only.
-server_name | server hostname | Specifies location of DP4 server by hostname. Cannot be used with multiple resiliency. Applies to TCP/IP only. From 4.620 hostname may also be in the form #<ip_address>, for example #192.168.160.13 , though you are recommended not to use this facility unless absolutely necessary.
-threads | n | Applies to 32 bit Windows requesters only. Specify number of threads to be available for processing requests on remote servers. The default value is 1 if multiple resilience is in use, or otherwise two threads per installed processor. If you are not planning to make use of the named connection facility that allows simultaneous connections to multiple servers you may want to specify -threads 1 to disable multi-threading as it will not be beneficial.

Network manager command tails:

Tail | Parameter | Action
-allow_execute |  | enables the facility for DP4 requesters to run programs on the server (required for DP4 Terminal Server operation). This option, introduced in release 4.620, removes a potential security hazard from DP4 Server operation. can be used where security of files on the server is important.
-aux |  | Used in Multiple Resilience Combined Client/Server configuration and Named Connection Configuration
-chain | number | Used in Multiple Resilience Combined Client/Server configuration
-deadtime | seconds | see TCP/IP specific section
-detach |  | causes network manager to detach itself from the console
-hostname | host name | force the use of a specified host name. TCP/IP Specific. This command tail is not used from 4.621 as TCPMGR automatically accepts connections on any available network adapter.
-nocopy |  | disables remote file facilities (and hence prevents some DP4 utilities from working) as a protection against malicious use of DP4 to gain unauthorised access to a computer's file systems. This option, introduced in release 4.620, can be used where security of files on the server is important. Refer to Secure Operation of DP4 in the Guide to DP4 Configuration for more information.
-nonames |  | stops TCPMGR from attempting to resolve IP addresses to names for display purposes. You should probably always use this option if you expect connections to made over a private WAN or the Internet.
-online |  | This option can be used in Duplex Configurations. It causes the network manager to notify AUXDISTR at commit time when a server has come back online that should be updated.
-quiet |  | Network Manager uses a cut-down form of its display that does not use the DP4 terminal manager. This is useful if you want to see diagnostic messages from the network manager on operating systems other than Windows.
-reverse |  | Instructs tcpmgr to attempt to run another program each time a new machine is connected. The other program is specified in the onconnect entry in the [tcpmgr] section of the DP4 Configuration file. Normally the program will attempt to create a "reverse" connection using the requester machine as a server. The requester machine wlil normally be configured using the named connection configuration. This option is new in release 4.620.
-rocopy |  | prevents remote file facilities from writing to files on the server (and hence prevents some DP4 utilities from working) as a protection against malicious use of DP4 to gain unauthorised access to a computer's file systems.. This option, introduced in release 4.620, can be used where security of files on the server is important, but where the DP4 Terminal Server is required.Refer to Secure Operation of DP4 in the Guide to DP4 Configuration for more information.
-scrnsave | timeout in seconds | The display of DP4 network managers is switched to a "screen saver" after the timeout has elapsed. The normal display is restored when a key is pressed. Probably only useful on Unix
-shutdown | hh:mm, time in hours and minutes | if running in foreground mode, causes the network manager to exit if no users are logged on at the time specified
-spx_socket | number | Used to change the IPX/SPX socket number used by IPXMGR.NLM from the default of 16384.
-stimeout | seconds | See Network Status Table in Configurations Using Multiple Resilience section.
-usenames |  | Causes IPXMGR.NLM to attempt to display computer name rather than socket address.

DP4 Terminal Client command tails:

Tail | Parameter | Action
-debug -debug_error |  | Enable diagnostics. -debug_error = error diagnostics only. -debug = complete diagnostics.
-nodetach |  | do not detach from console, for debugging use. Applies to Unix/Linux
-hostname | host name | force the use of a specified host name.

Network Router (AUXDISTR) command tails

Tail | Parameter | Action
-debug -debug_error |  | Enable diagnostics. -debug_error = error diagnostics only. -debug = complete diagnostics.
-mustuse | n | Specify vital server number to access for tables using default layout. Usually specified at most once but can be specified multiple times. See Vital Servers for further information.
-noblock |  | turns off block fetch optimisation - see below. Does not apply from 4.621 N.B. This tail was previously documented incorrectly.
-nocrash -nofail -nocrash -readlocal |  | Resilience Command Tails. See Network Error Handling Options with Multiple Resilience for details.
-nodetach |  | do not detach from console, for debugging use. Applies to Unix/Linux and 4690 OS.
-nolocal |  | Specifies that there are no local databases
-prefer_vital |  | Specifies that where a table can be fetched from more than one server that AUXDISTR should use one of the "vital" servers in preference to the local machine. This option is only meaningful where there is at least one "vital" server, usually specified via the −mustuse command tail.
-remain -noremain |  | -noremain: stop DP4 service after all DP4 applications have terminated (Windows) -remain: stay loaded after all DP4 applications have terminated (others)
-server | n | Specifies that this is a combined client/server machine. See How to Load Multiple Resilience Combined Client/Server for details.
-use | n | Specify server number to access for tables using default layout. Can be specified multiple times

Automatic Reconciliation Program (ONLINE) command tails

Tail | Parameter | Action
-db | dbname | Name of application database to be brought up to date
-localfile | filename | Specify file to read on local machine instead of default transaction log
-lockup |  | Lock up local server after processing
-remotefile | filename | Specify file to read on remote machine instead of default transaction log
-server | number | Number of database server from which to copy application database
-setsessn |  | Perform equivalent of a SETSESSN -DELETE on both databases after processing

-nofaster and -noblock Tails [4.520]

The information in this section applies for release prior to 4.621. These command tails have no effect on DP4 network requesters from version 4.621, both of the optimisations to which they were relevant have been removed - block fetch optimisations should be done using the bf_() functions in C/C++ code, and are done automatically by QAB and other DP4 utilities. The effect of the old block post optimisation is now automatically enabled on TCP/IP because DP4 no longer requires an immediate response from the server for calls to rec_post() or a number of other functions.

These command tails were not documented correctly in earlier releases. Prior to 4.520 -nofaster disabled both block fetch and block (queued) updates.

The network requester contains code to "queue" updates to the database in order to reduce the number of calls across the network, and code to "block fetch" records being fetched with a sequence of rec_fetch_main(NEXT) or rec_fetch_main(PREV) calls. This code can be disabled using -nofaster. The block fetch can be disabled with -noblock. Normally both these command tails are supplied to the network requester. When multiple resilience is used, the -noblock command tail should be specified on auxdistr instead. From 4.521/4.617 onwards, queued updates are automatically disabled when auxdistr is in use.

It is suspected that the former feature no longer improves performance significantly - (except during batch updates, where it can be quite effective) . The block fetch feature is potentially beneficial. However applications can perform block fetching more efficiently for themselves using the bf_fetch() family of functions. QAB does this for multiple choice access and also for sorts. However, these features are retained until such time as we can test a wide range of applications on different network configurations.

You are therefore recommended to experiment with loading the requester software with and without the -nofaster command tail and to use whichever configuration performs best. We would be interested to here whether performance is better with or without -nofaster.


Platform Support

This section details the platforms for which DP4 networking is currently supported. This page is for TCP/IP networks. Information on other network protocols can be found here.

TCP/IP Networks

Clients

Win32 clients using Microsoft TCP/IP are supported on all Microsoft Win32 platforms, (the Winsock version is not significant). In theory other versions of TCP/IP such as Novell's can also be used. However problems will normally only be investigated at Itim Technology Solutions if they are reproducible on the standard Microsoft Client, and we strongly recommend our customers to use the Microsoft TCP/IP protocol.

DP4 software for Windows CE has been tested on the Cassiopeia Pocket PC (made by Casio). The same files will probably work unchanged on any other MIPS based Pocket PC. ARM based software has been tested on a variety of Compaq/Hewlett Packard devices. Intel X-Scale devices are also supported as of release 4.621. Contact us if you require DP4 software on another, incompatible CE device.

Win16 client software is supported on Windows 3.x. We do not officially support running 16 bit networking software on Win32 platforms, as there are severe reliability problems with such a configuration. It does not work at all on Windows 98 (at least for TCP/IP). It may work on Windows 95 or Windows Me. It may be better to run Win32 DP4 networking and use the WIN1632W program to allow communication from the 16 bit program. Ideally the application should be converted to 32 bit. Further notes on Win16 operation can be found here.

Unix or Linux clients. We actively support Red Hat and Suse Linux clients, SCO Open Server 5, and Unixware 7.1. In the past we also supported TCP clients for a number of other versions of Unix. These are no longer being maintained, but could be brought up to date if necessary, assuming we can be given access to a suitable development and test environment. Further notes on Unix operation can be found here.

MS-DOS clients using PC TCP/IP from FTP Software Inc, on MS-DOS 6.x. Other protocols are not supported, in particular the Microsoft MS-DOS TCP/IP is not supported.

We also support connecting to TCP/IP from 4690 OS and Generic FlexOS. Details on request.

Servers

Win32 Servers. Windows NT 4 service pack 5 or 6a is recommended, or Windows 2000 or later. NT server is not required. However it is usually easier to set up DP4 TCP/IP networking on machines that are part of an NT domain.

Unix Linux Servers. As for Clients, with the addition of the RS6000 platform. tcpmgr also exists for a number of other non Intel platforms, such as the HP9000, Sun Solaris, and DG Aviion. However, these are not actively maintained as we do not currently have access to suitable equipment.

Connection Limits for TCP/IP

DP4 Multiple Resilience configurations limit the number of servers that a client can connect with to 99. The maximum number of named connections that can be opened simultaneously is 199.

Multiple resilienece limits individual tables on a database to a maximum of 32 servers simultaneously.

Individual implementations of TCP/IP may limit the number of connections further (the limit will be the same as for the maximum number of connections supported by a DP4 server as explained below).

The maximum number of connections supported by the network manager depends on the platform. For Win32 platforms limits this value to 127 machines per server, but you can use Concentrators to allow more machines to connect to one server. Again this may be further limited by the underlying software layers. On UNIX servers the limit is the smaller of 256 and the defined value of FD_SETSIZE. This is defined in the system header files.

Itim Technology Solutions normally tests networking software with 3 or 4 clients, except after major changes when 15 to 20 clients are tested, depending on available hardware.


Platform Support - Other Networks

NetBIOS protocol servers are available for Win32 and Win16 platforms, OS/2,and MS-DOS, as are clients. We last tested the programs for these environments for release 4.520. They are no longer being actively maintained. We recommend using TCP/IP protocol instead wherever possible.

It is possible to use DP4 NetBIOS networking for NetBIOS over TCP/IP. This is not recommended, and can be tricky to set up. We have not succeeded in getting it to run with NetBIOS over IPX/SPX. More details on NetBIOS can be found here.

IPX protocol is supported for IPXMGR.NLM running on Novell 3.x and 4.x. We do not yet support Novell 5, and have not ourselves tested DP4 on it, though it is known to work. We have no plans to support any future release of Novell Netware. The only client actively supported is the Win32 client running on the Novell IPX/SPX protocol. Microsoft IPX compatible protocol is NOT supported. More details on IPX/SPX can be found here.


NetBIOS Networking

On NetBIOS networks server number as set up with -server servernr command tails on the network manager and requester are translated into NetBIOS names internally. There is no equivalent of the various naming schemes available with TCP/IP. The network manager adds its name to the NetBIOS network, and requesters talk to this name. Therefore the -server servernr command tails must match everywhere.

You are recommended not to use NetBIOS for DP4 networking if at all possible. TCP/IP networking is our preferred network protocol. On Windows XP we only support TCP/IP protocol. NetBIOS protocol networking is only tested on Windows 95 and NT4, and not on any later versions of Windows.

There are special considerations when using our NetBIOS networking on Windows NT and 95:

  • On Windows NT the NetBIOS interface may be implemented over a variety of protocols. If you select properties for 'network neighbourhood', then select the 'Services' tab and double click on NetBIOS Interface you will usually see a variety of 'Lana numbers' and network routes. When loading NTBMGR.W32 or NTBW.W32 you should specify -lana n as a command tail to select the required route/protocol. If you are using a copy of NTBMGR.W32 dated prior to 23 September 1997 this command tail is not available, and you must change the lana numbers so that lana 0 is the one you want.

    On Windows 95 the -lana command tail does not work (at least it did not for us). You have to use -lana 0 (the default). The protocol this uses will depend on the Properties of the network neighbourhood. If Windows 95 is using NETBEUI as its default protocol for NetBIOS it can talk to an NT server that has loaded NTBMGR on NetBIOS routed through Nbf. If you have selected TCP as the default protocol you can use NTBMGR via NetBT. We could not get a connection to NTBMGR via NetBIOS over IPX/SPX to work, but this may be a peculiarity of our network configuration.

  • MS-DOS clients on Win32 machines

    If you can get NTBW.W32 to work with Win32 DP4 you will be able to load NTB2 in a DOS box and run DP4 MS-DOS software, though we do not officially support this.

    If you do this you will find that DOS DP4 software runs very slowly unless you press a key, and this happens however you set the 'idle sensitivity' of the DOS box and whatever protocol you use beneath NetBIOS. The cause of this problem is that in order to avoid our code going into a 'busy loop' while waiting for a network command to complete we are calling an undocumented network idle interrupt each time we check for completion. It would appear that doing this has an extremely adverse effect on Windows 95. You should supply the -dosidle command tail to DOS DP4 network programs running in a DOS box. This forces the software to use the slightly less undocumented DOS idle interrupt instead (int 28h), and the performance is then very good.

    This command tail is available on all MS-DOS networking components, and should be used if you are running MS-DOS networking components on any version of Windows, not just Windows 95.


IPX Networking

Please refer to the DP4 Installation Manual for Novell (Novell.chm or Novell.hlp on the DP4 CD ROM) for information on installing DP4 on a Novell Network. Unfortunately that manual is somewhat out of date, and we have not had time to update it, or extract the still useful information into this section. Here are a few notes on setting up Windows clients for a Novell server, as that subject is not covered.

  • When setting up DP4 networking for Windows and IPX/SPX, please remember that only the Novell Client is supported, not the Microsoft client for Netware. This is installed using a special Novell setup program, and not by using the Control Panel.
  • The only combinations we now test with IPX protocol is IPXMGR.NLM running on Novell 3.x and 4.x, with ipxw.w32 running on Windows 95 or NT4. We do not officially support Novell 5.x, though we now know from our users that DP4 runs OK on it. We do not actively support the old IPX/SPX components for MS-DOS and 16 bit Windows, though support is available on demand. We have only done cursory tests on Windows 98, and the software appears to work but Customers have reported reliability problems with this configuration.

Setting up DP4 networking using ipxw.w32 (or the 16 bit ipxw.exe) is fairly similar to the TCPW.W32 setup. You can have an [spx_servers] section in the DP4 configuration file similar to the [tcpip_servers] section. The entries in this section are identical in purpose - they map the generated names for the various server numbers into network addresses. However the values to the right of the dp4serverXX entries are numbers of the form network.node.socket, so you need to know the address of your Novell server in this form.

This can be found by running nlist server /b on a workstation. This gives the network and node part of the address. The socket number is selected by IPXMGR and defaults to 16384, but is configurable with the -spx_socket N command tail on IPXMGR.NLM. Thus typical entries in the DP4 configuration file might be:

[startup]
1 = ipxw.w32 -server 1

[spx_servers]
dp4server01=3.1.16384

You shouldn't need any command tails on the IPXMGR.NLM. By default it will try and use socket 16384. If in doubt you can use the "spx_socket n" command tail to force a particular value to be used, say n = 20000. If you do this you must change the dp4server01 entry in the DP4 configuration file accordingly.

As an alternative you can use the command tail -dp4serverNN net.node.socket.

IPX/SPX-Specific Command Tails

The information below applies only to the MS-DOS IPX/SPX programs. The command tails listed have no meaning for the Windows IPX requesters or the Novell ipxmgr

ipx2

−RETRIES n

n is the number of times to retry the IPX broadcast to locate a DP4 server (default 3). This tail is applied to IPX2

ipxmgr.exe

−RECEIVES n | n is the number of SPX receive buffers (default 100, minimum 40, max 200).
−MEMORY n | n is the amount in kbytes of conventional memory available for buffering messages (default 64kbytes, max 256 kbytes).
−USERS n | n is the maximum number of users the system is to support (default 16, max 64).
−EXTENDED | Forces IPXMGR to use only extended memory to buffer messages.

ipx2.exe and ipxmgr.exe

−SOCKET n

n is the socket number on which the DP4 server listens for address queries from requesters. The value must be different from the value used for spx_socket.


16 bit Windows

Loading

Networking components are loaded from the command line, where available or from an icon on the desktop. The DP4 client appears as an icon on the desktop. The network server can be run in foreground mode or minimised to appear as an icon on the desktop. Name-to-address mapping mechanisms are identical to those described for WIN32 programs. Command tails may be supplied either on the command line/icon properties or in the appropriate program section in the DP4 configuration file. To specify command tails in the [program] section of the DP4 configuration file use this syntax:

[tcpw]
tcpw=-server 99 -timeout 20

Note If any command tails are supplied as part of the command line, the program section in the DP4 configuration file is not processed at all. You cannot mix the two methods by specifying some tails on the command line and some in the DP4 configuration file.

The required network components can be added to the startup group so that they are loaded automatically. If a network server is to be started automatically from the startup group, only tcpmgrw.exe should be added to the startup group and the autoload facility must be enabled in the DP4 configuration file.

Unloading the Network Software

The client, tcpw.exe, can be unloaded by clicking on the icon and choosing the close option. This will only succeed if no DP4 applications have an active connection at the time. The DP4 shutdown program can also be used.

The network manager and database manager must be closed separately in the normal way. If a network server is closed down, network clients with active connections will generate a Fail 7 network error when they next access the server.

Diagnostics

tcpw.exe and tcpmgrw.exe (and the corresponding programs for NetBIOS and IPX/SPX) accept the -debug and -debug_error command tails. Diagnostic output is limited to screen output via the DBWIN utility (supplied with Visual C 1.52 or the Windows SDK).

Command Tail specific to 16 bit Windows Network Requesters

-ALLOW_CLOSE n where n is 0, 1 or 2

  • n = 0 gives the default behaviour described above. tcpw (and ipxw,ntbw) ignores the close command if any DP4 applications are still active.
  • n = 1 forces tcpw to ask the operator to confirm that it should close if the close option is selected while DP4 applications are still active. If confirmed, tcpw exits and any DP4 applications still active will generate an error when they next try to access the server.
  • n = 2 forces tcpw to exit immediately. Any applications still active will generate an error when they next try to access the server.


DP4 Networking Notes for Unix and Linux

When loading the network manager on Intel-based UNIX machines two copies of tcpmgr are started. The second copy is used only when operating as a combined client-server. This second copy is not present on non-Intel machines.

The UNIX TCP network manager is a process that forks. As each client establishes a TCP connection a separate copy of tcpmgr is launched to service that client connection. At any time the number of copies running will be the number of active client connections plus two on Intel based machines or plus one on non-Intel machines.

Most other aspects of DP4 networking on Unix and Linux are very different between releases up to 4.523, and releases from 4.525, and so they are documented separately.

  • DP4 Networking Notes for Unix and Linux (up to 4.523)
  • DP4 Networking Notes for Unix and Linux (from 4.525)


DP4 Networking Notes for Unix and Linux (4.523)

Unloading the Network Software

tcp3 will unload once all DP4 applications using it have terminated unless the −remain command tail is specified. If loaded with −remain then the tcp3 process may be removed using the UNIX kill command or by running the DP4 utility shutdown

If loaded in the foreground tcpmgr can be terminated by pressing Esc when there are no users logged on. If run as a background process using the −detach command tail then the process(es) may be removed either using the UNIX kill command or by running the DP4 shut_tcp utility. If clients still have active TCP connections open, they will generate a network error, Fail 7. If the main listening thread is killed, all other threads will terminate. If a thread which has been launched to service a particular client connection is killed, all other threads continue to operate normally. There is no way to easily identify which thread is which from a listing of the running threads.

The shut_tcp program can identify the main thread and it will signal this thread to terminate. This in turn will cause all client threads to terminate. When tcpmgr threads are terminated either by the UNIX kill command or using shut_tcp they perform an orderly shutdown, logging off all database users. This does not happen if the UNIX command kill −9 is used. This command should not be used to close DP4 processes unless absolutely unavoidable.

Diagnostics

When diagnostics are enabled each thread of the DP4 networking programs produces a log file recording network activity. The file associated with the main listening thread of tcpmgr is named listenthread while each client that connects produces a file named according to its IP-address. The location of the log files can be specified by command tail (See below). The log files grow to a fixed size before being renamed with a .bak extension and restarted. The .bak files are overwritten. The maximum size that a log file will grow to can be specified. For example:
tcpmgr −server 99 −detach −debug −logpath /usr/datafit/trace −maxsize 256

In the above example each client process will produce a log file whose name is just the IP address of the client. These files will be created in the directory /usr/datafit/trace. The maximum disk space required will be 512 kb per client (256 kb for the log file and 256 kb for the .bak file)

To make it easier to decipher diagnostics from tcp3 it is a good idea to run it in a different console from DP4 applications.

Command Tails Specific to UNIX

tcpmgr and tcp3, but not tstcp3, accept the following diagnostic command tails in addition to −debug and −debug_error

−logpath path specifies the location of the diagnostic trace files, by default log files are written to the /usr/datafit directory

−echo echoes diagnostic output to the screen. In this case you should use the −quiet command tail to suppress the normal TCPMGR output, and not use the −detach command tail.

−notiming turns off timing information in the log files. (In post 4.523 releases this is the default, and you need to specify −timing to turn timing information on.)

−maxsize n limits diagnostic files to a maximum size of n kb, default 128.


DP4 Networking Notes for Unix and Linux (4.525)

Unloading the Network Software

In 4.525 tcp3 will unload once all DP4 applications using it have terminated if it is loaded with the −noremain command tail. The tcp3 process may be removed using the UNIX kill command or by running the DP4 utility shutdown, or, assuming tcp3 was loaded under dbdaemon, by running dbdaemon -stop or killing dbdaemon.

If loaded in the foreground tcpmgr can be terminated by pressing Esc when there are no users logged on. If run as a background process using the −detach command tail or under dbdaemon, then the process(es) may be removed either using the UNIX kill command, or , assuming tcpmgr was loaded under dbdaemon, by running dbdaemon -stop, or killing dbdaemon). If clients still have active TCP connections open, they will generate a network error, Fail 7. If the main listening thread is killed, all other threads will terminate. If a thread which has been launched to service a particular client connection is killed, all other threads continue to operate normally. It may not be possible to identify which thread is which from a listing of the running threads, but if the -display_address command tail is used, the ps information may allow you to do so.

When tcpmgr threads are terminated they perform an orderly shutdown, logging off all database users. This does not happen if the UNIX command kill −9 is used. This command should not be used to close DP4 processes unless absolutely unavoidable.

There is no equivalent of the shut_tcp command supplied up to release 4.523

Diagnostics

From 4.525 diagnostics produced by DP4 Networking programs are sent to stdout, and the programs no longer do their own diagnostic log management. If you want to collect diagnostics from DP4 network programs run them under dbdaemon, and enable error logging using errlog. See Diagnostics in the Guide to DP4 Configuration for more details.


OnConnect

[TCPMGR]
onconnect=

Processed when TCPMGR is loaded with the -reverse command tail, and a machine establishes a new session with TCPMGR.

The value specified should be a command line for TCPMGR to execute (locally). TCPMGR appends the IP address of the incoming connection to the command line and executes the specified commmand.


DP4 Thin Client (Terminal Server)

Introduction

Typical DP4 components For many years DP4 has had an extremely well modularised and portable structure which lends itself to distributed processing. Until recently this has primarily been used to create applications with a typical client/server architecture with a database distributed between one or more servers, and perhaps also held locally on each client to provide resilience. The typical DP4 components are shown in the diagram on the left.

With the decreased cost and increasing power of PC based servers, as well as the increasing complexity and cost of deployment of PC based software a new type of solution has become popular - "Thin Client", or "Ultra Thin". In this architecture the absolute minimum of software is deployed on client PCs and applications run primarily on the server with the client computer acting more or less solely as a terminal (possibly a very sophisticated one).

In fact this largely represents a return to the typical model of computer systems before the invention of the PC. Thin Client solutions should offer cost savings through reduced cost of deployment and administration, and savings on hardware costs through reuse of legacy hardware for clients and reduced cost of new hardware.

From version 4.523/4.619 DP4 can run in this type of configuration. Existing applications are able to run with little or no modification using the new DP4 Terminal Server.

Existing Thin Client Technologies Compared with DP4 Terminal Server

The original thin client software was probably Telnet - this allowed any computer to act as a dumb terminal to any other computer running as a Telnet server to which it could connect. Telnet is still a very useful program - its primary disadvantage is that it can only run text based programs. Later on the Unix X Windows system allowed GUI based programs to run in a similar fashion, though this does not seem to have become very commonly used as yet (at least in the commercial environment).

In recent years two more types of technology have emerged:

  • Windows Terminal Servers - the various Citrix products, the Terminal Server Edition of Windows NT, and latterly the Terminal Services built into Windows 2000.
  • Web based architectures such as Java, ASP etc.

DP4 systems can already be deployed using most, if not all, of these technologies. However the DP4 Terminal Server architecture has unique advantages over all of them. Before going into this in more detail a few terms will be defined:

  • The Application Host is the server class computer where programs executed with the DP4 Terminal Server run.
  • An MS Architecture Terminal Server is a server that runs as a Windows Terminal Server using one of the products mentioned above.
  • The Client Machine is the computer where an application executed with the DP4 Terminal Server interacts with the user and the operating system - i.e. where screen output is displayed and keyboard and other input is captured. Normally printing will be done and non database files accessed on this computer though this will be configurable at the system, per client and application level.

Here then are the advantages of the DP4 Terminal Server Architecture:

  • Cross Platform Portability

    The DP4 Terminal Server will be based around the existing DP4 API. As such an application might be running on a Unix or Linux based Server, but the client machine might be running Windows 9x,Windows CE (or even Windows 3.1). To the user on such a device the application will appear exactly like an ordinary Windows application. In due course GUI execution on Linux will be supported. Text based execution is also be supported - a major strength of the DP4 API is that is almost completely indifferent to the type of display device in use. Thus an existing text based Unix based QAB system will automatically use the mouse correctly when run with a GUI DP4 Terminal Server.

  • Cost

    The Application Host will need to be sufficiently powerful to run the expected maximum number of DP4 programs simultaneously. However typically a machine previously acting as a DP4 server in a client server architecture will already be sufficiently powerful to run as the application host, though possibly some extra RAM will be required, particularly if a large number of different executables are used. Typically the additional processing required for running the application programs on the server will be largely offset by reduced network traffic, though this will of course depend on the application to some extent. An important point to note is that no special operating system version will be required for running as a DP4 Application Host - in particular there will be no need for the machine to run Windows NT/2000 Server rather than workstation (unless it is felt that additional CPUs are required beyond the two supported by NT Workstation). If desired the Application Host can run one of the MS Architecture terminal server operating systems so that a mix of client architectures can be used. Deploying an application using the DP4 Terminal Server will offer the benefits in terms of reduced cost of deployment typical of Thin Client Solutions. In particular upgrading a system and making database changes should be much easier in a Terminal Server environment.

    Any PC capable of running DP4 in client server mode can be used as a client. An absolute minimum spec would be a 386 DOS PC with FTP TCP/IP protocol, or a Windows 3.1 PC with Microsoft TCP/IP. Any higher spec will be sufficient. Non PC hardware such as Winterms will not be supported, though these can of course be used if the Application Host is also an MS Architecture Terminal Server.

  • Extensibility and device support

    With most existing Thin Client architectures programs typically have no access to devices on the client. For example in the Telnet client there is no facility to save a file from the server to the client and the FTP program must be used instead. Access to specialised hardware such as retail devices is out of the question. In Web Based architectures it may be possible to gain access to the local machine, but typically there are difficult security issues to be addressed, and unpleasant cross language software has to be written - for example to bridge from Java classes to an existing device driver with a C/C++ interface. In some cases even if access to devices is possible existing devices drivers may have to be completely rewritten.

    With the DP4 Terminal Server is possible to use an existing OSDI more or less unchanged, as the OSDI is an interface between the DP4 terminal manager and the hardware, and not between the application and the hardware.

  • Performance

    The DP4 Terminal Server architecture should offer improved performance compared with Remote Display Protocol type servers (X Servers and MS Terminal Servers). This is because the DP4 API is much more abstract than most RDP architectures. For example when a user is selecting options from a menu an MS Terminal Server will typically have to send many messages across the network for the client to to tell the application server where the mouse is, and from the server to the client to repaint the display. In the DP4 architecture this will be accomplished by a single API executed entirely on the client, with a one time transfer of the text in the various menus from the database on the server. In this respect the DP4 architecture is more like a Web architecture where client side processing reduces the load on the server. On the other hand the DP4 architecture does not suffer from the high round trip cost for going back to the server inherent in current web based architectures. The vast majority of processing is carried out on the most appropriate computer - display, editing and device handling on the client, database and business logic on the server.

Architecture

To understand the DP4 Terminal Server Architecture it is only really necessary to understand the basics of the regular DP4 architecture.

A pure DP4 application program never interacts with any devices on the computer directly. All such interaction is done via one of two interfaces - the DP4 database interface, and the DP4 terminal interface. The DP4 database interface implements the calls necessary to access and update a DP4 database. The DP4 terminal interface implements a set of APIs which is sufficient for all typical application needs - it contains calls to display screens, collect data from the user, access and create non database files, and print to regular or special printers or files. Specialised hardware is via the terminal interface accessed using abstract devices.

The word interface was used advisedly above. It is common to speak of a program "calling" the database manager, but this is not strictly accurate, it is an interface that is called, and the components that implement that interface can be extremely varied - anything from a standalone DP4 database residing locally through to a mixture of DP4 and non DP4 databases residing on different servers running different operating systems on different and incompatible processor architectures and accessed through differing network protocols. The application need not care about the implementation of the DP4 database at all.

Internally this is achieved by defining a simple protocol which is common to all calls to the database interface. This protocol is completely independent of the operating system and hardware used by the program. All calls to the DP4 database interface work by constructing a simple parameter block which describes the function to be called and the data to be passed in and out of the function. In the simplest case calling the interface will send some kind of message directly to the DP4 database manager. In a more complicated set up the same message may be picked up and passed through one or more ADC components and then passed into one of the DP4 network requesters. This then communicates with a DP4 network manager (receiver) running on another server and transfers to it the parameter and data associated with the call. The call can then be handed on to the DP4 database interface on that server. From the point of view of whatever is implementing the DP4 interface on the server computer the call is more or less identical to one made by a DP4 application running locally - so the same process may be repeated to pass the call through several more layers. Eventually (usually almost instantaneously) the destination will be reached, the call will be processed and the answer will be returned to the calling application, which will have no idea where or what actually did the processing. The DP4 networking components function much like software equivalents of electrical extension cables and adapters that convert British to Continental plugs or vice versa. (A slightly better analogy might be with surge protector type adapters or UPS as the DP4 networking software also contains logic to deal with network problems of various types)

Diagram of DP4 Terminal Server architecture The DP4 terminal interface is logically identical to the DP4 database interface. In exactly the same way a parameter block is constructed to describe the function to be called and its input and output data. Indeed the format of the parameter block is compatible, and in principle exactly the same network components can be used to transfer a call from one machine to another. (The same interface is now also used for calling the DP4 SQL Engine and will be used for any future interface components). The implementation of the DP4 terminal server, shown in the diagram on the right, is almost as simple as replacing the regular DP4 terminal manager component with the existing network requester and making the existing DP4 network receiver accept and pass on Terminal Interface calls as well as Database Interface calls.

In fact things are not quite as simple as this as there do need to be some additional "components" and there are some minor changes required to the regular DP4 network architecture. For example the client machine has to have some program running that is capable of starting an application on a remote server, and at the same time creating an instance of the "terminal receiver" that will execute its client side code. Whereas the regular DP4 network requester is completely single threaded a separate "terminal requester" is created for each application (since otherwise all the applications on the server could be blocked if a remote user went to lunch with a DP4 program sitting in a data entry screen). The regular DP4 Network Receiver is used as the program which processes requests from clients to launch applications. Each application is supplied with an environment variable that tells its terminal requester which client machine to connect to.

The diagram shows the networking components as using TCP/IP protocol. In principle there is no reason why other protocols could not be supported.

Implementation

The Terminal Requester is built into the regular DP4 Terminal Manager. An application will automatically select whether to use the regular terminal manager code, or the requester version. Thus there is no problem with running DP4 applications locally on the application host.

No additional components are required on the application host. On the client side you can choose either to install a simple client configuration (which comfortably fits on a single diskette) without installing any other DP4 components, or you can install the client as an additional package. This might be convenient where you want to deploy some, but not all of an application locally. You can run almost any mix of local,client-server, and DP4 terminal server applications all at once, connecting to as many servers as you want!

The two special components for the client side are :

  • The program to launch applications on the application host. This is a simple command line utility which can easily be replaced by a customised version - you can launch applications yourself by directly calling the same API that TSRUN uses.

  • The terminal receiver program TSTCPx. This is the program that communicates with the remotely running application. An initial copy of this program must be loaded as part of the DP4 service, (which will be started automatically when a request to start a DP4 program is made).

In addition to this a skeleton DP4 installation is required - the install program for the client will do this automatically. For example on Linux the files tcp3,trm3 and userdata.sys are required. On Windows a few additional DLLs are also required and the DP4 service program srvw32.exe . The OSDI must be installed on the clients if required.


Vital Servers: The -mustuse command tail

From version 4.523 a previously undocumented command tail −mustuse n is reintroduced and officially supported. This command tail was available prior to version 4.521 but never documented. It was removed from that version as it was believed it was never used, but in fact some systems rely on it.

The −mustuse n command tail is similar to −use n - it adds the specified server to the list of servers used by tables following the default layout, but it affects the operation of commit()/checkpoint(): servers specified with −mustuse are considered to be "vital". If any update in a database transaction tries to update a "vital" server that is offline, then the following commit() will definitely fail (with COMMS_FAILURE), unless there are no servers at all online, and the -NOFAIL mode of operation is in use (See Network Error Handling). Note that transactions that do not update such a server may succeed even when the server is offline. Typically a system will only specify at most one "vital" server. This will function as a "master" machine to which all database updates must be sent.

Applications can use the vital_machines field in the network layout table to get or set the list of vital servers


Fail Error 7

Message

7. Network error

Comment

The additional error code is a network-specific error code, if it is less than 256 or over 280 or a DP4 network error code (listed in NETWORK.H) otherwise. Windows uses error codes above 10000 for TCP/IP errors. From version 4.620, where appropriate the network error code will be reported as an operating system error, rather than as an additional error. This allows the operating system message associated with socket related errors to be displayed, though this is only supported on Windows 2000 or later. (Other versions of Windows will display a blank or corrupt message).

Your strategy for recovering from this type of error should be to use the correct level of resilience unless you are using the named server configuration and manage your own connections.

You can attempt to carry on after this error, but will need to establish a new connection by closing the old connection and reinitialising.

Common additional error codes are:

Code | Meaning
256 | The requester believes the DP4 server is not running
259 | The application program is trying to access a server without initialising properly. This error commonly arises if a program that has encountered a network error and has been logged off as a result attempts to continue to use its "dead" connection.
264 | The network requester believes a reply from a server is invalid.
268 | The network requester and server or AUXDISTR are from incomptible releases of DP4.
10053 | Winsock - Connection aborted
10054 | Winsock - Connection reset
10060 | Winsock - Connection time-out
10061 | Winsock - Connection refused
10064 | Winsock - Host down
10065 | Winsock - Host unreachable

Other Errors in the 10000 range are documented in the Windows header file winsock.h. On Windows 2000 and Windows XP you can run FAIL 7 100nn to display the text associated with Winsock errors.
Unix socket errors will usually be found in the header file /usr/include/sys/errno.h

If you are using NetBIOS programs (NTBW and NTBMGR for example), the additional error code is a NetBIOS return code.


NETBIOS return codes

     NB_ILLEGAL_BUFFER_LEN                        1
     NB_INVALID_COMMAND                           3
     NB_COMMAND_TIME_OUT                          5
     NB_MESSAGE_INCOMPLETE                        6
     NB_DATA_NOT_RECEIVED                         7
     NB_ILLEGAL_LOCAL_SESSION                     8
     NB_NO_RES_AVAILABLE                          9
     NB_SESSION_CLOSED                            10
     NB_COMMAND_CANCELLED                         11
     NB_DUPLICATE_NAME                            13
     NB_NAME_TABLE_FULL                           14
     NB_CMND_CMPL_DEREGISTERED                    15
     NB_SESSION_TABLE_FULL                        17
     NB_SES_OPEN_REJECTED                         18
     NB_ILLEGAL_NAME_NUMBER                       19
     NB_REMOTE_NAME_NOT_FOUND                     20
     NB_LOCAL_NAME_NOT_FOUND                      21
     NB_NAME_IN_USE                               22
     NB_NAME_DELETED                              23
     NB_SESSION_ENDED_ABNORMALLY                  24
     NB_NAME_CONFLICT                             25
     NB_INTERFACE_BUSY                            33
     NB_MAX_CMNDS_EXCEEDED                        34
     NB_INVALID_ADAPTER                           35
     NB_CMND_ALREADY_COMPLETED                    36
     NB_CMND_INVALID_TO_CANCEL                    38
     NB_NAME_DEFINED_BY_OTHERS                    48
     NB_ENVIRONMENT_NOT_DEFINED                   52
     NB_NO_OS_RESOURCES                           53
     NB_MAX_APPL_EXCEEDED                         54
     NB_NO_SAP_AVAILABLE                          55
     NB_INADEQUATE_RESOURCES                      56
     NB_INVALID_NCB_ADDRESS                       57
     NB_RESET_INVALID                             58
     NB_INVALID_DD_ID                             59
     NB_SEGMENT_LOCK_UNSUCCESSFUL                 60
     NB_DD_OPEN_ERROR                             63

     NB_OS_ERROR_DETECTED                         64
     NB_PERM_RING_STATUS                          79

     NB_UNEXPECTED_CCB_ERROR                      246
     NB_ADAPTER_OPEN_ERROR                        248
     NB_ADAPTER_HANDLER_ERROR                     249
     NB_ADAPTER_CHECK                             250
     NB_CODE_NOT_OPERATIONAL                      251
     NB_OPEN_FAILURES                             252
     NB_UNEXPECTED_CLOSE                          253
     NB_COMMAND_IN_PROCESS                        255