Intelligent Messaging Administrator's Guide
Chapter 1 - How the Mail Service Transports Messages
This chapter describes the major concepts and features of the Intelligent Messaging™ mail service. The mail service provides mailbox functions to local users and transfers messages to remote users. This chapter describes how the mail service transports messages.
Using StreetTalk Explorer or a combination of the MMAIL, MSERVICE, and MUSER management programs, administrators can manage mailbox functions and monitor transfer processes of the mail service.
For instructions on:
Managing the mail service, refer to Chapter 2 Maintaining user mailboxes, refer to Chapter 3
On the server, two components provide mail service, as follows:
The mail service provides mailbox service to the workstation client. Mailbox service includes creating, deleting, sending, and organizing messages. When a user runs Intelligent Messaging at a workstation, the corresponding mail service (specified in the user profile) allows the user access to a mailbox. Mailbox service also includes the janitor, which empties all the Wastebasket folders at a specified time every day. The transfer agent on the same server, provides two-way message transfer between the local mail service and other transfer agents. Note If your Intelligent Messaging mail service is running on a Windows NT Server, be aware that your Windows NT Server does not act as a mail gateway. For example, you cannot use an Intelligent Messaging mail service on a Windows NT Server to send and receive SMTP mail. If your network requires a mail gateway, you must install and configure an Intelligent Messaging mail service and the gateway software on a VINES Server.
Figure 1-1 shows the relationship among the mail client, mail service, and transfer agent.
The mail service features a message store that is a database system organized by folder. For each folder, the mail service consolidates messages into a database.
Note Some mail applications allow you to query the message store database. Such an application could request a list of messages in a folder, sorted by the value of a particular envelope field. For more information, refer to the documentation that shipped with the mail application.
The following section describes how the mail service transports messages.
Transporting Messages Using Multi-Threaded Processing
When a user sends a message, the transfer agent of the mail service specified in the user profile resolves all the addresses associated with the message. Addresses can be StreetTalk:
Names List names Nicknames Patterns Remote addresses, such as Remote Electronic Mail Address (REMA) addresses
To resolve an address, the transfer agent determines the destination transfer agent to which it will transport the message, so that the message can be ultimately delivered to a recipient.
Once an address is resolved, the transfer agent transports the message as follows:
If the recipient's mailbox is maintained on the local server, the transfer agent delivers the message to the mailbox. If the recipient's mailbox is stored on another server, the transfer agent transports the message to the destination transfer agent.
The process of resolving addresses and transporting messages is multi-threaded.
Multi-threaded processing allows the transfer agent to process multiple messages at one time. Multi-threaded processing also allows messages that may be resolved and transported easily to be delivered while the transfer agent continues to work on difficult messages.
For example, if a message is addressed to one individual whose mailbox is maintained on the local mail service and to a second individual whose mailbox is maintained on a remote mail service over a slow link, the transfer agent can resolve and deliver the message to the local addressee while it continues to process the remote-destination addressee.
Using Mail Queues for Multi-Threaded Processing
To accomplish multi-threaded processing, the mail service uses multiple queues, visible as special mail folders. Each queue is a directory on the server that the mail service uses to store messages being processed by specific tasks.
Messages in a queue are distinguished from one another by their message IDs. Copies of the same message each have a message ID with the same first 10 characters and are distinguishable by a unique 11th character.
For example, if the mail service receives a message that is destined for four users on different destination mail services, the mail service makes four copies of the message. The copies are distinguished from one another in a mail queue by the last character of their message IDs, as shown in the example below:
yYv7+QFz5da
yYv7+QFz5db
yYv7+QFz5dc
yYv7+QFz5dd
The mail queues are described in the sections that follow:
Accept queue Send queue Resolve queue Transfer queue
All messages that a mail service receives from other mail services are initially received in the Accept queue. These mail messages reside here until they are moved into the first Resolve queue, Resolve0.
All messages that a mail service receives from local users are initially received in the Send queue. These mail messages reside here until they are moved into the first Resolve queue, Resolve0.
The transfer agent uses three Resolve queues to process the StreetTalk names of message recipients. They are as follows:
Resolve0 Resolve1 Resolve2
Resolve0
Messages are first received in the Resolve0 queue. The transfer agent passes through the queue, working to resolve each message for a specific amount of time. After the time limit, the transfer agent moves to the next message in the queue.
Only messages that can be resolved easily will be completely resolved in the short time that the transfer agent works on a message. For any recipient resolved within that time, a copy of the message is moved to the first Transfer queue, Transfer0.
Figure 1-2 shows an easily resolved message moving from Resolve0 to Transfer0.
Normal and Low priority messages with recipients that are not resolved within the specified time limit are moved into the second Resolve queue, Resolve1.
Figure 1-3 shows an unresolved message moving from Resolve0 to Resolve1.
High priority messages remain in Resolve0. Periodically, the transfer agent works on resolving any unresolved recipients, until the message is fully resolved.
If a message has been deferred, it resides in Resolve0 until the specified defer time. At that time, the transfer agent begins resolving the message.
Resolve1
Normal and Low priority messages that are not fully resolved in Resolve0 are received in the Resolve1 queue. The transfer agent passes through the queue, working to resolve each message for a specific amount of real time. After the time limit, the transfer agent moves to the next message in the queue.
The amount of resources the transfer agent allocates to processing a message depends on the priority of the message. More resources are spent processing Normal priority messages than Low priority messages.
For any recipient resolved, a copy of the message is moved into the first Transfer queue, Transfer0.
Figure 1-4 shows a resolved message moving from Resolve1 to Transfer0.
If the transfer agent is not able to fully resolve the message after trying each recipient in the message twice, the transfer agent assumes that the group or user exists, but is temporarily unavailable (such as, when a server is temporarily unavailable). The transfer agent moves the message to the third Resolve queue, Resolve2.
Figure 1-5 shows an unresolved message moving from Resolve1 to Resolve2.
Resolve2
Messages that are not fully resolved in Resolve1 are received in the Resolve2 queue. The transfer agent works on resolving each message periodically and after network events, such as a server coming on-line.
The amount of resources the transfer agent allocates to processing a message depends on the priority of the message. More resources are spent processing Normal priority messages than Low priority messages.
For every recipient resolved, a copy of the message is moved into the first Transfer queue, Transfer0.
Figure 1-6 shows a fully resolved message moving from Resolve2 to Transfer0.
If a message expires before it is fully resolved, it is returned to the sender with an Undeliverable status.
The transfer agent uses three queues to transfer messages.
Transfer0 Transfer1 Transfer2
Transfer0
Resolved messages are first received in the Transfer0 queue. The transfer agent uses network routing information to determine if the mail server is up as well as the network cost of transferring each message, based on the line speed between the local and destination mail services. These cost metrics are updated periodically from network routing table metrics, yet maintain historic information, such as servers that were previously on-line. The historic information is used to predict network cost when one link in a series is temporarily inactive.
Once the network cost has been determined, the transfer agent moves the message into one of the other transfer queues, depending on its cost. If a message is a low-cost message, it is moved into the second Transfer queue, Transfer1.
Figure 1-7 shows a resolved, low-cost message moving from Transfer0 to Transfer1.
If the message is a high-cost message, the message is moved into the third Transfer queue, Transfer2.
Figure 1-8 shows a resolved, high-cost message moving from Transfer0 to Transfer2.
Transfer1
Low-cost messages are received in the Transfer1 queue. If the recipient's mailbox is maintained on the local server, the transfer agent delivers the message to the mailbox. If the recipient's mailbox is stored on another server, the transfer agent contacts the destination mail service and transfers the message to the Accept queue of the destination mail service.
If the transfer agent is not able to fully transfer a message (for example, because a link terminates), the message remains in the Transfer1 queue. The transfer agent tries to retransfer the message periodically and after a network event, such as a server coming on-line.
For information on how the mail service recovers partially transferred messages, refer to "Checkpointing Messages" later in this chapter.
Transfer2
High-cost messages are received in the Transfer2 queue. The transfer agent contacts the destination mail service and transfers the message. If the transfer agent is not able to fully transfer a message, the message is placed at the bottom of the Transfer2 queue. The transfer agent retries to transfer the message every specified time period and after a network event, such as a server coming on-line.
Figure 1-9 shows some sample paths that messages take through the system as they are processed and transported.
Examples of Multi-Threaded Processing
The examples that follow illustrate Intelligent Messaging's multi-threaded processing feature. The examples show how the mail transfer agent resolves and transports mail messages.
Example 1 - Sending a Message to a Local User and a Remote User
James Smythe of WCTUS in Chicago sends a high-priority message to two addresses:
Liz Epstein, a user with the same mail service as James Nigel Waters, of WCTAU in Sydney, a user on a remote mail service located over a low-speed link
The message is initially received in the Send queue because the message was sent by a local user, James Smythe. It resides there until it is picked up by the transfer agent.
Step 1: The transfer agent picks up the message
The message is moved from the Send queue into the Resolve0 queue. The transfer agent works to resolve the message for a specified period of time.
Step 2: The transfer agent resolves both recipients
As a high-priority message, it resides in the Resolve0 queue while the transfer agent resolves both recipients. The transfer agent determines that the message is destined for two mail services.
Step 3: The transfer agent copies the message
The transfer agent makes two logical copies of the message: one for the local destination and one for the remote destination. The logical copies are linked to the original and do not use extra disk space.
Step 4: The transfer agent determines the cost
Both copies are moved into the Transfer0 queue, where the transfer agent determines the network cost of transferring each message.
Step 5: The transfer agent delivers the local message
The transfer agent determines that the message to Liz Epstein is a low-cost message to be delivered locally and moves it into the Transfer1 queue. The transfer agent then delivers the message to Liz's mailbox.
Step 6: The transfer agent transfers the remote message
The message to Nigel Waters is a high-cost message because it must be sent over a low-speed link. The message is moved into the high-cost queue, Transfer2.
The transfer agent then contacts the destination mail service and transfers the message.
Example 2 - Sending a Message to a User and a List
John Williamson of WCTUS in Chicago sends a Normal priority message to two addressees:
· Cathy McIntire, a user · Project Team@Research@WCTUS, a list
The message is initially received in the Send queue, where it resides until it is picked up by the transfer agent.
Step 1: The transfer agent picks up the message
The message is moved from the Send queue into the Resolve0 queue. The transfer agent works on resolving the message for a specified period of time.
Step 2: The transfer agent resolves three recipients
The transfer agent completes the following tasks within the time limit:
· Resolves the recipient Cathy McIntire · Enumerates the list to include these recipients: - Joe Thomas
- MaryLou Lane
- Sue Ralph
· Resolves the first two recipients of the list-Joe and MaryLou
Step 3: The transfer agent copies the message
The transfer agent makes three logical copies of the message, one for each of the following:
· Remote recipient's mail service (Cathy McIntire) · Both local recipients' mail service (Joe Thomas and MaryLou Lane) · Unresolved recipient (Sue Ralph)
Step 4: The transfer agent works to resolve the last recipient
The transfer agent moves the copy of the message destined for the unresolved recipient into the Resolve1 queue, where it works to resolve the message for a specified period of time.
Step 5: The transfer agent determines the cost
The transfer agent moves the two copies destined for the resolved recipients into the Transfer0 queue, where it determines the network cost of transferring each message.
Step 6: The transfer agent delivers local messages; transfers remote message
The transfer agent determines that the message to MaryLou and Joe and the message to Cathy are both low-cost messages. It moves the messages to the Transfer1 queue. It then completes the following tasks:
· Contacts the remote destination mail service and transfers the message · Delivers the messages to the two local recipients' mailboxes
Step 7: The transfer agent works to resolve the last recipient
In the Resolve1 queue, the transfer agent is unable to resolve the message after trying the recipient twice. The transfer agent moves the message into the Resolve2 queue where it works on resolving the message.
Step 8: The transfer agent resolves the last recipient
The transfer agent resolves the recipient, Sue Ralph, after a network event.
Step 9: The transfer agent determines the cost
The transfer agent moves the message into the Transfer0 queue, where it determines the network cost of transferring the message.
Step 10: The transfer agent transfers the remote message
The message to Sue is a high-cost message because it must be sent over a low-speed link. The transfer agent moves the message into the high-cost queue. The transfer agent then contacts the destination mail service and transfers the message.
The mail service uses a compression utility to compress messages. You can configure the mail service to trigger message compression when a message is greater than a specified size and will incur a specified network cost to be transferred. (For instructions on configuring the mail service, refer to "Configuring the Mail Service" in Chapter 2.) By default, the mail service compresses a message when the message and its attachments have a combined size of 5K or greater. The benefit of compressing only large messages is that small messages can be transferred quickly, rather than consuming time and resources to compress them.
The mail service compresses messages in both the Transfer0 and Transfer2 queues. In the Transfer0 queue, the transfer agent compresses large messages for which network cost cannot be determined (for example, when the destination mail service cannot be found). The mail service compresses these messages in Transfer0 rather than waiting until the destination service comes on-line. Then, as soon as the destination mail service comes on-line, the mail service will be ready to determine cost and transfer the message. When transient links are involved, this process increases the likelihood that the transfer agent can successfully transfer the entire message in the brief period that a transient link is up.
In the Transfer2 queue, the transfer agent compresses high-cost messages that are large. If the transient link is lost during the time that a message is being compressed, the transfer agent moves the message into Transfer0 to redetermine cost.
Before transferring a compressed message, the mail service contacts the destination mail service and determines if the two services support a common compression algorithm. Only a destination service that supports a compression algorithm compatible with that of the source service can receive the compressed attachment. If the destination service does not support a compatible compression algorithm, the source mail service transfers a noncompressed version of the message.
In addition to the message-compression triggers that are set globally on the mail service, some mail applications allow you to set a compression trigger on an individual message basis. For information about setting such a compression trigger, refer to the documentation that shipped with the mail application.
The mail service uses checkpointing to transfer messages over transient links. Checkpointing allows the destination mail service to recover partially transferred messages and attachments when there is an interruption in the transfer of the message over the link. When the link becomes complete again, the source mail service can continue the transfer, starting with the next block in the message, rather than initiating the transfer again at the first block.
Mail service logs contain information on system and service activity, including transport statistics. This information can be useful in a variety of situations, such as troubleshooting, diagnosing problems, and measuring usage.
Transport statistics are written approximately every hour to the mail service log. These statistics include the number of messages, their average size, and total size, in bytes. The statistics are divided into the following categories:
Messages sent from local users to the mail service Messages received from a remote mail service Messages delivered to local users Messages transferred to a remote mail service Checkpointed messages (number of messages only) Messages regarding the janitor
You can generate a report of service log messages. To do so, use the StreetTalk Explorer, or the MSERVICE and OPERATE programs. For detailed instructions, refer to Banyan Server Operations Guide. You can view the report on-line, or write it to a file.
