YMODEM

YMODEM

Communication protocol
Purpose	file transfer protocol
Developer(s)	Chuck Forsberg
Introduction	1985 (1985)
Based on	XMODEM
Influenced	ZMODEM
Hardware	modems

YMODEM is a file transfer protocol used between microcomputers connected together using modems. It was primarily used to transfer files to and from bulletin board systems. YMODEM was developed by Chuck Forsberg as an expansion of XMODEM and was first implemented in his CP/M YAM program. Initially also known as YAM, it was formally given the name "YMODEM" in 1985 by Ward Christensen, author of the original XMODEM.

YMODEM extended XMODEM in three ways, combining features found in other extended XMODEM varieties. Like XMODEM-CRC, YMODEM replaced the 8-bit checksum with a 16-bit cyclic redundancy check (CRC), but made it the default form of correction instead of optional. From TeLink it added the "block 0" header that sent the filename and size, which allowed batch transfers (multiple files in a single session) and eliminated the need to add padding at the end of the file. Finally, YMODEM allowed the block size to be increased from the original 128 bytes of data to 1024, as in XMODEM-1k, which greatly improved throughput on faster modems.

Forsberg built the standard with all of these features as runtime options, allowing a single protocol driver to fall back to XMODEM-CRC or even XMODEM when connecting to non-YAM systems. He believed that programmers would want to implement as many of these features as possible on any given platform. He was dismayed to find that the majority of implementations were actually providing nothing more than 1k block size with CRC-16, failing to implement the "block 0" while continuing to use the YMODEM name. The result was the release of many mutually incompatible YMODEM implementations, and the use of the name YMODEM Batch to clearly indicate those versions that did support the complete standard.

Features

XMODEM

The original XMODEM was a very simple protocol and that is the reason for its success; it could be implemented on practically any machine of the era, even those with very limited processors and storage. It worked by breaking up the data to be sent into 128-byte packets, adding a 3-byte header and 1-byte checksum footer, and sending the resulting 132-byte packets out in order. The receiving computer recalculated the checksum from the 128 bytes of data, and if it matched the checksum sent in the footer it sent back an ACK, and if it did not match, a NAK. When the sender received an ACK it sent the next packet, while a NAK caused it to re-send the last one.

There were a number of problems with the protocol. The use of a simple checksum meant some common errors could go unnoticed. The small packet size and requirement to wait for the ACK or NAK led to slow performance on higher speed links or ones with significant latency. Finally, as the transfer contained no details of the file, every file had to be manually started, which could be a chore when many small files were being transferred.

Solutions to these problems were developed during the early 1980s. XMODEM-CRC replaced the checksum with a 16-bit cyclic redundancy check (CRC), which was much more resistant to common errors. XMODEM-1k expanded the packet size from 128 bytes to 1024, improving performance on higher-speed connections, while others, like WXMODEM and SEAlink instead introduced sliding window systems to combat both performance and latency, at the cost of some complexity. Still others, like TeLink and MODEM7, added file information so that a single transfer could contain multiple files, allowing batches of files to be sent with a single command.

YMODEM

Chuck Forsberg, author of the CP/M "Yet Another Modem program", or YAM, decided to write a single protocol driver that supported many features compared to XMODEM and called it YMODEM. When the users started a transfer they could indicate which options they wanted on the command line, for instance, saying they wished to use CRC. The protocol was written so that it would attempt this style, but gracefully fall-back to match whatever capabilities the remote software did implement.

Abort

One problem with the original XMODEM was that there was no defined way to abort the transfer once started. The normal solution was to send NAKs to every subsequent packet if the user requested it. Since the XMODEM protocol defined a limit of ten NAKs to abort a send, and each packet might take a second to send, this meant there was a ten second delay where the sender continually sent data that was simply ignored.

Some implementations had added the ability to send a CAN instead of ACK or NAK at the end of a received packet to indicate an abort. Unfortunately, there was the possibility that a CAN could be generated by line noise and trigger an abort. YAM thus modified this slightly to require two CANs back-to-back, which would immediately perform a "graceful abort" on the sender end.

CRC

CRC support had been introduced in XMODEM-CRC. This was a very simple change to the original protocol; if requested, the receiver would attempt to trigger the transfer by sending an initial C instead of a NAK. If the remote sender supported the CRC option, it would begin sending packets as normal, but with a 16-bit CRC in the footer rather than the 1-byte checksum. YAM supported this option with no changes.

1k

1024-byte packets had been introduced in XMODEM-1k. This version did not change the trigger character from the receiver, so there was no way for the sender to know whether the receiver supported larger packets. Instead, XMODEM-1k was presented as a separate protocol on both ends of the connection. When such a connection was started, the sender could choose to send either 1024 bytes in a packet or 128, indicating the larger with an STX character in the header rather than the normal SOH. Normally only the last few packets would use the smaller packets, to avoid sending large amounts of padding. 1k also assumed CRC for all connections. YAM supported 1k with no changes.

Zero packet

In order to support automated transfers of FidoNet mail, MODEM7 introduced the ability to send the filename as plain text before sending the first block of data. This was not reliable, and TeLink improved this by placing the filename, and optionally other data like the creation date and file length, in a complete 128-byte packet. XMODEM started transfers with packet number one, so TeLink sent this packet as number zero. This "zero packet" or "block zero" became common in other FidoNet systems like SEAlink and others.

YAM supported the zero packet format, but it was ignored by many 3rd party implementations of YMODEM. When one implementation attempted to send the zero packet to a non-aware version, the receiver would naturally NAK the packet, as packet zero is illegal. The sender would then see the NAK as a transmission error and try to send the packet again, attempting this ten times before failing.

For reasons that are not entirely clear, many implementations of YMODEM did not implement this feature. Because they were unaware of it, they sent a NAK, triggering a series of resend attempts before failing out. This meant that if the user chose to use a compliant YMODEM with a non-compliant version, the transfers would fail. Nevertheless, such non-compliant versions were common.

As a result, it was common to see both YMODEM and YMODEM Batch listed as two separate protocols. Further confusion was created by the similarity between XMODEM-1k and these non-compliant YMODEMs, which were similar to the point that they were often incorrectly listed as the same.

Streaming support

YMODEM-G is a streaming variant used for error-free connections. It does not wait for an ACK to be received before sending the next packet; XON/XOFF is used for flow control. The protocol is faster than YMODEM because no latency is introduced between packets, but has no capability for correcting errors. It is reliant on the underlying connection being error-free, which is the case for modems supporting MNP for instance.

Normally, a YMODEM transfer would be started by the receiver sending a C to indicate it wants to use the 128-byte format with CRC, or NAK if it wishes to use the original checksum system. When g-protocol is desired, the transfer is instead triggered by sending a G. If the sender does not support the g-protocol, it considers this to be an error and ignores it, but if it does support g it begins sending packets in a continual stream. It only expects a single ACK after the final packet is received, which is indicated by the presence of an EOT character in the data. YMODEM-g assumes 1k packets are available.

However, despite this protocol potentially being faster than ZMODEM it was still rarely used. This was partially due to the lack of other functionality, but also a more serious issue. Before the emergence of the 16550 UART, there was a substantial risk of buffer overrun on the serial port. Although this would be detected by YMODEM-g it could not be corrected since no block re-transmission is possible. The receiver would have to cancel and restart the whole transmission from the beginning. ZMODEM on the other hand has transfer resume capability which made it more appealing.

References

• XMODEM / YMODEM Protocol Reference by Chuck Forsberg, October 10, 1985
• XMODEM / YMODEM Protocol Reference by Chuck Forsberg, June 18, 1988 (document reformatted October 14, 1988) (HTML version with text issues)