Soft hyphen

From Wikipedia, the free encyclopedia
Jump to: navigation, search

In computing and typesetting, a soft hyphen (ISO 8859: 0xAD, Unicode U+00AD soft hyphen, HTML: ­ ­) or syllable hyphen (EBCDIC: 0xCA), abbreviated SHY, is a code point reserved in some coded character sets for the purpose of breaking words across into lines by inserting visible hyphens. Two alternative ways of using the soft-hyphen character for this purpose have emerged, depending on whether the encoded text will be broken into lines by its recipient or has already been formatted by is originator.[1][2][3]

Use in text to be formatted by the recipient[edit]

The use of SHY characters in text that will be broken into lines by the recipient is the application context considered by the post-1999 HTML and Unicode specifications, as well as some word-processing file formats. In this context, the soft hyphen may also be called a discretionary hyphen or optional hyphen. It serves as an invisible marker used to specify a place in text where a hyphenated break is allowed without forcing a line break in an inconvenient place if the text is re-flowed. It becomes visible only after word wrapping at the end of a line. The soft hyphen's Unicode semantics and HTML implementation are in many ways similar to Unicode's zero-width space.

To show the effect of a soft hyphen in HTML, the following words have been separated with soft hyphens:

Margaret­Are­You­Grieving­Over­Goldengrove­Unleaving­Leaves­Like­The­Things­Of­Man­You­With­Your­Fresh­Thoughts­Care­For­Can­You­Ah­As­The­Heart­Grows­Older­It­Will­Come­To­Such­Sights­Colder­By­And­By­Nor­Spare­A­Sigh­Though­Worlds­Of­Wanwood­Leafmeal­Lie­And­Yet­You­Will­Weep­And­Know­Why­Now­No­Matter­Child­The­Name­Sorrows­Springs­Are­The­Same­Nor­Mouth­Had­No­Nor­Mind­Expressed­What­Heart­Heard­Of­Ghost­Guessed­It­Is­The­Blight­Man­Was­Born­For­It­Is­Margaret­You­Mourn­For

On HTML browsers supporting soft hyphens, resizing the window will re-break the above text only at word boundaries, and insert a hyphen at the end of each line.

HTML4 describes it as a "hyphenation hint", though it suggests that that interpretation is not universal:[4]

In HTML, there are two types of hyphens: the plain hyphen and the soft hyphen. The plain hyphen should be interpreted by a user agent as just another character. The soft hyphen tells the user agent where a line break can occur. Those browsers that interpret soft hyphens must observe the following semantics. If a line is broken at a soft hyphen, a hyphen character must be displayed at the end of the first line. If a line is not broken at a soft hyphen, the user agent must not display a hyphen character. For operations such as searching and sorting, the soft hyphen should always be ignored.

Use in text formatted by the originator[edit]

The SHY character is also being used in text where paragraphs have already been broken into lines, such as certain plain text files, text sent to VT100-style terminal emulators or printers, or pages represented in page description languages. This is the application context originally considered by the EBCDIC and ISO 8859-1 standards and implemented in many VT100 terminal emulators.[1][2]

Here, SHY is a visible hyphen that is usually visually indistinguishable from a regular hyphen, but has been inserted solely for the purpose of line breaking. The purpose of the soft hyphen here is to distinguish it from any regular hyphen that might have been part of the original spelling of the word. This distinction helps to reuse already formatted text, when line breaks and soft hyphens inserted during word wrapping have to be removed to convert the text back into its unformatted form. For example, the copy or paste function of a terminal emulator can offer to replace line breaks with a space character, and remove any soft hyphens including any immediately following whitespace characters.

An example application that outputs soft hyphens for this reason is the groff text formatter as used on many Unix/Linux systems to display man pages.

Encodings and definitions[edit]

SHY characters in coded characters sets, roughly in chronological order:

  • EBCDIC placed a SHY character (known there as a "syllable hyphen") at position 202 = 0xCA.[1][5] IBM defined its purpose as a "hyphen used to divide a word at the end of a line [that] may be removed when a program adjusts lines."[6]
  • ISO 8859-1:1986 (Latin 1) inherited SHY from EBCDIC, but called it "soft hyphen", placed it at position 0xAD (hexadecimal), and stated its purpose as "for use when a line break has been established within a word". Other ISO 8859 parts placed it at the same position, with the exception of ISO 8859-11 (Latin/Thai), which lacks it.
  • IBM code page 850 (an MS-DOS character set covering all ISO 8859-1 characters) placed it at position 240 = 0xF0.
  • Unicode 1.0 (1991) and ISO 10646 (1993) took the first 256 code positions from ISO 8859-1, resulting in SHY at Unicode codepoint of U+00AD.
  • HTML 2 (1995) incorporated the "­" character entity from SGML, but explicitely discouraged its use.
  • HTML 4 (1999) redefined the purpose of the character as marking a hyphenation opportunity, which only becomes visible as a hyphen at the end of a line after formatting.
  • Unicode 4.0 (2002) changed the category of its SHY character from previously "Pd" (punctuation, dash) to "Cf" (other, format), thereby aligning its interpretation of the characters with that of HTML 4.

Other commands for marking hyphenation opportunities in text formatting languages (similar to the HTML 4 and Unicode 4.0 interpretation of SHY):

[7]

Security issues[edit]

Soft hyphens have been used to obscure malicious domains or URLs in e-mail spam.[8][9]

See also[edit]

References[edit]

  1. ^ a b c Jukka Korpela (Revision as of January 2011). "Soft hyphen (SHY) – a hard problem?". Tampere University of Technology. Retrieved 2011-04-08. 
  2. ^ a b Markus G. Kuhn (2003-06-04). "Unicode interpretation of SOFT HYPHEN breaks ISO 8859-1 compatibility". Unicode Technical Committee. L2/03-155R. 
  3. ^ Eric Muller (2002-08-14). "Yes, SOFT HYPHEN is a hard problem". Unicode Technical Committee. L2/02-279. 
  4. ^ "9.3.3 Hyphenation". HTML 4.01 Specification. World Wide Web Consortium. 24 December 1999. Retrieved 2011-04-08. 
  5. ^ "Extended Binary-Coded Decimal Interchange Code - S/390". comsci.us. Retrieved 2011-04-08. 
  6. ^ "Glossary". IBM. Retrieved 2011-04-08. 
  7. ^ "Commonly Confused Characters". Greg Baker, Simon Fraser University. Retrieved 2011-07-12. 
  8. ^ "Spammers Using Soft Hyphen To Hide Malicious URLs". Slashdot. October 7, 2010. Retrieved 2011-04-08. 
  9. ^ "Soft Hyphen – A New URL Obfuscation Technique". Symantec. Retrieved 2011-04-08.