Lotus Multi-Byte Character Set

From Wikipedia, the free encyclopedia
Jump to navigation Jump to search

The Lotus Multi-Byte Character Set (LMBCS) is a proprietary multi-byte character encoding originally conceived in 1988 at Lotus Development Corporation with input from Bob Balaban and others.[1] Created around the same time and addressing some of the same problems, LMBCS could be viewed as parallel development and possible alternative to Unicode.[1] For maximum compatibility, later issues of LMBCS incorporate UTF-16 as a subset.[2][3]

Commercially, LMBCS was first introduced as the default character set of Lotus 1-2-3 Release 3 for DOS in March 1989[1][4] and Lotus 1-2-3/G Release 1 for OS/2[1] in 1990 replacing the 8-bit Lotus International Character Set (LICS) and ASCII used in earlier DOS-only versions of Lotus 1-2-3 and Symphony.[5] LMBCS is also used in IBM/Lotus SmartSuite, Notes and Domino,[1] as well as in a number of third-party products.

LMBCS encodes the characters required for languages using the Latin,[6] Arabic, Hebrew, Greek and Cyrillic[6] scripts, the Thai, Chinese, Japanese[6] and Korean writing systems, and technical symbols.

Encodings[edit]

Technically, LMBCS is a lead-byte encoding where code point 00hex as well as code points 20hex (32) to 7Fhex (127) are identical to ASCII[1] (as well as to LICS).[5]

Code point 00hex is always treated as NUL character to ensure maximum code compatibility with existing software libraries dealing with null-terminated strings[1] in many programming languages such as C.[nb 1]

Code points 01hex to 1Fhex, which serve as control codes in ASCII, are used as lead bytes to switch the definition of code points above 7Fhex between several code groups (similar to code pages) and at the same time determine either a single- or multi-byte nature for the corresponding code group.[1]

For example, code group 1 (with group byte 01hex)[1] is almost identical to the SBCS code page 850, whereas code group 16 (with group byte 10hex)[1] is similar to the Japanese MBCS code page 932. Multi-byte characters can thus occupy two or three bytes.[7][6]

In canonical LMBCS, each character starts with its group byte.[1] To reduce the length, in optimized or compressed LMBCS a default code group or optimization group code can be defined on a per application or process basis (ideally chosen according to the highest likelihood of occurrence)[1] and must be communicated to the interpreting code in some way (f.e. by specifying the corresponding "LMBCS-n" name).[8] Thereby, the group byte can be omitted for these characters.[1] Lotus 1-2-3 retrieves the optimization group code from the file header of the corresponding source file,[7] whereas for Lotus Notes the optimization group code is fixed to be always 01hex.[2][7]

Default Group Bytes Description
N/A 00hex 1[7] NUL
LMBCS-1 01hex 2[7] Code page 850 (DOS Latin-1)[2][7]
LMBCS-2 02hex 2[7] Code page 851 (DOS Greek)[2][7]
LMBCS-3 03hex 2[7] Code page 1255 (Windows Hebrew)[2][7]
LMBCS-4 04hex 2[7] Code page 1256 (Windows Arabic)[2][7]
LMBCS-5 05hex 2[7] Code page 1251 (Windows Cyrillic)[2][7]
LMBCS-6 06hex 2[7] Code page 852 (DOS Latin-2)[2][7]
N/A 07hex 1[7] BEL[2]
LMBCS-8 08hex 2[7] Code page 1254 (Windows Turkish)[2][9][7]
N/A 09hex 1[7] TAB[2][9][7]
N/A 0Ahex 1[7] LF[2][9][7]
LMBCS-11 0Bhex 2[7] Code page 874 (Thai)[9][7]
(LMBCS-12) 0Chex 2[7] Reserved[2]
N/A 0Dhex 1[7] CR[2][9][7]
(LMBCS-14) 0Ehex 2[7] Reserved[2]
(LMBCS-15) 0Fhex 2[7] Remapped C0/C1 control codes[7]
LMBCS-16 10hex 3[7] Code page 932/[2]943[7] (Japanese / Shift-JIS)[2][9]
LMBCS-17 11hex 3[7] Code page 949/[2]1261[7] (Korean)[2][9]
LMBCS-18 12hex 3[7] Code page 950[2][7] (Traditional Chinese / Taiwan / Big5)[2][9]
LMBCS-19 13hex 3[7] Code page 936/[2]1386[7] (Simplified Chinese)[2][9]
(LMBCS-20) 14hex 3[7] UTF-16 (Unicode)[2][3][7]
N/A 15hex 3 Reserved[2]
N/A 16hex 3 Reserved[2]
N/A 17hex 3 Reserved[2]
N/A 18hex 3 Reserved[2]
N/A 19hex 1[7] Lotus 1-2-3 system range[9][7]
N/A 1Ahex 3 Reserved[2]
N/A 1Bhex 3 Reserved[2]
N/A 1Chex 3 Reserved[2]
N/A 1Dhex 3 Reserved[2]
N/A 1Ehex 3 Reserved[2]
N/A 1Fhex 3 Reserved[2]

Character set[edit]

Without prefix byte the code points 32 (20hex) to 127 (7Fhex) are interpreted as follows (corresponding to LMBCS codes 32 to 127):

  Letter   Number   Punctuation   Symbol   Other   undefined

Single byte codes (ASCII/ISO-646-US[10])
_0 _1 _2 _3 _4 _5 _6 _7 _8 _9 _A _B _C _D _E _F
2_ SP
0020
32
!
0021
33
"
0022
34
#
0023
35
$
0024
36
%
0025
37
&
0026
38
'
0027
39
(
0028
40
)
0029
41
*
002A
42
+
002B
43
,
002C
44
-
002D
45
.
002E
46
/
002F
47
3_ 0
0030
48
1
0031
49
2
0032
50
3
0033
51
4
0034
52
5
0035
53
6
0036
54
7
0037
55
8
0038
56
9
0039
57
:
003A
58
;
003B
59
<
003C
60
=
003D
61
>
003E
62
?
003F
63
4_ @
0040
64
A
0041
65
B
0042
66
C
0043
67
D
0044
68
E
0045
69
F
0046
70
G
0047
71
H
0048
72
I
0049
73
J
004A
74
K
004B
75
L
004C
76
M
004D
77
N
004E
78
O
004F
79
5_ P
0050
80
Q
0051
81
R
0052
82
S
0053
83
T
0054
84
U
0055
85
V
0056
86
W
0057
87
X
0058
88
Y
0059
89
Z
005A
90
[
005B
91
\
005C
92
]
005D
93
^
005E
94
_
005F
95
6_ `
0060
96
a
0061
97
b
0062
98
c
0063
99
d
0064
100
e
0065
101
f
0066
102
g
0067
103
h
0068
104
i
0069
105
j
006A
106
k
006B
107
l
006C
108
m
006D
109
n
006E
110
o
006F
111
7_ p
0070
112
q
0071
113
r
0072
114
s
0073
115
t
0074
116
u
0075
117
v
0076
118
w
0077
119
x
0078
120
y
0079
121
z
007A
122
{
007B
123
|
007C
124
}
007D
125
~
007E
126
DEL (⌂)
007F (2302)
127
_0 _1 _2 _3 _4 _5 _6 _7 _8 _9 _A _B _C _D _E _F

Group 1[edit]

LMBCS group 1 code points 128 (80hex) to 255 (FFhex) are identical to the corresponding code points in code page 850 (DOS Latin-1), whereas code points 1 (01hex) to 127 (7Fhex) are defined according to the following exception list (corresponding to LMBCS codes 256 to 383):

LMBCS Group 1, lower half[11][10]
_0 _1 _2 _3 _4 _5 _6 _7 _8 _9 _A _B _C _D _E _F
0_ NUL
0000
0

263A
1

263B
2

2665
3

2666
4

2663
5

2660
6

2022
7

25D8
8

25CB
9

25D9
10

2642
11

2640
12

266A
13

266B
14

263C
15
1_
25BA
16

25C4
17

2195
18

203C
19

00B6
20
§
00A7
21

25AC
22

21A8
23

2191
24

2193
25

2192
26

2190
27

221F
28

2194
29

25B2
30

25BC
31
2_ ¨
00A8
32
~
007E
33
˚
02DA
34
^
005E
35
`
0060
36
´
00B4
37

201C
38
'
0027
39

2026
40
[nb 2]
2013[nb 2]
41

2014
42

2018
43[nb 3]

2019
44[nb 3]


45[nb 3]

2039
46

203A
47
3_ ¨[nb 4]
00A8
48
~[nb 4]
007E
49
˚[nb 4]
02DA
50
^[nb 4]
005E
51
`[nb 4]
0060
52
´[nb 4]
00B4
53

201E
54

201A
55

201D
56

2017
57


58[nb 3]
NBSP
00A0
59[nb 3]


60[nb 3]

FFFD
61[nb 3]


62[nb 3]


63[nb 3]
4_ Œ
0152
64
œ
0153
65
Ÿ
0178
66
˙
02D9
67[nb 3]
˚[nb 4]
02DA
68[nb 3]


69[nb 3]

255E
70

255F
71

258C
72

2590
73

25CA
74[nb 3]

2318
75[nb 3]

F8FF
76[nb 3]

F8FE
77[nb 3]
Ω
2126
78[nb 3]


79[nb 3]
5_
2568
80

2564
81

2565
82

2559
83

2558
84

2552
85

2553
86

256B
87

256A
88

2561
89

2562
90

2556
91

2555
92

255C
93

255B
94

2567
95
6_ ij
0133
96
IJ
0132
97
/
FB01/F8FD
98[nb 5]
/
FB02/F8FC
99[nb 5]
ʼn
0149
100
ŀ
0140
101
Ŀ
013F
102
¯
00AF
103[nb 3]
˘
02D8
104[nb 3]
˝
02DD
105[nb 3]
˛
02DB
106[nb 3]
ˇ
02C7
107[nb 3]
~[nb 4]
007E
108[nb 3]
^[nb 4]
005E
109[nb 3]


110[nb 3]


111[nb 3]
7_
2020
112

2021
113
Ħ
0126
114[nb 3]
ħ
0127
115[nb 3]
Ŧ
0166
116[nb 3]
ŧ
0167
117[nb 3]

2122
118

2113
119
Ŋ
014A
120[nb 3]
ŋ
014B
121[nb 3]
ĸ
0138
122[nb 3]


123[nb 3]
(Kr)
F8FB
124[nb 6]

2310
125

20A4
126

20A7
127
LMBCS Group 1, lower half
_0 _1 _2 _3 _4 _5 _6 _7 _8 _9 _A _B _C _D _E _F

Group 2[edit]

LMBCS group 2 code points 128 (80hex) to 255 (FFhex) are identical to the corresponding code points in code page 851 (DOS Greek), whereas code points 1 (01hex) to 127 (7Fhex) are defined according to the following exception list:[nb 7]

LMBCS Group 2, lower half[11]
_0 _1 _2 _3 _4 _5 _6 _7 _8 _9 _A _B _C _D _E _F
0_ NUL
0000
0
ͺ
037A
1
΅
0385
2
Ϊ
03AA
3
Ϋ
03AB
4

2015
5
΄
0384
6
ʼ
02BC
7
ʽ
02BD
8

203E
9


10


11


12


13

F862
14

F863
15
1_
F864
16

F865
17

21D5
18

215E
19

215D
20

215C
21

215B
22

F867
23

21D1
24

21D3
25

21D2
26

21D0
27

F868
28

21D4
29

F869
30

F89F
31
2_
F89E
32

F89D
33

F89C
34

F89B
35

F89A
36

F899
37

F898
38

F897
39

F896
40

F895
41

F894
42

F893
43

F892
44

F891
45

F890
46

F88F
47
3_
F88E
48

F88D
49

F88C
50

F88B
51

F88A
52

F889
53

F888
54

F887
55

F886
56

F885
57

F884
58

F883
59

F882
60

F881
61

F880
62

F866
63
4_
2220
64

2207
65

F87F
66

F87E
67

F87D
68

F87C
69

F87B
70

F87A
71

F879
72

F878
73

F877
74

F876
75

F875
76

F874
77

F873
78

F872
79
5_
2202
80

2135
81

2111
82

211C
83

F871
84

F870
85

F86F
86

F86E
87

F86D
88

F86C
89

F86B
90

220B
91

2208
92

2209
93

2286
94

2287
95
6_
2297
96

2295
97

2713
98

22C0
99

2201
100

222B
101

2200
102

2203
103

F86A
104

2032
105

2033
106

221E
107

221D
108
φ
03C6
109

222A
110

2229
111
7_
2261
112

2245
113

2265
114

2264
115

2320
116

2321
117

2260
118

2248
119

2044
120

2219
121

2030
122

221A
123

207F
124

2205
125

2282
126

2283
127
LMBCS Group 2, lower half
_0 _1 _2 _3 _4 _5 _6 _7 _8 _9 _A _B _C _D _E _F

Group 6[edit]

LMBCS group 6 code points 128 (80hex) to 255 (FFhex) are identical to the corresponding code points in code page 852 (DOS Latin-2), whereas code points 1 (01hex) to 127 (7Fhex) are defined according to the following exception list:[nb 7]

LMBCS Group 6, lower half[11]
_0 _1 _2 _3 _4 _5 _6 _7 _8 _9 _A _B _C _D _E _F
0_ NUL
0000
0
ā
0101
1
Ĉ
0108
2
ĉ
0109
3
Ċ
010A
4
ċ
010B
5
Ē
0112
6
ē
0113
7
Ė
0116
8
ė
0117
9
Ĝ
011C
10
ĝ
011D
11
Ġ
0120
12
ġ
0121
13
Ģ
0122
14
ģ
0123
15
1_ Ĥ
0124
16
ĥ
0125
17
Ĩ
0128
18
ĩ
0129
19
Ī
012A
20
ī
012B
21
Į
012E
22
į
012F
23
Ĵ
0134
24
ĵ
0135
25
Ķ
0136
26
ķ
0137
27
Ļ
013B
28
ļ
013C
29
Ņ
0145
30
ņ
0146
31
2_ Ō
014C
32
ō
014D
33
Ŗ
0156
34
ŗ
0157
35
Ŝ
015C
36
ŝ
015D
37
Ũ
0168
38
ũ
0169
39
Ū
016A
40
ū
016B
41
Ŭ
016C
42
ŭ
016D
43
Ų
0172
44
ų
0173
45
Ā
0100
46


47
3_

48


49


50


51


52


53


54


55


56


57


58


59


60


61


62


63
4_

64


65


66


67


68


69


70


71


72


73


74


75


76


77


78


79
5_

80


81


82


83


84


85


86


87


88


89


90


91


92


93


94


95
6_

96


97


98


99


100


101


102


103


104


105


106


107


108


109


110


111
7_

112


113


114


115


116


117


118


119


120


121


122


123


124


125


126


127
LMBCS Group 6, lower half
_0 _1 _2 _3 _4 _5 _6 _7 _8 _9 _A _B _C _D _E _F

See also[edit]

Notes[edit]

  1. ^ Lotus 1-2-3 Release 3.0 for DOS and newer versions are written in C.
  2. ^ a b (U+2010), (U+2011), (U+2012), (U+2013)
  3. ^ a b c d e f g h i j k l m n o p q r s t u v w x y z aa ab ac ad ae af ag ah ai According to the documentation this code point is not supported by Lotus 1-2-3 Release 3.1+ for DOS and OS/2 and earlier.
  4. ^ a b c d e f g h i For compatibility with Lotus 1-2-3 Release 5.0.
  5. ^ a b LMBCS group 1 code points 0162hex (LATIN SMALL LIGATURE FI) and 0163hex (LATIN SMALL LIGATURE FL) originally overlapped with Microsoft's Private Use Area (PUA) but were later redefined according to new Unicode definitions.
  6. ^ Unicode does not define a glyph for the crown currency symbol (Krone aka "Kr"), therefore this points to F8FBhex in the Unicode Private Use Area (PUA).
  7. ^ a b According to the documentation code points 1 to 127 in this group are not supported by Lotus 1-2-3 Release 3.1+ for DOS and OS/2 and earlier. These versions only supported LMBCS code points 0 to 511, covering group 0 and 1 only.

References[edit]

  1. ^ a b c d e f g h i j k l m Balaban, Bob (2001). "Multi-Language Character Sets – What They Are, How To Use Them" (PDF). Looseleaf Software, Inc. Archived (PDF) from the original on 2016-11-25. Retrieved 2016-11-25. 
  2. ^ a b c d e f g h i j k l m n o p q r s t u v w x y z aa ab ac ad ae af ag ah "Appendix A. Encoding Schemes". IBM Character Data Representation Architecture. IBM (CDRA). Lotus Multi-byte Character Set (LMBCS). Archived from the original on 2016-11-26. Retrieved 2016-11-26. For optimization purposes, the group byte is omitted in Notes for single-byte values between X'20' and X'FF'. For example, LMBCS is always optimized to group 0x01, which means that any character where the first byte is greater than 0x1F, has an implicit group byte of 0x01. 
  3. ^ a b Scherer, Markus; Murray, Brendan (2000-06-02). "Re: MS Excel, Lotus 123 & Unicode". Archived from the original on 2016-12-06. Retrieved 2016-12-06. 
  4. ^ "Kapitel 4. Kompatibilität mit anderen 1-2-3 Versionen – Zeichensätze" [Chapter 4. Compatibility with other 1-2-3 Versions – Character Sets]. Lotus 1-2-3 Version 3.1 Upgrader's Handbuch [Upgrader's handbook] (in German) (1 ed.). Cambridge, MA, USA: Lotus Development Corporation. 1989. pp. 4–10–4–11. 302173. 
  5. ^ a b Kamenz, Alfred; Vonhoegen, Helmut (1992). Das große Buch zu Lotus 1-2-3 für DOS (in German) (1 ed.). Data Becker. pp. 131–132, 357–358. ISBN 3-89011-375-3. 
  6. ^ a b c d Lotus – Inside Notes – The Architecture of Notes and the Domino Server (PDF). Lotus Development Corporation. 2000. Archived (PDF) from the original on 2016-12-12. Retrieved 2016-12-12. […] Notes uses a single character set, the Lotus Multibyte Character Set (LMBCS), to encode all text data used internally by its programs. Whenever Notes first inputs text encoded in a character set other than LMBCS, it translates the text into a LMBCS string, and whenever it must output text in a character set other than LMBCS, it translates the internal LMBCS string into the appropriate character set. Because all text is internally formatted by LMBCS, all text-processing operations […] are done in only one way. LMBCS uses up to three bytes in memory to represent a single text character […] 
  7. ^ a b c d e f g h i j k l m n o p q r s t u v w x y z aa ab ac ad ae af ag ah ai aj ak al am an ao ap aq Murray, Brendan; Snyder-Grant, Jim, eds. (2016) [2000-02-09]. "ucnv_lmb.c". w32 tex-src. International Business Machines (IBM). Archived from the original on 2016-12-20. Retrieved 2016-12-06. 
  8. ^ Batutis, Edward J. (2001-11-03). "Re: converter types". International Components for Unicode (ICU). Archived from the original on 2016-12-06. Retrieved 2016-12-06. 
  9. ^ a b c d e f g h i j "LMBCS" (in Japanese). 2009-02-03. Archived from the original on 2016-11-26. Retrieved 2016-11-26. [1]
  10. ^ a b "Anhang 2. Der Lotus Multibyte Zeichensatz (LMBCS)" [Appendix 2. The Lotus Multibyte Character Set (LMBCS)]. Lotus 1-2-3 Version 3.1 Referenzhandbuch [Lotus 1-2-3 Version 3.1 Reference Manual] (in German) (1 ed.). Cambridge, MA, USA: Lotus Development Corporation. 1989. pp. A2–1 – A2–13. 302168. 
  11. ^ a b c "lmb-excp.ucm". megadaddeln / icu_chrome. 2010 [1995]. Archived from the original on 2016-12-06. Retrieved 2016-12-06.  [2]

Further reading[edit]

External links[edit]