CCSID explained
A CCSID (coded character set identifier) is a 16-bit number that represents a particular encoding of a specific code page. For example, Unicode is a code page that has several character encoding schemes (referred to as "transformation forms")—including UTF-8, UTF-16 and UTF-32—but which may or may not actually be accompanied by a CCSID number to indicate that this encoding is being used.
Difference between a code page and a CCSID
The terms code page and CCSID are often used interchangeably, even though they are not synonymous. A code page may be only part of what makes up a CCSID. The following definitions from IBM help to illustrate this point:
- A glyph is the actual physical pattern of pixels or ink that shows up on a display or printout.
- A character is a concept that covers all glyphs associated with a certain symbol. For instance, "F", "F", "F", "", "", and "" are all different glyphs, but use the same character. The various modifiers (bold, italic, underline, color, and font) do not change the F's essential F-ness.
- A character set contains the characters necessary to allow a particular human to carry on a meaningful interaction with the computer. It does not specify how those characters are represented in a computer.[1] This level is the first one to separate characters into various alphabets (Latin, Arabic, Hebrew, Cyrillic, and so on) or ideographic groups (e.g., Chinese, Korean). It corresponds to a "character repertoire" in the Unicode encoding model.
- A code page represents a particular assignment of code point values to characters. It corresponds to a "coded character set" in the Unicode encoding model. A code point for a character is the computer's internal representation of that character in a given code page. Many characters are represented by different code points in different code pages. Certain character sets can be adequately represented with single-byte code pages (which have a maximum 256 code points, hence a maximum of 256 characters), but many require more than that. Examples include JIS X 0208 and Unicode.
- An encoding scheme is the byte format of a code page. It maps code point values to sequences of one or more byte values in a computer.[2] For example, UTF-8 and UTF-16BE are two encodings of the same Unicode code page. (Varying only in how many bytes are needed to represent a particular Unicode character value, how it is contained within those bytes, and how the presence of Unicode information is indicated.) Meanwhile, in IBM's character data representation architecture (CDRA), this is typically represented with an ESID (encoding scheme identifier).[3] EUC and ISO-2022 are other examples of encoding schemes.
- A coded character set identifier (CCSID) contains all of the information necessary to assign and preserve the meaning and rendering of characters through various stages of processing and interchange. This information always includes at least one code page, but may include multiple code pages of differing byte-lengths. The CCSID also has an associated encoding scheme that governs how various code points are to be handled. This mechanism allows a program to recognize bidirectional orientation, character shaping (mainly of Arabic characters), and other complex encoding information.
Examples
The following examples show how some CCSIDs are made up of other CCSIDs.
CCSID 932[4] | Character set | Code page | CCSID | Encoding scheme |
|---|
| 01122 | 00897 | 897 | |
| 00370 | 00301 | 301 | | |
CCSID 942[5] | Character set | Code page | CCSID | Encoding scheme |
|---|
| 01172 | 01041 | 1041 | SBCS |
| 00370 | 00301 | 301 | DBCS | |
CCSID 5028[6] | Character set | Code page | CCSID | Encoding scheme |
|---|
| 01170 | 00897 | 4993 | SBCS |
| 00370 | 00301 | 301 | DBCS | |
All three of these variant Shift-JIS CCSIDs are multi-byte character sets (MBCS): the single-byte character set (SBCS) portion of each CCSID is different. The double-byte character set (DBCS) portion is the same across each CCSID. CCSID 5028 uses an updated code page 897 called CCSID 4993. CCSID 932 uses the original code page 897, which is CCSID 897. CCSID 942 uses a different SBCS from the other two CCSIDs, which is 1041.
Also notice how CCSID 5028 and 4993 are different by 4096 (1000 in hexadecimal) from the predecessor CCSID with the same code page identifier. This is a common way that CDRA denotes an upgraded CCSID.
There are a few reasons for this complexity:
- Many of the CCSIDs are used in IBM databases, like IBM Db2, where a database field only supports an SBCS, DBCS or MBCS string. CCSIDs allow programs to differentiate between which one is being used.
- When characters are added or replaced, like the Euro currency sign introduction, one can know whether the stored strings support or do not support those character additions because a different CCSID is being used. This versioning is important for the integrity of the data.
- It enables reuse of resources among similar CCSIDs.[7]
External links
Notes and References
- Web site: IBM Terminology—Terms C . . 2013-01-25.
- Web site: Character Data Representation Architecture . Appendix A. Encoding Schemes . . 2019-06-29.
- Web site: Character Data Representation Architecture . Chapter 3. CDRA Identifiers, section "Long-Form Identification" . . 2019-06-29.
- Web site: Japanese PC Data Mixed including 1880 UDC . IBM . Globalization . dead . February 20, 2012 . https://web.archive.org/web/20120220153419/http://www-01.ibm.com/software/globalization/ccsid/ccsid932.html . November 29, 2011 .
- Web site: Japanese PC Data Mixed including 1880 UDC, Extended SBCS . IBM . Globalization . dead . December 1, 2014 . https://web.archive.org/web/20141201234629/http://www-01.ibm.com/software/globalization/ccsid/ccsid942.html . November 29, 2011 .
- Web site: Japanese PC Data Mixed including 1880 UDC (Katakana - PC common set for SBCS) . IBM . Globalization . dead . November 29, 2014 . https://web.archive.org/web/20141129225910/http://www-01.ibm.com/software/globalization/ccsid/ccsid5028.html . November 29, 2011 .
- Web site: Us-en_software_HP. 9 November 2020.
