The m17n Library 1.8.4
|
Coding system objects and API for them. More...
Data Structures | |
struct | MConverter |
Structure to be used in code conversion. More... | |
struct | MCodingInfoISO2022 |
Structure for a coding system of type MCODING_TYPE_ISO_2022. More... | |
struct | MCodingInfoUTF |
Structure for extra information about a coding system of type MCODING_TYPE_UTF. More... | |
Functions | |
MSymbol | mconv_define_coding (const char *name, MPlist *plist, int(*resetter)(MConverter *), int(*decoder)(const unsigned char *, int, MText *, MConverter *), int(*encoder)(MText *, int, int, unsigned char *, int, MConverter *), void *extra_info) |
MSymbol | mconv_resolve_coding (MSymbol symbol) |
Resolve coding system name. | |
int | mconv_list_codings (MSymbol **symbols) |
List symbols representing coding systems. | |
MConverter * | mconv_buffer_converter (MSymbol name, const unsigned char *buf, int n) |
Create a code converter bound to a buffer. | |
MConverter * | mconv_stream_converter (MSymbol name, FILE *fp) |
Create a code converter bound to a stream. | |
int | mconv_reset_converter (MConverter *converter) |
Reset a code converter. | |
void | mconv_free_converter (MConverter *converter) |
Free a code converter. | |
MConverter * | mconv_rebind_buffer (MConverter *converter, const unsigned char *buf, int n) |
Bind a buffer to a code converter. | |
MConverter * | mconv_rebind_stream (MConverter *converter, FILE *fp) |
Bind a stream to a code converter. | |
MText * | mconv_decode (MConverter *converter, MText *mt) |
Decode a byte sequence into an M-text. | |
MText * | mconv_decode_buffer (MSymbol name, const unsigned char *buf, int n) |
Decode a buffer area based on a coding system. | |
MText * | mconv_decode_stream (MSymbol name, FILE *fp) |
Decode a stream input based on a coding system. | |
int | mconv_encode (MConverter *converter, MText *mt) |
Encode an M-text into a byte sequence. | |
int | mconv_encode_range (MConverter *converter, MText *mt, int from, int to) |
Encode a part of an M-text. | |
int | mconv_encode_buffer (MSymbol name, MText *mt, unsigned char *buf, int n) |
Encode an M-text into a buffer area. | |
int | mconv_encode_stream (MSymbol name, MText *mt, FILE *fp) |
Encode an M-text to write to a stream. | |
int | mconv_getc (MConverter *converter) |
Read a character via a code converter. | |
int | mconv_ungetc (MConverter *converter, int c) |
Push a character back to a code converter. | |
int | mconv_putc (MConverter *converter, int c) |
Write a character via a code converter. | |
MText * | mconv_gets (MConverter *converter, MText *mt) |
Read a line using a code converter. | |
Variables: Symbols representing coding systems | |
MSymbol | Mcoding_us_ascii |
Symbol for the coding system US-ASCII. | |
MSymbol | Mcoding_iso_8859_1 |
Symbol for the coding system ISO-8859-1. | |
MSymbol | Mcoding_utf_8 |
Symbol for the coding system UTF-8. | |
MSymbol | Mcoding_utf_8_full |
Symbol for the coding system UTF-8-FULL. | |
MSymbol | Mcoding_utf_16 |
Symbol for the coding system UTF-16. | |
MSymbol | Mcoding_utf_16be |
Symbol for the coding system UTF-16BE. | |
MSymbol | Mcoding_utf_16le |
Symbol for the coding system UTF-16LE. | |
MSymbol | Mcoding_utf_32 |
Symbol for the coding system UTF-32. | |
MSymbol | Mcoding_utf_32be |
Symbol for the coding system UTF-32BE. | |
MSymbol | Mcoding_utf_32le |
Symbol for the coding system UTF-32LE. | |
MSymbol | Mcoding_sjis |
Symbol for the coding system SJIS. | |
Variables: Parameter keys for mconv_define_coding(). <br> | |
MSymbol | Mtype |
MSymbol | Mcharsets |
MSymbol | Mflags |
MSymbol | Mdesignation |
MSymbol | Minvocation |
MSymbol | Mcode_unit |
MSymbol | Mbom |
MSymbol | Mlittle_endian |
Variables: Symbols representing coding system types. <br> | |
MSymbol | Mutf |
MSymbol | Miso_2022 |
Variables: Symbols appearing in the value of Mflags parameter. <br> | |
Symbols that can be a value of the Mflags parameter of a coding system used in an argument to the mconv_define_coding() function (which see). | |
MSymbol | Mreset_at_eol |
MSymbol | Mreset_at_cntl |
MSymbol | Meight_bit |
MSymbol | Mlong_form |
MSymbol | Mdesignation_g0 |
MSymbol | Mdesignation_g1 |
MSymbol | Mdesignation_ctext |
MSymbol | Mdesignation_ctext_ext |
MSymbol | Mlocking_shift |
MSymbol | Msingle_shift |
MSymbol | Msingle_shift_7 |
MSymbol | Meuc_tw_shift |
MSymbol | Miso_6429 |
MSymbol | Mrevision_number |
MSymbol | Mfull_support |
Variables: Others | |
Remaining variables. | |
MSymbol | Mmaybe |
Symbol whose name is "maybe". | |
MSymbol | Mcoding |
The symbol Mcoding . | |
Coding system objects and API for them.
The m17n library represents a character encoding scheme (CES) of coded character sets (CCS) as an object called coding system. Application programs can add original coding systems.
To encode means converting code-points to character codes and to decode means converting character codes back to code-points.
Application programs can decode a byte sequence with a specified coding system into an M-text, and inversely, can encode an M-text into a byte sequence.
enum MConversionResult |
Codes that represent the result of code conversion.
One of these values is set in MConverter->result
.
enum MCodingType |
Types of coding system.
Enumerator | |
---|---|
MCODING_TYPE_CHARSET | A coding system of this type supports charsets directly. The dimension of each charset defines the length of bytes to represent a single character of the charset, and a byte sequence directly represents the code-point of a character. The m17n library provides the default decoding and encoding routines of this type. |
MCODING_TYPE_UTF | A coding system of this type supports byte sequences of a UTF (UTF-8, UTF-16, UTF-32) like structure. The m17n library provides the default decoding and encoding routines of this type. |
MCODING_TYPE_ISO_2022 | A coding system of this type supports byte sequences of an ISO-2022 like structure. The details of each structure are specified by |
MCODING_TYPE_MISC | A coding system of this type is for byte sequences of miscellaneous structures. The m17n library does not provide decoding and encoding routines of this type. They must be provided by the application program. |
enum MCodingFlagISO2022 |
Bit-masks to specify the detail of coding system whose type is MCODING_TYPE_ISO_2022.
MSymbol mconv_define_coding | ( | const char * | name, |
MPlist * | plist, | ||
int(*)(MConverter *) | resetter, | ||
int(*)(const unsigned char *, int, MText *, MConverter *) | decoder, | ||
int(*)(MText *, int, int, unsigned char *, int, MConverter *) | encoder, | ||
void * | extra_info | ||
) |
MSymbol mconv_resolve_coding | ( | MSymbol | symbol | ) |
Resolve coding system name.
The mconv_resolve_coding() function returns symbol if it represents a coding system. Otherwise, canonicalize symbol as to a coding system name, and if the canonicalized name represents a coding system, return it. Otherwise, return Mnil.
int mconv_list_codings | ( | MSymbol ** | symbols | ) |
List symbols representing coding systems.
The mconv_list_codings() function makes an array of symbols representing a coding system, stores the pointer to the array in a place pointed to by symbols, and returns the length of the array.
MConverter * mconv_buffer_converter | ( | MSymbol | name, |
const unsigned char * | buf, | ||
int | n | ||
) |
Create a code converter bound to a buffer.
The mconv_buffer_converter() function creates a pointer to a code converter for coding system name. The code converter is bound to buffer area of n bytes pointed to by buf. Subsequent decodings and encodings are done to/from this buffer area.
name can be Mnil. In this case, a coding system associated with the current locale (LC_CTYPE) is used.
NULL
and assigns an error code to the external variable merror_code. MERROR_SYMBOL
, MERROR_CODING
MConverter * mconv_stream_converter | ( | MSymbol | name, |
FILE * | fp | ||
) |
Create a code converter bound to a stream.
The mconv_stream_converter() function creates a pointer to a code converter for coding system name. The code converter is bound to stream fp. Subsequent decodings and encodings are done to/from this stream.
name can be Mnil. In this case, a coding system associated with the current locale (LC_CTYPE) is used.
NULL
and assigns an error code to the external variable merror_code. MERROR_SYMBOL
, MERROR_CODING
int mconv_reset_converter | ( | MConverter * | converter | ) |
Reset a code converter.
The mconv_reset_converter() function resets code converter converter to the initial state.
void mconv_free_converter | ( | MConverter * | converter | ) |
Free a code converter.
The mconv_free_converter() function frees the code converter converter.
MConverter * mconv_rebind_buffer | ( | MConverter * | converter, |
const unsigned char * | buf, | ||
int | n | ||
) |
Bind a buffer to a code converter.
The mconv_rebind_buffer() function binds buffer area of n bytes pointed to by buf to code converter converter. Subsequent decodings and encodings are done to/from this newly bound buffer area.
MConverter * mconv_rebind_stream | ( | MConverter * | converter, |
FILE * | fp | ||
) |
Bind a stream to a code converter.
The mconv_rebind_stream() function binds stream fp to code converter converter. Following decodings and encodings are done to/from this newly bound stream.
MText * mconv_decode | ( | MConverter * | converter, |
MText * | mt | ||
) |
Decode a byte sequence into an M-text.
The mconv_decode() function decodes a byte sequence and appends the result at the end of M-text mt. The source byte sequence is taken from either the buffer area or the stream that is currently bound to converter.
NULL
and assigns an error code to the external variable merror_code. MERROR_IO
, MERROR_CODING
MText * mconv_decode_buffer | ( | MSymbol | name, |
const unsigned char * | buf, | ||
int | n | ||
) |
Decode a buffer area based on a coding system.
The mconv_decode_buffer() function decodes n bytes of the buffer area pointed to by buf based on the coding system name. A temporary code converter for decoding is automatically created and freed.
NULL
and assigns an error code to the external variable merror_code. MERROR_IO
, MERROR_CODING
MText * mconv_decode_stream | ( | MSymbol | name, |
FILE * | fp | ||
) |
Decode a stream input based on a coding system.
The mconv_decode_stream() function decodes the entire byte sequence read in from stream fp based on the coding system name. A code converter for decoding is automatically created and freed.
NULL
and assigns an error code to the external variable merror_code. MERROR_IO
, MERROR_CODING
int mconv_encode | ( | MConverter * | converter, |
MText * | mt | ||
) |
Encode an M-text into a byte sequence.
The mconv_encode() function encodes M-text mt and writes the resulting byte sequence into the buffer area or the stream that is currently bound to code converter converter.
MERROR_IO
, MERROR_CODING
int mconv_encode_range | ( | MConverter * | converter, |
MText * | mt, | ||
int | from, | ||
int | to | ||
) |
Encode a part of an M-text.
The mconv_encode_range() function encodes the text between from (inclusive) and to (exclusive) in M-text mt and writes the resulting byte sequence into the buffer area or the stream that is currently bound to code converter converter.
MERROR_RANGE
, MERROR_IO
, MERROR_CODING
int mconv_encode_buffer | ( | MSymbol | name, |
MText * | mt, | ||
unsigned char * | buf, | ||
int | n | ||
) |
Encode an M-text into a buffer area.
The mconv_encode_buffer() function encodes M-text mt based on coding system name and writes the resulting byte sequence into the buffer area pointed to by buf. At most n bytes are written. A temporary code converter for encoding is automatically created and freed.
MERROR_IO
, MERROR_CODING
int mconv_encode_stream | ( | MSymbol | name, |
MText * | mt, | ||
FILE * | fp | ||
) |
Encode an M-text to write to a stream.
The mconv_encode_stream() function encodes M-text mt based on coding system name and writes the resulting byte sequence to stream fp. A temporary code converter for encoding is automatically created and freed.
MERROR_IO
, MERROR_CODING
int mconv_getc | ( | MConverter * | converter | ) |
Read a character via a code converter.
The mconv_getc() function reads one character from the buffer area or the stream that is currently bound to code converter converter. The decoder of converter is used to decode the byte sequence. The internal status of converter is updated appropriately.
EOF
without changing the external variable merror_code. If an error is detected, it returns EOF
and assigns an error code to merror_code. MERROR_CODING
int mconv_ungetc | ( | MConverter * | converter, |
int | c | ||
) |
Push a character back to a code converter.
The mconv_ungetc() function pushes character c back to code converter converter. Any number of characters can be pushed back. The lastly pushed back character is firstly read by the subsequent mconv_getc() call. The characters pushed back are registered only in converter; they are not written to the input source. The internal status of converter is updated appropriately.
EOF
and assigns an error code to the external variable merror_code. MERROR_CODING
, MERROR_CHAR
int mconv_putc | ( | MConverter * | converter, |
int | c | ||
) |
Write a character via a code converter.
The mconv_putc() function writes character c to the buffer area or the stream that is currently bound to code converter converter. The encoder of converter is used to encode the character. The number of bytes actually written is set to the nbytes
member of converter. The internal status of converter is updated appropriately.
EOF
and assigns an error code to the external variable merror_code. MERROR_CODING
, MERROR_IO
, MERROR_CHAR
MText * mconv_gets | ( | MConverter * | converter, |
MText * | mt | ||
) |
Read a line using a code converter.
The mconv_gets() function reads one line from the buffer area or the stream that is currently bound to code converter converter. The decoder of converter is used for decoding. The decoded character sequence is appended at the end of M-text mt. The final newline character in the original byte sequence is not appended. The internal status of converter is updated appropriately.
NULL
and assigns an error code to merror_code. MERROR_CODING
MSymbol Mcoding_us_ascii |
Symbol for the coding system US-ASCII.
The symbol Mcoding_us_ascii has name "us-ascii"
and represents a coding system for the CES US-ASCII.
MSymbol Mcoding_iso_8859_1 |
Symbol for the coding system ISO-8859-1.
The symbol Mcoding_iso_8859_1 has name "iso-8859-1"
and represents a coding system for the CES ISO-8859-1.
MSymbol Mcoding_utf_8 |
Symbol for the coding system UTF-8.
The symbol Mcoding_utf_8 has name "utf-8"
and represents a coding system for the CES UTF-8.
MSymbol Mcoding_utf_8_full |
Symbol for the coding system UTF-8-FULL.
The symbol Mcoding_utf_8_full has name "utf-8-full"
and represents a coding system that is a extension of UTF-8. This coding system uses the same encoding algorithm as UTF-8 but is not limited to the Unicode characters. It can encode all characters supported by the m17n library.
MSymbol Mcoding_utf_16 |
Symbol for the coding system UTF-16.
The symbol Mcoding_utf_16 has name "utf-16"
and represents a coding system for the CES UTF-16 (RFC 2279).
MSymbol Mcoding_utf_16be |
Symbol for the coding system UTF-16BE.
The symbol Mcoding_utf_16be has name "utf-16be"
and represents a coding system for the CES UTF-16BE (RFC 2279).
MSymbol Mcoding_utf_16le |
Symbol for the coding system UTF-16LE.
The symbol Mcoding_utf_16le has name "utf-16le"
and represents a coding system for the CES UTF-16LE (RFC 2279).
MSymbol Mcoding_utf_32 |
Symbol for the coding system UTF-32.
The symbol Mcoding_utf_32 has name "utf-32"
and represents a coding system for the CES UTF-32 (RFC 2279).
MSymbol Mcoding_utf_32be |
Symbol for the coding system UTF-32BE.
The symbol Mcoding_utf_32be has name "utf-32be"
and represents a coding system for the CES UTF-32BE (RFC 2279).
MSymbol Mcoding_utf_32le |
Symbol for the coding system UTF-32LE.
The symbol Mcoding_utf_32le has name "utf-32le"
and represents a coding system for the CES UTF-32LE (RFC 2279).
MSymbol Mcoding_sjis |
Symbol for the coding system SJIS.
The symbol Mcoding_sjis has name "sjis"
and represents a coding system for the CES Shift-JIS.
MSymbol Mtype |
Parameter key for mconv_define_coding() (which see).
MSymbol Mcharsets |
MSymbol Mflags |
MSymbol Mdesignation |
MSymbol Minvocation |
MSymbol Mcode_unit |
MSymbol Mbom |
MSymbol Mlittle_endian |
MSymbol Mutf |
Symbol that can be a value of the Mtype parameter of a coding system used in an argument to the mconv_define_coding() function (which see).
MSymbol Miso_2022 |
MSymbol Mreset_at_eol |
MSymbol Mreset_at_cntl |
MSymbol Meight_bit |
MSymbol Mlong_form |
MSymbol Mdesignation_g0 |
MSymbol Mdesignation_g1 |
MSymbol Mdesignation_ctext |
MSymbol Mdesignation_ctext_ext |
MSymbol Mlocking_shift |
MSymbol Msingle_shift |
MSymbol Msingle_shift_7 |
MSymbol Meuc_tw_shift |
MSymbol Miso_6429 |
MSymbol Mrevision_number |
MSymbol Mfull_support |
MSymbol Mmaybe |
Symbol whose name is "maybe".
The variable Mmaybe is a symbol of name "maybe"
. It is used a value of Mbom parameter of the function mconv_define_coding() (which see).
MSymbol Mcoding |
The symbol Mcoding
.
Any decoded M-text has a text property whose key is the predefined symbol Mcoding
. The name of Mcoding
is "coding"
.