International Components for Unicode
ICU 2.0 ReadMe

Version: 2002-Feb-04
Copyright © 1997-2002 International Business Machines Corporation and others. All Rights Reserved.

Table of Contents

• Introduction
• Getting started
• What is new in this release?
• How to Download the Source Code
• ICU Source Code Organization
• How to Build And Install ICU
  • Supported Platforms
  • Windows
  • Unix
  • OS/390 (zSeries)
  • OS/400 (iSeries)
• Important Notes About Using ICU
  • Windows Platform
  • Unix Type Platforms
  • Using the default codepage
  • Methods for enabling deprecated APIs
• Platform Dependencies

Introduction

Today's software market is a global one in which it is desirable to develop and maintain one application (single source/single binary) that supports a wide variety of languages. The International Components for Unicode (C/C++) provides tools to help write platform-independent applications that are internationalized and localized, with support for:

• Support for the latest version of the Unicode standard
• Character set conversions, with support for over 200 codepages
• Locale data for more than 160 locales
• Text collation (sorting) based on the Unicode Collation Algorithm (=ISO 14651), customizable and tailored for national standards
• Transliteration services for script<->script transliterations and general text operations
• Resource bundles for storing and accessing localized information
• Date/Number/Message formatting and parsing of culture-specific input/output formats
• Text boundary analysis for finding characters, word and sentence boundaries

ICU has a sister project ICU4J that extends the internationalization capabilities of Java to a level similar to ICU. The ICU C/C++ project is also called ICU4C when a distinction is necessary.

Getting started

This document describes how to build and install ICU on your machine. For other information about ICU please see the following table of links.
The ICU homepage also links to related information about writing internationalized software.

Important: Please make sure you understand the Copyright and License Information.

What is new in this release?

The following list concentrates on changes that affect existing applications migrating from previous ICU releases. For more news about this release, see the ICU 2.0 download page.

Support for Unicode 3.1.1

ICU 2.0 has been upgraded to support Unicode 3.1.1, which includes the addition of 44,946 new encoded characters. These characters cover several historic scripts, several sets of symbols, and a very large collection of additional CJK ideographs.

As part of this upgrade, a number of ICU services have been reviewed and improved with regards to handling supplementary characters (surrogate pairs). Especially, normalization is revamped for support of supplementary characters and higher performance.

Euro transition

Locale data for countries that are switching their national currencies to the Euro is updated to use the Euro symbol and appropriate currency formatting. The old data is available in _PREEURO locale variants. The _EURO variant selector can still be used to unambiguously get Euro currency symbol formatting. For some time around the transition, software should explicitly specify _PREEURO and _EURO variants to make sure to get the intended currency format.

For more on this topic see the developerWorks article "Are you really ready for the Euro?".

API changes

Functions that take C-style string input arguments with const UChar *src and int32_t srcLength now consistently treat srcLength==-1 to mean that the input string is NUL-terminated and get srcLength=u_strlen(src).

Functions that take C-style string output arguments with UChar *dest and int32_t destCapacity now handle NUL-termination of the output string consistently. If the output length is equal to destCapacity, then dest is filled with the output string and a warning code is set. For details about string handling see the User's Guide Strings chapter.

Some APIs have been deprecated for a long time (more than a year) and have been removed now.
Some other APIs have been marked as deprecated because they are replaced by improved APIs; the newly deprecated APIs will be available for another year. In particular, the C++ classes UnicodeConverter, Unicode, and BiDi are deprecated in favor of the equally powerful C APIs.
A few draft APIs have changed, especially for transliteration.

APIs that take a rules or pattern string (for collation, transliteration, message formats, etc.) now also take a UParseError structure that is filled with useful debugging information when a rule syntax error is detected. This makes it easier in large rules to find problems. As a result, the signatures of some functions have changed. The old signatures will be available for about a year by #defining a constant. See affected header files for details.

The C++ Normalizer class had a partially broken model for iterative normalization; this is redone in a more consistent way. See the Normalizer API documentation for details.

Memory and resource cleanup

ICU is carefully tested for memory leaks. Some memory is held in internal caches that do not normally get released during normal operation. These are not leaks because ICU continues to use them as necessary.

For testing purposes (for memory leaks) and for a small number of applications it can be useful to close all the memory that is allocated for a library. ICU 2.0 supports this with a new function u_cleanup() that may be called after an application has released all ICU objects. u_cleanup() will then release all of ICU's internal memory. The ICU libraries can then even be unloaded cleanly without shutting down the process.

ICU versioning - C++ namespaces

Beginning with ICU 2.0, multiple releases of ICU can be used in the same process. Together with an arbitrary number of post-2.0 releases, one pre-2.0 release can be loaded and active.

This is achieved by renaming all library exports to include a release number suffix. Each global function and each class is renamed in this way using a header file with #defines. For C++, if the compiler supports namespaces, all ICU C++ classes are defined in the "icu" namespace. If the compiler does not support namespaces, then the classes are renamed instead. This change also reduces the chance of naming collisions with other libraries.

For details see the User's Guide Design Chapter.

Data loading changed

ICU data loading is simplified for most users. By default, the ICU build creates a DLL/shared library that is linked directly with the common library ([lib]icuuc). By placing all ICU libraries including the data library into the same folder, ICU should start up and find its data immediately. Dynamic loading of data from DLLs/shared libraries is not supported any more.

Before ICU 2.0, ICU did not itself link directly with its data library, but some ICU applications did (like the Xerces XML parser) and called udata_setCommonData(). This is not necessary any more in the default case.
On the other hand, this same technique can now be used to efficiently load application data (e.g., for its own localization). An application can build a data DLL/library of its own, link it, and call the new API udata_setAppData().

For details on finding and loading ICU data and on options for portable, common data files etc. see the User's Guide ICU Data Chapter.

Collation improvements

The performance of Japanese Katakana collation is improved, and the Japanese collation is changed for conformance with the JIS X 4061 standard. The improvement is in the handling of the length and iteration marks, making the processing of regular letters faster.

The JIS X 4061 standard specifies a 5-level sorting algorithm. Sorting with all five levels according to JIS is achieved in ICU 2.0 with the "identical" strength. The fifth level distinguishes regular character codes from compatibility variants.

There is special code to handle the fourth (quarternary) level of the JIS standard, which distinguishes between Hiragana and Katakana letters. In ICU 2.0 string comparisons (like ucol_strcoll), when using the "shifted" option, this is slow because it generates complete sort keys for both strings. This is not an issue if the "shifted" option is not used, or if the string comparison is done with fewer levels.

Quarternary strength, without the "shifted" option, is the default for Japanese collation in ICU 2.0.

Three-level sorting (tertiary strength) and lower — if sufficient — is faster even with "shifted" on (for string comparisons: much faster in this case).

License Change (for ICU 1.8.1 and up)

The ICU projects (ICU4C and ICU4J) have changed their licenses from the IPL (IBM Public License) to the X license. The X license is a non-viral and recommended free software license that is compatible with the GNU GPL license. This is effective starting with release 1.8.1 of ICU4C and release 1.3.1 of ICU4J. All previous ICU releases will continue to utilize the IPL. New ICU releases will adopt the X license. The users of previous releases of ICU will need to accept the terms and conditions of the X license in order to adopt the new ICU releases.

The main effect of the change is to provide GPL compatibility. The X license is listed as GPL compatible, see the gnu page at http://www.gnu.org/philosophy/license-list.html#GPLCompatibleLicenses.

The text of the X license is available at http://www.x.org/terms.htm. The IBM version contains the essential text of the license, omitting the X-specific trademarks and copyright notices.

For more details please see the press announcement and the Project FAQ.

Transliterator improvements

The transliterator service has undergone an extensive overhaul, in both the rule-based engine and the built-in system rules. For a complete description see the User's Guide chapter on transliteration.

• New or rewritten rules: Any-Accents, Any-Publishing, Cyrillic-Latin*, Greek-Latin*, Greek-Latin/UNGEGN (aka el-Latin), Hiragana-Latin*, and Latin-Katakana*. New algorithmic rules include Any-Name*, the normalization rules Any-NFC, Any-NFKC, Any-NFD, and Any-NFKD, casing rules Any-Upper, Any-Lower, and Any-Title. Unicode-Hex* has been renamed Any-Hex*. Any-Remove deletes its input. [*applies to reverse rule as well]
• Indic script rules: Transliterators between Indic scripts and from each script to and from Latin have been completely revised. Scripts included are Bengali, Devanagari, Gujarati, Gurmukhi, Kannada, Malayalam, Oriya, Tamil, and Telugu. Taking Bengali as an example, transliterators Bengali-X and X-Bengali exist, where X is any of the other listed Indic scripts, or Latin.
• Deleted rules: UnicodeName-UnicodeChar has been replaced by Any-Name*. Latin-Arabic* and Latin-Hebrew* have been removed until they can be rewritten. KeyboardEscape-Latin1 has been replaced by Any-Accents and Any-Publishing. Latin-Kana* has been replaced by Latin-Katakana* and Latin-Hiragana*. [*applies to reverse rule as well]
• ID syntax changes: Transliterator IDs ignore case and whitespace now. They now have the standard form [filter]source-target/variant. The "[filter]" element is optional; if present, it limits the characters that the transliterator operates on. The "source-" element is optional; if omitted, it is taken to be Any. The "/variant" element is also optional; if present, it selects between different flavors of a related set of transliterators, for example, Greek-Latin and Greek-Latin/UNGEGN. The source, target, and variant specifiers are case-insensitive strings of the form /[_[:L:]][_[:L:][:N:]]*/.
• Locale support: The source, target, or both may be locales. In this case the transliterator rules will be looked up in the system locale resource bundles. Rules are sought under three tags, listed below. The text after the underscore in each tag is always canonicalized to uppercase before lookup. Note: The underscore is currently omitted from ICU4C tags, but will be restored when possible.
  • TransliterateTo_SCRIPT: Unidirectional rules from the enclosing locale to another script or specifier.
  • TransliterateFrom_SCRIPT: Unidirectional rules from another script or specifier to the enclosing locale.
  • Transliterate_SCRIPT: Bidirectional rules, with the forward direction being To and the reverse direction being From.
  Lookup proceeds in the following order:
  • In the dynamic registry: source-target
  • In the source locale: TransliterateTo_TARGET then Transliterate_TARGET (forward direction)
  • In the target locale: TransliterateFrom_SOURCE then Transliterate_SOURCE (reverse direction)
  If either the source or target specifier is not a locale then the corresponding locale lookup is skipped. If either is a locale, then locale fallback from aa_BB_CCC to aa_BB to aa is performed (where aa, BB, and CCC are the locale language, country, and variant). The final fallback is from the specifier, whether it is a locale or not (e.g., script abbreviation), to the long script name associated with that specifier. If a tag lookup succeeds, the attached element should be a string array of 2n items where n >= 1. Each pair of strings is a variant name and rule string. The variants are matched against the requested variant. If no variant is specified then the first variant is considered to match.
• Filters on compounds IDs: A filter on a compound transliterator can now be specified by giving a leading entry that contains a filter and no transliterator ID. For example, "[abc]; Latin-Katakana; Katakana-Hiragana" submits only the characters contained in the UnicodeSet [abc] to the compound transliterator Latin-Katakana; Katakana-Hiragana.
• Explicit reverse IDs: Typically if a transliterator A-B is formed, and its inverse is requested, the system tries to create B-A. That is, the source and target are exchanged. In some cases, the user may wish a different transliterator to be considered the reverse. In order to do this, the reverse ID is specified in parentheses immediately following the ID. For example, "A-B (B-C)" is a transliterator A-B whose inverse is B-C. If the ID of the inverse is requested, "B-C (A-B)" is returned. The forward or reverse component may be empty, so "(B-C)" and "A-B()" are legal IDs with Null transliterator for the forward and reverse direction, respectively. This is most useful in compounds where one element has no inverse or where a different inverse from the standard inverse is desired. For example, "Any-Lower(); Latin-Cyrillic".
• Quantifiers: Transliterator rules may now contain quantifiers '*', '+', and '?'. These indicate zero or more, one or more, and zero or one matches, respectively. Quantifiers apply to the last element, be it a single character, a UnicodeSet, a segment definition, or a quote; the entire preceding element is repeated. Quantifiers are implemented as greedy, non-backtracking matchers, unlike their typical implementation in regular expressions. As a result, expressions that match in a traditional regular expression engine (e.g., Perl) will not match in transliterator. E.g., "[a-z]+ q > x;" will not match "abcq", since the '+' quantifier consumes all four characters.
• Dot character: A new special character is recognized in rules, '.' (U+0020). This character matches any characters in the set [^[:Zp:][:Zl:]\r\n$]. Note the trailing '$' in the set pattern, which indicates that the ETHER character is not matched by '.'.
• ::ID blocks in rules: Transliterator IDs may now be included in rule sets. These may occur in two locations: as one contiguous block before any other rules, and as one contiguous block after all rules. The effect of placing ::IDs into a rule set is to enclose the rule-based transliterator within a compound transliterator containing the indicated IDs. The ::ID syntax is exactly the same as the standard ID syntax, with the difference that each ID element is preceded by the special token "::".
• Segment definitions more flexible: Segment definitions may be nested and are now unlimited in number. Prior to 2.0, segments could not be nested and were limited to nine ($1 to $9).
• Variable range pragma: A new pragma is supported. This follows the syntax:use variable range 0xE800 0xEFFF; (Any two code points may be specified.) The code points are specified as decimal constants, octal constants with a leading '0', or hexadecimal constants with a leading "0x". The given range is used internally for stand-in characters during processing. The default range is 0xF000..0xF8FF. If a rule set explicitly uses characters in the default variable range, a new range, not containing any characters in use in the rule set, must be specified. Note: This is the first of several planned pragmas.
• Factory method registration: Factory methods (function pointers in ICU4C; functor objects in ICU4J) may be registered against transliterator IDs. This is generally more efficient than the registration of singleton prototypes, since no actual transliterator object need be created until the user requires one. See the registerFactory() method in Transliterator.
• Filtering semantics changed for subclasses: Subclasses now need not concern themselves with filters. Instead, they may assume that all characters received by handleTransliterate() have already passed through the filter. This simplifies subclass code greatly.

UnicodeSet Improvements

• [:Any:] set: The set [:Any:] matches all Unicode code points, that is, U+0000..U+10FFFF.
• \p{} syntax: UnicodeSet now recognizes a Perlish syntax for character properties. Any property designated as [:Foo:] may equivalently be designated \p{Foo}.
• Short, medium, and long property names: In addition to the short property names, such as [:Ll:], equivalent medium (e.g., [:gc=Ll:]) and long (e.g., [:GeneralCategory=LowercaseLetter:]) forms are recognized. See the UnicodeSet Properties design document for details. As of this release, general categories, numeric value, and script are supported.

How to Download the Source Code

There are two ways to download ICU releases:

• Official Release Snapshot:
  If you want to use ICU (as opposed to developing it), you should download an official packaged version of the ICU source code. These versions are tested more thoroughly than day-to-day development builds of the system, and they are packaged in zip and tar files for convenient download. These packaged files can be found at http://oss.software.ibm.com/icu/download/.
  The packaged snapshots are named icu-nnnn.zip or icu-nnnn.tgz, where nnnn is the version number. The .zip file is used for Windows platforms, while the .tgz file is preferred on most other platforms.
  Please unzip this file. It will reconstruct the source directory, including anonymous CVS control directories (see below).
• CVS Source Repository:
  If you are interested in developing features, patches, or bug fixes for ICU, you should probably be working with the latest version of the ICU source code. You will need to check the code out of our CVS repository to ensure that you have the most recent version of all of the files. See our CVS page for details.

ICU Source Code Organization

In the descriptions below, <ICU> is the full path name of the icu directory - the top level directory from the distribution archives - in your file system.

How To Build And Install ICU

Supported Platforms

Key to testing frequency

Reference platform

ICU will work on these platforms with these compilers

Regularly tested

ICU should work on these platforms with these compilers

Rarely tested

ICU has been ported to these platforms but may not have been tested there recently

How To Build And Install On Windows

Building International Components for Unicode requires:

• Microsoft NT 4.0 and above, or Windows 98 and above
• Microsoft Visual C++ 6.0 (Service Pack 2 is required to work with the release build of max speed optimization).

The steps are:

1. Unzip the icu-XXXX.zip file into any convenient location. Using command line zip, type "unzip -a icu-XXXX.zip -d drive:\directory", or just use WinZip.
2. Be sure that the ICU binary directory, <ICU>\bin\, is included in the PATH environment variable. The tests will not work without the location of the ICU dll files in the path.
3. Set the TZ environment variable to PST8PDT. The tests will not work in any other timezone.
4. Open the "<ICU>\source\allinone\allinone.dsw" workspace file in Microsoft Visual C++ 6.0. (This workspace includes all the International Components for Unicode libraries, necessary ICU building tools, and the intltest and cintltest test suite projects). Please see the note below if you want to build from the command line instead.
5. Set the active Project to the "all" project. To do this: Choose "Project" menu, and select "Set active project". In the submenu, select the "all" workspace.
6. Set the active configuration to "Win32 Debug" or "Win32 Release" (See note below).
7. Choose the "Build" menu and select "Rebuild All". If you want to build the Debug and Release at the same time, see the note below.
8. Run the C++ test suite, "intltest". To do this: set the active project to "intltest", and press F5 to run it.
9. Run the C test suite, "cintltst". To do this: set the active project to "cintltst", and press F5 to run it.
10. Make sure that both "cintltst" and "intltest" passed without any errors. The return codes are non-zero when they do not pass. Visual C++ will display the return codes in the debug tag of the output window. When "intltest" and "cintltest" return 0, it means that everything is installed correctly. You can press Ctrl+F5 on the test project to run the test and see what error messages were displayed (if any tests failed).
11. Reset the TZ environment variable to its original value, unless you plan on testing ICU any further.
12. You are now able to develop applications with ICU.

Using MSDEV At The Command Line Note: You can build ICU from the command line. Assuming that you have properly installed Microsoft Visual C++ to support command line execution, you can run the following command, 'msdev <ICU>\source\allinone\allinone.dsw /MAKE "ALL"'.

Setting Active Configuration Note: To set the active configuration, two different possibilities are:

• Choose "Build" menu, select "Set Active Configuration", and select "Win32 Release" or "Win32 Debug".
• Another way is to select "Customize" in the "Tools" menu, select the "Toolbars" tab, enable "Build" instead of "Build Minibar", and click on "Close". This will bring up a toolbar which you can move aside the other permanent toolbars at the top of the MSVC window. The advantage is that you now have an easy-to-reach pop-up menu that will always show the currently selected active configuration. Or, you can drag the project and configuration selections and drop them on the menu bar for later selection.

Batch Configuration Note: If you want to build the Debug and Release configurations at the same time, choose "Build" menu and select "Batch Build..." instead (and mark all configurations as checked), then click the button named "Rebuild All". The "all" workspace will build all the test programs as well as the tools for generating binary locale data files. The "makedata" project will be run automatically to convert the locale data files from text format into icudata.dll.

How To Build And Install On Unix

Building International Components for Unicode on Unix requires:

A UNIX C++ compiler, (gcc, cc, xlc_r, etc...) installed on the target machine. A recent version of GNU make (3.7+). For a list of OS/390 tools please view the OS/390 build section of this document for further details.

The steps are:

1. Decompress the icuXXXX.tar (or icuXXXX.tgz) file. For example, gunzip -d < icuXXXX.tgz | tar xvf -
2. Change directory to the "icu/source".
3. chmod +x runConfigureICU install-sh
4. Run the runConfigureICU script for your platform. If you are not using the runConfigureICU script or your platform is not supported by the script, you need to set your CC, CXX, CFLAGS and CXXFLAGS environment variables, and type "./configure". You can type "./configure --help" to print the available options.
5. Type "gmake" to compile the libraries and all the data files.
6. Optionally, type "gmake check" to verify the test suite.
   • Note: You may have to set certain variables if you with to run test programs individually, that is apart from "make check". The TZ environment variable needs to be set to PST8PDT. Also, the environment variable ICU_DATA must be set to the full pathname of the data directory, to indicate where the locale data files and conversion mapping tables are. The trailing "/" is required after the directory name (e.g. "$Root/source/data/" will work, but the value "$Root/source/data" is not acceptable).

     When running samples or other applications, ICU_DATA only needs to be set if the data is not installed (such as via 'make install') into the default location.

7. Type "gmake install" to install.

Some platforms use package management tools to control the installation and uninstallation of files on the system, as well as the integrity of the system configuration. You may want to check if ICU can be packaged for your package management tools by looking into the "packaging" directory. (Please note that if you are using a snapshot of ICU from CVS, it is probable that the packaging scripts or related files are not up to date with the contents of ICU at this time, so use them with caution.)

OS/390 (zSeries) Platform

If you are building on the OS/390 UNIX System Services platform, it is important that you understand a few details:

• The gnu utilities gmake and gzip/gunzip are needed and can be obtained for OS/390 from z/OS Unix - Tools and Toys. Documentation on these tools can be found at the Open Source Software for OS/390 UNIX Red Book.
• Encoding considerations: The source code assumes that it is compiled with codepage ibm-1047 (to be exact, the UNIX System Services variant of it). The pax command converts all of the source code files from ASCII to codepage ibm-1047 (USS) EBCDIC. However, some files are binary files and must not be converted, or must be converted back to their original state. You can use the unpax-icu.sh script to do this for you automatically. It will unpackage the tar file and convert all the necessary files for you automatically.

• OS/390 supports both native S/390 hexadecimal floating point and, (with Version 2.6 and later) IEEE binary floating point. This is a compile time option. Applications built with IEEE should use ICU dlls that are built with IEEE (and vice versa). The environment variable IEEE390=1 will cause the OS/390 version of ICU to be built with IEEE floating point. The default is native hexadecimal floating point.
  Important: Currently (ICU 1.4.2), native floating point support is sufficient for codepage conversion, resource bundle and UnicodeString operations, but the Format APIs, especially ChoiceFormat, require IEEE binary floating point.

  Examples for configuring ICU:
  Debug build: IEEE390=1 ./runConfigureICU --enable-debug zOS/cxx
  Release build: IEEE390=1 ./runConfigureICU zOS/cxx

• Since the default make on OS/390 is not gmake, the pkgdata tool requires that the "make" command is aliased to your installed version of gmake.
• The makedep executable that is used with the OS/390 ICU build process is not shipped with ICU. It is available at the z/OS Unix - Tools and Toys site. The PATH environment variable should be updated to contain the location of this executable prior to build. Alternatively, makedep may be moved into an existing PATH directory.
• To run all of the tests for ICU, use "gmake check". When running individual tests of the test suite, the TZ environment variable should be set to export TZ="PST8PDT" so that time zone comparisons are correct. Building and testing ICU without using gmake requires that the ICU libraries in the LIBPATH. In other words, the LIBPATH should contain (each path prepended with the root directory that contains the icu directory):
  • icu/source/common
  • icu/source/data
  • icu/source/i18n
  • icu/source/tools/ctestfw
  • icu/source/tools/toolutil
  • icu/source/extra/ustdio

OS/390 Batch (PDS) support

By default, ICU builds its libraries into the HFS. However, there is a 390-specific switch to build some libraries into PDS files. The switch is the environmental variable OS390BATCH, and if set, the following libraries are built into PDS files: libicuucXX.dll, libicudtXXe.dll, libicudtXXe_390.dll, and libtestdata.dll. Turning on OS390BATCH does not turn off the normal HFS build, thus the HFS dlls will always be created.

The names of the PDS files are determined by the value of the environmental variables LOADMOD and LOADEXP. These variables must contain the target PDS names whenever the OS390BATCH variable is set. LOADMOD is the library (.dll) target dataset and LOADEXP is the side deck (.x) target dataset.

The PDS member names are as follows:

IXMIXXUC --> libicuucXX.dll
IXMIXXDA --> libicudtXXe.dll
IXMIXXD1 --> libicudtXXe_390.dll

Example PDS attributes are as follows:

Data Set Name . . . : USER.ICU.LOAD
General Data
Management class. . : **None**
Storage class . . . : BASE
Volume serial . . . : TSO007
Device type . . . . : 3390
Data class. . . . . : LOAD
Organization  . . . : PO
Record format . . . : U
Record length . . . : 0
Block size  . . . . : 32760
1st extent cylinders: 40
Secondary cylinders : 59
Data set name type  : PDS

Data Set Name . . . : USER.ICU.EXP
General Data
Management class. . : **None**
Storage class . . . : BASE
Volume serial . . . : TSO007
Device type . . . . : 3390
Data class. . . . . : **None**
Organization  . . . : PO
Record format . . . : FB
Record length . . . : 80
Block size  . . . . : 3200
1st extent cylinders: 3
Secondary cylinders : 3
Data set name type  : PDS

OS/400 (iSeries) Platform

ICU Reference Release 1.8.1 contains partial support for the 400 platform, but additional work by the user is currently needed to get it to build properly. A future release of ICU should work out-of-the-box under OS/400.

• Requirements:
  • QSHELL interpreter installed (install base option 30, operating system)
  • ILE C/C++ Compiler for iSeries, LPP 5722-WDS
  • The latest GNU facilities (You can get the GNU facilities for OS/400 from http://www.as400.ibm.com/developer/porting/gnu_utilities.html). Older versions may not work properly.
• Build environment setup:
  1. Create AS400 target library. This library will be the target for the resulting modules, programs and service programs. You will specify this library on the OUTPUTDIR environment variable in step 2.
     

     CRTLIB LIB(libraryname)
     

     
     
  2. Set up the following environment variables in your build process (use the libraryname from the previous step)

     ADDENVVAR ENVVAR(ICU_DATA) VALUE('/icu/source/data')
     ADDENVVAR ENVVAR(CC) VALUE('/usr/bin/icc')
     ADDENVVAR ENVVAR(CXX) VALUE('/usr/bin/icc')
     ADDENVVAR ENVVAR(MAKE) VALUE('/usr/bin/gmake')
     ADDENVVAR ENVVAR(OUTPUTDIR) VALUE('libraryname')
     

     libraryname identifies target as400 library for *module, *pgm and *srvpgm objects.
     
     
  3. In order to get the tests to run correctly, the QUTCOFFSET needs to be set to the Pacific Time Zone offset.
     
     To check your QUTCOFFSET:

     DSPSYSVAL SYSVAL(QUTCOFFSET)
     

     
     To change your QUTCOFFSET:
     

     CHGSYSVAL SYSVAL(QUTCOFFSET) VALUE('-0800')
     

     You should change -0800 to -0700 for daylight savings.
     
     
  4. Run 'CHGJOB CCSID(37)'
  5. Run 'QSH'
  6. Run gunzip on the ICU source code compressed tar archive (icu-X-Y.tar.gz or icu-X-Y.tgz).
  7. Run unpax-icu.sh on the tar file from the ICU download page.
  8. Change your current directory to icu/source.
  9. Run 'as_is/os400/configure --host=as400-os400 --with-data-packaging=archive'
  10. Run 'gmake -e'. The '-e' option is needed to pickup the compilers.
  11. Run 'gmake -e check' to run the tests.

Important Notes About Using ICU

Windows Platform

If you are building on the Win32 platform, it is important that you understand a few of the following build details.

DLL directories and the PATH setting

As delivered, the International Components for Unicode build as several DLLs which are placed in the "<ICU>\bin" directory. You must add this directory to the PATH environment variable in your system, or any executables you build will not be able to access International Components for Unicode libraries. Alternatively, you can copy the DLL files into a directory already in your PATH, but we do not recommend this. You can wind up with multiple copies of the DLL and wind up using the wrong one.

Changing your PATH

• Windows 2000: Use the System Icon in the Control Panel. Pick the "Advanced" tab. Select the "Environment Variables..." button. Select the variable PATH in the lower box, and select the lower "Edit..." button. In the "Variable Value" box, append the string ";<ICU>\bin" to the end of the path string. If there is nothing there, just type in "<ICU>\bin". Click the Set button, then the OK button.
• Windows NT: Use the System Icon in the Control Panel. Pick the "Environment" tab, and select the variable PATH in the lower box. In the "value" box, append the string ";<ICU>\bin" at the end of the path string. If there is nothing there, just type in "<ICU>\bin". Click the Set button, then the OK button.
• Windows 95/98/ME: Edit the autoexec.bat, and add the following line to the end of file, "SET PATH=%PATH%;<ICU>\bin"

Note: when packaging a Windows application for distribution and installation on user systems, copies of the ICU dlls should be included with the application, and installed for exclusive use by the application. This is the only way to insure that your app is running with the same version of ICU, built with exactly the same options, that you developed and tested with. Refer to Microsoft's guidelines on the usage of dlls, or search for the phrase "dll hell" on msdn.microsoft.com.

Linking with Runtime libraries

All the DLLs link with the C runtime library "Debug Multithreaded DLL" or "Multithreaded DLL." (This is changed through the Project Settings dialog, on the C/C++ tab, under Code Generation.) It is important that any executable or other DLL you build which uses the International Components for Unicode DLLs links with these runtime libraries as well. If you do not do this, you will get random memory errors when you run the executable.

Unix Type Platform

If you are building on a Unix platform, it is important that you add the location of your ICU libraries (including the data library) to your LD_LIBRARY_PATH environment variable. The ICU libraries may not link or load properly without doing this.

Using the default codepage

ICU has code to determine the default codepage of the system or process. This default codepage can be used to convert char * strings to and from Unicode.

Depending on system design, setup and APIs, it may not always be possible to find a default codepage that fully works as expected. For example,

• On Windows there are three encodings in use at the same time. Unicode (UTF-16) is always used inside of Windows, while for char * encodings there are two classes, called "ANSI" and "OEM" codepages. ICU will use the ANSI codepage. Note that the OEM codepage is used by default for console window output.
• On some Unix-type systems, non-standard names are used for encodings, or non-standard encodings are used altogether. Although ICU supports 200 encodings in its standard build and many more aliases for them, it will not be able to recognize such non-standard names.
• Some systems do not have a notion of a system or process codepage, and may not have APIs for that.

If you have means of detecting a default codepage name that are more appropriate for your application, then you should set that name with ucnv_setDefaultName() as the first ICU function call. This makes sure that the internally cached default converter will be instantiated from your preferred name.

Starting in ICU 2.0, when a converter for the default codepage cannot be opened, a fallback default codepage name and converter will be used. On most platforms, this will be US-ASCII. For OS/390 (z/OS), ibm-1047-s390 is the default fallback codepage. For AS/400 (iSeries), ibm-37 is the default fallback codepage. This default fallback codepage is used when the operating system is using a non-standard name for a default codepage, or the converter was not packaged with ICU. The feature allows ICU to run in unusual computing environments without completely failing.

Methods for enabling deprecated APIs

C

Some deprecated C APIs can be enabled without recompiling the ICU libraries. This can be achieved by defining certain symbols before including the ICU header files. For example, to enable deprecated C APIs for formatting.

#ifndef U_USE_DEPRECATED_FORMAT_API
#  define U_USE_DEPRECATED_FORMAT_API 1
#endif

#include "unicode/udat.h"

int main(){
    UDateFormat *def, *fr, *fr_pat ;
    UErrorCode status = U_ZERO_ERROR;
    UChar temp[30];

    fr = udat_open(UDAT_FULL, UDAT_DEFAULT, "fr_FR", NULL,0, &status);
    if(U_FAILURE(status)){
        printf("Error creating the french dateformat using full time style\n %s\n",
            myErrorName(status) );
    }
    /* This is supposed to open default date format,
       but later on it treats it like it is "en_US".
       This is very bad when you try to run the tests
       on a machine where the default locale is NOT "en_US"
    */
    def = udat_open(UDAT_SHORT, UDAT_SHORT, "en_US", NULL, 0, &status);
    if(U_FAILURE(status)){
        .... /* handle the error */
    }
}

C++

Deprecated C++ APIs cannot be enabled without recompiling ICU libraries. Every service has a specific symbol that should be defined to enable the deprecated API of that service. For example: To enable deprecated APIs in Transliteration service, the U_USE_DEPRECATED_TRANSLITERATOR_API symbol should be defined before compiling ICU.

Platform Dependencies

The platform dependencies have been mostly isolated into the following files in the common library. This information can be useful if you are porting ICU to a new platform.

• unicode/platform.h.in (autoconf'ed platforms)
  unicode/pXXXX.h (others: pwin32.h, pmacos.h, ..): Platform-dependent typedefs and defines:
  
  
  • XP_CPLUSPLUS for C++ only.
  • TRUE and FALSE, UBool, int8_t, int16_t etc.
  • U_EXPORT and U_IMPORT for specifying dynamic library import and export
  
  
• unicode/putil.h, putil.c: platform-dependent implementations of various functions that are platform dependent:
  
  
  • uprv_isNaN, uprv_isInfinite, uprv_getNaN and uprv_getInfinity for handling special floating point values.
  • uprv_tzset, uprv_timezone, uprv_tzname and time for getting platform specific time and timezone information.
  • u_getDataDirectory for getting the default data directory.
  • uprv_getDefaultLocaleID for getting the default locale setting.
  • uprv_getDefaultCodepage for getting the default codepage encoding.
  
  
• umutex.h, umutex.c: Code for doing synchronization in multithreaded applications. If you wish to use International Components for Unicode in a multithreaded application, you must provide a synchronization primitive that the classes can use to protect their global data against simultaneous modifications. See Users' guide for more information.
  
  
  • We supply sample implementations for WinNT, Win95, Win98, Sun/Solaris, RedHat/Linux, HP-UX and for AIX on an RS/6000.
  
  
• umapfile.h, umapfile.c: functions for mapping or otherwise reading or loading files into memory. All access by ICU to data from files makes use of these functions.
  
  
• For the Intltest test suite, intltest.cpp in "icu/source/test/intltest/" contains the method pathnameInContext, which must also be adapted to any new platform.
• Using platform specific #ifdef macros are highly discouraged outside of the scope of these files. When the source code gets updated in the future, these #ifdef's can cause testing problems for your platform.

It is possible to build each library individually. They must be built in the following order:

1. stubdata
2. common
3. i18n
4. toolutil
5. makeconv
6. genrb
7. gentz
8. genccode
9. gennames
10. genuca
11. gennorm
12. makedata (a project on Windows, or source/data/Makefile on Unix)
13. ctestfw, intltest and cintltst, if you want to run the test suite.

Copyright © 1997-2002 International Business Machines Corporation and others. All Rights Reserved.
IBM Globalization Center of Competency - San Jose,
5600 Cottle Road, San José, CA 95193
All rights reserved.