allow msgids in !=English languages (based on Stefan Kowski's patch)

git-svn-id: https://svn.wxwidgets.org/svn/wx/wxWidgets/trunk@27980 c3d73ce0-8a6f-49c7-b76d-6d57e0e08775

docs/changes.txt

@@ -108,6 +108,8 @@ All:
 
 - number of fixes to wxPluginManager (Rick Brice, Hans Van Leemputten)
 - fixed memory leak in wxURL when using a proxy (Steven Van Ingelgem)
+- it's now possible to use msgids in other languages than English with
+  wxLocale (based on patch by Stefan Kowski)
 
 All (GUI):
 

docs/latex/wx/locale.tex

@@ -348,17 +348,42 @@ set locale is restored and so the changes described in
 
 \func{bool}{AddCatalog}{\param{const char }{*szDomain}}
 
+\func{bool}{AddCatalog}{\param{const char }{*szDomain}, \param{wxLanguage}{msgIdLanguage}, \param{const char }{*msgIdCharset}}
+
 Add a catalog for use with the current locale: it is searched for in standard
 places (current directory first, then the system one), but you may also prepend
 additional directories to the search path with 
 \helpref{AddCatalogLookupPathPrefix()}{wxlocaleaddcataloglookuppathprefix}.
 
-All loaded catalogs will be used for message lookup by GetString() for the
-current locale.
+All loaded catalogs will be used for message lookup by
+\helpref{GetString()}{wxlocalegetstring} for the current locale.
 
 Returns true if catalog was successfully loaded, false otherwise (which might
 mean that the catalog is not found or that it isn't in the correct format).
 
+The second form of this method takes two additional arguments,
+\arg{msgIdLanguage} and \arg{msgIdCharset}.
+
+\arg{msgIdLanguage} specifies the language of "msgid" strings in source code
+(i.e. arguments to \helpref{GetString}{wxlocalegetstring},
+\helpref{wxGetTranslation}{wxgettranslation} and the 
+\helpref{\_()}{underscore} macro). It is used if AddCatalog cannot find any
+catalog for current language: if the language is same as source code language,
+then strings from source code are used instead. 
+
+\arg{msgIdCharset} lets you specify the charset used for msgids in sources
+in case they use 8-bit characters (e.g. German or French strings).  This
+argument has no effect in Unicode build, because literals in sources are
+Unicode strings; you have to use compiler-specific method of setting the right
+charset when compiling with Unicode.
+
+By default (i.e. when you use the first form), msgid strings are assumed
+to be in English and written only using 7-bit ASCII characters.
+
+If you have to deal with non-English strings or 8-bit characters in the source
+code, see the instructions in 
+\helpref{Writing non-English applications}{nonenglishoverview}.
+
 \membersection{wxLocale::AddCatalogLookupPathPrefix}\label{wxlocaleaddcataloglookuppathprefix}
 
 \func{void}{AddCatalogLookupPathPrefix}{\param{const wxString\& }{prefix}}
@@ -497,6 +522,9 @@ message catalog is found \arg{szOrigString} is returned if `n == 1',
 otherwise \arg{szOrigString2}.
 See \urlref{GNU gettext manual}{http://www.gnu.org/manual/gettext/html\_chapter/gettext\_10.html\#SEC150} for additional information on plural forms handling.
 
+This method is called by the \helpref{wxGetTranslation}{wxgettranslation}
+function and \helpref{\_()}{underscore} macro.
+
 \wxheading{Remarks}
 
 Domains are searched in the last to first order, i.e. catalogs
@@ -597,7 +625,8 @@ The call of this function has several global side effects which you should
 understand: first of all, the application locale is changed - note that this
 will affect many of standard C library functions such as printf() or strftime().
 Second, this wxLocale object becomes the new current global locale for the
-application and so all subsequent calls to wxGetTranslation() will try to
+application and so all subsequent calls to
+\helpref{wxGetTranslation()}{wxgettranslation} will try to
 translate the messages using the message catalogs for this locale.
 
 Returns true on success or false if the given locale couldn't be set.

docs/latex/wx/tnoneng.tex

@@ -88,6 +88,29 @@ user's operating system. This is default behaviour of the
 \helpref{wxLocale}{wxlocale} class; you can disable it by {\bf not} passing
 {\tt wxLOCALE\_CONV\_ENCODING} to \helpref{wxLocale::Init}{wxlocaleinit}.
 
+\wxheading{Non-English strings or 8-bit characters in the source code}
+
+By convention, you should only use characters without diacritics (i.e. 7-bit
+ASCII strings) for msgids in the source code and write them in English.
+
+If you port software to wxWindows, you may be confronted with legacy source
+code containing non-English string literals. Instead of translating the strings
+in the source code to English and putting the original strings into message
+catalog, you may configure wxWidgets to use non-English msgids and translate to
+English using message catalogs:
+
+\begin{enumerate}
+\item{If you use the program {\tt xgettext} to extract the strings from
+the source code, specify the option {\tt --from-code=<source code charset>}.}
+\item{Specify the source code language and charset as arguments to
+\helpref{wxLocale::AddCatalog}{wxlocaleaddcatalog}. For example:
+\begin{verbatim}
+locale.AddCatalog(_T("myapp"),
+                  wxLANGUAGE_GERMAN, _T("iso-8859-1"));
+\end{verbatim}
+}
+\end{enumerate}
+
 \wxheading{Font mapping}
 
 You can use \helpref{wxMBConv classes}{mbconvclasses} and 

include/wx/intl.h

@@ -455,6 +455,8 @@ public:
     //
     // Returns 'true' if it was successfully loaded
     bool AddCatalog(const wxChar *szDomain);
+    bool AddCatalog(const wxChar *szDomain,
+                    wxLanguage msgIdLanguage, const wxChar *msgIdCharset);
 
     // check if the given catalog is loaded
     bool IsLoaded(const wxChar *szDomain) const;

src/common/intl.cpp

@@ -887,7 +887,8 @@ public:
               wxPluralFormsCalculatorPtr& rPluralFormsCalculator);
 
     // fills the hash with string-translation pairs
-    void FillHash(wxMessagesHash& hash, bool convertEncoding) const;
+    void FillHash(wxMessagesHash& hash, const wxString& msgIdCharset,
+                  bool convertEncoding) const;
 
 private:
     // this implementation is binary compatible with GNU gettext() version 0.10
@@ -963,7 +964,8 @@ class wxMsgCatalog
 {
 public:
     // load the catalog from disk (szDirPrefix corresponds to language)
-    bool Load(const wxChar *szDirPrefix, const wxChar *szName, bool bConvertEncoding = FALSE);
+    bool Load(const wxChar *szDirPrefix, const wxChar *szName,
+              const wxChar *msgIdCharset = NULL, bool bConvertEncoding = false);
 
     // get name of the catalog
     wxString GetName() const { return m_name; }
@@ -1214,7 +1216,9 @@ bool wxMsgCatalogFile::Load(const wxChar *szDirPrefix, const wxChar *szName0,
   return true;
 }
 
-void wxMsgCatalogFile::FillHash(wxMessagesHash& hash, bool convertEncoding) const
+void wxMsgCatalogFile::FillHash(wxMessagesHash& hash,
+                                const wxString& msgIdCharset,
+                                bool convertEncoding) const
 {
 #if wxUSE_WCHAR_T
     wxCSConv *csConv = NULL;
@@ -1222,7 +1226,15 @@ void wxMsgCatalogFile::FillHash(wxMessagesHash& hash, bool convertEncoding) cons
         csConv = new wxCSConv(m_charset);
 
     wxMBConv& inputConv = csConv ? *((wxMBConv*)csConv) : *wxConvCurrent;
+
+    wxCSConv *sourceConv = NULL;
+    if ( !msgIdCharset.empty() && (m_charset != msgIdCharset) )
+        sourceConv = new wxCSConv(msgIdCharset);
+
 #elif wxUSE_FONTMAP
+    wxASSERT_MSG( msgIdCharset == NULL,
+                  _T("non-ASCII msgid languages only supported if wxUSE_WCHAR_T=1") );
+    
     wxEncodingConverter converter;
     if ( convertEncoding )
     {
@@ -1259,11 +1271,17 @@ void wxMsgCatalogFile::FillHash(wxMessagesHash& hash, bool convertEncoding) cons
     for (size_t i = 0; i < m_numStrings; i++)
     {
         const char *data = StringAtOfs(m_pOrigTable, i);
-#if wxUSE_WCHAR_T
+#if wxUSE_UNICODE
         wxString msgid(data, inputConv);
 #else
-        wxString msgid(data);
+        wxString msgid;
+#if wxUSE_WCHAR_T
+        if ( convertEncoding && sourceConv )
+            msgid = wxString(inputConv.cMB2WC(data), *sourceConv);
+        else
 #endif
+            msgid = data;
+#endif // wxUSE_UNICODE
 
         data = StringAtOfs(m_pTransTable, i);
         size_t length = Swap(m_pTransTable[i].nLen);
@@ -1300,6 +1318,7 @@ void wxMsgCatalogFile::FillHash(wxMessagesHash& hash, bool convertEncoding) cons
     }
 
 #if wxUSE_WCHAR_T
+    delete sourceConv;
     delete csConv;
 #endif
 }
@@ -1310,7 +1329,7 @@ void wxMsgCatalogFile::FillHash(wxMessagesHash& hash, bool convertEncoding) cons
 // ----------------------------------------------------------------------------
 
 bool wxMsgCatalog::Load(const wxChar *szDirPrefix, const wxChar *szName,
-                        bool bConvertEncoding)
+                        const wxChar *msgIdCharset, bool bConvertEncoding)
 {
     wxMsgCatalogFile file;
 
@@ -1318,7 +1337,7 @@ bool wxMsgCatalog::Load(const wxChar *szDirPrefix, const wxChar *szName,
 
     if ( file.Load(szDirPrefix, szName, m_pluralFormsCalculator) )
     {
-        file.FillHash(m_messages, bConvertEncoding);
+        file.FillHash(m_messages, msgIdCharset, bConvertEncoding);
         return TRUE;
     }
 
@@ -2490,27 +2509,47 @@ bool wxLocale::IsLoaded(const wxChar *szDomain) const
 
 // add a catalog to our linked list
 bool wxLocale::AddCatalog(const wxChar *szDomain)
+{
+    return AddCatalog(szDomain, wxLANGUAGE_ENGLISH, NULL);
+}
+
+// add a catalog to our linked list
+bool wxLocale::AddCatalog(const wxChar *szDomain,
+                          wxLanguage    msgIdLanguage,
+                          const wxChar *msgIdCharset)
+
 {
   wxMsgCatalog *pMsgCat = new wxMsgCatalog;
 
-  if ( pMsgCat->Load(m_strShort, szDomain, m_bConvertEncoding) ) {
+  if ( pMsgCat->Load(m_strShort, szDomain, msgIdCharset, m_bConvertEncoding) ) {
     // add it to the head of the list so that in GetString it will
     // be searched before the catalogs added earlier
     pMsgCat->m_pNext = m_pMsgCat;
     m_pMsgCat = pMsgCat;
 
-    return TRUE;
+    return true;
   }
   else {
     // don't add it because it couldn't be loaded anyway
     delete pMsgCat;
 
-    // it's OK to not load English catalog, the texts are embedded in
-    // the program:
-    if (m_strShort.Mid(0, 2) == wxT("en"))
-        return TRUE;
+    // It is OK to not load catalog if the msgid language and m_language match,
+    // in which case we can directly display the texts embedded in program's
+    // source code:
+    if (m_language == msgIdLanguage)
+        return true;
 
-    return FALSE;
+    // If there's no exact match, we may still get partial match where the
+    // (basic) language is same, but the country differs. For example, it's
+    // permitted to use en_US strings from sources even if m_language is en_GB:
+    const wxLanguageInfo *msgIdLangInfo = GetLanguageInfo(msgIdLanguage);
+    if ( msgIdLangInfo &&
+         msgIdLangInfo->CanonicalName.Mid(0, 2) == m_strShort.Mid(0, 2) )
+    {
+        return true;
+    }
+
+    return false;
   }
 }