What are product scenarios and requirements related to use of multibyte and handling encodings ?

Information provided by Ken Frank, Tools I18N quality and testing

Contact kfrank@netbeans.org for more information or with comments or questions about these pages.

This document does not include info on which actual encodings or type of encodings to use to solve these issues - guidelines on that is in another attached document.

This information is about what scenarios might be in which users would use non ascii and multibyte in their projects or applications and can help see then what encoding handling approaches might be needed to enable this to happen.

NOTE - there is a new project encoding property and more enhanced handling of project and file encoding in netbeans 6.0. The overall name for these features is file encoding query (FEQ)

see these nb api javadoc classes on these for more information at implementation level:



This document tells more details about this; it is a collection of information from various developers who implemented it for projects and various file types.

and see these documents
http://wiki.netbeans.org/wiki/view/FaqI18nProjectEncoding http://wiki.netbeans.org/wiki/view/FaqI18nChangeProjectEncodingImpact http://wiki.netbeans.org/wiki/view/FaqI18nFileEncodingQueryObject

and these other documents on the wiki about encoding:





The information in these docs and api might replace some details below if they refer to specific api, but the principles of encoding handling and how to test it are the same.

Use of multibyte in file names and contents as well as in properties and other data

         o as long as use of multibyte in name or contents in various files or in other module data or properties is legal in java language and in j2ee, ejb, web services or jsp/web apps specifications, then it should be legal to use multibyte in these places, assuming that use would be legal if english was being used in any given place.
   *  does the module code handle the encoding of such file names and data correctly, in all cases, which are elaborated below ?
         * What might be analyzed to meet this requirement:
                -- list the names of the all the control and configuration files generated by or used by the modules (or created by user) as well as data values/properties that can be modified by user with string/character data (includes deployment related config files also)
             	  + this would include data both sent from module to outside itself or ide, as well as data from external apps or processes coming into the module. (like for database or external web services, or info from appserver, etc.)
               +   for each of the above files/filetypes, which of them have been coded such that multibyte can be used:
  	 * in names of the files
  	 * in file contents where otherwise legal
 	  * in properties or values generated or that user can input that result in files being created or modified (sometimes the use of multibyte is explicit by user, other times ide takes some property with multibyte and uses it to create names of other objects and files)
 	  *  how encoding is handled for each file or data, both internal to the same module, between modules and between ide and external products or processes -- which is needed to ensure  that any multibyte will display properly
	   * that no data loss or corruption would occur
   * often this part is understanding when to use utf-8 encoding and when to use encoding of the locale the user is in (system locale)
   * this could apply to data shown in UI of product, data sent to or received from external process/application or data or program/app sent to browser.
   * (for user app seen in browser, its expected user would set encoding in jsp, but if module is doing this for some other reason, it needs to handle encoding setting so that data it sends is shown ok)
   * be sure to make sure the encoding has been handled ok  for generated code also - if user sets a property value in property editor to have multibyte, make sure encoding was handled ok and displays properly in the generated java or other code

Viewing of multibyte (ie encoding handling)

that multibyte can be displayed correctly in all applicable ide windows and dialogs, which includes various properties windows and editors, output window, etc.

What might be analyzed to meet this requirement is:
                know how encoding for data in each UI area is to be procesed so that multibyte will be handled ok. Sometimes the data is from files previously written and reopened on ide restart.
               to not hardcode font names but use standard jdk font names
                know that ide output window just displays what is sent to it - it assumes that proper encoding and font handling has been done by the module that sends data to it.

Viewing of multibyte after restart of ide after files/data modified (ie how are files/data are generated, read or written/saved as to encoding ?)

         * Another common situation is where user uses multibyte in file name or data and then saves or the creates or saves the file or data and then user restarts ide
         * the multibyte might look ok when user originally was in the ide, but if code does not handle encoding propertly when creating or saving the file or data or when reading it in on restart, the multibyte might not display properly and/or the data is not valid, since encoding was not handled properly.
       What might be analyzed to meet this requirement is:
                know how encoding is handled for files/data being created, saved and for files/data being read in

Files or data in userdir should not be used as assumption of what encoding to use since user might use same userdir but be in a different locale.

     What might be analyzed to meet this requirement is:
          Are there any files in userdir or other places not in actual installed product, where information is kept that would effect encoding to be used or that would be basis of a message or label shown to user ?
Not logged in. Log in, Register

By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo