Current Status of Sinhala Unicode with issues identified and workarounds, if available, based upon testing performed on selected Mobile and Desktop Operating Systems

Current Status of Sinhala Unicode with issues identified and workarounds, if available, based upon testing performed on selected Mobile and Desktop Operating Systems

Harsha Wijayawardhana B.Sc. (Miami), CITP (UK), FBCS (UK)

Introduction

Help Centre for Sinhala and Tamil Unicode functions as a hotline as well as a Help Centre, catering to Sinhala and Tamil Unicode users to address technical issues. Theekshana set up the Help Centre with financial assistance from LK Domain Registry in 2018. It not only filled a vacuum of non-availability of help in addressing Sinhala and Tamil Unicode problems but also in the formulation of Sinhala and Tamil Unicode policy. The non-availability of technical assistance in Sinhala and Unicode became further aggravated due to the dormancy of the Local Language Working Group (LLWG) of ICTA. The absence of LLWG led to many contentious issues relating to the Sinhala Unicode Chart where Unicode Consortium encoded Chandra Bindu in Sinhala BMP without considering the opposition of Sri Lankan scholars.

In 2020/21, Help Centre became pivotal in the resurrection of former dormant LLWG. The new LLWG regrouped in the latter 2021 due to primary efforts of the Help Centre and LK Domain Registry. In addition, the Help Centre provided technical assistance to Government organizations on Unicode issues in parallel while helping the public. Several individuals and Government organizations sought assistance in solving technical issues, and some wanted Theekshana to get involved in mediation with multinationals such as Microsoft, Oracle, and Google relating to Unicode issues-found. As a result, Theekshana conducted regular testing on Sinhala Unicode using known word processors and Operating Systems (OSs). The following are findings of such testing conducted recently on different versions of Microsoft word (primarily on Windows 10), Apple IOS, Android and on Social Media such as Facebook. Also, Theekshana tested using the same test cases on Windows 11 and logged the findings/issues with Microsoft recently. In addition to popular Mobile, Desktop Operating Systems, Theekshana intends to test popular Linux flavors.     

Findings and workarounds for common issues

1. Testing Part 1 – Sinhala Unicode Typing on Microsoft Office 2013, 2016 and 2019

Testing has been carried out by using the Sinhala alphabet and sample words which are commonly used.

1) Testing Results and Summary for Microsoft Office Professional Plus 2013 with Iskoola Pota Unicode Font

• MS Word 2013

            No Issues Found. Sample words and Letters can be typed properly.

• MS Excel 2013

            No Issues Found. Sample words and Letters can be typed properly.

• MS PowerPoint 2013

Words with Yansaya, Repaya, and Rakaransaya cannot be typed properly. Following   figure 1 presents the testing results of sample Sinhala word list

• Solution / Workaround – To solve the above problem, as a workaround, type those words in a notepad and then copy and paste to the word document. Another way is typing those words in google input tools and then copy and paste them onto a word document.

2) Testing Results and Summary for Microsoft Office 2016 with Iskoola Pota Unicode Font

• MS Word 2016

                         No Issues Found. Sample words and Letters can be typed properly.

• MS Excel 2016

                         No Issues Found. Sample words and Letters can be typed properly.

• MS PowerPoint 2016

                         No Issues Found. Sample words and Letters can be typed properly.

3) Testing Results and Summary for Microsoft Office 2019 with Iskoola Pota Unicode Font

• MS Word 2019

       No Issues Found. Sample words and Letters can be typed properly.

• MS Excel 2019

       No Issues Found. Sample words and Letters can be typed properly.

• MS PowerPoint 2019

No Issues Found. Sample words and Letters can be typed properly.

              Microsoft Office Packages Testing Results – Summary

•  For the testing purpose, we have used MS Office packages. We have used Iskoola Pota Unicode as all the government departments/offices use Iskoola Pota.
• In Office 2013, Sinhala words with Yansaya, Repaya, and Rakaransaya cannot be    typed properly.
• In Office 2016 – Office 2016 (earlier versions)– For the earlier versions of Office 2016, have some problems like switching letters in certain words when letter “ර්” is presented. For example, this problem has occurred in words like “වර්ග”, “දෙපාර්තමේන්තුව”, “නිවර්තන”
• Solution – The above problem does not occur in the latest versions of MS Office 2016. Therefore, by upgrading to the latest 2016 MS Office package, this can be solved.
•  In Office 2019 – There are no issues found.

Testing Part 2 – Sinhala Unicode Typing on Facebook, YouTube and Gmail with Web Browsers

1. Facebook

Web Browsers

• Tested with Google Chrome – Version 104.0.5112.80 (Official Build) (64-bit)
• Tested with Firefox – Version 103.0.1 (64-bit)
• Tested with Microsoft Edge – Version 104.0.1293.47 (Official build) (64-bit)

Operating System

• Windows 10 & Mac Operating System (OS Version: High Sierra) – Desktop

• In Google Chrome, Firefox and Microsoft Edge Sinhala words with Rakaransaya, Yansaya, Repaya and Sinhala Conjunct letters can be typed properly.
  • ක්‍රමය, ශ්‍රමය
  • අධ්‍යාපනය
  • වර්‍ගඵලය
  • ක්‍ෂය

• After posting on Facebook, those words display as follows on desktop. – Figure 2

• Copy and paste the correct words does not solve the issue. After posting, those words display incorrectly on desktop.
• On Mobile Devices – Using Facebook App or using Web browser those words are displayed properly.

• Gmail

Web Browsers

• Tested with Google Chrome – Version 104.0.5112.80 (Official Build) (64-bit)
• Tested with Firefox – Version 103.0.1 (64-bit)
• Tested with Microsoft Edge – Version 104.0.1293.47 (Official build) (64-bit)

Operating System 

• Windows 10 (Desktop)

• In Google Chrome, Firefox and Microsoft Edge Sinhala words with “rakaransaya”, “yansaya”, “repaya” and Sinhala Conjunct letters can be typed and displayed properly on desktop.
  • ක්‍රමය, ශ්‍රමය
  • අධ්‍යාපනය
  • වර්‍ගඵලය
  • ක්‍ෂය

• YouTube

 Web Browsers

• Tested with Google Chrome – Version 104.0.5112.80 (Official Build) (64-bit)
• Tested with Firefox – Version 103.0.1 (64-bit)
• Tested with Microsoft Edge – Version 104.0.1293.47 (Official build) (64-bit)

Operating System

• Windows 10 (Desktop)

• In Google Chrome, Firefox and Microsoft Edge Sinhala words with Rakaransaya, Yansaya, Repaya and Sinhala Conjunct letters can be typed and displayed properly on desktop.
  • ක්‍රමය, ශ්‍රමය
  • අධ්‍යාපනය
  • වර්‍ගඵලය
  • ක්‍ෂය

  Testing Part 3 – Microsoft Windows 10 Terminal/Command Prompt

Theekshana tested comprehensively Terminal Window or Command- Prompt Window of windows 10. Terminal Window of windows 10 failed to render Sinhala Unicode. At the Command Prompt or Shell, it displayed squares.

After setting up UTF-8 by typing, at the command prompt, “chcp 65001”, Unicode Sinhala is not rendered. We diagnosed that a Sinhala font is not properly installed. After following the instructions, we made an entry to the Windows Registry of a font. It still gives squares.

Conclusion

Help Centre staff has found several issues relating to Unicode rendering, although most have been ironed out. The staff of the Help Centre will carry out further tests using the same test cases on Linux-based Operating Systems. Help Centre will inform relevant companies, especially Facebook and Microsoft, on the issues found. It will also carry out these testing on a regular intervals. And it hopes to publish findings on Helpcentre. lk.