Commit Graph

2423 Commits

Author SHA1 Message Date
Behdad Esfahbod
be218c688c Pass this object to trace macros 2012-11-23 15:32:14 -05:00
Behdad Esfahbod
902cc8aca0 [OTLayout] Start unbreaking tracing 2012-11-23 15:23:30 -05:00
Behdad Esfahbod
dabe698fcb Minor 2012-11-23 14:21:35 -05:00
Behdad Esfahbod
c779d82b2f Fix warnings 2012-11-23 14:09:21 -05:00
Behdad Esfahbod
81822528ef Minor 2012-11-23 13:27:16 -05:00
Behdad Esfahbod
1d67ef980f Move code around 2012-11-22 16:47:53 -05:00
Behdad Esfahbod
ec35a72a44 [OTLayout] Port apply() operator to process() template 2012-11-22 16:33:46 -05:00
Behdad Esfahbod
2005fa5340 [OTLayout] Port would_apply() and get_coverage() to process() templates 2012-11-22 16:33:46 -05:00
Behdad Esfahbod
44fc237b53 [OTLayout] Port closure() to process() template 2012-11-22 16:33:46 -05:00
Behdad Esfahbod
5be86b1bb4 [ucdn] Make data tables const! 2012-11-22 16:33:46 -05:00
Behdad Esfahbod
7c5b7fe686 Fix hb_shape_plan_get_shaper() 2012-11-22 16:33:46 -05:00
Behdad Esfahbod
ac064a2db2 Rename hb_set_population() to hb_set_get_population() 2012-11-21 01:14:19 -05:00
Behdad Esfahbod
16c914c2a6 [Indic] One more try at unbreaking Khmer fonts
See comments and discussion on the list.
2012-11-21 01:04:15 -05:00
Behdad Esfahbod
e8cfdd7fa8 Start implementing collect_glyphs() operation
Not functional yet.
2012-11-16 19:07:06 -08:00
Behdad Esfahbod
7d52e6601f Whitespace 2012-11-16 18:49:54 -08:00
Behdad Esfahbod
51bb498b7b Minor 2012-11-16 14:08:05 -08:00
Behdad Esfahbod
89ca8eeb83 Implement hb_ot_layout_get_glyphs_in_class() 2012-11-16 13:53:40 -08:00
Behdad Esfahbod
5a08ecf920 Implement hb_ot_layout_get_glyph_class() 2012-11-16 13:34:29 -08:00
Behdad Esfahbod
f9edd5d56b Implement hb_shape_plan_get_shaper()
Untested.
2012-11-16 13:23:37 -08:00
Behdad Esfahbod
43b6531500 [Indic] Another try to unbreak Sinhala split matras
Just read the comments...
2012-11-16 13:14:26 -08:00
Behdad Esfahbod
977f1740ac Unbreak tests 2012-11-16 13:10:07 -08:00
Behdad Esfahbod
eba312c8d1 Plumbing to get shape plan and font into complex decompose function
So we can handle Sinhala split matras smartly...  Coming soon.
2012-11-16 12:58:38 -08:00
Behdad Esfahbod
3f82f8ff07 Rename hb_buffer_guess_properties() to hb_buffer_guess_segment_properties() 2012-11-15 18:48:10 -08:00
Behdad Esfahbod
f30641038b Bunch of independent changes (ouch)
API additions:

	hb_segment_properties_t
	HB_SEGMENT_PROPERTIES_DEFAULT
	hb_segment_properties_equal()
	hb_segment_properties_hash()

	hb_buffer_set_segment_properties()
	hb_buffer_get_segment_properties()

	hb_ot_layout_glyph_class_t

	hb_shape_plan_t
	hb_shape_plan_create()
	hb_shape_plan_create_cached()
	hb_shape_plan_get_empty()
	hb_shape_plan_reference()
	hb_shape_plan_destroy()
	hb_shape_plan_set_user_data()
	hb_shape_plan_get_user_data()
	hb_shape_plan_execute()

	hb_ot_shape_plan_collect_lookups()

API changes:

	Rename hb_ot_layout_feature_get_lookup_indexes() to
	hb_ot_layout_feature_get_lookups().

New header file:

	hb-shape-plan.h

And a bunch of prototyped but not implemented stuff.  Coming soon.
(Tests fail because of the prototypes right now.)
2012-11-15 18:48:10 -08:00
Behdad Esfahbod
e05a999495 Add hb_face_[sg]et_glyph_count() 2012-11-15 16:23:21 -08:00
Behdad Esfahbod
aec89de564 Add / modify set API a bit 2012-11-15 16:15:42 -08:00
Behdad Esfahbod
c54599ad26 Minor 2012-11-15 16:14:39 -08:00
Behdad Esfahbod
d1aa143ca4 [Thai] Remove U+0E2C from "AC" consonants
WinXP doesn't include it.
2012-11-15 15:38:08 -08:00
Behdad Esfahbod
362a990b22 Rename hb_ot_layout_would_substitute_lookup() and hb_ot_layout_substitute_closure_lookup()
To match upcoming API.
2012-11-15 14:57:31 -08:00
Behdad Esfahbod
3cec819d39 Make the OT shaper default, even if CoreText or Uniscribe is enabled 2012-11-15 13:15:39 -08:00
Behdad Esfahbod
072ae7a982 Add hb_buffer_serialize_list_formats() 2012-11-15 13:14:12 -08:00
Behdad Esfahbod
f9edf16725 Add buffer serialization / deserialization API
Two output formats for now: TEXT, and JSON.  For example:

  hb-shape --output-format=json

Deserialization API is added, but not implemented yet.
2012-11-15 13:10:07 -08:00
Behdad Esfahbod
fd0de881f4 Avoid C++ undefined behavior
https://bugzilla.mozilla.org/show_bug.cgi?id=810823
2012-11-15 10:48:50 -08:00
Behdad Esfahbod
f41dc2d35b Fix undefined behavior in Indic dottedcircle
Chromium Issue 158998:	Conditional jump in harfbuzz-ng
http://code.google.com/p/chromium/issues/detail?id=158998
2012-11-15 10:36:43 -08:00
Behdad Esfahbod
1eb3e94fe9 [Thai] Implement PUA-based fallback shaping
As explained here:

  http://linux.thai.net/~thep/th-otf/shaping.html

Our output now matches Uniscribe for old fonts (eg. XP Tahoma) with no
Thai GSUB table.
2012-11-14 17:53:09 -08:00
Behdad Esfahbod
851784f837 Improve shaper selection 2012-11-14 17:53:09 -08:00
Behdad Esfahbod
43f04a7456 Move Thai shaper into a separate file 2012-11-14 15:51:54 -08:00
Behdad Esfahbod
ba82325b7a Add note re 'Phags-pa letter U+A872, which is Joining_Type=L 2012-11-14 15:36:53 -08:00
Behdad Esfahbod
d469fadce8 [Indic] Exchange abort() for assert() 2012-11-14 15:07:36 -08:00
Behdad Esfahbod
0f80a89de9 Don't route Kharoshthi through the Indic shaper
It's a simple, right-to-left, script.
2012-11-14 15:05:19 -08:00
Behdad Esfahbod
e67072bb17 [Indic] Handle overstruck matra position 2012-11-14 15:00:53 -08:00
Behdad Esfahbod
7e99e4f074 Reposition Lao marks
Lao marks are center-aligned, unlike Thai ones.
2012-11-14 14:09:46 -08:00
Behdad Esfahbod
865745b5b8 Don't do fallback positioning for Indic and Thai shapers 2012-11-14 13:48:26 -08:00
Behdad Esfahbod
981748cb2e [Indic] If Khmer fonts have a 'liga' feature, use generic shaper
Seems to produce more coherent results than trying the Indic shaper on
them.  I'm looking at you, Kh-* fonts...
2012-11-14 13:38:16 -08:00
Behdad Esfahbod
dde5506fd9 [Indic] Don't move virama with left matra
This is important for the Sinhala U+0DDA split matra since it decomposes
to U+0DD9,U+0DCA where U+0DD9 is a left matra and U+0DCA is the virama.
We don't want to move the virama with the left matra.
TEST: U+0D9A,U+0DDA

Note that we were already doing this in the Uniscribe bug compatibility
mode.  We now do it all the time.
2012-11-14 11:37:04 -08:00
Behdad Esfahbod
92f9bfed42 Minor 2012-11-13 16:50:45 -08:00
Behdad Esfahbod
66ac2ff32e API change: Remove "mask" from hb_buffer_add()
I don't expect anybody using hb_buffer_add(), so this shouldn't break
anyone's code.
2012-11-13 16:26:32 -08:00
Behdad Esfahbod
e13f8d280b Fix UTF-8 backward iteration
Ouch!
2012-11-13 15:12:06 -08:00
Behdad Esfahbod
5669a6cf41 [Arabic] Fix post-context handling
Ouch!
2012-11-13 15:11:51 -08:00
Behdad Esfahbod
0c7df22228 Add buffer flags
New API:

	hb_buffer_flags_t

	HB_BUFFER_FLAGS_DEFAULT
	HB_BUFFER_FLAG_BOT
	HB_BUFFER_FLAG_EOT
	HB_BUFFER_FLAG_PRESERVE_DEFAULT_IGNORABLES

	hb_buffer_set_flags()
	hb_buffer_get_flags()

We use the BOT flag to decide whether to insert dottedcircle if the
first char in the buffer is a combining mark.

The PRESERVE_DEFAULT_IGNORABLES flag prevents removal of characters like
ZWNJ/ZWJ/...
2012-11-13 14:42:35 -08:00
Behdad Esfahbod
1c7e55511a Minor fix
Ouch
2012-11-13 14:42:22 -08:00
Behdad Esfahbod
82ecaff736 Add hb_buffer_clear()
Which is like _reset(), but does NOT clear unicode-funcs.
2012-11-13 14:10:00 -08:00
Behdad Esfahbod
0736915b8e [Indic] Decompose Sinhala split matras the way old HarfBuzz / Pango did
Had to do some refactoring to make this happen...

Under uniscribe bug compatibility mode, we still plit them
Uniscrie-style, but Jonathan and I convinced ourselves that there is no
harm doing this the Unicode way.  This change makes that happen, and
unbreaks free Sinhala fonts.
2012-11-13 12:35:35 -08:00
Behdad Esfahbod
6fd5335622 [Indic] Update auto-generated Indic machine to reflect previous commit 2012-11-12 18:42:18 -08:00
Behdad Esfahbod
9cac1338c4 [Indic] Allow Consonant_Medial's after Consonant's
Mostly affects Myanmar, but also Tai Tham, Javanese, and Cham.  The
latter three are untested (no fonts!).
2012-11-12 18:41:22 -08:00
Behdad Esfahbod
d187099cba [Indic] Categorize Myanmar "tone marks" as nuktas 2012-11-12 18:38:06 -08:00
Behdad Esfahbod
8173f23f3f [Indic] Add config for Myanmar 2012-11-12 18:37:20 -08:00
Behdad Esfahbod
9e92978c8a [Indic] Route "new" Myanmar tag through the Indic shaper
Windows 8 adds a Myanmar shaper using the 'mym2' tag.  Route that
through the Indic shaper.  It's still very broken, but at least this
does NOT break old-style Myanmar shaping using the generic shaper.
2012-11-12 18:36:10 -08:00
Behdad Esfahbod
5ab3855f81 Choose shaper based on chosen OT script tag
For Arabic and Indic shapers, if the font doesn't have a script system
for the script, use default shaper.

Make an exception for Arabic script since we have fallback logic for
that one.
2012-11-12 18:27:42 -08:00
Behdad Esfahbod
9b37b4c580 Make planner available to complex shaper choosing logic 2012-11-12 18:23:38 -08:00
Behdad Esfahbod
6fddf2d739 Refactoring ot-map building to make chosen script available earlier 2012-11-12 18:03:07 -08:00
Behdad Esfahbod
de796a6fb9 Add "new" Myanmar OT Script tag
Windows 8 added support for Myanmar shaping using the "mym2" script tag,
even though Windows never supported the old "mymr" tag.
2012-11-12 17:27:51 -08:00
Behdad Esfahbod
e9334ce97b Break build when ragel is needed and missing 2012-11-12 14:57:02 -08:00
Behdad Esfahbod
dba186711e [Indic] Make more room in the table
To be used in upcoming commits.
2012-11-12 14:48:33 -08:00
Behdad Esfahbod
c4be991743 Typo 2012-11-12 14:27:33 -08:00
Behdad Esfahbod
56be677781 [Indic] Port 'pref' logic to look into font tables
...instead of using a hardcoded list of Ra characters.
2012-11-12 14:09:40 -08:00
Behdad Esfahbod
f2c0f59043 [Indic] Port reph handling logic to look into font features
...instead of using a hardcoded list of Ra characters.
2012-11-12 14:02:02 -08:00
Behdad Esfahbod
43149afbc0 Route MEETEI_MAYEK through the Indic shaper
Since it has a couple of left-"matras".
2012-11-12 13:34:17 -08:00
Behdad Esfahbod
d0905c3400 Minor 2012-11-12 13:03:52 -08:00
Behdad Esfahbod
365f27ab5b Work around older compilers
As reported on the list:

I am seeing a similar problem building harfbuzz 0.9.5 with Apple gcc
4.0.1 on OS X 10.5 Leopard:

hb-ot-layout-common-private.hh:406: error: 'struct
OT::CoverageFormat1::Iter' is private
hb-ot-layout-common-private.hh:646: error: within this context
hb-ot-layout-common-private.hh:500: error: 'struct
OT::CoverageFormat2::Iter' is private
hb-ot-layout-common-private.hh:647: error: within this context
make[4]: *** [libharfbuzz_la-hb-ot-layout.lo] Error 1

Also reported as happening with MSVC 2005.
2012-11-12 11:16:57 -08:00
Behdad Esfahbod
6b389ddc36 [Indic] Don't apply 'liga'
Uniscribe doesn't.  And some fonts abuse this feature to get Indic
shaping working in non-complex applications like Adobe's apps.

No change in numbers:

BENGALI: 353897 out of 354188 tests passed. 291 failed (0.0821598%)
DEVANAGARI: 707337 out of 707394 tests passed. 57 failed (0.00805774%)
GUJARATI: 366440 out of 366457 tests passed. 17 failed (0.00463902%)
GURMUKHI: 60704 out of 60747 tests passed. 43 failed (0.0707854%)
KANNADA: 951046 out of 951913 tests passed. 867 failed (0.0910798%)
KHMER: 299074 out of 299124 tests passed. 50 failed (0.0167155%)
LAO: 53611 out of 53644 tests passed. 33 failed (0.0615167%)
MALAYALAM: 1048011 out of 1048334 tests passed. 323 failed (0.0308108%)
ORIYA: 42320 out of 42329 tests passed. 9 failed (0.021262%)
SINHALA: 271666 out of 271847 tests passed. 181 failed (0.0665816%)
TAMIL: 1091754 out of 1091754 tests passed. 0 failed (0%)
TELUGU: 970557 out of 970573 tests passed. 16 failed (0.00164851%)
TIBETAN: 208469 out of 208469 tests passed. 0 failed (0%)
2012-11-12 11:02:56 -08:00
Behdad Esfahbod
d05ac7dc3f Fix hb-ft glyph name for broken fonts that return empty glyph names 2012-11-12 10:26:50 -08:00
Behdad Esfahbod
4899801155 U+A872 PHAGS-PA SUPERFIXED LETTER RA is "Right"-Joining 2012-11-08 15:08:26 -08:00
Behdad Esfahbod
22a685836a Adjust Mongolian shaping
For U+1880..U+1886 Uniscribe thinks they are non-joining.
For U+1887 Uniscribe thinks it's joining, but looks wrong to me.
For now, match Uniscribe.
2012-11-05 15:20:10 -08:00
Behdad Esfahbod
c26a52fbe6 Minor 2012-11-04 16:48:45 -08:00
Behdad Esfahbod
f60d3ed35d Minor 2012-11-04 16:44:47 -08:00
Behdad Esfahbod
10a33296e6 Minor 2012-11-02 13:38:55 -07:00
Behdad Esfahbod
3ba7bc14ea Implement 'Phags-pa shaping
Through the Arabic shaper.  It's similar to Mongolian.
2012-11-01 20:05:04 -07:00
Behdad Esfahbod
da70111ab2 Don't clear buffer pre-context if no new context is being provided
Patch from Jonathan Kew.

Part of fixing:

Mozilla Bug 801410 - avoid inserting dotted-circle for run-initial
Unicode combining characters in "simple" scripts such as Latin

https://bugzilla.mozilla.org/show_bug.cgi?id=801410
2012-10-31 13:45:30 -07:00
Behdad Esfahbod
0bc7a38463 [OT] Fix ReverseChainingSubst
We should make it clear that we don't want output buffer in this case,
otherwise buffer->backtrack_len() would be wrong.
2012-10-29 22:02:45 -07:00
Behdad Esfahbod
2616689d15 More tracing fixups 2012-10-29 21:51:56 -07:00
Behdad Esfahbod
937f8d3871 [Arabic] Enable dlig and mset for Arabic
That's what the spec says, and what Uniscribe does.
2012-10-29 21:49:33 -07:00
Behdad Esfahbod
bc513add79 Add missing TRACE_RETURN 2012-10-29 19:03:55 -07:00
Behdad Esfahbod
88d3c98e30 [Indic] Position pre-base reordering Ra after Chillus in Malayalam
The logic for pre-base reordering follows the left matra logic.
We had an exception for Malayalam/Tamil in the left matra repositioning
which was not reflected in pre-base reordering.

Malayalam failures down from 337 to 323.

BENGALI: 353996 out of 354285 tests passed. 289 failed (0.0815727%)
DEVANAGARI: 707339 out of 707394 tests passed. 55 failed (0.00777502%)
GUJARATI: 366489 out of 366506 tests passed. 17 failed (0.0046384%)
GURMUKHI: 60769 out of 60809 tests passed. 40 failed (0.0657797%)
KANNADA: 951086 out of 951913 tests passed. 827 failed (0.0868777%)
KHMER: 299106 out of 299124 tests passed. 18 failed (0.00601757%)
LAO: 53611 out of 53644 tests passed. 33 failed (0.0615167%)
MALAYALAM: 1048011 out of 1048334 tests passed. 323 failed (0.0308108%)
ORIYA: 42320 out of 42329 tests passed. 9 failed (0.021262%)
SINHALA: 271726 out of 271847 tests passed. 121 failed (0.0445103%)
TAMIL: 1091837 out of 1091837 tests passed. 0 failed (0%)
TELUGU: 970558 out of 970573 tests passed. 15 failed (0.00154548%)
TIBETAN: 208469 out of 208469 tests passed. 0 failed (0%)
2012-10-29 16:46:44 -07:00
Behdad Esfahbod
21bf796954 Add missed file 2012-10-29 14:21:09 -07:00
Behdad Esfahbod
02ed52169a Improve license information 2012-10-28 21:26:19 -07:00
Behdad Esfahbod
4c1d924461 Minor 2012-10-28 20:27:25 -07:00
Behdad Esfahbod
38b015e57f Fix hb_buffer_set_length(buffer, 0)
Was causing invalid realloc()s.
2012-10-28 20:11:47 -07:00
Behdad Esfahbod
b7115b63be Add XXX 2012-10-28 20:11:42 -07:00
Behdad Esfahbod
71ee1f2450 Port to ICU LayoutEngine C API
Incidentally, this makes it not crash with icu-le-hb anymore...
I'm not smart / stupid enough to spend two more days debugging C++
linking issues, and this is ABI-stable at least.
2012-10-28 19:18:11 -07:00
Behdad Esfahbod
0144f05e57 Remove unused members 2012-10-26 13:48:06 -07:00
Behdad Esfahbod
cf3afd8979 Rename and revamp is_zero_width() to be is_default_ignorable()
That's really the logic desired.  Except that MONGOLIAN VOWEL SEPARATOR
is not default_ignorable but it really should be.  Reported to Unicode.

Based on suggestion from Konstantin Ritt.
2012-10-25 16:32:54 -07:00
Behdad Esfahbod
fecdfa95da Fixup hb_ot_shape_closure()
Broke it when merged cmap mapping and normalizer.  Ouch!
2012-10-07 17:19:58 -04:00
Behdad Esfahbod
2d1dcb3ce3 Mark debug message functions static 2012-10-07 17:13:46 -04:00
Behdad Esfahbod
9947bd6daf Update UCDN to upstream commit 3f159c87824230b59af56e40e2db32caf6afa51a
- Unicode 6.2.0 goodness,
- Unassigned codepoints now have correct properties.  Passes test suite.
2012-10-02 20:44:43 -04:00
Behdad Esfahbod
32dbfcf763 Fix visibility of UCDN symbols 2012-10-02 17:42:13 -04:00
Behdad Esfahbod
3f33f0d1f2 Import UCDN into source tree
https://github.com/grigorig/ucdn
2012-10-02 16:23:29 -04:00
Behdad Esfahbod
0e292eb2a2 Remove Glib thread-safety support
Now that we have pthread detection in configure, we don't need Glib
anymore.  Glib will only be a Unicode data provider.
2012-10-02 15:09:38 -04:00
Behdad Esfahbod
66efe89648 Check for pthreads 2012-10-02 14:55:32 -04:00
Behdad Esfahbod
f2eb3fa9dc [OT] Only insert dottedcircle if at the beginning of paragraph
If the first char in the run is a combining mark, but there is text
before the run, don't insert dottedcircle.

Part of addressing:
https://bugzilla.redhat.com/show_bug.cgi?id=858736
2012-09-25 21:35:35 -04:00
Behdad Esfahbod
bdc2fc8294 [Arabic] Respect Arabic joining from neighboring context
Now we respect Arabic joining across runs.
2012-09-25 21:32:35 -04:00
Behdad Esfahbod
05207a79e0 [buffer] Save pre/post textual context
To be used for a variety of purposes.  We save up to five characters
in each direction.  No public API changes, everything is taken care
of already.  All clients need to do is to call hb_buffer_add_utf* with
the full text + segment info (or at least some context) instead of
just passing in the segment.

Various operations (hb_buffer_reset, hb_buffer_set_length,
hb_buffer_add*) automatically reset the relevant contexts.
2012-09-25 21:32:21 -04:00
Behdad Esfahbod
89ac39dbbe Add hb_utf_prev() 2012-09-25 13:59:24 -04:00
Behdad Esfahbod
70ea4ac688 Slightly optimize UTF-8 parsing 2012-09-25 12:30:16 -04:00
Behdad Esfahbod
4445e5e2ec [buffer] Cleanup / optimize UTF-16 parsing a bit 2012-09-25 12:26:12 -04:00
Behdad Esfahbod
1f66c3c1a0 Add hb_utf_strlen()
Speeds up UTF-8 parsing by calling strlen().
2012-09-25 11:42:16 -04:00
Behdad Esfahbod
7f19ae7b9f [buffer] Templatize UTF handling
Also move UTF routines into a separate file, to be reused from shapers
that need it.
2012-09-25 11:23:55 -04:00
Behdad Esfahbod
0e0a4da9b7 [buffer] Towards template'izing different UTF adders 2012-09-25 11:09:04 -04:00
Behdad Esfahbod
7d37280600 Minor 2012-09-25 11:04:41 -04:00
Behdad Esfahbod
54d5da4ee9 Remove unused indic.cc 2012-09-25 10:51:42 -04:00
Behdad Esfahbod
fab7a71f11 [Indic] Import ragel-generated Indic machine in git
I don't expect ragel to be creating too much noise in its generated
output, and including this in-tree helps users right now.  We can
revisit this later if it proved to be too much trouble.
2012-09-24 21:51:13 -04:00
Behdad Esfahbod
20a840c7cd Use a C++ linker on Windows
On Windows we don't care whether or not we link to libstdc++.
Seems to fix build with mingw32 on msys, as reported by Werner.
2012-09-24 20:23:00 -04:00
Behdad Esfahbod
eb7669a380 Better autofoo 2012-09-18 19:42:06 -04:00
Behdad Esfahbod
d00f7d8375 Fix dependencies 2012-09-17 20:59:09 -04:00
Behdad Esfahbod
811eefe225 Return NULL, not false
Oh well...
2012-09-10 09:56:27 -04:00
Behdad Esfahbod
166b5cf7ec [Indic] Find syllables before any features are applied
With FreeSerif, it seems that the 'ccmp' feature does ligature
substituttions.  That was then causing syllable match failures.  We now
find syllables before any features have been applied.

Test sequence: U+0D9A,U+0DCA,U+200D,U+0DBB,U+0DCF
2012-09-07 14:56:01 -04:00
Behdad Esfahbod
96fdc04e5c Add hb_buffer_[sg]et_content_type
And hb_buffer_content_type_t and enum values.
2012-09-06 22:30:53 -04:00
Behdad Esfahbod
e30ebd2794 Add hb_feature_to/from_string() 2012-09-06 22:09:06 -04:00
Behdad Esfahbod
f67917161b [OT] Do per-ligature-component fallback mark positioning
With this in place, you can remove GDEF/GSUB/GPOS tables from Arabic
fonts and still get per-component marks positioned on
oh-yeah-fallback-formed LAM-ALEF ligatures with marks in between the LAM
and ALEF.

Now *that*'s pretty cool, if a bit anachronistic...
2012-09-06 17:22:31 -04:00
Behdad Esfahbod
525c685578 [OT] Make fallback mark positioning more robust
...with clusters spanning multiple base characters.
2012-09-06 16:02:07 -04:00
Behdad Esfahbod
5d502443f5 [old] Clear offset array 2012-09-06 15:29:29 -04:00
Behdad Esfahbod
9433c218b4 [OT] Simplify fallback positioning condition 2012-09-06 14:27:15 -04:00
Behdad Esfahbod
028a1706f8 Refactor common macro 2012-09-06 14:25:48 -04:00
Behdad Esfahbod
07cfbe21b5 [OT] Streamline Arabic fallback shaping table 2012-09-06 01:16:39 -04:00
Behdad Esfahbod
82f6b6f388 Minor 2012-09-06 01:12:50 -04:00
Behdad Esfahbod
fabd3113a9 [OT] Port Arabic fallback shaping to synthetic GSUB
All of init/medi/fina/isol and rlig implemented.

Let there be dragons... ⻯
2012-09-06 00:51:44 -04:00
Behdad Esfahbod
f0b8ed1b6d [Indic] Allow "H,ZWJ,M"
Uniscribe accepts a Halant,ZWJ before matras.  Allow that.

BENGALI down from 295 to 291
DEVANAGARI down from 69 to 57
GUJARATI down from 19 to 17
KANNADA down from 871 to 867
MALAYALAM down from 340 to 337
TELUGU down from 20 to 16

Currently at:

BENGALI: 353897 out of 354188 tests passed. 291 failed (0.0821598%)
DEVANAGARI: 707337 out of 707394 tests passed. 57 failed (0.00805774%)
GUJARATI: 366440 out of 366457 tests passed. 17 failed (0.00463902%)
GURMUKHI: 60704 out of 60747 tests passed. 43 failed (0.0707854%)
KANNADA: 951046 out of 951913 tests passed. 867 failed (0.0910798%)
KHMER: 299077 out of 299124 tests passed. 47 failed (0.0157125%)
LAO: 53611 out of 53644 tests passed. 33 failed (0.0615167%)
MALAYALAM: 1047997 out of 1048334 tests passed. 337 failed (0.0321462%)
ORIYA: 42320 out of 42329 tests passed. 9 failed (0.021262%)
SINHALA: 271666 out of 271847 tests passed. 181 failed (0.0665816%)
TAMIL: 1091754 out of 1091754 tests passed. 0 failed (0%)
TELUGU: 970557 out of 970573 tests passed. 16 failed (0.00164851%)
TIBETAN: 208469 out of 208469 tests passed. 0 failed (0%)
2012-09-05 17:41:08 -04:00
Behdad Esfahbod
4ed717ef61 [Indic] Relax grammar
Now that we insert dotted-circle, tests break more easily when our indic
machine breaks.

In particular, a few Devanagari tests were having sequences like
"C,H,ZWJ,N", and because of the ZWJ the Nukta does NOT get reordered to
before the Halant as the grammar used to expect...  Fixup.

Another case is as simple as "C,ZWJ,SM".

Fixes 10 out of 79 failures:

DEVANAGARI: 707325 out of 707394 tests passed. 69 failed (0.00975411%)
2012-09-05 17:21:17 -04:00
Behdad Esfahbod
aa7141efe4 [Indic] Fix Khmer syllable-final coeng-consonant
Brings down Khmer failures from 162 to 47.

KHMER: 299077 out of 299124 tests passed. 47 failed (0.0157125%)

Also rebaselined some of the test files that had only-inherited lines.
Removing those, the stats are:

BENGALI: 353893 out of 354188 tests passed. 295 failed (0.0832891%)
DEVANAGARI: 707315 out of 707394 tests passed. 79 failed (0.0111678%)
GUJARATI: 366438 out of 366457 tests passed. 19 failed (0.00518478%)
GURMUKHI: 60704 out of 60747 tests passed. 43 failed (0.0707854%)
KANNADA: 951042 out of 951913 tests passed. 871 failed (0.0915%)
KHMER: 299077 out of 299124 tests passed. 47 failed (0.0157125%)
LAO: 53611 out of 53644 tests passed. 33 failed (0.0615167%)
MALAYALAM: 1047994 out of 1048334 tests passed. 340 failed (0.0324324%)
ORIYA: 42320 out of 42329 tests passed. 9 failed (0.021262%)
SINHALA: 271666 out of 271847 tests passed. 181 failed (0.0665816%)
TAMIL: 1091754 out of 1091754 tests passed. 0 failed (0%)
TELUGU: 970553 out of 970573 tests passed. 20 failed (0.00206064%)
TIBETAN: 208469 out of 208469 tests passed. 0 failed (0%)

Still some regressions, but some of the more egregious cases are
addressed.
2012-09-05 17:14:52 -04:00
Behdad Esfahbod
27bd55bd2c [Indic] Tamil does not have half-forms either
The Win7 Tamil font does not realy on this behavior, but the WinXP
version does.  Handle Tamil like Malayalam: Matras always move to
before base.

WinXP Tamil failures went down from 168964 (15.4752%) to 167
(0.0152953%) (two orders of magnitude reduction!).

Included in this is a minor fixup that actually fixed a few tests
with non-Tamil too.  Numbers at:

BENGALI: 353997 out of 354285 tests passed. 288 failed (0.0812905%)
DEVANAGARI: 707339 out of 707394 tests passed. 55 failed (0.00777502%)
GUJARATI: 366489 out of 366506 tests passed. 17 failed (0.0046384%)
GURMUKHI: 60769 out of 60809 tests passed. 40 failed (0.0657797%)
KANNADA: 951086 out of 951913 tests passed. 827 failed (0.0868777%)
KHMER: 299106 out of 299124 tests passed. 18 failed (0.00601757%)
LAO: 53611 out of 53644 tests passed. 33 failed (0.0615167%)
MALAYALAM: 1048104 out of 1048416 tests passed. 312 failed (0.0297592%)
ORIYA: 42320 out of 42329 tests passed. 9 failed (0.021262%)
SINHALA: 271747 out of 271847 tests passed. 100 failed (0.0367854%)
TAMIL: 1091837 out of 1091837 tests passed. 0 failed (0%)
TELUGU: 970558 out of 970573 tests passed. 15 failed (0.00154548%)
TIBETAN: 208469 out of 208469 tests passed. 0 failed (0%)
2012-09-05 15:22:02 -04:00
Behdad Esfahbod
87b75d0a4a [OT] Allow adding features with fallback implementation 2012-09-04 23:06:38 -04:00
Behdad Esfahbod
1d3947a6bd Minor 2012-09-04 22:42:17 -04:00
Behdad Esfahbod
b3b89b6658 [OT] Add SubstLookup serialize API 2012-09-04 21:28:33 -04:00
Behdad Esfahbod
715e03bc21 Minor 2012-09-04 20:10:17 -04:00
Behdad Esfahbod
652d1e0d64 [OT] Start adding Lookup-level serialize API 2012-09-04 20:00:44 -04:00
Behdad Esfahbod
a930c68e9c [OT] More serialize. Implements all basic GSUB subtables 2012-09-04 19:16:09 -04:00
Behdad Esfahbod
1b38b4e817 Minor 2012-09-04 18:17:21 -04:00
Behdad Esfahbod
2bd9fe3598 Refactor 2012-09-04 15:15:19 -04:00
Behdad Esfahbod
a5ddd9e31c [OT] Really fix possible NULL dereference this time 2012-09-04 14:55:00 -04:00
Behdad Esfahbod
2941683358 [OT] Implement serialize() for AlternateSubst 2012-09-03 23:31:14 -04:00
Behdad Esfahbod
1f07e3382a [OT] Implement serialize() for MultiSubst 2012-09-03 23:28:34 -04:00
Behdad Esfahbod
4912030dfb Minor 2012-09-03 21:00:48 -04:00
Behdad Esfahbod
f8fa2b5cf6 Fix possible NULL dereference
As reported by Kenichi Ishibashi.
2012-09-03 20:19:46 -04:00
Behdad Esfahbod
4b312fb288 [OT] Remove serialize alignment
Will reintroduce in a different way when we actually need it.
2012-09-01 21:56:06 -04:00
Behdad Esfahbod
c61be03d6d [OT] A bit more serialize 2012-09-01 21:49:44 -04:00
Behdad Esfahbod
abcc5ac1fd [OT] Improve serialize syntax
For some definition of improvement...
2012-09-01 21:30:17 -04:00
Behdad Esfahbod
bc5be24014 [OT] Restart work on serialize() 2012-09-01 21:25:20 -04:00
Behdad Esfahbod
6912e476dd [OT] Insert dotted-circle for run-initial marks
Unfortunately if the font has GPOS and 'mark' feature does
not position mark on dotted-circle, our inserted dotted-circle
will not get the mark repositioned to itself.  Uniscribe cheats
here.

If there is no GPOS however, the fallback positioning kicks in
and sorts this out.

I'm not willing to address the first case.
2012-09-01 20:38:45 -04:00
Behdad Esfahbod
1d581ec384 [OT] Fallback-position ccc=0 Thai / Lao marks
Not perfect, but so is fallback positioning in 2012...
2012-09-01 20:06:26 -04:00
Behdad Esfahbod
3992b5ec4c Move code around 2012-09-01 19:20:41 -04:00
Behdad Esfahbod
b85800f9de [Indic] Implement dotted-circle insertion for broken clusters
No panic, we reeally insert dotted circle when it's absolutely broken.

Fixes most of the dotted-circle cases against Uniscribe. (for Devanagari
fixes 80% of them, for Khmer 70%; the rest look like Uniscribe being
really bogus...)

I had to make a decision.  Apparently Uniscribe adds one dotted circle
to each broken character.  I tried that, but that goes wrong easily with
split matras.  So I made it add only one dotted circle to an entire
broken syllable tail.  As in: "if there was a dotted circle here, this
would have formed a correct cluster."  That works better for split
stuff, and I like it more.
2012-08-31 19:18:20 -04:00
Behdad Esfahbod
327d14ef18 [Indic] Start adding dotted-circle instrastructure 2012-08-31 16:49:34 -04:00
Behdad Esfahbod
1be368e96f Minor 2012-08-31 16:29:17 -04:00
Behdad Esfahbod
784f29d061 Minor 2012-08-31 14:06:26 -04:00
Behdad Esfahbod
5a7f18767a [OT] Better fallback-position Thai / Lao ccc!=0 marks 2012-08-30 22:53:29 -04:00
Behdad Esfahbod
9f2348de58 [OT] Add serialize() for Coverage 2012-08-29 21:08:59 -04:00
Behdad Esfahbod
e901b954c6 [OT] Start adding serialize() API 2012-08-29 20:26:08 -04:00
Behdad Esfahbod
965c280de0 Add HB_BUFFER_ASSERT_VAR
To be used in places we access buffer vars...
2012-08-29 14:02:37 -04:00
Behdad Esfahbod
0ccf9b6473 Move code around 2012-08-29 14:02:37 -04:00
Behdad Esfahbod
2fcbbdb41a Port Arabic fallback ligating to share code with GSUB
This will eventually allow us to skip marks, as well as (fallback)
attach marks to ligature components of fallback-shaped Arabic.
That would be pretty cool.  I kludged GDEF props in, so mark-skipping
works, but the produced ligature id/components will be cleared later
by substitute_start() et al.

Perhaps using a synthetic table for Arabic fallback shaping was a better
idea.  The current approach has way too many layering violations...
2012-08-29 14:01:22 -04:00
Behdad Esfahbod
5e399a8a45 Minor 2012-08-29 10:40:49 -04:00
Behdad Esfahbod
a177d027d1 [GSUB] Move ligation logic over 2012-08-28 23:18:22 -04:00
Behdad Esfahbod
191fa885d9 [GSUB] Merge Ligature and context input matching
Looks better now...
2012-08-28 22:58:55 -04:00
Behdad Esfahbod
93814ca7dc Start converging Ligature and match_input 2012-08-28 22:39:10 -04:00
Behdad Esfahbod
2eef71737e [hb-icu-le] Add visbility 2012-08-28 19:16:38 -04:00
Behdad Esfahbod
d59e28e492 Minor 2012-08-28 19:08:36 -04:00
Behdad Esfahbod
af169d2813 Minor 2012-08-28 19:08:22 -04:00
Behdad Esfahbod
52ff2681d8 Use VisualStudio-style atomic intrinsics on mingw32 2012-08-28 18:03:35 -04:00
Behdad Esfahbod
7c8e844d92 Use namespace for OpenType tables
Avoids USHORT, SHORT, ULONG, LONG clashes with Windows API.
2012-08-28 17:57:49 -04:00
Behdad Esfahbod
dc5df5af6b Revert "Minor"
This reverts commit 3e0a03978b.

I know remember why that line is there :).
2012-08-28 16:31:23 -04:00
Behdad Esfahbod
3e0a03978b Minor 2012-08-27 17:10:02 -04:00
Behdad Esfahbod
667218a5b1 Minor 2012-08-27 17:00:44 -04:00
Behdad Esfahbod
30dd62251f Only fallback-position glyphs if we have the ccc
Previously, ccc=0 Thai / Lao marks were being
mispositioned.  Don't touch them.
2012-08-27 16:54:34 -04:00
Behdad Esfahbod
e1ba62811a Center unknown marks horizontally 2012-08-27 16:28:05 -04:00
Behdad Esfahbod
23b0e9d7dc [Indic] Fix switch
D'oh.  Was working by pure chance :)).
2012-08-26 14:30:38 -04:00
Behdad Esfahbod
56e878ab87 [graphite2] Cleanup scratch buffer allocation 2012-08-24 00:41:51 -04:00
Behdad Esfahbod
2f7586c622 [icu-le] Implement icu layout engine shaper 2012-08-24 00:00:33 -04:00
Behdad Esfahbod
ba7f6c3797 [icu-le] Hook up to hb_face_t 2012-08-24 00:00:33 -04:00
Behdad Esfahbod
e96bb36995 [icu-le] Actually use the FontTableCache 2012-08-24 00:00:33 -04:00
Behdad Esfahbod
7d242364ea [icu-le] Start adding a icu-layout-engine backend
Import PortableFontInstance and add shaper stub.
2012-08-24 00:00:29 -04:00
Behdad Esfahbod
b5584ee4be [Indic] For old-spec, match non-zero context
Fixes consonant-position with old-spec Malayalam.  Uniscribe seem to be
doing this.  Fixes below-base La (eg. Pa,H,La) with AnjaliNewLipi.ttf.
Doesn't regress new-spec or other scripts.
2012-08-23 16:26:07 -04:00
Behdad Esfahbod
d9b204d3d2 [GSUB] Allow non-zero-context matching in would_apply()
To be used in the next patch.
2012-08-23 16:22:28 -04:00
Behdad Esfahbod
1f2bb172fe Revert "[Indic/GSUB] Ignore context when matching would_apply()"
This reverts commit 24dd4e5674.

Oops.  My bad.  The change _regressed_ Malayalam test suite, not
improved it.  I'll redo it, differentiating between old-spec and
new-spec cases.
2012-08-23 16:10:37 -04:00
Behdad Esfahbod
24dd4e5674 [Indic/GSUB] Ignore context when matching would_apply()
The MS Indic specs say "...all classifications are determined ... using
context-free substitutions."  However, testing shows that MS's Malayalam
shapers (both old and new), "match" even if there is no zero-context rule.
We follow.

Fixes below-base La (eg. Pa,H,La) with AnjaliNewLipi.ttf (old spec).
Moreover, test suite Malayalam failures are down to 312 from 875!  No
change in other scripts.

Current numbers:

BENGALI: 353996 out of 354285 tests passed. 289 failed (0.0815727%)
DEVANAGARI: 707339 out of 707394 tests passed. 55 failed (0.00777502%)
GUJARATI: 366489 out of 366506 tests passed. 17 failed (0.0046384%)
GURMUKHI: 60769 out of 60809 tests passed. 40 failed (0.0657797%)
KANNADA: 951086 out of 951913 tests passed. 827 failed (0.0868777%)
KHMER: 299106 out of 299124 tests passed. 18 failed (0.00601757%)
LAO: 53611 out of 53644 tests passed. 33 failed (0.0615167%)
MALAYALAM: 1047541 out of 1048416 tests passed. 875 failed (0.0834592%)
ORIYA: 42320 out of 42329 tests passed. 9 failed (0.021262%)
SINHALA: 271726 out of 271847 tests passed. 121 failed (0.0445103%)
TAMIL: 1091837 out of 1091837 tests passed. 0 failed (0%)
TELUGU: 970558 out of 970573 tests passed. 15 failed (0.00154548%)
TIBETAN: 208469 out of 208469 tests passed. 0 failed (0%)
2012-08-23 15:47:10 -04:00
Behdad Esfahbod
6732d62e78 [Indic] Implement pre-base reordering Ra for old-spec Malayalam
Fixes Pa,H,Ra sequence with AnjaliNewLipi.ttf.
2012-08-23 15:32:12 -04:00
Behdad Esfahbod
80cd92326f [Indic] Only apply basic features per-syllable
Free up syllables and let features work across syllables for the
presentation forms features and GPOS.

Fixed:
- 1 GURMUKHI test (remains 40)
- 12 KHMER tests (remains 18)
- 11 SINHALA tests (remains 121)

Regresses:
- 5 MALAYALAM tests (up to 312)

Current numbers:

BENGALI: 353996 out of 354285 tests passed. 289 failed (0.0815727%)
DEVANAGARI: 707339 out of 707394 tests passed. 55 failed (0.00777502%)
GUJARATI: 366489 out of 366506 tests passed. 17 failed (0.0046384%)
GURMUKHI: 60769 out of 60809 tests passed. 40 failed (0.0657797%)
KANNADA: 951086 out of 951913 tests passed. 827 failed (0.0868777%)
KHMER: 299106 out of 299124 tests passed. 18 failed (0.00601757%)
LAO: 53611 out of 53644 tests passed. 33 failed (0.0615167%)
MALAYALAM: 1048104 out of 1048416 tests passed. 312 failed (0.0297592%)
ORIYA: 42320 out of 42329 tests passed. 9 failed (0.021262%)
SINHALA: 271726 out of 271847 tests passed. 121 failed (0.0445103%)
TAMIL: 1091837 out of 1091837 tests passed. 0 failed (0%)
TELUGU: 970558 out of 970573 tests passed. 15 failed (0.00154548%)
TIBETAN: 208469 out of 208469 tests passed. 0 failed (0%)
2012-08-23 12:06:14 -04:00
Behdad Esfahbod
df5d5c68f3 Whitespace 2012-08-23 09:33:30 -04:00
Behdad Esfahbod
2f1747ed7d Add comment 2012-08-16 11:46:46 -04:00
Behdad Esfahbod
bd08d5d126 [OT] Fix Arabic shaper OOB access
https://bugzilla.mozilla.org/show_bug.cgi?id=782908
2012-08-16 11:35:50 -04:00
Behdad Esfahbod
daf0731865 [ICU] Fix includes
As reported by Steven Loomis, including uversion.h works everywhere.
2012-08-16 07:32:59 -04:00
Behdad Esfahbod
a67ba9c0fe Whitespace 2012-08-15 18:52:17 -04:00
Behdad Esfahbod
45c1383cc7 Minor 2012-08-14 09:33:18 -04:00
Behdad Esfahbod
4ac4c6f2e1 Fix ICU build with older ICUs 2012-08-13 10:52:52 -04:00
Behdad Esfahbod
d5045a5f40 [ICU] Use new normalizer2 compose/decompose API
It's considerably faster than the fallback implementation we had
previously!
2012-08-11 21:27:15 -04:00
Behdad Esfahbod
9f9f04c222 [OT] Unbreak Thai shaping and fallback Arabic shaping
The merger of normalizer and glyph-mapping broke shapers that
modified text stream.  Unbreak them by adding a new preprocess_text
shaping stage that happens before normalizing/cmap and disallow
setup_mask modification of actual text.
2012-08-11 18:34:13 -04:00
Behdad Esfahbod
e9f28a38f5 [OT] Add shape_plan to Arabic shaper 2012-08-11 18:20:54 -04:00
Behdad Esfahbod
daf13afb08 [OT] Implement fallback mark positioning for "double" combining marks 2012-08-10 16:38:44 -04:00
Behdad Esfahbod
d345313104 [OT] Fix fallback mark positioning with left-to-right text
Ouch!
2012-08-10 16:34:04 -04:00
Behdad Esfahbod
f4cb476298 [OT] Slightly adjust normalizer
The change is very subtle.  If we have a single-char cluster that
decomposes to three or more characters, then try recomposition, in
case the farther mark may compose with the base.
2012-08-10 03:51:44 -04:00
Behdad Esfahbod
07d6828063 Minor 2012-08-10 03:28:50 -04:00