Commit Graph

3082 Commits

Author SHA1 Message Date
Behdad Esfahbod
bcc8f32b4c [buffer] Minor 2013-02-27 18:49:10 -05:00
Behdad Esfahbod
d3e14aafff [buffer] Move buffer serialization code to a new file 2013-02-27 18:49:05 -05:00
Behdad Esfahbod
8da5111269 Minor 2013-02-27 18:49:05 -05:00
Behdad Esfahbod
94789fd601 [Indic] Sort pre-base reordering consonants with post-forms
Before, we were marking them as below-form for initial reordering.
However, there is a rule that says "post consonants should follow
below consonsnts" for base determination purposes.  Malayalam has
port-form YA/VA, and RA is pre-base.  As such, for a sequence like
YA,Virama,YA,Virama,RA, the correct base is at index 0.  But
because the code was seeing RA as a below-base, it was stopping at
the second YA as base, instead of jumping it as a post-base.

By treating prebase-reordering consonants like post-forms, this
is fixed.

MALAYALAM went down from 351 to 265.  Other numbers didn't change:

BENGALI: 353686 out of 354188 tests passed. 502 failed (0.141733%)
DEVANAGARI: 707305 out of 707394 tests passed. 89 failed (0.0125814%)
GUJARATI: 366262 out of 366457 tests passed. 195 failed (0.0532122%)
GURMUKHI: 60706 out of 60747 tests passed. 41 failed (0.067493%)
KANNADA: 950680 out of 951913 tests passed. 1233 failed (0.129529%)
KHMER: 299074 out of 299124 tests passed. 50 failed (0.0167155%)
LAO: 53611 out of 53644 tests passed. 33 failed (0.0615167%)
MALAYALAM: 1048069 out of 1048334 tests passed. 265 failed (0.0252782%)
ORIYA: 42320 out of 42329 tests passed. 9 failed (0.021262%)
SINHALA: 271539 out of 271847 tests passed. 308 failed (0.113299%)
TAMIL: 1091753 out of 1091754 tests passed. 1 failed (9.15957e-05%)
TELUGU: 970555 out of 970573 tests passed. 18 failed (0.00185457%)
TIBETAN: 208469 out of 208469 tests passed. 0 failed (0%)
2013-02-26 21:22:37 -05:00
Behdad Esfahbod
6d69a2cec1 [tests] Add Malayalam tests frim cibu 2013-02-26 21:11:05 -05:00
Behdad Esfahbod
1edc6b97b6 Update Android.mk 2013-02-26 15:01:45 -05:00
Behdad Esfahbod
05d5d3cd63 Fix "comparison between signed and unsigned" warnings
Patch from Jonathan Kew.
2013-02-25 23:57:51 -05:00
Behdad Esfahbod
07a52b6211 Give up sanitizing after 100 edit requests
Such fonts are *definitely* really broken.  Give up.
Limits time spent in sanitize for extremely / deliberately broken
fonts.  For example, two fonts with these md5sum / names:

  9343f0a1b8c84b8123e7d201cae62ffd.ttf
  eb8c978547f09d368fc204194fb34688.ttf

were spending over a second in sanitize!  Not anymore.
2013-02-25 19:09:57 -05:00
Behdad Esfahbod
29ff254815 Add HB_DEBUG_VERBOSE
Only affects the verbosity of function name printing right now.
2013-02-25 19:04:56 -05:00
Behdad Esfahbod
05686b5f69 0.9.13 2013-02-25 18:19:20 -05:00
Behdad Esfahbod
9e5ac7b8c0 Fix blob test to match c3ba49b6fa 2013-02-25 18:19:02 -05:00
Behdad Esfahbod
c3ba49b6fa Always create sub-blobs in MEMORY_MODE_READONLY
This fixes a design bug with sanitize and sub-blobs that can
cause crashes.  Jonathan and I found and debugged this issue
when we tested a corrupt font with the md5sum / filename:
ea395483d37af0cb933f40689ff7b60a.  Two hours of intense
debugging we found out that the font has overlapping GSUB/GPOS
tables, and as such, sanitizing the second table can modify
the first one, which can cause all kinds of undefined behavior.

The correct way to fix this is to make sure sub-blobs are
always created readonly, since we consider the parent blob
to be a shared resource and can't modify it, even if it *is*
writable.

This essentially makes the READONLY_MAY_MAKE_WRITABLE mode
unused...  Maybe we should simply remove / deprecate it.
2013-02-25 17:06:35 -05:00
Behdad Esfahbod
57542d7f41 Minor 2013-02-21 15:55:49 -05:00
Behdad Esfahbod
d46606e119 Port TrueType-kerning to use skippy iterator
It skips joiners and default ignorables now.  Skips marks too, but only
if there is proper GDEF classes for marks.
2013-02-21 15:55:49 -05:00
Behdad Esfahbod
722e8b857e Fixup previous commit
Was not decreasing num_items.  Ouch!
2013-02-21 15:37:51 -05:00
Behdad Esfahbod
ba87b8fc89 Move code around 2013-02-21 15:23:39 -05:00
Behdad Esfahbod
2b2a6e8944 [OTLayout] Correctly skip Default_Ignorable when match_func not set
When a match_func was not set on the matcher_t object (ie. from GPOS),
then the Default_Ignorables (including joiners) were never skipped.
This meant that they were not skipped as they should during GPOS
matching.  Fix that.

A few Indic numbers have "regressed": BENGALI and DEVANAGARI went
up from 290 and 58 respectively, but in both cases new results are
superior to Uniscribe, as they apply GPOS when we weren't (and
Uniscribe isn't) before.
BENGALI: 353686 out of 354188 tests passed. 502 failed (0.141733%)
DEVANAGARI: 707305 out of 707394 tests passed. 89 failed (0.0125814%)
GUJARATI: 366262 out of 366457 tests passed. 195 failed (0.0532122%)
GURMUKHI: 60706 out of 60747 tests passed. 41 failed (0.067493%)
KANNADA: 950680 out of 951913 tests passed. 1233 failed (0.129529%)
KHMER: 299074 out of 299124 tests passed. 50 failed (0.0167155%)
LAO: 53611 out of 53644 tests passed. 33 failed (0.0615167%)
MALAYALAM: 1047983 out of 1048334 tests passed. 351 failed (0.0334817%)
ORIYA: 42320 out of 42329 tests passed. 9 failed (0.021262%)
SINHALA: 271539 out of 271847 tests passed. 308 failed (0.113299%)
TAMIL: 1091753 out of 1091754 tests passed. 1 failed (9.15957e-05%)
TELUGU: 970555 out of 970573 tests passed. 18 failed (0.00185457%)
TIBETAN: 208469 out of 208469 tests passed. 0 failed (0%)
2013-02-21 15:07:03 -05:00
Behdad Esfahbod
ff93ac8cb2 Minor 2013-02-21 14:51:40 -05:00
Behdad Esfahbod
e0486fc1af [tests] Add Myanmar torture tests from Martin Hosken 2013-02-19 00:58:10 -05:00
Behdad Esfahbod
a3df9a7bf8 Minor
Moving files around
2013-02-19 00:50:46 -05:00
Behdad Esfahbod
b1f4407591 [SEA] Fix order of pre-base reordering Ra and left matras
The code was confused because it was expecting left matra to have
POS_PRE_M, like we do in the Myanmar shaper, but that is not what
we were doing in this shaper.  Rewrite to rely on category only.

Test case: U+AA06,U+AA34,U+AA2F
2013-02-17 12:12:37 -05:00
Behdad Esfahbod
99fa9ea020 Minor 2013-02-15 11:47:24 -05:00
Behdad Esfahbod
27589620ba [OTLayout] Remove unused code 2013-02-15 11:47:09 -05:00
Behdad Esfahbod
ebb77b9c5e Remove TODO items that don't make sense
The spec says those features need to be disabled by default.
2013-02-15 09:33:10 -05:00
Behdad Esfahbod
c2a1cdc4c4 [Arabic] Fix shaping of left-joining 'Phags-Pa U+A872
This is the first character in Unicode to have Arabic left-joining
behavior.  Update the machine to recognize that.

Test case: U+A840,U+A872,U+A840.
2013-02-15 09:27:02 -05:00
Behdad Esfahbod
05ac87813d [tests] Add Syriac Alaph shaping test cases 2013-02-15 09:26:41 -05:00
Behdad Esfahbod
c462b32dcb Disable automatic segment properties guessing
Before, if one called hb_shape() without setting script, language, and
direction on the buffer, hb_shape() was calling
hb_buffer_guess_segment_properties() on the user's behalf to guess
these.

This is very dangerous, since any serious user of HarfBuzz must set
these properly (specially important is direction).  So now, we don't
guess properties by default.  People not setting direction will get
an abort() now.  If the old behavior is desired (fragile, good for
simple testing only), users can call
hb_buffer_guess_segment_properties() on the buffer just before calling
hb_shape().
2013-02-15 07:51:47 -05:00
Behdad Esfahbod
7abddbb47a Add assertions for a couple programmer errors 2013-02-15 07:46:57 -05:00
Behdad Esfahbod
038c98f686 Allow disabling of TrueType kerning
Responds to the same feature tag that GPOS kerning does:
'kern' for horizontal and 'vkrn' for vertical.
2013-02-15 07:41:07 -05:00
Behdad Esfahbod
398238a252 Fix partial disabling of default-on features
Surprisingly, if user ever tried to turn a default feature off partially
(say, disable liga for a range), the feature was being turned off
globally!  Fixed now.
2013-02-15 07:40:10 -05:00
Behdad Esfahbod
cb90b1bbe6 [OTLayout] Respect syllable boundaries for backtrack/lookahead matching
Originally we meant to match backtrack/lookahead across syllable
boundaries.  But a bug in the code meant that this was NOT done for
backtrack.  We "fixed" that in 2c7d0b6b80,
but that broke Myanmar shaping.

We now believe that for Indic-like shapers (which is where syllables are
used), all basic shaping forms should be fully contained within their
syllables, so now we limit backtrack/lookahead matching to the syllable
too.  Unbreaks Myanmar.
2013-02-15 07:02:08 -05:00
Behdad Esfahbod
ee9c3a17d0 Minor refactoring 2013-02-15 06:22:52 -05:00
Behdad Esfahbod
cfc507c543 [Indic-like] Disable automatic joiner handling for basic shaping features
Not for Arabic, but for Indic-like scripts.  ZWJ/ZWNJ have special
meanings in those scripts, so let font lookups take full control.

This undoes the regression caused by automatic-joiners handling
introduced two commits ago.

We only disable automatic joiner handling for the "basic shaping
features" of Indic, Myanmar, and SEAsian shapers.  The "presentation
forms" and other features are still applied with automatic-joiner
handling.

This change also changes the test suite failure statistics, such that
a few scripts show more "failures".  The most affected is Kannada.
However, upon inspection, we believe that in most, if not all, of the
new failures, we are producing results superior to Uniscribe.  Hard to
count those!

Here's an example of what is fixed by the recent joiner-handling
changes:

  https://bugs.freedesktop.org/show_bug.cgi?id=58714

New numbers, for future reference:

BENGALI: 353892 out of 354188 tests passed. 296 failed (0.0835714%)
DEVANAGARI: 707336 out of 707394 tests passed. 58 failed (0.00819911%)
GUJARATI: 366262 out of 366457 tests passed. 195 failed (0.0532122%)
GURMUKHI: 60706 out of 60747 tests passed. 41 failed (0.067493%)
KANNADA: 950680 out of 951913 tests passed. 1233 failed (0.129529%)
KHMER: 299074 out of 299124 tests passed. 50 failed (0.0167155%)
LAO: 53611 out of 53644 tests passed. 33 failed (0.0615167%)
MALAYALAM: 1047983 out of 1048334 tests passed. 351 failed (0.0334817%)
ORIYA: 42320 out of 42329 tests passed. 9 failed (0.021262%)
SINHALA: 271539 out of 271847 tests passed. 308 failed (0.113299%)
TAMIL: 1091753 out of 1091754 tests passed. 1 failed (9.15957e-05%)
TELUGU: 970555 out of 970573 tests passed. 18 failed (0.00185457%)
TIBETAN: 208469 out of 208469 tests passed. 0 failed (0%)
2013-02-14 13:10:54 -05:00
Behdad Esfahbod
0b45479198 [OTLayout] Add fine-grained control over ZWJ matching
Not used yet.  Next commit...
2013-02-14 13:02:13 -05:00
Behdad Esfahbod
607feb7cff [OTLayout] Ignore default-ignorables when matching GSUB/GPOS
When matching lookups, be smart about default-ignorable characters.
In particular:

Do nothing specific about ZWNJ, but for the other default-ignorables:

If the lookup in question uses the ignorable character in a sequence,
then match it as we used to do.  However, if the sequence match will
fail because the default-ignorable blocked it, try skipping the
ignorable character and continue.

The most immediate thing it means is that if Lam-Alef forms a ligature,
then Lam-ZWJ-Alef will do to.  Finally!

One exception: when matching for GPOS, or for backtrack/lookahead of
GSUB, we ignore ZWNJ too.  That's the right thing to do.

It certainly is possible to build fonts that this feature will result
in undesirable glyphs, but it's hard to think of a real-world case
that that would happen.

This *does* break Indic shaping right now, since Indic Unicode has
specific rules for what ZWJ/ZWNJ mean, and skipping ZWJ is breaking
those rules.  That will be fixed in upcoming commits.
2013-02-14 12:57:50 -05:00
Behdad Esfahbod
ec5448667b Add hb_ot_map_feature_flags_t
Code cleanup.  No (intended) functional change.
2013-02-14 12:53:57 -05:00
Behdad Esfahbod
e7ffcfafb1 Clean-up add_bool_feature 2013-02-14 11:58:13 -05:00
Behdad Esfahbod
e7562f53fe Fix compile warnings for ragel-generated machines 2013-02-14 11:58:13 -05:00
Behdad Esfahbod
4e51df73a3 [OTLayout] Remove unused function 2013-02-14 07:42:42 -05:00
Behdad Esfahbod
8820bb235b [OTLayout] Port apply_lookup to skippy_iter 2013-02-14 07:41:23 -05:00
Behdad Esfahbod
dfca269f06 [OTLayout] Port ligate_input to skippy_iter 2013-02-14 07:41:23 -05:00
Behdad Esfahbod
7e53415c2d [OTLayout] Minor fix for apply_lookup()
Should NOT change behavior, since first glyph is a match.
2013-02-14 06:24:30 -05:00
Behdad Esfahbod
6880f7e19d [OTLayout] Make table type known to apply context 2013-02-13 12:17:25 -05:00
Behdad Esfahbod
2c7d0b6b80 [OTLayou] Unbreak backtrack matching
Was introduced by 28b9d502bb.
2013-02-13 12:10:08 -05:00
Behdad Esfahbod
c074ebc466 [OTLayout] Minor refactoring 2013-02-13 11:22:42 -05:00
Behdad Esfahbod
407fc12466 [OTLayout] Remove bogus caching of glyph property 2013-02-13 11:13:06 -05:00
Behdad Esfahbod
6b1e3502e2 Remember ZWNJ
To be used in upcoming changes.
2013-02-13 11:02:54 -05:00
Behdad Esfahbod
1f91c39677 Indent 2013-02-13 09:38:40 -05:00
Behdad Esfahbod
a0cb9f33ee [Indic] Improve base finding in final_reordering
Fixes 5 Malayalam failures!

MALAYALAM: 1048016 out of 1048334 tests passed. 318 failed (0.0303338%)
2013-02-13 09:26:55 -05:00
Behdad Esfahbod
126f39cd16 Add more dot-reph tests 2013-02-13 08:29:21 -05:00