User login

Adaptive and interactive recognition systems

We are investigating fully automatic methods for whole-book recognition. We have proposed an information-theoretic framework for identifying significant disagreements between models, the iconic model and the linguistic model, and interpreting these as candidates for corrections of one or the other of the two models so that, when the updated models are reapplied to perform recognition, a lower error rate on the entire passage results.

Our research builds on over a decades of work showing that adaptive classifiers can improve accuracy without human intervention. Tao Hong showed that within a book, strong "visual" (image-based, iconic) consistency-constraints support automatic post-processing that reduces error. These successes appear, to us, to be due largely to isogeny, the tendency of particular documents to contain only a small subset of all the type, faces, languages, image qualities, and other variabilities that oc- cur in large collections. It is well known that if models of the typefaces, languages, etc were known, even if only approximately, optimizing recognition jointly across all the models improves the accuracy. In a long, highly isogenous book, identical (or similar) character images will occur multiple times, and the same word will also occur multiple times, independently. If the models are inaccurate, the resulting errors cause repeated disagreements between the models, which can be measured at character, word, and pas- sage scales. Correct model adaptation, which leads to a better ac- curacy, will presumably also lower passage-scale model disagreement. Therefore passage-scale mutual entropy can drive model correction and reduce error rates.