Historical Document Image Analysis

Distinction between handwritten and machine-printed text

Nowadays, one can observe a rapidly growing number of digitization initiatives in libraries and archives, involving a variety of document types. Among several other obstacles, the presence of printed and handwritten text in the same document image gives rise to significant issues since each modality requires different treatment to recognize the corresponding characters. Furthermore, the automatic processing of application forms, bank checks, petitions, mail items, etc. makes imperative the distinction between handwritten and machine-printed text.

It is, therefore, necessary to separate the two types of text so that it becomes feasible to apply different recognition methodologies to each modality.

A new approach is proposed which strives towards identifying and separating handwritten from machine printed text using the Bag of Visual Words model (BoVW). Initially, blocks of interest are detected in the document image. For each block, a descriptor is calculated based on the BoVW. The final characterization of the blocks as Handwritten, Machine Printed or Noise is made by a decision scheme which relies upon the combination of binary SVM classifiers. The promising performance of the proposed approach is shown by using a consistent evaluation methodology which couples meaningful measures along with new datasets dedicated to the problem upon consideration.

  • K. Zagoris, I. Pratikakis, A. Antonacopoulos, B. Gatos and N. Papamarkos, “Distinction between handwritten and machine-printed text based on the bag of visual words model”, Pattern Recognition, 47 (3), (1051 - 1062), 2014
  • Zagoris, K. and Pratikakis, I. and Antonacopoulos, A. and Gatos, B. and Papamarkos, N., “Handwritten and Machine Printed Text Separation in Document Images using the Bag of Visual Words Paradigm”, 13th International Conference on Frontiers in Handwriting Recognition (ICFHR'12), 2012

Word Spotting in Historical Handwritten Document Images

Word spotting strategies employed in historical handwritten documents face many challenges due to variation in the writing style and intense degradation. The VCG created a new method that permits effective word spotting in handwritten documents that it relies upon document-oriented local features which take into account information around representative keypoints as well as a matching process that incorporates spatial context in a local proximity search without using any training data. Experimental results on four historical handwritten datasets for two different scenarios (segmentation-based and segmentation-free) using standard evaluation measures show the improved performance achieved by the proposed methodology.

The main novelties of the proposed method are:

  • Use of local features that takes into consideration the handwritten documents particularities. Therefore, it is able to detect meaningful points of the characters that reside in the documents independently of its scaling.
  • It provides consistency between different handwritten writing variations.
  • Use of the same operational pipeline in both segmentation-based and segmentation-free scenarios
  • Incorporation of spatial context in the local search of the matching process by integrating a near neighbor search procedure relative to each keypoint.

The keypoint detection method is able to detect meaningful points of the characters that reside in the documents independently of its scaling. Moreover, the linear quantization and the resulting CCs represent chunks of strokes that correspond to different writing directions between them. A subset of these CCs should be stable between different scaling and handwriting styles as some of those chunks remain the same.

The word matching method is motivated by the Nearest Neighbor Search (NNS) by incorporating a spatial context suitable for document images. The advantage of the proposed matching is three-fold: (i) it enables a local search instead of searching in a brute force manner, (ii) it incorporates spatial context and (iii) it is suitable under both segmentation-based or segmentation-free scenarios.

A Framework for Efficient Transcription of Historical Documents Using Keyword Spotting

The VCG proposed a framework that employs KeyWord Spotting to enhance the efficiency in the manual transcription process, thus, reducing drastically the cost of training data creation. The core principle relies upon the ability of robust document-specific descriptors to produce meaningful similarities between a chosen word image for transcription and the corresponding word images in the full dataset under consideration. In the proposed framework, KWS is coupled with a relevance feedback mechanism which further enhances retrieval performance while being independent to the chosen KWS algorithm. The efficiency of the proposed pipeline is showcased via a user-friendly web-based prototype http://vc.ee.duth.gr/ws/.

The major achievement of the proposed framework is the reduction in time expenses required to achieve transcription data which could feed a Handwriting Text Recognition engine for training. Furthermore, the keyword spotting pipeline is coupled with a relevance feedback mechanism which introduces the user in the retrieval loop, thus, improving the final retrieval performance.

  • K. Zagoris, I. Pratikakis, B. Gatos, “Unsupervised Word Spotting in Historical Handwritten Document Images using Document-oriented Local Features,” in IEEE Transactions on Image Processing, vol.PP, no.99, pp.1-1
  • K. Zagoris, I. Pratikakis, and B. Gatos, “Segmentation-based historical handwritten word spotting using document-specific local features,” in Frontiers in Handwriting Recognition (ICFHR), 2014 14th International Conference on, Sept 2014, pp. 9–14.
  • K. Zagoris, I. Pratikakis, and B. Gatos, “A framework for efficient transcription of historical documents using keyword spotting,” in Historical Document Imaging and Processing (HIP'15), 3rd International Workshop on, August 2015, pp. 9–14.