Word Spotting

Word Spotting in Historical Handwritten Document Images

Word spotting strategies employed in historical handwritten documents face many challenges due to variation in the writing style and intense degradation. The VCG created a new method that permits effective word spotting in handwritten documents that it relies upon document-oriented local features which take into account information around representative keypoints as well a matching process that incorporates spatial context in a local proximity search without using any training data. Experimental results on four historical handwritten datasets for two different scenarios (segmentation-based and segmentation-free) using standard evaluation measures show the improved performance achieved by the proposed methodology.

The main novelties of the proposed method are:

  • Use of local features that takes in consideration the handwritten documents particularities. Therefore, it is able to detect meaningful points of the characters that reside in the documents independently of its scaling.
  • It provides consistency between different handwritten writing variations.
  • Use of the same operational pipeline in both segmentation-based and segmentation-free scenarios
  • Incorporation of spatial context in the local search of the matching process by integrating a near neighbor search procedure relative to the each keypoint.

The keypoint detection method is able to detect meaningful points of the characters that reside in the documents independently of its scaling. Moreover, the linear quantization and the resulting CCs represent chunks of strokes that correspond to different writing directions between them. A subset of these CCs should be stable between different scaling and handwriting styles as some of those chucks remains the same.

The word matching method is motived from the Nearest Neighbor Search (NNS) by incorporating a spatial context suitable for document images. The advantage of the proposed matching is three-fold: (i) it enables a local search instead of searching in a brute force manner, (ii) it incorporates spatial context and (iii) it is suitable under both segmentation-based or segmentation-free scenarios.

A Framework for Efficient Transcription of Historical Documents Using Keyword Spotting

The VCG proposed a framework that employs KeyWord Spotting to enhance the efficiency in the manual transcription process, thus, reducing drastically the cost of training data creation. The core principle relies upon the ability of robust document-specific descriptors to produce meaningful similarities between a chosen word image for transcription and the corresponding word images in the full dataset under consideration. In the proposed framework, KWS is coupled with a relevance feedback mechanism which further enhances retrieval performance while being independent to the chosen KWS algorithm. The efficiency of the proposed pipeline is showcased via a user-friendly web-based prototype http://vc.ee.duth.gr/ws/.

The major achievement of the proposed framework is the reduction in time expenses required to achieve transcription data which could feed a Handwriting Text Recognition engine for training. Furthermore, the keyword spotting pipeline is coupled with a relevance feedback mechanism which introduces the user in the retrieval loop, thus, improving the final retrieval performance.

  • K. Zagoris, I. Pratikakis, B. Gatos, “Unsupervised Word Spotting in Historical Handwritten Document Images using Document-oriented Local Features,” in IEEE Transactions on Image Processing, vol.PP, no.99, pp.1-1
  • K. Zagoris, I. Pratikakis, and B. Gatos, “Segmentation-based historical handwritten word spotting using document-specific local features,” in Frontiers in Handwriting Recognition (ICFHR), 2014 14th International Conference on, Sept 2014, pp. 9–14.
  • K. Zagoris, I. Pratikakis, and B. Gatos, “A framework for efficient transcription of historical documents using keyword spotting,” in Historical Document Imaging and Processing (HIP'15), 3rd International Workshop on, August 2015, pp. 9–14.
☀️