Open Access Open Access  Restricted Access Subscription or Fee Access

An Efficient Method of Video Mining to Recognize the Speech of the Video Objects

P. Peer Fatima, M. Parveen, Dr.M. Mohamed Sathik


Mining is the process of extracting particular information from the large amount of data stored in the database or data warehouse. The focusing area in mining process is very vast including the process of Knowledge Mining, Gold Mining and so on. Mining is an important task since there exist many unwanted data along with the needed data. In the stepping stone of data mining, there may the need for mining in the area of Video. The process of extracting the particular object from the video is termed by a misnomer called ―Video Mining‖. Video Mining is one of the emerging technologies in this research world. Due to these properties, in this paper, we presented the concept of Video Mining as the process to extract the particular object from the playing video and analyses its role. Also we can able to retrieve the extracted object’s name and recognize the speech of the object through the technique of Speech Recognizer. Thus this paper produces the valuable technique that will be efficient to carry out the task of Video Mining.


Data Warehouse, Gold Mining, Knowledge Mining, Misnomer, Speech Recognizer, Stepping Stone, Video Mining.

Full Text:



―Lip Localization and Viseme Classification for Visual Speech Recognition Salah Werda, Walid Mahdi and Abdelmajid Ben Hamadou‖ MIRACL: Multimedia Information systems and Advanced Computing Laboratory Higher Institute of Computer Science and Multimedia, Sfax, Tunisia; [walid.mahdi, benhamadou.abdelmajid]

McGurck et J. Mcdonald. ―Hearing lips and seeing voice‖. Nature, 264 : 746-748, Decb 1976.

P. Daubias, ―Modèles a posteriori de la forme et de l’apparence des lèvres pour la reconnaissance automatique de la parole audiovisuelle‖. Thèse à l’Université de Maine France 05-12-2002.

R. Goecke, ―A Stereo Vision Lip Tracking Algorithm and Subsequent Statistical analyses of the Audio-Video Correlation in Australian English‖. Thesis Research School of Information Sciences and Engineering. The Australian National University Canberra, Australia, January 2004.

Y. Nakata and M. Ando. ―Lipreading Method Using Color Extraction Method and Eigenspace Technique‖, Systems and Computers in Japan, Vol. 35, No. 3, 2004.

Petajan, E. D., Bischoff, B., Bodoff, D., and Brooke, N. M., ―An improved automatic lipreading system to enhance speech recognition,‖ CHI 88, pp. 19-25, 1985.

J. WU, S. Tamura, H. Mitsumoto, H. Kawai, K. Kurosu, and K. Okazaki. ―Neural network vowel recognition jointly using voice features and mouth shape image‖. Pattern Recognition, 24(10): 921-927, 1991.

U. Meier, R. Stiefelhagen, J. Yang et A. Waibe. ―Towards unrestricted lip reading‖. Proc 2nd International conference on multimodal Interfaces (ICMI), Hong-kong, Jan 1999.

G. Potamianos, H. P. Graft et E. Gosatto. An Image transform approach For HM based automatic lipreading‖. Proc, ICIP, Volume III, pages 173-177, Chicago, IL, USA Octb 1998.

I. Matthews, J. Andrew Bangham, and Stephen J. Cox. ―Audiovisual speech recognition using multiscale nonlinear image decomposition‖. Proc . 4th ICSLP, volume1, page 38-41, Philadelphia, PA, USA, Octob 1996.

K. Prasad, D. Stork, and G. Wolff, ―Preprocessing video images for neural learning of lipreading,‖ Technical Report CRCTR- 9326, Ricoh California Research Center, September 1993.

R. Rao, and R. Mersereau, ―On merging hidden Markov models with deformable templates,‖ ICIP 95, Washington D.C., 1995.

P. Delmas, Extraction des contours des lèvres d’un visage parlant par contours actif (Application à la communication multimodale) ‖. Thèse à l’Institut National de polytechnique de Grenoble, 12-04-2000.

E. D. Petajan, N. M. Brooke, B. J. Bischoff, and D. A. Bodoff. ―An improved automatic lipreading system to enhance speech recognition‖. In E. Soloway, D. Frye, and S. B. Sheppard, editors, Proc. Human Factors in Computing Systems, pages 19-25 ACM, 1988.

A. J. Goldschen, O. N. Garcia, and E. Petajan. ―Continuous optical automatic speech recognition by lipreading.‖ In 28th Annual Asilmomar Conference on Signals, Systems, and Computer, 1994.

L. R. Rabiner. ―A tutorial on Hidden Markov models and selected applications in speech recognition‖. Proceedings of the IEEE, 77(2) : 257-286, February 1989.

C. Bregler and Y. Konig. Eigenlips for robust ―speech recognition. In Proc. IEEE Int. Conf. on Acoust.‖, Speech, and Signal Processing, pages 669-672, Adelaide, 1994.

Y. Nankaku, K. Tokuda. ―Normalized Training for HMM-Based Visual Speech Recognition‖. Electronics and Communications in Japan, Part 3, Vol. 89, No. 11, 2006.

N.Eveno, ―Segmentation des lèvres par un modèle déformable analytique‖, Thèse de doctorat de l’INPG, Grenoble, November 2003.

N. Eveno, A. Caplier, and P-Y Coulon, ―Accurate and Quasi-Automatic Lip Tracking‖ , IEEE Transaction on circuits and video technology, Mai 2004.

―A Unified Framework for Object Retrieval and Mining‖ by Arasanathan Anjulan and Nishan Canagarajah, Member, IEEE

U. Fayyad and R. Uthurusamy, ―Data mining and knowledge discovery in databases: Introduction to the special issue,‖ Commun. ACM, vol. 39, pp. 24–26, 1999.

R. J. Brachman, T. Khabaza, W. Kloesgen, G. Piatetsky-Shapiro, and E. Simoudis, ―Mining business databases,‖ Commun. ACM, vol. 39, pp. 42–48, 1999.

O. Etzioni, ―The world-wide web: Quagmire or gold mine?,‖ Commun. ACM, vol. 39, pp. 65–68, 1999.

J. Oh, J. Lee, S. Kote, and B. Bandi, ―Multimedia data mining framework for rawvideo sequences,‖ Mining Multimedia and Complex Data, vol. 2797, Lecture Notes in Artificial Intelligence, pp. 18–35, 2003.

Z. Zhang, ―Mining surveillance video for independent motion detection,‖ in Proc. Int. Conf. Data Mining, 2002, pp. 741–744.

S.-C. Chen, M.-L. Shyu, M. Chen, and C. Zhang, ―A decision treebased multimodal data mining framework for soccer goal detection,‖ in Proc. Int. Conf. Multimedia Expo, 2004, pp. 265–268.

J. Oh, J. Lee, and S. Kote, ―Real time video data mining for surveillance video streams,‖ in Proc. 7th Pacific-Asia Conference Advances in Knowledge Discovery and Data Mining, 2003, pp. 222–233.

S. C. Chen, M. L. Shyu, C. Zhang, and J. Strickrott, ―Multimedia data mining for traffic video sequence,‖ in MDM/KDDWorkshop, 2001, pp. 78–86.

H. J. Zhang, A. Kankanhalli, and S. W. Smoliar, ―Automatic partitioning of full-motion video,‖ Multimedia Syst., vol. 1, pp. 10–28, 1993.

A. Hampapur, R. Jain, and T.Weymouth, ―Digital video segmentation,‖ in Proc. ACM Multimedia, 1994, pp. 357–364.

A. Nagasaka and Y. Tanaka, ―Automatic video indexing and full-video search for object appearences,‖ in Proc. Visual Database Syst., 1992, pp. 113–127.

R. Kasturi and R. Jain, Computer Vision: Principles. New York: IEEE Computer Society Press, 1991.

S. V. Porter, M. Mirmehdi, and B. T. Thomas, ―Video cut detection using frequency domain correlation,‖ in Proc. Int. Conf. Pattern Recognit., 2000, pp. 413–416.

A. Akutsu, Y. Tonomura, H. Hashimoto, and Y. Ohba, ―Video indexing using motion vectors,‖ in Proc. SPIE Vis. Commun. Image Process., 1992, pp. 1522–1530.

B. Shahraray, ―Scene change detection and content-based sampling of video sequences,‖ in Proc. SPIE Digital Video Compression: Algorithms Technologies, 1995, pp. 2–13.

H.-Y. Chen and J.-L. Wu, ―A multi-layer video browsing system,‖ IEEE Trans. Consumer Electron., vol. 44, pp. 842–850, 1995.

B. Gunsel and A. M. Tekalp, ―Content-based video abstraction,‖ in Proc. Int. Conf. Image Process., 1998, pp. 128–132.

E. Ardizzone and M. L. Cascia, ―Video indexing using optical flow field,‖ in Proc. Int. Conf. Image Process., 1996, pp. 831–834.

J. Vermaak, P. Peraz, M. Gangnet, and A. Blake, ―Rapid summarization and browsing of video sequences,‖ in Proc. British Mach. Vis. Conf., 2002, pp. 424–433.

J. Calic, N. Campbell, B. T. Thomas, R. Laborde, S. Porter, and N. Canagarajah, ―ICBR—Multimedia management system for intelligent content based retrieval,‖ in Proc. CIVR, 2004, pp. 601–609.

―Character Identification in Feature-Length Films Using Global Face-Name Matching‖ by Yi-Fan Zhang, Student Member, IEEE, Changsheng Xu, Senior Member, IEEE, Hanqing Lu, Senior Member, IEEE, and Yeh-Min Huang, Member, IEEE

Dr.M. Mohamed sathik, M. Parveen, P. Peer Fathima ,‖An efficient method to find video objects‖, International Journal of Advanced Research in Computer Science(IJARCS),volume-1,No-1,July-Aug 2010, ISSN No. 0976-5697 ,

Dr.M. Mohamed sathik, M. Parveen, P. Peer Fathima ,‖Extraction of object from the video‖, International Journal of Advanced Research in Computer Science(IJARCS),volume-1,No-3,Jsept-Octb,2010, ISSN No. 0976-5697 ,


  • There are currently no refbacks.

Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 License.