CONTENT BASED IMAGE RETRIEVAL

A Requirement Specification

Submitted to

Rajiv Gandhi Prodyogiki Vishwavidyalaya, Bhopal

Towards partial fulfillment for

The degree of

Bachelor of engineering in Information Technology

2010-2011

Submitted to Guided by Submitted By Project co-ordinators Mr. / Ms._____ Your name

Department of Computer Science and Engineering

Acropolis Institute of Technology and research, Indore (M.P.)

December-2010

Department of Computer Science

Acropolis Institute of Technology and Research, Indore (M.P.)

CERTIFICATE

The project entitled “YOUR Project name” submitted by YOUR names is a satisfactory account of the bona fide work done under our supervision is recommended towards the partial fulfillment for the award of the bachelor of engineering by Rajiv Gandhi Prodyogiki Vishwavidyalaya, Bhopal.

Head of the Dept. Project Coordinator Principal

Dr. D.K Mishra Ms. Sarika Jain Dr. M.D. Agrawal

Department of Computer Science

Acropolis Institute of Technology and Research, Indore (M.P.)

PROJECT APPROVAL SHEET

The project title entitled “YOUR project name” submitted by YOUR names is approved as partial fulfillment for the award of the bachelor of engineering degree by Rajiv Gandhi Prodyogiki Vishwavidyalaya, Bhopal.

Internal Examiner External Examiner

Date: Date:

ACKNOWLEDGEMENT

There are two ways of spreading the light, to be a candle, or the mirror, which reflects it. In relation to the light of knowledge o, this work carried out by us is just a ‘mirror’. There are some candles on the other side of the mirror. We would like to avail this opportunity to express our sincere thanks to all those who helped us in making this project. Even a most vivid collection of words, yield to express our heart fully thank towards one and all to have successfully assisted us in our expenditure of carrying out this project.

We wish to express our deep sense of gratitude to H.O.D Dr. D.K Mishra, our project coordinator Ms. Sarika Jain, our project guide ________ and the whole faculty members of the department of Computer Science for encouraging and giving moral support, not only regarding this project but also throughout our studies at this institute. And also to all of my fellow classmates, friends and well wishers for their support and cooperation towards me.

Your names

CONTENTS

Chapter No Description Page No.

1. Introduction ……………...……………………. . ….1

1.1 Purpose……………………………………........

1.2 Scope……………………………………….. …..

1.3 Definition, Acronyms and Abbreviations….. …

1.4 Intended Audience………………………………

1.5 References……………………………………….

2. Existing System……………………………………….

2.1 Process Flow……………………………………

2.2 Limitations……………………………………..

3. Proposed System………………………………………

3.1 Process Flow……………………………………

3.2 Data Flow……………………………………….

3.3 Model Diagram…………………………………

4. System Environment…………………………………..

4.1 Hardware Interface……………………………..

4.2 Client Side……………………………………….

4.3 Server Side……………………………………….

4.4 Software Interface……………………………….

4.5 Communication Interface……………………….

5. Data requirements……………………………………..

6. Application Interfaces………………………………….

6.1 Input Output Interfaces………………………….

6.2 report Formats……………………………………

7. Change Management Process………………………….

8. Assumptions, Constraints and Risks…………………..

8.1 Assumptions……………………………………….

8.2 Constraints………………………………………...

8.3 Risks……………………………………………….

9. Other Requirements…………………………………….

9.1 Appendix A: Glossary…………………………….

9.2 Appendix B: Analysis Models…………………….

9.3 Appendix C: Issues List…………………………..

10. Acceptance Criteria…………………………………….

INTRODUCTION 1.1 Purpose

In last few years the potential growth in digitization of images has occurred,with immense amount of information flowing and stored in the database of world wide web.the No. of users exploiting the WWW has increased tremendously while accessing and manipulate remotely-stored images in all kinds of new and exciting ways.However, they are also discovering that the process of locating a desired image in a large and varied collection can be a source of considerable frustration.The problems of image retrieval are becoming widely recognized, and the search for solutions an increasingly active area for research and development.Some indication in form of No. of research and development in field of CBIR.No.of journal articles,research papers, appearing each year on this subject.

Traditional way to search image in database is to create a textual description of all the images in the database and use the methods from text-based information retrieval to search based on the textual descriptions.Unfortunately, this method is not feasible. On the one handannotating images has to be done manually and is a very time-consuming task and on theother hand images may have contents that words cannot convey.

This has given rise in interest of techniques for retrieving images on the basis of automatically-derived features such as colour, texture and shape – a technology now generally referred to as Content-Based Image Retrieval (CBIR)

1.2  SCOPE

The software product is content based image reteival (CBIR)  is about developing an image search engine, not only by using the text annotated to the image by an end user (as traditional image search engines), but also using the visual contents available into the images itselves.Initially, CBIR system should has a database, containing several images to be searched. Then, it should derive the feature vectors of these images, and stores them into a data structure like on of the “Tree Data Structures” (these structures will improve searching efficiency).A CBIR system gets a query from user, whether an image or the specification of the desired image. Then, it searchs the whole database in order to find the most similar images to the input or desired image.

CBIR usually deals with large image collection of low level and high level features,which directly influence indexing and retival complexity,memorey and disk space requirment.due to high memorey and processing power requirment,cbir has not widely been appplied on platforms having limited resouces,such as mobile devices

1.3 Definition, Acronyms and Abbreviationsdefinitions:

A feature vector vˆI of an image  can be thought of as a point in Rn space: ~

vˆI = (v1, v2, ..., vn), where n is the dimension of the vector.

Examples of possible feature vectors are a color histogram , a multiscale fractalcurve , and a set of Fourier coefficients

The row mean vector is the set of averages of the intensity values of the respective rows.

The column mean vector is the set of averages of the intensity values of the respective columns.

kekre’s transform:- Kekre’s transform matrix is the generic version of Kekre’s LUV color space matrix. Kekre’s transform matrix can be of any size NxN, which need not have to be in powers of 2 (as is the case with most of other transforms). All upper diagonal and diagonal values of Kekre’s transform matrix are one, while the lower diagonal part except the values just below diagonal is zero.

Generalized NxN Kekre’s transform matrix can be given as:

         The formula for generating the term Kxy of Kekre’s transform matrix is:

Euclidean distance ~ the ordinary distance between two points that one would measure with a ruler, and is given by the Pythagoras

AcronymsCBIR: content based image retrievalQBIC: query by image contentCBVIR :content-based visual information retrieval

Key words:

CBIR, Image Splitting, Energy Compaction, Kekre Transform,Row feature vector, Column feature vector

1.4 Intended Audience

A wide range of possible applications for CBIR technology has been identified Potentially fruitful areas include:· Crime prevention· The military· Intellectual property· Architectural and engineering design· Fashion and interior design· Journalism and advertising· Medical diagnosis· Geographical information and remote sensing systems· Cultural heritage· Education and training· Home entertainment· Web searching.

1.5 References

H.B.Kekre, Sudeep D. Thepade, “Color Traits Transfer to Grayscale Images”, IEEE –Int. Conference on Emerging Trends in Engineering and Technology, ICETET-2008, 16-18 July 2008, Raisoni College of Engineering, Nagpur.

E. Saber, A.M. Tekalp, ”Integration of color, edge and texture features for automatic region-based image annotation and retrieval,” Electronic Imaging, 7, pp. 684–700, 1998.

H.B.Kekre, Tanuja Sarode, Sudeep D. Thepade, “Color-Texture Feature

S. Santini and R. Jain, “Similarity measures,” IEEE Trans. PatternAnal.Mach. Intell., vol. 21, no. 9, pp. 871–883, Sep. 1999.

H.B.Kekre, Sudeep D. Thepade, “Scaling Invariant Fusion of Image Pieces in Panorama Making and Novel Image Blending Technique”, International Journal on Imaging (IJI), Autumn 2008, Volume 1, No. A08, Available online at www.ceser.res.in/iji.html (ISSN: 0974-0627).

EXISTING SYSTEM       2.1 Process Flow             

THE TRADITIONAL TEXT BASED IMAGE SEARCH ENGINE

Image meta search (or image search engine) is a type of search engine specialised on finding pictures, images, animations etc. Like the text search, image search is an information retrieval system designed to help to find information on the Internet and it allows the user to look for images etc. using keywords or search phrases and to receive a set of thumbnail images, sorted by relevancy.The metadata of the image is indexed and stored in a large database and when a search query is performed the image search engine looks up the index, and queries are matched with the stored information. The results are presented in order of relevancy. The usefulness of an image search engine depends on the relevance of the results it returns, and the ranking algorithms are one of the keys to becoming a big player.

Page parser is used to analyze Web pages so as to find informative images and extract associated text for indexing.Page parser also tries to determine which parts of the hosting Web page are likely relevant to the contained images and extract corresponding text as for indexing.

Index builder is used to build indexing structure for efficient search of images. The methods adopted are quite similar to general web search, except that each image is treated as a text document rather than a Web page.

Image searching is actually processed in the server side. It accepts users queries and compares with indexed images. When generating the final search result, it considers a number of factors, e.g. the similarity between the query and an indexed image, the image quality, etc. Google claims to present high-quality images first so as to improve the perceived accuracy.

Text-based image search engines index images using the words associated with the images. Depending on whether the indexing is done automatically or manually, image search engines adopting this approach may be further classified into two categories: Web image search engine or collection-based search engine. Web image search engines collect images embedded in Web pages from other sites on the Internet, and index them using the text automatically derived from containing Web pages. Most commercial image search engines fall into this category. On the contrary, collection-based search engines index image collections using the keywords annotated by human indexers. Digital libraries and commercial stock photo collection providers are good examples of this kind of search engines.

"Spiders" take a Web page's content and create key search words that enable online users to find pages they're looking for.

However, text-based image retrieval also faces many challenges. One major problem is that the task of describing image content is highly subjective. The perspective of textualdescriptions given by an annotator could be different from the perspective of a user. A picture can mean different things to different people. It can also mean different things to the same person at different time. Furthermore, even with the same view, the words used to describe the content could vary from one person to another . In other words, there could be a variety of inconsistencies between user textual queries and image annotations or descriptions.

2.2 Limitations

LIMITATION OF TRADITIONAL TEXT BASED IMAGE RETRIEVAL APPROACH:

Problem of image annotation Large volumes of databases Valid only for one language – with image retrieval this limitation should not exist Problem of human perception Subjectivity of human perception Too much responsibility on the end-user Problem of deeper (abstract) needs Queries that cannot be described at all, but tap into the visual features of images.

Proposed System

3.1 Process Flow

CBIR the term content-based image retrieval in the literature survey  describe automatic retrieval of images from a database by colour and shape feature. The term has since been widely used to describe the process of retrieving desired images from a large collection on the basis of features (such as colour, texture and shape) that can be automatically extracted from the images themselves. The features used for retrieval can be either primitive or semantic, but the extraction process must be predominantly automatic. Retrieval of images by manually-assigned keywords is definitely not CBIR as the term is generally understood – even if the keywords describe image content.

Feature of CBIRThese schemes used primitive features of color, shape and texture over the entirei mage to retrieve relevant images. Global features view the image as a whole and calculate its features. Some of the predominantly used features are color histograms, color moments, color sets, gabor filters, co-occurrence matrix, shape context, etc.Later, spatial layout based schemes, sampled images in finer detail by dividing them into many small, usually equal sized parts. They then continued to extract the local features from each part.In  this general framework, the image is segmented into different homogeneous regions based on either colour, texture, shape or all three of them. These schemes range from segmenting the image into objects to segmenting them into homogeneous color patches.

Image features/Histogram

CBIR operates on a totally different principle, retrieving stored images from a collection by comparing features automatically extracted from the images themselves. The commonest features used are mathematical measures of color, texture or shape,

1) Color retrieval

methods for retrieving images on the basis of colour similarity ,Each image added to the collection is analyzed to compute a colour histogram which shows the proportion of pixels of each colour within the image. The colour histogram for each image is then stored in the database. At search time, the user can either specify the desired proportion of each colour (75% olive green and 25% red, for example), or submit an example image from which a colour histogram is calculated. Either way, the matching process then retrieves those images whose colour histograms match those of the query most closely

2) Texture retrieval

The ability to retrieve images on the basis of texture similarity may not seem very useful. But the ability to match on texture similarity can often be useful in distinguishing between areas of images with similar colour (such as sky and sea, or leaves and grass).

Essentially, these calculate the relative brightness of selected pairs of pixels from each image. From these it is possible to calculate measures of image texture such as the degree of contrast, coarseness, directionality and regularity or periodicity, directionality and randomness Texture queries can be formulated in a similar manner to colour queries, by selecting examples of desired textures from a palette, or by supplying an example query image. The system then retrieves images with texture measures most similar in value to the query.

3) Shape retrieval

The ability to retrieve by shape is perhaps the most obvious requirement at the primitive level. Unlike texture, shape is a fairly well-defined concept – and there is considerable evidence that natural objects are primarily recognized by their shape.Two main types of shape feature are commonly used – global features such as aspect ratio, circularity and moment invariants and local features such as sets of consecutive boundary segments

Representations of imagethe images will be stored in their original analogue form, in wallets, files or folders, which in turn will be arranged on shelves, in drawers or in cabinets. The level of indexing associated with manual image collections will be closely related to the importance of the collection, the way it is used, and the time and resources allocated to the task. Retrieval of particular images from such collections is inherently labor intensive and often serendipitous.

Generations of feature vector

For feature extraction in CBIR there are mainly two approaches feature extraction in spatial domain and feature extraction in transform domain. The feature extraction inspatial domain includes CBIR techniques based on histograms The transform domain methods are widely used in image compression, as they give high energy compaction in transformed image  So it is obvious to use images in transformed domain for feature extraction in CBIR . But taking transform of image is time consuming, this complexity is reduced to a great extent by the proposed technique. Reducing the size of feature vector using pure image pixel data in spatial domain only and till getting the improvement in performance of image retrieval is the theme of the work presented. Many current CBIR systems use Euclidean distance on the extracted feature set as a similarity measure. The Direct Euclidian Distance between image P and query image Q can be given as equation 1, where Vpi and Vqiare the feature vectors of image P and Query image Q respectively with size ‘n’.

The formation of feature vector is core process of any CBIR system .In this paper First we split the image into R, G and B planes For each of these three planes we are calculating mean vectors for row and columns of images over which we applied Kekre’s transformation and obtained the feature database for first approach and in second approach we calculated the variances of row and column vectors which is followed by the application of Kekere’ transform to obtain set of coefficients which creates the feature vector database for the second approach. In feature database1 we have six

feature vectors (RRK, RCK, GRK, GCK, BRK and BCK ) foreach of the  imagesEach feature vector in FDB1 and FDB2 represents the row and column mean and variance Kekre’s transform coefficients respectively for planes R, G and B of an image. Whenever we give a query image as query by example to our system, it calculates the feature vectors in the similar manner as explained above. Next step the system performs is the comparison of query image with database images using thesimilarity measure Euclidean distance . Once we have calculated the Euclidean distance between query and database images we retrieve the image from Image database where the Euclidean distance is less than preselected threshold. Determination of threshold is trial and error method.

3.2 Data Flow  

3.3 Model Diagram

System Environment                                   4.1 Hardware Interface

The content based image retrieval (CBIR) does not require any  external hardware requirements apart from a mouse and keyboard that would facilitate giving proper inputs in the form of images.it may also require a scanner to scan external images like medical x-rays .

4.2 Client Side

There is no such client side interface we used here, our system will  interact with users with the help of user friendly GUI.The GUI will be self contained and added with Help function for initial users.                                    4.3 Software Interface

 4.4 Communication Interface:

CBIR does not communicate with any external interface .all its requirements are met from inside the softwar so no communication interface required.

Data requirements

Metadata (‘data about data’) form an essential part of any data archive, providing descriptive data about each stored object which form the main basis for their retrieval. Such metadata typically include catalogue information about Image. Since image will be represented by many of its feature vectors.The image will be tagged with its color ,shape ,texture ,etc.CBIR will deal with huge amount of images as it is intended to do, will require a database to store this image and associated Metadata with it.A standard size and compressed image will be stored in the data base which will require a database engine able to deal with images as data type.A logical database will be required to maintain the information about the images more precisely each image will be associated with its feature vectors like row-mean ,column-mean ,RGB components of image, major color intensity for retrieval on the basis of color,etc.

Application Interfaces                                   

6.1 Input Output Interfaces

CBIR system mainly focus on the algorithm of image retrieval .The input output interface are secondary part and it will depend on implementation of algorithm.

                                    6.2 report Formats

As such there will be no report generated in our application as an output. the input and output interface will be a GUI .the output of the system is a group of images displayed in matrix format. the help function will be there to assist users on the working of s/w .when the user will ask for help the txt file will be generated in front of them .

Change Management Process

The cbir  system is based around an algorithm for calculation of feature vector of images by calculating row mean and coloumn mean and than applying kekre’s transform, here fast searching and quick response of query content should be efficient to give the exact results. so as far as searching speed is concern which is a major issue in our application the algorithm should be fast and efficient, if in case the algorithm is not working effciently we will opt for the other algorithm to give more effcient output corresponding to the query input. when implementing process, technology or organizational change managing resistance can be reduced by making the searching of query content faster and instant response of qurey by  changing the algorithm type.

Assumptions, Constraints and Risks                                     8.1 Assumptions

Images given as input for a query will be on given standard size like 80*80 resolution

Minimum resources are available in system in context of memory and CPU frequency

initial system may have less precision and recall

  8.2 Constraints  

Nature of digital images: arrays of numbers Descriptions of images: high-level concepts.

o Sunset, mountain, dogs, … … Semantic gap

o Discrepancy between low-level features and high-level conceptso High feature similarity may not always correspond to semantic similarityo Different users at different time may give different interpretations for the same

image

                                     8.3 Risks:

Searching of image could be a slow process. Maintenance and updation of databases. Overflow of memory

Other Requirements9.1 Appendix A: Glossary

WWW:-The World Wide Web, abbreviated as WWW and commonly known as the Web, is a systemof interlinked hypertext documents accessed via the Internet. With a web browser, one can viewweb pages that may contain text, images, videos, and other multimedia and navigate between them by via hyperlinks.

Data Structures:-In computer science, a data structure is a particular way of storing and organizing data in a computer so that it can be used efficiently

Euclidean distance :- the ordinary distance between two points that one would measure with a ruler, and is given by the pythagorous formula.

metadata:-Metadata is loosely defined as data about data. Metadata is traditionally found in the card catalogues of libraries and is today commonly used to describe three aspects of digital documents and data: 1) definition, 2) structure and 3) administration. By describing the contents andcontext of data files, the quality of the original data/files is greatly increased.

database index is a data structure that improves the speed of data retrieval operations on a database table at the cost of slower writes and increased storage space

feature vector:-In pattern recognition and machine learning, a feature vector is an n-dimensional vector of numerical features that represent some object. Many algorithms in machine learning require a numerical representation of objects, since such representations facilitate processing and statistical analysis. When representing images, the feature values might correspond to the pixels of an image, when representing texts perhaps to term occurrence frequencies.

Thresholding:- Thresholding is the simplest method of image segmentation. From a grayscale image, thresholding can be used to create binary images

                                    

9.2 Appendix B: Analysis Models

9.3 Appendix C: Issues List->>How the image database will be maintained in our systm still need to be resolved.->>initial system will be working on a Database maintained in a PC,after its version wil be improved to work on WWW->>Our system will be eithe  a object oriented model or  highly specific functional model for faster algorithm i

lementation like inmatlab.A cceptance CriteriaSpecified in dicators or measures employed in assessi ng the abil ity of a co mponent, structure, or system to p

Target dates: 12 march 2011 Major functions: content based search which gives exact output as query input. Appearance: Personnel level required to use/operate a deliverable: Performance levels: Capacity:based on the size of database Accuracy: good Availability: on windows operating system Reliability:efficiently reliable , can retreive fastly even on large image database (Mean/maximum time to repair, mean time between failures):fair Security:secure Ease of use:very easy