Ernesto Coto and Andrew Zisserman

Overview

The VGG Image Classification (VIC) Engine is an open source project developed at the Visual Geometry Group and released under the BSD-2 clause. VIC is a web application that serves as a web engine to perform image classification queries over an user-defined image dataset. It is based on the original application created by VGG to perform visual searchers over a large dataset of images from BBC News.

Features

  • Performs queries by entering a text or an image
  • Automatically downloads training images from Google
  • Performs automatic training, classification and ranking of results
  • Automatically caches query results
  • Provides an user management interface
  • Allows further query refinement
  • Enables users to create curated queries using their own training images
  • Is capable of data ingestion, i.e., users can search their own dataset and define their own metadata
  • Can be executed with GPU support
  • May 2018: Enables users to create queries based on the metadata of their images
  • March 2019: VIC now can ingest images from repositories supporting the International Image Interoperability Framework (IIIF)
  • June 2019: Published documentation about the API of the category-search backend
  • May 2020: VIC has now moved to Python3

Components

VIC is actually the union of two components:

  • A web frontend: Developed using Django, the source code is available in the vgg_frontend Git repository. The provided version runs in DEBUG mode. Should you wish to deploy it to a production environment, please refer to Django's documentation. The web frontend also includes an image downloading service, which corresponds to a reduced version of the imsearch-tools tool. The version provided in the vgg_frontend repository includes only the google web image search engine.

  • A category-search backend: VIC uses the image classification backend engine described in the paper VISOR: Towards On-the-Fly Large-Scale Object Category Retrieval. This backed in called vgg_classifier and the source code is available at the vgg_classifier Git repository.
    • API documentation: It is possible for you to create your own application (e.g. your own web frontend) for interacting with the category-search backend. The API is documented at this Wiki and you can use the test scripts as examples of the interaction.

Screenshots


Home page

Getting started tour

Processing a query for "plane"

Results for the "plane" query

Refining the query for "plane"

Details of a result

Getting Started

A quick tour to the system can be accessed here. This tour is also available by clicking the "Getting Started" link on the home page of VIC.

In addition, there are more advanced usage instructions in the VIC Docker deployment pages, such as: "How to search your own image dataset", "How to use your own training images" and "How to use your own metadata for searching".

More details about the compilation, configuration and execution of each of VIC's components can be found in the README files available at the aforementioned source code repositories.

Docker version

A very quick and simple deployment is available for Linux, Mac and Windows via Docker technology. Please read the page corresponding to the platform in which you wish to perform the deployment.

Downloads

For VIC to be able to work properly, it needs to pre-process the set of images that are going to be classified. Below we provide files corresponding to sample image datasets and the pre-processed data extracted from it. You could use these files to start using VIC straight away. Should you wish to use your own image dataset, please read the "How to use your own images" section in either of the VIC Docker deployment pages.

  • vic_data_COCO_v2.tar: This file contains a small subset of COCO2014 (100 images), along with an example metadata file and sample negative images. It also contains pre-processed data files for the category search backend, computed out of the sample COCO2014 images. A few cached queries are delivered as well and a set of files so as to be able to perform a curated query for the "car" category. A .zip version of the file can be found here.

    DISCLAIMER: Please note that VGG does not own the copyright of the COCO2014 images. See the COCO Dataset Terms of Use.

  • vic_data_paintings_v1.tar: This file contains a small subset of the Paintings Dataset (104 images), along with an example metadata file and sample negative images. It also contains pre-processed data files for the category search backend, computed out of the sample Painting images. A few cached queries are delivered as well and a set of files so as to be able to perform a curated query for the "apple" category. A .zip version of the file can be found here.

    DISCLAIMER: Please note that VGG does not own the copyright of the ArtUK images provided. See the ArtUK pages for Image Use.

Contact

Software bug reports and feature requests for vgg_frontend should be submitted here (requires an account at gitlab.com).

Software bug reports and feature requests for the category search backend (vgg_classifier) should be submitted here (requires an account at gitlab.com).

For all other queries, please contact Ernesto Coto

Citation

If you use this software, please cite this page as shown below and acknowledge the Seebibyte grant as follows: "Development and maintenance of the VGG Image Classification (VIC) Engine is supported by EPSRC programme grant Seebibyte: Visual Search for the Era of Big Data (EP/M013774/1)".

@misc{ coto2017vic,
  author = "Coto, E. and Zissermann, A.",
  title = "{VGG} Image Classification ({VIC}) Engine",
  year = "2017",
  howpublished = "http://www.robots.ox.ac.uk/~vgg/software/vic/",
  note = "Version: X.Y.Z, Accessed: INSERT_DATE_HERE"
}

Acknowledgements

This work is supported by EPSRC programme grant Seebibyte: Visual Search for the Era of Big Data (EP/M013774/1).

EPSRC logo Seebibyte Logo