Research Statement
Social networks such as Facebook, video and photo hosting sites like YouTube and Flickr, file-hosting service sites like Dropbox, business review sites like Yelp, and e-commerce sites like eBay, have become more and more popular, allowing people to easily upload, share and annotate massive amounts of images and videos. Web-scale visual search thus has recently become a very active inter-disciplinary area, involving computer vision, multimedia, machine learning, information retrieval, and data mining. Most content based image/video processing, such as retrieval or classification, are performed on image "features", which are much more compact relative to the raw data. Yet the volumes of such image features still can be huge due to the amount and the dimensionality.
Hence, my research is focused on addressing some essential aspects of the problem, including scalable clustering, approximate nearest neighbor search, and fast bin matching for near-duplicate image/video detection/mining. The experimental results have shown the proposed methods are comparable or better than existing solutions in terms of trade-off among search time, memory usage, database storage, and search accuracy.
Visual Search: The Future
- Google's similar image search seems to be built upon their powerful keyword-based image search results. That is the initial search results are refined by certain real content based techniques.
- Baidu's content based image search is featured by Deep Learning techniques plus memory-based indexing search.
- Orbeus is highlighted for their cloud-computing framework; they provide face recognition, scene understanding, and fairly impressive object recognition.
- This demo image search engine shows power of some advanced nearest neighbor search techniques, which was developed by Herve Jegou et al..
Academic Resources
- VisionBib: a great collection of resources related to the area of Computer Vision, with a maintenance of its conference deadlines
- Algorithms for NN search: a tutorial for nearest neighbor search algorithms created in 2007
- Wiki NN: an ongoing maintained Wiki-page for nearest neighbor search problems
Software Libraries
- Yael: a C & Python library for fundamental computation and algorithms of ANN search problems
- ANN: a C++ library for ANN search, including kd-trees, box-decomposition trees, and some search strategies implementations. Support up to 20 dimensional data. (from David Mount, UMD)
- STANN: a C++ library for ANN search, mainly for lower dimensional data (from Piyush Kumar, FSU)
- PQ-Codes (Matlab): a library for ANN search, mainly for high dimensional data, i.e. SIFT, GIST... (from Hervé Jégou, INRIA, France)
- VLFeat: an open source library implements popular computer vision algorithms including SIFT, MSER, k-means, hierarchical k-means...
- LIBSVM: a library for SVM
Dataset Links
- BIGANN: a big ANN search benckmark with 1 billion SIFT and 1 million GIST descriptors
- ImageNet: an image database (with over 14 millions images) organized according to the WordNet hierarchy
- TinyImage: a database with 80 millions tiny images and GIST features
- MNIST: a collection of images with hand-written digits