Hi!
I'm Vatsal

Visual Question Answering

A VQA system takes as input an image and a free-form, open-ended, natural language question about the image and produces a natural language answer as the output

• Dataset: Toronto COCO-QA, VQA dataset. Both of these datasets contain images from MS COCO and Abstract Scenes with ~5 questions per image and 3 plausible answers per question
• Model architecture comprises of 4 parts - Image Encoder, Question Encoder, Fusion technique and feed forward classifier
• VGG16 encoder combined with concatenated word embeddings from BERT combined using a stacked attention network based fusion technique was our best performing model with a top 5 accuracy of 70.32%

Automated chest radiograph interpretation at the level of practicing radiologists using CheXpert - a large dataset of chest X-rays

• Trained two separate DenseNets - one trained only on lateral images and one trained only on frontal images
• Combined the results from these two models at a study level by taking the maximum predictions for two views
• Acheived AUC Score of 0.861 on the test set and the model performed better than predictions by a single radiologist.
• Interpreted the model using Class Activation Mapping - highlighting features/sections in the xray most important for the classification.
• Also tried guided backpropagation to identify the locations in the image that activate the final layer neurons.

SearchSearchGo

SearchSearchGo is a academic version of a search engine consisting of a distributed crawler, hadoop map-reduce based indexer and pagerank, a query engine with Berkley DB as the store and UI built on ReactJS.

• Crawled ~1 million pages using a Apache storm based crawler distributed across 10 EC2 machines.
• Created the inverted indexes, tf-idf values for the crawled corpus using a hadoop map-reduce based Indexer
• Generated pageranks for the webpages using a map-reduce pagerank algorithm executed on Amazon EMR
• Query engine combined the results from cosine similarity between the query and the documents and the pagerank values.
• Integrated shopping and weather results from ebay and weather.com apis/li>
• The UI was built using ReactJS

Roof Identification

Used Mask RCNNs fine tuned on SpaceNet dataset to identify roofs in satellite imagery to identify potential slums and informal settlements in Latin American Countries.

• Exploratory data analysis of SpaceNet Rio data and the data from dymaxion labs for latin american countries.
• Converting data from different formats from SpaceNet Rio and Dynamion data to a common format consistent with MS COCO input format
• Fine tuning the non backbone layers of the Mask RCNN on SpaceNet Rio Data using multiGPU training on Google Cloud
• Used several data augmentation techniques for regularization and performance improvements on the slums data

Face Antispoofing

Developed a technique that can distinguish the spoof face accesses from the genuine ones. This will reduce the possibility of a facial biometric system from being deceived or spoofed by non-real faces.

• Collected face data using digital and 3d printed masks based spoofing.
• Used a combination of hand engineered textural features using Local Binary Patterns and Haralick texture features and deep features from a pretrained Resnet50 CNN model on VGG-Face dataset.

Intelligent Online Clothing Store

Developed a system to recognise clothing style and colour in apparel images using deep features and recommend visually similar products.
The system has the functionality to search for products available in the store by clicking photographs of the desired products.

• Automated product tagging using a color X category classifier trained on images scraped from google
• Searching products with images
• Review based product recommendation for users

Healthcare Outcomes

Developed a system to Predict health behaviors and outcomes for US counties as a function of

• Demographics
• Socio-Economic Status (SES)
• Urban/rural environment
• Words in tweets from that county

Game of Hex

Hex is a strategy board game usually played on a 11x11 hexagonal grid. Wrote the code for both a Player vs Player and Player vs CPU hex game. The AI Bot used Monte Carlo Simulation to determine the next best move.

Movie Rating Predictor

Predict success for films measured by their gross earnings and average user rating using support vector machine trained and tuned on IMDb movie data for 80,000 movies (Pruned to include only movies for which the desired information is present)