Projects
Multimodal Fusion in Affective Computing - Master Thesis
Exploring contributions of bio-signals and facial expressions for multi-modal emotion recognition and developing fusion strategies using deep learning.
Thesis(Low-Res) Thesis(High-Res)
Efficient architectures for Image Segmentation
Studying and analyzing latest developments in efficient architectures of Image Segmentation. Project: Create UNet like architectures based on more efficient and sparse architectures like Depth-Wise separable and grouped convolutions. Goal: Experiment with Atrous Convolutions, Depth-Wise Separable Convolutions, Channel Shuffle, Bottleneck Layers etc to reduce FLOPs while maintaining accuracy of the model.
Oracle: Diagnostics for Intent Detection
Oracle was built to make data centric ML development more efficient. It helps data analysts see the problems with the data for intent classification task at Haptik. Oracle Insights such as "Points of intent(class) overlap" - by analysing the ditribution of prediction probabilities, helped data analysts modify the dataset effectively leading to more accurate results.
Sequence Tagging for NER module
IVA makes a decision on what the user wants by detecting Intent and Entities from the incoming user-query. I worked on the NER module that Haptik's IVAs use. Multiple experiments were done using BERT encoder with a CRF head on SNIPS custom-intents dataset. Considering the requirement that the system should be able to detect different types of entites from small, ill-formed sentences we settled with CRF classifer on top of GloVe embeddings. The main problem we faced was that since most entities in our IVAs used be OOVs, in cases when in-vocabulary word(s) appeared in an entity it wasn't recognized. After a lot of analysis we found the problem was in GloVe vectors where due to very strong representation, the word was almost never classified as an entity. We solved the problem by implementing a dropout like scheme for this embedding vector.
Domain Classification with Freebase data
Domain Classification/Tagging based on entity embeddings. Parsed 400GB freebase data to extract entities and their definitions/descriptions. Finetuned GloVe vectors to include these entities. Trained a text classification system using Flair and pooled embeddings to pool BERT outputs with custom GloVe vectors
Supervised Short-Text Similarity
LSTM and Bilinear Layer model to interpret semantic similarity between two short excerpts of text( messages in a chat) It consisted of an LSTM encoder followed by a bilinear-layer(tensor layer). The model was trained on chat data. Pretrained glove(twitter) embeddings were first fine-tuned on 200MB of textual chat-like data For eg. youtube comments, twitter conversations, daily-dialog, etc. and then used while training the model. The model was able to differentiate between adversarial sentences like “I love you” and “I hate you”; this is one of the cases where existing(at that time) sentence encoders fail. This architecture was used to deploy models for Smalltalk/chit chat for 10 languages
View Project
Paraphrase generation with Variational Autoencoders
Variational autoencoder was used to build an architecture proposed by Gupta et al, A Deep Generative Framework for Paraphrase Generation. It was trained using quora similar questions dataset.
View Project
Founding flowgiga.com
We are trying to help content creators in India monetize their content, either with private delivery channels or finding them the right distributor. Apart from leading and managing a team of 4 engineers, for frontend and UI development. I worked on the backend myself developing a Content Management Platform using Django framework and a video encoding service using ffmpeg, bento tools. Deployed the entire system on AWS ECS and AWS Batch(for encoding jobs).
View Project
IVA Development Platform
I co-built the IVA development platform at jubi.ai. The system consisted of modules like text ranking, intent and entity detection.