Home          Experience          Research          Teaching          Publications

Projects

• Machine Learning
• Robotics
• Computer Vision

Machine Learning

---

Sentiment Analysis (using Deep Learning) on Amazon Review Data

Amazon product reviews from users are not always consistent with the numerical ratings provided. The ratings can be lower or higher than the overall sentiment the review text conveys. In this work, we use existing work in sentiment analysis using recurrent neural networks and apply it to text reviews from Amazon. Our goal is to provide the prospective buyer with an adjusted rating that agrees more with the sentiment of the text. We compare the adjusted ratings from our system and human predictions of the sentiment and find 70% agreement, encouraging further exploration.

---

Concept Learning using Interactive Clustering

In this project we use concepts of Active Learning to interact and obtain labels from a human to learn simple concepts using clustering algorithms like kNN and GMMs. The concepts we dealt with included positional attributes of a table clean-up task (like sink, trash), attributes used to identify animals (like tail, fins, claws etc.) and attributes used to categorize people into social groups (friends, family, colleagues and so on).  We would incrementally learn these concepts by asking the human specific questions and then expand our concept set by generating conjunctions and disjunctions to enable higher-level concepts. These combinations were randomly generated but constrained by the labels received as well as task-specific heuristics. This simple approach proved to work reasonably well to enable fast instantiation of complex concepts.

---

POMDP-based Planning for a Table-top Search and Find Task

For robot mobile manipulators in human environments, an important task is object retrieval. We investigate the task of optimally locating and grasping a goal object in a cluttered environment. We model the world as a Partially Observable Markov Decision Process and use two forms of task representation. Our first design is a grid-based representation which takes different views of the scene and the uncertainty associated with robot vision. Our second approach attempts to exploit a more informed vision system. A tree of obstructing objects is gathered from the scene and planning is done over the possible tree configurations. We solve the POMDP using Point-based Value Iteration algorithms and evaluate the performance on few sample search scenarios.

Download the report here.

---

Efficient Apprenticeship Learning with Smart Humans - AAAI Learning by Demonstration Challenge

We develop a generalized apprenticeship learning protocol for reinforcement-learning agents with access to a teacher. The teacher interacts with the agent by providing policy traces (transition and reward observations). We characterize sufficient conditions of the underlying models for efficient apprenticeship learning and link this criteria to two established learnability classes (KWIK and Mistake Bound).

We demonstrate our approach in a conjunctive learning task that would be too slow to learn in the autonomous setting. We show that the agent can guarantee near-optimal performance with only a polynomial number of examples from a human teacher and can efficiently learn in real world environments with sensor imprecision and stochasticity.

Compiled video here.
It describes, 1. Autonomous Model Learning, 2. Human Interaction and 3. Execution of the learned policy.

Download the report here.

---

State Space Abstraction with the Highway Car Domain

In this project, we use the help of an expert human to learn the task of navigating on a simulated highway. We take advantage of the different forms of input that can be given by the human and map them to the agent's world. The human interaction could be in one of two ways - by providing rewards or by providing a policy. We introduce a novel approach where the humans provides high level state abstractions. The criteria used by the human was - "states are similar if the same optimal action is to performed in both the states". This interactive abstraction significantly sped-up the performance of the agent.

Simulated video here.
Details can be found in Chapter 4 of my Thesis.

---

Introduction to Model-based Reinforcement Learning - IJCAI 2009, Best Narration Award

We created a reinforcement-learning demo -- a simple robot navigation task -- and took it to the public to teach them about AI and robotics. The video shows the system adapting in real time to various modifications to the robot's design and provides a very gentle introduction to the idea of model-based reinforcement learning.

You can find the video here.

---

Multi-Dimensional Particle Swarm Optimization - A Parallel Approach

Particle Swarm Optimization (PSO) is a population-based stochastic algorithm that deals with real world optimization problems. When using this technique to optimize multi-dimensional functions, a large amount of computational power is required to achieve efficient results. To alleviate this issue a parallel algorithm was implemented in an MPI cluster. The advantage of this was it produced the same result as that obtained from the serial algorithm with greatly reduced computation time and increased scalability. Large swarm population sizes can be easily managed when using multiple processors.